The transformer model has two main components — the encoder and decoder. The encoder stacks and decoder stacks (Nx) are identical layers, either of encoders or decoders. The original paper had Nx=6.

Encoder

Encoder processes the input iteratively, layer by layer and generates a sequence of hidden states to represent the input sequence. The decoder uses the output of the encoder to generate the output sequence. The decoder attends to the encoder’s output and the output so far generated to generate the next word in the sequence.

Decoder

The decoder as we already know generates output sequence by attending to the input sequence and the output that has been generated so far. There are decoder layers in a decoder which process the encoder’s output iteratively, layer by layer.