Stacks of attention and feed-forward blocks, with shortcuts, read all tokens at once.
- EmbedTokens become vectors plus positions.
- Self-attentionEvery token gathers context from the others.
- Feed-forwardEach token's vector is transformed on its own.
- Residual + normAdd the input back and normalise.
- Repeat N timesStack 12 to 100+ blocks.
Explain 1/6
Source: Vaswani et al.: Attention Is All You Need