Several attentions in parallel, each free to specialise.
Made for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
The model splits its width into h smaller heads. Each head has its own Q, K, V projections and runs attention independently.
One attention gives one weighting per token. Several heads can track different relations at once, for example nearby words and long-range references. What each head learns is observed after training, not assigned.
Head outputs are concatenated and passed through one more learned matrix so the result is back at the model width.
Step 1
Project per head
Smaller Q, K, V for each head.
Step 2
Attention per head
Independent weights.
Step 3
Concatenate
Join head outputs.
Step 4
Output projection
Mix heads, back to model width.