Loading
Why similarity is a dot product and why it is rescaled.
Made for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
The dot product is large when two vectors point the same way, so it is a cheap similarity score between a query and every key, computed for all pairs as one matrix multiply.
With independent unit-variance entries, a dot product of d-dimensional vectors has variance d. Dividing by sqrt(d) keeps the scale steady so softmax does not saturate as models widen.
Step 1
Q times K transposed
A score for every token pair.
Step 2
Divide by sqrt(d)
Keep variance near 1.
Step 3
Softmax
Turn scores into weights.