Making attention fit in memory and serve more users.
Made for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
Standard attention writes the full n by n score matrix to slow memory. FlashAttention computes in tiles that fit in fast on-chip memory and never stores the whole matrix. The result is the same, with much less memory traffic.
Multi-query attention shares one K and V across all query heads; grouped-query attention shares them within groups. Fewer K and V means a smaller KV cache and faster decoding, with a small quality trade-off that varies by model.
Step 1
Tile the computation
Process blocks in fast memory.
Step 2
Never store full scores
Same output, less memory.
Step 3
Share K, V across heads
GQA groups heads; MQA uses one.
Step 4
Smaller KV cache
More concurrent users per device.