Concept 1 of 27
LLM
A large language model predicts the next token from the tokens before it, trained on huge text corpora. Everything it appears to do, answering, coding, summarising, comes from that one objective plus instruction tuning.
Pre-training
Once · monthsLearn language by predicting the next token across a huge amount of text.
- Training textWebBooksCode
- Tokenizetext → tokens
- Predict next tokenbillions of times
Tap any box to follow it.
Instruction tuning
Once · weeksTeach it to follow requests instead of just continuing text.
- Prompt + answer pairswritten examples
- Human feedbackrank better answers
Tap any box to follow it.
Generation
Every requestYour prompt goes in; the answer comes out one token at a time.
- Your promptas tokens
- Next-token oddspick one
- Stop token?no: append it and predict again
- Answerdetokenized text
Tap any box to follow it.
Check yourself
0 of 3 knownThink first, then reveal
What does an LLM actually do at each step?
Think first, then reveal
Why can an LLM sound confident and be wrong?
Think first, then reveal
What is instruction tuning?