When to adapt a model, and when a smaller model is enough.
Starting with When to fine-tune, Building a training set, Supervised fine-tuning, LoRA and adapters, Evaluating a fine-tune, Small models and distillation, and more
In a picture
Collect good examples of the exact behaviour you want.
Continue training a base model on your examples.
Prove it beats the prompt-only version.
Stage 1 of 3: Prepare data
Open the full conceptMade for you now: slide lessons from basics to production, usually with code and a quiz. See a finished one
When to adapt a model, and when a smaller model is enough.
AI EngineeringStage 14 of 17: Fine-tuning and small models
Start with these 6. 10 more are one tap away.
Fine-tuning versus prompting versus RAG.
Collect, clean and format examples the model should copy.
Train on pairs of input and ideal output.
Cheap fine-tuning that changes few weights.
Compare against the base model on cases it never trained on.
Get most of the quality at a fraction of the cost.