Pretraining teaches general language from huge text; fine-tuning adapts it to your task.
- PretrainTrillions of tokens, weeks of GPUs.
- Base modelKnows language, not your task.
- Fine-tuneHundreds to thousands of your examples.
- Or prompt / RAGOften enough without training.
Explain 1/5
Source: Hugging Face: Fine-tune a pretrained model