Store weights with fewer bits so big models fit and run on smaller hardware.
- Full precisionWeights as 16-bit numbers.
- QuantiseRound to 8 or 4 bits.
- CalibratePick scales so rounding loses little accuracy.
- CheckCompare quality before and after.
Explain 1/5
Source: Hugging Face: Quantization overview