Quantization

Fewer bits, same model — mostly: what gets quantized, the methods that made it routine, and how to check what it cost you.

Content last verified 2026-09.

Lessons

1. Fewer Bits: What Quantization Is
2. The Methods That Made It Routine
3. What It Costs You — Measure, Don’t Assume
4. Formats and Deployment

Sources

• Frantar et al. (2022) — GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
• Lin et al. (2023) — AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
• Dettmers et al. (2023) — QLoRA: Efficient Finetuning of Quantized LLMs
• Pope et al. (2022) — Efficiently Scaling Transformer Inference