The Methods That Made It Routine
Lesson 2 of 4 in Quantization.
Everything in this module is post-training quantization (PTQ): take a finished Checkpoint, produce a lower-bit artifact, no retraining. The naive version — round every weight to the nearest representable value — holds up surprisingly well at 8-bit for many models, but tends to fall apart as you push toward 4 bits. The methods that made low-bit weights routine are cleverer about which rounding error to accept, and both need only a small calibration set — a few samples of representative text run through the model to collect statistics.
GPTQ (Frantar et al., 2022) is error-compensating rounding. Instead of rounding all weights independently, it quantizes them a group at a time and then nudges the not-yet-quantized weights to soak up the error the rounding just introduced, using approximate second-order information about how sensitive the layer’s output is to each weight. The paper demonstrated accurate 4-bit (and even 3-bit, in its experiments) weight quantization for models with tens of billions of parameters — the result that made very-low-bit serving credible.
AWQ (Lin et al., 2023) is activation-aware scaling. Its observation: weights are not equally important — a small fraction of weight channels sit opposite large activation magnitudes and dominate output quality. AWQ uses calibration activations to find those salient channels and rescales them before rounding, so the precious few are represented finely and the rounding error lands on channels that matter less. No weight is left in high precision; protection comes entirely from the scaling.
Key terms: Quantization, GPTQ, AWQ, KV cache, Mixed precision
| Name | Core idea | Where you meet it |
|---|---|---|
Round-to-nearest | Round every weight independently; per-group scale factors do the heavy lifting | The baseline inside most 8-bit paths; the thing better methods beat at 4-bit |
GPTQ | Error-compensating rounding: quantize step by step, adjust remaining weights to absorb the error | 4-bit GPU serving artifacts; a standard format on open-weights model hubs |
AWQ | Activation-aware scaling: protect the few weight channels that face large activations, then round | 4-bit GPU serving artifacts; the other standard hub format |
GGUF (llama.cpp) | A file format, not an algorithm: one artifact carrying weights in the llama.cpp ecosystem’s quantization schemes at many bit-widths | The Open weights hobbyist-to-edge ecosystem: laptops, CPUs, consumer GPUs |
QLoRA | A training method — quantize the base model to 4-bit so a small adapter can be fine-tuned on top cheaply (Dettmers et al., 2023) | The fine-tuning story (Adapting LLMs domain) — listed here only because the name gets mixed in |
A note on the ecosystem you will actually download from. llama.cpp is a widely used open project for running LLMs on ordinary hardware, and GGUF is its artifact format; community members publish GGUF conversions of popular open-weights checkpoints at a whole menu of bit-widths, from conservative to aggressive. GPU serving engines load their own quantized checkpoint styles — GPTQ- and AWQ-format artifacts are common — which is a compatibility question we take up in lesson four. The method decides the quality; the format decides what can load it.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.