Prefill and Decode

One request, two very different phases: the parallel prefill and the serial decode — the split behind every latency number and price sheet in this field.

Content last verified 2026-09.

Lessons

1. Two Phases, Two Personalities
2. Why Output Costs More Than Input
3. The Serving Loop

Sources

• Pope et al. (2022) — Efficiently Scaling Transformer Inference
• Kwon et al. (2023) — Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM)