The Map: Four Stages of a Model’s Life

Lesson 1 of 3 in The LLM Lifecycle Map.

You already know what an Large language model (LLM) is: a next-token predictor whose knowledge lives in billions of learned Weights. This module answers the next question — where do these models come from, and what happens to one between a pile of raw text and the assistant answering your users?

The answer is a map with four stages, and it is the map this entire academy is organized around.

Pre-training turns raw text into a base model. Enormous amounts of text — much of the public internet, code, books — are fed through the next-token game you already know, and gradient descent adjusts the weights, over and over, until the model is very good at continuing text. The result is powerful and strange: a base model completes documents, but it does not naturally answer questions, follow instructions, or decline harmful requests. It continues text, because that is all it was trained to do.

Post-training shapes that raw ability into behavior. The model is trained further — on demonstrations of good answers, on human (and AI) feedback about which responses are better — until it behaves like an assistant: it follows instructions, adopts a persona, refuses some requests. Same architecture, same order of magnitude of weights; very different behavior. This is where a base model becomes an instruction-tuned model.

Inference is everything after training: the weights are frozen, and the model spends the rest of its life answering requests, one token at a time. Every chat, every API call, every generated line of code is inference. It looks like the boring stage and is anything but — serving a large model fast and cheaply is a serious engineering discipline, and it is where the ongoing bills live.

Adaptation is making a general-purpose model yours without building one: writing prompts that steer it, retrieving your documents into its Context window so it can answer from them (retrieval-augmented generation — RAG), or fine-tuning a small slice of behavior onto it. This is the stage where most engineers, and most readers of this site, actually work.

The LLM lifecycle — four stages, two cross-cutting concerns

  1. Raw text at scale

    Web pages, code, books — trillions of tokens, filtered and deduplicated before any training begins.

  2. Stage 1 · Pre-training

    Next-token prediction over the corpus, for months, on thousands of accelerators. The weights absorb the statistics of everything read.

  3. Base model

    A checkpoint of weights that continues text brilliantly but does not naturally follow instructions.

  4. Stage 2 · Post-training

    Instruction tuning and preference-based training (RLHF and successors) shape the base model into an assistant.

  5. Instruction-tuned model

    The artifact you actually meet behind a chat window or an API: helpful by training, not by nature.

  6. Stage 3 · Inference & serving

    Weights frozen. The model answers requests token by token — latency, throughput, and the ongoing bill are decided here.

  7. Stage 4 · Adaptation

    Prompting, RAG, fine-tuning: builders bend the general model to their domain and data.

  8. Your application

    What users see. Every layer below it was someone’s stage of the map.

  9. Evaluation & security — every stage

    Not a fifth stage but a lens on all four: labs run evals between training stages, red teams attack the trained model, platforms defend the serving stack, builders test their adaptations.

Two things about this map are worth pinning down before anything else on this site will make sense.

First, evaluation and security are not stages — they are lenses on every stage. Labs evaluate checkpoints during pre-training to decide whether a run is healthy. Post-training is steered by evaluations and stress-tested by red teams before release. Serving stacks are attacked (prompt injection, data exfiltration) and must be defended. And your adaptation — the prompt, the RAG pipeline — needs its own tests, because upstream scores say nothing about your use case. Whenever you see an evaluation number or a safety claim, your first question should be: at which stage was this measured?

Second, the map tells you where weights change. Pre-training and post-training rewrite them; fine-tuning (inside adaptation) nudges some of them; inference never touches them. That single fact explains a lot of everyday model behavior — including one that surprises almost everyone at first.

Why the stages split where they do

The four-stage split is not arbitrary — it follows from what changes, on how much data, at what cost.

Pre-training vs post-training is a split of objective and scale. Pre-training minimizes Cross-entropy Loss on next-token prediction over a corpus measured in trillions of tokens; it is where nearly all the compute is spent and where capabilities come from. Post-training reuses the same next-token machinery but points it at much smaller, deliberately constructed data: curated demonstrations of good behavior, then comparisons where humans (or a model standing in for them) rank candidate responses. Ouyang et al. (2022) showed why this stage earns its keep: users preferred the outputs of a comparatively small instruction-tuned model over a far larger base model. Behavior, it turns out, is much cheaper to shape than capability is to create.

Training vs inference is a split of computation. Training runs the model forward and backward — computing the loss, back-propagating gradients, updating every Parameter — over the same data many times. Inference is the forward pass alone, once per generated token, with weights read-only. That asymmetry is why training a frontier model is a rare capital event while running one is a permanent operating expense, and why the two are engineered by different teams with different tools.

The Checkpoint is the handoff artifact. Each stage ends by writing the weights to disk; the next stage begins by loading them. A “model” you download or call is nothing more than a specific, named checkpoint from somewhere along this pipeline — which is why the same family name can cover a base model, an instruction-tuned sibling, and a dozen fine-tunes, all different checkpoints with a shared ancestor.

Adaptation is a spectrum of how much you touch. Prompting changes only the input; RAG changes what enters the context window; fine-tuning changes some weights. The further right you go, the more power you get over behavior — and the more of the map you become responsible for. That trade-off gets its own domain later in the site.

When each stage entered the story

  • 2019-02-14 — GPT-2 and staged release:

    A 1.5B-parameter GPT trained on web text showed surprising zero-shot ability across tasks, and OpenAI staged its release over misuse concerns. It was the first clear signal that scaling one recipe broadens capability — and the first mainstream release-policy debate.

  • 2020-01-01 — Scaling laws for neural LMs:

    Kaplan et al. measured loss falling as smooth power laws in parameters, data, and compute, with architectural details mattering far less. It turned "make it bigger" from a hunch into a predictive engineering discipline.

  • 2020-05-01 — GPT-3: in-context learning:

    At 175B parameters, GPT-3 exposed a new interface: in-context learning, where the model performs tasks from instructions and a few examples in the prompt, with no gradient updates. Prompting began to replace fine-tuning as the primary way to program a model.

  • 2022-03-01 — Chinchilla: compute-optimal training:

    Hoffmann et al. showed most large models were undertrained: at a fixed compute budget, parameters and training tokens should grow roughly in step (about 20 tokens per parameter). It reset scaling strategy toward smaller models trained on far more data.

  • 2023-02-01 — LLaMA: the open-weights era begins:

    Meta trained 7B–65B models Chinchilla-style on trillions of tokens and released the weights to researchers; the 13B model rivaled GPT-3. The weights spread far beyond the research program and ignited the open-weights ecosystem of fine-tunes and local inference.

  • 2023-03-14 — GPT-4:

    A multimodal model (text and image input) with large capability jumps — top-decile bar-exam performance among them — whose technical report disclosed neither architecture nor training data. It marked both the new capability frontier and the industry’s turn away from openness.

  • 2023-07-18 — Llama 2: open weights go commercial:

    Meta released retrained 7B–70B models plus RLHF-tuned chat variants under a license permitting most commercial use. Open weights became a viable production choice rather than a research artifact, anchoring the open-versus-closed split that defines the current landscape.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.