The Words People Use
Lesson 4 of 4 in What Is a Large Language Model?.
Model announcements, docs, and meetings run on a small working vocabulary. Five clusters cover most of it — get these solid and the rest of the academy reads twice as fast.
Parameters and weights. Two names for the same thing: the adjustable numbers inside the network. When a model is described as “7B”, that is the Parameter count — seven billion Weights. The count is the standard shorthand for a model’s size, and size correlates with capability, memory footprint, and serving cost — correlates, not determines.
Checkpoint. A Checkpoint is a snapshot of all the weights at one moment of training, saved to disk. Every model you download or call is, concretely, a checkpoint — training produced a long trajectory of them, and someone chose one, named it, and shipped it. When people say “the new checkpoint fixed it”, they mean the weights changed, not the code around them.
Context window. The Context window is how much text — measured in tokens, never words or pages — the model can consider in a single request. It is working memory, not long-term memory: everything the model should account for must fit inside it, every request, and nothing persists between calls unless your application resends it.
Base model
A Base model is what pre-training alone produces: a pure next-token predictor with no further shaping. Give one the prompt “Write a haiku about the ocean” and a typical continuation is more text like the training data — perhaps “…in exactly seventeen syllables. Exercise 12: Write a sonnet about…” — because the prompt resembles a worksheet, and worksheets continue with more exercises. The base model is not disobeying. It is doing exactly its one job: continuing the text.
Instruction-tuned model
An Instruction-tuned model is a base model given additional training on examples of instructions paired with good responses, so that answering becomes the most probable continuation. The same prompt now yields an actual haiku. Nearly every model behind a chat product is instruction-tuned; base checkpoints are mostly used as raw material for further training. How that shaping works — and what it costs in side effects — is the Post-training & Alignment domain’s story.
Open vs closed weights. The last axis is access to the file itself. Open weights: the checkpoint is published, and anyone can download it and run it on their own hardware — door three from the previous lesson. Closed weights: the checkpoint stays on the provider’s servers, reachable only through an app or API. The practical stakes are exactly the responsibility split you just learned: data location, customization depth, and who operates the stack. A model’s Model card — its published spec sheet — usually states the release type along with the license; the Model Landscape domain teaches how to read one critically.
Interactive sorting exercise: Sort each statement onto the term it describes.
Key terms: Parameter, Checkpoint, Context window, Base model, Instruction-tuned model, Open weights
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.