The context window — and the amnesia behind it
Lesson 2 of 5 in LLM Mechanics for Agent Builders — No Math Required.
The context window is the model’s entire world: a bounded number of tokens that must hold everything at once — the system prompt, the tool definitions, the whole conversation and tool-result history, and room for the reply the model is about to write. If a fact is not in that window right now, then as far as this call is concerned, it does not exist. The model cannot peek at your database, your last session, or the file it read yesterday.
Windows are large in 2026 — commonly hundreds of thousands of tokens, sometimes more — but bounded is the operative word. An agent that greps a big repo or pulls a long API response can fill any window, and every token in it is paid for on every call.
Watch it happen inside one agent run. The user says “fix the failing test.” Call 1 sends the system prompt, the tool definitions, and that goal; the model replies with a grep tool call. Now the runtime makes call 2 — and it cannot just send the grep results, because the model has already forgotten the goal, the tools, and its own plan. Call 2 must resend everything from call 1, plus the tool call and its result. Call 3 resends all of that plus the next exchange. The transcript is the agent’s only memory, and it travels in full, every time.
| Call | What the runtime must send (the entire prompt, rebuilt from scratch) | Input tokens |
|---|---|---|
Call 1 | System prompt + tool definitions + “fix the failing test” | ~4,200 |
Call 2 | Everything above + the model’s grep call + 3 pages of grep results | ~7,400 |
Call 3 | Everything above + the model’s read-file call + the 200-line source file | ~11,600 |
Call 12 | Everything above, ten tool exchanges deeper — edits, test runs, stack traces | ~48,000 |
Interactive sorting exercise: Where does each of these live, from the model’s point of view during one call?
Key terms: context window, system prompt, statelessness, compaction, context engineering, memory
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.