Context rot: the slow failure of long runs
Lesson 3 of 5 in Memory Architectures: Engineering the Context at Scale.
Context rot is the gradual degradation of an agent’s decision quality as its context accumulates noise: duplicated file reads, stale tool output, dead-end trails, boilerplate, half-relevant retrieved chunks. Nothing crashes. The agent keeps producing plausible turns — it just follows instructions less reliably, repeats work more often, and reaches for the wrong tool more frequently than it did at turn five.
The mechanism is attention, not amnesia. Every token in the window is still there; what changes is the model’s ability to allocate attention across an ever-noisier set. Long-context models that ace needle-in-a-haystack retrieval benchmarks still degrade on instruction-following and multi-step reasoning as realistic, messy context grows — and content buried in the middle of a long context reliably gets less attention than content at the beginning or end (the lost-in-the-middle effect). A rule stated once, 40,000 tokens ago, mid-context, is a rule the model will eventually miss.
| Symptom | Root cause | Countermeasure |
|---|---|---|
Ignores a rule it followed earlier — the system prompt says “ask before deleting,” turn 70 deletes without asking | Instruction dilution: the rules are a shrinking fraction of a growing context, stranded far from the model’s current focus. | Keep the rulebook short; re-inject or pin critical rules near the end of the context; compact earlier so instructions stay proportionally large. |
Repeats work — re-reads files, re-runs the same search | Earlier results are buried mid-context under newer noise — present but effectively invisible. | Deduplicate tool results at ingestion; maintain a visible, current task ledger; summarize the stale middle. |
Retries known dead ends | The negative result was buried — or was destroyed by a compaction that had no do-not-retry discipline (lesson 2). | Preserve negative results explicitly, in the checkpoint and in a mandatory summary section. |
Cost and latency climb every turn | The whole context is re-sent and re-priced on every call — an uncompacted history grows the bill linearly per turn, quadratically over the run. | Compaction on a cadence, not just at the wall; truncate tool outputs at ingestion; cap the retrieval budget per turn. |
Quality falls off a cliff right after a big tool result | A single dump displaced the working state — plan, constraints, and instructions all lost share at once. | Hard caps on tool-result size entering context; extract-then-drop; route bulk analysis to a subagent (below). |
The countermeasures reduce to four principles. Hygiene at ingestion: the cheapest place to fight rot is the door — truncate, extract, and deduplicate tool results before they enter the context, because evicting noise later costs a compaction. Cadence: compact on a schedule tied to quality, not only when the window forces you. Position: put what must be followed where attention is strongest — instructions at the start, the current task and critical rules restated near the end, nothing load-bearing in the middle. Isolation: give noisy work to a subagent with its own clean, task-scoped context; the parent receives only the conclusion, and the ten thousand tokens of search transcript die with the subagent (the subagents and context isolation module in this domain is built on exactly this move).
And when a context is too far gone, there is a nuclear option that works surprisingly well: checkpoint and restart. Write the durable state out (lesson 2), kill the context entirely, and start a fresh one that reads the checkpoint. A clean context plus a good checkpoint routinely outperforms a rotted context that “knows” more.
Myth: “Million-token windows make all of this obsolete”
Bigger windows raise the overflow ceiling; they do nothing for attention quality, and you pay for every token on every call regardless. In practice a bigger window is a longer runway to accumulate more noise — teams that treat capacity as a license to skip hygiene meet the same rot, later and at higher cost per turn. The budget discipline of lesson 1 applies at every window size.
Myth: “The model forgets things on long runs”
Nothing is forgotten — every token of an unbroken context is still in the input. What degrades is attention allocation across a noisy set, aggravated by position effects. The distinction matters because it dictates the fix: repetition helps a little (a re-stated rule lands in a high-attention position), but subtraction helps more — remove the noise competing with the signal instead of shouting over it.
Myth: “A rotted run is a ruined run”
Rot is recoverable precisely because the model is stateless: the damage lives entirely in the context, and the context can be rebuilt. Checkpoint the durable state, restart fresh, re-read the checkpoint — the agent resumes with its knowledge and without its noise. Architectures that plan for this (regular checkpoints, resumable state) treat context rot as a maintenance event, not an incident.
Key terms: context rot, lost in the middle, compaction, subagent, context isolation, checkpoint
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.