The resend tax: where a run’s tokens actually go
Lesson 1 of 5 in Cost and Latency Budgets You Can Defend.
Most teams estimate agent cost the way they estimate chatbot cost: tokens in the question, tokens in the answer, multiply by traffic. Then the first invoice arrives and it is 5–10× the estimate, and nobody can say why.
The reason is structural, and it is the single most useful thing in this module. An LLM is stateless: it remembers nothing between calls. So every iteration of the agent loop must resend the entire transcript so far — system prompt, tool schemas, the goal, every model turn, every tool result — just to add one new turn on the end. Call it the resend tax: you do not pay for the conversation, you pay for the conversation once per turn.
That changes the shape of the cost curve. If the fixed prefix is P tokens and each turn appends g tokens, then call n sends P + (n−1)·g input tokens, and a whole n-turn run sends
total input ≈ n·P + g·n(n−1)/2
The second term is a triangular number, so total input grows with the square of the turn count. Not exactly quadratic — the P term is linear and g is only an average — but close enough that you should budget as if it were. Here is what that looks like on one concrete run.
| Turn | What this call resends | Input tokens, this call | Cumulative input | Cumulative output |
|---|---|---|---|---|
1 | prefix only | 3,000 | 3,000 | 400 |
2 | prefix + turn 1 | 5,000 | 8,000 | 800 |
3 | prefix + turns 1–2 | 7,000 | 15,000 | 1,200 |
4 | prefix + turns 1–3 | 9,000 | 24,000 | 1,600 |
5 | prefix + turns 1–4 | 11,000 | 35,000 | 2,000 |
6 | prefix + turns 1–5 | 13,000 | 48,000 | 2,400 |
7 | prefix + turns 1–6 | 15,000 | 63,000 | 2,800 |
8 | prefix + turns 1–7 | 17,000 | 80,000 | 3,200 |
9 | prefix + turns 1–8 | 19,000 | 99,000 | 3,600 |
10 | prefix + turns 1–9 | 21,000 | 120,000 | 4,000 |
Key terms: token, context window, agent loop, resend tax, token budget, tool call
The resend tax has a second consequence that changes how you debug bills: where a token lands in the run decides what it costs. A thousand tokens added on turn 1 of a ten-turn run are billed ten times. The same thousand tokens added on turn 10 are billed once. Cost is not a property of a token; it is a property of a token and its position.
Interactive sorting exercise: Same 10-turn run as the table above. Sort each change by how it hits the input-token bill. (Hint: ask “how many later calls will resend this?”)
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.