What agents actually cost
Lesson 2 of 4 in When NOT to Build an Agent.
Agent proposals usually arrive with one number: the token bill from the demo. That number is off by an order of magnitude, and understanding why is the fastest way to earn a reputation for judgment.
Start with the run-time arithmetic. An agent’s cost per task is roughly tokens per iteration × iterations × (1 + retry rate) — and every factor is variable. Worse, the factors multiply each other: each loop iteration appends the previous tool results to the context window, so iteration ten re-reads everything iterations one through nine produced. A 15-step run doesn’t cost 15 model calls; it costs 15 calls of cumulatively growing size. Cost is superlinear in steps, and the model — not you — decides how many steps there are.
Line 1 · The token bill — you pay for the conversation, repeatedly
Every iteration replays the growing context. Rules of thumb: doubling the average number of steps more than doubles the bill; verbose tool outputs (full web pages, raw logs) are the silent multiplier. This line is at least visible — it shows up on the invoice, which is why it is the only line most projections include.
Line 2 · The retry tax — non-determinism means paying for failures
If 15% of runs fail and get re-run — by your harness or by an annoyed user — you pay for ~1.18 runs per task, and the failed runs are often the longest ones (the agent flails before giving up). A turn limit caps the worst case; nothing eliminates the tax.
Line 3 · The debugging tax — failures without stack traces
When a script fails you get a stack trace pointing at a line. When an agent fails you get a trace: forty model decisions, one of which was subtly wrong, none of which will reproduce on re-run. Debugging becomes reading transcripts and forming hypotheses about a probabilistic system — senior-engineer hours, not junior ones. Teams consistently report this, not tokens, as the dominant cost.
Line 4 · The eval tax — testing becomes a standing project
You cannot assert on an agent, so you must evaluate it: build a golden dataset, score runs (often with an LLM-as-judge), and re-run the whole suite on every prompt tweak and model upgrade — and model deprecations mean upgrades are not optional. A workflow’s test suite is an artifact; an agent’s eval suite is a subscription.
Line 5 · The ops tax — humans in the loop are a cost line
Monitoring dashboards, guardrails, incident playbooks — and above all, reviewer time. If a human approval gate takes 30 seconds per action at 2,000 actions a day, you have hired two full-time reviewers without noticing. The agentops domain exists because someone has to own all five of these lines.
Key terms: token, context window, non-determinism, eval, golden dataset, debugging tax
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.