A budget without enforcement is a wish

Lesson 4 of 5 in Cost and Latency Budgets You Can Defend.

Write “keep each run under $0.50” in the system prompt and you have written a preference. The model cannot count its own tokens, cannot see your rate card, and can be talked out of prose instructions by its own reasoning or by text it reads while working. A budget lives in the runtime or it does not exist.

Concretely, the runtime keeps a ledger per run and checks it before each model call. Six meters are worth tracking, and they fail differently:

Tokens — the raw meter every provider reports in each response’s usage fields; count cached and uncached input separately or your cost maths will be wrong.
Money — tokens × your rate table. What the business asked for, and the only meter that survives a model swap.
Turns — the cheapest to implement and the best early proxy, because turn count multiplies everything else.
Wall clock — the meter that protects the user; a run inside its token budget can still blow a 60-second promise.
Tool calls — total, and per tool: 200 searches is a loop bug, and 200 emails is an incident.
Spend rate — cost per minute across all runs, which is the only meter that catches a fleet-wide regression before the invoice does.

The enforcement stack — narrowest control at the top

Pre-call check — the only place a run is stopped before it spends
Immediately before each model call, compare the ledger plus this call’s known input size plus the configured output cap against the remaining budget. If it would breach, do not make the call — switch to the degradation path instead. Everything below this tier reacts after money is spent; only this tier prevents the spend.
Per-run ledger — one run id, six meters, inherited by children
Accumulate usage from every response into a structure keyed by run id, and pass the remaining budget into every subagent and nested tool-driven call. A parent cap that child runs do not inherit is not a cap — this is the single most common budget bug in multi-agent systems.
Per-user, per-tenant, per-feature quotas — blast-radius containment for cost
Daily and monthly caps at the identity boundary. This is what stops one enthusiastic customer, one scripted client, or one runaway integration from consuming the budget of everyone else. Same instinct as least privilege, applied to spend.
Platform quotas, rate limits, concurrency caps — independent of your application code
Provider-side rate limits, gateway budgets, and your own concurrency limiter. Blunt and lossy — they shed load rather than degrade gracefully — but they hold when your application logic is the thing that is broken, which is exactly when you need them.
Spend alarms and the kill-switch — a smoke detector, not a budget
Anomaly alerts on cost per minute, plus a documented, tested way to turn the feature off in seconds. Detection latency here is minutes to hours, so this tier cannot be your primary control — but every serious deployment has one, and the kill switch is the part people forget to test.

Enforcement is only half the design. The other half is what the agent does when it hits the wall — and “throw an exception” is a product decision made by accident.

Three behaviours are defensible, and the choice is not the agent’s to make at random. Degrade gracefully: spend a reserved final turn producing the best available answer plus a handover note naming what is unfinished. Report: return a structured incomplete result carrying the run id, the meter that tripped, and the spend so far, so a caller or a human can decide. Stop: end the run without side effects and without silently retrying it — a retry at the same budget just pays twice for the same failure.

Note the word reserved. Graceful exits cost tokens, so the enforcement threshold has to fire before the cap, not at it. Budget about a turn’s worth of headroom, or your degradation path will itself be refused by the pre-call check.

The ledger says this run breaches its budget within two turns. What should the runtime do?

Interactive decision tree — outcomes:

Degrade gracefully — enter finish-up mode
Spend the reserved final turn on an instruction like “this is your last turn: produce your best answer and a handover note listing what is unverified.” Return it flagged as partial, with the run id and the meter that tripped. This is the best outcome available, and it only exists if you reserved the headroom in advance.
Ask for more — never extend silently
A guardrail plus real progress is a case for an increment, not for removing the cap. Surface the run id, spend so far, the progress evidence, and a bounded increment to a human or a policy service — an approval gate on spend. Auto-extension with no policy is how a $2 run becomes a $200 run, and how one bad prompt becomes a four-figure night.
Stop and report
End the run. Return a structured incomplete result naming the meter, the spend, and the trace id. Do not auto-retry at the same budget: if the loop was thrashing, a retry reproduces the thrash at full price. Feed the trace into your regression set instead — a budget breach is a bug report, not a billing event.
Unwind first, then stop
Take no new actions, run the compensating steps (or open a ticket that names each one precisely), then stop. A budget stop must never leave orphaned side effects — half a migration, a payment with no receipt, an email promising a follow-up nobody will send. If your agent cannot unwind, the enforcement threshold must be low enough to leave room to try.

Budget the worst case — the multipliers are hiding in your architecture

A “$0.50 per run” cap describes a mean. Your invoice is set by the worst case, and four multipliers stack on top of it:

Retries. A 3-attempt policy triples the ceiling. Retrying a whole run is the expensive kind; retrying one failed tool call is the cheap kind. Know which one your framework does by default.

Fan-out. A supervisor that spawns five subagents, each running its own loop with its own transcript, costs five runs plus the supervisor’s own turns. Fan-out is the fastest way to multiply a token bill ever invented, and it is invisible in a per-call view.

Reflection and critic passes. Every “now check your work” pass adds turns on top of the biggest transcript of the run — the most expensive place to add them.

The evaluation bill. Judge calls, regression suites, and online sampling all consume tokens. Teams are regularly surprised to find evaluation costs more than production. It is worth every cent — but it belongs in the budget, on purpose.

So write the ceiling down: worst-case run cost ≈ per-run cap × retry factor × fan-out. If that number would embarrass you at your P99 traffic, the cap is not the thing that needs changing — the architecture is.

Key terms: token budget, graceful degradation, kill switch, subagent, approval gate, trace

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.