Cost and Latency Budgets You Can Defend

Where an agent run’s tokens actually go (the whole transcript resends every turn, so cost grows quadratically-ish with turn count), which levers really cut cost and latency, how to enforce a budget in the runtime instead of wishing for one in the prompt, and how to bake off cost, latency, and quality on the same golden set.

Content current as of 2026-09.

Lessons

1. The resend tax: where a run’s tokens actually go
2. The cost levers, honestly sized
3. Time-to-first-token vs time-to-done
4. A budget without enforcement is a wish
5. The bake-off: cost, latency, and quality on one table