Tokens: the meter everything runs on

Lesson 1 of 5 in LLM Mechanics for Agent Builders — No Math Required.

An LLM never sees words. Before your prompt reaches the model, a tokenizer chops it into tokens — subword chunks like agent, ic, or * the* (spaces included). The model reads tokens in and produces tokens out, one at a time. That is the entire I/O of the machine.

Why should an engineer care about a text-encoding detail? Because every number that constrains your agent is denominated in tokens: what a call costs (price per million tokens, in and out), how fast it runs (tokens generated per second), and how much the model can consider at once (the context window, measured in tokens). Tokens are the meter, and an agent — an LLM called in a loop — is a machine for running that meter.

The rough conversion worth memorising: 1 token ≈ ¾ of an English word, or about 4 characters. So 100 tokens ≈ 75 words; a dense page of text ≈ 500 tokens; this lesson so far ≈ 300 tokens.

Why are tool results so token-hungry?

Because machines write verbose text. A JSON API response repeats every key on every record; a stack trace repeats file paths; a grep across a repo can return thousands of lines. A single careless tool call can inject 50,000 tokens into the transcript — which the agent then resends on every subsequent call. Trimming tool output before it enters the context is one of the highest-leverage optimisations in agent engineering.

Do all languages cost the same?

No. Tokenizers are trained mostly on English-heavy data, so English compresses best. The same sentence in Japanese, Hindi, or Greek can cost 1.5–3× more tokens — same meaning, bigger bill, less room in the window. If your agent serves non-English users, budget for it.

Why can’t the model count the letters in “strawberry”?

Because it never sees letters — it sees whole token-chunks. Asking an LLM to count characters is like asking you to count the pixels in a photo of a word. This is a tokenizer artifact, not a reasoning failure, and it is why string-exact work (character counts, precise offsets, checksums) belongs in a tool, not in the model.

Are input and output tokens priced the same?

Almost never. Output tokens typically cost 3–5× more than input tokens (illustrative ratio — check your vendor’s current pricing), because generating a token is sequential work while reading input is parallel. Agents flip the usual intuition, though: their transcripts grow so fast that input usually dominates the bill anyway.

Worked example: pricing a 20-turn agent run (illustrative numbers)

Take a coding agent whose system prompt plus tool definitions weigh 4,000 tokens, and where each turn appends ~2,000 tokens to the transcript — a ~500-token model reply plus a ~1,500-token tool result. Because the whole transcript is resent every call, call 1 sends 4,000 input tokens, call 2 sends ~6,000, call 3 sends ~8,000… and call 20 sends ~42,000. Add it up: about 460,000 input tokens and 10,000 output tokens (20 × 500) for one run.

At illustrative prices of $3 per million input tokens and $15 per million output tokens (real prices vary by model and change often — always check current vendor pricing), that is roughly $1.38 + $0.15 ≈ $1.50 per run. Two lessons hide in that number: input costs nine times the output because of resending, and a 40-turn run would cost roughly 4× — not 2× — this one, because the transcript grows as it goes.

Key terms: token, tokenizer, LLM, context window, tool call

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.