Token economics: estimate, track, attribute

Lesson 1 of 3 in GenAI Cost Engineering.

At the associate level you learned that Bedrock on-demand meters tokens in and tokens out (the MLA cost module covers that shape). The professional exam assumes it and asks the operator's questions instead: how many tokens will this feature burn before it launches, how do you know what it burns now, and whose budget does each token land on? Three disciplines — estimate, track, attribute — and each has a named mechanism.

Estimation happens before traffic exists. Count the tokens of your real prompt templates with the model's tokenizer (English prose runs roughly 3–4 characters per token, but code, JSON, and non-Latin scripts tokenize heavier — measure, don't assume), add the retrieval payload at your real chunk sizes, and multiply by expected output length. The asymmetry matters: output tokens price higher than input tokens on most models, and output length also drives latency — which is why a max-token cap is both a cost control and a latency control.

Horizontal bar chart of the illustrative token share of a single retrieval-augmented chat request: retrieved context 30%, conversation history 22%, system prompt 18%, few-shot examples 14%, model output 12%, and the user question only 4%. The hints name the lever for each slice: rerank and prune retrieval, summarize or window history, compress the system prompt and cache it as a static prefix, cut redundant examples, cap max output tokens. Relative weights for one illustrative workload, not pricing data.

Where the tokens of one RAG chat turn live — and which lever shrinks each slice (illustrative — source: Amazon Bedrock pricing — how inference is billed)

Tracking is built in once you know where to look. Every Converse/InvokeModel response carries a usage block with input and output token counts, and CloudWatch aggregates the same truth in the AWS/Bedrock namespace: InputTokenCount and OutputTokenCount by model ID, alongside Invocations and InvocationThrottles. For per-request forensics — which prompt burned 40,000 tokens? — enable model invocation logging and query the records (the monitoring lesson goes deep). The CloudWatch GenAI observability dashboards chart token consumption without you building anything.

Attribution is the step teams skip until finance asks. One account, one model, five product features — the bill is a single line unless you split it. The professional-grade mechanism is application inference profiles: create a profile per workload (it wraps the model or a cross-Region profile), tag it with cost allocation tags, activate those tags in the Billing console, and invoke through the profile's ARN. Spend then slices by feature in Cost Explorer and the Cost and Usage Reports. Tags are not retroactive — only spend after activation is tagged — so wire attribution in before launch, not after the surprise.

The token FinOps loop

  1. Estimate

    Tokenize the real prompt templates, add retrieval payload at real chunk sizes, multiply by expected output length and traffic. A measured pilot beats any rule of thumb.

  2. Measure

    Usage blocks on every response; InputTokenCount / OutputTokenCount by model ID in CloudWatch; invocation logs for per-request forensics.

  3. Attribute

    Tagged application inference profiles per workload; activate the cost allocation tags; slice spend by feature in Cost Explorer and CUR.

  4. Within budget?

    AWS Budgets alerts on forecasted overrun; Cost Anomaly Detection flags unusual spend patterns; CloudWatch anomaly detection catches token bursts in near-real time.

  5. Keep shipping

    Attribution stays on — the next feature inherits the discipline for free.

  6. Pull a lever

    Compress, tier, batch, cache — the next two lessons. Then re-measure: every lever claims savings; only the token metrics confirm them.

Key terms: AWS Cost Anomaly Detection, AWS Cost Explorer, AWS Budgets, Model invocation logging

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.