What crosses the boundary: payload, identity, budget
Lesson 3 of 5 in Orchestration End to End: One Request Through the Whole System.
Three things travel across every hop in a multi-step system. Most designs specify the first, half-specify the second, and forget the third entirely.
The payload — what the next hop needs to do its job. The identity — whose run this is, which step, on whose behalf. The budget — how much time and money is left.
Omit the identity and you get a system that cannot be traced or made idempotent. Omit the budget and every inner call quietly uses its own default timeout, so an outer deadline of 20 seconds waits 30 for one warehouse.
The payload
The rule that saves the most pain: pass references and decisions, not prose.
When the fan-out step briefs the entitlement subagent, the weak version is “here is the customer email and the conversation so far, figure out what they’re entitled to.” The strong version names the contract ids, the SKU, the shortage quantity, the deadline, and the exact question to answer — plus the output shape, with a required citation.
Why it matters at the seam: every hop that re-summarises is a hop that loses information and adds a place for facts to mutate. That is the telephone game, and it compounds across hops in exactly the way chain errors do. A reference (contract_id: MSA-2019-0442) survives arbitrarily many hops unchanged; a sentence about the contract does not.
The corollary is unpopular and correct: do not pass the raw customer message downstream once you have resolved it. Keeping it in context invites every later hop to re-read untrusted prose and prefer it over resolved facts — and it is the transport by which an injected instruction reaches the hop with the dangerous tools. Resolve, then drop.
The identity
Four identifiers should ride along on every hop, including into subagents and out to tool servers:
- run id — this attempt at this task. The unit an operator cancels or replays.
- session id — the longer conversation or case this run belongs to.
- step id — the hop, so spans nest into a tree rather than a list.
- principal — the human or agent identity on whose behalf the work happens, ideally as a downscoped credential rather than a service account with everything.
Two capabilities fall straight out of threading these. Idempotency: an execution key of {run_id}:{step_id}:{action} makes a retried reship a no-op instead of a second pallet. A trace that reads as one story: lesson five is only possible because the ids exist.
And one that is easy to lose: on-behalf-of identity across hops. If the subagent reads contracts using a service principal with access to every customer’s agreements, then a scoping bug in the parent becomes a cross-customer data leak in the child. Narrow at the boundary, not at the end.
The budget
Deadlines and spend limits must be arguments, not globals. Deadline propagation means the caller computes what remains and passes it down: “you have 4.2 seconds and 6,000 tokens.”
Without it, three ordinary things happen. Inner calls use library defaults — the 30-second HTTP default under a 20-second SLO. A subagent with its own 15-turn cap outlives the parent that is waiting for it, so the parent’s deadline is decoration. And a fan-out that inherits the full budget per branch authorises N times the spend you approved.
With it, timeouts shrink down the call tree, which is the only arrangement where an outer bound means anything. The practical shape: pass deadline_ms_remaining and token_budget_remaining in the same envelope as the ids, refuse to dispatch a hop whose minimum cost exceeds what is left, and record both in the span so you can see afterwards which hop ate the budget.
Which brings us to the question the boundary makes unavoidable: as facts accumulate — the resolved order, the warehouse answers, the entitlement clause, the approver’s edit — where do they live? “State management” is usually discussed as one thing. In a real system it is four stores with different lifetimes, and most state bugs are a fact written to the wrong one.
| Store | Lives for | Holds | Misplacement bug |
|---|---|---|---|
Model context | One model call | The prompt: instructions, resolved facts, recent tool results, the working scratchpad | Treating it as memory. It is rebuilt every call, so anything you need later and do not write down is gone — and anything you keep “just in case” is paid for on every call and eventually causes context rot |
Run state | One run, across suspends and resumes | Which hops completed, their outputs, the current proposal, the checkpoint the approval gate suspends on | Keeping it only in the transcript. Then a resume after approval must re-derive the run’s position by reading messages, which is how divergent state appears |
Case / session memory | The customer relationship | Prior exceptions for this customer, standing preferences, “they always want reship not credit” | Writing model inferences here as facts. Unprovenanced entries become permanent — the poisoning and provenance problem, at the seam where a run decides what is worth remembering |
System of record | Forever, and audited | The credit memo, the replacement shipment, the approval and who gave it — the business facts | The one that matters most. If the only record that a $4,000 credit was approved lives in your agent’s state store, you have built a financial system with no ledger. The agent holds a pointer plus its own reasoning; the business holds the fact |
Key terms: delegation brief, telephone game, deadline propagation, on-behalf-of (OBO), run, checkpoint
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.