Deterministic fallbacks: degrade, do not improvise

The degradation ladder

1. Full agent autonomy — the loop absorbs the failure itself

   The plumbing told the model the truth and the model re-planned: a different tool, a narrower query, a cached read. Most transient failures should end here — which is only possible if failures arrive as typed observations rather than exceptions. This rung is where good error messages pay for themselves.

2. Constrained agent — same loop, smaller world

   Still model-directed, but with authority trimmed: a read-only tool set, a cached data source, a cheaper or secondary model, a tighter step budget. Useful when the goal is still reachable at lower fidelity. Requires that the constrained configuration be evaluated too — a degraded mode nobody ever ran is a second, untested agent.

3. Deterministic scripted path — no model in the loop

   The boring workflow that handles the common case: a template reply, a rules-based router, the pre-agent pipeline you probably still have running. It is right less often and predictable always — during an incident, predictable is worth more. Keep it exercised; a fallback path that is never run in normal operation rots.

4. Human queue — a person finishes the job

   A durable handoff carrying the run’s state: what was attempted, what succeeded, what is still in unknown state, and the trace link. This rung only exists if the queue is real, staffed, and monitored for age — an unstaffed escalation queue is an honest error with extra latency and false comfort.

5. The honest error — refuse clearly, with what is known

   The floor, and a legitimate outcome: "I could not complete this because the inventory system is unavailable. Nothing was charged. Here is your reference number." It states what was and was not done, and it never asserts a fact the run could not verify. Ranked below a human queue, ranked far above a confident guess.

The model provider throws 503s for 40 seconds

Plumbing: per-model breaker with a latency-based trip, backoff with jitter on the model call, and admission control at the edge so queued work is not accepted faster than it can drain. Fallback: secondary model if your evals cover it, otherwise the scripted path. Anti-pattern: every worker retrying in lockstep and turning a 40-second wobble into a 10-minute outage.

One tool times out on 30% of calls

Plumbing: per-tool breaker, and drop the tool from the tool list while the breaker is open so the model does not spend turns on it. Fallback: typed tool_unavailable observation naming the permitted alternative. Anti-pattern: a longer timeout, which makes the healthy runs slower and the sick ones no more likely to succeed.

The agent issued the same refund twice

Plumbing: runtime-derived idempotency keys on every side-effecting tool, an atomic claim in durable state, the response stored for replay, and the key passed downstream. Fallback: on ambiguous timeout, reconcile by re-presenting the key — never a blind retry. Anti-pattern: adding "do not call refund twice" to the system prompt and calling it fixed.

Queue depth is climbing and the oldest message is 20 minutes old

Plumbing: backpressure — bound the queue, shed or defer at admission, and alert on oldest-message age rather than depth. Fallback: tell the caller you are at capacity with a realistic window. Runs that exhaust their retry budget go to a dead-letter queue a human actually drains — not back onto the same queue, where they consume the capacity the healthy work needs. Anti-pattern: an unbounded queue, which converts backpressure into invisible latency until every message ages past its deadline at once.

A tenant burns its monthly token budget on the 28th

Plumbing: per-tenant budgets tracked in your own runtime and checked before the call, with quota exhaustion modelled as a state in the design. Fallback: the scripted path plus a clear message and reset window. Anti-pattern: discovering the limit from the provider’s error and surfacing it as a 500.

The agent looped 40 times, then produced a confident wrong answer

Plumbing: a wall-clock run deadline and a step budget, plus a run-level retry budget so per-call limits cannot multiply. Fallback: at the budget boundary, an honest partial result or a human handoff — never a final answer synthesised from failed steps. Anti-pattern: treating "the process got killed" as the stopping condition.