The hybrid: code for the skeleton, model for the judgment
Lesson 4 of 5 in State Machines vs LLM Loops: Who Owns Control Flow?.
Strip the framework debates away and production systems keep converging on one shape: a coded skeleton with model judgment installed inside specific states. Code owns anything with a right answer or a hard requirement — sequencing, retries, permission checks, money movement, approval gates. The model owns anything that requires reading — classifying a complaint, extracting fields from a messy document, diagnosing a failure, drafting language a human will see.
The joint between the two is structured outputs. The model doesn’t get to say “I think we should probably refund this one” in prose; it returns { verdict: 'refund', amount: 41.50, confidence: 0.92 } against a schema, code validates it, and code executes the consequence. Judgment flows through a typed, checkable interface — never directly into control.
A hybrid refund flow — who owns what
- Refund request arrives
- Validate order (code)
Deterministic checks: order exists, within return window, not already refunded. No model needed — there is a right answer.
- Model: classify claim, extract fields
The judgment call: what is the customer actually claiming? Returns a structured verdict against a schema — never free prose.
- Within policy?
Code inspects the validated verdict and the policy table. The model informed this decision; code executes it.
- Issue refund (code, idempotent)
Money moves only through code — retry-safe, logged, permission-checked. The model has no tool that touches the ledger.
- Model: draft customer message
Language is a judgment task — the model’s home turf. Output goes to a template slot, not to a send button.
- Send + write audit record (code)
- Escalate to human queue
Out-of-policy or low-confidence verdicts land here — the explicit exception state every skeleton needs.
- Resolved
Who should own this decision?
Interactive decision tree — outcomes:
- Code owns it
Write the rule, test the rule, never spend a token on it. Putting a model where an if-statement belongs buys you non-determinism and latency in exchange for nothing.
- Model informs, code decides, gate the action
The model may produce a structured verdict; code validates it against policy and executes through an idempotent, permission-checked path — with a human-in-the-loop gate where stakes demand one. The model never holds the pen on irreversible actions.
- Model-routed graph
Enumerate the destinations as nodes; let the model pick the edge via a routing function. You keep testability and audit legibility while buying semantic routing.
- LLM loop — with runtime guardrails
Genuinely open-ended work earns the loop end of the spectrum. Grant it deliberately: budget caps, step limits, sandboxing, a kill switch, and least privilege tools — because the runtime, not the prompt, must enforce the boundaries.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.