Picking a pattern
Lesson 6 of 6 in Workflow Patterns: Five Ways to Compose Model Calls.
Five patterns, one selection principle: start with the simplest structure that fits the task, and add machinery only when a measured failure demands it. A single well-gated call beats an unnecessary chain; a fixed chain beats an orchestrator that rediscovers the same plan on every run. Complexity is spent, not accumulated — every pattern you add is more calls to pay for, more latency to explain, and more places to debug.
The matrix below is the module in one table. The two columns that decide real designs are the last two: what you reach for it to get, and what it costs you.
| Pattern | Shape | Reach for it when… | Cost profile | Signature failure |
|---|---|---|---|---|
Fixed sequence; gates between steps | The task decomposes into known, ordered steps and accuracy beats speed | Cost and latency stack linearly with steps — fully predictable | Early errors laundered into polished downstream output; gates rot lax or strict | |
Classify once, dispatch to a specialist path | Inputs fall into distinct categories that deserve different prompts, tools, or models | Near single-call: one cheap classify + one specialist path; big savings from matching model size to category | Misclassification cascades — specialists execute the wrong assignment with full confidence | |
Fan-out / fan-in; branches fixed by code | Independent sections need full attention each (sectioning), or one judgment is worth N independent opinions (voting) | Cost × N branches, latency ≈ one call — tokens buy time | Merge conflicts between non-independent sections; correlated votes that fake confidence | |
Model plans subtasks at runtime; code dispatches workers | Subtasks genuinely cannot be enumerated before runtime | Unpredictable by design — engineer the bound (worker caps, spend ceilings), not the average | Runaway decomposition; overlapping or orphaned briefs surfacing at integration | |
Generate → critique → refine loop; code owns exit | First drafts measurably miss a bar you can write down as a rubric | ≈ 2 calls × rounds; cap rounds in code — diminishing returns after 2–3 | Vague rubrics that never converge; oscillating critiques; generator gaming the judge |
Which pattern does this task need?
Interactive decision tree — outcomes:
- Prompt chaining
Fixed, dependent steps → chain them, and put a code gate after every step whose failure would poison the rest. Cost and latency stack linearly, so keep it short.
- Routing
Classify first with a small, cheap model, then dispatch to specialist paths. Budget real effort for the router’s eval set and a low-confidence fallback route — the cascade is your failure mode.
- Parallelization — sectioning
Fan the independent jobs out concurrently and merge in code. You pay tokens for all branches but wait only for the slowest — the pattern of choice when latency matters.
- Parallelization — voting
Sample the judgment several times and act on agreement — but decorrelate the votes (temperature, varied framings, mixed models) or the ensemble is one opinion photocopied.
- Orchestrator–worker
Let a model write the plan, then have code validate it — worker caps, spend ceilings, coverage checks — before dispatch. Engineer the cost bound, not the average.
- Evaluator–optimizer
Generate, critique against the rubric, refine — with an iteration cap and a best-so-far exit in code. Expect two to three rounds of gain, then diminishing nits.
- Stop — no rubric, no loop
An evaluator without explicit acceptance criteria never converges; it just manufactures new findings each round. Write the rubric first, or route drafts to human review until you can.
- You have left workflow territory
Open-ended tool use with mid-course decisions is an agent loop, not a workflow — the model owns control flow now. That is a legitimate choice with its own module, budgets, and guardrails. Make it deliberately.
Interactive sorting exercise: A colleague describes their task. Which pattern should they reach for?
Tool: Orchestration Sandbox — Wire the five patterns yourself: compose chains, routers, and fan-outs against sample traffic and watch where the cost and the failures land.