Parallelization: sectioning and voting

Lesson 3 of 6 in Workflow Patterns: Five Ways to Compose Model Calls.

Chains and routes are sequential: one call finishes before the next begins. Parallelization runs multiple calls at the same time and merges the results in code — the classic fan-out / fan-in shape. You reach for it in two situations that look similar and are not: when a task splits into independent pieces (sectioning), and when you want independent opinions on the same piece (voting).

Fan-out / fan-in

Task arrives
Code splits the work
The fan-out is decided by code before any model runs: three fixed aspects (sectioning) or N samples of one question (voting). The model does not choose the branches — that distinction becomes the next lesson.
Call A
Sectioning: the security review. Voting: sample 1. Each branch runs with its own clean context, blind to its siblings.
Call B
Runs concurrently with A and C — wall-clock time is one call, not three.
Call C
Branches cannot contradict each other mid-flight because they never communicate. Contradictions surface at the merge — by design.
Code merges results
Sectioning: concatenate or synthesize the sections. Voting: count votes against a threshold. The merge is code — deterministic and testable.
Result

Sectioning

Split the work; each branch does a different job. A pull-request review fans out into three simultaneous calls — one hunting security issues, one checking performance, one verifying the docs still match — and the merge assembles one review.

Why not one call with all three instructions? The same reason routing beat the mega-prompt: attention is a budget. A model told to check five things checks the first two thoroughly. Three focused calls each do one job with full attention — and the whole review still takes one call’s worth of wall-clock time.

The precondition is real independence: each section must be assessable without the others’ output. Sectioning a task whose pieces need to agree with each other produces the merge-conflict failure below.

Voting

Same question, multiple attempts; merge by agreement. A moderation system asks “does this post violate the harassment policy?” five times and acts only when four or more say yes. Because LLM outputs are non-deterministic, each sample is a slightly different draw — voting turns that noise from a bug into a confidence signal.

Voting buys reliability on judgment calls, and you tune the threshold to the error you fear most: require near-unanimity to act when false positives are expensive, act on a bare majority when misses are worse.

The price is blunt: five votes cost five calls for one answer. Voting is for decisions that matter enough to pay a 5x premium — not a default garnish on every call.

Failure modes: merge conflicts and voting clones

Sectioning fails at the fan-in. If the sections were not truly independent, the merge inherits their disagreements: the summary branch calls the contract low-risk while the liability branch flags a poison clause, and naive concatenation ships both verdicts side by side. The merge step needs a real policy — precedence rules in code, or a final synthesis call that must resolve conflicts, not paste sections together.

Voting fails when the votes are correlated. Five samples from the same model share the same training, the same blind spots, and — if you kept the temperature near zero — nearly the same output. That is not five opinions; it is one opinion photocopied five times, and unanimity tells you nothing. Decorrelate the ensemble: vary the prompt framing, raise sampling temperature, or mix models. An ensemble’s value is exactly the independence of its members — the same principle that makes LLM-as-judge panels stronger than a single judge.

Fix the cost model in your head, because it is the opposite of chaining’s. A chain of N steps costs N calls and N calls of latency. A fan-out of N branches costs N calls but roughly one call of latency — you pay tokens to buy time. That makes parallelization the pattern of choice when a deadline (a user staring at a spinner) matters more than the invoice, and voting the pattern of choice when one answer is worth several tries.

Key terms: parallelization, fan-out / fan-in, sectioning, voting, non-determinism

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.