The retry ladder: when garbage arrives anyway

The retry ladder

1. Model response arrives

   Raw text, or a tool-call payload. Untrusted either way.

2. Parse + validate at the boundary

   Syntax, then schema, then semantic invariants and existence checks. One function, one place.

3. Conforms and passes invariants?

   Two separate questions with one answer: the shape check and the truth check both have to pass before anything downstream sees the object.

4. Typed object → continue

   Downstream code cannot tell whether this was a first-pass success or a rung-two recovery. That is the goal.

5. Attempts left, and no side effect fired?

   Check the budget before spending it. If the failed call already sent the email or moved the money, retrying the call is not a formatting fix — it is a duplicate action.

6. Classify the failure

   The validator already told you which class it is: parse error, schema violation, or a business-rule failure. Route on that, not on vibes.

7. Rung 0 · mechanical salvage

   No model call. Strip code fences, take the first balanced JSON value, tolerate a trailing comma. Meaning-preserving edits only — and log every salvage as a defect.

8. Rung 1 · repair prompt (format only)

   A fresh, cheap call: “here is text, here is the schema, return conforming JSON.” No task reasoning, no tools, low temperature.

9. Rung 2 · re-ask with the error

   Back to the original call, with the offending output and the validator’s message appended. The only rung that can fix wrong values, because the task context is still there.

10. Rung 3 · fail closed

    Typed error, no partial object, escalate to a human or a fallback path — and save the raw output as a regression fixture.

Rung 0 · Reparse — mechanical salvage, zero model calls

Deterministic, microseconds, free. Strip ```json fences. Extract the first balanced object or array. Use a lenient parser for trailing commas. Unwrap a single-element array when you expected an object.

Rule: only edits that cannot change meaning. Filling in a missing field, guessing an enum value, or coercing "none" to null is not salvage — it is fabrication wearing a parser’s coat.

Rule: count every salvage in a metric. A silent salvage path is how a prompt regression stays invisible for three months, right up until the day the noise stops being mechanical.

Rung 1 · Repair prompt — a second model, one job

One extra call whose entire brief is reformatting: the offending text, the schema, “return only a conforming JSON document.” Often a smaller and cheaper model, low temperature, no tools, no task context.

Fixes: shape problems in output whose content was fine — a missing wrapper, a stringified number, an extra commentary field.

Cannot fix: anything that requires knowing the task. A repairer handed {"priority": "urgent-ish"} and an enum of p1|p2|p3 will pick one, and its pick is a coin flip dressed as a fix. Worse: a repairer facing a required field the source text never mentioned will invent a value, because the schema gives it no way to say “absent”.

Rung 2 · Re-ask with the error — the original call, better informed

Go back to the original request with the full task context, append what came back and what the validator said: “end_date (2025-01-04) is before start_date (2025-03-11); both must be ISO dates and end must not precede start.”

Fixes: wrong values, contradictory fields, hallucinated ids (paired with a lookup tool), and shape problems the repairer would have guessed at.

Costs: a full call, full latency, and full context — the expensive rung. Cap it at one or two attempts, and make the error message specific: a validator dump the model cannot read is a wasted call. If attempt two fails the same way, attempt three usually will too.

Rung 3 · Fail closed — the rung that protects everything downstream

Return a typed failure. Do not pass along a partial object, a coerced default, or an empty array that reads like “nothing found”. Escalate: a human queue, a deterministic fallback, a retry on a later schedule, or an honest error to the caller.

This is a design decision you make once, in advance. The tempting alternative — best-effort partial data — converts a loud, cheap failure into a quiet, expensive one: garbage lands in your database indistinguishable from real records, and the incident starts weeks later when someone notices the numbers.

Always: log the raw output, the schema version, the model and prompt version, and the validator error together. That bundle is a fixture, and the next lesson pair — evals in CI — is where it earns its keep.