Parse, never trust

Lesson 2 of 5 in Structured Outputs: Getting Data, Not Prose.

Here is the sentence to tattoo on the inside of your eyelids: schema conformance is a type guarantee, not a truth guarantee.

A constrained decoder proves the document has the fields you asked for, with the types you asked for, drawn from the enums you listed. It proves nothing about whether the values are true, consistent, permitted, or complete. Those are properties of the world, and the grammar has never seen the world.

Which is why the boundary between model output and your code gets a validator on every rung of the ladder — including the top one.

1 · Semantic correctness — right shape, wrong answer

{"total_cents": 420000, "currency": "EUR"} is a flawless instance of your schema and off by a factor of ten. Every arithmetic slip, misread column, and confidently wrong classification passes a validator untouched.

Defence: cross-field checks and recomputation. If the model returns line items and a total, add the line items yourself and compare. Never ask a model for a number you can compute.

2 · Referential existence — the id that fits the pattern and names nothing

You wrote "pattern": "^ORD-[0-9]{6}

quot; and the model returned ORD-884213. It conforms. The order does not exist.

This is the counterintuitive part: tight patterns make hallucinated identifiers look more legitimate, not less. Before the pattern, a bad id was the customer’s order and blew up loudly. After it, a bad id is a plausible key that flows into a query, returns zero rows, and gets logged as “customer not found”.

Defence: every model-supplied identifier gets an existence check against the system of record before it is used — and identifiers that can be looked up should be looked up with a tool rather than generated.

3 · Internal consistency — fields that contradict each other

start_date after end_date. status: "resolved" with resolution: null. A refund_cents larger than the order total. Individually each field is valid; together they describe an impossible object.

Defence: invariants in the validator, not in the prompt. Most schema languages express some of this (dependent required fields, bounds); the rest is a handful of assertions you run immediately after parsing.

4 · Completeness — the silently truncated list

You asked for every overdue invoice; the model returned four of the eleven it saw, in a conforming array. Nothing in the shape records what is missing. Long inputs, token limits, and “summarise” framings all quietly encourage abridgement.

Defence: make counts explicit and checkable (total_found alongside items), prefer paginated tool calls over one giant extraction, and treat an array that exactly hits your maxItems as suspicious.

5 · Provenance and authorization — conforming data from an untrusted source

The values came from somewhere: a web page, a document, a tool result an attacker could influence. A conforming object is a perfectly good envelope for poisoned content — a note field carrying injected instructions, a url pointing at an exfiltration endpoint, a sql_fragment doing exactly what its name suggests.

Defence: structured output is not sanitisation. Strings inside a valid object still need escaping for their destination, least-privilege limits on what the resulting action may touch, and an authorization check that the current user may act on this record — the model has no idea whose data it just cited.

Sort those defences and you get four layers at the boundary. They fail differently, they are enforced by different code, and skipping one does not make the others cover for it.

Four checks at the boundary — all four run on every response, constrained or not
Layer	The check	Catches	Blind to
Syntax	`JSON.parse` (or the equivalent) succeeds.	Fences, preambles, truncation, malformed escapes.	Everything about meaning. Valid JSON can be entirely wrong.
Schema	A real validator against the same schema you sent the model — Zod, Pydantic, a JSON Schema library.	Missing required fields, wrong types, unknown keys, out-of-enum values, broken patterns.	Values that are well-typed and false.
Semantics	Business invariants and cross-field assertions, plus existence lookups for every id.	Impossible date ranges, sums that do not add up, refunds over policy, ids that name nothing.	Whether the requester is allowed to do this at all.
Authorization	The harness re-checks the acting identity against the resolved record, before any side effect.	An agent acting on another tenant’s order because a poisoned document named it.	Nothing at this layer — but it must be its own layer, not a field the model fills in.

Key terms: schema validation, hallucination, tool-output poisoning, least privilege, provenance, agent harness

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.