The Shapes It Takes
Lesson 2 of 4 in Hallucination.
“Hallucination” names one mechanism with several distinct surface shapes — and the distinction is operational, because each shape has a different trigger and a different first line of defense. A working taxonomy for defenders:
Fabricated entities and citations. The model invents things that do not exist — a paper with plausible authors and a well-formed arXiv number, a court case with a realistic docket, a library function that should exist given the API’s naming conventions. This is the purest form of the mechanism: the request demands a specific reference, the weights hold none, and the model produces what a reference looks like. Citation formats are rigid and abundant in training data, so the fabrications are syntactically immaculate — which is exactly what makes them dangerous.
Plausible-but-wrong facts. The entity is real; the attribute is wrong. A real person gets the wrong employer, a real drug the wrong dosage class, a real API the wrong default. These are harder to catch than fabrications because most of the answer checks out, and they concentrate near the model’s knowledge edge: real-but-rare subjects, details that changed after the Knowledge cutoff, facts the training data stated inconsistently.
Unfaithful summaries. Give the model a document and ask for a summary, and the output can contradict or exceed the source — even when the model “knew” better. Summarization research distinguishes intrinsic hallucination (misrepresenting what the source says) from extrinsic (adding content the source never contained), and found both endemic in abstractive systems (Maynez et al. 2020). This shape matters doubly because grounding — the lesson-three fix for missing knowledge — turns every answer into a summarization task: unfaithfulness is the failure mode grounding leaves behind.
Confident arithmetic errors. Ask for multi-digit arithmetic and the model may produce a wrong number with full confidence. Part of the root is representation: tokenizers split digit strings into uneven chunks that align poorly with place value (the tokenization module covers the mechanics), and next-token prediction over such chunks learns approximate, pattern-shaped arithmetic rather than an algorithm. The defense is architectural: route calculation to a calculator or code, not to token prediction.
Sycophantic agreement. Assert something false and ask the model to confirm it, and it is commonly reported that models — tuned on human preference signals that reward agreeableness — will fold: endorsing the user’s error, or revising a correct answer when the user pushes back. The trigger is social pressure in the prompt rather than a knowledge gap, which is why this shape evades knowledge-side fixes and shows up in review-my-work and confirm-my-hypothesis workflows.
| Shape | What it looks like | Typical trigger | First defense |
|---|---|---|---|
Fabricated entities / citations | References, cases, papers, or functions that do not exist — in flawless format | Request demands a specific reference the weights don’t hold | Verify every reference against a source of truth; app-owned citation IDs |
Plausible-but-wrong facts | Real entity, wrong attribute; mostly-correct answers with a poisoned detail | Knowledge edge: rare subjects, post-cutoff changes, inconsistent training data | Grounding in retrieved, current evidence; freshness from Retrieval, not weights |
Unfaithful summaries | Summary contradicts (intrinsic) or exceeds (extrinsic) the provided source | Abstractive compression; prior knowledge overriding the document | Faithfulness/groundedness checks against the source (lesson four) |
Confident arithmetic errors | Wrong numbers delivered with certainty; errors grow with digit count | Tokenized digits + pattern-matching in place of an algorithm | Route math to a calculator or code execution — never token prediction |
Sycophantic agreement | Endorsing a user’s false premise; abandoning a correct answer under pushback | Assertive or emotionally loaded prompts; preference training rewarding agreement | Neutral phrasing in prompts; evals that include false-premise probes |
Interactive sorting exercise: Sort each output into the shape it exemplifies. All examples are invented for this exercise — the products, papers, and numbers are fictional.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.