Not a Database, Not a Search Engine
Lesson 2 of 4 in What Is a Large Language Model?.
The most expensive misconception about LLMs is quiet and reasonable-sounding: it answered my question, so it must have looked the answer up somewhere.
It did not, because there is nowhere to look. When a database answers you, it retrieves a record that someone explicitly stored, and the record either exists or it does not. When a search engine answers you, it retrieves documents other people wrote and shows you where each one lives. When an LLM answers you, nothing is retrieved at all. The training text is gone — what remains is the set of Weights it adjusted. The answer is manufactured on the spot, token by token, from the statistical patterns those weights encode. It is less like pulling a file from a cabinet and more like asking a well-read person to write from memory, if that person were made entirely of probabilities: a blend of everything the model read, weighted by what best fits your prompt.
| Database | Search engine | LLM | |
|---|---|---|---|
Where answers come from | Records someone explicitly stored, returned verbatim | Documents on the live web, found and ranked | Patterns compressed into the model’s weights during training — the answer is generated, not fetched |
Can it cite a source? | Yes — the record is the source | Yes — every result is a link to a real page | Not by itself — a “citation” it prints is generated like every other token, and the referenced thing may not exist |
Wrong but confident? | No — a query finds matching data or fails loudly | Rarely in this way — a bad result is visibly a bad link, not a fabricated one | Yes, by construction — fluent prose is what it was trained to produce, and fluent prose can be false |
How it updates | Instantly, on every write | Continuously, as crawlers revisit pages | Only when someone trains it again — built-in knowledge freezes at the Knowledge cutoff |
Read the third row again, because it is the one that costs money in production. A database has a built-in failure mode that protects you: no rows found. The LLM’s generation loop has no such state. It must produce a next token every round, and the only thing it can consult is its own probability blend. When your question sits in a region the training text covered densely, the most probable continuation is usually true. When it sits in a sparse region — a niche API, an event after the cutoff, a person with three mentions on the internet — the most probable continuation is whatever sounds like the training text, and the model produces it with exactly the same fluent confidence. A wrong answer of this kind is called a Hallucination, and it is not a bug that will be patched out next release; it is the default behavior of a probability blend asked to go beyond what it absorbed. The Security & Risk domain takes hallucination apart properly — mechanisms, measurement, and mitigations.
Where the knowledge lives, then
If nothing is stored, how does the model “know” that Paris is the capital of France? Think of the weights as a lossy compression of the training text. Facts repeated millions of times across the corpus press deep, reliable grooves into the parameters — the model reproduces them almost perfectly. Facts mentioned a handful of times leave shallow traces that blur into neighboring patterns — the model reconstructs them approximately, and approximately is where hallucination lives: a plausible-but-wrong name, a date off by two years, an API parameter that belongs to a similar library. This weight-encoded knowledge is called parametric knowledge, and its two structural limits follow directly from the mechanism: it is frozen at the Knowledge cutoff, and it has no notion of provenance — the model cannot tell you where a groove came from, because grooves do not carry return addresses.
Two refinements keep the picture honest. First, compression is lossy but not uniformly so: text repeated verbatim often enough can be memorized nearly exactly, which is why models sometimes regurgitate famous quotations — and why training-data leakage is a real security topic. Second, the standard engineering response to both limits is not to retrain the model but to hand it the facts at question time — paste the current documents into the prompt so the model works from text it can actually see. That pattern, retrieval-augmented generation, gets a full treatment in the Adapting LLMs domain.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.