Grounding with Context
Lesson 3 of 4 in Context Engineering at the Model Level.
A model’s built-in knowledge is frozen at its Knowledge cutoff, blurry on specifics, and delivered with the same confidence whether it is right or hallucinated. For questions about your facts — your policies, your codebase, your product — the fix is Grounding: put the facts in the context and constrain the answer to them. The model stops being asked to know and starts being asked to read, which is the task it is reliably good at.
The baseline implementation is almost embarrassingly simple, which is exactly why it is worth taking seriously: paste the relevant material into the prompt, and instruct the model to answer from that material only. No embedding model, no index, no retriever — the documents section from last lesson’s skeleton, filled by hand or by a static loader. Two instructions do the heavy lifting: a restriction (“answer using only the material inside the documents section; if it is not there, say so”) and a citation rule (“name the ID of every document you rely on”). The restriction converts unanswerable questions from hallucination opportunities into honest refusals; the citation rule makes every answer checkable against a source a reviewer can open.
Answer the question using only the material inside <documents>.
- If the documents do not contain the answer, reply exactly: "The provided documents do not answer this." Do not guess.
- After every claim, cite the supporting document id in brackets, like [{{DOC_ID_EXAMPLE}}].
- If documents conflict, say so and cite both sides.
<documents>
{{DOCUMENTS_EACH_TAGGED_WITH_A_SHORT_ID}}
</documents>
Question: {{USER_QUESTION}}The refusal phrase is deliberately exact: a fixed string is easy to detect downstream, so your application can distinguish “answered from the documents” from “nothing found” without parsing prose. This same contract survives unchanged when the documents section is later filled by a retriever instead of a loader.
When does this honest baseline actually win? More often than the architecture diagrams admit. Small, stable corpora — a policy handbook, a product FAQ, an API reference — often fit in a modern window whole, and a corpus that fits needs no machinery for deciding what to include. Whole-corpus questions — “summarize the differences between these contracts”, “which of our policies mention remote work?” — are places where fetching the top few fragments serves you worse than showing everything, because the answer lives in the sweep, not in a passage. And on every axis of operations, nothing beats no moving parts: what the model saw is exactly what you pasted, so a wrong answer is debuggable by reading one prompt — no index staleness, no Retrieval misses, no Chunking decisions to second-guess. With the corpus laid out as a stable prefix, Prefix caching means you are not even paying full price to re-send it every call.
| Dimension | Stuff-the-docs baseline | RAG pipeline |
|---|---|---|
Freshness | As fresh as your prompt assembly: re-fetch the source at request time and the context is current | As fresh as the index: new and edited documents appear only after re-ingestion, so staleness is a pipeline property to monitor |
Scale | Hard wall at the window: the corpus (plus instructions, history, headroom) must fit — beyond that, no option exists | Effectively unbounded: the index holds millions of documents; only the retrieved top-k enters the window |
Cost shape | Pay per request for the whole corpus as input tokens — softened by Prefix caching, but the meter still runs on corpus size | Pay per request for only the retrieved slices, plus standing costs: embedding, indexing, and the retrieval infrastructure itself |
Citability | Excellent: every document is present and IDs are yours, so citations point at material you can verify in the same prompt | Good, with a caveat: the model can only cite what was retrieved — a retrieval miss silently removes the right source from existence |
Failure modes | One layer: the model misreads or ignores what is plainly there — debuggable by reading the prompt | Two layers: retrieval can fetch the wrong material, and the model can still misuse the right material — each needs its own diagnosis |
Engineering surface | A template and a loader — reviewable in an afternoon | An Embedding model, Chunking policy, vector index, retriever, and their synchronization — a system you operate |
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.