The Levers: Instructions, Examples, Structure
Lesson 1 of 4 in Prompting as Engineering.
Prompting is the first adaptation lever because it is the cheapest and the most reversible: you change the input, not the model. No training run, no new artifact to host, no rollback plan beyond reverting a string. Before anyone reaches for Retrieval-augmented generation (RAG) or Fine-tuning, the honest question is whether the behavior you want can be specified in the prompt — and for a large share of tasks, it can.
Why does this work at all? An Instruction-tuned model was post-trained to follow directions placed in its Context window, and — more fundamentally — a large model can pick up a task from the input itself. Brown et al. (2020) demonstrated this with GPT-3: given a natural-language description and a handful of demonstrations inside the prompt, the model performs tasks it was never explicitly trained on, with no gradient updates. They called it In-context learning, and it is the mechanism this whole module rides on. The taxonomy falls straight out of it: Zero-shot means the prompt contains instructions only; Few-shot means it also contains worked examples. Nothing else about the model changes — every prompt “technique” is a different way of spending input tokens.
That framing turns prompting from folklore into engineering. Each lever fixes a specific class of failure, and each has a cost you can measure:
| Lever | What it fixes | What it costs | Where it fails |
|---|---|---|---|
Instructions | Wrong task, wrong audience, wrong tone. Be specific: what to do, for whom, what to refuse, what to do when unsure. | Cheap — tens to hundreds of tokens, sent every call. | Underspecified edge cases: the model fills gaps with its own defaults, which look like randomness. |
Examples (few-shot) | Behavior that is easier to show than to describe: label sets, judgment calls, house style. | Moderate — each example is paid for on every request, forever. | Inconsistent or unrepresentative examples silently redefine the task (next lesson). |
Output structure | Unparseable output. Demand a schema: JSON with named keys, fixed labels, delimiters around free text. | Cheap — and it pays back by making failures machine-detectable. | Structure alone cannot fix wrong content — a well-formed JSON object can still contain a Hallucination. |
The most common prompt failure is not a missing trick — it is an underspecified task. “Summarize this document” leaves the model to decide length, audience, what counts as important, and what to do about missing information. Every one of those unstated decisions becomes variance in your output. A production prompt states them:
You are preparing a briefing for {{AUDIENCE}}.
Task: Summarize the document below in at most {{MAX_BULLETS}} bullet points.
Rules:
- Cover decisions made and questions left open; skip pleasantries and boilerplate.
- Quote numbers exactly as written; do not round or convert units.
- If the document does not state something, write "not stated" — do not guess.
Output format: a Markdown list, one bullet per finding, no preamble.
Document:
"""
{{DOCUMENT}}
"""Every {{PLACEHOLDER}} is a variable your code fills in. Task, audience, length, refusal behavior, and output format are all explicit — nothing is left to the model’s defaults. The delimiters around the document separate your instructions from untrusted content.
Why in-context learning exists at all
Nobody designed in-context learning as a feature. Brown et al. (2020) framed it as an emergent consequence of the pre-training objective: a model trained to predict the next token across an enormous corpus keeps meeting text where a pattern is established and then continued — lists, translations side by side, Q&A pairs, worked problems. Getting the next token right in such passages requires inferring the local pattern and extending it. Scale the model and the data far enough, and that pattern-completion machinery becomes general enough that a prompt full of demonstrations reads, to the model, like one more pattern to continue.
Two engineering consequences follow. First, few-shot performance in the paper generally improved with model scale — in-context learning is substantially a property of large models, so the same prompt can behave very differently across model sizes. Second, nothing persists: the “learning” is conditioning, recomputed from the context on every request. That is precisely why prompting is instantly reversible — and why you pay for your examples on every single call. Why the mechanism works as well as it does remains an active research area; treat any tidy explanation, including this one, as a useful approximation rather than settled theory.
Key terms: Zero-shot, Few-shot, In-context learning, System prompt, Chain-of-thought (CoT)
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.