Few-Shot Mechanics
Lesson 2 of 4 in Prompting as Engineering.
Instructions describe a task; examples define it. When you put demonstrations in the prompt, the model does not treat them as decoration — it treats them as the specification, and it continues whatever pattern they establish. This is In-context learning doing exactly what lesson one said it does, and it cuts both ways: examples can pin down behavior no instruction quite captures, and examples can quietly redefine the task in ways you never intended.
Think of a few-shot prompt as a tiny, implicit training set that ships with every request. Everything you know about training data applies in miniature. Are the examples representative of real inputs, or only the easy ones? Do they cover every label, including the boring catch-all? Do they demonstrate the exact output format your code parses? An example set that is unrepresentative, incomplete, or internally inconsistent specifies an unrepresentative, incomplete, or inconsistent task — and the model will faithfully deliver it.
Zero-shot
Classify the customer message into one category:
billing, bug, feature request, or other.
Message: "I was charged twice for March."
A plausible reply — invented for teaching, as is everything in these tabs:
This looks like a billing issue — the customer appears
to have been double-charged.
Correct, and useless to a parser. Nothing specified that the answer should be the bare label, so the model chose a helpful-sounding sentence. Zero-shot leaves format to the model’s defaults.
Few-shot, same task
Classify each customer message into exactly one category.
Respond with the category only.
Message: "Why did my invoice go up this month?"
Category: BILLING
Message: "The export button crashes the app on iPad."
Category: BUG
Message: "Any plans for a dark mode?"
Category: FEATURE_REQUEST
Message: "I was charged twice for March."
Category:
Now the expected continuation is a single token-cheap label:
BILLING
The examples demonstrated the label set, the casing, and the one-word format — three things the instructions alone left ambiguous. (Both transcripts are invented illustrations, not captured model outputs.)
Two facts about few-shot prompts deserve engineering respect.
Selection and ordering effects are real. The literature has repeatedly documented that few-shot performance can shift with which examples you choose and what order they appear in — sensitivity is well established, though the size of the effect varies by task, model, and era, so treat any specific number you read with suspicion. The engineering response is not to memorize folklore about the best ordering; it is to treat the example set as a tunable input and measure your candidates on your own evaluation data. If a reordering changes your metrics materially, you have learned something about how fragile that prompt is.
Format consistency beats cleverness. The single highest-leverage property of a few-shot prompt is that every example demonstrates the same input-output shape — same delimiters, same label style, same field order — and that this shape is exactly what your downstream code expects. A boring, rigorously consistent example set outperforms an ingenious, slightly inconsistent one as a specification, because the model generalizes the pattern you actually showed it, not the one you meant.
Interactive sorting exercise: Each card changes a few-shot classification prompt. Does it strengthen or weaken the implicit task specification?
Classify each customer message into exactly one category:
BILLING, BUG, FEATURE_REQUEST, or OTHER.
Respond with the category only.
Message: "Why did my invoice go up this month?"
Category: BILLING
Message: "The export button crashes the app on iPad."
Category: BUG
Message: "Any plans for a dark mode?"
Category: FEATURE_REQUEST
Message: "What are your office hours?"
Category: OTHER
Message: "{{USER_MESSAGE}}"
Category:One example per label, identical formatting throughout, and the prompt ends mid-pattern — "Category:" — so the model’s most natural continuation is exactly the label your code parses. Remember: all four examples are re-sent, and re-billed, on every request.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.