The Limits of Imitation
Lesson 4 of 4 in Supervised Fine-Tuning.
Here is the uncomfortable truth that sets up the rest of this domain: SFT teaches the form of a good answer, everywhere — including where the model has nothing true to say.
Think about what the demonstration set actually demonstrates. Every example shows a confident, complete, well-structured response. Almost none show “I’m not sure” — because the annotator, who did know the answer, had no reason to write that. But the annotator’s knowledge and the model’s knowledge are different sets, and the training data was written from the wrong one. So the model learns the lesson the data actually teaches: whatever the question, produce a fluent, authoritative, answer-shaped output. When the question lands inside its knowledge, that policy looks like competence. When it lands outside — past its Knowledge cutoff, in a domain pre-training barely covered — the same policy produces Hallucination with perfect grammar and a confident tone. Fluent wrongness is not a glitch in SFT; it is SFT working as specified on a specification nobody meant to write.
The deeper limit is structural. A demonstration is a single target: “this is the answer.” It carries no information about alternatives — no way to say this answer is better than that one, that two responses are both acceptable but one is clearer, or that a confident wrong answer is far worse than an honest “I don’t know”. Imitation has no vocabulary for better; it only has match this.
| Goal | Can SFT deliver it? | Why |
|---|---|---|
Answer format and structure | Yes — its core strength | Format is visible in every demonstration; next-token imitation copies it directly |
Tone, persona, refusal style | Yes | Averaged out of the demonstration set — this is what curation buys |
Instruction-following and task compliance | Largely yes | The prompt→response pattern is demonstrated thousands of times across task types |
Adding new factual knowledge | Mostly no | A few epochs over a small set barely moves what the model knows — but it does teach the model to talk as if it knows |
Calibrated uncertainty — saying “I don’t know” where it doesn’t | Not by itself | Demonstrations are written from the annotator’s knowledge, not the model’s — the data cannot mark where the model’s knowledge ends |
Graded quality — “prefer A over B” | No — structurally | A demonstration is one target, not a comparison; the loss can only say match this, never this beats that |
The loss itself makes the same point in miniature. Token-level Cross-entropy measures deviation from the reference text — it has no idea which deviations matter. A response that paraphrases the reference gracefully and one that swaps a critical fact can sit at similar loss, because the loss counts token mismatches, not consequences. Everything you actually care about at the margin — truthful over plausible, honest over confident, helpful over verbose — is a comparison between outputs, and the SFT objective has nowhere to put it.
That is the handoff. To go further you need training data that says “A is better than B” — Preference data — and an objective that can consume it. That is Reinforcement learning from human feedback (RLHF) and its successors, the subject of the next module. SFT is not deprecated by any of this: it remains the stage that gets the model into the game — speaking the format, holding the persona — so that preference training has something coherent to compare.
In production
The managed “fine-tuning” services on the major clouds are, mechanically, SFT: you upload prompt→response examples, the platform runs supervised next-token training on the response spans. Everything this module said about SFT is therefore a statement about what those services can and cannot fix.
AWS
Amazon Bedrock’s labeled-data model customization is supervised fine-tuning on your prompt-completion pairs. Expect it to move what SFT moves: output format, tone, task compliance, domain vocabulary. Do not expect it to install new facts or make the model honestly uncertain — teams that need current or proprietary knowledge pair a fine-tuned model with retrieval, using SFT to teach the shape of grounded answers rather than the knowledge itself.
Azure
Azure AI Foundry’s supervised fine-tuning consumes chat-formatted training files — literally the conversations-with-roles structure from lesson two, templated and loss-masked to assistant turns by the service. That makes your dataset the product: coverage, consistency, and tone of your examples become the deployed behavior, and the LIMA lesson applies directly — a smaller, editor-reviewed set routinely beats a bulk export of raw transcripts.
Google Cloud
Vertex AI’s supervised tuning follows the same contract: example conversations in, imitation out. The evergreen discipline is to evaluate before and after against the failure you are actually trying to fix — if the failure is missing knowledge or fabricated details, no amount of supervised examples of other answers will repair it, because imitation transfers form, not facts.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.