Fine-tuning strategies and catastrophic forgetting

Lesson 2 of 4 in Customizing Foundation Models.

Fine-tuning continues an FM's training on labeled prompt-response pairs so the desired behavior stops being a per-request instruction and becomes part of the weights. On AWS it comes in two managed flavors: Bedrock fine-tuning (select models; you supply JSONL pairs, Bedrock trains and hosts a private custom model — the base is never mutated for anyone else) and SageMaker AI training jobs over open-weight models when you need full control of the process. One serving note worth knowing: customized Bedrock models are typically served through dedicated Provisioned Throughput capacity rather than on-demand token pricing — check current support per model.

Full fine-tuning vs parameter-efficient fine-tuning (PEFT), conceptually
Approach	What trains	Compute & data appetite	Forgetting risk
Full fine-tuning	Every weight in the model	Highest — large GPU budgets, more examples to stay stable	Highest: aggressive updates can overwrite general ability
Parameter-efficient (adapters/LoRA-style)	Small added weight matrices; the base stays frozen	A fraction of full tuning — often one modest GPU class	Lower by construction: the base weights never move

Line chart with fine-tuning steps on the x axis and benchmark score on the y axis. One curve, labeled target task, rises steeply from about 0.45 to about 0.9 as tuning proceeds. A second curve, labeled general capability, starts near 0.8 and sinks to roughly 0.58 over the same steps. The widening gap between the rising task curve and the falling general curve is catastrophic forgetting: narrow gradients overwriting broad ability.

The catastrophic-forgetting trade: narrow fine-tuning drives the target task up while general capability drifts down. The mitigations flatten the falling curve — mix general data into the tuning set, lower the learning rate, or freeze the base with parameter-efficient tuning. (illustrative — source: Amazon Bedrock user guide — custom models)

What does Bedrock fine-tuning data look like?

JSONL of prompt-completion pairs (format varies slightly by model family). Quality beats quantity: a few hundred to a few thousand clean, representative pairs typically outperform mountains of noisy ones.

Full or parameter-efficient — what does the exam want?

Conceptual discrimination only: full tuning updates everything (max adaptation, max cost, max forgetting risk); PEFT trains small added weights against a frozen base (cheaper, safer, usually sufficient). Scenario says "limited budget / preserve general capability" → PEFT.

Does fine-tuning add knowledge?

It is the wrong tool for facts. Tuned-in facts go stale and cannot be cited. Behavior, format, tone, task skill → fine-tune. Knowledge → RAG. Domain language itself → continued pre-training (next lesson).

Can I combine rungs?

Routinely: a fine-tuned model behind a RAG pipeline is the standard production stack — tuned behavior, retrieved knowledge. The ladder is cumulative, not either/or.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.