RAG, fine-tuning, or prompt engineering
Lesson 5 of 5 in Choosing a Modeling Approach.
Choosing the FM is half the decision; the other half is how to make it yours. Three levers, in cost order: prompt engineering (instructions and examples — no training), RAG (retrieve your documents into the prompt — adds knowledge), and fine-tuning (training on labeled pairs — changes behavior). The mantra that answers most exam items: RAG for knowledge, fine-tuning for behavior, prompts before either. The full customization ladder gets its own module: Customizing Foundation Models.
The RAG request path
- User question
- Embed the query
The same embedding model that indexed the documents turns the question into a vector.
- Vector search
The vector store returns the top-k most similar chunks. Bedrock Knowledge Bases manage ingestion, chunking, and retrieval.
- Augment the prompt
Retrieved chunks are injected into the prompt as grounding context.
- FM generates
The model answers from the supplied context — groundable and citable.
- Answer + citations
| Approach | Fixes | Data needed | Latency & cost effect | Knowledge freshness |
|---|---|---|---|---|
Prompt engineering | Format, tone, task framing | None — just examples in the prompt | Slightly longer prompts; no training cost | Frozen at the base model's cutoff |
RAG | Missing or private knowledge; hallucinated facts; citations | Your documents, chunked and embedded | Retrieval hop + bigger prompts per request | As fresh as the index — update documents, not the model |
Fine-tuning | Persistent behavior: style, format, domain task skill | Labeled prompt-response pairs | Training cost up front; shorter prompts at runtime | Still frozen — new facts need retraining or RAG on top |
The Domain 2 fork: service, FM, built-in, or custom?
Interactive decision tree — outcomes:
- No model at all
Implement the rule. Deterministic logic in code is cheaper, exact, and fully auditable — and "no model" is a legitimate modeling decision, not a cop-out.
- Pre-trained AI service
Textract, Rekognition, Comprehend (+Medical), Transcribe, Translate, Polly, Lex, Personalize. Don't build what AWS sells pre-trained; chain services for multi-step tasks.
- Bedrock FM + prompt engineering
Pick the right model tier on Bedrock and engineer the prompt (system rules, few-shot examples, structured output). Cheapest customization; iterate in minutes.
- Bedrock FM + RAG
Ground the model in your documents via a Knowledge Base / vector store. Knowledge stays fresh and citable without touching weights.
- Customize the FM
Fine-tune (or go deeper — see Customizing Foundation Models) when behavior must persist beyond what prompts and retrieval deliver.
- SageMaker built-in algorithm
XGBoost, Linear Learner, K-Means, Random Cut Forest, DeepAR and friends: your data, AWS's algorithm, no model code to maintain.
- Script mode
Your training script on a managed framework container — custom modeling with managed infrastructure. The default for custom work on MLA-C02.
- Bring your own container
Full control, full responsibility. C02 keeps BYOC light — know when it is forced (exotic dependencies), not its mechanics.
Key terms: Foundation model (FM), Retrieval-augmented generation (RAG), Fine-tuning, Prompt engineering, SageMaker built-in algorithms, Script mode
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.