Optimizing retrieval: embeddings and rerankers
Lesson 4 of 4 in Customizing Foundation Models.
When a RAG system answers badly, the generator usually gets the blame — but retrieval quality is the ceiling: the best FM cannot answer from chunks that were never retrieved, so the retrieval side has tuning levers of its own and the exam asks about them directly. Chunking strategy lives in Domain 1; this lesson covers the rest: the embedding model, its dimensionality, top-k, rerankers, and hybrid search.
| Lever | What it changes | Pull it when |
|---|---|---|
Embedding model | How meaning is encoded — domain fit, languages supported | Retrieval misses obvious matches; the corpus is specialized or multilingual |
Dimensionality | Richness of each vector vs storage and per-query compute (some models offer selectable output sizes) | Scale up for nuance; scale down when index cost and latency dominate |
Top-k | How many chunks reach the prompt | Raise when answers lack context; lower when noise crowds out the signal |
Reranker | A second-stage model re-orders candidates by true relevance to the query | The right chunk is usually retrieved but not in the top spots |
Hybrid search | Combines semantic (vector) search with keyword matching | Queries contain exact identifiers — SKUs, error codes, part numbers — that embeddings blur |
Hybrid retrieval with reranking
- Query
- Semantic (vector) search
Embedding similarity finds conceptually related chunks — strong on paraphrase, weak on exact strings.
- Keyword search
Lexical matching catches exact identifiers (error codes, SKUs) that embeddings blur together.
- Merge candidates
- Rerank by relevance
A reranking model scores each candidate against the query, promoting the truly relevant few.
- Top chunks → prompt
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.