The Full Pipeline

Lesson 1 of 4 in RAG End to End.

The previous module opened up the machinery — embedding models, Chunking, approximate nearest-neighbor search. This one assembles the machine. Retrieval-augmented generation, named and framed by Lewis et al. (2020), is the idea of bolting a retriever onto a generator so the model answers from fetched evidence instead of from whatever its weights happen to remember. In production, that idea becomes a system with two halves that run on completely different clocks.

Ingest runs offline, on the corpus’s clock. Documents are parsed out of their native formats (PDF, HTML, wikis, tickets) into clean text, chunked into retrieval-sized passages, embedded into vectors by an embedding model, and indexed into a Vector database — often alongside a lexical index like BM25 for Hybrid search. This pipeline runs when the data changes: nightly, hourly, or on document-update events.

Serving runs online, on the user’s clock. A query arrives and is embedded with the same embedding model that embedded the chunks — mismatched models produce vectors in unrelated spaces, and Retrieval silently returns noise. The index returns candidate chunks; a reranker reorders them by true relevance; the survivors are assembled into the prompt; and the model generates an answer instructed to cite the evidence it was given. Every stage exists to protect the one non-negotiable property: the answer should be grounded in what was retrieved.

RAG end to end — offline ingest feeds the index; online serving reads it

  1. Source documents

    PDFs, HTML, wikis, tickets, transcripts — every format your organization actually stores knowledge in.

  2. Parse

    Extract clean text and structure (headings, tables) from native formats. Garbage here poisons every later stage.

  3. Chunk

    Split into retrieval-sized passages that each carry one coherent idea, with enough context to stand alone.

  4. Embed chunks

    Run every chunk through the embedding model once, offline. This is what makes serving-time search cheap.

  5. Index

    Store vectors in an ANN index (e.g. HNSW) inside a vector database — often with a lexical index beside it for hybrid search. Re-runs when documents change: this is where freshness lives.

  6. User query

    The online path begins here — everything below runs per request, on the user’s latency budget.

  7. Embed query

    The same embedding model that embedded the chunks. A different model means a different vector space — and meaningless neighbors.

  8. Retrieve candidates

    ANN (and optionally lexical) search returns the top candidates by similarity — optimized for recall, not final ordering.

  9. Rerank

    A cross-encoder rescores each query–chunk pair jointly and reorders. Precision applied to dozens, not millions — next lesson.

  10. Assemble context

    The surviving chunks are formatted into the prompt with identifiers for citation, strongest evidence placed where the model reads best.

  11. Generate with citations

    The model answers from the assembled evidence and cites which chunks support which claims — the grounding contract.

The two-clock structure explains a whole class of production surprises. Freshness is an ingest property, not a serving property. A document that changed an hour ago does not exist for retrieval until the pipeline re-parses, re-chunks, re-embeds, and re-indexes it — the serving path can only read what the index holds. Teams discover this when the model keeps confidently citing last quarter’s policy: the Large language model (LLM) is fine, the index is stale. Decide your re-indexing cadence from how fast your corpus actually changes, and index document timestamps as metadata so the system can prefer — or at least disclose — recency.

The assembled context is where this module meets the previous two: everything you know about Context engineering applies to the final prompt, and where a chunk sits in the Context window affects whether it gets used at all. One boundary note: when a single retrieve-then-generate pass is not enough and you want the system to reformulate queries, retrieve again, and loop, you are orchestrating retrieval with an agent — our sister AI Agent Academy teaches that layer; here we go deep on the pipeline itself.

The grounded-answer assembly prompt
You are answering questions using only the evidence provided below.

Evidence (each chunk has an ID):
{{RETRIEVED_CHUNKS}}

Question: {{QUESTION}}

Rules:
- Answer only from the evidence above. Do not use outside knowledge.
- Cite the chunk ID after every claim, like [chunk-3].
- If the evidence does not contain the answer, say exactly:
  "The retrieved documents do not answer this." Do not guess.

The explicit refusal path is the load-bearing line: without it, the model fills retrieval gaps from its weights and you get fluent, uncited fabrication. Chunk IDs make grounding checkable — you can verify every citation against the evidence.

Key terms: Retrieval-augmented generation (RAG), Retrieval, Chunking, Vector database, Reranking, Grounding

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.