Phase 11 — Retrieval-Augmented Generation & Vector Search

The phase that teaches a model to look things up instead of hallucinating from stale weights. A frozen LLM knows only what was in its training data, has no idea what changed yesterday, and cannot cite a source. RAG fixes all three by retrieving relevant documents at query time and grounding the answer in them — and the entire art is in the retrieval, not the generation. This phase builds the retriever from the metal: the vector-search internals (exact kNN, then ANN), the retrieval-quality metrics that let you debug RAG, chunking, reranking, and the grounded prompt. It is the most-asked applied-AI system-design topic of the era.

Why this phase exists

Every team that ships an LLM feature ends up building RAG, and almost every one of them gets it wrong the same way: they treat it as "embed some docs, stuff the top-5 into the prompt" and then wonder why the answers are confidently wrong. The senior move is to see RAG as two systems with two separate evaluations:

1. A retriever — turns a query into the right k chunks. Graded by recall@k, nDCG, MRR. This is a search problem, and search is governed by the recall / latency / memory triangle (exact is accurate but slow; ANN trades a little recall for a lot of speed; quantization trades a little recall for a lot of memory). If retrieval is bad, no generator can save you.
2. A generator — writes a grounded, cited answer from the retrieved context. Graded by faithfulness (did it stick to the context?) and citation accuracy. Its failure modes — lost-in-the-middle, prompt injection hiding in a retrieved doc — are different from the retriever's.

Get fluent here and you can debug the question every RAG team asks at 3 p.m.: "the answer is wrong — is it because we didn't retrieve the right doc, or because the model ignored it?" You can only answer that if you measured the two halves apart.

Concept map

                 ┌────────────────────────────────────────────┐
                 │   RAG = RETRIEVER  +  GENERATOR             │
                 │   (two systems, two evaluations)           │
                 └────────────────────────────────────────────┘
                       │                          │
        ┌──────────────┘                          └───────────────┐
        ▼                                                          ▼
   RETRIEVER (this lab)                                      GENERATOR
   ────────────────────                                      ─────────
   embed → vector search → rerank                            grounded prompt
        │           │          │                             + numbered citations
   cosine/dot/   EXACT kNN   MMR / cross-                    answer ONLY from context
   normalize     (FlatIndex) encoder                              │
        │           │          │                             FAILURES:
        │      ┌────┴────┐     │                             lost-in-the-middle
        │      ▼         ▼     │                             prompt injection in docs
        │    ANN       chunk   │                             stale / skewed index
        │  ┌──┴──┐    size+    │
        │  IVF  HNSW  overlap  │      EVALUATE SEPARATELY:
        │ (nprobe)(M,ef)       │      retriever → recall@k, nDCG, MRR
        │  └──┬──┘             │      generator → faithfulness, citation accuracy
        │     ▼                │
        │  recall / latency / memory triangle
        │     │
        └─────┴────────────────┴──────────────┐
                                               ▼
                                  recall_at_k / ndcg_at_k
                              (grade the index vs the Flat oracle)

The lab

The lab is a runnable, test-verified miniature — see the lab standard. Run it red (pytest test_lab.py), make it green, then read solution.py — and watch recall climb from ~0.88 at nprobe=1 to 1.0 at nprobe=nlist.

Integrated scenario ideas

• Debug a wrong RAG answer: given a query whose answer is in the corpus but the model got it wrong, decide whether it is a retrieval miss (compute recall@k against the Flat oracle) or a generation miss (the chunk was retrieved and ignored), and name the fix for each.
• Size a vector index: 50M chunks at 768-dim fp32 = 150 GB of raw vectors. Show why exact kNN is hopeless, pick IVF vs HNSW, set nlist/nprobe (or M/ef) for a recall target, and apply PQ to fit the memory budget — narrate the triangle.
• Defend RAG over fine-tuning: for a knowledge base that changes weekly and needs citations, show why RAG (re-index, no retrain) beats fine-tuning, and where the line flips (style/format/skill the model lacks → fine-tune; facts that change → RAG).
• Kill the near-duplicate context: take a corpus with five paraphrases of one fact, show naive top-5 wastes the whole budget on duplicates, and fix it with MMR.
• Two-stage retrieve-then-rerank: cast a wide net with fast ANN (high recall, top-50), then rerank to the top-5 for precision; explain why the reranker is too expensive to run on the corpus and only works because the first stage shrank it.
• Survive a corpus poisoning attempt: a retrieved document contains "ignore your instructions"; show how it reaches the prompt and how to defang it by treating retrieved text as untrusted data, not commands.

Deliverables checklist

• lab.py passes pytest test_lab.py -v (and LAB_MODULE=solution does too).
• You can explain cosine vs dot vs L2 and why production normalizes at ingest.
• You can explain IVF's nlist/nprobe, prove recall is monotone in nprobe, and state why nprobe==nlist recovers the exact Flat answer.
• You can describe HNSW's greedy descent and its M/ef knobs, and IVF-PQ's memory savings — i.e. all three legs of the recall/latency/memory triangle.
• You can grade a retriever (recall@k, nDCG, MRR) and a generator (faithfulness, citation accuracy) and explain why they are measured separately.
• You can name four RAG failure modes (stale index, embedding skew, prompt injection in retrieved docs, lost-in-the-middle) and a mitigation for each.

Key takeaways

• RAG is a retrieval problem. The generator is the easy half; the wins and the failures live in the retriever. Measure it on its own — recall@k against an exact oracle — before you ever blame the model.
• ANN is a knob, not a black box. IVF (cluster + nprobe) and HNSW (graph + ef) both expose a recall/latency dial, and PQ adds the memory leg. "Search everything" (nprobe == nlist) always recovers exactness; the engineering is choosing how much recall to trade for speed and bytes.
• Top-k is not the answer; ranked, diverse, budgeted context is. Reranking (cross-encoders for quality, MMR for diversity) and a context budget that respects "lost in the middle" are what turn raw hits into a useful prompt.
• Evaluate the two halves separately, or you will debug blind. Retriever metrics (recall@k, nDCG, MRR) and generator metrics (faithfulness, citation accuracy) catch different bugs; conflating them is the most common RAG mistake.
• RAG beats fine-tuning when the facts change or must be cited. Fine-tuning teaches skills and style; RAG injects current, attributable knowledge. Know which lever the problem needs — and that the common answer is both, fine-tune the behaviour and RAG the knowledge.
• The vector DB is the commodity; retrieval quality is the craft. Chunking, embedding choice, reranking, and a two-sided evaluation move the needle far more than which store you pick — don't agonise over the database, agonise over recall.

Next: Phase 12 — Agentic Reasoning, Tools & Memory.