Lab 09 — Hybrid Search from Scratch

Runs fully offline (Python 3 only, no downloads). Implement the core RAG retrieval algorithms — BM25, dense cosine, and Reciprocal Rank Fusion — and see why hybrid beats either alone. Concepts: Phase 9.05, cheatsheet 14.

Goal

Build sparse (BM25) + dense (embedding cosine) retrieval and fuse them with RRF; demonstrate that hybrid handles both exact-term and paraphrase queries.

Files

lab-09-hybrid-search/
  README.md
  hybrid_search.py     ← runnable: BM25 + toy hashing embedding + RRF over a toy corpus

Run

PYTHONHASHSEED=0 python3 hybrid_search.py   # PYTHONHASHSEED=0 makes the toy embedding deterministic

Output shows BM25 top-k, dense top-k, and the fused (RRF) top-k for an exact-keyword query, a paraphrase query, and another paraphrase — hybrid is robust to all.

Exercises

  1. Sparse vs dense: find a query where BM25 wins (exact rare term) and one where dense wins (paraphrase). Explain.
  2. RRF k: change the RRF constant k (60) and observe ranking sensitivity.
  3. Add reranking (stretch): add a toy cross-encoder score (e.g., token-overlap) and rerank the fused top-k → precision boost.
  4. Recall@k: add gold relevance labels per query and compute recall@k for sparse vs dense vs hybrid (reuse lab-05 scorers.py).
  5. Real embeddings (stretch): swap the hashing embedding for a real model (sentence-transformers) and compare.

Deliverables

  • A query where hybrid beats both sparse and dense alone, with the explanation.
  • recall@k for sparse vs dense vs hybrid on ≥5 labeled queries.
  • A note: "when do you need hybrid vs pure vector?"

Why it matters (interview)

RAG quality is dominated by retrieval. Showing you can implement BM25 + RRF and reason about sparse-vs-dense tradeoffs is the core RAG-engineer signal (interview-prep 05).