Phase 06 — Recommendation Systems
Difficulty: ⭐⭐⭐⭐⭐ Estimated Time: 2 weeks (35–45 hours) Roles Supported: the JD's "recommendation" — the canonical ML-system-design interview and the archetype of personalization at scale
Why This Phase Exists
"Design a recommendation system" is the single most common senior-MLE system-design prompt, and recsys is where every theme of this track converges: two-stage retrieval+ranking (Phases 04–05), feature freshness (Phase 03), feedback loops and position bias (Phase 05), online experimentation as the only truth (Phase 11).
The modeling core this phase implements by hand is implicit-feedback matrix factorization trained with BPR — the pairwise-ranking view of collaborative filtering that connects directly to Phase 05's machinery — evaluated honestly against the baseline every recsys paper must beat and many products don't: popularity.
Concepts
- Explicit vs implicit feedback; why "no interaction" ≠ "dislike" (the missing- not-at-random problem)
- Collaborative filtering: neighborhood methods → matrix factorization; user/item embeddings as learned representations
- BPR (Bayesian Personalized Ranking): pairwise loss over (user, seen, unseen) triples — Phase 05's pairwise idea, recsys edition
- The production architecture: candidate generation (two-tower / ANN — Phase 04's indexes) → filtering → ranking (Phase 05) → re-ranking (diversity, business rules)
- Cold start (users and items), popularity bias, feedback loops
- Evaluation: leave-last-out / time-based splits, recall@k / NDCG@k, coverage and popularity-slice metrics; why offline recsys metrics are weak proxies
Labs
Lab 01 — Implicit-Feedback Recommender (flagship, implemented)
| Field | Value |
|---|---|
| Goal | Implement MF with BPR on numpy: embeddings, sampled (user, pos, neg) triples, SGD; evaluate recall@10 with leave-last-out against popularity and random baselines; measure catalog coverage; handle seen-item exclusion and cold users |
| Concepts | BPR objective and gradients, negative sampling, baseline discipline, coverage vs accuracy |
| Steps | 1. Interaction matrix + leave-last-out split; 2. popularity + random baselines; 3. BPR training loop; 4. top-k recommendation with seen-item masking; 5. recall@k + coverage eval; 6. cold-user fallback |
| How to Test | pytest test_lab.py — 12 tests incl. BPR beating popularity on structured data and the seen-item mask |
| Talking Points | Why pairwise for implicit data? Why is popularity hard to beat (and what's wrong with only beating it on recall)? Where do the trained item embeddings get reused? (candidate-gen ANN!) |
| Resume Bullet | Built an implicit-feedback recommender (BPR matrix factorization) from scratch with leave-last-out evaluation; beat popularity baseline on recall@10 while measuring catalog-coverage trade-offs |
→ Lab folder: lab-01-bpr-recommender/
Extension Project A — Two-Tower on MovieLens (spec)
Train a two-tower model (torch) on MovieLens-100K with in-batch negatives; export item embeddings into your Phase 04 dense index; measure candidate-gen recall@100; compare against the BPR embeddings.
Extension Project B — Feedback-Loop Simulation (spec)
Simulate a recsys serving its own training data for 20 "days" (users click what's shown, model retrains on clicks): measure popularity concentration (Gini) over time; add an exploration epsilon; measure again. The degenerate-loop intuition, by hand.
Guides in This Phase
Deliverables Checklist
- Lab 01 implemented; all 12 tests pass
- Extension A recall table BPR vs two-tower
- Extension B concentration curves