Lab 01 — Implicit-Feedback Recommender (BPR)

Phase: 06 — Recommendation Systems | Difficulty: ⭐⭐⭐⭐⭐ | Time: 6–8 hours

Personalization, measured honestly: BPR matrix factorization built from four gradient lines, evaluated leave-last-out against the baseline most products never beat. On the reference run: popularity recall@10 = 0.18, BPR = 0.73 — and BPR's coverage is 1.0 where popularity's is 0.28. Both numbers are the lesson.

What you build

  • leave_last_out — the time-respecting split (random splits leak the future)
  • Baselines first — popularity (with seen-item masking) and random; the discipline every recsys claim is measured against
  • BPRModel — user/item embeddings, (user, positive, negative) triple sampling, the four-line SGD update; recommend with seen-masking and a popularity fallback for cold users
  • recall_at_k and coverage — accuracy and catalog health, because winning recall by showing everyone the bestsellers is a measured failure here
  • A representation test: items from the same latent cluster end up closer in embedding space — the "MF is embedding learning" claim, asserted

Key concepts

ConceptWhat to understand
Implicit dataPositive-only, missing-not-at-random — rank, don't regress
BPRRankNet's pairwise logistic loss with (u, i, j) triples; δ = 1 − σ(x) drives all three updates
Seen-item maskingA product decision implemented as −inf on a copy (mutating scores in place is the classic bug)
CoverageThe anti-concentration metric; popularity's 0.28 is the feedback-loop warning rendered as a number
Clustered synthetic dataStructure where personalization is learnable — so the BPR-beats-popularity test is a theorem about your code, not luck

Files

FilePurpose
lab.pySkeleton with # TODO markers (data generator provided)
solution.pyComplete reference; python solution.py prints the three-way table
test_lab.py12 tests: split, baselines, gradient direction, masking, cold fallback, metrics, the beats-popularity and coverage claims, embedding clustering
requirements.txtnumpy, pytest

Run

pytest test_lab.py -v
LAB_MODULE=solution pytest test_lab.py -v
python solution.py   # the table

Success criteria

  • All 12 tests pass — test_bpr_beats_popularity_on_clustered_data and test_embeddings_cluster_by_latent_group are the certificates
  • You can write the four BPR update lines from memory and say what δ → 0 means (the pair is already ordered; learning stops — saturation, by design)
  • You can explain why popularity's coverage is so low and what that predicts about feedback loops (WARMUP Ch. 7)

Extensions

  • Popularity-weighted negative sampling — measure the recall and tail-recall change
  • Export model.Q into your Phase 04 DenseIndex: item-to-item recommendations and the candidate-generation pattern, literally connected
  • AUC evaluation per user; compare its ranking of models against recall@10's

References

  • Rendle et al., BPR (UAI 2009)
  • Covington et al., Deep Neural Networks for YouTube Recommendations (RecSys 2016)
  • WARMUP.md chapters 2–8