Warmup Guide — Observability, Drift & Governance

Zero-to-expert primer for Phase 12: keeping shipped models honest — the monitoring stack, drift mathematics (PSI/KS), retraining as policy, and the governance artifacts that make ML auditable.

Table of Contents

• Chapter 1: Why Models Rot
• Chapter 2: The Five-Layer Monitoring Stack
• Chapter 3: The Drift Taxonomy
• Chapter 4: PSI and KS — the Workhorse Detectors
• Chapter 5: Monitoring Without Labels
• Chapter 6: Retraining as Policy
• Chapter 7: Model Risk and Governance Artifacts
• Chapter 8: Responsible AI, Operationally
• Lab Walkthrough Guidance
• Success Criteria
• Interview Q&A
• References

Chapter 1: Why Models Rot

A deployed model is a frozen snapshot of a moving world. It degrades through three distinct mechanisms, each needing different detection:

1. The world changes (drift proper): user behavior shifts, new products launch, a pandemic rewrites every prior. Gradual or sudden; inevitable.
2. The plumbing breaks (silent feature failure): an upstream schema change, a default-to-null deploy, a stale cache — the model sees garbage and produces confident, plausible garbage. This is the most common "drift" in practice and the most preventable (Phase 01's contracts + this phase's distribution monitors).
3. The model changes the world (feedback): the recommender trains on clicks it caused (Phase 06's loop); the fraud model changes fraudster behavior (adversarial drift). Monitoring must account for the model's own fingerprints in the data.

The phase's stance: rot is not an if but a when-and-how-fast; the engineering question is detection lag — how long between the world changing and you knowing.

Chapter 2: The Five-Layer Monitoring Stack

Each layer catches what the previous can't; mature teams run all five:

Two structural facts: layer 4 is gated on labels (often delayed days or forever — Ch. 5 exists because of this), and alerts must carry attribution — "PSI breach on merchant_category (0.31), top contributing bins: …" is actionable; "model drift alert" is noise that trains on-call to ignore the channel (alert fatigue is the death of monitoring programs; tune for precision first).

Chapter 3: The Drift Taxonomy

With features $X$ and target $y$, the joint $P(X, y)$ can shift three ways:

• Covariate drift: $P(X)$ changes, $P(y|X)$ stable — new traffic mix, new demographics. The model may still be correct where it's now being asked to predict — but it's extrapolating into thinner training support. Detectable without labels (layer 3 sees it).
• Prior/label drift: $P(y)$ changes — fraud base rate doubles. Calibration breaks even if ranking survives; thresholds need re-tuning. Visible in the prediction distribution before labels confirm.
• Concept drift: $P(y|X)$ changes — the same features now mean something different (a behavior that signaled churn becomes normal). The damaging one, and invisible to feature-distribution monitors — only label-based layer 4 (or its proxies, Ch. 5) sees it.

The diagnostic discipline: feature drift detected ≠ model damaged (impact depends on the feature's importance and the direction of the shift); model damaged ≠ feature drift visible (concept drift hides). Drift monitors are smoke detectors, not damage meters — they tell you where to look, which is exactly how the lab's report ranks features.

Chapter 4: PSI and KS — the Workhorse Detectors

PSI (Population Stability Index) — the credit-risk industry's standard, because it's interpretable and decomposable:

1. Bin the reference distribution into quantile bins (deciles standard) — reference quantiles, so expected mass is uniform 10% per bin; current data is then counted into those fixed bins. (Binning on current data instead is the classic implementation bug — it hides exactly the shifts you seek.)
2. With reference share $r_i$ and current share $c_i$ per bin:

$$\text{PSI} = \sum_i (c_i - r_i) \ln!\frac{c_i}{r_i}$$

3. Conventional thresholds: < 0.1 stable, 0.1–0.25 moderate, > 0.25 action. These are folklore, not theory — PSI grows with sample size sensitivity and bin count; calibrate thresholds on your metric's historical week-over-week PSI (the lab's no-shift test makes this concrete). Per-bin terms give attribution: which part of the distribution moved.

KS (Kolmogorov–Smirnov): max distance between empirical CDFs, $D = \max_x |F_{ref}(x) - F_{cur}(x)|$, with a p-value. Binning-free and principled — but at production sample sizes (millions), it flags trivially small shifts as significant (the statistical-vs-practical significance gap). Practical pattern: KS/PSI as the detector, an effect-size threshold (PSI level, or KS D itself) as the alerter. For categorical features: chi-square or PSI over category shares (with smoothing for empty cells — the lab handles this).

Prediction-distribution drift (PSI/KS on the model's scores) deserves special status: it summarizes all features through the model's own lens, catches prior drift early, and needs no labels — the single highest-value drift monitor per line of code.

Chapter 5: Monitoring Without Labels

Ground truth arrives late (fraud: 30–90 days), sampled (manual review), or never (what would the rejected loan have done?). The proxy toolbox, in order of deployment frequency:

1. Prediction-distribution monitoring (Ch. 4) — always available, immediate.
2. Feature drift with importance weighting: weight per-feature drift by the feature's model importance — drift in the top feature outranks drift in a minor one (the lab's report does this).
3. Confidence/entropy trends: rising average uncertainty often precedes measurable degradation.
4. Delayed-label backfill: when labels do arrive, join them back to the predictions of that time (event-time discipline — Phase 02!) and compute trailing metrics with their inherent lag; this is layer 4's actual implementation, and it also feeds Ch. 6's performance trigger.
5. Canary cohorts with fast labels: a small segment where ground truth is cheap (e.g., manually reviewed sample) as a leading indicator.

The honest sentence to internalize: between label arrivals, you are flying on proxies; the engineering goal is to choose proxies whose failure correlates with the model's.

Chapter 6: Retraining as Policy

"When do we retrain?" answered by vibe is answered wrong. The policy is a state machine (the lab implements exactly this):

HEALTHY → (drift breach | schedule due | perf breach) → RETRAINING
RETRAINING → (new model trained) → VALIDATING
VALIDATING → (gates pass) → PROMOTING → HEALTHY
VALIDATING → (gates fail) → INCIDENT  (human in the loop — do NOT auto-loop)

The design decisions that matter:

• Triggers are tiered: scheduled retraining (the baseline cadence — data freshness has value even without detected drift), drift-triggered (early, cheap), performance-triggered (late, definitive). Most teams run scheduled + drift-as-accelerator.
• The validation gate compares challenger vs incumbent on current data (the freshest labeled window — not the original test set, which the world has left behind). Phase 08's quality-gate machinery, pointed at a moving target.
• The trap the FSM must encode: drift fired because data is broken (Ch. 1 mechanism 2) → retraining on broken data launders the breakage into the model. Hence the data-quality gate precedes the retrain trigger in the policy: drift + contract violations = data incident, not retraining.
• Failed validation goes to INCIDENT, not auto-retry: a challenger that can't beat the incumbent on current data means something structural changed — a human decides (more data? new features? concept shift needing redesign?). Auto-looping retrains burns compute to hide a signal.

Chapter 7: Model Risk and Governance Artifacts

The JD's "model risk assessment" — the practice (from banking's SR 11-7 heritage, now spreading everywhere via AI regulation):

• Risk tiering: classify each model by decision impact (advisory → automated with human override → fully automated), affected population, and reversibility. The tier sets the process weight: a Tier-1 credit model gets independent validation and quarterly review; a Tier-3 internal search ranker gets CI gates. Proportionality is the point — governance that treats everything as Tier-1 gets routed around.
• The model card (the lab generates one): intended use and out-of-scope uses, training-data provenance (Phase 08's lineage), evaluation results including slice performance, known limitations and failure modes, monitoring plan + retraining policy, owner and approval chain. The test of a good card: an engineer who has never seen the model can decide from it whether a proposed new use is safe.
• Auditability: every production prediction traceable to (model version → training run → data snapshot → approval record) — Phase 08's ledger + registry is this; governance is where it pays off.

Chapter 8: Responsible AI, Operationally

Stripped of slogans, the engineering content:

• Fairness is measured on slices: per-protected-group performance (TPR/FPR/ precision gaps — equalized odds family; selection-rate ratios — demographic parity family). The metrics conflict mathematically (you can't satisfy all simultaneously except in degenerate cases) — choosing which matters is a product/policy decision you document, then monitor like any other metric (Extension B wires it as a CI gate). Small slices need CIs (Phase 11's statistics) or they generate false alarms.
• Human override paths for consequential decisions: the override rate is itself a monitored metric — rising overrides = the model is losing operator trust, often before metrics confirm why.
• Documentation as control: model cards + decision logs + known-limitation registers are what "responsible AI" audits actually inspect (EU AI Act high-risk requirements are, to first order: risk tiering + documentation + monitoring + human oversight — i.e., this chapter).
• The line engineers hold: no governance artifact substitutes for the Phase 11 question "did we measure the harm?" — guardrail metrics in experiments (complaint rates, slice deltas) are responsible AI practiced, not just documented.

Lab Walkthrough Guidance

Lab 01 — Drift Monitor & Retraining Policy, suggested order:

1. psi — reference-quantile binning first (the test plants a shift detectable only with correct binning), smoothing for empty bins, then the hand-computed case.
2. ks_statistic — the two-sample D; check against scipy in the test.
3. DriftMonitor.report — per-feature PSI + KS, importance-weighted ranking, the no-shift calibration case (your false-alarm rate on stable data is the threshold-calibration lesson).
4. RetrainingPolicy — the FSM with guards; the two tests to respect: the data-quality gate preempts the drift trigger, and validation failure lands in INCIDENT (no auto-loop).
5. model_card — assemble from the monitor + policy + provided metadata; the completeness test enumerates the required sections (Ch. 7's list).

Success Criteria

You are ready for Phase 13 when you can, from memory:

1. Name the three rot mechanisms and which monitoring layer catches each.
2. Define the three drift types and state which are label-free detectable.
3. Compute PSI by hand on a 3-bin example; state the binning rule and why; recite the threshold folklore and its caveat.
4. Explain the statistical-vs-practical significance gap at production sample sizes and the detector/alerter split.
5. Draw the retraining FSM with both trap-guards (broken-data laundering; no-auto-loop).
6. List the model card's sections and the risk-tiering axes.

Interview Q&A

Q: Your fraud model's AUC dropped from 0.92 to 0.87 over two months. Walk through the investigation. Timeline first: sudden drop vs slow slide (sudden = plumbing or upstream change — check data-quality layer and deploy/schema timelines; slow = world drift). Then attribution: per-feature PSI ranked by importance — a top feature with PSI 0.4 and a null-rate change is a broken join, not drift; fix the data, don't retrain. If features are healthy: prediction-distribution + prior check (fraud base rate moved? → threshold/calibration fix may suffice) before concept drift (adversarial adaptation — needs new features/labels, not just fresh training). Retraining is the last answer, after the data is known-good — retraining on broken data launders the breakage in.

Q: How do you monitor a model whose labels arrive 60 days late? Layered proxies: prediction-distribution drift (immediate), importance-weighted feature PSI (immediate), confidence trends, a fast-label canary cohort (sampled manual review), and the 60-day backfill join computing true metrics with honest lag (event-time join — predictions matched to their moment). Set the retraining schedule from drift velocity measured on the proxies; let the delayed true metric audit the proxy choices quarterly. Name the residual risk: concept drift that leaves feature distributions intact will take up to 60 days + detection to surface — that bound should be in the model card.

Q: The compliance team asks "how do you know this model is safe to keep running?" Give the systems answer. Point at artifacts, not assurances: the five-layer monitoring stack with current dashboards (data contracts green, PSI under calibrated thresholds, trailing metrics within control bands, slice gaps monitored with CIs), the retraining policy FSM with its gates and history, the model card with known limitations and out-of-scope uses, and the lineage chain from every prediction to an approved training run. Safety isn't a property we assert; it's a process whose evidence is queryable — and here's the query.

References

• Breck et al., The ML Test Score (2017) — monitoring sections
• Gama et al., A Survey on Concept Drift Adaptation (ACM CS 2014) — the taxonomy, rigorously
• Rabanser et al., Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift (NeurIPS 2019)
• Evidently AI docs — the industrial version of the lab; read after building
• Mitchell et al., Model Cards for Model Reporting (FAT* 2019) — arXiv:1810.03993
• Federal Reserve SR 11-7, Guidance on Model Risk Management — the governance origin document
• Hardt et al., Equality of Opportunity in Supervised Learning (NeurIPS 2016) — the fairness-metric mathematics
• EU AI Act high-risk requirements overview — the regulatory shape