Design 05 — Forecasting Platform

"Design demand forecasting for 50k SKUs × 500 stores, horizon 1–14 days, feeding replenishment." Phase 07 built the methodology; Phases 08, 09, 12 the machinery; capstone 04 assembled them.

1. Requirements

Clarify first (these change the design): consumer = automated replenishment (over/under costs asymmetric → quantile forecasts, not points — the P10/P50/ P90 the inventory policy consumes); 25M series at SKU-store grain; histories from 3 years to 3 weeks; promos/holidays known in advance; nightly cadence with a morning deadline; coherent totals wanted by planners (hierarchy).

Metric: MASE per series (scale-free across 25M series — Phase 07's argument); fleet judged on median AND p90 (the tail is where stockouts live); pinball loss for the quantiles. Baseline: seasonal-naive everywhere, forever — it's the promotion gate's second hurdle, not scaffolding.

2. Architecture

sales/inventory/promo/calendar sources
        │ contracts (P01): the stockout-censoring flag is a FIRST-CLASS column
        ▼
nightly DAG (P08: cached, lineaged):
  validate ─► feature build (lag ≥ horizon!, P07) ─► segment router
        │                                               │
        │                  ┌────────────────────────────┼─────────────┐
        │            smooth/rich history          intermittent     short/new
        │            global GBM per horizon       Croston-class    seasonal-naive
        │            (quantile loss)                               + category prior
        │                  └────────────────────────────┼─────────────┘
        ▼                                               ▼
  rolling-origin backtest gate (P07): challenger vs incumbent vs seasonal-naive,
  identical folds, MASE + pinball — fold table archived as the approval artifact
        ▼
  batch scoring run ─► forecast store (versioned) ─► reconciliation (bottom-up /
        │                                            MinT) ─► consumers
        ▼
  accuracy ledger: forecasts ⨝ actuals as they arrive (event-time, P02)
  per-series trailing MASE control bands ─► retraining FSM (P12)

3. Deep dives

One global model, not 25M models (the scale answer): a single GBM per horizon-bucket trained across all rich-history series, with series identity as features (category, store cluster, price band) — cross-series learning is why ML beats per-series classical methods at retail scale (the M5 lesson); per- series methods remain for segments where the global model's features can't exist (3-week histories). The segment router is itself backtested — assignment is a model decision, gated like one.

The leakage discipline (Phase 07's rule, at platform scale): every feature lag ≥ horizon or known-in-advance (promo flags, calendar); enforced structurally — the feature builder takes horizon as a parameter and the CI suite includes the reached-into-the-future test. One leaked lag produces a fleet of beautiful backtests and a quarter of bad orders.

Stockout censoring (the domain trap — raise it unprompted): sales ≠ demand when shelves were empty; training on censored sales teaches the model that stockouts reduce demand → orders shrink → more stockouts (a feedback loop with a supply chain attached). Mitigation: censoring flags from inventory data, uncensoring estimates (or masking censored days from loss), and monitoring the censored fraction as a data-quality metric.

Quantiles & reconciliation: quantile GBM (pinball loss) per required quantile; calibration monitored (P90 should cover ~90% — a measurable promise, Phase 12's control bands). Hierarchy: bottom-up as the default (simple, coherent); MinT where forecast error covariance justifies it — but reconciliation of quantiles is genuinely hard; say so and scope it (coherent medians + approximate quantile adjustments is the honest v1).

The nightly economics (Phase 08): content-hash caching means unchanged series' pipelines skip — most series don't change most nights; the cold-vs- cached task count is a tracked metric. Scoring is embarrassingly parallel; training is the bottleneck → weekly full retrains + nightly scoring with drift-triggered acceleration (Phase 12 FSM), not nightly retrains by reflex.

4. Operations

Per-series trailing MASE with control bands (25M alert streams is noise — alert on aggregates by segment + the worst-N mover list); residual-drift monitoring distinguishing regime change (→ retrain path) from broken covariates (→ data INCIDENT — the laundering guard, Phase 12); the planted-regime-change fire drill from capstone 04 as a staging test. Forecast versioning: consumers pin to a forecast-run id; revisions are new versions (planners diffing yesterday's plan against a silently mutated forecast is how trust dies).

5. Failure modes

6. Evolution

Probabilistic deep models (DeepAR-class) where the GBM plateaus — gated by the same backtest harness (the harness outlives every model choice; that's the design's actual thesis); intermittent-demand upgrades (Croston → TSB); causal promo-effect models feeding what-if planning; reconciliation done properly (MinT with shrinkage) once consumers demand coherent quantiles.