Capstone 05 — Multi-Model Dashboard

Phase: 11 — Capstone | Difficulty: ⭐⭐⭐⭐⭐ | Time: 5–6 hours

The final capstone: benchmark multiple model variants across multiple tasks, run Pareto analysis, detect regressions, and emit a structured JSON dashboard report.

What you build

  • ModelVariant / BenchmarkTask / TaskResult — data structures
  • run_benchmark_matrix — all variant×task combinations
  • aggregate_results — per-model averages, sorted by accuracy
  • dominates / compute_pareto_front / find_knee — Pareto analysis
  • detect_regression — compare to baseline with delta threshold
  • DashboardReport + build_dashboard — JSON with Pareto front, knee, regression flags, recommended_variant

Integrates

Phase 09 Pareto + Phase 09 regression + Phase 07 profiling + Phase 11 CI pipeline

Sample Output

{
  "models": [...],
  "pareto_front": ["model_A", "model_C"],
  "knee_point": "model_A",
  "regressions": [],
  "recommended_variant": "model_A"
}

Run

pip install -r requirements.txt
python lab.py
pytest test_lab.py -v