Model Selection Memo Template

Phase 3 · Document 07 · Model Cards and System Cards — Template artifact (not a 15-section concept doc) Prev: 06 — Model Card Checklist · Up: Phase 3 Index

A reusable, copy-paste template for recording a model decision so it's defensible, reproducible, and reviewable. Produce one for every non-trivial model choice. It turns the checklist and your eval into a durable artifact (an Architecture Decision Record for models).

Why write a memo

  • Reproducibility: pins the exact (versioned) model + provider + settings you chose.
  • Defensibility: records why over alternatives, with eval numbers — not vibes.
  • Reviewability: lets teammates, security, and finance check the decision.
  • Future-proofing: when prices/limits/models change (Phase 0.02), the memo shows what to re-verify.

How to use it

  1. Fill a checklist for each candidate first.
  2. Run a small eval on your task (Phase 1.07) — benchmarks shortlist, your eval decides.
  3. Complete the memo; commit it to your repo (e.g. decisions/model-<usecase>-YYYY-MM.md).
  4. Revisit on a schedule or when a gate input changes (price, deprecation, policy).

The template (copy this block)

# Model Selection Memo — <use case>
Date: YYYY-MM-DD   Author: ____   Status: proposed | accepted | superseded
Decision: <chosen model @ versioned ID> via <provider/channel>

## 1. Context & Requirements
- Use case: ____
- Quality bar (how measured): ____
- Latency targets: TTFT p95 ____ / TPOT ____ / total ____
- Cost budget: per request ____ / per month ____
- Data policy: can data leave our env? residency? retention/no-train required? ____
- Required capabilities: tools / structured / reasoning / modality / context: ____
- Volume: ____ requests/day, ~____ in / ____ out tokens each

## 2. Candidates Evaluated
| Model (versioned ID) | Provider | Context | In $/1M | Out $/1M | Cached $/1M | Tools | Structured | Reasoning | Weights/License |
|---|---|---|---|---|---|---|---|---|---|
|  |  |  |  |  |  |  |  |  |  |

## 3. Evaluation Results (your task, not benchmarks)
| Model | Quality score | TTFT p95 | TPOT | Cost/req | JSON valid % | Tool-call valid % | Notes |
|---|---|---|---|---|---|---|---|
|  |  |  |  |  |  |  |  |
- Eval set: <link/size/source>   Method: <programmatic / judge (calibrated) / human>

## 4. Decision Rationale
- Why the chosen model beat the alternatives (tie to §1 requirements + §3 numbers): ____
- Trade-offs accepted: ____

## 5. Gates Cleared
- [ ] License permits our use      - [ ] Data retention/residency acceptable
- [ ] Limitations acceptable        - [ ] Safety/over-refusal acceptable (system card + our probes)

## 6. Risks & Mitigations
- Risk: ____ → Mitigation: ____
- Risk: ____ → Mitigation: ____

## 7. Fallback Plan
- Primary:  <model @ id> / <provider>
- Fallback 1 (on error/ratelimit/timeout): <model> / <provider>
- Fallback 2 (degraded/local): <model> / <provider>
- Routing rule: ____   (see Phase 8 gateway)

## 8. Cost Estimate
- Per request: (in_tok×in$ + out_tok×out$ − cache savings) = $____
- Monthly: requests/day × 30 × per-request = $____
- Margin impact / levers (caching, routing, retrieval): ____

## 9. Operational Notes
- Versioned ID pinned: ____    Deprecation date (if known): ____
- Inference params (temp/max_tokens/etc.): ____
- Monitoring: cost, latency, error/refusal rate dashboards: ____
- Re-verify trigger: price change / deprecation / new model / policy change

## 10. Sign-off
- Engineering: ____   Security/Privacy: ____   Product/Finance: ____

Filled mini-example (illustrative, abbreviated)

# Model Selection Memo — Support-bot RAG answers
Date: 2026-06-13   Status: accepted
Decision: example-instruct-8b-2026-04 via OpenRouter (fallback: Claude Haiku via API)

1. Context: ground answers in help-center docs; TTFT p95 <1.5s; <$0.004/req; no-train required; needs structured citations; ~40k req/day.
2. Candidates: 8B-instruct ($0.20/$0.60), mid-tier ($0.50/$1.50), premium ($3/$15).
3. Eval (200 golden Qs): 8B faithfulness 0.91 / cost $0.0026 / JSON 99% → best cost-quality; premium +0.03 quality at 8× cost (not worth it).
4. Rationale: 8B clears quality bar at the lowest cost; reasoning not needed for grounded Q&A.
5. Gates: Apache-2.0 ✔ · no-train confirmed ✔ · over-refusal probed ✔.
6. Risks: long-tail hallucination → citation enforcement + faithfulness eval gate.
7. Fallback: Claude Haiku on error/ratelimit.
8. Cost: ~$0.0026/req → ~$3.1k/mo; prefix-cache system prompt → ~$2.2k/mo.
9. Pinned versioned ID; dashboards for cost/latency/faithfulness; re-verify on price change.

Cross-references

Up: Phase 3 Index · Next: Phase 4 — Model Catalogs and Trend Reading