« Phase 16 · Warmup · Track Overview
Hitchhiker's Guide — Two-in-a-Box & Engineering Leadership
The fast orientation. What the pieces are, what they are called, and how they fit — before the deep dive takes them apart.
Table of Contents
- 1. Don't panic: the one-paragraph version
- 2. The map
- 3. The vocabulary
- 4. The instrument table
- 5. The ORR gate, memorized
- 6. The five forums, on one card
- 7. The five things that will surprise you
- 8. Reading an ADR
- 9. Where the neighbouring phases connect
- 10. What to build first
1. Don't panic: the one-paragraph version
Two-in-a-box means accountability is undivided: the engineering lead and the product owner are both accountable for availability, cost, security posture, the roadmap and the pager. It exists because an AI platform's decisions — autonomy bands, onboarding pace, cost per action, whether to ship into a breach — are simultaneously product and engineering decisions, and a partition puts them on a boundary where neither party has the full picture. Shared accountability without shared instruments is two people blaming each other, so the phase builds them: an error-budget policy signed before the first breach, a decision router that says who must sign, a disagreement protocol that separates facts from values, immutable ADRs, an ORR that is a gate rather than a grade, a standing design-review checklist, and an action tracker that measures the only thing that matters. And the organizing idea: a standard in code is a control; a standard in a wiki is a suggestion.
2. The map
┌─────────────────────────────────┐
│ ENGINEERING LEAD + PRODUCT │
│ OWNER — undivided accountability│
│ availability · cost · security │
│ roadmap · architecture · pager │
└───────────────┬─────────────────┘
│
┌─────────────┬───────────────┼───────────────┬─────────────┐
▼ ▼ ▼ ▼ ▼
ERROR-BUDGET DECISION DESIGN ORR INCIDENT
POLICY ROUTER REVIEW GATE COMMAND
4 states reversible? 5 questions mandatory mitigate ≠
exceptions → signers + red flags + advisory resolve
expire │ │
▼ ▼
DISAGREEMENT POST-MORTEM
PROTOCOL action completion
facts → measure is the metric
values → escalate BOTH
│
▼
ADRs — immutable, superseded, costs named
│
▼
┌──────────────────────────────────────────────────────────────┐
│ EA · CYBER · MODEL RISK · INTERNAL AUDIT · CTTO │
│ five audiences, five different answers │
└──────────────────────────────────────────────────────────────┘
3. The vocabulary
| Term | Means |
|---|---|
| Two-in-a-box | undivided accountability between two owners, including the pager |
| Undivided | both own the same surface — not "you own tech, I own product" |
| Error-budget policy | budget state → permitted change classes |
| Freeze | budget exhausted; only emergency fixes and reliability work |
| Exception | a deliberate, expiring, both-signed departure from the policy |
| Exception rate | the health metric for the policy itself |
| One-way door | an irreversible decision (Bezos) — both owners sign |
| Disagree and commit | the loser records the disagreement and commits, in writing |
| Falsifier | "what would change your mind?" — the factual/values test |
| ADR | Architecture Decision Record: context, options, decision, consequences |
| Superseded | how an accepted ADR changes; never edited |
| ORR | operational readiness review — the gate into production |
| Mandatory / advisory | gates versus weighted criteria |
| Incident commander | decides; does not debug |
| Mitigation | users no longer affected — not the fix |
| Blameless | the review examines the system, not the person |
| Action completion rate | the only honest measure of a post-mortem culture |
| Paved road | a compliant path that is faster than the non-compliant one |
4. The instrument table
Each removes one specific ambiguity. That is how to remember them:
| Instrument | Removes the question |
|---|---|
| Error-budget policy | "should we ship into a breach?" |
| Decision router | "does this need both of us?" |
| Disagreement protocol | "how do we settle this?" |
| ADRs | "what did we decide, and why?" |
| Design-review checklist | "is this design good enough?" |
| ORR | "is it ready?" |
| Action tracker | "did the post-mortem change anything?" |
And the four states of the budget policy, which is the one you will be asked to recite:
| State | Budget | Permits |
|---|---|---|
| Normal | > 50% | everything |
| Elevated | 20–50% | everything except experiments |
| Reliability focus | < 20% | emergency, reliability, bug fixes |
| Freeze | 0% | emergency + reliability only |
5. The ORR gate, memorized
Classic (8):
SLOs instrumented alerts tested BY INJECTING FAILURE
runbook rehearsed rollback tested
BY AN OUTSIDER dependencies mapped w/ blast radius
capacity vs PROVIDER limit on-call trained + shadow shift
degradation ladder documented
Agent-specific (6):
eval suite at the tier threshold red-team passing ON CONTAINMENT
tool scopes reviewed per-tenant cost ceiling
autonomy band assigned evidence pack generable
And the two scoring rules:
- any mandatory failure fails at 100% advisory;
- every criterion names its evidence — "yes" with no artifact is a belief.
6. The five forums, on one card
| Forum | Bring | Do not say |
|---|---|---|
| Enterprise Architecture | the five-layer reference architecture, the ADRs | "we built something bespoke" |
| Cyber | identity model, containment argument, red-team containment rate | "we prevent prompt injection" |
| Model Risk | inventory entry, validation pack, tiering rationale | "the model is the weights" |
| Internal Audit | a generated evidence pack, the control→evidence map | "we have a control for that" |
| Group CTTO | cost per successful action, capacity forecast, exit readiness | anything at the code level |
7. The five things that will surprise you
1. The product owner carries the pager. It sounds performative and it is the single most effective mechanism in the model — it changes roadmap decisions without anyone lobbying for them.
2. A freeze must still permit reliability work. Obvious once stated, frequently got wrong, and a freeze that blocks the work that would restore the budget extends itself.
3. A high exception rate is data, not misbehaviour. Ten exceptions a quarter means the SLO is wrong. The response is to renegotiate it, not to keep granting them or to start refusing.
4. Never average a values disagreement. The midpoint of two coherent positions has the costs of both and the benefits of neither.
5. An ADR without negative consequences cannot be accepted. Every real decision costs something, and a successor reads the costs first — it is how they tell whether the trade-off still holds.
8. Reading an ADR
The shape, with the parts that carry the weight marked:
# ADR-0007: Self-host the 70B model for restricted-data workloads
## Status
Accepted (layla.almansouri, omar.haddad) — 2026-03-14
Superseded by ADR-0019 ← never edited
## Context
Residency requires in-region inference. PTU capacity in uaenorth is
constrained and the queue is 6 weeks.
## Options considered ← at least TWO
1. Azure OpenAI PTU only
2. Self-host on an AKS GPU pool
3. Hybrid: PTU for internal, self-host for restricted
## Decision
Hybrid, routing by data classification at the gateway.
## Consequences
### Positive
- residency is provable per record
- the self-hosted path doubles as the exit plan for concentration risk
### Negative ← REQUIRED
- a GPU node pool to operate, and a 9-minute cold start
- two model behaviours to evaluate and two prompt variants to maintain
- on-call now needs GPU expertise the team does not have
Three habits when reading one:
- Read the negatives first. They tell you what was accepted, which is what tells you whether the decision still holds.
- Check the options. One option means it was a description, not a decision.
- Check the signers against the class. An irreversible decision with one name is a process failure, whatever the content says.
9. Where the neighbouring phases connect
| Phase | Gives this phase | Takes from this phase |
|---|---|---|
| 00 — Platform model | the five design-review questions | — |
| 02 — MCP | publish() refusing an unclassified tool | the standards-as-controls pattern |
| 09 — Control plane | the eval gate | ORR row 9 |
| 10 — Action gateway | the autonomy ladder | the band assigned at ORR |
| 11 — Guardrails | red-team containment | the Cyber briefing |
| 13 — Backbone | paved-road modules | the mentorship pattern |
| 14 — SRE | the error budget | the policy that governs it |
| 15 — Governance | evidence packs, the inventory | the Audit and Model Risk briefings |
| 17 — Capstone | — | the ORR the capstone must pass |
10. What to build first
- The error-budget policy, signed, on day one — while it is abstract and nobody is under pressure. This is the only item whose value depends on being early.
- The decision router. One table, ten minutes, and it prevents the "did we both need to agree?" conversation entirely.
- The ADR template, with negative consequences required. Then write the first one together.
- The design-review checklist, from the five questions plus whatever red flags you already know. It grows.
- The ORR, before the first production promotion. Retrofitting a gate onto something already live is a negotiation you will lose.
- The disagreement protocol, before the first disagreement. Same argument as the budget policy.
- The action tracker. Trivial, and it is the thing that makes the post-mortems worth writing.
- The forum briefing cards, as you meet each forum for the first time.