« Phase 16 · Warmup · Track Overview

Core Contributor — The Literature and the Open Tooling

This phase has no engine to contribute to in the way Kafka or OpenTelemetry do. What it has is a literature, a small set of open tools, and a genuine gap. Read this if you want the primary sources rather than the summaries.

Table of Contents

• 1. Why read the primary sources
• 2. The SRE Workbook, chapter by chapter
• 3. Where ADRs came from
• 4. ADR tooling
• 5. Incident command and its origins
• 6. DORA, and what it actually measured
• 7. Operational readiness in the wild
• 8. Policy-as-code for process
• 9. The gap: agentic ORR and eval gating
• 10. Building in-house operating-model tooling
• 11. Contributing

1. Why read the primary sources

Because the summaries lose the conditions. "Error budgets" as commonly repeated is a chart. The SRE Workbook's chapter is mostly about the policy — what happens when it is exhausted, who agreed, and how exceptions are handled — and that is the part that determines whether it works.

Because the practices have known failure modes, documented by people who ran them at scale, and most of them are the ones you will hit. Reading the source is much cheaper than rediscovering them.

Because "blameless" is more precise than it sounds. Blameless does not mean consequence-free; it means the review examines the system that let a person make an ordinary mistake. Getting that distinction wrong produces post-mortems that are either accusatory or useless.

2. The SRE Workbook, chapter by chapter

sre.google/workbook — free, and the chapters that matter for this phase:

The error-budget chapter is the one people cite and have not read. Three things in it that the popular summary omits:

The policy is a written agreement with named signatories, not a convention. It states who agreed, when, and what happens on breach. That formality is the mechanism.

There is an explicit escalation path for when the policy's consequences are disputed — because the authors knew it would be, and building the path in advance is what stops the dispute becoming a referendum on the policy.

Exceptions are anticipated and bounded. The chapter treats them as normal and specifies who may grant one, which is exactly the design in this phase's lab.

And the sentence worth carrying from the on-call chapter: an on-call rotation that generates more than about two pages per shift is not sustainable, which is the number that makes shared on-call either a leadership practice or a punishment.

3. Where ADRs came from

Michael Nygard's 2011 post is two pages and it is the whole idea. Worth reading in the original because the framing has been lost in the templates.

His argument: architecture documents go stale because they describe a state, and states change. A decision does not change — it was made, at a time, with certain information. So record decisions, not state, and the record stays true forever even when the architecture moves.

Which is the justification for immutability. An edited ADR is a state document again.

The template evolution:

MADR's addition of per-option analysis is the meaningful one, and the reason is the same as the lab's two-option rule: one option is not a decision, it is a description. Forcing the author to articulate what they rejected is where most of the value sits.

The Y-statement form is worth knowing for a different reason — it is the shortest thing that still carries the trade-off, and "accepting…" is the negative-consequences field in miniature. For a small decision it is enough.

4. ADR tooling

Small tools, and the smallness is appropriate — ADRs are markdown in the repo.

adr-tools (npryce/adr-tools) — bash, and the reference implementation:

adr new "Self-host the 70B model for restricted-data workloads"
adr new -s 7 "Move restricted-data inference back to PTU"   # supersedes ADR-7
adr generate toc
adr generate graph | dot -Tpng > adrs.png                   # the supersession graph

The -s flag is the interesting part: superseding is a first-class operation that writes the link in both directions. Which is exactly the property the lab enforces, and it is worth noticing that the reference tool made the same choice.

log4brains (thomvaill/log4brains) — Node, adds a web UI and a searchable timeline. Useful once there are more than about thirty.

adr-viewer, dotnet-adr, and a dozen language-specific ports — all small, all doing the same thing.

The pattern to adopt regardless of tool: ADRs live in the repository they govern, numbered, in markdown, with the supersession link. Not in a wiki — the point is that they are versioned with the code and travel with it.

And the practice worth stealing from §8 of the deep dive: a comment in the module naming the ADR that governs it. The next person to change that code encounters the reasoning, which is the only mechanism that reliably surfaces an ADR at the moment it matters.

5. Incident command and its origins

PagerDuty's Incident Response documentation (response.pagerduty.com) is the best public material on this, and it is open source (PagerDuty/incident-response-docs).

The roles come from the Incident Command System — developed for wildland firefighting in California in the 1970s, adopted by emergency services worldwide, and adapted by tech via Google and PagerDuty. Knowing the origin explains the design:

The third is the one tech teams break most, and it is the one this phase names: a commander who is debugging is not commanding. The symptom is that nobody decides to degrade until it is too late, because the person who would decide is deep in a stack trace.

Worth reading in the PagerDuty docs: the "during an incident" section, which is a script, and the severity definitions, which are worth copying almost verbatim because getting severity definitions wrong causes more pain than getting the roles wrong.

6. DORA, and what it actually measured

dora.dev, and the Accelerate book. The four keys:

   deployment frequency        lead time for changes
   change failure rate         time to restore service

Two findings that are routinely misread:

Speed and stability are not a trade-off. High performers are better at both, simultaneously. Which is a strong argument for the mechanisms in this phase: they are not a brake, they are what makes speed safe. It is worth having this to hand when a mechanism is characterized as slowing things down.

The predictors are practices, not tools. Trunk-based development, continuous testing, loosely coupled architecture, and — relevant here — a generative culture (Westrum), which is measured by things like whether messengers are punished and whether failure leads to inquiry.

That last one is the connection to blameless post-mortems, and it is empirical rather than ideological: Westrum's typology is a measured predictor of performance, not a values statement.

Where DORA does not transfer cleanly to an AI platform, and worth being able to say:

• "Change failure rate" assumes changes are deployments. A prompt edit (Phase 15) is a change, and most teams do not count it.
• "Time to restore" assumes a restore is possible. A quality regression from a provider-side model change has no rollback.
• None of the four measure quality, which for this platform is half the risk.

7. Operational readiness in the wild

Public ORR material is thinner than it should be, and what exists is worth reading:

AWS Well-Architected — Operational Readiness Reviews (docs) is the most complete public treatment. Its central insight is worth quoting in spirit: an ORR checklist should be built from your own incidents. Every question earns its place by having prevented something, which is why importing somebody else's checklist wholesale produces a form.

Google's Production Readiness Review — described in the SRE Book's Evolving SRE Engagement Model chapter. The interesting part is not the checklist; it is that the PRR is the entry point to a support relationship, so passing it means SRE takes on the pager. That coupling — the gate is tied to who carries the consequences — is what gives it teeth.

Microsoft's Azure Well-Architected operational excellence pillar covers similar ground with different emphasis.

What none of them cover, and what this phase adds: the agent-specific rows. Eval suite, red-team containment, tool scopes, cost ceiling, autonomy band, evidence-pack generability. There is no public ORR checklist that includes them, which is §9.

8. Policy-as-code for process

The mechanisms in this phase can be enforced by the same tooling that enforces infrastructure policy (Phase 13):

OPA/Conftest works for all of these — the input is a PR's metadata rather than a Terraform plan, and the policy is the same shape:

package deploy

deny[msg] {
    input.change_class == "feature"
    input.budget_state == "freeze"
    not input.exception_id
    msg := "budget is frozen; features require an exception granted by both owners"
}

GitHub rulesets / branch protection covers the simpler cases (required labels, required reviewers, required checks) with no new tooling.

And the observation worth making: this is the same "standards as controls" argument applied to process rather than to code. A rule that the pipeline enforces is a rule; a rule in a wiki is a suggestion — and process rules are not exempt from that.

9. The gap: agentic ORR and eval gating

The genuine gap in the public material, and the place where a bank running agents in production has something to contribute.

What does not exist publicly:

The red-team row is the one with the largest gap and the clearest argument. Every public red-teaming tool reports a detection rate, and detection is the wrong metric — a payload the scanner missed that could not reach a side-effecting tool is a pass, and grading on detection rewards a scanner that blocks everything (Phase 11).

Places that would take a contribution:

• AWS Well-Architected has a Generative AI Lens and its operational-readiness content is thin.
• OWASP GenAI (genai.owasp.org) publishes practical guides and takes contributions.
• CNCF TAG App Delivery / TAG Security produce whitepapers and have an active AI workstream.
• NIST AI RMF profiles are open to community contribution.

None of this is code, and all of it is the kind of contribution that has more effect than a pull request — because the checklist somebody else adopts is a checklist that prevents an incident you will never hear about.

10. Building in-house operating-model tooling

If you build the mechanisms as code, the properties that make them survive:

The criteria are data, not code. A YAML file per checklist, versioned, reviewed like anything else. Which means adding a criterion is a pull request with a discussion, and that is exactly the right amount of friction.

Every criterion names its evidence. The field that turns a belief into a check (Phase 15's argument, applied to readiness).

Generate the template from the criteria. So the ORR form and the scorer cannot drift apart — the same argument as generating the design-review template from the review rules.

Deterministic scoring. Injected clock, sorted output, no randomness. The lab's 113 tests run in under a tenth of a second for this reason, and it means a scoring change is reviewable as a diff.

Property tests on the invariants:

# any mandatory failure => not passed, for ANY set of advisory answers
# an unanswered mandatory criterion is a failure
# tighter budget states permit a subset of looser ones
# an accepted ADR is never mutated by any sequence of operations
# the supersession chain is acyclic
# dropped actions never appear in the completion denominator
# any blocker => not approved

The first is the one worth writing first: Hypothesis will generate the advisory combination that accidentally passes, if your implementation has that bug.

Store the outcomes. An ORR result, a design review, an ADR — all of them are evidence (Phase 15), and Internal Audit will ask for the ORR that preceded a production incident.

11. Contributing

PagerDuty Incident Response (PagerDuty/incident-response-docs) — markdown, Apache 2.0, and it takes contributions. Genuinely useful additions from this domain: incident types for non-deterministic failures, and a triage tree that separates availability from quality.

adr-tools (npryce/adr-tools) and log4brains (thomvaill/log4brains) — small, and approachable. A decision-class field (reversible / irreversible) with a required-signers check would be a natural addition and does not exist.

OWASP GenAI (genai.owasp.org) — the most likely home for an agentic ORR checklist and for red-team acceptance criteria based on containment. They actively want practitioner input, and a checklist derived from real production experience is exactly the shape of contribution they publish.

AWS Well-Architected Generative AI Lens — feedback is accepted, and the operational-readiness section is the thinnest part.

CNCF TAG App Delivery — whitepapers on platform engineering, with an active AI workstream and an open contribution process.

The DORA research (dora.dev) runs an annual survey. Participating from a regulated-AI-platform context adds a data point that the dataset currently has very few of.

And the honest framing: this phase's contribution opportunity is not code. It is a checklist, a template or a set of acceptance criteria, published where somebody else will adopt it. That is a lower-status contribution than a pull request and a higher-leverage one, because a checklist prevents incidents in organizations you will never hear about.