« Phase 17 · Warmup · Lab 01

Staff Notes — Judgment, Review Signal, War Stories

The things that do not fit in a lab: what to look for in a design review, what a good answer sounds like, and the failures that teach.

Table of Contents

1. The question that finds every composition bug

One question, asked of any distributed design:

What travels the whole way, and where could it stop?

Answers, for an agent platform: the human's identity, the tenant, the data classification, the trace id, the region, the freshness of the policy bundle. Six things, and every one of them has a hop where it can quietly stop travelling.

The follow-up that turns it into a review: "show me the line of code where it is attached, and the line where it is read at the far end." If those are the same line, it is not travelling — it is being reconstructed, and reconstruction is where things go missing.

I have never asked this question in a design review without finding something.

2. Review signal: what a strong design says

Phrases that indicate the author has built one of these before:

"The residency check lives inside the router, so the fallback goes through it too."

Structural. They have seen a fallback breach residency, or thought hard enough to see it coming.

"That control is non-blocking — it removes the document and the request continues."

They have distinguished acting from halting, which means their availability SLI is not measuring their barrier policy.

"If we can't staff the review, we shouldn't choose that autonomy band."

They understand escalation as a capacity decision. This is rarer than it should be.

"The scanner is deliberately weak in the reference implementation, because the design shouldn't depend on it."

Structural thinking about an adversarial problem. Compare to "we use a state-of-the-art classifier".

"We haven't tested the exit, so I'd call the concentration risk accepted rather than mitigated."

Honest about the difference between configured and tested. Regulators can tell.

"That's a policy decision — let's write down who owns it and revisit it when the input changes."

Knows which decisions are theirs.

3. Review signal: what a weak one says

"We have defence in depth."

Ask: how deep? Which layers, for which attack, and what makes them independent? Usually there is one layer and a diagram with three boxes.

"Prompt injection is handled by our guardrails."

Ask: what happens when the scanner misses? If there is no answer, they have a filter, not an architecture.

"It fails closed."

Ask about the control plane. Fail-closed there means the control plane's availability becomes the platform's, and nobody has computed that composite number.

"We have a fallback provider."

Ask when it last served production traffic. Configured is not tested.

"The audit log has everything."

Ask them to answer, from the log, "who authorized the release of PMT-771?" If they cannot join approval to action, the log has everything except the join key.

"We'll add observability later."

Observability added later measures what is easy to measure. The seams are not.

"The agent is autonomous but a human reviews everything."

Those are different bands, and the sentence means nobody has decided.

4. Four bugs from building this lab

The reference implementation had four composition bugs. Every one was invisible in the components and obvious in the demo output — which is the phase's argument, arrived at the hard way.

1. The budget refused the happy path. Per-request budget 20,000 µUSD; the frontier route projects 27,000. Every legitimate request was DENIED at the model layer. Both numbers were defensible in isolation; nothing had multiplied them together. Lesson: a budget is only a number once something has priced the path.

2. A barrier filter halted the request. The MNPI memo was removed, correctly, and the run reported DENIED — because "a control denied" and "the request is refused" were the same field. Every filtered document read as an outage. The fix was Denial.blocking, and the distinction only becomes visible when controls compose. Lesson: acting and halting are different, and you cannot see it from inside one control.

3. Short-circuiting under-counted defence depth. Natural implementation: something denied, stop evaluating. Consequence: attack A-01 reported depth 1 instead of 2, because the gateway never ran after the taint rule blocked. The platform's headline security claim was unmeasurable due to an optimization nobody had thought about. Lesson: measurement and efficiency conflict, and measurement wins on the security path.

4. A successful fallback reported as a failure. _route returned the reasons it skipped routes for, and the caller logged them as denials. A fallback that worked came out DENIED with an explanation of why the primary was skipped. Lesson: diagnostics and refusals look identical in a list of strings, and only the caller knows which it has.

Three of the four made the platform look more broken than it was. That is the direction composition bugs usually run in a well-built system, and it is why teams learn to distrust their integration suite — which is exactly the wrong lesson.

5. War stories

The audit trail that named a robot. A trade-surveillance escalation, reviewed nine months later. Every record named surveillance-agent-3. The human who initiated the review was in the session log, and the session log had been rotated at 90 days. The chain had been replaced at one hop, in a helper written by someone who has since left, in a change that passed review because the helper's tests all passed. Nobody could say who asked. The remediation took a quarter and the finding took four minutes to write.

The fallback that went to Ireland. Discovered during a routine review of the model gateway's logs, eleven months after go-live. The primary endpoint had failed over twice, for a total of about forty minutes, and both times traffic went to a West Europe deployment because it was the next entry in the list. The routing code was correct. The residency policy was correct. The residency policy applied to the configured primary. Forty minutes of confidential data in the wrong jurisdiction is a notifiable event in some readings and a very uncomfortable conversation in all of them.

The rung that ate the scanner. An emergency change during a Sev-1 added an injection-scan bypass as a degradation rung. Reviewed, approved, correct in the moment. Fourteen months later a routine load test tripped the ladder to level 2 and the platform served 40 minutes of unscanned retrieval on an ordinary Tuesday. Nobody noticed for a week. The person who added it had been promoted; the person who found it was writing an unrelated audit script. This is the story that made me put is_control on the rung.

The idempotency key that wasn't. A payment release retried by a transport-layer retry policy somebody added to a shared HTTP client to fix an unrelated flakiness. The gateway required an idempotency key; the client generated a fresh one per attempt. Two payments. The client change was eleven lines and had a test.

The approval queue nobody staffed. Escalation designed, built, tested, shipped. Queue grew to 300 items in a fortnight. A manager was given "temporary" bulk-approve access to clear it. That access existed for two years. The control was perfect and the capacity plan did not exist.

6. The eight-minute whiteboard, rehearsed

You will be asked to draw the platform. Rehearse this until it is boring.

0:00–1:00 — the layers. Five, bottom to top: infrastructure, model, knowledge, kernel, channels. Then three cross-cutting: control plane, identity, guardrails. Say "cross-cutting" out loud.

1:00–4:00 — one request. The payment investigation. Each step, and what it denies. This is the core; if you are cut off here you have already delivered the thing being assessed.

4:00–5:00 — the seams. Three: identity propagation, fallback residency, the join key. Say why no component test sees them.

5:00–6:00 — degradation. The five rungs in order, then the invariant unprompted: "and no control is on it — quality may degrade, safety may not."

6:00–7:00 — the evidence pack. Seven questions, where each is answered, and "complete or it names what is missing".

7:00–8:00 — what breaks it. One honest limitation with its mitigation. Correlated failures is the best choice.

Three rules for the delivery. Draw the flow, not the boxes — an architecture diagram of boxes is what everyone brings. Say what each thing denies — capability lists are cheap and refusals are not. Welcome the interruption — the interruption is the interview, and getting to minute 8 uninterrupted usually means you were not saying anything they wanted to push on.

7. Things that sound smart and are not

"We use a multi-agent architecture." Agent count is not a design. What is the delegation boundary, what does the chain carry, and what is the depth bound?

"Everything is event-driven." Then what is the ordering guarantee, and what happens to a control that needs to run before a side effect in a system where nothing is synchronous?

"We're model-agnostic." Almost never true, and prompts do not transfer between families cleanly. Say "we run 5% on the secondary continuously" or do not make the claim.

"We have 99.99% availability." Of what event? An agent platform's availability is a validity predicate over outcomes (Phase 14), not a count of 200s. A degraded run that answered from a stale cache is a 200.

"The LLM decides." The LLM proposes. The gateway decides. If the LLM decides, you have no platform — you have a model with credentials.

"We red-teamed it." How many cases, generated from what, and what is the containment rate? "We tried some jailbreaks" is not a control.

8. Things that sound boring and are not

The join key. One field on every artifact. Its absence is the difference between an evidence pack and a pile of logs, and it is discovered by an auditor rather than a test.

Canonical JSON. sort_keys=True. Its absence makes every historical hash unverifiable the day you change Python versions.

The must_deny=False control case. One test case that asserts a legitimate request is not denied. Without it, a platform that refuses everything passes your entire security suite.

Derived identifiers. REF-0001 instead of uuid4(). It is what makes the whole run reproducible and therefore what makes the safety assertions enforceable in CI. Flaky safety tests get deleted.

Emitting on denial. policy_decision with effect="deny". It is the difference between "the control refused" and "we cannot tell whether the control ran", and it is the first thing an auditor asks for.

The staleness hard stop. One integer. It is the difference between fail-static and fail-open over a long outage.

9. Mentoring on this material

Engineers arrive at this phase good at components. Three moves that help.

Give them the demo output before the code. The four bugs in §4 are visible in the printed run and invisible in the source. Ask them to find one. It teaches the habit of reading the composition's behaviour rather than its structure.

Make them break a seam deliberately. Change delegate_to to replace instead of append, run the suite, watch exactly one test fail — and notice that every component test still passes. Nothing teaches the point faster.

Have them predict before they inject. Give them a chaos case with the expected_ fields blank. Their prediction is a measurement of their model of the system, and getting it wrong is the most useful five minutes in the phase.

The failure mode to watch for: an engineer who builds beautiful components and cannot say what happens when two of them disagree. The tell is that their design documents describe capabilities and never refusals.

10. The first ninety days

If you take a job like the one this track answers, the composition is not what you build first. In order:

Weeks 1–3 — find out what exists. There are already agents in production somewhere, built by a team who did not ask. Find them. Not to shut them down — to understand what path they take to tools, because that is the boundary you will have to enforce at.

Weeks 3–6 — the action boundary. Before the control plane, before the evidence pack: make sure nothing reaches an irreversible tool without a contract check. It is the cheapest control with the largest blast-radius reduction, and it is enforceable at one chokepoint.

Weeks 6–10 — one evidence pack, end to end. Pick the single most sensitive existing agent and make its runs auditable. You will find every seam doing it, and you will have an artifact to show the CTTO's office that is worth more than any roadmap.

Weeks 10–13 — the ladder and the ORR. Written in daylight, with the product owner, before the first incident. Then the readiness review that everything else has to pass.

What not to do first: build the full composition. You will build it for agents that do not exist yet, to a policy nobody has agreed, and the teams you need will be the ones you bypassed.

11. What to say when you do not know

The most common failure in a principal interview is not ignorance; it is a confident guess. The formula that works:

"I don't know. Here's how I'd find out, here's what I'd expect, and here's what would change my answer."

Three examples worth having ready:

"I don't know what CBUAE's current position is on cross-border inference for confidential data. I'd get it from the outsourcing guidance and Compliance rather than infer it, and I'd design assuming in-country until told otherwise, because that assumption is cheap to relax and expensive to add."

"I don't know whether our injection scanner catches that class. I'd measure it against AgentDojo and report the containment rate rather than the detection rate, because the architecture is designed to survive detection failures."

"I don't know how long revocation actually takes here. It's measurable — revoke a test agent and time it — and I'd expect the real number to be worse than the estimate, because it's the sum of bundle propagation, cache TTL and in-flight requests."

Each names the source, the expectation, and the falsifier. That is what a senior technical forum rewards, and it is the same discipline as the disagreement protocol in Phase 16: what would change your mind?