« Phase 08 · Warmup · Track Overview

Principal Deep Dive — Architecture, Tradeoffs & Blast Radius


Table of Contents


1. The three tradeoffs

Tradeoff 1 — lifetime versus availability. Short credentials bound the damage of a leak and make revocation a timeout. They also mean every hop mints, which puts the STS on the critical path of every call — and a 60-second TTL means a 60-second STS outage stalls the platform.

The resolution is short lifetimes plus an STS built as a data-plane component: stateless, horizontally scaled, no synchronous dependencies, and — the part people miss — the resource server keeps verifying during an STS outage, because verification needs only the public key. Existing credentials keep working for their remaining life, so an STS outage degrades new work rather than in-flight work. That asymmetry is what makes 60 seconds survivable.

Tradeoff 2 — narrowing versus round trips. Narrowing at every hop means an exchange per hop. Three hops is three STS calls inside the latency budget from Phase 00.

The resolution is to narrow at trust boundaries, not at every function call. A hop that stays inside one service, one tenant and one blast radius does not need a new credential. The boundaries worth an exchange are: entering the platform, crossing into another team's agent, and calling a system of record. That is typically two or three exchanges per request, not ten — and each costs a single-digit millisecond because the STS is doing an HMAC and some string comparisons.

Tradeoff 3 — delegation richness versus interoperability. The act chain is precise and it is a claim that many systems do not understand. A legacy core-banking API sees a bearer token and a sub, and the chain is invisible to it.

The resolution: enforce on the chain at the last hop you control, and make the audit record carry it regardless. If core banking cannot read act, the action gateway (Phase 10) reads it, decides, and records it — so the evidence exists even when the ultimate consumer is chain-blind. What you must not do is drop the chain because the last hop ignores it.

2. Where the STS sits

Three placements, and the choice determines your operating model:

PlacementShapeFits
Entra aloneon-behalf-of flow for every hopyou can express your delegation model in Entra's, and you accept a network round trip to the IdP per hop
Entra + a platform STS ← the defaultEntra authenticates the human; a thin internal STS performs agent-to-agent exchangesyou need act chains, per-task scopes and second-scale lifetimes that the enterprise IdP will not issue
Platform STS aloneyour own issuer for everythingalmost never right — you have rebuilt an IdP and inherited its security burden

The middle row is where banks land, and the reason is specific: the enterprise IdP is optimized for human sessions. Minimum token lifetimes are measured in minutes, custom claims go through a governance process, and issuing thousands of 60-second tokens per second is not what it is sized for. A thin STS that consumes Entra tokens and issues platform-scoped ones gets you both: the enterprise remains the authority on who the human is, and the platform controls the agent model.

The critical design rule for that STS: it authenticates the caller, it does not authorize the action. It answers "may this credential be exchanged for that one" — narrowing, chain, lifetime. Whether the resulting action is permitted is Phase 09. Conflating them produces an STS that needs the policy engine on its critical path, and then a policy outage is a total outage.

3. The parked-task problem

The hardest interaction in the phase, and the one most designs miss.

A task pauses for a human approval. Four hours later the approver responds. Every credential involved expired three hours and fifty-nine minutes ago.

Three wrong answers:

Wrong answerWhy
Long-lived tokensdefeats the entire model; a four-hour credential is a service account with extra steps
Refresh tokens held by the agentthe agent now holds durable authority, which is the thing you removed
Silently re-authenticate as the platformthe user vanishes; every action after the pause is attributed to a service

The correct shape: a parked task holds no live credential at all. It holds a reference — the session id, the user id, the delegation chain, the intended action — and on resume the platform re-mints, which means:

  • re-verifying the user's session is still valid (they may have logged out, or been offboarded);
  • re-evaluating policy, because entitlements, agent posture and risk signals have all had four hours to change;
  • issuing a fresh short-lived credential for the remaining work.

Which reframes the lifecycle states from Phase 03: input-required and auth-required are not merely UX states. They are the points at which authority is re-established, and that is why they exist as distinct states rather than one "paused".

The uncomfortable consequence to state in a design review: if the user's authority is gone when the approval arrives, the task fails. That is correct, and it is a behaviour to design the UX around rather than engineer away.

4. Scaling envelope

DimensionFirst constraintSecond
Exchanges/secSTS CPU (signing)replay-cache write rate
Credential lifetime ↓mint rate ↑ proportionallySTS availability becomes load-bearing
Chain depthreliability \( p^n \) and latencypolicy evaluation over a longer chain
NHIsregistry size (trivial)ownership review load — the real limit
Trust domainsSPIRE server capacityfederation bundle management
Key rotationoverlap window managementverifier cache staleness

Two that bite.

Halving the TTL doubles the mint rate. Going from 5-minute to 60-second credentials is a 5× increase in STS load, and the STS is now on the critical path of every request. That is affordable — an HMAC is microseconds — but it must be planned, and it is why the STS is stateless and horizontally scaled rather than a singleton.

Ownership review is the human bottleneck. At 500 NHIs, a quarterly attestation that each still has a valid owner and appropriate scopes is 500 decisions. Nobody does 500 careful decisions, so attestation becomes rubber-stamping — which is worse than not doing it, because it produces evidence of a control that is not operating. The mitigations are structural: default expiry on identities (an NHI that is not re-attested retires automatically), grouping by owner, and risk-tiering so only high-impact identities get individual review.

5. Failure modes and blast radius

FailureBlast radiusDetectionMitigation
STS unavailableall new credentials; in-flight work continuesSTS error ratestateless + multi-replica; verification needs no STS
Signing key compromisedevery token until rotationnone, directlyshort key rotation, HSM/Key Vault custody, asymmetric so verifiers never hold minting keys
A verifier skips audthat service accepts tokens meant for othersnone at runtimea shared verification library, not per-service code
Chain dropped at a hopattribution lost from there onchain-depth distribution as a metricreject a chain-less token for user-scoped actions
Agent compromisedthat agent's scopes, for the TTL, on one audienceanomalous scope/ratenarrow scopes + short TTL: this is what bounds it
Clock driftintermittent, fleet-wide token rejectionsrejection-rate spike correlated to a hostNTP, and skew tolerance sized to observed drift
Replay cache lostone-time tokens replayable within their TTLnoneshort TTLs make the window small; the cache is an optimization, not the guarantee
SPIRE server downno new SVIDs; existing ones valid until expirySVID issuance rateHA SPIRE; TTL long enough to survive a restart
Over-broad registration entryany workload matching one selector gets the identitynone at runtimeall-selectors matching, plus review of entries
Offboarded human still owns 40 NHIsthose identities are unownedjoiner-mover-leaver integrationtie NHI ownership to HR feed; auto-flag on leaver

Two rows worth expanding.

The signing key is the crown jewel. Compromise mints anything, and there is no runtime detection — the tokens are valid. The controls are custody (HSM or Key Vault, never a file), short rotation with an overlap window, and asymmetric signing so that no verifier holds a key capable of minting. The lab uses HMAC for zero dependencies, and its own docstring says this is the one simplification that would be unacceptable in production.

Agent compromise is where the model pays off. With a service account, a compromised agent has everything, everywhere, permanently. With this model it has one audience, one task's scopes, for sixty seconds — and it cannot pivot, because the credential is refused everywhere else on audience alone. That containment is the whole return on the phase's complexity, and it is the sentence to use when someone asks whether all this is worth it.

6. Revocation, honestly

The honest statement, which many designs avoid: an issued credential cannot be recalled. It is a signed assertion; the holder has it; the verifier does not phone home.

So "revocation" is really four different mechanisms with different costs:

MechanismLatencyCost
Expirythe TTLnone — this is the default and should carry most of the load
Refuse re-issuancethe TTLnone — registry state, checked at mint
A deny list at the verifiernear-instantthe verifier now has a synchronous dependency, with the fail-open/fail-shut question
Key rotationnear-instant, for everythinga fleet-wide outage of in-flight credentials

The design that follows: short TTLs plus registry state for the general case, a deny list only for high-impact identities, and key rotation as a break-glass.

The deny list is where the availability question from Phase 00 returns. If the verifier must check a revocation service on every request, that service's availability multiplies into every API's. The answer is the same as for policy: a pushed, cached deny list that fails static — the verifier holds the last known-good list, alarms on staleness, and hard-stops past a threshold. Which means the deny list is only worth carrying for the identities where seconds matter, because it costs an availability dependency.

State the revocation latency explicitly in the design. "Sixty seconds, or two seconds for the twelve identities on the fast path" is an answer a risk function can accept. "We can revoke instantly" is a claim that will not survive the first incident.

7. Decisions that look wrong but are intentional

Distinct error codes internally, one generic error to the caller. Looks like it hurts debuggability. The distinct codes are in the log and the audit record, where a legitimate developer can see them; the caller gets a 401 because varying errors are an oracle for mapping trust boundaries.

Impersonation is a constructor flag, not a request parameter. Looks inflexible. A per-request flag lets any caller choose to erase the chain, which is precisely the decision that must be a deployment policy with a change record.

may_delegate is a claim rather than inferred from depth. Looks redundant given the depth check. It expresses a different thing: this credential is a leaf and must not be traded onward, independent of how deep the chain currently is. A tool-scoped credential should be non-delegable even at depth 1.

The cycle check refuses actor == subject. Looks like an edge case. It stops an agent appending the user as an actor, which would let it launder its own actions as the user's — a subtle and complete defeat of the attribution model.

Attestation refuses ambiguity instead of picking the most specific match. Looks unhelpful. "Most specific" is a policy, and it means the identity a workload receives depends on a tie-break rule nobody remembers. An error at attestation time is a configuration bug surfaced where it can be fixed.

A registration entry with no selectors is refused. Looks like defensive over-engineering. all(x for x in ()) is True, so an empty selector set matches everything — vacuous truth turning into a total authorization bypass.

Verification does not consult the identity registry. Looks like a missing check — surely a suspended agent's token should be refused? That would put a registry lookup on every API call, making the registry a synchronous dependency of the whole platform. Suspension takes effect at issuance, and the TTL bounds the gap. The deny list in §6 is the escape hatch for the cases where that gap is too long.

8. What changes at 10×

At 20 agents and one trust domain, the lab's model is close to shippable. At 500 agents, four trust domains and two external counterparties:

  • Asymmetric signing with a JWKS endpoint stops being optional, because verifiers proliferate and none of them should hold a minting key. Key rotation with an overlap window becomes a scheduled operation.
  • The STS needs its own SLO, because it is now on the critical path of every request. Phase 00's composition arithmetic applies: an STS at 99.9% caps the platform at 99.9%.
  • NHI attestation becomes a programme: automated expiry, ownership tied to the HR feed, risk-tiered review, and a "discovered but unmanaged identity" detection sweep — because there will be identities nobody registered.
  • Federation management becomes real work. Trust bundles rotate, and a stale bundle silently breaks cross-domain mTLS. It needs monitoring like a certificate does.
  • Per-task scope generation replaces static registration: an agent's scopes come from the task, computed by the control plane, rather than from a fixed list. This is the natural endpoint of "the task's scope, not the user's", and it requires Phase 09's policy engine.
  • Chain depth needs a metric, not just a limit. A rising average depth means delegation is spreading, which is a reliability and cost signal before it is a security one.
  • PAM integration for the privileged paths, because at 500 agents some of them will touch systems where session recording is mandatory.

Seams to build now, cheap today: asymmetric signing from day one even if you keep one key; kid in every header; the chain in every audit record even where the consumer ignores it; a mandatory owner field with an HR-linked identifier rather than a free-text name; and the revocation latency written down as a number.