« Phase 08 · Warmup · Track Overview
Principal Deep Dive — Architecture, Tradeoffs & Blast Radius
Table of Contents
- 1. The three tradeoffs
- 2. Where the STS sits
- 3. The parked-task problem
- 4. Scaling envelope
- 5. Failure modes and blast radius
- 6. Revocation, honestly
- 7. Decisions that look wrong but are intentional
- 8. What changes at 10×
1. The three tradeoffs
Tradeoff 1 — lifetime versus availability. Short credentials bound the damage of a leak and make revocation a timeout. They also mean every hop mints, which puts the STS on the critical path of every call — and a 60-second TTL means a 60-second STS outage stalls the platform.
The resolution is short lifetimes plus an STS built as a data-plane component: stateless, horizontally scaled, no synchronous dependencies, and — the part people miss — the resource server keeps verifying during an STS outage, because verification needs only the public key. Existing credentials keep working for their remaining life, so an STS outage degrades new work rather than in-flight work. That asymmetry is what makes 60 seconds survivable.
Tradeoff 2 — narrowing versus round trips. Narrowing at every hop means an exchange per hop. Three hops is three STS calls inside the latency budget from Phase 00.
The resolution is to narrow at trust boundaries, not at every function call. A hop that stays inside one service, one tenant and one blast radius does not need a new credential. The boundaries worth an exchange are: entering the platform, crossing into another team's agent, and calling a system of record. That is typically two or three exchanges per request, not ten — and each costs a single-digit millisecond because the STS is doing an HMAC and some string comparisons.
Tradeoff 3 — delegation richness versus interoperability. The act chain is precise and it is
a claim that many systems do not understand. A legacy core-banking API sees a bearer token and a
sub, and the chain is invisible to it.
The resolution: enforce on the chain at the last hop you control, and make the audit record
carry it regardless. If core banking cannot read act, the action gateway
(Phase 10) reads it, decides, and records it — so the
evidence exists even when the ultimate consumer is chain-blind. What you must not do is drop the
chain because the last hop ignores it.
2. Where the STS sits
Three placements, and the choice determines your operating model:
| Placement | Shape | Fits |
|---|---|---|
| Entra alone | on-behalf-of flow for every hop | you can express your delegation model in Entra's, and you accept a network round trip to the IdP per hop |
| Entra + a platform STS ← the default | Entra authenticates the human; a thin internal STS performs agent-to-agent exchanges | you need act chains, per-task scopes and second-scale lifetimes that the enterprise IdP will not issue |
| Platform STS alone | your own issuer for everything | almost never right — you have rebuilt an IdP and inherited its security burden |
The middle row is where banks land, and the reason is specific: the enterprise IdP is optimized for human sessions. Minimum token lifetimes are measured in minutes, custom claims go through a governance process, and issuing thousands of 60-second tokens per second is not what it is sized for. A thin STS that consumes Entra tokens and issues platform-scoped ones gets you both: the enterprise remains the authority on who the human is, and the platform controls the agent model.
The critical design rule for that STS: it authenticates the caller, it does not authorize the action. It answers "may this credential be exchanged for that one" — narrowing, chain, lifetime. Whether the resulting action is permitted is Phase 09. Conflating them produces an STS that needs the policy engine on its critical path, and then a policy outage is a total outage.
3. The parked-task problem
The hardest interaction in the phase, and the one most designs miss.
A task pauses for a human approval. Four hours later the approver responds. Every credential involved expired three hours and fifty-nine minutes ago.
Three wrong answers:
| Wrong answer | Why |
|---|---|
| Long-lived tokens | defeats the entire model; a four-hour credential is a service account with extra steps |
| Refresh tokens held by the agent | the agent now holds durable authority, which is the thing you removed |
| Silently re-authenticate as the platform | the user vanishes; every action after the pause is attributed to a service |
The correct shape: a parked task holds no live credential at all. It holds a reference — the session id, the user id, the delegation chain, the intended action — and on resume the platform re-mints, which means:
- re-verifying the user's session is still valid (they may have logged out, or been offboarded);
- re-evaluating policy, because entitlements, agent posture and risk signals have all had four hours to change;
- issuing a fresh short-lived credential for the remaining work.
Which reframes the lifecycle states from Phase 03:
input-required and auth-required are not merely UX states. They are the points at which
authority is re-established, and that is why they exist as distinct states rather than one
"paused".
The uncomfortable consequence to state in a design review: if the user's authority is gone when the approval arrives, the task fails. That is correct, and it is a behaviour to design the UX around rather than engineer away.
4. Scaling envelope
| Dimension | First constraint | Second |
|---|---|---|
| Exchanges/sec | STS CPU (signing) | replay-cache write rate |
| Credential lifetime ↓ | mint rate ↑ proportionally | STS availability becomes load-bearing |
| Chain depth | reliability \( p^n \) and latency | policy evaluation over a longer chain |
| NHIs | registry size (trivial) | ownership review load — the real limit |
| Trust domains | SPIRE server capacity | federation bundle management |
| Key rotation | overlap window management | verifier cache staleness |
Two that bite.
Halving the TTL doubles the mint rate. Going from 5-minute to 60-second credentials is a 5× increase in STS load, and the STS is now on the critical path of every request. That is affordable — an HMAC is microseconds — but it must be planned, and it is why the STS is stateless and horizontally scaled rather than a singleton.
Ownership review is the human bottleneck. At 500 NHIs, a quarterly attestation that each still has a valid owner and appropriate scopes is 500 decisions. Nobody does 500 careful decisions, so attestation becomes rubber-stamping — which is worse than not doing it, because it produces evidence of a control that is not operating. The mitigations are structural: default expiry on identities (an NHI that is not re-attested retires automatically), grouping by owner, and risk-tiering so only high-impact identities get individual review.
5. Failure modes and blast radius
| Failure | Blast radius | Detection | Mitigation |
|---|---|---|---|
| STS unavailable | all new credentials; in-flight work continues | STS error rate | stateless + multi-replica; verification needs no STS |
| Signing key compromised | every token until rotation | none, directly | short key rotation, HSM/Key Vault custody, asymmetric so verifiers never hold minting keys |
A verifier skips aud | that service accepts tokens meant for others | none at runtime | a shared verification library, not per-service code |
| Chain dropped at a hop | attribution lost from there on | chain-depth distribution as a metric | reject a chain-less token for user-scoped actions |
| Agent compromised | that agent's scopes, for the TTL, on one audience | anomalous scope/rate | narrow scopes + short TTL: this is what bounds it |
| Clock drift | intermittent, fleet-wide token rejections | rejection-rate spike correlated to a host | NTP, and skew tolerance sized to observed drift |
| Replay cache lost | one-time tokens replayable within their TTL | none | short TTLs make the window small; the cache is an optimization, not the guarantee |
| SPIRE server down | no new SVIDs; existing ones valid until expiry | SVID issuance rate | HA SPIRE; TTL long enough to survive a restart |
| Over-broad registration entry | any workload matching one selector gets the identity | none at runtime | all-selectors matching, plus review of entries |
| Offboarded human still owns 40 NHIs | those identities are unowned | joiner-mover-leaver integration | tie NHI ownership to HR feed; auto-flag on leaver |
Two rows worth expanding.
The signing key is the crown jewel. Compromise mints anything, and there is no runtime detection — the tokens are valid. The controls are custody (HSM or Key Vault, never a file), short rotation with an overlap window, and asymmetric signing so that no verifier holds a key capable of minting. The lab uses HMAC for zero dependencies, and its own docstring says this is the one simplification that would be unacceptable in production.
Agent compromise is where the model pays off. With a service account, a compromised agent has everything, everywhere, permanently. With this model it has one audience, one task's scopes, for sixty seconds — and it cannot pivot, because the credential is refused everywhere else on audience alone. That containment is the whole return on the phase's complexity, and it is the sentence to use when someone asks whether all this is worth it.
6. Revocation, honestly
The honest statement, which many designs avoid: an issued credential cannot be recalled. It is a signed assertion; the holder has it; the verifier does not phone home.
So "revocation" is really four different mechanisms with different costs:
| Mechanism | Latency | Cost |
|---|---|---|
| Expiry | the TTL | none — this is the default and should carry most of the load |
| Refuse re-issuance | the TTL | none — registry state, checked at mint |
| A deny list at the verifier | near-instant | the verifier now has a synchronous dependency, with the fail-open/fail-shut question |
| Key rotation | near-instant, for everything | a fleet-wide outage of in-flight credentials |
The design that follows: short TTLs plus registry state for the general case, a deny list only for high-impact identities, and key rotation as a break-glass.
The deny list is where the availability question from Phase 00 returns. If the verifier must check a revocation service on every request, that service's availability multiplies into every API's. The answer is the same as for policy: a pushed, cached deny list that fails static — the verifier holds the last known-good list, alarms on staleness, and hard-stops past a threshold. Which means the deny list is only worth carrying for the identities where seconds matter, because it costs an availability dependency.
State the revocation latency explicitly in the design. "Sixty seconds, or two seconds for the twelve identities on the fast path" is an answer a risk function can accept. "We can revoke instantly" is a claim that will not survive the first incident.
7. Decisions that look wrong but are intentional
Distinct error codes internally, one generic error to the caller. Looks like it hurts debuggability. The distinct codes are in the log and the audit record, where a legitimate developer can see them; the caller gets a 401 because varying errors are an oracle for mapping trust boundaries.
Impersonation is a constructor flag, not a request parameter. Looks inflexible. A per-request flag lets any caller choose to erase the chain, which is precisely the decision that must be a deployment policy with a change record.
may_delegate is a claim rather than inferred from depth. Looks redundant given the depth
check. It expresses a different thing: this credential is a leaf and must not be traded onward,
independent of how deep the chain currently is. A tool-scoped credential should be non-delegable
even at depth 1.
The cycle check refuses actor == subject. Looks like an edge case. It stops an agent
appending the user as an actor, which would let it launder its own actions as the user's — a
subtle and complete defeat of the attribution model.
Attestation refuses ambiguity instead of picking the most specific match. Looks unhelpful. "Most specific" is a policy, and it means the identity a workload receives depends on a tie-break rule nobody remembers. An error at attestation time is a configuration bug surfaced where it can be fixed.
A registration entry with no selectors is refused. Looks like defensive over-engineering.
all(x for x in ()) is True, so an empty selector set matches everything — vacuous truth
turning into a total authorization bypass.
Verification does not consult the identity registry. Looks like a missing check — surely a suspended agent's token should be refused? That would put a registry lookup on every API call, making the registry a synchronous dependency of the whole platform. Suspension takes effect at issuance, and the TTL bounds the gap. The deny list in §6 is the escape hatch for the cases where that gap is too long.
8. What changes at 10×
At 20 agents and one trust domain, the lab's model is close to shippable. At 500 agents, four trust domains and two external counterparties:
- Asymmetric signing with a JWKS endpoint stops being optional, because verifiers proliferate and none of them should hold a minting key. Key rotation with an overlap window becomes a scheduled operation.
- The STS needs its own SLO, because it is now on the critical path of every request. Phase 00's composition arithmetic applies: an STS at 99.9% caps the platform at 99.9%.
- NHI attestation becomes a programme: automated expiry, ownership tied to the HR feed, risk-tiered review, and a "discovered but unmanaged identity" detection sweep — because there will be identities nobody registered.
- Federation management becomes real work. Trust bundles rotate, and a stale bundle silently breaks cross-domain mTLS. It needs monitoring like a certificate does.
- Per-task scope generation replaces static registration: an agent's scopes come from the task, computed by the control plane, rather than from a fixed list. This is the natural endpoint of "the task's scope, not the user's", and it requires Phase 09's policy engine.
- Chain depth needs a metric, not just a limit. A rising average depth means delegation is spreading, which is a reliability and cost signal before it is a security one.
- PAM integration for the privileged paths, because at 500 agents some of them will touch systems where session recording is mandatory.
Seams to build now, cheap today: asymmetric signing from day one even if you keep one key; kid in
every header; the chain in every audit record even where the consumer ignores it; a mandatory owner
field with an HR-linked identifier rather than a free-text name; and the revocation latency written
down as a number.