« System Design · Track Overview

Design 01 — The Enterprise Agent Platform

"Design the AI and agentic platform for a Tier-1 bank. Twelve thousand employees will use it, three business units will build agents on it, and it has to satisfy the CBUAE."

The question it turns on: can you compose an SLO and name what each layer denies?

Table of Contents

• 1. Constraints before components
• 2. The request path, layer by layer
• 3. Compose the SLO
• 4. The latency budget
• 5. Identity, end to end
• 6. Failure modes and blast radius
• 7. Evidence
• 8. What you build first
• 9. What changes at 10×
• 10. The questions you will be asked

1. Constraints before components

Ask these before drawing anything. Half the design space disappears here, and discovering that after you have drawn the architecture wastes twenty minutes of a forty-five-minute interview.

Two of those are load-bearing and worth saying out loud:

Residency eliminates routes, not regions. It is not "deploy in UAE North" — it is "every model endpoint that can ever serve a confidential request, including the fallback and the fallback's fallback, is in an approved region." That is a property of the router, not of the deployment.

"Hostile-by-default tenancy" is a design input, not a policy. It means the isolation must be structural — separate indexes, tenant as the first component of every cache key — rather than a filter applied after retrieval.

2. The request path, layer by layer

   Teams / web / API / batch
        │
   ┌────▼───────────────────────────────────────────────────────────┐
   │ USERS & CHANNELS       denies: an unauthenticated human,        │
   │                                a channel not approved for the   │
   │                                data class                       │
   └────┬───────────────────────────────────────────────────────────┘
   ┌────▼───────────────────────────────────────────────────────────┐
   │ CONTROL PLANE          denies: an unregistered agent, a         │
   │  (KYA, policy, quota)          suspended one, a stale           │
   │                                evaluation, an over-quota tenant │
   └────┬───────────────────────────────────────────────────────────┘
   ┌────▼───────────────────────────────────────────────────────────┐
   │ AGENT KERNEL           denies: a run past its step budget,      │
   │  (loop, memory, state)         a delegation past its depth      │
   └────┬───────────────────────────────────────────────────────────┘
        ├──────────────► KNOWLEDGE FOUNDATION
        │                denies: a document behind a barrier, above
        │                        the viewer's clearance, or outside
        │                        their desk
        ├──────────────► MODEL LAYER (gateway)
        │                denies: a route breaching classification,
        │                        residency or budget
        │
   ┌────▼───────────────────────────────────────────────────────────┐
   │ GUARDRAILS             denies: an injected instruction, an      │
   │  (twice: retrieval,            unapproved side-effecting action │
   │   proposed action)             derived from tainted content     │
   └────┬───────────────────────────────────────────────────────────┘
   ┌────▼───────────────────────────────────────────────────────────┐
   │ ACTION GATEWAY         denies: an unregistered tool, a schema   │
   │                                violation, a missing idempotency │
   │                                key, an action without dual      │
   │                                control above threshold          │
   └────┬───────────────────────────────────────────────────────────┘
        ▼
   THE ESTATE  (core banking, payments, CRM, data platform)

Every layer denies something. A layer that denies nothing is a layer you can delete — say that in the review, because it is the fastest way to show the diagram is a design rather than a picture.

Note the two things the diagram makes explicit that most do not:

• Knowledge and the model layer are drawn as a fan-out from the kernel, not as steps in a chain. They are called by the kernel, possibly repeatedly, and treating them as a linear pipeline produces a latency budget that does not match reality.
• Guardrails appear once as a box but run twice — over retrieved content before it enters the prompt, and over the proposed action before it reaches the gateway. Those are different attacks against different targets.

3. Compose the SLO

The single most common failure in this conversation is quoting a target instead of composing one.

Serial dependencies multiply:

$$A_{\text{series}} = \prod_i A_i$$

Naively serial: \( 0.9995 \times 0.999^5 \times 0.995 = 0.9896 \) → 98.96%, which is 7 h 30 m/month. Say the number before anyone asks for it.

Now the three moves that recover it, in order of leverage:

1. Make dependencies degradable rather than serial. The knowledge foundation being down should produce a degraded answer, not a failed request. Same for the reranker, the graph, and the delegate agent. Every dependency you move from "serial" to "degradable" leaves the product.

$$0.9995 \times 0.999^4 \times 0.995 = 0.9906$$

2. Add redundancy where the failure domains are genuinely independent. Two model providers with working fallback: \( 1 - 0.001^2 = 0.999999 \). But only if the failover fits the latency budget (§4) and the failure modes are actually independent — the same region is not independent.

$$0.9995 \times 0.999^3 \times 0.999999 \times 0.995 = 0.9916$$

3. Take core banking off the synchronous path where the action allows it. A payment release must be synchronous. A CRM note does not have to be — an outbox and a relay turn a 99.5% dependency into an eventual one, and the user's request succeeds.

$$0.9995 \times 0.999^3 \times 0.999999 = 0.9965$$

→ 99.65%, or about 2 h 30 m/month. State it as a decision: "I can promise 99.65% for the answer path and 99.5% for anything that must touch core banking synchronously, and those are different SLOs because they have different dependency sets."

The error budget that follows: at 99.65%, 2 h 32 m/month. The burn-rate alerting derives from it — page at 14.4× (2% of the budget in an hour), ticket at 6× and 1× (Phase 14).

And the SLI is not "HTTP 200". For a non-deterministic workload, availability is the fraction of requests satisfying a validity predicate: an answer was produced, it cited at least one retrieved document, and no guardrail blocked it. A 200 carrying "I'm sorry, I can't help with that" is not availability.

4. The latency budget

Write it before designing. Target: 3,000 ms p95.

Four rules, each of which is a red flag if violated:

1. A timeout must be smaller than the remaining budget. A 5-second model timeout inside a 3-second budget is not a timeout, it is a fiction.
2. A fallback that does not fit the headroom is decoration. 570 ms of headroom means the fallback model must reach TTFT in under 570 ms, which usually means it is the small model, not the second frontier provider. Say that — it is the real trade-off.
3. Parallelize everything with no data dependency. Input guardrails ∥ retrieval; BM25 ∥ dense. The 80 ms of input guardrails costs zero wall-clock.
4. p95 of a serial chain is worse than the max of the components' p95s. Tails compound. Budget at p95 per stage and expect the composed p95 to exceed the sum of medians substantially — which is precisely why the headroom line exists.

5. Identity, end to end

Trace one credential from the human to core banking. Name the exchange at each hop and what narrows.

  human in Teams
      │  Entra ID → OIDC id_token + access_token (aud: platform)
      │  claims: sub=layla.almansouri, tid, groups, amr
      ▼
  CHANNEL — validates the token, builds the Principal ONCE
      │  RFC 8693 token exchange
      │  actor: agent SPIFFE ID · subject: the human
      ▼
  AGENT KERNEL — acts as agent-on-behalf-of-user
      │  scopes NARROW: payments.read, kb.read  (not payments.write)
      │  chain: layla.almansouri -> orchestrator
      │  mTLS, SPIFFE SVID, workload identity
      ▼
  DELEGATE AGENT (Group Compliance)
      │  another exchange; chain APPENDS, never replaces
      │  chain: layla.almansouri -> orchestrator -> compliance-agent
      │  depth-bounded (max 3), cycle-checked
      ▼
  ACTION GATEWAY
      │  JIT credential, minted per action, TTL 60 s,
      │  audience-bound to the specific core-banking endpoint,
      │  scoped to the single payment id
      ▼
  CORE BANKING — sees a short-lived, narrowly-scoped credential
                 that names the human and the chain

The five sentences that carry this section:

• "The chain is derived from a verified credential, never asserted in the request." A chain in a request body is a chain the caller can forge.
• "It appends, never replaces." The human stays at the head. The failure this prevents is an audit record that names a robot.
• "Scopes narrow at every hop." An agent gets the task's privileges, not the user's.
• "Credentials are minted just in time and die in a minute." A standing service account with payments.write is the thing this whole architecture exists to avoid.
• "Depth-bounded and cycle-checked." A→B→A is not hypothetical; two agents that each consider the other authoritative will do it on the first ambiguous input.

6. Failure modes and blast radius

The degradation ladder, written in daylight and rehearsed:

Then the sentence, unprompted: "and no control is on it — quality may degrade, safety may not." If a control is too expensive to run at peak, you shed traffic (rung 5), not the control.

One nuance worth raising before they find it: rung 3 is dangerous for some query classes. A sanctions-status answer served from a six-hour-old cache is not a degraded answer, it is a wrong one. So the ladder is per query class: cache-only is fine for "why was this held", and for "is this counterparty sanctioned" the correct rung is refuse. That is a product decision, made with the Product Owner, before the incident.

7. Evidence

In a regulated design, raising this unprompted is one of the strongest signals available.

Three properties, and each is a sentence worth having ready:

"Generated, not assembled." Each artifact is emitted by the step that had the information, at the moment it had it. Assembling at the end means reconstructing, and reconstruction is where fields go missing.

"One join key, stamped in one place." The trace id on every artifact, set by a single emit function so no layer can forget it. The realistic failure is that seven artifact types carry it and one does not — and the one that does not is the approval.

"Complete, or it names what is missing." evidence_complete=False starts a hunt. missing=['approval'] ends one.

And the honest limit, said before they ask: the check verifies an artifact is present, not that it is true. Presence is mechanically checkable; truth needs independent validation and a human panel.

8. What you build first

A design with no sequencing is a wish list. In order, with the reason:

1. The action gateway. Before the control plane, before the evidence pack. It is the cheapest control with the largest blast-radius reduction, and it is enforceable at one chokepoint. Nothing reaches an irreversible tool without a contract check and an idempotency key.

2. Identity propagation. The chain, end to end, with the token exchange at each hop. Retrofitting this is the single most expensive thing on the list, because every downstream artifact written before it is unattributable forever.

3. The model gateway. Routing, residency, budget, and a fallback that fits the headroom. This is where cost and residency are enforced, and both are easier to enforce from day one than to retrofit into twelve agent codebases.

4. Observability and the evidence pack. Spans and artifacts at agent-and-tool granularity. Not because of the regulator — because you cannot operate what you cannot see, and the first incident arrives before the first audit.

5. The control plane. Registries, KYA, policy. By now you have three agents in production and you know what the policy model needs to express, which you did not on day one.

6. Knowledge foundation. Hybrid retrieval, barriers, the graph. Deliberately late: it is the most visible and the least dangerous.

Deferred, explicitly: multi-agent delegation, the semantic cache, self-hosted models, the knowledge graph. Each is a capability; none is a control. Name them as deferred rather than omitted — a deferral is a decision and an omission is an oversight.

9. What changes at 10×

80,000 actions/day, thirty agents, six business units.

The control plane becomes a hot path. At 8,000/day a network policy call is fine; at 80,000 it is a serial dependency on every request. The answer is a local bundle with a TTL, pushed rather than pulled, with staleness alarms and the hard stop — which is why fail-static was designed in at 1× rather than bolted on at 10×.

Tenancy stops being logical. Six business units with hostile-by-default isolation and a shared vector store becomes six indexes, and the cost of that decision is felt in embedding spend. Make it structural early; retrofitting isolation into a shared index is a data-migration project.

Capacity moves from PAYG to a mixed floor. Size dedicated capacity to p50 demand, spill the rest to PAYG. The break-even is a utilization number, and utilization depends on your input/output token mix far more than on price.

The escalation queue becomes a staffing plan. At 80,000 actions and a 2% escalation rate that is 1,600 human reviews a day. If those people do not exist, the escalation is not a control — it is a queue that grows until somebody approves in bulk.

The platform team becomes the bottleneck. If onboarding an agent needs a change in the orchestrator, adoption stalls at exactly the teams with the most leverage. The fix is that agents arrive as registry entries, not branches.

10. The questions you will be asked

"Why not just use LangGraph / Bedrock Agents / Foundry?" — Use them, in the kernel. The platform is the layers around the kernel: identity, policy, gateway, evidence. Frameworks model the flow and none of them model the principal, which is the gap an enterprise platform fills.

"Your SLO is lower than the business wants." — Then here are the three changes that move it, in cost order, and here is what each one buys. That conversation is a design conversation; quoting a higher number is not.

"What if the model hallucinates a payment?" — It cannot execute one. The model proposes; the gateway disposes. The proposal fails schema validation, or the contract check, or dual control, or the taint rule. Four independent refusals, and the defence-depth harness measures how many actually fire.

"How do you know it's secure?" — Because defence depth is a number. For every case in the attack suite I count how many distinct layers denied, require at least two for anything irreversible, and require a written explanation for any result of one.

"What breaks first at scale?" — The control plane on the synchronous path, then the escalation queue. Both are visible in the design before they happen, which is why both have a stated answer.