Enterprise AI & Agentic Platform — Engineering & Architecture
Target role: Senior Engineer, Platform Engineering and Architecture — the senior technical authority for a Tier-1 bank's enterprise AI & Agentic Platform, operating two-in-a-box with the Platform Product Owner, under CBUAE regulation. Full brief: jd.md.
What this track is. A build-it-yourself curriculum for the person who owns the substrate that a whole bank's agents run on. Not "how to build an agent" — how to build the agentic runtime, model gateway, integration fabric, identity layer, and infrastructure backbone that dozens of agent teams share, and then run it in production, on-call, with an auditor watching.
Every phase builds a runnable, test-verified stdlib miniature of a real platform mechanism: the MCP tool registry, the A2A task machine, the LLM gateway's routing/fallback/semantic-cache path, the PTU capacity planner, the hybrid retriever, the SHACL validator and SPARQL engine, the RFC 8693 token-exchange chain, the Rego-style policy engine, the idempotent action gateway with saga compensation, the guardrail chain, the ISO 20022 parser and transactional outbox, the Terraform-style resource graph with drift detection, the burn-rate alerting engine, and the model-risk evidence graph. You do not configure these — you build them, which is the only way to defend them in a principal-level design review and debug them at 3 a.m.
Table of Contents
- Who this is for
- Relationship to other tracks
- The five-layer stack this track builds
- Phase roadmap
- Suggested schedule
- How to work a phase
- The document set
- Running the labs
- What "done" looks like
- References
Who this is for
You are (or want to be) the engineer who can hold all of these in one head:
- A distributed-systems engineer — because an agent platform is a distributed system with an unusually chaotic client (a language model) and unusually expensive calls.
- An identity engineer — because agent identity is the hardest unsolved problem in the JD, and it is where the bank's risk actually concentrates.
- An SRE — because you carry the pager for a system whose correctness is probabilistic and whose cost is per-request.
- A regulated-industry architect — because every design must produce evidence, and "we log it" is not evidence.
Prerequisite: comfortable Python, comfortable reading a protocol spec, comfortable with HTTP, TLS, and containers at a working level. Everything else — MCP, A2A, OAuth 2.1, SPIFFE, SHACL, PTUs, KV-cache, burn-rate alerting, ISO 20022 — is built from zero inside the track.
Relationship to other tracks
This repo already has two neighbouring tracks. This one is deliberately not a superset of either; where they go deep, we cross-link instead of duplicating.
| Track | Its lens | This track's lens |
|---|---|---|
| Agentic AI Engineer | How an agent works: ReAct/ReWOO loops, tool calling, RAG, durable execution, sandboxing, evals, framework internals (LangGraph, ADK, Bedrock AgentCore, …). | How a bank runs a thousand of them: registries, gateways, identity chains, policy planes, capacity contracts, regulator evidence, run-state ownership. |
| Principal Azure Cloud Engineer | The Azure control plane in general: ARM, Entra, RBAC, landing zones, networking, APIM, Functions. | The AI platform's slice of it: private-endpoint topology for model endpoints, GPU node pools, AI-workload policy-as-code, egress control for a model gateway. |
| Senior AI Engineer | Model internals: transformers, autograd, quantization, PagedAttention. | Serving economics: when a PTU beats pay-as-you-go, what a KV-cache eviction does to your p99, how sovereignty forces self-hosting. |
If you have done the Agentic track, phases 01, 02, 06, and 11 here will feel like a platform re-derivation of familiar mechanisms — that is intentional, and the new material (registries, contracts, tenancy, evidence) is the point.
The five-layer stack this track builds
The JD names five layers. Read them bottom-up as a request's journey:
┌──────────────────────────────────────────────────────────┐
│ USERS & CHANNELS Teams · web · API · batch · IVR │ identity starts here
└────────────────────────────┬─────────────────────────────┘
│ user token
┌────────────────────────────▼─────────────────────────────┐
│ CONTROL PLANE registries · KYA · policy · evals · │ every arrow below is
│ tracing & lineage · quotas │ admitted by this layer
└────────┬────────────────────────────────────┬────────────┘
│ admit / deny / require-approval │ trace
┌────────▼─────────────────────────┐ ┌───────▼───────────────┐
│ AGENT KERNEL │ │ KNOWLEDGE FOUNDATION │
│ lifecycle · plan loop · memory │◄─┤ vectors · BM25 · │
│ state · session affinity │ │ graph · grounding │
└────────┬─────────────────────────┘ └───────────────────────┘
│ proposed action
┌────────▼─────────────────────────────────────────────────┐
│ ACTION GATEWAY contracts · idempotency · circuit │ the enforcement
│ breakers · sagas · audit-grade log │ boundary
└────────┬─────────────────────────────────────────────────┘
│ mediated call, JIT credential, delegated identity
┌────────▼─────────────────────────────────────────────────┐
│ BANK ESTATE core banking · payments · treasury · risk │
└──────────────────────────────────────────────────────────┘
cross-cutting: MODEL LAYER (gateway, routing, capacity) · IDENTITY LAYER (NHI, OAuth 2.1,
token exchange, mTLS) · INFRASTRUCTURE BACKBONE (Terraform, AKS, mesh, networking, CI/CD)
The single most important sentence in the whole track: the model proposes, the platform disposes. Every layer above exists to make sure that a probabilistic component's suggestion becomes a bank action only after identity, policy, contract, quota, and evidence have all said yes.
Phase roadmap
Build status. Complete. All eighteen phases — 00–17 — carry seven teaching documents
each (README, Warmup, Hitchhiker's, Deep Dive, Principal Deep Dive, Core Contributor, Staff Notes)
plus a runnable, test-verified lab: 1,768 tests, all green under LAB_MODULE=solution, with
every lab.py a genuine TODO skeleton and every solution.py runnable as a worked example. The two
supporting chapters are complete as well — five full system designs and four interview-prep
chapters including a 150-question rapid-fire bank and six architecture-review drills.
| # | Phase | You build | JD line it answers |
|---|---|---|---|
| 00 | Platform Mental Model | a five-layer reference-architecture model with composed availability, error-budget split, and a cost-per-action calculator | five-layer stack; two-in-a-box; availability/performance/cost accountability |
| 01 | The Agent Kernel | an agent runtime: lifecycle state machine, plan/act/observe loop, short-term + long-term + episodic memory, scratchpad persistence, session affinity, execution chains | "design the platform's agent kernel…" |
| 02 | MCP — The Tool Plane | a JSON-RPC MCP server + client, a tool registry with semantic versioning, capability advertisement, JSON-Schema enforcement, runtime discovery, and a tool-estate catalog | "engineer the platform's tool layer and MCP server estate" |
| 03 | A2A & ACP — Agent Interop | an A2A miniature: Agent Cards, the task lifecycle state machine, messages/parts/artifacts, streaming updates, push notifications, plus an ACP-style envelope bridge and a hyperscaler-fabric adapter | "MCP … A2A … ACP … interoperable with hyperscaler agent fabrics" |
| 04 | LLM Gateway & Model Abstraction | the gateway: provider adapters, a normalized request/response, routing policy, fallback & retry with budget, rate limiting, token accounting, cost attribution, tenant isolation | "architect and engineer the platform's LLM gateway and model abstraction layer" |
| 05 | Serving, Capacity & Caching | a capacity planner (PTU vs PAYG vs self-hosted), a three-tier cache (exact / prefix / semantic), and a continuous-batching + KV-cache admission simulator | "model serving patterns … PTUs … vLLM/TGI/Triton … KV-cache … batching" |
| 06 | Knowledge Foundation | a hybrid retriever: BM25 + dense + RRF + cross-encoder rerank, per-tenant namespaces, chunking and embedding strategy, freshness and grounding checks | "architect the platform's knowledge foundation … hybrid retrieval … grounding patterns" |
| 07 | Financial Knowledge Graphs | an RDF triple store, an RDFS/OWL entailment subset, a SHACL validator, a SPARQL BGP engine, and FIBO-shaped graph grounding for retrieval | "knowledge graph integration (FIBO, OWL, SHACL, SPARQL)" |
| 08 | Agent & Workload Identity | the identity fabric: NHI lifecycle, OAuth 2.1 + PKCE, OIDC ID tokens, RFC 8693 token exchange with a delegation chain, SPIFFE-style SVIDs, mTLS binding, JIT credentials | "agent identity and workload identity model … OAuth 2.1 … SPIFFE/SPIRE … mTLS" |
| 09 | Control Plane: KYA & Zero Trust | agent + tool registries, a Rego/Cedar-style policy engine, KYA enforcement, behavioral posture checks, continuous authorization, and capability discovery | "design the platform's control plane … KYA enforcement at runtime" |
| 10 | The Action Gateway | contract enforcement, idempotency keys with a replay store, circuit breakers, saga compensation, dual-control approval, and a hash-chained audit log | "design the action gateway as the bank's enforcement boundary for agentic action" |
| 11 | Runtime Guardrails | the guardrail chain: PII/PHI/MNPI detection and masking, prompt-injection defense, output filtering, sensitive-action approval, HITL escalation, OWASP LLM Top 10 coverage matrix | "prompt and output guardrails … human-in-the-loop escalation patterns" |
| 12 | Integration Fabric | an ISO 20022 pain.001 parser/validator, a transactional outbox with exactly-once effects, a schema registry with compatibility rules, and a data-product contract checker | "integration architecture … core banking, payments … Kafka, Event Hubs … data product layer" |
| 13 | Cloud & Infrastructure Backbone | a Terraform-style resource graph with plan/apply/drift, a policy-as-code admission gate, and a private-networking reachability checker for a model-gateway topology | "Terraform … AKS … service mesh … private networking … policy-as-code" |
| 14 | SRE for Non-Deterministic AI | SLI/SLO/error-budget math, multi-window multi-burn-rate alerting, an OTel-shaped span tree at agent+tool granularity, a degradation ladder, and cost governance | "own platform SRE … SLO design … OpenTelemetry … capacity planning … cost governance" |
| 15 | Governance, Model Risk & Evidence | a model/agent inventory with risk tiering, a lineage & evidence graph, a data-residency policy checker, third-party model governance, and an auditor evidence-pack generator | "harden the platform to meet CBUAE, internal model risk, and Group governance requirements" |
| 16 | Two-in-a-Box & Engineering Leadership | an operational-readiness-review scorer, an architecture-decision-record engine, an error-budget policy machine, and the regulator/audit conversation playbooks | "operate in genuine two-in-a-box … drive engineering excellence … represent the platform" |
| 17 | Capstone — The Bank-Grade Platform | one AIPlatform.handle() composing all five layers end-to-end for a wholesale-banking payment-investigation agent, with full evidence output | the whole JD |
Supporting chapters:
| Chapter | Contains |
|---|---|
| Interview Prep | the battle plan (this JD line by line), a 150-question rapid-fire bank, six architecture-review drills with the red flags planted, and the behavioural round |
| System Design Walkthroughs | five full designs at the altitude this role is interviewed and reviewed at — the platform, the gateway, identity across three hops, authorized retrieval, the evidence platform |
Suggested schedule
Phases are sized at roughly one per 1.5 weeks at 10–12 h/week — about 27 weeks (≈6 months) for the core eighteen, plus the capstone. Two faster paths:
- Interview sprint (6 weeks) — 00, 04, 08, 09, 10, 14, then Interview Prep and two system designs. This is the minimum set that lets you hold a credible architecture conversation for this JD.
- Run-state sprint (4 weeks) — 13, 14, 15, 16. For someone who already knows agents but has never carried a regulated pager.
| Weeks | Phases | Theme |
|---|---|---|
| 1–3 | 00, 01 | the platform model and the runtime it hosts |
| 4–7 | 02, 03 | the protocol stack: tools, then agents |
| 8–11 | 04, 05 | the model layer: gateway, then economics |
| 12–15 | 06, 07 | knowledge: vectors, then ontology |
| 16–20 | 08, 09, 10 | the trust spine: identity → policy → enforcement |
| 21–23 | 11, 12 | safety envelope and the bank estate |
| 24–27 | 13, 14 | backbone and run-state |
| 28–30 | 15, 16 | evidence and the operating model |
| 31–33 | 17 | capstone |
How to work a phase
- Read
README.md— the why, the concept map, the lab spec, and the deliverables. - Read
WARMUP.mdend-to-end before touching the lab. It is the zero-to-principal primer: every term from first principles, the mechanism under the hood, the production significance, and the misconception that bites people. - Implement the lab's
lab.pyTODOs. Runpytest test_lab.py -vuntil green. - Compare against
solution.py, then runpython solution.pyfor the worked trace. - Read
DEEP-DIVE.md(internals & complexity),PRINCIPAL-DEEP-DIVE.md(tradeoffs, scaling, blast radius),CORE-CONTRIBUTOR.md(how the real system does it and what we simplified), andSTAFF-NOTES.md(judgment, review red flags, interview signal). - Skim
HITCHHIKERS-GUIDE.mdthe day before an interview — it is the compressed recall pass.
The document set
| Document | Voice | What it gives you |
|---|---|---|
README.md | the syllabus | why the phase exists, concept map, lab table, integrated scenario, deliverables, key takeaways |
WARMUP.md | the professor | zero-to-principal primer with TOC, first-principles derivations, worked math, mechanism diagrams, lab walkthrough, interview Q&A, references |
HITCHHIKERS-GUIDE.md | the senior who's been there | 30-second mental model, numbers to memorize, war stories, vocabulary, beginner mistakes |
DEEP-DIVE.md | the core contributor | data structures, algorithms, invariants, complexity, a worked step-by-step trace |
PRINCIPAL-DEEP-DIVE.md | the principal engineer | tradeoffs, scaling envelope, failure modes and blast radius, "looks wrong but intentional" |
CORE-CONTRIBUTOR.md | the maintainer | how the real system implements this, its sharp edges, and what our miniature simplifies |
STAFF-NOTES.md | the staff engineer | build-vs-buy, decision framework, code-review red flags, production war stories, exact interview signal |
Standards for the runnable material are in LAB-STANDARD.md. Vocabulary is in GLOSSARY.md; the numbers and one-liners worth memorizing are in CHEATSHEET.md.
Running the labs
cd ai-platform-architect/phase-04-llm-gateway-model-abstraction/lab-01-llm-gateway
pip install -r requirements.txt # pytest only — labs are pure stdlib
pytest test_lab.py -v # against your lab.py (red until implemented)
LAB_MODULE=solution pytest test_lab.py -v # against the reference (must be green)
python solution.py # the worked example, printed
Run every lab in the track:
cd ai-platform-architect
for d in phase-*/lab-*; do (cd "$d" && LAB_MODULE=solution python -m pytest -q) || echo "FAIL $d"; done
What "done" looks like
You are ready for this role's loop when you can, from a blank whiteboard:
- Draw the five layers, name what each one owns, and say which layer denies a bad action and why the others cannot.
- Explain agent identity end to end: how a user token becomes a delegated, short-lived, audience-scoped credential at hop three of a multi-agent flow, and what breaks if you use a service account instead.
- Defend a model-routing and capacity decision with arithmetic — PTU break-even, cache hit-rate effect on cost, p99 impact of a fallback.
- Specify the evidence a CBUAE examiner would ask for after an agent moved money, and point at the component that emits each artifact.
- State an SLO for a non-deterministic system — including what you don't put in the SLI — and run the burn-rate arithmetic that pages you.
- Run a design review that catches the four review red flags in Phase 16.
References
Protocols & standards
- Model Context Protocol specification — dated revisions; read the current one and one prior to see the evolution.
- Agent2Agent (A2A) protocol — Agent Cards, tasks, artifacts.
- Agent Communication Protocol (ACP).
- RFC 6749 / RFC 9700 (OAuth 2.0 + security BCP), OAuth 2.1 draft, RFC 7636 (PKCE), RFC 8693 (Token Exchange), RFC 8705 (mTLS client auth), RFC 9068 (JWT access tokens), RFC 7519 (JWT), OpenID Connect Core 1.0.
- SPIFFE / SPIRE — workload identity, SVIDs, trust domains.
- W3C: RDF 1.1, OWL 2, SHACL, SPARQL 1.1.
- ISO 20022 message catalogue.
Books
- Newman, Building Microservices, 2nd ed. — contracts, sagas, boundaries.
- Kleppmann, Designing Data-Intensive Applications — the distributed-systems spine.
- Beyer et al., Site Reliability Engineering and The Site Reliability Workbook — SLOs, error budgets, on-call.
- Rosenthal & Jones, Chaos Engineering — how to earn confidence in a system you cannot prove.
- Allemang & Hendler, Semantic Web for the Working Ontologist — RDF/OWL/SHACL, practically.
- Ford, Richards et al., Software Architecture: The Hard Parts — the tradeoff vocabulary this role is assessed on.
Regulatory & risk
- Federal Reserve SR 11-7 — Guidance on Model Risk Management (the ancestor of most bank internal AI-model standards).
- OWASP Top 10 for LLM Applications & Gen AI.
- NIST AI Risk Management Framework (AI RMF 1.0) and the Generative AI Profile.
- CBUAE rulebook and outsourcing/cloud guidance.
- EU AI Act — useful as the strictest reference regime even where it does not apply.