« Track Overview · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes
Phase 13 — Secure Multi-Tenant Platform: AuthN/Z, Isolation, Secrets & Quotas
Answers these JD lines: Citi's "multi-tenant agentic ecosystems" and "secure distributed systems" (Lead/Staff Agentic AI); OpenAI's "secure REST APIs, OAuth, authorization, encryption" and "agent infrastructure/security"; and the tenant-isolation, secrets, and quota concerns every enterprise platform role in jd.md shares (Docker, Wolters Kluwer, RBC). This is the phase where "the model proposes, the application executes" (Phase 00) meets "one platform, many customers, zero leakage."
Why this phase exists
An enterprise agent platform is never single-tenant. One deployment serves dozens of business units or thousands of customers on shared compute, one vector database, one cache, one process. That sharing is the entire economic argument for SaaS — pool the infrastructure, amortize the cost — and it is simultaneously the entire security problem: tenant A's request and tenant B's data live in the same process, so the only thing keeping them apart is your code. Get the boundary right and you have a platform; get it wrong once and you have a breach, a compliance failure, and a headline.
Five ideas do the load-bearing work, and this phase builds every one:
- AuthN answers who, AuthZ answers may they. Authentication verifies identity (a signed token); authorization decides permission (RBAC, default-deny). They are different questions with different failure modes, and conflating them is a classic bug.
- The tenant comes from the token, never the request. A verified, signed token carries the
tenant id. If you read
tenant_idfrom a request body or query param, you have handed the caller the keys — that is privilege escalation. Derive the tenant from trusted auth and thread it through every query. - Isolation is enforced in the data path, not hoped for. Row-level security force-scopes every query to the caller's tenant; a shared vector index is namespaced; a cache key includes the tenant. A cross-tenant read must be impossible, not merely unlikely.
- The AI-specific leak channels are new and the most dangerous. A shared vector index is the #1 way multi-tenant AI platforms leak — RAG retrieves the nearest chunk regardless of owner, so B's document surfaces in A's answer. Prompt caches and agent memory leak the same way. None is fixable by prompting.
- Secrets, quotas, and audit are platform hygiene. Per-tenant secret custody (never logged), per-tenant token-bucket quotas (noisy-neighbor isolation), and an append-only audit trail (metadata, not content) are the difference between a demo and a SOC 2 platform.
Concept map
- AuthN: OAuth2/OIDC-shaped bearer token → JWT (
header.payload.signature) → HMAC signing withhmac/hashlib→ expiry via an injected clock → a trustedPrincipal. - AuthZ: roles → permissions (RBAC); ABAC as the generalization; default-deny; cross-tenant denied even for a tenant-admin.
- Isolation models: silo (dedicated infra per tenant) · pool (shared, isolated in the data path) · bridge (hybrid) — the AWS SaaS lens.
- Row-level security: thread
tenant_idfrom the verified token through every query; a crafted filter can never widen scope. - AI leak channels: shared vector index (#1) → per-tenant namespaces; prompt cache → tenant-keyed; agent memory → per-tenant; co-mingled fine-tunes → per-tenant adapters.
- Secrets: per-tenant custody, vault/BYOK, never in a log.
- Quotas: per-tenant token bucket, fair-share, never negative.
- Audit: append-only, who/what/tenant, correlation IDs, secrets redacted.
- Compliance: encryption in transit/at rest, SOC 2, GDPR data residency.
The lab
| Lab | You build | Proves you understand |
|---|---|---|
| 01 — Multi-Tenant Agent Gateway | a gateway that verifies HMAC-signed tokens, enforces RBAC default-deny, force-scopes every query/vector/secret to the tenant, meters a per-tenant token bucket, and audits with secrets redacted | that tenant isolation is an enforced property of the data path, derived from the trusted token — not a prompt, not a hope |
Integrated scenario (how this shows up at work)
Your company runs one agent platform for the whole enterprise: the Payments team, the HR team, and the Legal team all use the same "ask-your-documents" agent, backed by one vector database and one LLM gateway. A Legal user asks a question and the RAG step retrieves the three most similar chunks — and one of them is a Payments incident report, because the index is shared and similarity does not respect ownership. The model paraphrases it into Legal's answer. That is a cross-tenant data leak, and no prompt would have stopped it. The fix is architectural: each team is a tenant with its own vector namespace, its own cache partition, its own secrets, and its own quota; the gateway derives the tenant from the signed token and force-scopes every retrieval. This lab builds exactly that gateway — the one you point to in the design review when someone asks "how do we know Legal can't read Payments' data?"
Deliverables checklist
-
Lab 01 green under
LAB_MODULE=solution pytestand your ownlab.py. - You can explain AuthN vs AuthZ and why the tenant id must come from the token.
- You can name the three isolation models (silo/pool/bridge) and place a design in one.
- You can name the four AI-specific leak channels and the isolation control for each.
- You can whiteboard a token-bucket quota and argue noisy-neighbor isolation.
Key takeaways
- Trust the token, never the body. The tenant id is derived from verified auth and threaded
through every query; a request-supplied
tenant_idis a privilege-escalation bug. - A shared vector index WILL leak unless it is namespaced per tenant — it is the #1 multi-tenant AI leak channel, and prompt caches and agent memory are the runners-up.
- Default-deny or fail open. RBAC without a default-deny posture fails open the moment you forget a case; least privilege and allow-lists are the only safe defaults.
- Isolation is defense in depth. Authz rejects the cross-tenant request and the data layer is scoped by the trusted tenant regardless — two independent gates, so one bug is not a breach.
- Secrets, quotas, and audit are not optional. Per-tenant secret custody, fair-share quotas, and a redacted append-only trail are what turn a working agent into a compliant platform.