Tenant Isolation

Phase 14 · Document 04 · Security, Privacy and Governance Prev: 03 — Secrets and API Keys · Up: Phase 14 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

If you sell your LLM app to more than one customer, you are running a multi-tenant system — and the single worst thing that can happen is showing one customer another customer's data. Cross-tenant leakage is an instant trust-destroying, contract-breaking, often legally-reportable incident. LLM apps make this harder than traditional SaaS because data leaks through AI-specific channels that ordinary tenant-isolation reviews miss: a shared vector index that retrieves Tenant B's documents for Tenant A's query (Phase 9), a prompt cache / KV-cache shared across tenants (Phase 7.05), agent memory that carries one tenant's context into another's session (Phase 10.04), a fine-tuned model that memorized one tenant's data and emits it to others (Phase 13.06), or a mis-scoped BYOK key (03). This doc is how to keep tenants' data, context, keys, and models strictly separated.

2. Core Concept

Plain-English primer: every request must be locked to one tenant

A tenant is an isolated customer (a company, a workspace, sometimes an individual). Tenant isolation means: for any request, the system can only ever see, retrieve, compute on, and return data belonging to that one tenant. The core mechanism is a tenant identifier that is established from a trusted source (the authenticated session / token — never from user-supplied text or the model) and then threaded through every layer so each layer filters to that tenant.

auth → trusted tenant_id  →  must scope EVERY layer:
   DB query (WHERE tenant_id = ?) · VECTOR search (filter tenant_id) · CACHE key (include tenant_id) ·
   agent MEMORY (per-tenant store) · provider KEY (per-tenant, BYOK) · LOGS (tagged) · MODEL (shared base, isolated data)
A single layer that forgets the filter = cross-tenant leak.

The golden rule: isolation is only as strong as the weakest layer. Perfect DB row-level security means nothing if the vector index is shared and unfiltered.

The isolation spectrum (how hard you separate)

1. Silo (hard isolation) — separate infrastructure per tenant: separate databases, separate vector indexes/namespaces, sometimes separate deployments. Strongest isolation; highest cost/ops; common for large/regulated enterprise customers.
2. Pool (shared infrastructure, logical isolation) — shared DB/index/service, isolation enforced by a tenant_id filter on every operation (row-level security, per-tenant metadata filters). Cheapest/most scalable; isolation depends entirely on never missing a filter.
3. Bridge / hybrid — pool for small tenants, silo for big/regulated ones; a common real-world pattern.

Most SaaS starts pooled (cheaper) and offers siloed tiers for enterprise. The risk with pooled is human: one forgotten WHERE tenant_id or unfiltered vector query leaks everything.

The AI-specific leak channels (the part traditional reviews miss)

• Shared vector store / RAG index — the #1 LLM tenant-leak. If all tenants' documents share one index without a tenant filter on retrieval, Tenant A's query can retrieve and surface Tenant B's documents. Fix: per-tenant namespaces/collections, or a mandatory metadata filter (tenant_id) on every search, enforced server-side (not by the model) (Phase 9.04). Combine with document-level ACLs so a user only retrieves what they're allowed to see within a tenant.
• Prompt / prefix / KV cache — caching across tenants can serve one tenant's cached completion or cached prefix to another. Include tenant_id (and user/ACL scope) in the cache key; never share cache entries across tenants (Phase 7.05).
• Agent memory / conversation state — long-term or session memory must be per-tenant (and per-user); a shared memory store leaks context across sessions (Phase 10.04).
• Fine-tuned models — a model fine-tuned on Tenant A's data memorizes it and may emit it to others. Don't co-mingle tenants in one fine-tune unless that's explicitly acceptable; for per-tenant customization prefer per-tenant adapters (multi-LoRA) over one shared fine-tune (Phase 13.07/Phase 13.06).
• Provider keys (BYOK) — each tenant's key must be isolated and scoped to that tenant's requests (03); never let one tenant's traffic bill or route through another's key.
• Logs & observability — tagged with tenant_id, access-controlled, and never co-mingled in a way that exposes one tenant's prompts to another's support view (06).

Defense in depth for isolation

• Trusted tenant context — derive tenant_id from the authenticated token, never from request body, prompt, or model output (the model can be injected into claiming a different tenant, 01).
• Enforce at the data layer, not the app layer alone — row-level security (RLS) in Postgres, per-tenant collections in the vector DB — so a forgotten app-side filter still fails safe.
• Default deny — queries without a tenant scope return nothing, not everything.
• The model never decides access — retrieval filtering and authorization are deterministic code, not something you ask the model to respect (model proposes / app executes, Phase 10.05).
• Test for leakage — automated cross-tenant tests: as Tenant A, attempt to read/retrieve Tenant B's data through every channel.

Noisy-neighbor (the availability side of multi-tenancy)

Isolation isn't only about data — it's also performance/cost isolation: one tenant shouldn't be able to exhaust shared capacity (rate limits, token budgets per tenant) or run up shared cost. Enforce per-tenant quotas/rate limits so one tenant can't degrade or bankrupt the others (Phase 7.09/Phase 8.06).

3. Mental Model

   TENANT = isolated customer. ISOLATION = every request only sees/returns ITS tenant's data.
   ★ tenant_id from TRUSTED auth (NEVER from user text / model [01]) → thread through EVERY layer:
     DB (RLS WHERE tenant_id) · VECTOR search (namespace/metadata filter) · CACHE key (incl tenant_id) ·
     agent MEMORY (per-tenant) · provider KEY (per-tenant/BYOK [03]) · LOGS (tagged [06]) · MODEL (shared base, isolated data)
   isolation = only as strong as the WEAKEST layer. One missing filter = leak.

   SPECTRUM: SILO (separate infra/DB/index — strongest, costly) · POOL (shared + tenant_id filter — cheap, filter-dependent) · BRIDGE (hybrid)

   ★ AI-SPECIFIC LEAK CHANNELS (traditional reviews miss):
     - SHARED VECTOR INDEX → A's query retrieves B's docs → per-tenant namespace + mandatory metadata filter + doc ACLs [9.04]
     - PROMPT/PREFIX/KV CACHE → include tenant_id (+ACL) in cache key; never share across tenants [7.05]
     - AGENT MEMORY → per-tenant/per-user store [10.04]
     - FINE-TUNED MODEL → memorizes a tenant's data → don't co-mingle; prefer per-tenant adapters (multi-LoRA) [13.07/13.06]
     - BYOK KEYS → isolate/scope per tenant [03]
     - LOGS → tagged + access-controlled [06]

   DEFENSE IN DEPTH: trusted tenant ctx · ENFORCE AT DATA LAYER (RLS) not app-only · DEFAULT DENY (no scope→nothing) ·
                     model never decides access [10.05] · automated CROSS-TENANT leak tests
   NOISY NEIGHBOR: per-tenant quotas/rate limits/budgets so one tenant can't degrade/bankrupt others [7.09/8.06]

Mnemonic: derive the tenant from trusted auth and thread it through every layer — DB, vector index, cache, memory, keys, logs, model — because isolation is only as strong as the weakest layer; the AI-specific killers are the shared vector index, the shared cache, and co-mingled fine-tunes.

4. Hitchhiker's Guide

What to look for first: is the vector index/cache/memory shared across tenants, and is tenant_id enforced on every retrieval/lookup? That's where LLM apps leak. Then: where does tenant_id come from — trusted auth (good) or request/model (dangerous)?

What to ignore at first: full silo infrastructure for every tenant. Start pooled with rigorously enforced filters + RLS + cross-tenant tests; offer silo as an enterprise tier when needed.

What misleads beginners:

• Only isolating the database. The vector index, cache, memory, and fine-tunes leak too — isolation is only as strong as the weakest layer.
• Trusting the model/request for tenant identity. Derive tenant_id from the authenticated token; injection can make the model claim another tenant (01).
• App-layer filters only. A forgotten WHERE leaks everything — enforce at the data layer (RLS) so it fails safe.
• Co-mingling tenants in one fine-tune. The model memorizes and can emit cross-tenant — prefer per-tenant adapters (Phase 13.07).
• Sharing the prompt cache. Cache keys must include tenant_id (+ACL) or you serve one tenant's data to another (Phase 7.05).
• Ignoring noisy-neighbor. No per-tenant quotas → one tenant degrades/bankrupts the rest.

How experts reason: they establish tenant_id from trusted auth, thread it through every layer, enforce isolation at the data layer (RLS, per-tenant namespaces) with default-deny, keep the model out of access decisions, isolate AI-specific channels (vector index, cache, memory, fine-tunes, BYOK keys), add per-tenant quotas for noisy-neighbor, and run automated cross-tenant leak tests as a release gate. They silo big/regulated tenants.

What matters in production: zero cross-tenant data exposure through any channel; tenant identity from trusted auth; default-deny enforced at the data layer; per-tenant quotas; and continuous leak testing.

How to debug/verify: as Tenant A, try to retrieve/read Tenant B's data through each channel (DB, vector search, cache, memory, fine-tune output, logs). Any leak = the weakest layer. Verify tenant_id is never sourced from user input/model; verify a missing scope returns nothing.

Questions to ask: where does tenant_id come from? is it enforced on every layer (esp. vector index, cache, memory)? is it data-layer (RLS) or app-only? does a missing scope default-deny? are fine-tunes/keys per-tenant? are there per-tenant quotas? do cross-tenant leak tests run each release?

What silently leaks tenants: shared unfiltered vector index, shared prompt cache, shared agent memory, co-mingled fine-tunes, tenant_id from untrusted input, app-only filters, and no leak testing.

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• Cross-tenant data leakage is the worst multi-tenant failure — instant trust/contract/legal damage; LLM apps add AI-specific leak channels.
• Derive tenant_id from trusted auth, never from user text or the model (01), and thread it through every layer — isolation is only as strong as the weakest layer.
• Isolation spectrum: silo (separate infra, strongest), pool (shared + tenant filter, cheapest), bridge (hybrid) — most SaaS pools and offers silo tiers.
• The #1 LLM leak is a shared vector index — use per-tenant namespaces + a mandatory server-side metadata filter + doc ACLs (Phase 9.04).
• Prompt/prefix/KV cache must key on tenant_id — never share cache entries across tenants (Phase 7.05).
• Agent memory must be per-tenant/per-user (Phase 10.04); fine-tunes memorize data — don't co-mingle tenants; prefer per-tenant adapters (multi-LoRA) (Phase 13.07).
• Enforce at the data layer (RLS/namespaces) with default-deny; the model never decides access (Phase 10.05).
• Per-tenant quotas/rate limits/budgets prevent noisy-neighbor degradation and runaway cost (Phase 7.09/Phase 8.06); run cross-tenant leak tests every release.

9. Observations from Real Systems

• The shared-vector-index leak is the canonical LLM multi-tenancy bug — teams that bolt RAG onto a single shared index discover Tenant A retrieving Tenant B's docs; per-tenant namespaces/filters are the standard fix (Phase 9.04).
• Prompt-cache cross-tenant leaks have happened where cache keys omitted tenant scope — a subtle, high-severity bug (Phase 7.05).
• Per-tenant adapters (multi-LoRA) are how platforms offer per-customer fine-tuning without co-mingling data in one model (Phase 13.07).
• RLS at the database layer is favored because it makes a forgotten app-side filter fail safe rather than leak.
• Enterprise buyers demand silo isolation (separate infra/region) for regulated data — a common reason for a "dedicated/enterprise" tier (07).

10. Common Misconceptions

11. Engineering Decision Framework

TENANT ISOLATION FOR AN LLM APP:
 1. IDENTITY: derive tenant_id from TRUSTED auth/token — never from request body, prompt, or model output [01].
 2. THREAD it through EVERY layer: DB, vector index, cache key, agent memory, provider key, logs, model.
 3. MODEL: choose silo (separate infra — regulated/large tenants) vs pool (shared + filter — default) vs bridge (hybrid).
 4. ENFORCE at the DATA layer: RLS (Postgres), per-tenant namespaces (vector DB), DEFAULT-DENY (no scope → nothing).
 5. AI CHANNELS: per-tenant vector namespace + mandatory retrieval filter + doc ACLs [9.04]; tenant_id in cache key [7.05];
    per-tenant memory [10.04]; per-tenant adapters not co-mingled fine-tunes [13.07]; per-tenant BYOK keys [03].
 6. ACCESS DECISIONS are deterministic code, never the model [10.05].
 7. NOISY NEIGHBOR: per-tenant rate limits / token budgets / cost caps [7.09/8.06].
 8. TEST: automated cross-tenant leak tests through every channel, every release.

12. Hands-On Lab

Goal

Build a multi-tenant RAG slice and prove no cross-tenant leakage through the database, the vector index, and the cache — then break it and watch it leak.

Prerequisites

• A small RAG app with ≥2 tenants, a database, a vector store (with namespaces/metadata), and a prompt cache; auth that yields a tenant_id.

Steps

1. Establish trusted identity: derive tenant_id from the auth token; confirm it's not taken from the request body or model.
2. Scope every layer: DB query filters by tenant_id (ideally RLS); vector search uses a per-tenant namespace or mandatory tenant_id metadata filter; the cache key includes tenant_id (Phase 7.05).
3. Leak test (negative): as Tenant A, attempt to retrieve/read Tenant B's documents via each channel; confirm nothing comes back (default-deny).
4. Break it (demonstration): remove the vector filter (or drop tenant_id from the cache key) and show Tenant A now retrieves Tenant B's data — the weakest-layer lesson.
5. Restore + add doc ACLs: re-enforce filters and add within-tenant document ACLs so a user only retrieves permitted docs.
6. Noisy-neighbor: add a per-tenant rate limit / token budget and show one tenant can't exhaust the shared capacity (Phase 7.09).

Expected output

A multi-tenant RAG slice with tenant scoping on DB + vector index + cache, passing cross-tenant leak tests, plus a demonstrated leak when a single filter is removed — proving isolation is only as strong as the weakest layer.

Debugging tips

• If Tenant A ever sees Tenant B's data, find the unfiltered layer (usually the vector index or cache key).
• If tenant_id can be set from the request/prompt, that's an injection-exploitable hole (01).

Extension task

Add RLS in Postgres and confirm a deliberately-forgotten app-side filter still fails safe; add per-tenant adapters (multi-LoRA) instead of a co-mingled fine-tune (Phase 13.07).

Production extension

Wire cross-tenant leak tests into CI as a release gate; enforce per-tenant quotas at the gateway (Phase 8.06); tag all logs with tenant_id (06).

What to measure

Cross-tenant leak attempts (target 0), layers scoped (DB/vector/cache/memory/keys/logs), default-deny behavior, per-tenant quota enforcement.

Deliverables

• A multi-tenant slice with tenant scoping on DB + vector index + cache.
• A passing cross-tenant leak test suite + a demonstrated leak when a filter is removed.
• Per-tenant quotas + (stretch) RLS and per-tenant adapters.

13. Verification Questions

Basic

1. What is tenant isolation and why is cross-tenant leakage so severe?
2. Where must tenant_id come from, and where must it never come from?
3. Why is "isolation is only as strong as the weakest layer" the key principle?

Applied 4. Name the AI-specific leak channels beyond the database. 5. Why prefer per-tenant adapters over one co-mingled fine-tune?

Debugging 6. Tenant A retrieves Tenant B's documents. Where do you look first? 7. Why enforce isolation with RLS rather than app-layer filters alone?

System design 8. Design tenant isolation for a multi-tenant RAG agent (DB, vector index, cache, memory, keys, model, quotas).

Startup / product 9. When do you offer siloed isolation, and how do you message it to enterprise buyers?

14. Takeaways

1. Cross-tenant leakage is the worst multi-tenant failure — and LLM apps add AI-specific channels traditional reviews miss.
2. Derive tenant_id from trusted auth and thread it through every layer — DB, vector index, cache, memory, keys, logs, model; isolation = weakest layer.
3. The #1 LLM leak is a shared vector index — per-tenant namespaces + mandatory filter + ACLs; cache keys and memory must be per-tenant too (Phase 9.04/Phase 7.05).
4. Enforce at the data layer (RLS) with default-deny; the model never decides access (Phase 10.05); don't co-mingle tenants in fine-tunes — use per-tenant adapters (Phase 13.07).
5. Add per-tenant quotas (noisy-neighbor) and run cross-tenant leak tests every release (Phase 7.09).

15. Artifact Checklist

• tenant_id from trusted auth, threaded through DB, vector index, cache, memory, keys, logs.
• Data-layer enforcement (RLS / per-tenant namespaces) with default-deny.
• AI-channel isolation: per-tenant vector namespace + filter + ACLs; tenant in cache key; per-tenant memory/adapters/keys.
• A cross-tenant leak test suite as a release gate.
• Per-tenant quotas/budgets (noisy-neighbor).

Up: Phase 14 Index · Next: 05 — Policy and Guardrails