« Phase 13 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Staff Notes
Phase 13 — Core Contributor Notes: How the Real Systems Enforce This
This is the maintainer's-eye view of the real machinery our miniature imitates — JWT libraries, Postgres RLS, Pinecone/Weaviate namespaces, Vault, and the OWASP failure catalog — not the lab. The interesting engineering is in the seams: the places where the real implementations made a non-obvious choice, changed it, or left a sharp edge. Where I describe a pattern rather than a verified internal, I say so; I do not invent version numbers or URLs.
JWT verification: the vulnerabilities that shaped the API
Our verify_token is a faithful sketch of what a hardened JWT library does, and the reason those
libraries are shaped the way they are is a history of breaches:
alg: none. Early JWT libraries honored the token's own header. A header claimingalg: nonemeant "no signature required," so an attacker stripped the signature, setalg: none, and the library accepted the forged token. The fix that every serious library adopted: the caller pins the accepted algorithm(s); the token's header may never talk the verifier out of verifying. This is why modern APIs make you passalgorithms=["HS256"]explicitly and reject anything else.- Algorithm confusion (HS256/RS256). If a service verifies with a key that is sometimes RSA and
sometimes HMAC, an attacker takes the public RSA key (which is public), signs an
HS256token with it as the HMAC secret, and the naive verifier — using the same key bytes — validates it. The lesson baked into good libraries: the algorithm family and the key type are bound together, and you never let the token pick. - Constant-time comparison.
hmac.compare_digestexists because==on the signature leaks match progress through timing, and timing oracles are exploitable over a network. Our lab uses it for exactly this reason; a real library that used==would be a CVE.
Our miniature simplifies HMAC/HS256 only; production also runs RS256/ES256 (asymmetric, so verifiers
hold only the public key and cannot mint tokens) with JWKS key rotation — the kid header selects
a key from a published key set, and rotating a compromised key is a JWKS update, not a redeploy. We
skip all of that to keep the lab offline and deterministic, but the contract — verify before you
trust, pin the algorithm — is identical.
Postgres Row-Level Security: what "the database appends the predicate" really costs
Our TenantStore.query models CREATE POLICY ... USING (tenant_id = current_setting(...)). The real
feature has seams a committer learns the hard way:
- The
BYPASSRLSand table-owner escape. RLS does not apply to a table's owner or to superusers by default — the owner sees all rows. A common production mistake is running the app as the table owner, silently disabling the isolation you thought you had. Real deployments run the app as a non-owner role and often setFORCE ROW LEVEL SECURITYso even the owner is subject to policy. - The session variable is per-connection, and connection pools reuse connections. RLS reads
current_setting('app.current_tenant'), which is set per session. With a pool (PgBouncer), a connection carrying tenant A's setting can be handed to a request for tenant B if you don't reset it every checkout. The discipline isSET LOCALinside the transaction so the value dies at commit — a sharp edge our in-memory model doesn't have because it passes the principal explicitly. - Policies are
USING(read) vsWITH CHECK(write). AUSINGpolicy filters what you can see; aWITH CHECKpolicy constrains what you can insert/update, which is what stops a caller from writing a row stamped with someone else's tenant. Ourinsertstampsprincipal.tenantunconditionally, which is theWITH CHECKdiscipline collapsed into the write path.
Vector databases: namespace is a first-class primitive, and post-filtering is the trap
The real systems this phase's #1 leak channel mirrors:
- Pinecone namespaces partition an index so a query runs within one namespace — the tenant
filter is applied before the ANN search, which is exactly the property that makes it a security
boundary rather than a post-hoc filter. Our
VectorIndexkeying on tenant is this pattern. - Weaviate multi-tenancy makes tenant a first-class concept (a
tenantper object/class) rather than a metadata field, and can even keep inactive tenants' shards cold on disk — a scale property our in-memory dict doesn't model. - The metadata-filter trap. Many teams reach for a
WHERE tenant_id = ?metadata filter on a shared index instead of a namespace. Whether that filters before or after the ANN traversal is an implementation detail of the engine — and if it is a post-filter, the search still visits other tenants' vectors (a timing/existence signal at minimum, and a hard leak if the filter is ever misapplied). The maintainer's rule: a namespace is a partition; a metadata filter is a predicate, and for a security boundary you want the partition. Our lab deliberately models the partition. - The fine-tune channel has no query-time fix. Real platforms that must isolate at the weight level use per-tenant LoRA adapters loaded per request, or per-tenant models — there is no "namespace" for parameters, which is why this fourth channel is categorically harder than the other three.
Secrets: why Vault/KMS looks the way it does
Our SecretStore.get_secret((principal.tenant, name)) is the shape of a real secrets manager, and
the real ones add the properties we stub:
- Short-lived, dynamic secrets. Vault's signature feature is generating a database credential on demand with a lease, so a leaked secret expires on its own. Static secrets in a table are the thing you migrate away from.
- Not-found ≠ denied, on purpose. A cross-tenant secret read "does not exist" for that caller
rather than "is denied" — not leaking existence is itself a control, and real systems lean on it so
an attacker probing for
(other_tenant, name)learns nothing. - BYOK / customer-managed keys invert trust: the tenant holds the root key (in their KMS), so the platform can only decrypt with the tenant's cooperation and a platform-side compromise doesn't expose plaintext. Regulated customers ask for this by name. Our lab has no crypto-at-rest, but the custody boundary — secret scoped to tenant, fetched by trusted tenant — is the same.
The OWASP catalog is the design spec, not a checklist
Two entries are effectively the requirements doc for this phase. API1:2023 Broken Object-Level Authorization (BOLA/IDOR) is the trusted-request-body bug — the most common serious API vuln in the wild — and the reason the whole phase insists the tenant comes from the token. LLM06 Sensitive Information Disclosure in the OWASP LLM Top 10 is the vector-index / cache / memory leak family, the AI-specific extension classic training misses. A committer treats these as the enumeration of failure modes the code must structurally prevent, not a post-hoc audit.
What our miniature deliberately simplifies
- HS256 only, no asymmetric keys or JWKS rotation — the lab is offline and deterministic; real
auth is RS256/ES256 with rotating public keys and a
kidheader. - In-memory dicts instead of Postgres RLS + a connection pool — so we never confront the pool-reuse and table-owner escapes, which are the real system's sharpest edges.
- A namespace dict instead of Pinecone/Weaviate — same partition-before-search property, none of the sharding, cold-shard, or replication machinery.
- A plaintext secret map instead of Vault/KMS/BYOK — same custody boundary, no leases, no encryption at rest.
- An injected integer clock instead of wall time — deterministic expiry and refill; a real system reasons about clock skew and replay windows.
Know the seams — the alg pin, the RLS session-variable-per-pooled-connection trap, namespace vs
post-filter, not-found-as-a-control — and the real systems' docs read as confirmation of the shapes
this lab already made you build.