« Phase 17 README · Warmup · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes
Phase 17 — Hitchhiker's Guide
30-second mental model
A platform is layers composed in order with explicit trust boundaries. One request lifecycle:
authenticate+tenant → authorize+quota → input guardrail → context+retrieval (tenant-scoped) →
agent (metered, traced) → output guardrail → eval gate → audit. Turn any layer off and a specific
attack gets through. The five truths (trust boundary, 0.95^n, injection-is-architectural,
cost-is-engineered, evaluate-or-don't-ship) run through all of it.
The order to tattoo on your arm
authN+tenant → authZ+quota → INPUT guard → context+retrieve(scoped)
→ agent(meter+trace) → OUTPUT guard → eval gate → audit
| Rule | Why |
|---|---|
| authN before authZ | can't authorize an unknown identity |
| cheap checks (authz/quota) before expensive work | reject before spending tokens |
| input guard before the agent | catch injection before the model reads it |
| retrieve tenant-scoped, always | a shared index leaks across tenants |
| output guard + eval before returning | last line before a leak/hallucination ships |
| audit every path | incident response + compliance need the blocks too |
Framework one-liners
- The gateway (authn/authz/quota) = an API gateway / Phase 13 + 14.
- The guardrail service = Phase 10 wrapping the model in + out.
- The runtime = Phase 1/2 (loop + tools), durable via Phase 8, sandboxed via Phase 9, tools via MCP (Phase 3).
- The knowledge layer = Phase 5/6 retrieval on a real vector/graph DB.
- The quality layer = Phase 11 eval gate + CI regression.
- The observability layer = Phase 14 traces + cost meter + audit.
War stories
- The cross-tenant leak. Retrieval wasn't scoped by tenant; one customer's agent surfaced another's docs. Isolation must be threaded through the query, not checked after.
- The order bug. An input guard ran after context assembly, so the injection was already in the prompt. Order is a security property.
- The demo that had no audit. Great until the first incident, when nobody could answer "what did the agent do, for whom?" Audit every path.
Vocabulary
Request lifecycle · trust boundary (nested) · fail closed · least blast radius ·
defense in depth · cross-cutting concern · $/resolved-task · tenant-scoped ·
eval gate · audit substrate.
Beginner mistakes
- Treating the platform as a pile of features instead of ordered layers.
- Trusting
tenant_idfrom the request instead of the verified token. - Running the input guard after the untrusted text reached the model.
- An unscoped shared vector index (cross-tenant leak).
- Returning an answer with no output guard or eval gate.
- Auditing only the happy path.