Capstone — Security Checklist (Component 16)

The filled security posture for the platform. Based on the security checklist template and Phase 14. Fill the <...> for your build.

Threat model (one page)

  • Untrusted-text entry points: user input, RAG documents, tool/agent results, conversation memory.
  • Privileged actions: the agent's tools, DB writes, sends; provider keys.
  • Top risks (OWASP LLM): prompt injection, sensitive-info disclosure, insecure output handling, excessive agency, cross-tenant disclosure.

Controls (this platform)

Prompt injection (14.01)

  • Trust boundary: the agent model only proposes tool calls; agent/executor.py validates + permission-checks + executes.
  • Tools whitelisted; args validated against schema; least-privilege.
  • RAG/tool text treated as untrusted (isolated from instructions/secrets).
  • Output exfil scan (suspicious URLs/markdown) before render.
  • Approval gate on high-impact tools (agent/approval.py).
  • Red-team report: <link to labs/lab-14 run>.

Secrets (14.03)

  • Provider keys server-side only (gateway custody); clients get scoped internal keys.
  • No secrets in prompts/context/tool args/weights/logs.
  • Keys vaulted + scoped + spend-capped + rotatable; CI secret scanning.
  • BYOK (if used): per-tenant encrypted, never logged.

Data & privacy (14.02)

  • PII scrubbed before logging + before provider (where not needed).
  • Retention TTLs; deletion job; end-to-end erasure path (DB+logs+vector+provider).
  • Provider data policy: <no-train tier / DPA / ZDR / region>.

Tenant isolation (14.04)

  • tenant_id from trusted auth; threaded through DB (RLS), vector namespace, cache key, memory, logs.
  • Default-deny on missing scope.
  • Cross-tenant leak test suite (eval/leak_tests.py) in CI.
  • Per-tenant quotas/budgets (noisy-neighbor).

Guardrails (14.05)

  • Input + output guardrails (moderation, PII, schema, grounding, exfil).
  • Fail closed on guardrail dependency failure (high-risk paths).
  • Violation + over-refusal rates measured: <numbers>.

Audit & ops (14.06)

  • Audit log: consequential actions, append-only, tamper-evident, PII-safe, tenant-tagged.
  • Correlation IDs; traces; alerting; IR plan (RUNBOOK.md).

Compliance (if enterprise) (14.07)

  • SOC 2 (status); DPA + sub-processor list; SSO/RBAC; deployment model; data map.

The "can it be attacked?" trace

For each untrusted-text entry point, trace whether attacker text can reach a privileged action or another tenant's data. If yes → architectural fix (boundary/isolation), not a prompt tweak. Document the trace here.

Numbers to report

Injection red-team pass rate · cross-tenant leak tests (0 leaks) · PII-in-logs (0) · guardrail violation + over-refusal rates · secrets-reachable-by-model (0).