Template 03 — RAG Design Doc

Copy and fill when designing a RAG system. Full method: Phase 9, cheatsheet 14.

RAG Design: `<system name>`

Author: <name> · Date: <date>

1. Goal & data

Questions to answer: <scope>
Sources: <docs, formats, volume, update frequency>
Freshness requirement: <real-time / daily / weekly re-index>
Multi-tenant? <yes/no — if yes, isolation plan below>

2. Ingestion

Step	Choice
Loaders	`<PDF/HTML/MD/...>`
Cleaning	`<dedupe, boilerplate removal>`
Chunking	size `<200–500 tok>`, overlap `<%>`, structure-aware?
Embedding model	`<model>` (same for query + doc)
Store	`<vector DB>` + index `<HNSW>` (+ keyword index for hybrid)
Metadata	`<source, date, tenant_id, ACL>`

3. Retrieval

Step	Choice
Retrieval	dense `<top-k>` + BM25? + RRF fusion?
Rerank	cross-encoder → top `<n>`
Filters	`<tenant_id, ACL, freshness>` (server-side)
Query transform	`<HyDE / rewriting?>`

4. Generation

Context packing: order <best-first/last; lost-in-the-middle aware>, count <n chunks> (Phase 9.07)
Grounding prompt: "answer only from context; cite [n]; say 'I don't know' if absent"
Citations: <format> (Phase 9.08)
Model + params: <model>, temperature 0

5. Evaluation (split retrieval vs generation — Phase 9.09)

Retrieval: recall@k <target>, MRR <target>
Generation: faithfulness <target>, citation accuracy <target>
Golden set: <size/composition>

6. Security & multi-tenancy (Phase 14.04)

Tenant isolation: per-tenant namespace / mandatory tenant_id filter on retrieval; cross-tenant leak tests.
ACLs: user only retrieves permitted docs.
Untrusted content: treat retrieved text as untrusted (indirect injection, Phase 14.01).
PII: <scrubbing / handling> (Phase 14.02)

7. Cost & latency

Token budget: <context size cap>; prefix caching for shared context (Phase 7.05)
Cost per resolved query: <estimate> · p95 latency target: <ms>

8. Failure modes & mitigations

Symptom	Cause	Mitigation
"I don't know" but doc exists	retrieval miss	chunking/embedding/hybrid/top-k
hallucination	weak grounding	grounding prompt + faithfulness eval
cross-tenant leak	missing filter	server-side tenant filter + tests

Retrieval dominates RAG quality. Eval the two halves separately.