Phase 10 — GenAI in Production: RAG, Guardrails & Hallucination Analysis
Difficulty: ⭐⭐⭐⭐☆ Estimated Time: 2 weeks (40–50 hours) Roles Supported: the JD's "generative AI applications using embeddings, vector databases, RAG pipelines, agent workflows, prompt engineering, and guardrails" plus "hallucination analysis"
Why This Phase Exists
The llm-inference track teaches how RAG and agents work; this phase teaches what it takes to put them in front of enterprise users — which is a different discipline. Production GenAI is defined by its failure management: hallucinations measured rather than anecdotal, prompt injection treated as an attack surface, outputs validated against schemas before any downstream system trusts them, and an evaluation harness that runs on every change like a test suite.
The flagship lab builds that harness: groundedness scoring, injection canaries, a policy engine, and structured-output validation — deterministic, model-free (rule-based scorers + a mock LLM), so the system is what's tested. Swapping in a real LLM and LLM-as-judge scorers is the extension.
Concepts
- The production RAG loop: retrieve → assemble context (budgeted) → generate → validate → respond-or-fallback
- Groundedness/faithfulness: is every claim in the answer supported by the retrieved context? Sentence-level support scoring; the unsupported-claim rate as the hallucination metric for RAG
- Prompt injection: direct and indirect (poisoned documents); defense layers — privilege separation, input/output filtering, instruction hierarchy; canary-based regression testing
- Structured output: schema-constrained generation, validate-then-repair loops
- Refusal policy: what the system must not answer; topical guardrails
- Eval harnesses for GenAI: golden sets, canary suites, LLM-as-judge (and its biases), regression gating (Phase 12 wires this into CI)
Labs
Lab 01 — Guardrail & Eval Harness (flagship, implemented)
| Field | Value |
|---|---|
| Goal | Build the validation layer of a production RAG service: groundedness scorer, injection detector with a canary suite, refusal-policy engine, structured-output validator with one-shot repair, and the harness that runs all of it as a gate |
| Concepts | Sentence-level support, n-gram/token overlap scoring, canary suites, policy-as-data, validate-then-repair |
| How to Test | pytest test_lab.py — 14 tests: groundedness math on hand-built cases, injection canaries caught, clean inputs pass, policy decisions, schema repair loop, harness gating |
| Talking Points | Why is unsupported-claim rate the right hallucination metric for RAG? Why are canaries better than classifiers alone? What does LLM-as-judge add and what biases does it bring? |
| Resume Bullet | Built the guardrail layer for a RAG service: groundedness scoring, injection canary suite, policy engine, and schema-validated outputs with repair — wired as a regression gate on every prompt/retrieval change |
→ Lab folder: lab-01-guardrail-harness/
Extension Project A — Real-Model Harness (spec)
Swap the mock LLM for a real one (any provider); replace the rule-based groundedness scorer with an NLI model or LLM-as-judge; measure judge-vs-rule agreement on 50 hand-labeled cases; document the judge's failure modes (position bias, verbosity bias, self-preference).
Extension Project B — Indirect-Injection Red Team (spec)
Poison a document corpus with adversarial instructions; measure the attack success rate through your RAG pipeline with and without each defense layer; produce the defense-in-depth report.
Guides in This Phase
- WARMUP.md — the full zero-to-expert guide
Deliverables Checklist
- Lab 01 implemented; all 14 tests pass
- Extension A judge-agreement study
- Extension B red-team report