Lab 06 — Eval & Guardrails

Goal. Make the system measurably safe: build a groundedness/relevance eval harness (RAGAS + Foundry-style evaluators), a guardrail stack (Content Safety + PII redaction + output checks), and an eval gate you can wire into CI (Lab 05). JD4: LLM evaluation + security. Read K08 first.

Stack. RAGAS · Azure AI Content Safety · Azure AI Language PII / Microsoft Presidio · an LLM-judge for groundedness · pytest for the gate. Local fallback. RAGAS with a local judge model; Presidio for PII; a simple harm-word/regex filter standing in for Content Safety (note the difference in your write-up).

Run it (offline, no Azure)

pip install -r requirements.txt        # only pytest; pure stdlib
python run.py                          # metrics + stratified + guardrails + gate
python gate.py                         # the CI eval gate (exit 0 = PASS, 1 = FAIL)
pytest -q                              # 10 tests = success criteria

The harness, metrics, stratification, guardrail stack, and CI gate are real and run offline. The groundedness "judge" is a deterministic stand-in for an LLM-judge / Foundry evaluator; the PII/injection detectors stand in for Presidio / Content Safety Prompt Shields — swap those for the cloud services and the harness is unchanged.

Code tour

File	Teaches (K08)
evaluate.py	recall@k, groundedness/faithfulness (judge), answer relevance, refusal-correctness, stratify by language
guardrails.py	PII redaction (IBAN/PAN/Emirates-ID/email/phone), injection detection, output safety/leak check
dataset.py	golden set (incl. a hallucination + Arabic) + adversarial red-team set
gate.py	the CI eval gate that fails the build on a quality/safety regression
run.py / test_eval.py	demo + success criteria (the metric catches the hallucination; PII is redacted; gate flips on regression)

Steps

Eval set. Build a versioned golden set: ~25 (question, ideal answer, expected source) tuples — representative + edge + out-of-scope (must refuse) + Arabic and English. Plus an adversarial set: jailbreaks, indirect injection (instructions hidden in a doc), PII-extraction, "make up a rate" traps.
Retrieval metrics. On the golden set, compute recall@k / context precision for the Lab 01 retriever.
Generation metrics. Run RAGAS (faithfulness/groundedness, answer relevancy, context precision/recall) and an LLM-judge groundedness scorer (require it to cite the unsupported claim). Calibrate the judge against ~10 of your own human labels and report agreement.
Stratify. Report metrics per language and per intent so an Arabic regression can't hide in the average.
Input guardrails. Content Safety (harm categories) + Prompt Shields (jailbreak/injection) + PII detection/redaction before the model + scope check.
Output guardrails. Groundedness check + Content Safety on output + PII/secret leak filter + JSON-schema/citation validation.
Red-team. Run the adversarial set through the full stack; measure jailbreak/injection resistance and PII-leak rate; log every trigger.
CI gate. A pytest that fails if groundedness, refusal-correctness, or safety drops below thresholds — the gate Lab 05's pipeline calls.

Measurable result

A metrics report (recall@k, groundedness/faithfulness, answer relevance, refusal-correctness, jailbreak/injection resistance, PII-leak rate), stratified by language, plus a CI gate that goes red when you deliberately weaken the grounding prompt or add a leaky log line. Demonstrate Prompt Shields/least-privilege stopping an indirect-injection attempt.

Stretch

Use Foundry evaluators + continuous (online) evaluation on sampled traffic → Azure Monitor alerts (K08 §6).
Automate red-teaming with PyRIT.
Add an agent evaluator (tool-call accuracy, task success) for the Lab 02 agent.

Talking point

"In a bank, evaluation is what lets you ship at all. I gate releases offline on groundedness, refusal-correctness, and safety — stratified by language so Arabic can't regress unseen — and I monitor sampled production traffic continuously. The guardrail stack is defense-in-depth (Content Safety + Prompt Shields + PII redaction in and out), but the real safety is architectural: least-privilege tools and human approval mean even a successful injection can't move money."

Resume bullet

"Built a RAG/agent evaluation harness (RAGAS + Foundry evaluators + calibrated LLM-judge groundedness) with language-stratified metrics and a CI eval gate, plus a defense-in-depth guardrail stack (Content Safety harm filters, Prompt Shields, Presidio PII redaction in/out, citation/JSON validation) and an automated red-team suite for jailbreak and indirect-injection resistance."

AI Specialist