« Track Overview · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 11 — Agent Evaluation: LLM-as-Judge, Golden Datasets & Behavioral Regression

Answers these JD lines: Docker's "LLM-as-judge, behavioral regression testing, golden datasets"; Juniper Square's "evaluation frameworks" and "feedback systems"; Wolters Kluwer's "AI-agent evaluation"; Anthropic's "agent prompts & evals" — the discipline every one of these roles in jd.md treats as the line between a demo and a platform.

Why this phase exists

Load-bearing truth #5 of this whole track: an agent you cannot evaluate, you cannot ship. Everything before this phase builds agent behavior; this phase measures it, and measurement is what turns "it worked when I tried it" into a number you can defend, a regression you catch before the customer does, and a safety property you can prove holds on every commit.

The core problem is that an agent is a stochastic system, so a single successful run tells you almost nothing — it is one sample from a distribution you have not characterized. Four ideas do the work:

1. The golden dataset is the moat. You evaluate your task on your curated, versioned, labelled examples — not a public benchmark that measures something adjacent. The dataset is code: reviewed, diffed, and grown from every incident. It is the most valuable and least copyable artifact your team owns.
2. The scoring ladder: cheapest reliable rung first. Programmatic checks (exact/contains/ numeric/rubric) beat a calibrated LLM-judge beats a human — in cost, speed, and determinism. Reach for a model-based judge only for the parts genuinely too fuzzy to assert in code.
3. Grade the path, not just the destination. An agent that returns the right answer via the wrong tools, in the wrong order, or after an expensive detour is a latent bug. Trajectory evaluation grades the tool sequence; it is what separates agent eval from QA eval.
4. Evals are a CI gate, and safety is a gate, not an average. Re-run the golden set on every change; fail the build on a pass-rate regression; treat any safety failure as a hard block regardless of the aggregate. One number hides regressions — a per-label breakdown and a separate safety gate surface them.

And the load-bearing warning: an LLM-as-judge is a model, and a model can be wrong and biased. Before a judge is allowed to gate CI you must measure its agreement with human labels (Cohen's κ); a judge you haven't validated is a random-number generator in a lab coat.

Concept map

• Golden dataset: curated, versioned, representative + adversarial EvalCases with a reference answer, an expected tool trajectory, and slice labels. Evaluate your task, not a benchmark.
• Scoring ladder: programmatic (exact_match, contains, numeric_within, rubric_score) → calibrated LLM-judge → human. Cheapest reliable rung first.
• Trajectory evaluation: grade the tool sequence — exact order, soft order (LCS), and order-insensitive set-overlap via precision_recall_f1. Path, not just answer.
• LLM-as-judge: pointwise vs pairwise; the biases (position, verbosity, self-preference); validate with Cohen's κ against human labels before trusting it (judge_is_trustworthy).
• Regression testing: run_suite → pass rate + per-label breakdown + trajectory accuracy; regression_gate (fail on a drop vs baseline) and safety_gate (hard-fail on any safety miss).
• Online vs offline: offline golden-set evals in CI; online A/B tests, guardrail metrics, and human-feedback loops in production (Juniper's "feedback systems").

The lab

Integrated scenario (how this shows up at work)

Your team ships a support agent. It looks great in the demo. You build a golden set of 200 real tickets, each with a reference resolution, the expected tool trajectory (lookup_order → check_policy → draft_reply), and labels (refund, billing, safety). Programmatic scorers grade the closed-form parts; a rubric grades the reply; an LLM-judge — which you first validated at κ = 0.71 against a human-labelled slice — grades tone. CI runs the suite on every PR. A month later a prompt tweak lifts the overall pass rate 1% but the per-label breakdown shows the refund slice collapsed from 94% to 61% and one safety case now leaks an account id. The regression_gate and safety_gate block the merge automatically. You shipped nothing broken, and you can show the reviewer exactly why. That is the difference this phase trains.

Deliverables checklist

• Lab 01 green under LAB_MODULE=solution pytest and your own lab.py.
• You can name the scoring ladder and justify "programmatic before judge before human."
• You can explain trajectory evaluation and why a right answer via the wrong path is a bug.
• You can explain the three judge biases and why κ against humans gates a judge in CI.
• You can explain evals-as-CI and why safety is a separate hard gate, not a weighted term.

Key takeaways

• The golden dataset is the moat — versioned, labelled, grown from incidents; evaluate your task, not a public benchmark.
• Climb the scoring ladder from the bottom: programmatic checks are cheap, fast, and deterministic; escalate to a judge only when you must, to a human only rarely.
• Grade the trajectory, not just the answer; a lucky right answer is still a bug.
• An LLM-judge is a model with known biases — validate it with Cohen's κ before you trust it, and never let one aggregate number hide a per-slice or safety regression.