« Track Overview · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 33 — Eval-Driven Development (EDD): Building Agentic Systems for Software Delivery

Answers these JD lines: Cohere's Forward-Deployed-Engineer ask for "proven ability to build robust evaluation frameworks, moving well beyond trial and error, to measure agent accuracy, safety, and latency" and to "translate high-value, ambiguous business problems into well-framed agentic workflows with clear success criteria and evaluation methodologies"; the Anthropic / OpenAI-style "write evals" and "continuous evaluation" lines every applied-AI role now carries (jd.md). This phase is the methodology those lines describe: not one more eval mechanism, but the discipline of putting evals at the center of how you build — the way TDD puts tests at the center of how you write software.

Why this phase exists

You already have the mechanisms (Phase 11: trajectory eval, the LLM-judge, Cohen's κ, the regression gate) and the harness (Phase 15: spec → plan → patch → verify). What no earlier phase gives you is the development loop that decides what to build, in what order, and when it is safe to ship — with evals as the steering wheel rather than a report you run at the end.

Traditional testing quietly assumes a single correct output, a deterministic system, and a behavioral surface small enough to enumerate. An LLM/agentic system breaks all three: it samples (so one green run is one draw from a distribution you haven't characterized), there is often no single right answer (a summary, a refactor, a reply), and the behavioral surface is effectively infinite. "Vibe-checking" a few prompts in a playground feels like testing and scales to exactly zero. What carries over is the engineering spine — fixtures, CI, regression discipline, versioned datasets. What's new is that your success criteria are themselves a dataset you build, curate, and grow, and your "test suite" is a statistical instrument you must design and calibrate. EDD is the name for treating that seriously.

The load-bearing reframe: an eval is not a test you write after; it is the specification you write first. For a feature-delivery agent, "done" is not "the demo worked" — it is "the golden task set passes at the bar we agreed, no previously-passing case regressed, and no safety case failed." That is a number you can defend to a skeptical VP and a gate a CI job can enforce.

The EDD loop

The methodology is a loop, and the order matters — evals come first, before the agent is any good.

Steps 6→8 are a flywheel: production failures feed the eval set, a better eval set catches more regressions, and the agent's floor rises with every turn. The teams that win at agentic software delivery are the ones whose flywheel spins fastest.

Concept map

• Eval taxonomy for coding/delivery agents: capability evals (single-turn: does it write a correct function — HumanEval/MBPP-style pass@k with test execution as the grader); trajectory evals (sensible steps, right tools, in bounds); end-to-end task evals (SWE-bench-style: repo + issue → does the final patch make the tests pass); spec-conformance; safety/regression; and latency/cost budgets as first-class eval dimensions.
• Grader design: programmatic/assertion (execution — the gold standard for code) → rubric LLM- judge (with the calibration/bias problems anchored to Phase 11's κ treatment) → pairwise comparison → human-in-the-loop where judgment can't be mechanized.
• Statistics for small eval sets: the unbiased pass@k estimator, bootstrap confidence, why a 2-point delta on 50 tasks is noise, paired per-task comparison, and the training-on-the-test-set trap (overfitting prompts to the eval set) with held-out sets.
• CI/delivery integration: eval gates (cheap smoke per-commit, full suite nightly/pre-release), eval budgets ($/run, minutes), flake/retry policy for stochastic runs, dashboards tracking scores across versions.
• Real-world grounding: SWE-bench / SWE-bench Verified, HumanEval/MBPP, pass@k; the published eval guidance from Anthropic and OpenAI Evals; the "error analysis" practice from applied-AI practitioners (Hamel Husain, Eugene Yan); LangSmith / Braintrust / promptfoo-style tooling.

The labs

Integrated scenario (how this shows up at work)

Your team is asked to ship a PR-review / feature-delivery agent. Product's goal — "make our engineers faster without lowering quality" — is exactly the "high-value, ambiguous business problem" the Cohere JD names. You do not open a playground. On day one you frame success (resolve the issue, don't break existing tests, touch only in-scope files, land under a cost budget), build a golden set of 30 real closed tickets with their tests (Lab 03's dataset), and write execution graders (Lab 02: apply the patch, run the repo's tests). You baseline a naive one-shot agent — 40% resolved — and now every prompt, tool, and planning change is a paired comparison (Lab 01): you keep the ones that lift the score without regressing a task or tripping the safety (out-of-bounds edit) gate. When it ships, you sample production PRs, run error analysis (Lab 03), find that 30% of failures are one mode — the agent edits generated files it shouldn't — and promote those failures into new eval cases so they can never regress silently. A month later a "clever" prompt lifts the dev score 4 points but the held-out set drops 9 — the overfitting detector fires, the release is blocked, and you just avoided shipping a regression dressed up as an improvement. That is EDD, and it is the difference between a demo and a system.

Deliverables checklist

• Lab 01 green under LAB_MODULE=solution pytest and your own lab.py (34 tests).
• Lab 02 green (29 tests); you can explain why breaking a pass_to_pass test makes a task unresolved even when the feature works.
• Lab 03 green (32 tests); you can explain the error-analysis flywheel and why a held-out set catches what the dev gate can't.
• You can write the eight-step EDD loop from memory and say why evals come first.
• You can state the unbiased pass@k estimator and why the naive 1-(1-c/n)^k is wrong.
• You can name three eval types for a coding agent and the right grader for each.

Key takeaways

• Evals are the spec, written first. For an agentic system, "define done" is "write the eval." Adding evals later is flying blind and calling it agile.
• Execution beats judgment for code. Run the tests. An LLM-judge grades tone; it does not grade whether a function is correct.
• Never one number. A single accuracy scalar hides a collapsed slice, a flaky case, and a safety regression — report per-slice, treat flakiness as failure, keep safety a hard gate.
• The flywheel is the moat. Error analysis → promote failures → regression-gate → sample production → repeat. Curation beats volume; a held-out set keeps you honest.
• The senior framing: "I don't ship agent changes I can't defend with a number — a versioned golden set, an execution grader, a paired comparison, and a gate where safety never averages out."