« Overview

Agentic Engineering Lab Standard

Every lab in this track builds a runnable, test-verified miniature of a real piece of agent infrastructure — the ReAct/ReWOO control loop, the tool-call validator and repair loop, the MCP JSON-RPC server, the context assembler, the hybrid retriever, the RAPTOR tree, the multi-agent message bus, the durable/replay-safe workflow engine, the capability-gated sandbox, the prompt-injection guardrail chain, the LLM-as-judge eval harness, the async agent service, the multi-tenant gateway, the cost/latency/trace meter, the spec→plan→patch coding loop. You do not "call Runner.run(agent)"; you build the runner — the loop that turns a model step into a validated tool dispatch, records the trace, enforces the step budget, and survives a crash. That is what makes the knowledge defensible in a Staff/Principal interview and useful at 2 a.m. when an agent is looping, leaking, or lying.

Why miniatures (and not a real LLM behind a real framework)

A real agent stack is non-deterministic (the LLM samples), costs money (every step is tokens), needs credentials and network, and hides the mechanism behind a framework API (LangGraph's StateGraph, the OpenAI Agents SDK Runner, Temporal's worker, Docker's sandbox). A lab that reimplements the algorithm — the reason→act→observe loop, the JSON-Schema validator, the RRF fusion, the recursive-summary tree, the event-sourced replay, the capability check, the injection-scan, the judge-aggregation, the token-bucket quota — is offline, deterministic, free, and teaches the part interviewers actually probe. Each lab README ends with a "How this maps to the real stack" section that ties the miniature back to LangGraph / OpenAI Agents SDK / Google ADK / Temporal / MCP / pgvector / Neo4j / Docker / FastAPI, states its limits, and names the real API equivalent.

The one trick that makes agents testable: inject the model

The LLM is the only non-deterministic part of an agent. So every lab injects it as a plain callable — a Policy: Callable[[str], str] or a FakeLLM with a scripted, stateless response table. The "model" becomes a pure function of the scratchpad, the whole loop becomes reproducible, and the test can assert the exact trace. This is not a shortcut; it is the same seam real teams use to unit-test agents (record/replay, VCR-style fixtures, judge stubs). The lab teaches you to build code whose correctness does not depend on the model being right — which is the entire discipline of production agent engineering.

Required files (per lab)

The runnable core is Python (stdlib + pytest), offline, deterministic. No real LLM, no GPU, no CUDA, no network call to a provider, no model download, no pip install torch, no Date.now() / unseeded random. Where a lab needs vectors, embeddings, or matrices, implement them with stdlib lists / array / math so the mechanism is visible — a hashing embedder, a bag-of-words cosine, a hand-rolled BM25. NumPy is allowed only if a lab explicitly declares it in requirements.txt, and even then the algorithm (not a one-line library call) must be the thing the learner writes. A handful of platform labs may use fastapi/httpx in an Extensions stub, but the core lab always ships a stdlib fallback so it runs with zero installs. Multi-language / real-framework work (a real MCP server over stdio, a real Temporal worker, a real Docker sandbox) appears only as build specs in the README "Extensions" section for the learner's own machine.

Determinism rules (agents touch probability and time — be careful)

• Any randomness goes through an explicit, seeded random.Random(seed) passed in or defaulted; same seed → same bytes. Tests assert this.
• Any "current time" is an injected clock (now: Callable[[], float] or an integer tick counter), never time.time(). Retries, backoff, TTLs, quotas, and traces all need a fake clock so tests are reproducible — and a durable workflow engine that reads the wall clock is not replay-safe, which is itself a lesson (Phase 08).
• The injected "LLM" is a pure function of its input (scratchpad / messages). No hidden state, no counters that persist across calls unless the lab is explicitly testing memory.
• Floating-point comparisons in tests use pytest.approx (or an explicit abs-tolerance), never == on floats. Any softmax / log-sum-exp uses the max-subtraction / log-space trick so large-logit inputs don't overflow.

The test contract

import importlib, os
lab = importlib.import_module(os.environ.get("LAB_MODULE", "lab"))

Run both ways; the reference must pass, and your lab.py passes once the TODOs are filled:

pytest test_lab.py -v                       # against your lab.py (red until you implement)
LAB_MODULE=solution pytest test_lab.py -v   # against the reference (must be green)
python solution.py                          # the worked example

Test taxonomy every flagship lab includes: happy path; malformed / out-of-range input (a tool call with the wrong types, an unknown MCP method, an empty retrieval set, a degenerate preference pair) that becomes a structured error, not a crash; boundary cases — the off-by-one interviewers probe (max_steps reached, empty tool registry, a full cache, a quota at exactly the limit, a workflow replayed from step 0 vs step N, a tenant with no rows); security cases (an injected instruction does not escape the trust boundary; an exfil URL is caught; a capability check denies a disallowed path); invariants (the loop never exceeds max_steps; a replay produces byte-identical output; RRF is order-independent; the judge aggregation is monotone; the quota never goes negative); and deterministic output (same seed + same clock → same trace).

The four teaching docs (per phase)

Every phase explains the same material through four lenses, because senior-level understanding is their intersection:

Doc conventions (mdBook-compatible)

• WARMUP.md opens with a Table of Contents of working anchor links, kept in sync with the headings. mdBook lowercases, replaces spaces with hyphens, and strips punctuation to form anchors.
• MathJax for any math (\( … \) inline, $$ … $$ block). Standard Markdown only.
• No bare angle brackets in prose — wrap <like-this> in backticks so mdBook does not eat them as HTML. This bites constantly in this domain: <tool_call>, <|assistant|>, notifications/tools/list_changed, generic types — always fence them.
• File references use relative links. Code fences are language-tagged.

Definition of done

A lab is complete only when: the reference suite passes (LAB_MODULE=solution pytest), the learner lab.py passes after the TODOs are filled, python solution.py prints a sensible worked example, the README's success criteria are testable, and the "How this maps to the real stack" section is accurate about the production framework and its limits. A phase is complete when all four teaching docs exist, the WARMUP's ToC anchors resolve, and every lab in it is done. The track is complete when mdbook build agentic-engineer exits 0 and every lab in the repo passes under LAB_MODULE=solution.