« Phase 00 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 00 Warmup — The Agentic Engineer's Mental Model

Who this is for: someone who can write Python but has never built an agent, or has only called one through a framework. By the end you will hold the four numbers and one boundary that every later phase — and every Staff agent interview — depends on. Nothing here needs a GPU, an API key, or a framework. It is arithmetic and one idea about trust.

What is an "agent," precisely?
The trust boundary: model proposes, application executes
The anatomy of the agent loop
Reliability compounds: the 0.95^n law
Buying reliability back: retries, verification, and the step budget
The cost model: tokens, and why ReAct is quadratic
ReWOO and the plan-execute family: linear tokens, less adaptivity
Cost per resolved task: the number the business cares about
Latency: the tail is the story
The decision: workflow, agent, or agent-with-guardrails
Common misconceptions
Lab walkthrough
Success criteria
Interview Q&A
References

1. What is an "agent," precisely?

Strip away the marketing and an agent is three things wired into a loop:

A model — a large language model (LLM), which is a function from a text prompt to a probability distribution over the next token, sampled repeatedly to produce text. That is all it does: it reads text and writes text. It has no hands.
Tools — functions your program exposes (search the web, query a database, send an email, run code) that the model can ask to run by emitting a structured request.
A loop with memory — your code, which feeds the model a prompt, reads the text it writes, decides whether that text is a tool request or a final answer, runs the tool if so, appends the result to a running record (the scratchpad or transcript), and asks the model again — until the model says it's done or a budget runs out.

Contrast this with a plain LLM call: prompt in, text out, once. And contrast it with a workflow: a fixed sequence of steps you wrote by hand (maybe with LLM calls inside), where you decided the order, not the model. The defining feature of an agent is that the model chooses the next action at runtime. That single freedom is the source of all its power (it can handle tasks you didn't fully script) and all its danger (it can choose wrong, loop, or be manipulated).

A useful one-liner, due to Anthropic's Building Effective Agents: "Agents are systems where LLMs dynamically direct their own processes and tool usage." Workflows are systems where the path is fixed in code. Most production "agents" are, and should be, workflows — we return to this in §10.

Why start here and not with code? Because the single most common failure of junior agent engineers is to reach for an autonomous agent when a workflow would be more reliable, cheaper, and easier to debug. The mental model is the skill; the frameworks are commodity.

2. The trust boundary: model proposes, application executes

Here is the most important sentence in this entire track:

The model proposes; the application executes.

The LLM never actually does anything in the world. When an agent "sends an email," what physically happens is: the model emits text that looks like a request to send an email — say, {"tool": "send_email", "to": "finance@corp", "body": "..."} — and then your code parses that text, decides whether to honor it, and calls the real email API. The model's output is a proposal. Your code is the executor. The line between them is the trust boundary.

Why does this matter so much? Because the model's output is untrusted. It is the product of a stochastic process trained on the internet, and — critically — it will faithfully follow instructions that appear in its input, even when those instructions came from a malicious web page, a poisoned document, or a hostile user (this is prompt injection, Phase 10). So:

Everything on the model side of the boundary is untrusted text. Treat it like user input from the open internet, because that is exactly what it is.
Everything on the application side — the validator that checks the tool arguments, the allow-list of tools this agent may call, the sandbox the tool runs in, the authorization check that this tenant may touch this row — is where every safety and correctness property lives.

        UNTRUSTED                         TRUSTED (your code)
   ┌───────────────────┐        ┌──────────────────────────────────┐
   │  LLM emits text:  │        │  parse → validate schema/types   │
   │  "call send_email │  ───►  │  → check allow-list & authz      │  ───►  real
   │   to: finance…"   │        │  → run in sandbox w/ capabilities│        action
   └───────────────────┘        │  → scan output for exfil         │
      a PROPOSAL                 └──────────────────────────────────┘
                                     the EXECUTOR = the boundary

Internalize this and three whole phases stop being mysterious: tool validation (Phase 02) is guarding the boundary against malformed proposals; sandboxing (Phase 09) is containing what a proposal can touch once honored; the injection defenses (Phase 10) are assuming the proposal is hostile and refusing to let it escalate privilege or exfiltrate data. There is no prompt that makes the model trustworthy. The boundary is architectural.

3. The anatomy of the agent loop

Let's make the loop concrete, because every later mechanism plugs into it. In pseudocode:

def run(task):
    scratchpad = task                     # the running record
    for step in range(max_steps):         # the reliability & cost budget
        proposal = model(scratchpad)      # UNTRUSTED text out of the LLM
        if is_final_answer(proposal):
            return extract_answer(proposal)
        name, args = parse_action(proposal)   # cross the trust boundary
        if not registry.allows(name, args):   # validate on the trusted side
            observation = "ERROR: invalid or disallowed action"
        else:
            observation = registry.run(name, args)   # execute (sandboxed)
        scratchpad += f"\n{proposal}\nObservation: {observation}"   # remember
    return give_up_partial(scratchpad)    # the guard fired

Notice the four load-bearing details, each of which becomes a phase:

max_steps — the step budget. Without it a confused model loops forever, burning tokens and never terminating. This is your first, cheapest reliability control (§4).
parse_action — the trust-boundary crossing. The model's text becomes structured data your code can reason about, and malformed text becomes a recoverable observation, not a crash (Phase 02).
registry.allows(...) — authorization + validation. Least privilege lives here (Phases 02/09/10/13).
scratchpad += observation — memory. The scratchpad grows every step, which is exactly why cost is quadratic (§6) and why context engineering (Phase 04) exists.

That is the whole game. Every framework — LangGraph, the OpenAI Agents SDK's Runner, Google ADK — is an elaboration of this loop with better ergonomics, persistence, and observability. You will build it for real in Phase 01.

4. Reliability compounds: the 0.95^n law

Now the arithmetic that decides architecture. Suppose each step of your agent — one reasoning turn plus one tool call — succeeds independently with probability $p$. For the whole task to succeed, every step must succeed. The probability of that is the product:

$$P(\text{all } n \text{ steps succeed}) = p^n.$$

Why the product? Independent events combine by multiplication: the chance two independent things both happen is the chance of the first times the chance of the second. Extend to $n$ and you get $p^n$. Plug in the numbers interviewers love:

steps $n$	$0.95^n$	$0.9^n$	$0.99^n$
1	0.95	0.90	0.99
5	0.77	0.59	0.95
10	0.60	0.35	0.90
20	0.36	0.12	0.82
50	0.08	0.005	0.61

Read the bold cell: a ten-step agent whose steps each work 95% of the time — which sounds great — completes the whole task correctly only about 60% of the time. At 90% per step it is a coin flip by step 7. This is not pessimism; it is multiplication, and it is the reason "just let the agent run for 30 steps" is a beginner's plan.

Three consequences you should be able to recite:

Shorter chains are dramatically more reliable. Cutting steps from 10 to 5 at $p=0.95$ takes you from 0.60 to 0.77. This is why decomposing a task into fewer, larger, verified steps beats many tiny ones.
Per-step reliability matters more than it looks, because it's raised to a power. Moving $p$ from 0.95 to 0.99 at $n=10$ takes you from 0.60 to 0.90. Every bit of validation, every typed tool, every retry that raises $p$ pays off exponentially.
Independence is the pessimistic assumption, and reality is usually a bit better (a good plan makes later steps easier) or worse (a bad early step corrupts everything after). Either way, $p^n$ is the right first model, and it's the one you defend in the review.

The lab's compound_reliability(p, n) is literally p ** n. The whole point is that this trivial function encodes the deepest constraint in agent design.

5. Buying reliability back: retries, verification, and the step budget

If long chains are unreliable, how do real agents ever work? By buying reliability back. There are exactly three levers, and every reliability mechanism in this track is one of them:

Lever 1 — raise $p$ per step (retries). A step you can retry fails only if the original attempt and every retry fail. With independent attempts:

$$P(\text{success within } r \text{ retries}) = 1 - (1-p)^{r+1}.$$

A step at $p=0.8$ with 2 retries reaches $1-0.2^3 = 0.992$. But retries only work for idempotent steps — retrying "send the email" twice sends two emails. Making steps idempotent so they're safe to retry is a core durability topic (Phase 08).

Lever 2 — shorten the chain $n$. Decompose into fewer, larger steps; hard-code the parts that don't need the model (turn agent steps into workflow steps); cache results so a repeated sub-task isn't re-run.

Lever 3 — verify instead of trusting. Add a critic step (Phase 07) or an eval gate (Phase 11) that catches a bad step before it propagates, or a human approval on high-impact actions (Phase 10). Verification converts "hope the step was right" into "check that it was."

Putting levers 1 and 2 together gives you the step budget: given a per-step reliability $p$ and a required end-to-end target $T$, the largest number of unverified autonomous steps you can afford is the largest $n$ with $p^n \ge T$. Take logs (log is increasing, and $\log p < 0$ for $p<1$, which flips the inequality):

$$n \le \frac{\log T}{\log p} \quad\Rightarrow\quad n_{\max} = \left\lfloor \frac{\log T}{\log p} \right\rfloor.$$

At $p=0.95, T=0.90$: $\log 0.9 / \log 0.95 = 2.05$, so 2 steps. Beyond two unverified steps you must add a lever. That is the number the lab's max_autonomous_steps computes, and it's the number you put on the whiteboard to justify "we need a checkpoint here."

6. The cost model: tokens, and why ReAct is quadratic

Agents cost money, and the currency is tokens. A token is a chunk of text (roughly ¾ of a word) that the model reads or writes; providers bill per million tokens, and — this matters — output tokens usually cost 3–5× input tokens because generating is more expensive than reading. Phase 04 covers tokenization properly; here we just count.

The classic agent pattern is ReAct (Reasoning + Acting): the model interleaves a thought, an action (tool call), and then reads the observation, and repeats. The catch is in how the loop feeds the model. On every step, the entire scratchpad so far is resent so the model can reason about the latest observation in full context. So the input grows every step. Let each step contribute t thought/action tokens the model generates and o observation tokens the tool returns, on top of a fixed system prompt (with tool schemas) and task, both resent every call. On call $i$ (0-indexed) the scratchpad already holds $i$ completed steps:

$$\text{input}_i = \text{system} + \text{task} + i,(t + o).$$

Sum over $i = 0 \dots n-1$. The constant part is $n(\text{system}+\text{task})$; the growing part is $(t+o)\sum_{i=0}^{n-1} i = (t+o)\frac{n(n-1)}{2}$:

$$\text{total input} = n(\text{system}+\text{task}) + (t+o)\frac{n(n-1)}{2}.$$

That $n^2/2$ term is the headline: ReAct's input tokens grow quadratically in the number of steps. Double the steps and you more than double the token bill. A 20-step ReAct research agent can spend the overwhelming majority of its tokens re-reading its own scratchpad. This is not a bug; it's the price of interleaved reasoning — the model needs the history to reason well. But it's the first thing to profile when an agent's cost surprises you, and the reason context compression (Phase 04) and caching (Phase 14) exist.

The lab's react_cost computes exactly this, and a test asserts the super-linear growth.

7. ReWOO and the plan-execute family: linear tokens, less adaptivity

ReWOO — Reasoning WithOut Observation — attacks the quadratic directly. Instead of interleaving, it splits the agent into three modules:

Planner — one LLM call that writes the entire plan up front: a list of steps, each naming a tool and its arguments, using variable substitution (#E1, #E2, …) so a later step can consume an earlier step's result without the model seeing it yet.
Workers — the tools, run by your code without the LLM, filling in the evidence variables. The model isn't in this loop at all.
Solver — one final LLM call that reads the task plus all the gathered evidence and writes the answer.

So the model is called exactly twice, regardless of how many tool steps there are:

$$\text{total input} = \underbrace{(\text{system}+\text{task})}{\text{planner}} + \underbrace{(\text{system}+\text{task}+n\cdot o)}{\text{solver}}.$$

Input grows linearly in $n$ — that $n\cdot o$ evidence term is the only growth, and the giant quadratic scratchpad-resend is gone. ReWOO papers report multiple-× token savings on multi-step benchmarks for exactly this reason. In the lab's default 10-step example, ReWOO uses about 5–6× fewer input tokens than ReAct.

What do you give up? Adaptivity. ReWOO commits to the whole plan before seeing any observation, so if step 3's result invalidates the plan, ReWOO barrels on and the Solver has to cope with the mess; ReAct would have noticed at step 3 and changed course. This is the central tradeoff of the plan-execute family (ReWOO, LLMCompiler, "plan-and-execute" agents):

	ReAct (interleaved)	ReWOO / plan-execute
LLM calls	one per step ($n$)	two (plan + solve)
input tokens	quadratic $O(n^2)$	linear $O(n)$
adaptivity	high — reacts each step	low — commits to the plan
parallelism	none (sequential)	independent steps can run in parallel
best when	path is uncertain, needs mid-course correction	path is fairly predictable, cost matters

Real frameworks let you mix them: a LangGraph graph can have a planning node and interleaved worker nodes; the choice is per-task, and this is the analysis that justifies it. You build both cost models in the lab so the tradeoff is muscle memory, not a slide you once saw.

8. Cost per resolved task: the number the business cares about

Token cost per attempt is not what the business pays for; it pays for results. If an attempt costs unit_cost and only a fraction success_rate of attempts actually succeed (from your eval, Phase 11), the expected cost to get one good result is:

$$\text{cost per resolved task} = \frac{\text{unit_cost}}{\text{success_rate}}.$$

A "cheap" agent at $0.05/attempt that succeeds 40% of the time costs $0.125 per real result — more than an $0.10 agent that succeeds 95% ($0.105). This is why you cannot talk about cost without talking about reliability, and why the FinOps and eval conversations are the same conversation (Phase 14). It's also the denominator every pricing and margin discussion divides by: if you charge $0.20 per resolved task and it costs you $0.125, your gross margin is 37.5%, and a reliability regression that drops success to 60% quietly turns that margin negative.

9. Latency: the tail is the story

Users don't experience your average latency; they experience their latency, and some of them land in the slow tail. So agents are judged on percentiles, not the mean.

The p95 latency is the value below which 95% of requests complete — meaning 1 in 20 requests is slower than p95. p99 is 1 in 100. These tails are routinely several times the mean because latency distributions are right-skewed (a few requests hit a cold cache, a slow tool, a retried step, a GC pause). A system with a great 0.6s mean can still violate a 2s SLO at p99.

Two facts make this acute for agents:

Agent loops are sequential. Step $N$ needs step $N-1$'s observation, so total latency is the sum of step latencies, not the max. Ten steps at 0.5s each is 5s before you count the tail. (ReWOO's independent tool calls can run in parallel, turning a sum into a max for those steps — another point in its favor.)
The tail compounds down the chain. If each step's latency is itself a distribution, the chance that at least one step lands in its own slow tail rises with the number of steps.

So the design move is: measure per-step latency as a distribution, take the p95 per step, sum across the (sequential) chain, and check it against the budget. The lab's percentile, sequential_latency, and fits_budget do exactly this, and a test proves the p95 exceeds the mean so you never again quote an average in a capacity review.

10. The decision: workflow, agent, or agent-with-guardrails

Now assemble the numbers into the decision that opens every design review. The principle, straight from senior practice, is use the least-agentic thing that works:

Workflow — if the sequence of actions is known ahead of time and doesn't branch on runtime observations, hard-code it. You get determinism (Phase 08 durability comes almost free), lower cost (no LLM re-planning a fixed pipeline every run), and trivial debuggability. Most tasks marketed as "agents" are workflows with one fuzzy sub-step. A router that picks one of three known pipelines is still a workflow, not an agent.
Agent — if the task genuinely needs the model to choose the next action at runtime (the path depends on what it finds), and the required step count fits inside your reliable step budget from §5, a plain agent is fine.
Agent-with-guardrails — if it needs runtime flexibility but the step count exceeds the reliable budget, you need the agent and you must buy reliability back: checkpoints (Phase 08), a verifier/critic (Phase 07), an eval gate (Phase 11), or a human-in-the-loop approval (Phase 10). This is where most serious production agents land.

The lab's workflow_vs_agent encodes exactly this priority order and returns the reason, so you practice saying it out loud. The most impressive answer in an agent interview is frequently "this shouldn't be an agent, and here's the arithmetic."

11. Common misconceptions

"More autonomy is more advanced." No — more autonomy is more risk and more cost. Seniority is knowing when less autonomy is the better system. The impressive engineer removes agency, not adds it.
"95% per step is basically reliable." $0.95^{10}=0.60$. It is not.
"Tokens are cheap, don't optimize early." ReAct's quadratic growth means a long agent's bill is dominated by re-reading its own history; the shape bites before the per-token price does, and $/resolved-task (not $/attempt) is what you're actually paying.
"Our average latency is fine." The average is a liar; users live in the tail. Design to p95/p99 or get paged by the 1-in-20.
"The prompt will handle safety/edge cases." The prompt is a suggestion to an untrusted, stochastic text generator. Safety and correctness live on your side of the trust boundary, in code — validation, allow-lists, sandboxes, authz, human gates.
"ReWOO is just a cheaper ReAct." It's cheaper and less adaptive; it trades mid-course correction for token savings and parallelism. Picking between them is the analysis, not a default.

12. Lab walkthrough

Open lab-01-agent-reliability-cost-calculator/ and fill the TODOs top to bottom — the file is ordered to match this warmup:

Reliability — compound_reliability (p**n), max_autonomous_steps (the log formula, with the p==1.0 sentinel), reliability_with_retries (1-(1-p)**(r+1)). Validate inputs; the tests probe the edges (n=0, p=1.0, out-of-range).
Cost — TokenSpec.__post_init__ validation, dollars, then react_cost (mind the (steps-1)*steps//2 quadratic term and integer division) and rewoo_cost (two calls; steps==0 is (0,0,0.0)), then cost_per_resolved_task.
Latency — percentile (nearest-rank, ceil(p/100*N), 1-indexed; p==0 → min), sequential_latency (sum), fits_budget (<=).
Decision — workflow_vs_agent: compute the budget, decide needs_agency, branch in the priority order of §10.

Run LAB_MODULE=solution pytest test_lab.py -v first to see green, then make your lab.py match. Finish by reading solution.py's main() output — it's the whole warmup in numbers.

13. Success criteria

You can derive and interpret $0.95^{10}\approx0.60$ with no reference.
You can state the step budget for a given $p$ and target, and name the three levers to extend it.
You can explain, from the formula, why ReAct is quadratic and ReWOO linear — and when you'd pick each.
You can compute $/resolved-task and say why it, not $/attempt, is the real cost.
You can run the workflow-vs-agent decision on a fresh task out loud.
All 22 lab tests pass under both lab and solution.

14. Interview Q&A

Q: An engineer proposes a 15-step autonomous agent where each step is "about 95% reliable." What's your first question? A: "So the whole thing succeeds $0.95^{15}\approx0.46$ of the time — worse than a coin flip. Can we cut steps, add verification, or gate the risky ones behind a checkpoint or a human? What's the required end-to-end success rate, and what's our step budget to hit it?" You've reframed a vibe as arithmetic and pointed at the three levers.

Q: When would you choose ReWOO over ReAct? A: When the plan is fairly predictable and cost/latency matter: ReWOO calls the LLM twice instead of once per step, so input tokens are linear instead of quadratic (multiple-× savings on long tasks), and independent tool steps can run in parallel. The cost is adaptivity — ReWOO commits to the plan before seeing observations, so on tasks that need mid-course correction, ReAct's interleaving wins. In practice I'd profile both with the cost model and pick per task, or use a plan-execute graph that can re-plan.

Q: Your agent's token bill is 4× the estimate. Where do you look first? A: The scratchpad. ReAct resends the whole history every step, so input tokens grow as $n^2/2$; a long or looping agent spends most of its tokens re-reading itself. I'd check the step count (is it looping — is max_steps too high?), the observation sizes (are tools dumping huge blobs into context?), and whether context compression or caching is on. Output tokens priced 3–5× input also means verbose reasoning is a real cost.

Q: Why is "the model proposes, the application executes" a security statement, not just an architecture one? A: Because the model's output is untrusted — it will follow instructions embedded in its input (prompt injection), so any tool call it emits might be adversarially induced. Putting validation, allow-lists, authorization, sandboxing, and human approval on the application side of that boundary is the only place those controls can actually hold; no prompt can make the generator trustworthy.

Q: A PM insists a fixed fetch→match→flag→notify pipeline be built as an "AI agent." Push back. A: The path is known and doesn't branch on runtime observations, so it's a workflow: hard-code the four steps for determinism, ~10× lower cost (no LLM re-planning a fixed pipeline), and easy debugging, and reserve an LLM call only for the genuinely fuzzy sub-step (parsing a messy PDF). Same outcome, far more reliable and cheaper — and I can show the numbers.

Q: How do you set max_steps? A: From the reliable step budget plus a small margin: it's a safety guard against loops, not a target. If the task legitimately needs more steps than the budget allows, that's a signal to add checkpoints/verification, not to raise the cap — raising it just lets a confused agent burn more tokens before failing.

15. References

Anthropic, Building Effective Agents (2024) — the "workflows vs agents" framing and "least-agentic that works." https://www.anthropic.com/research/building-effective-agents
Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022). https://arxiv.org/abs/2210.03629
Xu et al., ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models (2023). https://arxiv.org/abs/2305.18323
Kim et al., An LLM Compiler for Parallel Function Calling (LLMCompiler, 2023) — parallel plan-execute. https://arxiv.org/abs/2312.04511
Gregg, Systems Performance (2nd ed.) — percentiles, tail latency, why the mean lies.
Dean & Barroso, The Tail at Scale, CACM 2013 — the canonical tail-latency paper. https://research.google/pubs/the-tail-at-scale/
OpenAI Agents SDK docs (the Runner/agent-loop model). https://openai.github.io/openai-agents-python/
LangGraph docs (graph-structured agent/workflow control). https://langchain-ai.github.io/langgraph/

Agentic AI Engineer — Agent Infrastructure, Platforms & Production Systems