« Phase 00 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Staff Notes

Phase 00 — Core Contributor Notes: Trust Boundaries, Reliability & Cost Math

This is a mental-model phase, so the "real system" it mirrors is not one library — it is the doctrine that shaped a generation of agent frameworks plus the concrete knobs those frameworks expose that encode exactly the four numbers from the lab. A maintainer's-eye view here means: where does n_{\max} live in real code, where does success_rate actually get computed, and where does the ReAct quadratic show up on a real bill. This document maps our stdlib miniature onto the systems that ship it.

The step budget is a real config key

Our max_autonomous_steps returns \(\lfloor \ln T/\ln p\rfloor\). No production framework computes that from reliability — but every one of them ships the guard it implies, because an unbounded agent loop over a stochastic oracle is an incident waiting to happen.

• LangGraph exposes recursion_limit, passed in the run config (config={"recursion_limit": N}). It bounds the number of super-steps (node executions) a graph will take before raising a recursion/GraphRecursionError. The documented default is on the order of 25. It is deliberately not a reliability computation — it is a blunt loop-guard whose value you set from exactly the reasoning this phase teaches: a confused graph that keeps re-entering a node should die, not spin. The maintainer's point is subtle: recursion_limit counts graph steps, which may not equal LLM turns one-to-one (a single super-step can fan out multiple nodes), so mapping our clean n onto it requires knowing your graph's shape.
• OpenAI Agents SDK exposes max_turns on Runner.run(...). A "turn" is one iteration of the agent loop — one model call plus the tool executions it triggers — and exceeding the cap raises a max-turns-exceeded error. The documented default is small (about 10). This is the closest real analog to our step budget: it is the "give up before you loop forever" guard, and setting it is a reliability-and-cost decision, not a target.

The gotcha every committer learns: these caps are safety guards, not step budgets. Raising recursion_limit or max_turns "just in case the agent needs more room" does not make the task more likely to succeed — it lets a confused agent burn more tokens before failing. The correct response to "it hit the cap" is usually fewer, verified steps, not a higher cap.

Where success_rate and $/resolved-task actually come from

The lab's cost_per_resolved_task = unit_cost / success_rate needs two real inputs, and in production they come from two different systems.

The numerator — per-request tokens, cost, and latency — is surfaced by the LLM gateway / observability layer, not computed by you:

• LiteLLM normalizes provider responses so every call carries a usage block (prompt/completion/total tokens) and computes a dollar cost from a maintained per-model price map, exposed on the response (a response_cost hidden param) and its callbacks. As a proxy it aggregates spend per key/user/model.
• Helicone sits as a logging proxy and records per-request tokens, cost, and latency — including time-to-first-token — and renders latency percentiles in its dashboards, which is where p95/p99 stops being theoretical.
• LangSmith traces each run/step with token counts, latency, and cost, and lets you roll those up per trace.

The denominator — success_rate — comes from an eval harness, and this is the part beginners skip. You attach an evaluator (an exact-match check, a rubric, an LLM-as-judge, a unit test as in SWE-bench-style "resolved" scoring) to a dataset of tasks; the harness runs the agent over every example, scores each, and reports the fraction that passed. That fraction is success_rate. LangSmith's evaluation flow, OpenAI Evals, and bespoke pytest-driven harnesses all follow the same shape: dataset × agent × scorer → aggregate score. The maintainer's insight is that the numerator and denominator come from different subsystems, so nobody computes $/resolved-task unless someone deliberately joins gateway cost telemetry to eval outcomes. That join is the metric this phase tells you to build.

The doctrine: Building Effective Agents and why defaults look the way they do

The workflow-vs-agent framing in this phase is not folklore; it traces to Anthropic's Building Effective Agents, which draws the sharp line: workflows are systems where LLMs and tools are orchestrated through predefined code paths; agents are systems where the LLM dynamically directs its own process and tool usage. Its governing advice — find the simplest thing that works and only add complexity (and autonomy) when the task demonstrably needs it — is the "least-agentic that works" rule the lab's workflow_vs_agent encodes.

That post also catalogs the composable patterns that framework APIs now mirror: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer (the critic loop of Phase 07). A committer reads current framework design — graph nodes you wire explicitly, first-class "hand off to a sub-agent," structured guardrail hooks — as the ecosystem converging on that doctrine: make the workflow composition easy and explicit, and make full autonomy an opt-in you reach for on purpose. The pointed critique in the same doctrine — that heavy framework abstraction can obscure the underlying prompt-and-loop and make debugging harder — is why this track has you build the raw loop in Phase 01 before touching a framework.

The papers behind the quadratic-vs-linear reality

Our react_cost and rewoo_cost are direct encodings of two papers. ReAct (Yao et al., 2022) established the interleaved thought→action→observation loop; because each step re-feeds the growing scratchpad, its context — and cost — grows super-linearly in steps, the \(\Theta(n^2)\) we derive. ReWOO (Xu et al., 2023) — Reasoning WithOut Observation — decouples reasoning from observation with an upfront plan using variable substitution and a final solver, calling the model a fixed number of times; the paper reports substantial token-efficiency gains on multi-step benchmarks (multiple-fold on tasks like HotpotQA) for exactly the linear-vs-quadratic reason. When you profile a real ReAct agent and watch input tokens balloon, you are watching Figure-N of that dynamic in your own gateway dashboard.

What our stdlib miniature deliberately simplifies

Being honest about the toy is a maintainer's habit. The lab trades fidelity for the shape of each relationship, and the shape is what does not change. Specifically it assumes away:

• Independent steps. Real steps are correlated: a good plan makes later steps easier (raising effective \(p\)), a bad early step poisons everything after (lowering it). p^n is the pessimistic first model, not a measurement — the real number comes from an eval, not the formula.
• A real tokenizer. The cost functions count abstract token units; production cost comes from the model's actual tokenizer (byte-pair encoding with model-specific vocab, Phase 04), where whitespace, punctuation, and non-English text change counts non-trivially.
• Clean latency. percentile over a fixed sample ignores queueing, network variance, cold caches, provider-side batching, and retries — the very things that fatten a real p99. Nearest-rank on 20 in-memory samples is a teaching device; a real SLO is measured over a rolling window of live traffic.
• Static prices. unit_cost is a constant; real per-token prices change, differ 3–5× between input and output, and are cut by prompt caching (Phase 14), which alone can reorder a ReAct-vs-ReWOO cost comparison.

The stdlib version is right about the direction and curvature of every relationship — geometric reliability decay, quadratic vs linear tokens, tail-heavy latency, reliability-weighted cost — and deliberately silent about the constants. That is the correct division of labor: this phase makes you fluent in the shapes so that when LangGraph, LiteLLM, or a LangSmith eval hands you the constants, you already know what curve they land on.