« Phase 00 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor

Phase 00 — Staff Engineer Notes: Trust Boundaries, Reliability & Cost Math

Nobody gets promoted for wiring a framework's Runner to a tool. This phase is where the seniority actually lives, because none of it requires a GPU, an API key, or a line of framework code — it requires judgment applied to four numbers. What separates someone who uses agents from someone trusted to own the platform is that the owner runs this arithmetic before the build, states the tradeoffs out loud, and defends the boring answer when the room wants the exciting one.

The decisions you own here

A staff engineer owns four decisions in this phase, and owns them by number, not by vibe:

  1. Workflow vs agent vs agent-with-guardrails. You run the least-agentic-that-works selection and you are willing to be the person who says "this shouldn't be an agent." That is not timidity; it is the arithmetic winning.
  2. Where to checkpoint. Given \(n_{\max}=\lfloor \ln T/\ln p\rfloor\), you place a verified anchor before the budget runs out, and you can point at the exact step where compounding drops you below target.
  3. What SLO. You pick the percentile (p95/p99), you set it at the tail, and you size the sequential chain by summing tail latencies — never the mean.
  4. What step budget / cap. You set max_turns / recursion_limit from the reliable budget plus a small margin, and you treat it as a loop-guard, not a target to grow.

A decision framework for reaching for autonomy

Ask, in order, and stop at the first "no":

  • Does the path branch on runtime observations? If no → workflow. Hard-code it. Determinism, ~10× cost reduction, free durability, trivial debugging. Reserve one LLM call for the single genuinely fuzzy sub-step if there is one.
  • If yes, does the required step count fit inside \(n_{\max}\)? If yes → a plain agent is fine.
  • If it exceeds the budget → agent-with-guardrails: checkpoints, a critic, an eval gate, or a human on the high-blast-radius actions. This is where most real production agents land, and knowing that is itself a signal.

The tell of seniority is running this out loud on a new task in under a minute and landing on the unglamorous answer without flinching.

Code-review red flags

These get a comment from me every time:

  • max_steps / max_turns set high "just in case." A high cap does not add capability; it lets a confused agent burn more tokens before failing. High-cap-as-safety-margin means the author has not internalized \(p^n\).
  • Mean latency quoted in a capacity or SLO review. The mean is a liar for a right-skewed distribution. If the number isn't p95/p99, it isn't an answer.
  • Agents compared on $/attempt. The business pays per result. Compare on $/resolved-task = unit_cost/success_rate or you are optimizing the wrong quantity — a "cheaper" agent that fails more can cost more per good answer.
  • A 15-step autonomous chain at "~95% per step." \(0.95^{15}\approx0.46\) — worse than a coin flip end to end. Ship it and it will "work in the demo" and page you in production.
  • "The prompt will handle authz / safety / the edge case." The prompt is a suggestion to an untrusted stochastic generator that will follow instructions injected into its input. Every safety property must live in code on the executor side of the trust boundary, or it does not exist.
  • A ReAct loop with big observations and no context management. The \(n^2/2\) scratchpad-resend term is a cost bomb; unbounded tool output dumped into context is the fuse.

Production war stories

  • The ReAct quadratic that 10×'d the bill. A research agent shipped at ~6 steps in testing, then real tasks ran 18–20 steps. Because ReAct resends the whole scratchpad every step, input tokens grow as \(n^2/2\): tripling the steps roughly nine-times'd the growing term, and most of the monthly spend turned out to be the model re-reading its own history. The fix was not a cheaper model — it was context compression and a hard step budget, i.e. changing the shape, and profiling $/resolved-task per route so it couldn't hide again.
  • The "agent" that was really a workflow, and failed quarter-end. An invoice-reconciliation "AI agent" had a fixed path — fetch → match → flag → notify — but was built as an autonomous loop because "agent" was the mandate. At quarter-end volume it hit nondeterministic ordering and looped on ambiguous matches; a hard-coded workflow with one LLM call for the messy-PDF sub-step would have been deterministic, ~10× cheaper, and debuggable. The postmortem line was: it was never an agent; the autonomy was the bug.
  • The p50 dashboard that lied. A service reported a comfortable 0.6s median and a green SLO board while a steady 1-in-20 of requests blew a 2s budget — the p50 dashboard simply could not see the tail the users lived in. Switching the SLO panel to p95/p99 histograms surfaced the violation that had been quietly generating support tickets for weeks.

The signal an interviewer or architecture review listens for

When someone describes a proposed agent, the reviewer is listening for whether the candidate reflexively converts vibe to arithmetic. "Fifteen steps, about 95% each" should trigger, unprompted: "so \(0.95^{15}\approx0.46\) end to end — what's the required success rate, what's our step budget, and where do we checkpoint or add a human?" The signal is not knowing the formula; it is reaching for it first. The second signal is willingness to recommend less autonomy — the senior move is often removing agency, not adding it. The third is talking cost as $/resolved-task and latency as p95, because those are the numbers the business and the pager actually run on. A candidate who says "it shouldn't be an agent, here's why in numbers" has demonstrated more seniority than one who describes an elaborate autonomous graph.

Closing takeaways

  • Draw the trust boundary first; it's the last thing you compromise. The model proposes, the application executes — and every safety property lives in your code, never in the prompt.
  • Reliability is a budget you spend. \(p^n\) decays geometrically; every checkpoint, critic, eval gate, and human gate in later phases exists to buy steps back. Know your \(n_{\max}\) cold.
  • Cost and latency are engineered before the build. ReAct's quadratic vs ReWOO's linear, output priced 3–5× input, the tail as the SLO — you predict these, you don't discover them from the bill.
  • $/resolved-task, not $/attempt, is the real cost — and a quiet reliability regression can invert your margin while every per-request dashboard stays green.
  • The most senior sentence in an agent review is often "this shouldn't be an agent." Least-agentic that works is the default, and reaching for autonomy is the exception you justify.

Internalize the framing that carries through the rest of this track: agent engineering is distributed systems with extra anxiety — an unreliable oracle in the loop, a stochastic component you can't unit-test to certainty, and a trust boundary an adversary is actively probing. The four numbers in this phase are how you turn that anxiety into a design you can defend.