« Phase 07 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor

Phase 07 — Staff Engineer Notes: Multi-Agent Orchestration

The Deep Dive taught the mechanism, the Principal doc taught the cost, the Core Contributor doc taught the real systems. This doc is about the thing none of those can hand you: judgment. Wiring up agents is a weekend. Being the person a team trusts to decide whether to go multi-agent at all — and to say no when the diagram is seductive — is the skill they are actually hiring and promoting for. This phase is unusual in the curriculum because its senior signal is largely a refusal.

The line between the two engineers

Watch two engineers get the same prompt: "design a multi-agent system for X."

The first draws five agents with clever names, arrows between all of them, and starts naming the frameworks. They have mistaken the artifact (a multi-agent diagram) for the goal (a task getting done reliably at an acceptable cost). Every arrow they draw is a seam, and every seam is a place a loop can start, an error can propagate, and a bill can grow — but the diagram hides all of that behind how sophisticated it looks.

The second asks a question first: "Before I add agents — could this be one agent with a couple of tools? If not, where is the one place a second agent earns its cost?" That question is the entire seniority signal, and the answer, nine times out of ten, is a critic — not more hands, a second opinion. Because the only lever that moves the reliability math is verification (1-(1-p)^{k+1}), routing agents around each other spreads the same unreliability across a bigger bill. The person who reaches for the swarm is demonstrating tool knowledge; the person who prices the swarm and declines it is demonstrating ownership. Organizations pay for the second one because the first one's designs are what the second one gets paged to fix.

The decision framework

Say this out loud in a review and you sound like the person who should own the call:

  1. Default: one agent with good tools. Start here always. It is the cheapest, lowest-latency, most-debuggable, single-owner design. Most "multi-agent" requirements dissolve into a well-tooled single agent under one question: what does the second agent buy?
  2. If the order is fixed and doesn't branch on runtime observations → it's a workflow. Hard-code the pipeline and get determinism for free (Phase 00). A model-driven supervisor over a fixed A→B→C is paying for adaptivity you are not using.
  3. If you need reliability → add exactly one critic. A separate critic with a different prompt is the single highest-value second agent. Bound it with max_rounds. This is the only addition that changes the reliability arithmetic, and it is cheap.
  4. If the path depends on runtime state → supervisor / orchestrator-worker. One router, N focused workers, routing off a blackboard, bounded by max_turns. This is the production default the moment you genuinely need adaptivity.
  5. If the sub-tasks are genuinely independent and numerous → parallel fan-out. The only topology that cuts latency. Worth the N× tokens when the work is embarrassingly parallel.
  6. Swarm / peer handoff and full group-chat are last resorts. Expressive, impressive, and the hardest to bound, cost, and debug. Reach for them only when routing genuinely cannot be centralized, and know you are signing up for the ping-pong failure mode.

The framework is a ladder you climb only as far as the task forces you, and you narrate why you stopped where you did. Climbing higher than the task requires is the most common architecture-review tell for a junior.

Code-review red flags

These are the things that make a reviewer stop and ask a hard question — and the things you scan for when you review someone else's multi-agent PR:

  • No termination bound on a loop. A supervisor loop or critique loop with no max_turns / max_rounds is not "usually fine" — it is an outage waiting for the day an agent misbehaves, which is exactly the day you cannot rely on it to stop itself. Every loop, bounded, returning a partial result with a stopped_reason. Non-negotiable.
  • Unbounded / uncentralized handoffs. Peer agents that can each hand off to any other with no hop budget and no progress check. Ask: whose turn is it, when does it stop, who owns the answer? If the design has no structural answer, it will ping-pong in production.
  • The whole transcript handed to every agent. Context explosion — cost grows with agents × turns and quality drops (lost-in-the-middle). Least-context handoffs are the default; a handoff carrying the full history is a bug, not a convenience.
  • Shared mutable state written by concurrent agents with no reducer. Last-writer-wins on a shared key is a race. Real frameworks solve it with channels/reducers (LangGraph); a hand-rolled system needs per-agent keys or an explicit merge.
  • A supervisor with no critic. Routing is not verification. This design distributes the same unreliability across more agents at higher cost and calls it an improvement.
  • An unvalidated handoff target. Routing to an agent that might not exist, failing silently. The target must be validated at construction (our make_handoff, the Agents SDK's declared handoffs).
  • No per-agent authz story. Each agent is a separate principal with a separate blast radius; a writer agent with database-write tools is not the same risk as a read-only researcher. Least privilege is per-agent.

War stories

  • The five-agent cathedral for a three-step task. Fetch → summarize → check shipped as five chatting agents with cute names. It was a two-worker pipeline plus one critic; the other three agents were pure cost and latency. Collapsing it cut the bill and raised reliability — fewer seams meant fewer places for an error to propagate. Fewer agents was the upgrade.
  • The critic that never approved. A picky reviewer and a stubborn writer with no max_rounds ping-ponged until someone noticed the job had been running for an hour. The fix was not a smarter writer; it was the bound that turns "hang forever" into "best draft, not approved" — and then a clearer critic so the best draft was usually good.
  • The context-explosion bill. Every agent was handed the full running transcript "for context." Token cost grew with agents × turns and quality dropped. Least-context handoffs fixed the bill and the quality in one change. The 15× multiplier is real and it compounds.
  • The hallucinated source nobody owned. The researcher invented a citation, the writer cited it, the weak critic approved it — responsibility diffusion, the bystander effect in software. A real fact-check critic with the final say and the authority to reject caught it. Someone has to own correctness; "the system" is not a someone.

The signal interviewers and reviews listen for

They are not listening for "I know AutoGen and LangGraph." That is table stakes — anyone can pip install. They are listening for whether you default to the simplest thing that works and can price the alternative. The exact senior move: given a task, argue for fewer agents, show the $/resolved-task arithmetic, name the one place a second agent (a critic) earns its cost, and reserve the elaborate topology for the case that genuinely demands it. The most senior sentence you can say in a multi-agent design review is frequently "this should be one agent with good tools" — backed by the numbers. Bonus signal: recognizing that multi-agent is a distributed-systems problem wearing an AI hat — ordering, idempotency, termination, cost attribution across actors — because that is the part that actually pages you, and it is why the bus uses a deterministic counter, not a clock.

Closing takeaways

  1. Default to one agent with good tools. Climb the topology ladder only as far as the task forces you, and narrate why you stopped.
  2. The critic is the only free lunch — and it isn't free. Verification (1-(1-p)^{k+1}) is the one thing that moves reliability. Routing does not. Add exactly one critic, bounded.
  3. Every loop is bounded. No max_turns / max_rounds is not a style choice; it is an outage with a delayed fuse. Return a partial result with a stopped_reason, never a hang.
  4. Least-context by default. A handoff carries the task plus one prior note, not the transcript. Context is a budget you spend, not a firehose you open.
  5. Boring topologies survive contact with a budget. A supervisor, two or three focused workers, one critic with the final say, a hard budget on every loop. Boring is auditable; auditable is reliable; reliable is what pays.
  6. The framework is the ergonomics; the four mechanisms are the engineering. Bus, roles, typed handoff, bounded verification. Know those and you can pick any framework, defend the choice, and debug it when it will not terminate.