« Phase 07 README · Warmup · Hitchhiker's · Deep Dive · Core Contributor · Staff Notes

Phase 07 — Principal Deep Dive: Multi-Agent Orchestration

The Deep Dive traced the mechanism. This doc is the argument you have to win in an architecture review: should this system be multi-agent at all, and if so, in what topology, and what does that cost. Multi-agent is the single most over-reached-for pattern in this field, and the reason is that the diagram is free and the bill is not. A principal's job is to price the diagram before it ships.

Where multi-agent sits in a real platform

A production agent platform is layers: a gateway (auth, rate limits, tenancy), a model router (Phase 24), tool/MCP integration (Phase 03), memory/context (Phase 04), durability (Phase 08), eval/observability (Phase 11), and — orthogonal to all of them — an orchestration layer that decides how many policies run and in what shape. The critical framing: orchestration is a composition concern, not a new primitive. Each agent is still the Phase 01 loop; the supervisor is a loop over policies; the bus is a channel; the budget is Phase 00's step budget applied per loop. Everything else in the platform plugs into the same bus — durability checkpoints the blackboard, tracing reads the log, authz gates each agent's tools. If your multi-agent design needs a new kind of control that single-agent didn't, you have probably designed it wrong; multi-agent needs the old controls applied at more seams, because it has more seams.

The topology decision and its cost envelope

There is no "multi-agent architecture"; there is a family of topologies, each with a different scaling, latency, and cost profile. The four that matter, priced:

• Sequential pipeline (A → B → C). Fixed order, no runtime branching. Latency is the sum of the hops; cost is the sum of each agent's tokens. This is the cheapest and most predictable shape — and the tell is that it is not really "agentic": if the order never branches on an observation, it is a workflow (Phase 00) and you should hard-code it and buy determinism for free. Most "multi-agent" systems that ship are this wearing a costume.
• Supervisor / orchestrator-worker. One router, N specialists, a decision each turn. This is the production default when the path depends on runtime state ("the researcher says it needs more data, route back"). Cost is the workers' tokens plus a supervisor LLM call per turn — the routing tax. Latency is sequential (one worker at a time) unless the supervisor fans out. It buys adaptivity; you pay one extra inference per routing decision for it.
• Parallel fan-out / map-reduce. The supervisor hands the same class of sub-task to N workers at once, then a reduce/join step (often a critic) combines them. This is the only topology that reduces wall-clock latency — N independent sub-tasks finish in ~1× latency instead of N× — at the cost of N× tokens and a join barrier that is a deadlock risk if a branch never returns. Reach for it only when the sub-tasks are genuinely independent (summarize 50 docs, probe 10 sources).
• Swarm / peer handoff. Agents transfer control to each other directly, no central router (OpenAI Swarm / Agents SDK). Flexible and expressive; the hardest to bound. "Whose turn is it? when does it stop? who owns the answer?" have no structural answer, so infinite handoff ping-pong is the default failure. Use it where the routing genuinely can't be centralized; otherwise a supervisor is easier to reason about, cost, and debug.
• Group-chat / round-robin. All agents share one conversation; a manager (or a fixed rotation) picks the next speaker (AutoGen). The cost trap is that if every agent sees the whole conversation, each turn's prompt grows with the transcript — cost scales with agents × turns, quadratically in a long chat. Cheap to prototype, expensive to run.

The one number to internalize: N agents that each make an LLM call is N× the tokens and N× the failure surface of a single call, and if they run sequentially, ~N× the latency too. A three-agent pipeline wrapped in a two-round critic is 5–10× a single well-tooled agent on cost. Anthropic's own production multi-agent research system runs about 15× the tokens of a single chat and they are explicit that it is worth it only because research is embarrassingly parallel and high-value. That 15× is the honest anchor: multi-agent is not a reliability upgrade you get for free, it is a cost multiplier you spend deliberately to buy parallelism or verification.

Capacity, cost, and latency math

Make the tradeoff arithmetic, not vibes. Let a task decompose into a supervisor plus k worker invocations plus r critic rounds, each averaging T tokens at C dollars per token:

• Cost ≈ (1 + k + 2r) × T × C — the 1 is the supervisor's per-turn routing calls, the 2r is draft + critique per round. Compare against a single agent at ~T × C. The multiplier is the whole decision: if (1 + k + 2r) is 8, you are paying 8× for whatever reliability or parallelism the extra agents buy. Measure it against $/resolved-task (Phase 00), not per-call cost — a swarm that resolves more tasks per attempt can be cheaper per resolved task even at higher per-call cost.
• Latency. Sequential: sum of hop latencies — a 4-hop pipeline at 2s/hop is 8s wall clock. Parallel fan-out: max of the branch latencies plus the join — this is the only way N agents beat one. The routing supervisor adds a full inference of latency per turn, which is why chatty supervisor loops feel slow even when each worker is fast.
• Reliability. This is the only axis where more agents can help, and only via verification. A worker correct with probability p shipped bare is reliability p; a critic that catches bad drafts and forces k revisions is 1 - (1-p)^{k+1} — 0.9 → 0.99 at one revision. Routing does not move this number; a supervisor without a critic distributes the same p across more agents at higher cost. If your multi-agent justification is not "a separate critic lifts reliability" or "genuine parallelism cuts latency," you do not have a justification, you have a diagram.

Failure modes and blast radius

Multi-agent adds failure modes a single agent cannot have, and — the part people miss — it widens the blast radius of each one because errors cross agent boundaries as trusted inputs:

• Non-termination. A supervisor that never emits done, or a critic that never approves, loops until the budget fires — or forever, if there is no budget. Blast radius: unbounded spend, a hung request. Guard: max_turns / max_rounds on every loop, returning a partial result with a stopped_reason.
• Infinite handoff ping-pong. In a swarm, agent A hands to B hands to A. Blast radius: same as non-termination but harder to detect because "progress" (messages posted) hides "no convergence." Guard: a global hop budget plus a progress check (has the blackboard actually advanced?).
• Cascading / propagated error. The researcher hallucinates a source, the writer cites it as fact, the weak critic approves it. Blast radius: one bad output becomes every downstream agent's trusted premise — the error amplifies instead of being caught. Guard: a real critic with the final say, typed results, validation at each handoff.
• Responsibility diffusion. With many agents, no single one owns correctness — each assumes another will catch it (the bystander effect, in software). Guard: an explicit owner/critic role with the final verdict and a single defined termination authority.
• Cost blowup. A routing loop that "works" but takes 12 turns instead of 3 is not a correctness bug and so ships — then the monthly bill is 4×. Guard: cost is an SLO; alert on $/resolved-task, not just error rate.

Cross-cutting concerns

• Observability. A single agent's trace is a linear transcript; a multi-agent trace is a graph — per-agent spans, correlated by a run id, with the handoffs as edges. If you cannot answer "which agent burned the tokens" and "where did the loop start" from your traces, you cannot operate this system. This is why the phase builds the bus with a deterministic log: it is the trace.
• Tenancy and authz. Each agent is a separate authorization principal. A researcher agent with web tools and a writer agent with database write access have different blast radii; least-privilege is per-agent, and a handoff must not launder one agent's authority into another. Guardrails/content safety and access control are separate axes (Phase 24) — routing is not authorization.
• Durability. A crash mid-handoff must resume from the blackboard, not restart the whole run (Phase 08). This is why the blackboard is a serializable dict and the log is replayable.

When not to go multi-agent

Default to a single well-tooled agent. Add a critic the moment reliability needs it — that is the one cheap, high-value second agent. Go to a full topology only when the structure genuinely demands it: genuine parallelism (independent sub-tasks worth fanning out), genuine specialization + verification (a separate critic measurably lifts 1-(1-p)^{k+1}), or genuine role separation (distinct sub-jobs each more reliable focused than combined). Absent one of those three, "this should be one agent with good tools" is the correct architecture — the multi-agent twin of "this should be a workflow," and just as often right. The senior signal is not the elaborate diagram; it is the priced refusal to draw one.