Planning vs Execution Models

Phase 11 · Document 06 · AI Coding Platforms Prev: 05 — Autocomplete Models · Up: Phase 11 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

This doc makes explicit the multi-model architecture that the latency tiers (00) and autocomplete (05) implied: a serious coding platform routes different sub-tasks to different models — a tiny FIM model for autocomplete, a strong reasoning model for planning, a code-specialized model for execution/editing, and a fast model for applying (04). Using one model for everything means either laggy autocomplete or a weak agent and a blown budget. Model routing (Phase 8.05, Phase 5.09) — picking the right model per sub-task — is the economic and quality lever of a coding platform, and the planning-vs-execution split is its sharpest example.

2. Core Concept

Plain-English primer: split the work, route to the right model

A coding request decomposes into sub-tasks with different requirements — and the platform routes each to a model suited to it. The recurring split:

• Planning ("how should I solve this across the codebase?") — needs strong reasoning + long context: decompose the task, decide which files/functions to change, sequence the steps. A frontier reasoning model (Phase 5.04). Done less often (once per task), so its cost/latency is tolerable.
• Execution / editing ("write the actual change to this function") — needs good code generation + instruction-following in a bounded scope. A code-specialized or strong instruct model (Phase 5.03). Done many times per task.
• Applying ("land this edit precisely") — needs exactness + speed, not reasoning. A small fast apply model (04).
• Autocomplete — needs extreme speed. A tiny FIM model (05).

TASK → PLAN (frontier reasoning, long ctx, 1×) → for each step: EXECUTE/EDIT (code model, N×)
     → APPLY (fast apply model [04]) → VERIFY (tests [Phase 10.06])
AUTOCOMPLETE (tiny FIM [05]) runs continuously, on a totally separate path

This is planner–executor (Phase 10.03) realized as a multi-model system: the planner and the executor can be different models, chosen for their job.

Why split (and not just use the best model for everything)

The cost-quality-latency triangle (Phase 5.09) makes one-model-for-all a bad point on every axis:

• Autocomplete can't wait for a frontier model (latency); planning needs more than a 7B (quality).
• Most tokens are "easy" (edits, applies, completions) — routing them to cheaper/faster models while reserving the frontier model for planning gives a far better blended cost/quality/latency than any single model (the difficulty-routing insight, Phase 5.09/Phase 8.05).
• Specialization beats generality per sub-task — a FIM model autocompletes better than a chat model; an apply model lands edits better than a reasoning model.

The routing decision

Routing picks the model per request by sub-task × constraints (Phase 8.05):

The router also considers context size (long context → a model that supports it), capability (tools/structured output for agent steps), cost budget, and BYOK/provider (07). It's exactly the Phase 8.05 routing engine (filter → select) applied inside the IDE.

Planner ≠ executor: a concrete pattern

The most valuable split in agent mode: a strong planner produces a plan/spec (which files, what changes, in what order), and a cheaper executor carries out each step (edit + apply + verify), escalating back to the planner if it gets stuck (Phase 10.03). This keeps the expensive reasoning model's usage to once per task while the high-frequency execution runs on cheaper models — big blended-cost wins on multi-step tasks.

Single-model reality check

Not every platform splits aggressively — a single strong model can do plan+execute+apply for agent mode, and many do (simplicity). But autocomplete is always separate (latency), and the apply step is often a separate fast model (apply-rate, 04). So even "single-model" platforms are really at least two-tier. The decision is how granular to route, traded against orchestration complexity (the Phase 10.03 "least-agentic/simplest that works" principle).

Where this lives

Routing is implemented via the platform's gateway/router (07, Phase 8), often with BYOK so users bring their own keys/providers. Evaluation (08) tells you whether a cheaper model is "good enough" for a tier (e.g., does the cheap edit model keep task-resolution high?).

3. Mental Model

   one coding request → MANY sub-tasks, each routed to the RIGHT model (multi-model architecture)
     AUTOCOMPLETE → tiny FIM (latency-extreme, often local) [05]   ← always its own tier
     EDIT (small) → fast code/instruct model
     APPLY        → small fast apply model (exactness+speed) [04]
     CHAT         → strong reasoning model
     AGENT: PLAN (frontier reasoning, long ctx, 1×) [5.04] → EXECUTE step (code model, N×) [5.03] → apply → verify [10.06]

   WHY split: cost-quality-latency [5.09] — one model is a bad point on every axis;
     most tokens are EASY → route easy→cheap/fast, reserve frontier for PLANNING (difficulty routing [8.05])
   ROUTER = filter (capability/context/cost) → select (sub-task) [8.05], via gateway/BYOK [07]
   even "single-model" platforms are ≥2-tier (autocomplete + apply separate); route as granularly as pays

Mnemonic: route each sub-task to the model built for it — tiny FIM for autocomplete, frontier for planning, code model for editing, fast model for applying. Most tokens are easy; reserve the expensive model for planning. It's Phase 8.05 routing inside the IDE.

4. Hitchhiker's Guide

What to look for first: is autocomplete a separate tier (it must be), and is the expensive reasoning model reserved for planning (not every step)? Those two choices drive most of the cost/latency.

What to ignore at first: maximal granular routing across many models. Start with autocomplete (FIM) + one strong model for chat/edit/agent + a fast apply model; split further only where eval shows it pays.

What misleads beginners:

• One model for everything. Laggy autocomplete or weak agent + blown budget — the latency tiers forbid it (00/05).
• Frontier model on every agent step. Planning needs it; execution/apply usually don't — reserve it for planning (Phase 5.09).
• Over-routing. Too many models = orchestration complexity + inconsistency; route only where it pays (Phase 10.03).
• Routing without eval. "Cheaper model is fine" must be measured (task-resolution/apply-rate hold?) (08).
• Ignoring capability/context in routing. An agent step needs tools/long-context; route to a capable model (Phase 8.05).

How experts reason: they decompose by sub-task (autocomplete/edit/apply/chat/plan/execute), route each to a fit model (latency-extreme→FIM, quality+context→frontier planner, code→executor, speed→apply), reserve the frontier model for planning (once/task), and measure that cheaper tiers hold quality (08). They implement it as a router/gateway (07/Phase 8.05) and keep routing as granular as pays (not more).

What matters in production: blended cost/quality/latency across tiers, autocomplete latency, planner usage frequency (cost), task-resolution with the chosen executor/apply models, and routing correctness (right model per sub-task/capability).

How to debug/verify: if cost is high → is the frontier model running on every step (should be planning-only)? if autocomplete lags → it's not a separate fast tier; if quality drops → a cheaper tier failed its job (eval it, 08).

Questions to ask: is autocomplete a separate tier? is the frontier model planning-only? what models per sub-task? is "cheaper is fine" eval-backed? is routing capability/context/cost-aware (Phase 8.05)?

What silently gets expensive/unreliable: one-model-for-all (latency/quality/cost all bad), frontier-on-every-step (cost blowup), over-routing (complexity/inconsistency), and unmeasured cheap tiers (silent quality drops).

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• A coding platform is multi-model: route each sub-task (autocomplete/edit/apply/chat/plan/execute) to a fit model (00).
• Autocomplete is always its own tier (tiny FIM, latency-extreme, 05); the apply step is often a separate fast model (04) — so even "single-model" platforms are ≥2-tier.
• Reserve the frontier reasoning model for planning (once per task); run high-frequency execution/apply on cheaper/faster models — big blended-cost wins (Phase 5.09).
• One model for everything is a bad point on every axis (cost, quality, latency).
• Planner ≠ executor: a strong planner produces the plan, a cheaper executor does the steps, escalating when stuck (Phase 10.03).
• Routing = filter (capability/context/cost) → select (sub-task) — the Phase 8.05 engine inside the IDE, via a gateway/BYOK (07).
• Route as granularly as pays — more models add orchestration complexity/inconsistency (Phase 10.03).
• "Cheaper tier is good enough" must be eval-backed (task-resolution/apply-rate hold?) (08).

9. Observations from Real Systems

• Cursor runs a fast autocomplete model, a dedicated apply model (04), and lets users pick strong models for chat/agent — a clear multi-model/per-task design (00).
• Agent modes (Cursor/Claude Code/Copilot) increasingly use planner–executor: a strong model plans, cheaper models execute steps (Phase 10.03).
• The biggest cost lever is keeping the frontier model to planning and routing the many execution/apply/autocomplete tokens to cheaper/faster models (Phase 5.09).
• Routing is the gateway's job — coding platforms embed a router/BYOK to pick models per sub-task (07, Phase 8.05).
• Eval gates the routing — teams verify a cheaper executor/apply model holds task-resolution before shipping it (08).

10. Common Misconceptions

11. Engineering Decision Framework

ROUTE MODELS IN A CODING PLATFORM:
 1. SEPARATE TIERS by sub-task × constraint:
      autocomplete → tiny FIM (latency-extreme, local) [05]
      apply        → small fast apply model (exactness+speed) [04]
      edit (small) → fast code/instruct model [5.03]
      chat         → strong reasoning model
      AGENT: PLAN → frontier reasoning + long ctx (1×) [5.04]; EXECUTE step → code model (N×) [5.03]
 2. RESERVE the frontier model for PLANNING; route the many easy tokens to cheaper/faster (difficulty routing [5.09]).
 3. IMPLEMENT via a router/gateway (filter capability/context/cost → select) [8.05], with BYOK [07].
 4. GRANULARITY: route as granularly as PAYS; avoid orchestration sprawl (least-agentic/simplest [10.03]).
 5. EVAL each tier: does the cheaper executor/apply model hold task-resolution/apply-rate? [08]

12. Hands-On Lab

Goal

Build a task router that sends sub-tasks to different models, and show that a planner–executor split (frontier plans, cheap executes) beats single-model on blended cost at equal task-resolution.

Prerequisites

• Access to ≥2 models (a strong + a cheap, via API or Phase 8); the code agent from Phase 10.06; a few multi-step coding tasks with tests.

Steps

1. Router: implement a route(subtask, constraints) that maps autocomplete→tiny/FIM, edit→cheap code model, apply→fast model, plan→strong model (Phase 8.05).
2. Single-model baseline: run the multi-step coding tasks with one strong model for plan+execute+apply; record task-resolution, tokens, cost, latency.
3. Planner–executor: use the strong model to plan once, then a cheaper model to execute each step (+ a fast apply, 04); record the same metrics (Phase 10.03).
4. Compare: show the split achieves similar task-resolution at lower blended cost/latency (the frontier model ran once, not per step) — or find where the cheap executor drops quality and adjust (Phase 5.09).
5. Autocomplete tier: confirm the autocomplete path uses the tiny FIM model and meets its latency budget — separate from the agent path (05).
6. Eval the tier: verify the cheap executor holds task-resolution; if not, route those steps up (08).

Expected output

A working sub-task router; a single-model vs planner–executor comparison (task-resolution vs cost/latency) demonstrating the blended-cost win; and a confirmation that autocomplete is a separate fast tier.

Debugging tips

• Planner–executor resolves fewer tasks → the cheap executor is under-powered for some steps; route harder steps up, or strengthen the plan.
• No cost win → the frontier model is still running per step (it should plan once).

Extension task

Add difficulty-based routing within execution (easy edits → cheapest, hard → stronger) and measure the additional blended-cost gain (Phase 5.09).

Production extension

Implement routing in the platform's gateway/BYOK (07/Phase 8.05); track blended cost/quality/latency per tier and gate model changes on eval (08).

What to measure

Task-resolution, tokens/cost/latency: single-model vs planner–executor; frontier-model call frequency; autocomplete latency; per-tier quality.

Deliverables

• A sub-task router (autocomplete/edit/apply/plan).
• A single-model vs planner–executor comparison (resolution vs blended cost/latency).
• A note on routing granularity + eval-backed tier choices.

13. Verification Questions

Basic

1. Why is a coding platform multi-model rather than single-model?
2. What are the planning, execution, apply, and autocomplete tiers, and their model profiles?
3. Why reserve the frontier model for planning?

Applied 4. How does planner–executor lower blended cost on multi-step tasks? 5. What factors (beyond cost) does the router consider per sub-task?

Debugging 6. Cost is high in agent mode. What routing mistake is likely? 7. A cheaper executor model dropped task-resolution. What do you do?

System design 8. Design the model-routing layer for a coding platform: tiers, planner–executor, gateway/BYOK, eval gates.

Startup / product 9. Why is model routing (not the base model) often the biggest margin and UX lever for a coding product?

14. Takeaways

1. A coding platform is multi-model — route each sub-task to a fit model (FIM autocomplete, frontier planner, code executor, fast apply).
2. Autocomplete (and usually apply) are always separate tiers — even "single-model" platforms are ≥2-tier.
3. Reserve the frontier model for planning (once/task); route the many easy tokens to cheaper/faster — the blended-cost win (Phase 5.09).
4. Planner ≠ executor (Phase 10.03); routing = filter→select via a gateway/BYOK (07/Phase 8.05).
5. Route as granularly as pays, and eval each tier (08) — model routing is the platform's core cost/quality/latency lever.

15. Artifact Checklist

• A sub-task router (autocomplete/edit/apply/plan/execute).
• A single-model vs planner–executor blended-cost comparison.
• A confirmation autocomplete is a separate fast tier.
• An eval-backed "cheaper tier is good enough" decision.
• A note on routing granularity (route as granularly as pays).

Up: Phase 11 Index · Next: 07 — BYOK and Provider Routing