AI Coding Platforms — Cursor-Style Architecture

Phase 11 · Document 00 · AI Coding Platforms Prev: Phase 10 — Agents and Tools · Up: Phase 11 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

AI coding tools — Cursor, GitHub Copilot, Windsurf, Claude Code, Zed AI — are the most-used LLM products by engineers and the clearest example of a complete LLM application: they combine everything in this curriculum — retrieval over a codebase (Phase 9), an agent loop with tools (Phase 10), multi-model routing (Phase 5.09), a gateway/BYOK (Phase 8), and tight latency/serving constraints (Phase 7) — wrapped in an IDE under brutal UX pressure (autocomplete must feel instant). Understanding their architecture makes you a far better user (you'll know why context matters and where they fail) and equips you to build coding assistants, internal IDE plugins, or code-review tools — a top product category. This overview is the map of Phase 11.

2. Core Concept

Plain-English primer: an IDE-wrapped RAG + agent over your code

A coding platform's job is to put the right code context in front of the right model at the right latency, then apply the result into the editor. Strip away the IDE chrome and it's the curriculum's pieces composed for code:

EDITOR (VSCode/JetBrains/own)          ← the surface: keystrokes, selection, diagnostics, diffs [01]
   │ capture context (current file, cursor, open tabs, errors, selection)
   ▼
CODEBASE INDEX  ← embeddings + symbol/AST graph over the repo [02,03]  (RAG for code [Phase 9])
   │ retrieve relevant code for the request
   ▼
CONTEXT ASSEMBLY  ← pack current file + retrieved chunks + diagnostics within the window [Phase 9.07]
   │
ROUTER  ← pick the model for the TASK + latency budget [06, Phase 5.09]
   ├ autocomplete → tiny fast FIM model (<~200ms)            [05]
   ├ inline edit  → instruction model + apply [04]
   ├ chat         → strong reasoning model
   └ agent        → strong tool-calling model in a loop      [Phase 10.06]
   ▼  (via a gateway/BYOK [07, Phase 8])
APPLY  ← turn model output into editor changes (diff/apply-patch), review before commit [04,10.05]

The central problem: context selection under a latency budget

The hard, recurring problem (what-happens §0): a real repo is millions of tokens; the window holds a fraction; and autocomplete has ~tens-of-ms budget while an agent edit can take a minute. So the platform must select the most useful context (current file + retrieved relevant code + diagnostics) fast enough for the task. This is RAG (Phase 9) plus retrieval-vs-stuffing (what-happens §3.4), specialized for code and squeezed by latency.

The five subsystems (the map of this phase)

1. The editor/extension surface (01) — how the tool plugs into VSCode/JetBrains: capture context, render completions/diffs, run commands.
2. Codebase indexing (02) — embeddings over code chunks for semantic retrieval (RAG for code).
3. Symbol search & AST (03) — structural retrieval (definitions, references, call/import graphs) that pure embeddings miss.
4. Inline edit & apply-patch (04) — turning model output into reliable edits (apply-rate; ties to Phase 10.06).
5. The multi-model engine — autocomplete models (05), planning-vs-execution routing (06), and BYOK/provider routing (07) — with evaluation tying it together (08).

The defining constraint: latency tiers

Coding platforms are unusual in LLM apps because different features have wildly different latency budgets, forcing a multi-model architecture (06):

You cannot serve all four with one model — autocomplete needs a different model and serving path than an agent. This latency-tiering is the architectural signature of the category.

It's a synthesis phase

Phase 11 doesn't introduce a new primitive — it assembles RAG (9), agents (10), routing/gateway (8), model selection (5), and serving (7) into a product. Read each doc as "how this curriculum concept specializes for code in an IDE."

3. Mental Model

   AI coding platform = IDE-wrapped RAG + agent over your code, latency-tiered

   EDITOR [01] → capture context → CODEBASE INDEX (embeddings [02] + symbol/AST graph [03]) →
   ASSEMBLE context (current file + retrieved + diagnostics, window-budgeted [9.07]) →
   ROUTER [06] by TASK × LATENCY → model (via gateway/BYOK [07/8]) → APPLY (diff/apply-patch, review) [04/10.05]

   CENTRAL PROBLEM: select the most useful context FAST enough (RAG [9] + retrieve-not-stuff, under latency)
   LATENCY TIERS force a MULTI-MODEL engine:
     autocomplete (tiny FIM, ~ms [05]) · edit (instruct+apply [04]) · chat (reasoning) · agent (tool-loop [10.06])

   SYNTHESIS phase: RAG(9) + agents(10) + routing/gateway(8) + model-selection(5) + serving(7), wrapped in an IDE

Mnemonic: a coding platform puts the right code context in front of the right model at the right latency, then applies the result. Context-selection-under-latency is the core problem; latency tiers force a multi-model engine; it's a synthesis of Phases 5/7/8/9/10.

4. Hitchhiker's Guide

What to look for first: how the platform does context selection (index + retrieval + what it puts in the prompt) and its latency tiers / model routing — those two define its quality and feel.

What to ignore at first: building your own from scratch. Start by understanding the subsystems via a small file-level assistant; the full platform is a large systems project.

What misleads beginners:

• Thinking it's "just a prompt to GPT." The value is context selection (indexing + retrieval) and the multi-model latency architecture, not the base prompt.
• One model for everything. Autocomplete and agent edits have ~1000× different latency budgets — they need different models/serving (05/06).
• Embeddings-only code retrieval. Code has structure (symbols, call graphs) that embeddings miss — combine with AST/symbol search (03).
• Auto-applying edits. Reliability (apply-rate) and review before commit matter — never silently mutate code (04, Phase 10.05).
• Ignoring privacy. Sending the whole codebase to a cloud model is a data-governance issue — indexing/BYOK choices matter (02/07).

How experts reason: they treat it as RAG + agents specialized for code, under a latency budget: invest in context selection (hybrid embeddings + symbol/AST), tier models by task (tiny FIM for autocomplete → strong agent for multi-file), make apply reliable (apply-rate), route via a gateway/BYOK, and evaluate on apply-rate / autocomplete-acceptance / task-resolution (08). They borrow the agent-safety model wholesale (Phase 10.05).

What matters in production: autocomplete latency/acceptance (p50/p95), retrieval relevance, apply-rate, task-resolution for agent mode, cost per task, and privacy of code/context (02/07).

How to debug/verify: trace a request through the subsystems — was the right context retrieved (02/03)? right model for the task (06)? did the edit apply (04)? is autocomplete within budget (05)?

Questions to ask: how is context selected (index + retrieval)? what models per latency tier? how reliable is apply (apply-rate)? is there review before commit? BYOK/privacy? how is it evaluated?

What silently gets expensive/unreliable: poor context selection (irrelevant/missing code → bad suggestions), one-model-for-all (laggy autocomplete or weak agent), low apply-rate (edits that don't land), and cloud-indexing the whole repo (privacy/cost).

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• A coding platform = codebase index + context collector + router + apply, wrapped in an editor — RAG (Phase 9) + agents (Phase 10) specialized for code.
• Context selection under a latency budget is the central problem — repos are huge, the window is small, autocomplete is ~ms (what-happens §3.4).
• Latency tiers force a multi-model architecture — tiny FIM model for autocomplete (05) → strong tool-calling model for agent mode (Phase 10.06); one model can't serve all.
• Code retrieval needs structure (symbols/AST), not just embeddings (03).
• Reliable apply (apply-rate) + review-before-commit are essential — never auto-mutate code (04, Phase 10.05).
• A gateway/BYOK handles provider routing, privacy, and cost inside the IDE (07, Phase 8).
• Privacy matters — indexing/sending a whole proprietary codebase is a governance decision (02/07).
• It's a synthesis phase — assembles Phases 5/7/8/9/10; evaluate it on coding-specific metrics (08).

9. Observations from Real Systems

• Cursor pioneered the multi-model coding IDE: a fast autocomplete model, a dedicated apply model (04), codebase indexing (02), and an agent mode — the canonical Cursor-style architecture.
• GitHub Copilot started as FIM autocomplete and grew chat + agents; Copilot/VS Code added BYOK (07, VS Code BYOK).
• Claude Code is the agent-first, terminal/IDE coding agent — read/edit/run loop with approval (Phase 10.06, what-happens).
• Windsurf, Zed AI, Continue, Aider explore variations (cascades, local models, open-source) of the same subsystem set.
• The differentiators across products are context selection + apply reliability + model routing — not the base LLM, which they often share (06).

10. Common Misconceptions

11. Engineering Decision Framework

BUILD / REASON ABOUT a coding platform:
 1. EDITOR SURFACE [01]: how you capture context + render completions/diffs + run commands (VSCode/JetBrains/own).
 2. CONTEXT SELECTION (the core): index code (embeddings [02] + symbol/AST [03]); retrieve + pack within window [9.07].
 3. LATENCY TIERS → MULTI-MODEL [06]:
       autocomplete → tiny fast FIM (<~200ms, local/edge) [05]
       inline edit  → instruction model + reliable APPLY [04]
       chat         → strong reasoning model
       agent        → strong tool-calling model in a loop [10.06]
 4. APPLY reliably (apply-rate) + REVIEW before commit; agent safety = sandbox/approval [04/10.05].
 5. ROUTE via gateway/BYOK (privacy, cost, provider choice) [07/8].
 6. EVALUATE: autocomplete acceptance + latency, apply-rate, task-resolution, cost/task [08].

12. Hands-On Lab

Goal

Build a minimal file-level coding assistant, then extend it toward platform features — feeling the context-selection and apply problems firsthand.

Prerequisites

• pip install openai; a Python file to edit; (optional) the agent from Phase 10.06.

Steps

1. File-level assistant: read a file, send it with a request, get back a search-replace diff (ORIGINAL/UPDATED), and apply it — with a diff preview + confirmation before writing (04).
2. Context selection: add multi-file context — use grep to find related files/symbols and include the relevant slices (proto-retrieval, 02/03); observe how better context improves edits.
3. Apply-rate: run 10 edit requests; measure how often the diff applies cleanly (exact-match); add re-read before edit and show apply-rate improve (04).
4. Routing by task: add an "autocomplete" path (FIM-style, fast model) vs an "edit" path (instruction model); note the latency/model difference (05/06).
5. Safety: ensure edits go to a branch/workspace with review, and any command execution is sandboxed (Phase 10.05).
6. Reflect: which edits worked? where did context selection or apply fail? Map each failure to a subsystem.

Expected output

A working file-level assistant with diff-apply + preview, basic multi-file context selection, an apply-rate measurement, and a task-routed model split — plus a subsystem-level failure analysis.

Debugging tips

• Edits don't apply → exact-match diff failure; re-read before editing (04).
• Wrong/irrelevant edits → context selection missed the right code (02/03).

Extension task

Add an embeddings index over the repo (02) and symbol search (03); compare retrieval quality vs grep-only.

Production extension

Wrap it in a VS Code extension (01), route models via a gateway/BYOK (07), add an agent mode (Phase 10.06), and evaluate (08).

What to measure

Apply-rate, edit correctness, context-selection relevance, latency per task tier, failure-by-subsystem.

Deliverables

• A file-level coding assistant (diff-apply + preview + confirmation).
• A context-selection improvement (grep/multi-file) before/after.
• An apply-rate measurement + a subsystem failure map.

13. Verification Questions

Basic

1. What are the five subsystems of a coding platform?
2. Why is context selection the central problem?
3. Why do latency tiers force a multi-model architecture?

Applied 4. Map each coding feature (autocomplete/edit/chat/agent) to a latency budget and model profile. 5. Why isn't embeddings-only retrieval enough for code?

Debugging 6. The assistant suggests irrelevant code. Which subsystem do you check? 7. Edits frequently fail to apply. Which subsystem, and the fix?

System design 8. Design a Cursor-style platform: editor surface, indexing, retrieval, multi-model routing, apply, BYOK, eval.

Startup / product 9. Since competitors often share the base LLM, what actually differentiates a coding platform?

14. Takeaways

1. A coding platform = codebase index + context collector + router + apply, in an editor — RAG (Phase 9) + agents (Phase 10) for code.
2. Context selection under a latency budget is the core problem (and the differentiator).
3. Latency tiers force a multi-model engine — tiny FIM autocomplete → strong agent (05/06).
4. Code retrieval needs structure (symbols/AST), not just embeddings; apply must be reliable with review (03/04).
5. It's a synthesis of Phases 5/7/8/9/10, routed via a gateway/BYOK and evaluated on coding metrics (07/08).

15. Artifact Checklist

• A file-level coding assistant (diff-apply + preview + confirmation).
• A context-selection (multi-file/grep, then index) before/after.
• An apply-rate measurement.
• A task-routed model split (autocomplete vs edit).
• A subsystem map of the platform with your design choices.

Up: Phase 11 Index · Next: 01 — VS Code Extension Architecture