« Track Overview · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 31 — Cohere Platform: Command, Embed, Rerank & Grounded Generation (North)

Answers these JD lines (Cohere — Forward Deployed Engineer, Agentic Platform / North): "design, build, and deploy production-grade RAG and agentic workflows for enterprise customers," "develop robust evaluation frameworks to measure quality, groundedness, and safety," "operate under enterprise security, private deployment, and auditability constraints," and — the line every other line hangs off — "deep, hands-on knowledge of Cohere's platform: Command, Embed, Rerank, and North." A forward deployed engineer is the person who sits inside a customer's environment and turns an ambiguous business problem into a working, defensible system built from exactly these pieces. This phase teaches the pieces at mechanism depth.

Why this phase exists

Every framework phase in this track (18–24) taught a runtime — where agents run, how loops execute, how a platform serves models. Cohere is a different animal: it is a model and API company whose entire product line is shaped around one enterprise thesis — retrieval-grounded, citable, privately deployable AI. Three ideas do most of the work:

1. Grounding is a first-class API contract, not a prompt trick. Cohere's Chat API takes a documents parameter and returns a citations list mapping spans of the generated answer to the documents that support them. The Command R family was trained to do this. For an enterprise that must answer "why did the assistant say that?" in an audit, the citation list is the difference between a demo and a deployment.
2. Retrieval is a two-stage discipline. Embed (a bi-encoder, with input_type-asymmetric query/document vectors and int8/binary/Matryoshka compression) retrieves wide and cheap; Rerank (a cross-encoder) re-scores the shortlist with real query×document interaction. Cohere productized both halves, and Rerank drops into any existing search stack — which is why it's the API even non-Cohere shops adopt first.
3. The deployment boundary is the product. Cohere models run as SaaS, in your VPC (AWS, Azure, GCP, OCI), on-prem, or air-gapped; Cohere does not train on your data by default; and North packages the whole stack — models, retrieval, agents, workplace-tool connectors — as a secure AI workspace that lives inside the customer's boundary. Security-first is not a compliance page; it is the sales pitch.

This phase builds the mechanisms behind all three as deterministic miniatures, then arms you with the platform fluency (model family, API shapes, deployment modes, competitive framing) an FDE interview probes.

Concept map

• Command — the generation family: Command R / R+ (RAG- and tool-use-optimized chat models, 128k context), Command R7B (small/edge), Command A (the current flagship line: 256k context, agent- and enterprise-tuned). Trained for grounded generation, citations, and structured tool calls — covered in the Warmup, exercised via the injected-generator seam in Labs 02–03.
• Chat API — messages, the documents parameter, tools (function calling), streaming, and the citations array in the response (Lab 02 builds the citation contract).
• Grounded generation with inline citations — Cohere's signature: each answer span carries {start, end, text, document_ids/sources}; the enforcement layer (drop what no document supports) and the faithfulness score are yours to build (Lab 02).
• Embed — v3/v4 embedding models; input_type (search_document vs search_query vs classification vs clustering) as asymmetric encoding, multilingual coverage, int8/binary/Matryoshka compressed embeddings (Lab 03).
• Rerank — the cross-encoder second stage over first-stage candidates: top_n, preserved original indices, long-document chunking, and the latency/quality tradeoff (Lab 01).
• North — the secure AI workspace: private/VPC/on-prem/air-gapped deployment, connectors to workplace tools, agents with tool access, RBAC/SSO, data that never leaves the customer's control — covered in the Warmup as the platform an FDE actually deploys.

The labs

82 tests total (27 + 28 + 27), pure stdlib, offline, deterministic — per the lab standard.

Integrated scenario (how this shows up at work)

You're the forward deployed engineer at a bank that just licensed North. Compliance wants every AI answer traceable to a policy document; the CISO wants nothing leaving the VPC; the business wants an assistant that actually answers procedure questions, not one that recites the policy index. You deploy North inside their VPC with connectors to the document store — data stays in their boundary, satisfying the CISO. You index policy docs with Embed (search_document), embed queries as search_query, and retrieve wide; Rerank fixes the "term-stuffed policy page beats the actual how-to" failure (Lab 01/03's planted example is this exact bug). Answers come from Command through the Chat API with documents, and every response ships with citations compliance can click through (Lab 02's contract) — plus your enforcement layer that drops any claim no document supports, because a model trained to cite can still miscite. You wire the Phase 11 eval harness to score faithfulness and citation accuracy on a regression set before every config change, and when the business asks for "an agent that files the exception ticket too," you extend the same stack with tool use (Phase 01's loop, Cohere's tools parameter) inside North's permission model.

Deliverables checklist

• Lab 01 green under LAB_MODULE=solution pytest and your own lab.py (27 tests); you can explain why the promoted document lost the first stage and won the rerank.
• Lab 02 green (28 tests); you can state Cohere's citation shape from memory and why the enforcement layer is yours to build.
• Lab 03 green (27 tests); you can explain what breaks (silently) when someone embeds queries with input_type="search_document".
• You can sketch the two-stage RAG pipeline and place Embed, Rerank, and Chat+documents on it without hesitating.
• You can describe North in two sentences: what it is, where it runs, what data leaves.
• You can argue Cohere vs OpenAI vs Anthropic vs open-weights for a regulated enterprise — by axis, not by favorite.

Key takeaways

• Cohere's differentiator is not raw model IQ — it is grounding + deployment boundary. Citations you can audit, models you can run inside your own walls, and no training on your data. That is the enterprise thesis, and every API reflects it.
• Rerank is the highest-leverage single call in retrieval: a cross-encoder over the top-100 candidates fixes the precision failures bi-encoders structurally cannot see, at a latency cost you control with top_n.
• input_type is load-bearing. Query and document embeddings are asymmetric on purpose; using the wrong one doesn't error — it silently degrades.
• Citations are a contract, not a garnish: offsets into the answer, ids into your documents, independently verifiable — and verification (plus dropping the unsupported) is the deployment's job, not the model's promise.
• The senior framing: "Cohere sells the enterprise the thing it actually lacks: not intelligence, but attributability — answers that cite, models that stay inside the boundary, and a retrieval stack precise enough to be worth citing."