« Phase 31 README · Warmup · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 31 — Hitchhiker's Guide

The compressed practitioner tour. Read the WARMUP for the mechanism; this is the stuff you say in the meeting.

30-second mental model

Cohere is the enterprise-RAG company. It does not try to win the frontier-IQ race; it sells attributable, deployable intelligence — answers that cite their sources, retrieval precise enough to be worth citing, and models that run inside the customer's boundary (VPC / on-prem / air-gapped) without training on their data. Four products, one thesis: Command (generation models tuned for grounded, cited, tool-using RAG), Embed (query/document-asymmetric embeddings with int8/binary/Matryoshka compression), Rerank (a cross-encoder that re-scores retrieval results — the most-adopted API, drops into anyone's stack), and North (a secure AI workspace packaging all three + agents + connectors, deployed where the data lives). The senior move: "Cohere sells the enterprise what it actually lacks — not intelligence, but attributability and deployment control."

The stack to tattoo on your arm

The distinctions that signal seniority

• Grounded generation vs naive RAG → naive RAG pastes chunks in the prompt and hopes; Cohere returns a structured, checkable span→source citation mapping the model was trained to emit.
• Citations vs faithfulness → citations are what the model returns; faithfulness is whether they're true. Cohere doesn't refuse uncited sentences — enforcement (drop/verify uncited claims) is YOUR job. A model trained to cite can still miscite.
• search_query vs search_document → asymmetric embeddings. Wrong one = silent quality loss, never an error.
• Bi-encoder vs cross-encoder → first-stage retrieval embeds query and doc separately (fast, blind to interaction); Rerank scores the pair jointly (accurate, O(candidates), so shortlist-only).
• int8 vs binary embeddings → int8 is a near-free 4× shrink; binary is ~32× and costs recall — a dial you measure, not damage.
• North vs "ChatGPT Enterprise for Cohere" → the point is where it runs and who controls the data, not the chat surface.
• Cohere vs OpenAI/Anthropic → not a model-IQ contest; it's the enterprise envelope (grounding, retrieval precision, deployment control). Pick by axis, not favorite.

The SDK one-liners

import cohere
co = cohere.ClientV2()  # CO_API_KEY

# Grounded generation with citations
resp = co.chat(
    model="command-r-08-2024",
    messages=[{"role": "user", "content": "What's our enterprise refund window?"}],
    documents=[{"id": "d1", "data": {"text": "Enterprise refunds: within 45 days of purchase."}}],
)
print(resp.message.content[0].text)   # grounded answer
print(resp.message.citations)         # span -> source mapping

# Embed: index docs one way, queries the other
doc_vecs = co.embed(texts=corpus, model="embed-english-v3.0",
                    input_type="search_document", embedding_types=["float"]).embeddings.float
q_vec    = co.embed(texts=[query], model="embed-english-v3.0",
                    input_type="search_query").embeddings.float[0]

# Rerank: cross-encoder over the first-stage shortlist
r = co.rerank(model="rerank-v3.5", query=query, documents=candidates, top_n=5)
for hit in r.results:                 # each hit: .index (into candidates), .relevance_score
    print(hit.index, hit.relevance_score)

War stories

• The "search returns the policy index, not the procedure" bug. First-stage embeddings ranked a term-stuffed policy page #1 because it repeated every query word; the actual how-to sat at #4. Bag-of-words retrieval loves repetition. Adding Rerank (a cross-encoder that sees phrasing, not just term presence) promoted the how-to to #1 — one API call, visible win. This is Lab 01's planted example, and it's the single most common reason teams adopt Rerank.
• The chatbot that embedded queries as search_document. Retrieval was "fine, not great" for months; nobody could say why. Someone had passed input_type="search_document" for the query side too, discarding the asymmetry the model was trained on. Flipping queries to search_query moved recall several points. No error was ever thrown — that's the trap.
• The audit that grounded generation survived. A regulated customer got asked, months later, on what basis the assistant told a client they qualified for a refund. Because every answer shipped with citations and an enforcement layer had dropped uncited claims, the answer was a link to a specific clause in the log — not "the model said so." That's the whole reason they'd picked Cohere.
• The binary-embedding index that lost the long tail. A team compressed to binary for 32× storage savings, shipped, and watched recall on hard/rare queries quietly drop. int8 would've been near-free; binary needed measuring first. Compression is a dial, not a default.

Vocabulary

Command R / R+ / R7B / A · Chat API · documents param · grounded generation · citations ({start, end, text, document_ids}) · faithfulness / grounding score · Embed · input_type (search_document / search_query / classification / clustering) · asymmetric embeddings · embedding_types (float / int8 / binary) · Matryoshka · multilingual embeddings · Rerank · cross-encoder vs bi-encoder · top_n · max_chunks_per_doc · two-stage retrieval · North · connectors · tool use · private / VPC / on-prem / air-gapped deployment · no training on customer data · recall@k / rerank lift / citation accuracy.

Beginner mistakes

1. Talking about Cohere like a chat vendor competing on model IQ — it's an enterprise-RAG platform competing on grounding + deployment control.
2. Assuming citations guarantee faithfulness — they don't; enforce and verify them yourself.
3. Embedding queries and documents with the same input_type — silent recall loss.
4. Treating Rerank as "a better embedding search" — it's a cross-encoder, runs on the shortlist, not the corpus.
5. Compressing straight to binary embeddings without measuring the recall cost (int8 is the safe default shrink).
6. Describing North as a chat UI — the point is private deployment and data control.
7. Pitching Cohere on benchmark scores instead of the enterprise envelope — you'll lose the room.