Cheat Sheet — Agentic AI Engineering

The numbers, formulas, and one-liners to have loaded before an interview or an incident. Every row ties to the phase that derives it.

The numbers to know cold

Number	Meaning	Phase
`0.95^10 ≈ 0.60`	ten 95%-reliable steps ≈ a 60% agent	00
`0.90^7 ≈ 0.48`	90%/step is a coin-flip by step 7	00
`n_max = ⌊log T / log p⌋`	reliable step budget for target T	00
`1 − (1−p)^(r+1)`	reliability of a step retried r times	00
ReAct input ≈ `(t+o)·n²/2`	quadratic — scratchpad resent each step	00/01
ReWOO input ≈ `O(n)`, 2 LLM calls	linear — plan once, solve once	00/01
output tokens ≈ `3–5×` input price	generating costs more than reading	00
`$/resolved = $/attempt ÷ success_rate`	the real unit cost	00/14
design to p95/p99	the mean is a liar	00/12/14
BM25 `k1≈1.5, b≈0.75`	TF saturation, length norm	05
RRF `k≈60`, `Σ 1/(k+rank)`	score-scale-free fusion	05
MCP protocol `2025-06-18`	version string in `initialize`	03
JSON-RPC error codes	`-32700/-32600/-32601/-32602/-32603`	03
Cohen's κ > ~0.6	judge↔human agreement floor to trust a judge	11

The five load-bearing truths

Model proposes, application executes — the trust boundary; all controls live on your side of it.
Reliability compounds — 0.95^n; buy steps back with retries, shorter chains, or verification.
Untrusted text becomes instruction — prompt injection has no general fix; contain it architecturally.
Cost and latency are engineered — tokens, cache, routing, quotas, p95; instrument day one.
What you can't evaluate, you can't ship — golden sets, judges, trajectory evals, regression gates.

The agent loop (memorize)

scratchpad := task
repeat up to max_steps:
    proposal := model(scratchpad)      # untrusted text
    if final(proposal): return answer
    name, args := parse(proposal)      # cross the trust boundary
    if not allowed(name, args): obs := "denied"        # authz, least privilege
    else: obs := run_in_sandbox(name, args)            # execute
    scratchpad += proposal + obs       # remember (grows → quadratic)
return give_up   # the guard fired

Orchestration modes (pick per task)

Mode	LLM calls	Adaptivity	Use when
ReAct	one per step	high	uncertain path, needs mid-course correction
Plan-execute-replan	1 + replans	medium	mostly-known path, occasional failures
ReWOO	2	low	predictable path, cost/latency matter, parallelizable

Tool calling (Phase 02)

Model → {"name": ..., "arguments": {...}}; validate arguments against JSON Schema before running; failures → structured error, not a crash.
Gotcha: arguments is often a JSON string inside JSON — parse twice.
Constrained decoding gives syntactic validity; you still need semantic validation + a repair loop. Few well-scoped tools beat many; error messages should be model-actionable.

MCP (Phase 03)

Handshake: client initialize (protocolVersion, capabilities, clientInfo) → server result (capabilities, serverInfo) → client notifications/initialized.
Discover: tools/list, resources/list, prompts/list. Use: tools/call, resources/read, prompts/get. Client primitives: sampling/createMessage, elicitation/create.
Notifications have no id. notifications/tools/list_changed → client refreshes.
Transport: stdio (local) / Streamable HTTP + OAuth (remote). A server is untrusted code — permission-gate and audit it.

Retrieval (Phase 05/06)

Pipeline: ingest → chunk → embed → index → (dense + BM25) → RRF → rerank → pack.
Hybrid beats either alone: BM25 catches rare tokens/IDs, dense catches paraphrase.
Bi-encoder retrieves (fast); cross-encoder reranks (accurate). Metrics: recall@k, MRR, nDCG.
Vector DBs: pgvector (already in Postgres), Pinecone/Weaviate/Milvus (managed ANN), FAISS (library), Chroma (local). Neo4j for graph.
GraphRAG → global/thematic questions; RAPTOR → multi-level abstraction; LightRAG → incremental dual-level.

Durable execution (Phase 08)

Workflow = deterministic orchestration; activities = the side-effecting steps (retried).
No wall-clock / random / I/O in workflow code — it must replay identically. Use injected time and activity results.
Idempotency keys make retries safe. Signals resume paused workflows (human approval).
"A Durable orchestrator is replay-safe; a raw agent loop is not."

Security (Phase 09/10/13)

Injection vectors: direct (user), indirect (fetched content/tool result), memory (poisoned store). Exfil via markdown image URLs / outbound calls.
Defenses (layered, none sufficient alone): least privilege + allow-lists, input/output guardrails, output exfil scan, human-in-the-loop on high-impact, sandbox + egress control, tenant isolation. Prompting is a mitigation, never a control.
Multi-tenant leak channels (rank order): shared vector index #1, prompt cache, agent memory, co-mingled fine-tunes. Thread tenant_id from trusted auth through every layer; default-deny.

Evaluation (Phase 11)

Golden dataset (versioned) → run → score (programmatic > calibrated judge > human) → regression gate in CI. Evaluate YOUR task, not benchmarks.
Grade the trajectory (tools, order), not just the final answer.
LLM-as-judge biases: position, verbosity, self-preference. Validate with Cohen's κ vs human.
Safety is a gate, not a weighted average.

Cost / latency / observability (Phase 14)

Levers: token budget, caching (semantic + prefix), model routing/cascade, distillation, degradation ladder. Target 70–80% gross margin on $/resolved-task.
Trace every run with OTel spans (LLM, tool, retrieval). Meter tokens per request from day one.

Interview reflexes

Asked to design an agent → first ask step count, per-step reliability, target, latency budget, tenancy, cost target. Turn vibes into numbers.
Asked "is this an agent?" → often "no, it's a workflow" is the senior answer.
Asked about safety → say architecture (trust boundary, least privilege, sandbox, HITL), not "we'll prompt it to be careful."
Asked about a flaky agent → diagnose structurally (loop guard? lossy parse? uncaught tool error? bad plan?), not "check the docs."
Always close a claim with a number: 0.95^n, $/resolved, recall@k, p95, κ.

Agentic AI Engineer — Agent Infrastructure, Platforms & Production Systems