« Overview

Glossary — Agentic AI Engineering

Every term an interviewer or an incident might throw at you, defined from first principles and tied to the phase that builds it. Alphabetical within sections.

Core agent concepts

  • Agent — an LLM in a loop that chooses its next action at runtime, with tools and memory. The defining property is runtime action selection by the model. (Phase 00/01)
  • Workflow — a system where the path is fixed in code, not chosen by the model. Most production "agents" are, and should be, workflows. (Phase 00)
  • Trust boundary — the line between the untrusted proposal (model text) and the trusted executor (your code). "The model proposes, the application executes." Every safety and correctness control lives on the application side. (Phase 00, threaded everywhere)
  • Scratchpad / transcript — the running record of thoughts, actions, and observations fed back to the model each step; it grows every turn (the source of ReAct's quadratic cost).
  • ReAct — Reasoning + Acting: interleave thought → action → observation, one LLM call per step. Adaptive; quadratic input tokens. (Phase 01)
  • ReWOO — Reasoning WithOut Observation: plan the whole task (1 call), execute tools without the LLM, solve once (1 call). Two LLM calls total; cheap; not adaptive. (Phase 01)
  • Plan-execute-replan — execute a plan and re-plan on failure (bounded). ReWOO's cost with recoverability. (Phase 01)
  • Policy (injected model) — a plain str -> str function that stands in for the LLM so the loop is deterministic and testable. (Phase 01, every lab)
  • Step budget / max_steps — the hard cap on loop iterations; a safety guard, not a target. Set from the reliable step budget plus margin. (Phase 00/01)
  • Error as observation — a tool failure returned as structured data the agent can recover from, never a raised exception that crashes the loop. (Phase 01)

Reliability, cost, latency

  • Compound reliability — end-to-end success of n independent steps at per-step p is \(p^n\). 0.95^10 ≈ 0.60. (Phase 00)
  • Reliable step budget — \(\lfloor \log T / \log p \rfloor\), the most unverified steps you can afford at target reliability T. (Phase 00)
  • Idempotent — safe to run more than once with the same effect; the precondition for a safe retry. (Phase 00/08)
  • Cost per resolved task — $/attempt ÷ success_rate; the real unit cost. (Phase 00/14)
  • Tail latency (p95/p99) — the value below which 95%/99% of requests complete; the SLO metric. The mean lies. (Phase 00/12/14)

Tools & structured output

  • Tool / function calling — the model emits a structured request (name + JSON arguments); your app validates and executes. (Phase 02)
  • JSON Schema — the contract for a tool's arguments (and for structured output): types, required fields, enums, ranges. Validated on the trusted side. (Phase 02)
  • JSON mode vs tool mode vs constrained/grammar decoding — three ways to get structured output; constrained decoding masks logits to a grammar (GBNF) so tokens are syntactically valid, but valid JSON can still be semantically wrong. (Phase 02)
  • Repair loop — validate model output; on failure, feed the errors back and re-emit, bounded. "Constrained ≠ correct." (Phase 02)
  • Parallel tool calls — a model requesting several independent tools in one turn. (Phase 02)

MCP (Model Context Protocol)

  • MCP — an open protocol standardizing how AI apps connect to tools/data: "USB-C for AI tools." Solves the M×N integration problem. (Phase 03)
  • Host / Client / Server — the host (AI app) runs one client per server connection; the server exposes context. (Phase 03)
  • JSON-RPC 2.0 — MCP's data-layer protocol: requests (with id), responses, and notifications (no id). (Phase 03)
  • Primitives — server-side: tools (actions), resources (data), prompts (templates); client-side: sampling (sampling/createMessage), elicitation (elicitation/create), logging. Discovered via */list. (Phase 03)
  • Transport — stdio (local process) or Streamable HTTP (remote, OAuth). (Phase 03)
  • Capability negotiation — the initialize handshake where client and server declare what they support. (Phase 03)

Context & memory

  • Context engineering — engineering the entire context window (system, tools, retrieved, memory, user), not just the instruction; the successor framing to "prompt engineering." (Phase 04)
  • Context window — the finite, billed token budget the model reads. (Phase 00/04)
  • Lost in the middle — models attend worst to the middle of a long context; put critical items at the edges. (Phase 04)
  • Intent routing / classification — deciding which skill/workflow a query belongs to; handles "workflow collisions" and enables cheaper per-intent models. (Phase 04)
  • Memory tiers — working/buffer (recent turns), session summary (compressed), long-term semantic/episodic (vector recall). (Phase 04)

Retrieval & RAG

  • RAG — Retrieval-Augmented Generation: fetch relevant context, then generate grounded on it. (Phase 05)
  • Chunking — splitting documents into retrievable units; size/overlap is a tradeoff. (Phase 05)
  • Embedding — a vector encoding meaning; similarity by cosine (on normalized vectors). (Phase 05)
  • ANN (HNSW/IVF/PQ) — approximate nearest-neighbor search; trades exactness for speed at scale. (Phase 05)
  • BM25 — a lexical ranking function (TF saturation + IDF + length norm); catches rare tokens/IDs dense retrieval misses. (Phase 05)
  • Hybrid search + RRF — combine lexical and dense results; Reciprocal Rank Fusion (Σ 1/(k+rank), k≈60) is score-scale-free. (Phase 05)
  • Bi-encoder vs cross-encoder — bi-encoder embeds query and doc separately (fast, retrieve); cross-encoder scores the pair jointly (slow, accurate, rerank). (Phase 05)
  • recall@k / precision@k / MRR / nDCG — retrieval metrics; retrieval and generation are evaluated separately. (Phase 05/11)
  • GraphRAG — build a knowledge graph (entities + relations) + community summaries; answers global/thematic questions vector RAG can't. (Phase 06)
  • LightRAG — dual-level (local + global) graph retrieval with incremental updates. (Phase 06)
  • RAPTOR — recursively cluster + summarize chunks into a tree; retrieve at multiple abstraction levels. (Phase 06)

Multi-agent & durability

  • Supervisor / worker / critic — orchestration roles: a planner delegates, workers execute, a critic verifies. (Phase 07)
  • Handoff — one agent transferring control (and context) to another. (Phase 07)
  • Message bus / blackboard — shared communication substrate for multi-agent coordination. (Phase 07)
  • Durable execution — a workflow that survives process crashes by event sourcing and deterministic replay; the model behind Temporal. (Phase 08)
  • Event sourcing / replay — persist a log of events; reconstruct state by replaying it. A workflow must be deterministic to replay safely (no wall-clock/random in the workflow). (Phase 08)
  • Signal / human-in-the-loop pause — an external event that resumes a paused workflow (e.g. an approval). (Phase 08/10)
  • Saga / compensation — undo steps for a partially-completed multi-step transaction. (Phase 08)

Security & isolation

  • Prompt injection — untrusted text (a web page, document, tool result, memory) becoming an instruction the model follows. No general fix; contained architecturally. (Phase 10)
  • Direct / indirect / memory injection — injection via the user prompt / via fetched content / via poisoned stored memory. (Phase 10)
  • Data exfiltration — an injected agent leaking secrets/data (e.g. via a markdown image URL or an outbound tool call). (Phase 10)
  • OWASP LLM Top 10 — the canonical LLM/agent threat list. (Phase 10)
  • Guardrails — deterministic input/output checks around the model (moderation, PII, schema, exfil scan, grounding). Output guardrails matter most. (Phase 10)
  • Sandbox / capability-based security — run agent-invoked code/tools with least-privilege capabilities (fs/net/exec) and resource limits; a container/microVM in production. (Phase 09)
  • Least privilege — grant the minimum tools/scopes/data an agent needs. (Phase 09/10/13)
  • Multi-tenancy — many customers on shared infra; isolation via silo/pool/bridge. The AI-specific leak channels: shared vector index, prompt cache, agent memory, co-mingled fine-tunes. (Phase 13)
  • RBAC / ABAC / OAuth / OIDC — authorization and authentication mechanisms. (Phase 13)

Evaluation & observability

  • Golden dataset — a curated, versioned set of inputs + expected outputs/labels; the eval ground truth and regression baseline. (Phase 11)
  • LLM-as-judge — using an LLM to score outputs; needs a human-agreement check (Cohen's κ) and bias controls before you trust it. (Phase 11)
  • Trajectory / step evaluation — grading the agent's path (tools, order), not just the final answer. (Phase 11)
  • Behavioral regression testing — re-running the golden set on every change to catch quality regressions; evals-as-CI-gates. (Phase 11)
  • OpenTelemetry (OTel) GenAI — the tracing standard for spans across an agent run (LLM calls, tool calls, retrieval). (Phase 14)
  • Semantic cache / prefix cache — cache by meaning / by shared prompt prefix to cut cost and latency. (Phase 14)
  • Model routing / cascade — send each request to the cheapest model that can handle it; escalate on low confidence. (Phase 14)

AI-native SDLC

  • Spec-Driven Development (SDD) — write an executable/verifiable spec first; the agent implements against it. (Phase 15)
  • Coding agent — an agent that reads a repo, plans, and applies patches (Claude Code, Codex, Cursor). (Phase 15)
  • Apply-patch / diff application — the mechanism a coding agent uses to edit files safely. (Phase 15)
  • Reusable command / skill — a packaged, parameterized agent workflow. (Phase 15)

Real-time / voice

  • VAD / endpointing — voice-activity detection / deciding when a speaker has finished a turn. (Phase 16)
  • Barge-in — the user interrupting the agent mid-speech. (Phase 16)
  • STT / TTS — speech-to-text / text-to-speech; the ends of a voice pipeline. (Phase 16)
  • WebRTC — the real-time transport for voice/video agents (LiveKit). (Phase 16)