Cheatsheet 01 — LLM Terminology

Fast-recall glossary of the terms you must define instantly in an interview. Grouped by area. Full treatment: Phase 1.

Core model concepts

Term	One-line definition
Token	Sub-word unit the model reads/writes (~4 chars / ~0.75 words in English).
Tokenizer	Maps text ↔ token IDs (BPE/SentencePiece).
Parameters	The learned weights; "7B" = 7 billion.
Context window	Max tokens (input + output) the model can attend to at once.
Embedding	A vector representing text meaning; powers retrieval.
Logits	Raw per-token scores before softmax → probabilities.
Vocabulary	The set of tokens the model knows.
Pre-training	Next-token prediction on huge corpora (the base model).
Fine-tuning	Further training to shape behavior (not facts).
Alignment	Making a model helpful/harmless/honest (SFT → RLHF/DPO).

Term	One-line definition
Prefill	Processing the whole prompt in parallel (compute-bound).
Decode	Generating tokens one at a time (memory-bandwidth-bound).
KV-cache	Cached keys/values per token so decode doesn't recompute attention.
Autoregressive	Each token is generated conditioned on all previous tokens.
TTFT	Time To First Token (latency of prefill).
TPOT / ITL	Time Per Output Token / inter-token latency (decode speed).
Throughput	Tokens/sec across all requests (a server metric).
Greedy / sampling	Pick the top token vs sample from the distribution.

Term	One-line definition
Temperature	Randomness; 0 = deterministic, higher = more diverse.
Top-p (nucleus)	Sample from the smallest set of tokens summing to prob p.
Top-k	Sample from the k most likely tokens.
Max tokens	Cap on output length.
Stop sequences	Strings that end generation.
Frequency/presence penalty	Discourage repetition / encourage new topics.
Seed	Fix randomness for reproducibility (where supported).

Term	One-line definition
In-context learning	Learning a task from examples in the prompt (no training).
Few-shot / zero-shot	With / without examples in the prompt.
Chain-of-thought (CoT)	Prompting the model to reason step-by-step.
Reasoning model	A model trained to "think" before answering (extra reasoning tokens).
Tool / function calling	Model emits a structured call the app executes.
Structured output	Model output constrained to a schema (JSON).
Multimodal	Handles text + images/audio/video.
RAG	Retrieval-Augmented Generation — answer from retrieved docs.
Agent	An LLM in a loop that calls tools to accomplish a goal.

Term	One-line definition
Transformer	The attention-based architecture under all LLMs.
Attention	Mechanism letting each token weigh all others.
MoE (Mixture-of-Experts)	Routes tokens to a few expert sub-nets (active < total params).
Active vs total params	Params used per token vs total (MoE distinction).
Quantization	Storing weights in fewer bits (FP16→INT8/INT4) to save memory.
LoRA / QLoRA	Low-rank adapter fine-tuning / on a quantized base.
SFT / RLHF / DPO	Supervised fine-tune / RL from human feedback / direct preference opt.
Distillation	Train a small "student" to mimic a big "teacher."
Hallucination	Confident but false/unsupported output.

Term	One-line definition
vLLM / TGI / SGLang	High-throughput serving engines.
PagedAttention	KV-cache memory management (OS-style paging) → high throughput.
Continuous batching	Add/remove requests from a batch each step → high GPU use.
Prefix/prompt caching	Reuse computation for shared prompt prefixes.
Speculative decoding / MTP	Draft model proposes tokens, big model verifies → lower latency.
Gateway / proxy	Layer that routes/meters/fallbacks across providers.
p50/p95/p99	Latency percentiles (tail latency matters most).

Why is decode slow? It's sequential and memory-bandwidth-bound — each token reloads the whole model + KV-cache from VRAM. (tok/s ≈ memory bandwidth ÷ model size in bytes.)
Why does context cost grow? KV-cache memory scales with context length × layers × heads — long context = more VRAM.
RAG vs fine-tuning? RAG adds knowledge (facts/context); fine-tuning shapes behavior (format/style/skill).
Why is structured output not enough for correctness? Constrained decoding guarantees valid JSON, not correct content.

Next: 02 — Model Card Reading · Full: Phase 1