Phase 1 · Document 09 · LLM Vocabulary and Mental Models
Prev: 08 — Business and Pricing Terms · Up: Phase 1 Index
All terms in one place. Use as a reference and self-test. Return to this after completing each phase.
Production-critical terms must be known cold; important terms matter for intermediate work; deep/specialized terms can wait until relevant.This glossary is the index ; the deep treatment of each cluster lives in the Phase 1 documents below. When a term is unfamiliar, read its home document for the full 15-section treatment (why it matters, mental model, lab, decision framework).
Domain Home document
Core mental model, token loop, the Six Laws 00 — Core Mental Model Tokens, tokenizer, vocabulary, context window 01 — Tokenization and Context Parameters, weights, checkpoints, dense/MoE, precision 02 — Parameters, Weights, and Checkpoints Sampling/generation parameters 03 — Inference Parameters Model families, variants, capabilities (tools, embeddings, multimodal) 04 — Model Capabilities Prefill/decode, KV cache, batching, TTFT/TPOT, throughput 05 — Serving Terms GGUF/safetensors, quantization, runtimes, hardware/memory 06 — Local Model Terms Benchmarks, evals, golden sets, LLM-as-judge, pass@k 07 — Evaluation Terms Providers, routing, pricing, limits, unit economics 08 — Business and Pricing Terms
This glossary is organized by domain. Cross-reference with phase documents for full explanations and labs.
Term Simple meaning Technical meaning Phase
LLM Large Language Model A transformer-based neural network trained on text to predict/generate tokens 1 Foundation model Pre-trained base model A large model trained on broad data, usable for many tasks with or without fine-tuning 1 Generative model Text-producing model A model that generates new content by sampling from a learned distribution 1 Base model Untrained-for-chat model A model trained on raw text prediction only; not yet fine-tuned for instruction following 1 Instruct model Chat-ready model A base model fine-tuned with SFT+RLHF/DPO to follow instructions 1 Chat model Conversational model A model fine-tuned for multi-turn dialogue 1 Reasoning model Extended-thinking model A model trained to generate internal chain-of-thought before answering 2 Multimodal model Text + image/audio model A model that accepts multiple input modalities 1 Embedding model Text → vector model A model that converts text into a dense vector for similarity search 9 Reranker Relevance scorer A model that scores (query, document) pairs for ranking, used after initial retrieval 9 Vision model Image-understanding model A model that processes images as input 1 Audio model Sound-processing model A model that processes audio input 1 Speech-to-text Transcription model Converts speech audio to text 1 Text-to-speech TTS model Converts text to audio 1
Term Simple meaning Technical meaning Phase
Transformer The core architecture Self-attention + feedforward layers stacked N times 2 Parameters/Weights Learned numbers Floating-point values updated during training; define model behavior 1 Checkpoint Saved model state A snapshot of weights at a point in training 1 Dense model All-neurons-active model All parameters are used for every token 2 MoE model Sparse model Mixture-of-Experts: only a subset of "expert" layers activate per token 2 Active parameters MoE active count Parameters actually used per token (smaller than total in MoE) 2 Total parameters Full model size All parameters, including inactive experts in MoE 2 Model family Series of related models E.g., Llama 3, Gemma, Qwen — models sharing architecture/training 1 Model variant Size/use-case version E.g., Llama-3-8B, Llama-3-70B, Llama-3-8B-Instruct 1 Architecture Model design Configuration: layers, heads, hidden dim, FFN size, attention type 2 Embedding layer Text → vectors Converts token IDs into high-dimensional float vectors 2 Attention Context-aware weighting Mechanism for tokens to attend to each other across the sequence 2 Multi-head attention Parallel attention Multiple attention heads running in parallel, each attending differently 2 Self-attention Token-to-token attention Tokens attend to all other tokens in the same sequence 2 Feed-forward network Per-token transformation Two-layer MLP applied to each token position independently 2 Layer norm Normalization Stabilizes activations; applied before or after attention/FFN 2 Residual connection Skip connection Adds input to output of each sublayer; prevents vanishing gradients 2 RoPE Rotary position encoding Relative positional encoding used by Llama, Gemma, Qwen 2 ALiBi Attention with linear biases Alternative positional encoding that extrapolates to longer sequences 2 FlashAttention Fast attention kernel Memory-efficient CUDA kernel for attention computation 7
Term Simple meaning Technical meaning Phase
Token Text unit Subword unit from BPE/SentencePiece vocabulary 1 Tokenizer Text splitter Maps string → token IDs using a vocabulary 1 Vocabulary Token dictionary The full set of tokens the model knows 1 Token ID Integer index Index into the vocabulary and embedding table 1 Prompt Your input All text sent to the model as input 1 Completion Model output The generated text returned by the model 1 Chat completion Multi-turn format API endpoint that accepts a messages array 1 System message Model instruction First message setting model behavior and persona 1 User message User input The human turn in the conversation 1 Assistant message Model output The model turn in the conversation 1 Context window Max input size Max tokens (input + output) in one call 1 Max input tokens Prompt size limit Hard limit on prompt length 1 Max output tokens Response size limit Hard limit on generated length 1 Input tokens Tokens sent Tokens in the prompt/context 1 Output tokens Tokens generated Tokens produced by the model 1 Cached input tokens Reused prompt tokens Input tokens served from KV cache; may cost less 7 Reasoning tokens Hidden thinking tokens Internal chain-of-thought tokens; billed but not returned 2 Autoregressive One-token-at-a-time Generating tokens sequentially, each depending on previous 2 Prefill Prompt processing phase Processing all input tokens to build KV cache 2 Decode Generation phase Generating output tokens one at a time 2 KV cache Attention memory Stored key/value vectors from previous tokens; avoids recomputation 2 PagedAttention Virtual memory for KV cache Manages KV cache in non-contiguous memory pages 7 Continuous batching Dynamic request batching Adds new requests to a running batch as slots free up 7 Chunked prefill Split prefill Processes long prompts in chunks to reduce batch stalls 7 Prefix caching Shared prompt caching Reuses KV cache for identical prompt prefixes 7 Speculative decoding Draft-then-verify Use small model to draft tokens; large model verifies in parallel 6 MTP Multi-Token Prediction Extension of speculative decoding predicting multiple tokens at once 6 Draft model Small fast model The model that proposes candidate tokens for speculation 6 Acceptance rate Speculation success rate Fraction of draft tokens accepted by the verifier 6 Greedy decoding Always-pick-best Select highest probability token at every step; temperature=0 1 Beam search Multi-path search Maintain k candidate sequences; pick highest overall probability 1 Streaming Token-by-token output Return generated tokens as they're produced via SSE 1 Structured output Schema-constrained output Force output to conform to a JSON schema 1 JSON mode Valid JSON output Constrain model to produce syntactically valid JSON 1 Tool calling Model-requested tools Model emits a structured call to an external tool; app executes 10 Function calling Same as tool calling OpenAI's original term for tool calling 10 Stop sequence Generation stopper String that halts generation when encountered 1
Term Simple meaning Technical meaning
temperature Randomness control Divides logits before softmax; 0=deterministic top_p Probability pool Sample from tokens covering top-p cumulative probability top_k Count pool Sample from top-k highest probability tokens min_p Probability floor Exclude tokens below min_p × max_token_probability repetition_penalty Repeat reducer Penalizes tokens based on prior occurrence (local models) presence_penalty Diversity booster Penalizes tokens that appeared at all frequency_penalty Repetition reducer Penalizes tokens proportional to frequency max_tokens Output length cap Hard limit on generated tokens seed Reproducibility RNG seed for deterministic sampling logprobs Token probabilities Log-probabilities of each output token
Term Simple meaning Technical meaning
TTFT Time to first token Latency from request to first generated token; measures prefill speed TPOT Time per output token Latency between consecutive generated tokens; measures decode speed Throughput Requests per second How many requests/tokens the server can handle per second Latency Response time Total time from request to complete response Tokens/second Generation speed How fast the server generates output tokens Concurrency Parallel requests Number of simultaneous requests handled Batch size Requests per batch Number of requests processed together Tensor parallelism Multi-GPU splitting Split model across GPUs on layer dimension Pipeline parallelism Multi-GPU stages Split model across GPUs by layer groups Data parallelism Multi-GPU replication Replicate model on multiple GPUs for throughput
Term Simple meaning Technical meaning
GGUF Local model format Quantized model format for llama.cpp; single file safetensors Safe weight format HuggingFace format for storing tensors safely PyTorch checkpoint Training save format .pt/.bin files; PyTorch's native format ONNX Cross-framework format Open Neural Network Exchange; portable model format TensorRT NVIDIA optimized format NVIDIA's compiled, optimized model format llama.cpp CPU/GPU inference engine C++ inference library for GGUF models llama-server HTTP server for llama.cpp OpenAI-compatible HTTP server using llama.cpp Ollama Local model runner Easy-to-use local model server with model registry LM Studio GUI local runner Desktop app for running local models MLX Apple Silicon inference Apple's ML framework optimized for Apple Silicon vLLM High-throughput server Python serving library with PagedAttention SGLang Structured generation server Fast inference with structured output support TensorRT-LLM NVIDIA serving library NVIDIA's optimized serving library TGI HuggingFace server Text Generation Inference by HuggingFace OpenAI-compatible API Standardized API REST API matching OpenAI's /v1/chat/completions format
Term Simple meaning Technical meaning
RAM System memory CPU-accessible memory; used for CPU inference VRAM GPU memory GPU-accessible memory; used for GPU inference Unified memory Shared CPU/GPU memory Apple Silicon: CPU and GPU share the same memory pool CUDA NVIDIA GPU programming NVIDIA's parallel computing platform ROCm AMD GPU programming AMD's equivalent to CUDA Metal Apple GPU programming Apple's GPU compute framework BF16 16-bit brain float Training-friendly 16-bit format; good range, lower precision FP16 16-bit float Half precision; common inference format FP8 8-bit float Emerging format; supported on H100+ INT8 8-bit integer 8-bit quantization; ~2x memory reduction INT4 / 4-bit 4-bit integer ~4x memory reduction; quality tradeoff Quantization Weight compression Reducing weight precision to save memory and speed inference GPTQ Post-training quant GPU-optimized post-training quantization AWQ Activation-aware quant Preserves important weights more carefully than GPTQ QAT Quantization-aware training Training with quantization simulation; higher quality Memory headroom Safety buffer Additional free memory needed beyond model size for KV cache and ops
Term Simple meaning Technical meaning
Provider Model host A company or service that serves the model via API Lab Model creator The organization that trained the model Aggregator Multi-provider router A service that routes to multiple providers (OpenRouter) Gateway Internal routing layer A proxy that adds auth, routing, metering, and policy Proxy Pass-through server A server that forwards requests with added functionality BYOK Bring Your Own Key Using your own API keys instead of the platform's Rate limit Request speed cap Maximum requests per minute/hour Quota Usage allowance Total tokens or requests allowed in a period Fallback Backup model A secondary model used when the primary fails SLA Service availability guarantee Uptime/availability promise from provider SLO Internal performance target Team's target for a service level metric Cost per 1M tokens Pricing unit Standard pricing: USD per million tokens Input price Prompt cost Cost per million input/prompt tokens Output price Generation cost Cost per million output/generated tokens Cached input price Cache hit cost Reduced cost when prefix is served from cache Data retention How long data is stored Whether provider stores your requests/responses Data residency Where data lives Which region/country data is processed and stored in License Usage terms Legal terms governing model use (commercial, research, etc.) Open weights Downloadable model Weights can be downloaded; does not imply open source Alias model Versioned shorthand e.g., "gpt-4o" may point to a specific versioned model Deprecation Model end-of-life Provider announcing a model will be retired
Term Simple meaning Technical meaning
RAG Retrieval-Augmented Generation Pattern of retrieving relevant docs and inserting into context Chunk Document segment A portion of a document used as a retrieval unit Embedding Text vector Dense float vector representing text semantics Vector database Embedding store Database optimized for similarity search over embeddings Semantic search Meaning-based search Search by embedding similarity rather than keyword match Hybrid search Combined search Combines dense (semantic) and sparse (BM25) retrieval BM25 Keyword search algorithm TF-IDF based ranking; good complement to semantic search Reranking Relevance re-scoring Re-order retrieved docs by relevance using a cross-encoder Faithfulness Groundedness metric Whether the answer is supported by retrieved documents Groundedness Citation accuracy Whether claims are traceable to source documents Context packing Optimal context filling Fitting retrieved chunks into context as efficiently as possible
Term Simple meaning Technical meaning
Agent Autonomous LLM loop LLM in a loop that can call tools and take actions Tool Callable function A function the model can request to run Tool schema Tool definition JSON Schema describing a tool's name, description, and parameters ReAct Reason + Act loop Agent pattern: reason about what to do, then act with a tool Planner Task decomposer Component that breaks a goal into subtasks Executor Task performer Component that carries out individual tool calls Approval gate Human checkpoint Pause requiring human confirmation before risky action Sandbox Isolated execution Safe environment for code or tool execution Memory Agent recall Mechanisms for agents to remember prior state Guardrail Safety check Checks that constrain or validate agent actions
Term Simple meaning Technical meaning
Eval Quality measurement Systematic test of model outputs against expected behavior Benchmark Standardized test Public test suite used to compare models Golden dataset Ground truth set Curated examples with known correct answers LLM-as-judge Model evaluator Using an LLM to score another LLM's output Pairwise eval Comparative scoring Comparing two model outputs head-to-head Regression eval Change detection Testing whether a model change hurt existing performance TTFT First token latency Time from request to first output token Hallucination Fabricated content Model output that is factually incorrect but sounds confident Faithfulness Citation groundedness Whether the answer is supported by the retrieved context
Can you explain these without looking at the glossary?
If you can explain all of these, you are ready for Phase 2.