« Phase 30 README · Warmup · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 30 — Hitchhiker's Guide

The compressed practitioner tour. Read the WARMUP for the mechanism; this is the stuff you say in the meeting.

30-second mental model

Hugging Face is an ecosystem around an artifact registry, not a model. transformers loads and runs any architecture by convention (Auto* classes dispatch on config.json); pipeline() wires tokenizer + model + task post-processing into one callable; the Hub is a git-backed store of models/datasets/Spaces with cards, licenses, gating, and revisions; datasets streams corpora bigger than RAM; Trainer/accelerate run fine-tuning; PEFT/LoRA adapts <1% of parameters into megabyte adapters; quantization (bitsandbytes/GPTQ/AWQ) shrinks the base; TGI serves it with continuous batching. The senior move: "HF is the packaging standard for open models — the answer to 'which model, which revision, tokenized how, adapted with what, quantized to fit where, served on which stack' — which is what makes open-weight strategy an engineering decision, not a leap of faith." The transformer internals are the Senior AI Engineer track; this is the workflow.

The pieces to tattoo on your arm

The distinctions that signal seniority

• pipeline() vs AutoModel → pipeline is the demo/default front door; production drops to Auto* for control over batching, devices, revisions — or to TGI, which replaces the loop.
• BPE vs WordPiece vs Unigram → merge-by-frequency (GPT) vs merge-by-likelihood-gain +## (BERT) vs start-big-and-prune + ▁ (T5/Llama SentencePiece). All trained, all shipped with the model.
• temperature vs top-k vs top-p → rescale distribution vs fixed-k cut vs adaptive cumulative-mass cut. Temperature never reorders; top-p adapts to model confidence.
• greedy vs beam vs sampling → argmax vs N-best-by-log-prob (closed-form tasks) vs draw (open-ended). generate() composes the knobs as a fixed LogitsProcessor chain.
• merge vs swap (LoRA) → merge folds A·B into W (identical output, zero overhead, no longer separable) vs keep adapters separate (one base, many tasks/tenants, tiny extra matmul).
• QLoRA quantizes the base, not the fine-tune → the artifact you ship is a normal LoRA adapter.
• safetensors vs pickle → a security property (no code execution), not just speed; and trust_remote_code=True is a separate, explicit "run this repo's Python" opt-in.
• static vs continuous batching → run-to-completion (idle GPU behind the straggler) vs reschedule every decode iteration (utilization stops being length-variance-hostage).

The one-liners

from transformers import (pipeline, AutoTokenizer, AutoModelForCausalLM,
                          TrainingArguments, Trainer)

# 1. pipeline — the front door
gen = pipeline("text-generation", model="mistralai/Mistral-7B-Instruct-v0.3", device_map="auto")
gen("Explain LoRA in one line:", max_new_tokens=64, temperature=0.7, top_p=0.9)

# 2. Auto* — load any architecture by convention; PIN THE REVISION
tok = AutoTokenizer.from_pretrained("gpt2", revision="e7da7f2")
model = AutoModelForCausalLM.from_pretrained("gpt2", revision="e7da7f2",
                                             torch_dtype="auto", device_map="auto")
ids = tok("Hello", return_tensors="pt")           # + attention_mask, offsets on fast tokenizers
out = model.generate(**ids, max_new_tokens=32, do_sample=True, top_p=0.9, repetition_penalty=1.1)
tok.decode(out[0], skip_special_tokens=True)

# 3. Trainer — the fine-tuning loop you don't rewrite
args = TrainingArguments(output_dir="out", per_device_train_batch_size=8,
                         gradient_accumulation_steps=4, learning_rate=2e-5,
                         num_train_epochs=3, bf16=True, push_to_hub=True)
Trainer(model=model, args=args, train_dataset=train, eval_dataset=dev,
        data_collator=collator, compute_metrics=metrics).train()

# 4. PEFT/LoRA — adapt <1% of params; QLoRA = the same over load_in_4bit
from peft import LoraConfig, get_peft_model
cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["q_proj", "v_proj"], task_type="CAUSAL_LM")
model = get_peft_model(base, cfg)
model.print_trainable_parameters()                 # "trainable: 4.2M || all: 7B || 0.06%"
# ... train ...
model.merge_and_unload()                            # fold A·B into W for zero-overhead serving

# 5. datasets + hub
from datasets import load_dataset
ds = load_dataset("imdb", split="train", streaming=True)      # zero-footprint iteration
model.push_to_hub("acme/my-adapter"); tok.push_to_hub("acme/my-adapter")

# serve it (TGI): one container, continuous batching, OpenAI-compatible /v1/chat/completions
docker run --gpus all ghcr.io/huggingface/text-generation-inference \
  --model-id acme/my-merged-model --quantize awq

War stories

• The "we swapped tokenizers to save tokens" quality collapse. Someone pointed the same model weights at a different tokenizer to cut token counts. Output turned to noise — the embedding matrix is indexed by the original vocabulary. Tokenizer and weights are one artifact; pin them together. (Its cousin: hand-formatting chat prompts instead of apply_chat_template and quietly losing the instruction-tuned behavior.)
• Batched generation that returned garbage for half the batch. A GPT-2-family model with no pad token, right-padded, batched. The short sequences were "continued" from padding. Two fixes nobody had set: pad_token = eos_token and padding_side="left".
• The unpinned from_pretrained("main") that changed behavior overnight. The model author pushed a new commit; a production eval that passed Friday failed Monday with no diff on our side. Pin a commit sha, snapshot the GenerationConfig, cache in CI. It's container-digest discipline for models.
• The trust_remote_code=True that shipped to prod because the quickstart had it. A repo with custom modeling code needed it; someone copy-pasted the flag into the service without reading the repo. That flag executes arbitrary Python at load — it's a code-review event, pinned to a revision, or it's an incident waiting.
• The 4-bit model that "worked" until it did math. AWQ 4-bit held general chat quality but visibly regressed on arithmetic and long-tail facts. Nobody re-ran evals after quantizing. Quantization degradation is task-dependent — measure on your tasks.

Vocabulary

transformers / tokenizers / datasets / accelerate / peft / huggingface_hub · the Hub (models/datasets/Spaces) · pipeline() · Auto* / from_pretrained / push_to_hub / save_pretrained · config.json / model_type / task heads · BPE / WordPiece / Unigram / SentencePiece (▁, ##) · special tokens / chat template / attention mask / left-padding / offset mapping · generate() / GenerationConfig · greedy / beam / temperature / top-k / top-p / repetition penalty / EOS · KV cache / prefill vs decode · model card / license / gated repo / revision (sha/tag/branch) · safetensors vs pickle / trust_remote_code · Trainer / TrainingArguments / data collator / SFTTrainer · LoRA (r/alpha/target_modules) / merge vs swap / QLoRA / NF4 / prompt tuning · bitsandbytes / GPTQ / AWQ / GGUF · TGI / vLLM / continuous batching / PagedAttention / Inference Endpoints.

Beginner mistakes

1. Treating the tokenizer as swappable or optional — it's a trained artifact versioned with the weights; mismatch = gibberish, and skipping the chat template = a "dumber" model.
2. Shipping pipeline() defaults (unpinned default model) to production instead of dropping to Auto* with a pinned revision.
3. Forgetting the attention mask / pad token / left-padding on batched generation — all silent quality bugs, not crashes.
4. Thinking temperature adds knowledge — it only rescales the existing distribution; at 0 it's argmax.
5. Believing generate() is one forward pass — it's a loop, one forward per token against the KV cache, which is why length drives latency.
6. Calling LoRA "approximate fine-tuning" — it's a low-rank parameterization, and a merged adapter's outputs are exactly equal to the unmerged ones.
7. Thinking QLoRA quantizes your fine-tune — it quantizes the frozen base; the adapter stays high-precision.
8. Using trust_remote_code=True as a compatibility flag — it runs arbitrary repo code; review and pin, or don't.
9. Assuming "open model" means "do anything" — Llama's community license, RAIL, and research-only releases all have real obligations. Read the card.
10. Quantizing and skipping the re-eval — 4-bit hits are task-dependent; measure on your workload.