Warmup Guide — CV Deep Learning: Detection & Segmentation

Zero-to-expert primer for Phase 05: the dense-prediction tasks — object detection (two-stage and one-stage), semantic and instance segmentation — and the shared machinery (anchors, IoU, NMS, mAP) that every system in this phase runs on.

Table of Contents

Chapter 1: The Task Taxonomy

Classification answers what; this phase's tasks answer what and where, at increasing granularity:

TaskOutputUnit of prediction
Classificationone labelimage
Object detectionboxes + labels + scoresobject
Semantic segmentationa label per pixelpixel (no instances: all "car" pixels are one class)
Instance segmentationa mask per objectobject × pixel
(Panoptic)both: stuff + thingseverything

The structural shift from classification: outputs are variable-length and spatial, which forces the machinery of Chapter 2 (how do you compare a predicted box set to a ground-truth box set?) and reshapes evaluation entirely (Chapter 3). Choosing the minimal sufficient task is a product decision you'll own: counting cars needs detection, not instance masks; measuring crack area needs semantic masks, not boxes — each step up the table costs labeling effort, compute, and annotation-quality sensitivity.

Chapter 2: The Shared Machinery — IoU, Anchors, NMS

Three concepts power every detector:

  • IoU (Intersection over Union): overlap area / union area of two boxes ∈ [0, 1] — the similarity metric of the field. Used to match predictions to ground truth (training assignment and evaluation), with thresholds as the vocabulary (IoU ≥ 0.5 = "correct" classically; 0.5:0.95 averaged = COCO's stricter modern standard).
  • Anchors / priors: predefined reference boxes tiled across feature-map positions at multiple scales/aspect ratios; the network predicts offsets from anchors rather than raw coordinates — turning detection into dense classification + regression on a grid. Anchor design (scales, ratios) encodes dataset priors; anchor-free detectors (FCOS, modern YOLO) replace them with per-location center/distance predictions — simpler, now standard, but the anchor mental model remains the key to reading the literature.
  • NMS (Non-Maximum Suppression): dense predictors fire many overlapping boxes per object; NMS keeps the highest-scoring box and removes neighbors above an IoU threshold, per class. Knobs that matter in production: the IoU threshold (crowded scenes need higher / Soft-NMS), the score threshold (the precision-recall dial — Ch. 3), and the fact that NMS is a post-process outside the network — exported models may or may not include it (a classic deployment surprise in Phase 07).

Chapter 3: Evaluating Detectors — mAP, Honestly

The metric pipeline, worth knowing mechanically because every published number hides choices inside it:

  1. Match predictions to ground truth greedily by score within a class: a prediction is TP if IoU ≥ threshold with an unmatched GT box, else FP; unmatched GT are FN.
  2. Sweep the score threshold → precision-recall curve per class.
  3. AP = area under that PR curve (interpolated); mAP = mean over classes; COCO's headline number additionally averages over IoU thresholds 0.5:0.05:0.95 (so "mAP 50.2" and "mAP@0.5 = 68" describe the same model — always check which).
  4. COCO also reports AP_small/medium/large — the size breakdown is where real systems are diagnosed (small-object AP is usually the disaster zone, and the reason FPNs exist).

The Phase 02 evaluation ethics apply unchanged: fixed eval sets, error bars across seeds where claims are close, and look at the errors — a TIDE-style breakdown (localization vs classification vs duplicate vs missed) tells you what to fix; a single mAP number doesn't.

Chapter 4: Two-Stage Detection — the R-CNN Line

The lineage that defined the field, compressed to its load-bearing ideas:

  • R-CNN (2014): external region proposals (selective search) + CNN per crop — accurate, absurdly slow (2K forward passes/image).
  • Fast R-CNN: run the backbone once; pool per-proposal features from the shared feature map (RoI Pooling) — proposals become cheap heads, not full passes. The shared-computation insight is the one that generalizes everywhere.
  • Faster R-CNN (the lab's subject): replace external proposals with a learned Region Proposal Network — a tiny conv head on the backbone predicting objectness + box offsets per anchor; top proposals feed the RoI head (classify + refine). End-to-end, ~real-time-ish, and still the accuracy-first template.
  • FPN (Feature Pyramid Network): lateral top-down connections build a multi-scale feature pyramid so small objects are detected on high-resolution levels and large on coarse — the single biggest small-object fix; standard in every modern detector.
  • RoIAlign (from Mask R-CNN, but used everywhere now): RoI Pooling's integer quantization misaligns features by pixels — fatal for masks, harmful for boxes; RoIAlign uses bilinear sampling at exact fractional coordinates. The lesson in miniature: dense prediction punishes coordinate sloppiness that classification never noticed.

Chapter 5: One-Stage Detection — YOLO and the Speed Frontier

One-stage detectors skip proposals: predict boxes + classes densely over the feature grid in a single pass.

  • The YOLO idea: divide the image into a grid; each cell (× anchor/scale) predicts objectness, class, and box — detection as one big regression. v1's coarse grid missed small/crowded objects; the series' evolution (v2–v8+) is a masterclass in iterative engineering: anchors then anchor-free, multi-scale heads (FPN-style necks — PANet), better label assignment (the quiet revolution: which predictions get assigned to which GT — SimOTA/TAL-class dynamic assignment), decoupled heads, and augmentation recipes (mosaic, mixup).
  • The historical accuracy gap vs two-stage was largely class imbalance: a dense grid is ~10⁴–10⁵ background predictions per handful of objects; easy negatives swamp the loss. Focal loss (RetinaNet) fixed it by down-weighting easy examples ($(1-p_t)^\gamma$ scaling) — after which one-stage matched two-stage and the speed/simplicity argument won the deployment world.
  • DETR-class (transformer set prediction, bipartite matching, no NMS) is the post-2020 branch — elegant, increasingly practical, worth knowing as the third paradigm even though this phase's labs are CNN-line.
  • Production framing for the YOLOv8 lab: you'll mostly fine-tune and deploy YOLOs, not invent them — the competencies are dataset preparation (label format, class balance), augmentation/hyperparameter sanity, reading the per-class/per-size AP breakdown, and the latency/accuracy operating-point choice (model size × input resolution).

Chapter 6: Semantic Segmentation — U-Net

Per-pixel classification needs per-pixel resolution — but backbones downsample 32×. The resolutions must come back:

  • Encoder-decoder: downsample for context (large receptive field, semantics), upsample (transposed conv or interpolation+conv) for resolution.
  • U-Net's contribution — skip connections from each encoder stage to the matching decoder stage: the decoder gets both deep semantics (what) and early-layer detail (precisely where). Without skips, boundaries blur into blobs; with them, U-Net segments crisply from tiny datasets — which is why it owns medical imaging (the capstone's domain) and why "U-Net-shaped" is now a generic architecture adjective (diffusion models included).
  • The practical loss story (Ch. 8 expands): per-pixel cross-entropy + Dice loss (overlap-based, robust to foreground/background imbalance — a 1% foreground organ drowns plain CE).
  • Evaluation: mIoU (per-class IoU averaged — pixel accuracy lies under imbalance, same disease as Phase 02 Ch. 6), boundary metrics where edges matter.

Chapter 7: Instance Segmentation — Mask R-CNN

The synthesis: Faster R-CNN + a third head. For each detected RoI, a small FCN predicts a binary mask within the box (28×28, class-specific). The design decisions that carry the lessons:

  • Detect-then-segment decomposition: instances come from the detector (boxes separate the objects); the mask head only answers "which pixels inside this box" — sidestepping the hard problem of separating touching same-class pixels that pure semantic segmentation can't express.
  • RoIAlign is mandatory (Ch. 4): mask quality is precisely where pooling misalignment became visible — the feature that motivated the fix.
  • Decoupled mask/class prediction: per-class binary masks with sigmoid (not a softmax competition across classes) — cleaner gradients, the paper's quiet ablation win.
  • Cost profile: heavier than detection (per-RoI mask head), annotation cost is the real bottleneck (polygon masks are ~10× box-labeling effort) — which loops back to Chapter 1's "minimal sufficient task" judgment.

Chapter 8: Training Dense Predictors — Losses and Imbalance

The cross-cutting training realities (all four labs hit them):

  • Multi-task losses: detection = classification loss + box-regression loss (smooth L1 / IoU-family losses — GIoU/DIoU/CIoU fix plain-IoU's zero-gradient-when- disjoint problem) (+ mask loss, + objectness), with weighting coefficients that are real hyperparameters.
  • Label assignment decides what each prediction learns: anchor-IoU rules classically; dynamic/learned assignment in modern YOLOs — when training is unstable or duplicates persist, suspect assignment before architecture.
  • Imbalance is the default condition: background≫foreground (focal loss, sampling ratios like RPN's 1:1 batch), class imbalance across categories (per-class AP exposes it), size imbalance (FPN + size-aware augmentation).
  • Augmentation is task-aware: geometric transforms must transform boxes/masks identically (the library does it — but verify visually; silent box-augmentation mismatch is a classic ruin), mosaic/scale jitter for small-object recall.
  • The debugging instrument: visualize predictions on training images early and often — anchor coverage, assignment, NMS behavior are all visible problems (Phase 00 Ch. 7's plot-everything ethic, at its highest value).

Chapter 9: Detection as Set Prediction — DETR to RT-DETR

Chapter 5 named DETR as the third paradigm; here is its machinery — because by 2026 the DETR line is a production option, not a curiosity.

The reframe. Every detector so far predicts densely — thousands of candidate boxes — and deduplicates afterward (NMS, Chapter 2). DETR (2020) instead treats detection as direct set prediction: output a fixed-size set of N predictions (N ≈ 100–300), trained so that each real object is owned by exactly one prediction slot.

  • Architecture: a CNN (or transformer) backbone produces a feature map, flattened into a token sequence (+ positional encodings, since attention is order-blind); a transformer encoder refines it; the decoder runs N learned object queries — trainable embedding vectors, one per prediction slot — which cross-attend into the image features ("where should I look?") and self-attend to each other ("what are the other slots already claiming?"). Each output embedding passes through small heads to a class distribution (including a "no object" ∅ class) and a box.
  • One-to-one bipartite matching: at training time the Hungarian algorithm computes the minimum-cost one-to-one matching between the N predictions and the ground-truth objects (leftover slots match ∅). The matching cost per candidate pair combines a class-probability term (how strongly this slot already predicts this GT's class), an L1 box distance, and a GIoU term — the same quantities the loss uses, so the assignment optimizes what the training optimizes. The loss is then computed only through that matching.
  • Why this deletes NMS: with exactly one slot assigned per object, any duplicate box is by definition matched to ∅ and trained to shut up; the self-attention among queries is the mechanism that lets slot \( j \) see that slot \( i \) already owns the object. Duplicate suppression becomes a learned, in-graph behavior instead of a hand-tuned IoU post-process — every NMS production knob from Chapter 2 (threshold tuning, crowded-scene suppression of real neighbors, the in-graph-vs-app-side export ambiguity) simply ceases to exist.

Why original DETR converged brutally slowly (500 COCO epochs vs the usual 12–36): first, each query's cross-attention starts near-uniform over every pixel and must learn to localize from scratch — dense detectors get localization for free from the grid structure; second, early in training the Hungarian matching is unstable — the slot assigned to a given object flips between iterations, so queries receive contradictory regression targets. Two fixes define the modern line:

  • Deformable DETR: replace global cross-attention with deformable attention — each query carries a reference point and predicts a small number of sampling offsets (e.g., 4 points per feature level) plus their attention weights, then attends only to those K bilinearly-sampled locations rather than the whole map. Cost drops from quadratic in pixels to linear in K, multi-scale feature maps become affordable (the FPN benefit, inside a transformer), and convergence speeds up ~10× because attention is born sparse and local instead of having to learn sparsity.
  • Denoising training (DN-DETR), in one paragraph: alongside the learned queries, feed noised copies of the ground-truth boxes as extra decoder queries whose assignment is known by construction — each must reconstruct its own GT. These bypass Hungarian matching entirely, so they deliver a stable "refine a rough box into this exact box" gradient from step one and stabilize what the matched queries learn. The DINO-DETR recipe (contrastive denoising, better query initialization) consolidated this into the standard training package.

RT-DETR (2023) is the "and now it's real-time" packaging, built from three decisions: run self-attention only on the top pyramid level (the smallest map, where quadratic attention is affordable and the semantics live); fuse the higher-resolution levels back in with a CNN-based cross-scale path instead of more attention; and initialize decoder queries by IoU-aware selection — pick top-K encoder features scored so that classification confidence reflects expected localization quality, handing the decoder good starting boxes instead of blank slates.

Its position vs YOLO in 2026, honestly: RT-DETR-class models sit on the modern YOLO accuracy/latency curve on server GPUs, with two structural advantages — no NMS-tuning surface (and deterministic latency: NMS cost varies with detection count, set prediction doesn't) and a clean single-graph TensorRT export — while YOLOs still win at the smallest edge/CPU budgets and in ecosystem depth (tooling, formats, community recipes). The open-vocabulary detection story and what foundation-model representations change for this line belong to Phase 06's warmup; this chapter stays at the detection-mechanics level.

Chapter 10: The YOLO Evolution, v8 → v10/v11 — and What Actually Changed

Honesty first: post-v8 YOLO releases are engineering increments, not revolutions — version numbers are partly release cadence, and reading the actual diffs beats believing launch blogs. What matters more than any single version is the shared modern settlement that v8-era detectors converged on:

  • Anchor-free, decoupled heads. Separate convolutional branches predict classification and box regression. Why coupling hurt: classification wants translation-invariant semantics ("what is it, wherever it sits"), regression wants translation-sensitive geometry ("where exactly is the edge") — one shared branch compromises both, and worse, it misaligns confidence with box quality, which directly degrades NMS (NMS ranks by score; a high score on a sloppy box beats a lower score on a tight one).
  • Task-aligned assignment (TAL): choose which predictions are positives by a combined score \( t = s^{\alpha},\mathrm{IoU}^{\beta} \) — classification score × localization quality. Why "aligned" beats center-based heuristics: geometric rules ("locations near the GT center are positives") encode a guess about which predictions should be good; TAL assigns based on which predictions are good at both subtasks, so the network is trained where it can win — and the classification score becomes an honest proxy for box quality, which is exactly what score-ranked deduplication needs.
  • Distribution Focal Loss (DFL) for box regression: instead of regressing each box side as a point value, predict a distribution over discretized offsets (n bins per side; the final coordinate is the expectation of the softmax). A point regression forces the network to commit when the true boundary is ambiguous — occlusion, motion blur, annotation noise; a distribution can be sharp where the edge is crisp and spread where it isn't. That gives cleaner gradients near the target (the loss focuses mass on the two bins bracketing it) and a usable localization-confidence signal for free.

The versions, at fair size: YOLOv9 is a training-dynamics contribution wearing a version number — GELAN (a re-engineered efficient-aggregation backbone) plus PGI, an auxiliary reversible branch that preserves information for gradient computation and is discarded at inference. YOLOv10's real contribution is NMS-free inference via consistent dual assignments — and the trick deserves a proper explanation. Training purely one-to-one (DETR-style, one positive per object) starves a dense head: most of the grid gets no positive gradient, and convergence suffers. v10 therefore trains two heads on the same backbone and neck: a one-to-many head with classic YOLO assignment (many positives per GT — rich, dense gradients) and a one-to-one head (single positive per GT). A consistent matching metric makes the one-to-one head's chosen positive coincide with the one-to-many head's top-ranked sample, so the two heads pull the shared features in the same direction instead of fighting. At deployment the one-to-many head is deleted: the one-to-one head emits non-duplicated boxes directly — DETR's no-NMS deployment story at YOLO cost. YOLO11 (2024) is the Ultralytics continuation — C3k2 blocks, partial attention modules, better parameters-per-mAP — same paradigm, same API; treat it as v8 with three more years of tuning.

The production reality most tutorials omit: licensing. Ultralytics code (v5, v8, v11) is AGPL-3.0. AGPL extends GPL's copyleft trigger to network use: users interacting with your service over a network counts as conveying the software, which obligates you to offer the corresponding source of your modified version — and depending on how your product embeds the library, plausibly the surrounding service code. Corporate legal teams treat that exposure as radioactive for closed-source products. The real options on the table: (1) buy the Ultralytics commercial license — that's their business model, and it's a legitimate answer; (2) use permissively-licensed alternatives — RT-DETR (Apache-2.0 in its original implementations), D-FINE, RF-DETR-class detectors and other Apache/MIT reimplementations; (3) train your own from the papers on a permissive codebase. Note the subtlety that weights trained with AGPL tooling inherit the question — assume they do until counsel says otherwise, and note that consuming RT-DETR through the ultralytics package is still AGPL-bound library use. Detector selection in 2026 is an accuracy × latency × license decision, and raising the third axis before legal does is a senior-engineer move.

Chapter 11: Multi-Object Tracking — Detection Over Time

Video turns detection into tracking: assign each object a persistent identity across frames. The dominant paradigm, tracking-by-detection, decomposes it cleanly — run a detector on every frame, then solve data association: which of this frame's boxes continues which existing track? Three components recur in every system: per-frame detection quality, a motion model predicting where each track should be now, and an assignment step matching predictions to detections.

  • SORT (2016) is the minimal honest implementation. Each track owns a Kalman filter whose state is the box's position and scale plus their velocities. Every frame it runs predict (advance the state under a constant-velocity model — uncertainty grows) then update (correct with the matched detection — uncertainty shrinks). The intuition: a principled weighted average between "where physics says it went" and "where the detector says it is", with weights set by each side's current uncertainty. Association is Hungarian assignment (the same algorithm as DETR's matching, Chapter 9) on the IoU between predicted track boxes and the new detections.
  • DeepSORT adds appearance: a ReID embedding network — trained on re-identification data so the same identity maps to nearby feature vectors — computes an embedding per detection, and the association cost mixes motion distance with appearance (cosine) distance. The point is occlusion robustness: motion predictions drift when a track goes unobserved, but appearance persists, so the pedestrian re-attaches to their old track when they emerge from behind the truck.
  • ByteTrack's key insight, explained properly: everyone had been discarding low-confidence detections before association — but an occluded true object usually still produces a box, just a low-score one, because the detector sees part of it. ByteTrack associates in two stages: high-confidence detections to tracks first; then the still-unmatched tracks against the low-confidence boxes. Occluded objects therefore keep their tracks alive with real observations instead of blind Kalman extrapolation. This single data-flow change outperformed systems with elaborate ReID machinery on the MOT benchmarks — the recurring engineering lesson: audit what your pipeline throws away before adding model capacity. (Its limit is full occlusion — no box at any score — which is where appearance and long-gap re-entry logic still earn their keep.)
  • One-liners for the current top of the leaderboards: OC-SORT re-anchors the Kalman state on observations to undo the drift accumulated while a track was occluded; BoT-SORT is ByteTrack plus camera-motion compensation and a ReID term.
  • MOT metrics, one honest sentence each: MOTA is detection-weighted — it is dominated by FP/FN counts, so a tracker can scramble identities constantly and barely lose MOTA; IDF1 is identity-weighted — it rewards keeping the right ID for a long time and forgives missed boxes more than MOTA does; HOTA decomposes performance into detection accuracy × association accuracy and takes their geometric mean — the balanced compromise, and the modern headline metric.
  • Production notes: the user-visible failure is the ID switch — analytics double-count a person, the alert fires on the wrong track — so watch IDF1/IDsw even when MOTA looks fine. And track lifecycle is a real tuning surface: birth (require N consecutive matches before confirming a track, or every detector false positive becomes a phantom) and death (max frames unmatched before deletion — too short kills tracks at every occlusion, too long glues a ghost track onto the next passerby). Both interact with frame rate and your scene's occlusion statistics; there are no universal defaults.

Chapter 12: Modern Training Recipes — What the YOLO Configs Actually Do

The config file is not boilerplate — it encodes a decade of empirical findings, and in practice the recipe is half the mAP. What each line buys:

  • Mosaic augmentation: four training images, randomly scaled and cropped, collaged into one. Three things it buys: every object appears at reduced scale (small-object exposure without collecting small-object data — recall Chapter 3's AP_small disaster zone); each sample carries four scenes' worth of variety (effective batch diversity at constant memory); and objects land in unnatural contexts, breaking background-co-occurrence shortcuts. Why close_mosaic disables it for the last N epochs: mosaic samples are distributionally wrong — collage seams, truncated boxes, impossible layouts — useful as mid-training regularization, harmful at the end; the final clean epochs let the box regressor and normalization statistics settle on test-like images, closing the train/test distribution mismatch.
  • MixUp (blend two images and their labels with a sampled weight) adds decision- boundary smoothness; copy-paste pastes segmented instances onto other images — it needs masks, which is why it's the signature augmentation of instance segmentation recipes, where it's reliably worth multiple mask-AP points.
  • EMA weights: maintain a shadow copy \( \theta_{\text{EMA}} \leftarrow d,\theta_{\text{EMA}} + (1-d),\theta \) (decay \( d \approx 0.999 \)) updated every step — and evaluate and deploy the shadow, not the raw weights. SGD's endpoint is one noisy sample from a trajectory bouncing around the optimum; the running average approximates ensembling along that trajectory for free. Every serious recipe has this on silently — if a released model beats your rerun and your curves jitter, check whether you evaluated raw weights while they shipped EMA.
  • Warmup + cosine LR: the first steps after initialization are the most dangerous — gradients are large and misdirected, the optimizer's second-moment estimates are unformed, and with a pretrained backbone under a random head, the head's garbage gradients flow straight into good backbone weights. Warmup ramps the LR from near-zero over the first epochs so nothing violent happens until statistics stabilize; cosine decay then anneals smoothly toward zero so late training refines instead of thrashing.
  • Multi-scale training: randomly resize inputs (±50%) across batches — resolution robustness, and the reason one checkpoint stays usable across the resolution operating points of Chapter 5.
  • Where albumentations fits: in custom (non-Ultralytics) pipelines it's the standard choice because its transforms are label-aware — boxes and masks move with the pixels, with clipping and minimum-visibility policies handled. Chapter 8's warning stands regardless: verify visually; a wrong bbox_params format silently corrupts every label while everything appears to train.

The closing honesty point: reproduction gaps between a paper's number and your run are more often recipe — augmentation schedule, EMA, assignment settings, training length — than architecture; and comparing detector A vs B under unequal recipes is meaningless. Diff the configs before crediting the architecture (Phase 02's controlled-comparison ethic, wearing a YOLO config).

Lab Walkthrough Guidance

Order: Lab 01 (YOLOv8) → Lab 02 (Faster R-CNN) → Lab 03 (U-Net) → Lab 04 (Mask R-CNN) — deploy-first, then anatomy, then pixels, then synthesis.

  • Lab 01: fine-tune on a custom small dataset end to end; deliverables are the per-class/per-size AP table, a TIDE-style error breakdown, and a latency/accuracy operating-point comparison (n/s/m model × 2 resolutions).
  • Lab 02: walk torchvision's Faster R-CNN internals — log and visualize RPN proposals, RoI assignments, and NMS inputs/outputs on a handful of images; swap the backbone and measure. The goal is anatomy fluency, not SOTA.
  • Lab 03: implement U-Net from scratch; train with CE vs CE+Dice on an imbalanced-foreground dataset and compare boundary quality; ablate the skip connections (the blurred-blob failure, personally witnessed).
  • Lab 04: fine-tune Mask R-CNN; inspect RoIAlign vs RoIPool on mask quality if feasible; report box AP vs mask AP and explain the gap on your data.

Success Criteria

You are ready for Phase 06 when you can, from memory:

  1. Choose the minimal sufficient task for three product scenarios and price the annotation delta.
  2. Define IoU, the anchor→offset scheme, and NMS with its production knobs.
  3. Walk the mAP pipeline including COCO's 0.5:0.95 convention and the size breakdown.
  4. Tell the R-CNN→Faster R-CNN story via the shared-computation and learned-proposal insights; explain FPN and RoIAlign.
  5. Explain why one-stage detectors needed focal loss, and what label assignment is.
  6. Draw U-Net and defend the skip connections; state Dice's role under imbalance.
  7. Describe Mask R-CNN's decomposition and its two key design choices.
  8. Explain DETR's query + Hungarian-matching mechanism, why one-to-one assignment deletes NMS, and the two fixes (deformable attention, denoising queries) for its slow convergence.
  9. State what actually changed post-v8 (TAL, DFL, v10's consistent dual assignment) and walk the AGPL decision as a license-vs-switch-vs-build call.
  10. Trace SORT → DeepSORT → ByteTrack by what information each adds, and say what MOTA, IDF1, and HOTA each reward.

Interview Q&A

Q: Your detector's mAP is 45 but customers complain it misses small objects. Diagnose and fix. The aggregate hides it: check AP_small (likely <15 while AP_large is 60+). Fixes in order of cost: raise input resolution (linear cost, immediate small-object gain); confirm FPN levels actually cover the size distribution (anchor/stride audit vs a histogram of GT sizes); augment with scale jitter/mosaic; tile-based inference for extreme cases (aerial imagery). And fix the metric: report size-stratified AP so the dashboard sees what customers see — the metric-design lesson is half the answer.

Q: Why does NMS exist, and when does it become the problem? Dense predictors are trained with many positives per object (assignment), so they fire duplicates by design; NMS deduplicates by score+IoU. It breaks: crowded same-class scenes (pedestrians — high IoU between real neighbors gets suppressed; Soft-NMS or higher thresholds), boxes of odd aspect ratios where IoU is a poor proximity measure, and deployment (is NMS inside the exported graph or app-side? mismatch = duplicated or missing boxes in prod). DETR's set-prediction-with-matching is the architectural answer to "what if no NMS" — name it as the trade: elegance vs the mature CNN ecosystem.

Q: Semantic vs instance segmentation — your client says "we need to know how much area is damaged." Which, and why? Semantic (U-Net-class): "how much area" is a per-pixel question with no instance identity needed — cheaper labels (paint the damage), cheaper model, simpler metric (IoU/area error). Instance segmentation earns its cost only when you must count or track discrete objects. Asking "is the unit of decision the pixel-region or the object?" is the consultative move the question is fishing for.

Q: Why does ByteTrack outperform trackers with fancy ReID? Because it fixed the information pipeline, not the model. Pre-ByteTrack systems discarded detections below a confidence threshold before association — and occlusion, the exact case ReID exists to handle, is precisely when a true object's score drops: the detector sees half a pedestrian and emits a low-score box. So the expensive appearance machinery was being asked to re-glue tracks that an upstream filter had broken. ByteTrack's two-stage association (high-confidence boxes to tracks first, then unmatched tracks against the low-confidence boxes) keeps occluded tracks fed with real observations, so identities don't break in the first place — prevention beat repair. The honest caveats: it requires the detector to produce some box under partial occlusion (full occlusion still needs appearance/long-gap logic — hence BoT-SORT), and low-confidence association leans on the motion gate to avoid latching onto background false positives — which conveniently rarely overlap a predicted track box. The meta-answer interviewers want: audit what your pipeline throws away before adding model capacity.

Q: Your YOLO's mAP is fine but users report duplicate boxes in production — walk the diagnosis, and how would a set-prediction detector change this? Fine offline mAP + duplicates in prod means the dedup machinery differs between eval and deployment. Diagnose in order: (1) Is NMS running at all in the deployed path? The exported ONNX/TensorRT graph may not contain NMS while offline eval used the framework's post-processing (Chapter 2's deployment surprise) — check the raw output count. (2) Parameter parity: deployed IoU threshold, score threshold, max-detections, and per-class vs class-agnostic NMS versus what eval used — per-class NMS lets near-duplicate class pairs both survive. (3) If NMS is genuinely identical, the model is emitting well-separated duplicates below the IoU threshold — that's an assignment/calibration symptom (Chapter 8): one-to-many training creates several positives per object, and if score doesn't track box quality (weak task alignment), two mediocre boxes both survive ranking. Note why the metric missed it: mAP barely punishes a duplicate (one extra FP among thousands), so add a TIDE-style duplicate-error count to the dashboard — metric design is half the fix. A set-prediction detector (RT-DETR, or YOLOv10's one-to-one deploy head) makes deduplication a learned in-graph behavior: no NMS config to drift between eval and prod, so this incident class disappears — replaced by its own trade: duplicates become a training problem you can't patch with a threshold at 2 a.m.

References