Warmup Guide — TensorFlow

Zero-to-expert primer for Phase 04: the TF/Keras ecosystem as a production CV engineer meets it — the Keras model-building spectrum, tf.data input pipelines, and the TFLite path to edge deployment.

Table of Contents

Chapter 1: Why Learn Both Frameworks

Pragmatics, not ideology: research and most new training code is PyTorch; a large share of deployed production CV — especially mobile/edge/embedded — runs TF/TFLite, and teams maintain TF estates that need extending. The job market rewards bilinguals, and the translation exercise itself deepens both: every concept in this phase maps to a Phase 03 concept (Keras Layer ↔ nn.Module, tf.data ↔ DataLoader, SavedModel ↔ TorchScript/ONNX export), and the one genuinely different idea — graph tracing (Ch. 3) — is a concept PyTorch later adopted in its own form (torch.compile). Learn TF as "same physics, different conventions, one new idea."

Chapter 2: The Keras API Spectrum

Three ways to define a model, in increasing flexibility — choosing the lowest sufficient tier is the Keras idiom:

  • Sequential: a linear stack. Fine for toy models; outgrown immediately by anything with skips or multiple inputs.
  • Functional (the lab's focus, and the production default): build a DAG by calling layers on symbolic tensors: 
    inputs = keras.Input(shape=(224, 224, 3))
x = layers.Conv2D(64, 3, padding="same")(inputs)
x = layers.add([x, shortcut])              # skips, multi-branch — all natural
model = keras.Model(inputs, outputs)
    Because the graph is explicit data, Keras can validate shapes at construction time, model.summary() and plot_model render it, and serialization is exact — the payoff of declaring structure rather than executing it.
  • Subclassing (keras.Model with call()): PyTorch-style imperative freedom; costs you construction-time shape checking and easy graph introspection. Use when control flow demands it.
  • Mental model for the spectrum: Functional = "the model is a data structure" (inspectable, exportable); Subclassing = "the model is code" (flexible, opaque). TF tooling (export, TFLite conversion, pruning) prefers data structures — another reason Functional is the deployment-minded default. Note TF is NHWC (channels-last) by default — the opposite of PyTorch; conversion bugs love this.

Chapter 3: Eager, tf.function, and Tracing — the One Deep Difference

TF2 executes eagerly by default (PyTorch-like). Performance comes from @tf.function: the first call traces the Python function — executes it with symbolic tensors, recording ops into a graph — and subsequent calls run the compiled graph, skipping Python entirely.

The three working rules (each a classic bug when violated):

  1. Python side effects run at trace time only: a print() fires once; tf.print fires every call. A Python counter increments once. If you need stateful behavior, use tf.Variable.
  2. Retracing triggers: new input shapes or dtypes, and new Python argument values each create a new trace — passing a Python int batch size retraces per value (use tensors or input_signature with None dims).
  3. Data-dependent Python control flow doesn't survive tracing: if tensor > 0: fails or bakes one branch; AutoGraph converts simple cases to tf.cond/ tf.while_loop, but the honest fix is tensor ops (tf.where).

This is the same capture problem the model-accuracy track studies deeply (its Phase 01 lab-03 builds a tracing inspector; its Phase 04 covers PyTorch's answer) — TF made you confront it a decade early. model.fit wraps its train step in tf.function automatically, which is why custom Keras code is fast by default and why violating rule 1 inside a custom layer produces silent weirdness.

Chapter 4: tf.data — Input Pipelines as Dataflow Graphs

tf.data.Dataset is the DataLoader counterpart with a more explicit philosophy: the pipeline is itself a graph of transformations, optimized and executed in C++:

ds = (tf.data.Dataset.from_tensor_slices(paths_labels)
      .shuffle(10_000)                       # before batch; buffer size matters
      .map(decode_and_augment, num_parallel_calls=tf.data.AUTOTUNE)
      .batch(64)
      .prefetch(tf.data.AUTOTUNE))           # overlap input with training — always last

The performance grammar: map parallelism for CPU-bound decode/augment; prefetch to overlap producer and consumer (the single most important call — omitting it serializes input and training); cache() after expensive deterministic steps when the dataset fits memory/disk; order matters (shuffle before batch; augment after cache). For real datasets, TFRecord files (protobuf records, sequential reads) replace per-file opens — the same large-scale I/O lesson as every other track's data phase: sequential, sharded, parallel. Diagnosis tooling: the TF Profiler's input- pipeline analyzer tells you whether you're input-bound — check it before buying GPUs (the universal lesson again).

Chapter 5: Training with Keras — fit() and Custom Loops

  • The compiled path: model.compile(optimizer, loss, metrics) + model.fit(ds, validation_data=...) — with callbacks as the extension points: ModelCheckpoint (save best), EarlyStopping (patience on val metric), ReduceLROnPlateau/LearningRateScheduler, TensorBoard. Ninety percent of training needs are a callbacks list away — resisting the urge to hand-roll what a callback does is the Keras discipline.
  • Custom training: override train_step() on the model (keeps fit's machinery — callbacks, distribution — while customizing the step), or go fully manual with tf.GradientTape: 
    with tf.GradientTape() as tape:
    loss = loss_fn(y, model(x, training=True))
grads = tape.gradient(loss, model.trainable_variables)
optimizer.apply_gradients(zip(grads, model.trainable_variables))
    The tape is autograd made explicit and scoped — record only inside the with. Note the training=True/False argument (TF's explicit version of model.train()/.eval() — BatchNorm/Dropout switch per call, and forgetting it at inference is the same bestiary bug as Phase 03's).
  • Transfer learning mechanics mirror Phase 03 Ch. 7: base.trainable = False for the probe stage — with the documented BN subtlety (frozen BN should run in inference mode; Keras handles it when trainable=False, if you pass training=False correctly — read the official transfer-learning guide's BN section once).

Chapter 6: SavedModel — the Deployment Contract

model.export(path) (or tf.saved_model.save) writes the SavedModel: the traced computation graph(s) + weights + signatures (named typed entry points) — self-contained, Python-free, loadable by TF Serving, TFLite converter, TF.js, and other languages. Signatures are the API contract: input names, shapes (including which dims are dynamic), dtypes — inspect with saved_model_cli show --all.

The discipline (identical in spirit to every export path in this curriculum — ONNX in Phase 07, the model-accuracy track's Phase 05): export, reload, and verify outputs match the in-memory model numerically before calling it done; bake preprocessing into the exported graph where feasible (a Normalization/resize layer in the model) so the train-time and serve-time transforms cannot drift — training-serving skew, pre-empted at the artifact level (Phase 02 Ch. 8's pipeline lesson, now for tensors).

Chapter 7: TFLite — the Edge Path

TFLite is the mobile/embedded runtime: a compact flatbuffer format + interpreter with CPU/GPU/NNAPI/EdgeTPU delegates. The conversion ladder (each rung trades accuracy risk for size/speed):

  1. Plain conversion (FP32): graph ops → TFLite kernels; size from graph trimming.
  2. Dynamic-range quantization: weights INT8, activations FP32 — ~4× smaller, modest speedup, near-zero accuracy risk; the default first rung.
  3. Full integer quantization: weights and activations INT8 — needs a representative dataset (a few hundred real inputs) to calibrate activation ranges; required for integer-only accelerators (EdgeTPU, many MCUs/NPUs).
  4. Quantization-aware training when rung 3's accuracy drop is unacceptable.

This ladder is precisely the quantization story of the model-accuracy track (its Phase 03 derives the math; its Phase 06 covers NPU deployment at depth) — at this phase's level, the required competencies are: run the conversion, measure the accuracy delta on your eval set per rung (never ship an unmeasured conversion), and benchmark on-device (or with the TFLite benchmark tool) rather than trusting desktop numbers. The lab's deliverable is exactly that table: size / latency / accuracy per rung.

Lab Walkthrough Guidance

Order: labs 01→03 as numbered.

  • Lab 01 (Keras Functional): rebuild your Phase 03 ResNet-ish CNN functionally — skips force the Functional idiom; use model.summary() to verify against your PyTorch parameter count (a bilingual sanity check); train with the callbacks stack (checkpoint + early stop + TensorBoard).
  • Lab 02 (tf.data): build the decode→augment→batch→prefetch pipeline from image files; benchmark with/without prefetch, num_parallel_calls, and cache — produce the throughput table; convert to TFRecords and measure again. Demonstrate one tf.function retracing trap and its fix.
  • Lab 03 (TFLite edge deploy): take lab 01's model through conversion rungs 1–3 (with a representative dataset for rung 3); produce the size/latency/accuracy table; verify SavedModel reload-equivalence first (Ch. 6's ritual).

Success Criteria

You are ready for Phase 05 when you can, from memory:

  1. Place a modeling task on the Sequential/Functional/Subclassing spectrum with the data-structure-vs-code rationale.
  2. State the three tf.function rules and the bug each prevents; explain trace-time vs run-time.
  3. Write the canonical tf.data pipeline and justify each stage's order; name the input-bound diagnosis tool.
  4. Use train_step/GradientTape and the training= flag correctly.
  5. Describe SavedModel signatures and the export-verify ritual.
  6. Recite the TFLite quantization ladder with each rung's requirement and risk.

Interview Q&A

Q: Your tf.data pipeline feeds a GPU at 40% utilization. Walk through the fix. Confirm input-bound with the Profiler's input analyzer (don't guess). Then in order: prefetch(AUTOTUNE) present and last? map parallelized (AUTOTUNE) and is decode the hot spot (pre-resize on disk / TFRecords to kill per-file opens)? cache() after deterministic expensive work if it fits? shuffle buffer too large (startup stall) or batch too small? Same playbook as PyTorch DataLoader tuning — the framework changes, the producer-consumer physics doesn't.

Q: A model behaves differently in fit() than when you call it in a loop. Name the suspects. training= flag (BN/Dropout modes — fit sets it true, your loop may not), tf.function trace staleness (your loop mutated Python state that was baked at trace time — rule 1), different input dtypes/shapes triggering retraces or silent casts, and regularization losses (model.losses) that fit adds but a naive custom loop forgets. The grader is listening for trace-time-vs-run-time reasoning — it's the chapter's one deep idea.

Q: Why does full-integer TFLite need a representative dataset when dynamic-range doesn't? Weights are static — their ranges are known exactly at conversion. Activations are input-dependent — INT8 activation quantization needs calibrated ranges (scale/zero-point per tensor), which only real sample inputs can provide. Dynamic-range sidesteps it by keeping activations float at runtime. Skimping on representativeness (wrong domain, too few samples) shows up as clipped activations and accuracy loss — measure per rung, always.

References