Lab 01 — OCI Registry Engine + AKS Scheduler

Phase: 07 — Containers, Docker, ACR & AKS Internals | Difficulty: ⭐⭐⭐☆☆ | Time: 4–5 hours

A container image is not a binary — it is an ordered stack of content-addressed layers plus a config, named by a sha256 digest, listed by a manifest, and pointed at by a mutable tag. Get that one model right and ACR's whole behaviour — dedup, layer-reuse pulls, garbage collection, geo-replication — falls out as arithmetic on digests. This lab makes you build the registry, then build the AKS scheduler that bin-packs pods onto nodes and tells you, with numbers, why a pod is Pending.

What you build

• digest(data) — "sha256:" + hexdigest; the content-addressing primitive everything else stands on (same bytes → same name; one flipped bit → a new name).
• Registry — a content-addressable OCI registry: put_blob (store-by-digest with dedup), put_manifest (store config+layers, return a content-addressed manifest digest), tag (mutable pointer → digest), resolve (tag or digest → manifest digest), pull(ref, client_has=…) (return only the missing blobs — layer reuse), blob_count.
• garbage_collect(registry) — mark-and-sweep from the tags: a shared layer survives while any tagged image references it; untagging reclaims only an image's exclusive blobs. Returns the swept digest set.
• replicate(src, dst, name) — ACR geo-replication: copy a tag's manifest + only the blobs the destination is missing.
• schedule(pods, nodes) — an AKS-style first-fit bin-packer: place each pod (by name order) on the first node whose remaining cpu and mem satisfy its requests and whose taints are all tolerated; decrement capacity as pods land; a pod that fits nowhere is Pending (None).

Key concepts

Files

Run

pip install -r requirements.txt
pytest test_lab.py -v                      # against your lab.py
LAB_MODULE=solution pytest test_lab.py -v  # against the reference
python solution.py                         # worked example

Success criteria

• All tests pass against your implementation (pytest test_lab.py -v).
• You can explain why test_shared_base_layer_stored_once asserts 5 blobs, not 6 — the base layer two images share is stored exactly once because blobs are keyed by content.
• You can explain why test_gc_shared_layer_survives_until_last_tag_gone only reclaims the base after both referencing tags are gone — GC reachability is computed from the tags.
• You can explain why test_schedule_is_deterministic_by_pod_name produces the same result regardless of input order, and why worker/zeta lands as Pending.
• Given a docker pull that "downloads almost nothing," you can say why (cached base layers), and given a Pending pod you can name the two possible causes (capacity, taints).

How this maps to real Azure

What the lab simplifies (know these for the interview): real manifests are JSON with media types and sizes (and a manifest list / image index for multi-arch); ACR GC also honours soft-delete retention and never deletes a blob mid-pull; the kube-scheduler runs a scoring phase (least-allocated, spread, affinity) after the filter step we model, and also weighs nodeSelector, affinity/anti-affinity, topology spread, and pod priority / preemption. Our first-fit is the filter step, deterministic and good enough to teach the mechanism.

Extensions

• Add a manifest list (multi-arch index): a manifest whose "layers" are themselves manifest digests, one per os/arch; make pull select by platform.
• Add content trust: sign a manifest digest (HMAC over the digest with a key) and make pull refuse an untrusted/altered manifest — model Docker Content Trust / Notation.
• Add soft-delete: untag moves a manifest to a tombstone with a retention window; garbage_collect only sweeps tombstones past their window; restore brings one back.
• Add a scheduler scoring pass: among nodes that pass the filter, pick least-allocated (spread) or most-allocated (bin-pack to scale down) and show the placement differs.
• Add pod priority + preemption: a high-priority Pending pod evicts a lower-priority pod to make room; show the evicted pod becomes Pending.

Resume / interview bullets

• Built a content-addressable OCI registry (digests, manifests, mutable tags) with cross-image layer deduplication, a missing-blob pull protocol, mark-and-sweep garbage collection, and missing-only geo-replication — the ACR data plane modelled.
• Implemented an AKS-style bin-packing scheduler (requests, taints/tolerations, capacity accounting) that explains Pending pods as requests-vs-capacity arithmetic.