« Phase 13 · Warmup · Track Overview

Lab 01 — The Resource Graph & the Reachability Prover

The problem

An examiner asks one question: "can you prove that no inference call on customer data leaves the UAE?"

The architect points at a diagram. There is a private endpoint on the box marked "Azure OpenAI". The VNet has no internet egress. The answer is obviously yes.

It is not. Somewhere in that estate:

• the private endpoint exists and its DNS zone was never linked, so the FQDN still resolves to a public IP and every packet takes the public path — while everything works perfectly;
• the storage account has a private endpoint and public network access still enabled, which are two different settings and only one of them was changed;
• and the platform VNet is peered to a shared services VNet, which is peered to a legacy VNet in West Europe that has had a NAT gateway since 2021.

None of these error. None alert. Each one is invisible in the diagram, and each one makes the answer to the examiner's question "no".

You build the thing that answers it properly: a reachability prover that searches the whole topology and returns a counter-example path when the answer is yes and should not be.

What you build

Key concepts

Files

Run

pip install -r requirements.txt
pytest test_lab.py -v
LAB_MODULE=solution pytest test_lab.py -v
python solution.py

Success criteria

• All 115 tests green against your lab.py.
• A cycle — including a self-dependency — is rejected before any apply.
• The topological order is identical across two runs of the same graph.
• plan on an unchanged graph is empty.
• Changing a FORCE_NEW attribute produces a REPLACE that names which attribute forced it.
• Deletes are ordered dependents-first.
• A failed resource skips its entire transitive dependent set, and not independent ones.
• Successful resources before the failure remain in state.
• Drift is reported per attribute, in all three categories.
• A private endpoint with public access still enabled is denied.
• A policy function that raises produces a deny, not an allow.
• Every violated rule is reported, not just the first.
• NSG evaluation is first-match-by-priority; swapping priorities swaps the outcome.
• A private endpoint in a different subnet of the same VNet is reachable.
• An unlinked DNS zone makes the endpoint not count, with a stated reason.
• A path through two peerings into another region is found, with a rendered counter-example.
• A gang that does not fit is not partially placed.
• An unplaced workload always carries a reason.

How this maps to the real stack

Honest limits. The reachability model has no route tables — a UDR pointing 0.0.0.0/0 at a firewall NVA is the single most common real topology and it is not represented, so a "no egress" proof here is weaker than the real question. There are no service tags (Storage, AzureCloud), which is how most real NSG rules are written, and no service endpoints, which are a different mechanism from private endpoints with different residency properties. DNS is a boolean where reality is a zone, a VNet link and an A record, each independently missable. The scheduler has no bin-packing, no preemption and no priority classes, so it will not reproduce the fragmentation patterns that make real GPU scheduling hard. And plan diffs against state with no notion of ignore_changes, lifecycle blocks, or computed attributes — which are where real Terraform plans get their surprises.

Extensions

1. Add route tables. A UDR sending 0.0.0.0/0 to a firewall NVA. Now the prover has to follow the next hop, and "no internet egress" becomes a much harder claim. This is the extension that makes the model realistic.
2. Service tags. Storage, AzureCloud, Internet — expand them into prefixes and watch how much broader real NSG rules are than they look.
3. Model DNS properly. A private DNS zone, a VNet link, and a record. Then reproduce the failure where the zone exists, the record exists, and the link to the consuming VNet does not.
4. Terraform state locking. Two concurrent applies. Show the interleaving that corrupts state, then add the lock.
5. ignore_changes. Add lifecycle rules and discover the drift that is deliberately invisible — and the argument for and against it.
6. Gatekeeper in Rego. Express three of these policies in Rego and run them against real Kubernetes manifests. Compare the expressiveness and the reviewability.
7. Bin-packing and preemption. Add priority classes, then preempt a low-priority job to fit a gang. The interaction between preemption and gang scheduling is where real schedulers get complicated.
8. A cost model. Attach a price to each pool, and answer "what does a warm pool cost per month, and what does the alternative cost in p99 latency?" (Phase 05)

Interview / resume bullets

• "Built a network reachability prover over the platform's topology that answers 'can this workload reach that endpoint, and does the traffic leave the region?' with a counter-example path — which turned a residency claim from an assertion about configuration into a provable property."
• "Found and closed a residency exposure that a per-VNet review had cleared: two peering hops into a legacy VNet with an internet-facing NAT gateway."
• "Enforced infrastructure policy at admission with deny-by-default and fail-closed evaluation, so a bug in a policy denies rather than admits — and every denial names its rule and its reason."
• "Caught the private-endpoint-with-public-access-still-enabled misconfiguration as a policy rule, because a private endpoint does not close the public path and the difference is invisible in a diagram."
• "Modelled GPU capacity with gang scheduling and MIG partitioning, which surfaced the deadlock where a tensor-parallel deployment holds six of the eight GPUs it needs and makes no progress."
• "Established that a nine-minute node-to-serving time makes reactive autoscaling an outage with a graph, and sized a warm pool against the measured arrival distribution instead."