Phase 01 — Azure Resource Manager & the Deployment Engine
Difficulty: ⭐⭐⭐☆☆ (the algorithm) → ⭐⭐⭐⭐⭐ (the judgment about modes, idempotency, and async) Estimated Time: 1 week (12–18 hours) Prerequisites: Phase 00 (control plane vs data plane, the resource-ID shape, the Well-Architected lens). Comfortable reading JSON. No live Azure subscription required for the lab.
Why This Phase Exists
Phase 00 established the single most important sentence in this entire track: every
write in Azure is one idempotent PUT to a resource ID, routed through Azure Resource
Manager. This phase opens that sentence up and builds the machine inside it.
Here is the thing almost nobody can actually explain in an interview, even people who've
used Azure for years: when you click "Create" in the portal, or run terraform apply,
or az deployment group create, what literally happens? They can name the services. They
cannot draw the request pipeline (authenticate → authorize → validate → route to the
resource provider), they cannot explain how ARM knows to build the virtual network
before the network interface that sits in it, and they go pale when you ask why
re-running the same deployment is safe but --mode Complete once deleted their
production database.
That last one is not hypothetical. The Complete-mode story is a real, recurring outage shape: an engineer deploys a template that's missing a resource someone added by hand, in Complete mode, and ARM — doing exactly what it was told — tears that resource down to make the resource group match the template. The blast radius is "everything in the RG not in this file." Understanding why that's the defined behavior (not a bug) is the difference between fearing the tool and wielding it.
So this phase makes you build the deployment engine — the dependency graph, the topological scheduler, the idempotent PUT, the mode logic, the what-if diff — because once you've implemented the algorithm, ARM stops being magic and Terraform's plan engine (Phase 02), CloudFormation, Pulumi, and Bicep all reveal themselves as the same graph algorithm with different front-end syntax. This is the keystone phase for all of IaC.
What "Principal-Level" Means Here
A senior engineer writes a working ARM/Bicep template and runs the deploy. A principal understands the engine well enough to:
- Predict the deploy order from the template without running it — by reading the
implicit
reference()/resourceId()edges, not just thedependsOnthey typed. - Debug a stuck or failed deployment by reasoning about
provisioningState, the async-operation poll, and which resource in the graph is blocking its dependents. - Choose the deployment mode deliberately and explain the blast radius of each — and never reach for Complete mode without a reason and a what-if.
- Defend idempotency as an architecture property: why CI can re-run the same template
on every commit, why drift converges, and where idempotency quietly breaks (a resource
with a generated name, a
reference()to something deleted out-of-band). - Read a
what-ifdiff like a code review and catch the Delete nobody intended before it ships.
Concepts
- The ARM control-plane request pipeline — every write is a REST call to
…/providers/{ns}/{type}/{name}?api-version=…; ARM authenticates it (validates the Entra JWT), authorizes it (RBACActions − NotActions, then deny assignments win), validates it (Azure Policy candeny/modify/append), and routes it to the resource provider that owns the type. Get this pipeline cold; it's the spine of P03, P04, and P05. api-version— every RP exposes a dated contract; the request must name one, and the version pins the request/response schema. New properties only exist on new versions.- Asynchronous operations — most provisioning is long-running: ARM returns
201/202with aprovisioningState(Accepted → Running → Succeeded|Failed) and anAzure-AsyncOperation/LocationURL the client polls until terminal. "The deploy is done" means the poll reachedSucceeded, not that the PUT returned. - ARM template anatomy —
$schema,contentVersion,parameters,variables,resources,outputs, and the template functions (resourceId(),reference(),concat(),parameters(),[...]expressions) the engine evaluates at deploy time. - The dependency graph — the heart of it. Edges come from explicit
dependsOnand the implicit dependencies ARM infers when one resource's propertyreference()s orresourceId()s another. ARM topologically orders the graph, deploys independent resources in parallel, and detects cycles. - Idempotency —
PUTof desired state (not an imperative command) means re-applying converges: Create if absent, Modify if drifted, NoChange if identical. This is the property the entire IaC discipline stands on. - Deployment modes — Incremental (default; adds/updates, leaves un-templated resources alone) vs Complete (makes the RG match the template exactly — deletes the rest). The foot-gun, and the cleanup tool.
- What-if — the dry-run diff that classifies every resource as
Create/Modify/Delete/NoChange/Ignorewithout touching anything. Your PR-time safety net. - Bicep — a typed DSL that transpiles to ARM JSON; same engine, symbolic
references (so dependencies are inferred from
vnet.id, not hand-writtendependsOn), modules, loops, and far less ceremony. - Deployment stacks & deny-settings — a stack manages a set of resources as a unit, with managed deletion (drop a resource from the stack → it's deleted) and deny-settings that block out-of-band changes (drift protection at the control plane).
Labs
Lab 01 — ARM Deployment Engine (flagship, implemented)
| Field | Value |
|---|---|
| Goal | Build a miniature ARM deployment engine: extract explicit + implicit dependencies from a resource, build the dependency graph, topologically order it deterministically with cycle detection, apply the template as an idempotent PUT (Create/Modify/NoChange), implement the Complete-mode delete, and produce a non-mutating what-if diff |
| Concepts | Implicit vs explicit dependencies; Kahn's topological sort with deterministic tie-break; cycle detection; idempotent desired-state convergence; Incremental vs Complete blast radius; read-only dry run |
| Steps | 1. parse reference()/resourceId() out of (nested) property strings; 2. union with dependsOn → extract_dependencies; 3. build_dependency_graph with unknown-reference + duplicate-name validation; 4. deterministic topological_order (Kahn's, tie-break by name, raise on cycle); 5. deploy with idempotent PUT + mode logic; 6. side-effect-free what_if |
| How to Test | pytest test_lab.py -v — 25 tests covering implicit + explicit extraction, nested references, graph validation, topo correctness + determinism, cycle + self-cycle detection, idempotent re-deploy (all NoChange), Modify on changed props, Incremental-leaves vs Complete-deletes, and what-if-doesn't-mutate |
| Talking Points | "Walk me through az deployment group create." / "How does ARM order the VNet before the NIC if I never wrote a dependsOn?" / "Incremental vs Complete — which is default and why is the other dangerous?" / "Why is an ARM PUT idempotent and why does CI depend on it?" |
| Resume bullet | Built a dependency-graph deployment engine (implicit/explicit reference resolution, deterministic topological scheduling with cycle detection, idempotent PUT convergence, non-mutating what-if diff) modeling the Azure Resource Manager control plane |
→ Lab folder: lab-01-arm-deployment-engine/
Integrated-Scenario Suggestions (carried through the whole track)
The phases compound. Keep these in mind as you build — each plugs the deployment engine into a larger system:
- Landing-zone bootstrap — a deployment that lays down a management-group subtree, policy assignments, and a hub VNet. The order (policy before the resources it governs; networking before the workloads) is pure dependency-graph reasoning. (→ P05)
- Idempotent CI deploy — a pipeline that runs
what-ifon PR (the diff is the review) anddeployon merge, safely re-applying on every commit because the PUT is idempotent. (→ P08) - Drift control — a deployment stack with
denySettingsthat blocks out-of-band portal edits; reconcile detected drift by re-deploying desired state. (→ P05, P02 for the Terraform-state version of the same idea) - Multi-resource provisioning order — a VM that needs a NIC that needs a subnet that needs a VNet, plus a Key Vault the VM reads a secret from — the canonical "why did this deploy in that order" graph. (→ P06, P12)
- Cross-RP routing — a single template touching
Microsoft.Storage,Microsoft.Network, andMicrosoft.Compute, each routed to a different resource provider, each with its ownapi-versionand async behavior. (→ all later phases)
Guides in This Phase
- HITCHHIKERS-GUIDE.md — the 30-minute orientation; read first
- WARMUP.md — the full primer; read slowly
- BROTHER-TALK.md — the candid version
Key Takeaways
- Everything in Azure is a
PUTthrough ARM, and the deploy engine is, at its core, a dependency-graph topological scheduler over idempotent PUTs. Build that once and every IaC tool — Bicep, Terraform, Pulumi, CloudFormation — becomes a front-end you already understand. - Implicit dependencies are real edges.
reference()andresourceId()create ordering constraints you never typed; the engine (and you) must find them. - Idempotency is the load-bearing property of IaC. Desired-state PUTs converge, which is why you can re-run a template on every commit and why drift is recoverable.
- Mode is a blast-radius decision. Incremental is additive and safe-ish; Complete makes the RG exactly the template — and deletes the rest. Never run Complete without a what-if you've read.
- Async is the default. "The PUT returned" is not "the resource is ready"; the
provisioningStatepoll is. Debugging stuck deploys lives here.
Deliverables Checklist
-
Lab 01 implemented; all 25 tests pass against
solution.pyand yourlab.py - You can draw the ARM request pipeline (authenticate → authorize → validate → route) from memory
- Given a template, you can list its implicit + explicit edges and write the deploy order without running it
- You can explain Incremental vs Complete and the exact blast radius of each
-
You can explain why
PUTis idempotent and trace a re-deploy to all-NoChange -
You can describe the
provisioningStateasync-poll loop