Lab 01 — Terraform State Engine

Phase: 02 — Terraform, CDKTF & IaC State Internals | Difficulty: ⭐⭐⭐☆☆ | Time: 3–4 hours

ARM (Phase 01) is stateless: every PUT re-derives reality from the resource provider, so there is no file to drift. Terraform is stateful: it remembers what it created in a terraform.tfstate file, and the entire tool is the diff between the world you wrote (config) and the world it remembers (state). This lab builds that engine — the per-resource diff, the ForceNew "replace not update" decision that destroys real infrastructure, the dependency ordering that flips for deletes, and the drift detector that catches the portal edit nobody told Terraform about.

What you build

A miniature Terraform plan/apply engine over three artifacts — config (desired), state (recorded reality), plan (the diff):

  • Resource — address ("azurerm_storage_account.main" = type.name, the state key), attributes (a flat dict), and the set of ForceNew attribute names.
  • diff_attributes(desired, current) — the leaf comparison: attribute → (old, new) for every changed / added / removed key.
  • plan_resource(desired, current, force_new) — the four-way decision: create (no state) · no-op (identical) · update (only mutable attrs moved) · replace (a ForceNew attr moved → destroy-then-create).
  • plan(config, state, force_new_by_type, deps) — the whole diff: classify every resource, mark orphaned state for delete, and order it — creates/updates/replaces in topological order, deletes in reverse topological order. Returns a Plan with .actions and a .summary count.
  • apply(plan, config, state) — execute the plan, return the new state without mutating the input (the property terraform apply depends on to be re-runnable).
  • detect_drift(state, real) — address → {attr: (state_val, real_val)} for resources whose real attributes diverge from recorded state: the out-of-band change terraform plan surfaces after its refresh.

Key concepts

ConceptWhat to understand
Three artifactsconfig = desired, state = recorded reality, plan = the diff; apply executes the plan
The plan algorithmper resource: compare config vs state → create / update / replace / delete / no-op
ForceNew → replacean immutable attribute changed ⇒ destroy-then-create (-/+), not in-place
Dependency DAGbuild from references; apply in topo order, destroy in reverse topo order
Driftreal ≠ state (out-of-band change); refresh surfaces it; state is a claim, not truth
Idempotencyre-planning a converged config is all no-op; apply never mutates the input

Files

FilePurpose
lab.pyskeleton with # TODO markers and signatures in place
solution.pycomplete reference; python solution.py runs a worked plan + drift example
test_lab.pythe proof — 28 tests; run them red, make them green
requirements.txtpytest only (pure stdlib otherwise)

Run

pip install -r requirements.txt
pytest test_lab.py -v                      # against your lab.py
LAB_MODULE=solution pytest test_lab.py -v  # against the reference
python solution.py                         # the worked example

Success criteria

  • All 28 tests pass against your implementation.
  • You can explain why test_plan_resource_force_new_boundary is the crux of the lab: the same diff is an update with an empty force_new and a replace the instant one changed attribute is immutable — and a replace destroys real infrastructure.
  • You can explain why test_plan_deletes_orphaned_state_in_reverse_order reverses the order: you cannot delete a subnet a NIC still sits in, so dependents die first.
  • You can explain why apply must not mutate its input state (a partially-applied run must leave the on-disk state recoverable; tests assert it stays untouched).
  • You can read a terraform plan and instantly flag every -/+ (replace) line as a potential outage before you type yes.

How this maps to real Azure (and real Terraform)

This labReal Terraform / AzureRM
state dict (address → attributes)terraform.tfstate JSON, stored in an Azure Storage blob backend
detect_driftthe refresh phase of terraform plan (reads each resource via the AzureRM provider, compares to state)
plan_resource → replacethe provider schema's ForceNew: true on an attribute (e.g. a storage account's location, account_kind)
deps mapedges Terraform infers from ${azurerm_x.y.id} references + explicit depends_on
reverse-order deletesTerraform walks the dependency graph backwards to destroy
apply returns a new stateterraform apply writes the updated state file (and takes a blob lease lock first so a teammate can't apply concurrently)

Two things the miniature deliberately omits but you must know:

  • Locking. Real remote state takes a lease on the Azure Storage blob for the duration of an apply; a second apply gets Error acquiring the state lock. Without it, two concurrent applies corrupt state. (WARMUP Ch. 6.)
  • create_before_destroy. A replace is destroy-then-create by default, which means downtime; the create_before_destroy lifecycle flag inverts the order to create the new resource first — but only works when names/addresses don't collide. (WARMUP Ch. 4.)

Extensions (for your own subscription / deeper study)

  • Add replace ordering. Make plan emit a replace as an explicit (address, "destroy") + (address, "create") pair, and support a per-resource create_before_destroy flag that flips their order. Test that a CBD replace creates before it destroys.
  • Add terraform import. A function that takes an unmanaged real resource and writes it into state (address → attributes) so the next plan is no-op instead of create — the real tool's answer to "ClickOps already made this."
  • Render the plan. Emit the + / ~ / -/+ / - symbols and a colored diff per attribute so your output reads like real terraform plan — then diff it against actual Terraform on a throwaway resource group.
  • Author it in CDKTF. Re-express the worked-example config in CDKTF (Python), cdktf synth it to cdk.tf.json, and confirm the synthesized JSON feeds the same plan/apply engine — the whole point of CDKTF.