« Track Overview · Warmup · Lab 01
Phase 13 — The Cloud & Infrastructure Backbone: Terraform, AKS, Mesh & Policy-as-Code
Answers these JD lines: "Lead the platform's cloud and infrastructure architecture across Azure (primary) and AWS, including infrastructure as code (Terraform), networking (private endpoints, peering, egress control), Kubernetes (AKS) and container orchestration, secrets management, and CI/CD pipelines" · "Deep proficiency in cloud-native platform engineering on Azure (preferred) and AWS, including Terraform, Kubernetes (AKS / EKS), Helm, service mesh (Istio, Linkerd), API gateways (APIM, Kong, Envoy), private networking, and policy-as-code (OPA, Azure Policy)."
Why this phase exists
The Principal Azure Cloud Engineer track covers the Azure control plane in general. This phase covers the AI platform's slice of it, and the slice has properties a normal workload does not:
- GPU node pools are expensive, scarce, gang-scheduled and slow to start — nothing about autoscaling a stateless web service transfers.
- Model endpoints are data-plane dependencies with residency implications. Whether an inference call leaves the region is a network-topology fact, and it must be provably true, not configured-and-hoped.
- Egress control is a security control here, not a hygiene one. An agent that can fetch a URL has an exfiltration channel (Phase 11), and the enforcement point is the network.
- Config is compliance. A routing rule that sends restricted data offshore is one commit away, and it takes effect everywhere at once — so infrastructure and policy changes need the same staged, reversible, audited treatment as code.
The phase is therefore built around one question: how do you make a topology claim you can prove?
Concept map
- Infrastructure as code: Terraform's model — a resource graph, state, plan as a diff of desired vs actual, and apply. Dependency ordering and why the graph is the important part.
- Drift: divergence between state and reality; detection, and the choice between reverting and absorbing. Drift as a run-state responsibility rather than a one-off.
- Kubernetes / AKS: the reconciliation loop as the same idea as Terraform's, run continuously; namespaces, workload identity, resource requests and limits.
- GPU node pools: taints and tolerations, device plugins, MIG partitioning versus whole-GPU, gang scheduling for tensor-parallel deployments, warm pools because engine startup is minutes.
- Helm: charts, values, releases, rollbacks — and why templating YAML is both the standard and a recurring source of incidents.
- Service mesh (Istio / Linkerd): sidecar or ambient; mTLS between workloads without
application change; retries, timeouts and traffic shifting; consistent-hash load balancing for
session affinity (Phase 01);
ext_authzas the PEP hook and thefailure_mode_allowsharp edge. - API gateways: APIM at the north-south boundary; Envoy as the mesh data plane; Kong as an alternative. Where the LLM gateway (Phase 04) sits relative to them.
- Private networking: private endpoints / Private Link for model endpoints, vector stores and key vaults; peering; DNS as the thing that actually breaks; egress control via firewall, NAT or proxy with an allow-list.
- Secrets: Key Vault, managed identity, workload identity federation — and the goal of no long-lived secrets anywhere, which is Phase 08's requirement expressed as infrastructure.
- Policy-as-code: Azure Policy for resource governance, OPA/Gatekeeper or Kyverno for admission, and the deny-by-default posture for anything touching restricted data.
- CI/CD: OIDC federation instead of stored credentials; staged rollout; supply-chain controls (SBOM, image signing, provenance, admission policies that refuse unsigned artifacts).
The lab
| Lab | You build | Proves you understand |
|---|---|---|
| 01 — The Resource Graph & the Reachability Prover | a Terraform-shaped resource graph with dependency ordering, a plan that diffs desired against actual, an apply with partial-failure semantics, and drift detection; a policy-as-code admission gate (deny-by-default for a restricted-data resource without a private endpoint, an unsigned image, or a public IP); and a network reachability prover that answers "can this workload reach that model endpoint, and does the traffic leave the region?" over a topology of VNets, peerings, private endpoints, NSG rules and egress policy — with a counter-example path when the answer is yes and should not be | that infrastructure claims must be provable, and that "we configured a private endpoint" is not the same statement as "no traffic leaves the region" |
115 tests, all green. Test contract: plan on an unchanged graph is empty; a cycle is rejected
at graph construction; a failed resource does not apply its dependents; drift is detected and
reported per attribute; the admission gate denies by default and names the rule; the reachability
prover finds a path through a peering that a naive per-VNet check misses; and every deny carries a
counter-example.
Documents
| Document | For |
|---|---|
| WARMUP.md | zero to principal on the platform's infrastructure — first principles, then the interview answers |
| HITCHHIKERS-GUIDE.md | the fast orientation: what the pieces are and how they fit |
| DEEP-DIVE.md | the mechanisms, in detail, with the failure modes |
| PRINCIPAL-DEEP-DIVE.md | the trade-offs you own at principal level |
| CORE-CONTRIBUTOR.md | what it takes to work on Terraform, Kubernetes, Envoy or Gatekeeper |
| STAFF-NOTES.md | judgment, review signal, war stories |
Deliverables checklist
- You can explain plan/apply/state/drift and why the graph is the important part.
- You can describe GPU scheduling differences: MIG, gang scheduling, warm pools, taints.
- You can place APIM, the mesh, and the LLM gateway relative to each other and justify it.
- You can explain how a private endpoint changes DNS, and what breaks when it does not.
- You can state an egress-control design and how it is proved rather than asserted.
- You can explain OIDC federation in CI/CD and what it removes.
- You can name three admission policies you would enforce for restricted-data workloads.
Key takeaways
- A topology claim must be provable. A reachability check that considers peerings is the difference between a configuration and a control.
- Reconciliation is one idea in two places — Terraform's apply and Kubernetes' controllers.
- GPUs break every autoscaling assumption: slow start, gang scheduling, scarcity, cost.
- Egress control is the exfiltration answer, and it lives in the network, not in a prompt.
- DNS is what actually breaks when you introduce private endpoints.
- No long-lived secrets, anywhere — managed identity, workload identity federation, JIT.
- Config is compliance. Stage it, sign it, make it reversible, and record who changed it.