Warmup Guide — Principal Architecture, FinOps & the Capstone

Zero-to-principal primer for Phase 15: the Well-Architected review as a repeatable method, the Architecture Decision Record that makes a call durable, the FinOps levers with their arithmetic, multi-region and DR design with RPO/RTO, and the secure event-driven serverless reference architecture that composes the entire track. This is the synthesis chapter — every engine you built plugs in here.

Table of Contents

• Chapter 1: The Principal's Job Is Composition
• Chapter 2: The Well-Architected Review as a Method
• Chapter 3: Scoring a Design (Five Pillars, From Evidence)
• Chapter 4: The Architecture Decision Record
• Chapter 5: FinOps — The Cost Levers, Quantified
• Chapter 6: Reliability Math — SLA Composition
• Chapter 7: Multi-Region & Disaster Recovery — RPO, RTO, the Spend
• Chapter 8: The Reference Architecture — The Whole Track in One Diagram
• Chapter 9: The Admission Gate — A Review You Can Run
• Lab Walkthrough Guidance
• Success Criteria
• Interview Q&A
• References

Chapter 1: The Principal's Job Is Composition

From zero. You have spent fourteen phases building engines: a JWT validator, an RBAC evaluator, a Service Bus broker, a Functions scale controller. Each is a unit. A senior engineer is fluent in units. The thing that separates a principal is not a fifteenth, bigger unit — it is the ability to compose the units into a platform, score the composition against conflicting requirements, and defend it in a room full of people whose job is to find the hole.

Why is composition the hard part? Because units don't add — they interact. Three 99.9% services in a request path do not give you 99.9%; they give you 99.7% (the math is in Chapter 6). A Managed Identity is a security win and an operability win and removes a cost (no secret-rotation tooling). A Private Endpoint buys you security and costs you DNS complexity that becomes the 2 a.m. incident. None of that is visible when you look at one unit. The principal sees the seams — exactly the lesson Phase 00 planted, now cashed in.

So this phase is not "learn a new Azure service." It is "learn to hold the whole platform at once, score it, and write down why." The lab is the executable form of that skill: a function that takes a deployment, runs every prior phase's check, and returns admit or reject, with a scorecard and an ADR.

Chapter 2: The Well-Architected Review as a Method

The Azure Well-Architected Framework (WAF) is five pillars:

The amateur treats WAF as a checklist to recite. The principal treats it as a method — a repeatable procedure you run on any design:

1. WORKLOAD FACTS  → RPS, payload, p99 SLO, availability target, data residency,
                     compliance regime, team size. (No numbers = no review.)
2. TWO PLANES      → control plane (tenant/MG/sub, RBAC, policy, IaC, secrets) AND
                     data plane (network, compute, events, storage, egress).
3. QUANTIFY        → compose the SLA, estimate $/month, size instances/partitions,
                     set the retry budget.
4. SCORE & TRADE   → score each pillar from the evidence; name the pillar you trade
                     and the number that justifies it.
5. FAIL IT         → blast radius, throttling limits, DR/region failover, rollback.
6. WRITE THE ADR   → the decision, the alternatives, what would reverse it.

The method is fractal: you run it on the whole platform, and you run it again on each component. It is also the same loop the lab automates — steps 1–4 are check_identity/check_authorization/check_policy/well_architected_score, and step 6 is the generated ADR.

Chapter 3: Scoring a Design (Five Pillars, From Evidence)

"Score it" sounds subjective. The principal move is to make each pillar a function of evidence you already computed, so the score is defensible and reproducible. The lab does exactly this:

• Reliability ← the composed SLA. Convert availability to nines: \( \text{nines} = -\log_{10}(1 - A) \). 99% → 2 nines, 99.9% → 3, 99.99% → 4. Map nines to 0..100 (five nines = 100). A 99.89% serial chain is ~2.96 nines → ~59/100 — and that low score is correct: three nines is a real ceiling, and the reviewer should see it.
• Security ← the posture score. Reward Managed Identity (no secret), private networking, Key Vault, customer-managed keys; penalize public exposure and stored secrets. The combination of public + stored secret is the worst case — one leaked config file becomes a breach.
• Cost ← the bill, against a documented reference. Cheaper is better, but the score makes the tradeoff visible: a multi-region active-active design scores low on cost and high on reliability, and the reviewer chooses with eyes open.
• Operational Excellence ← rewards the choices that reduce what you must run: Managed Identity (no rotation tooling), no stored secret (nothing to leak), Key Vault (centralized audited secrets), private networking (smaller surface to watch).
• Performance ← rewards redundancy (headroom under load) and a private fast-path (no cold public hop).

The teaching point — the same one from Phase 00's tradeoff modeler — is that the same two designs flip winners when the weights change. A payments workload weights reliability and security; an internal batch tool weights cost. "Best architecture" is a category error; "best for these requirements" is engineering. The scorecard makes the weights explicit so the argument is about the workload, not about taste.

Chapter 4: The Architecture Decision Record

A principal's decisions outlive their memory of them and must survive their absence. The ADR is the lightweight artifact that makes a decision reviewable, teachable, and reversible-on-purpose. The template you reuse forever:

# ADR-NNN: <short title>
Status: proposed | accepted | superseded by ADR-MMM
Date: YYYY-MM-DD

## Context
What forces are in play? The workload facts and the numbers (the five pillars).

## Decision
What we chose, in one sentence.

## Alternatives Considered
Each option, and the pillar it traded the wrong way for *our* numbers.

## Consequences
What gets better, what gets worse, what we must now monitor, and what would
make us revisit this.

The discipline the ADR enforces is exactly the review method: name the pillar, quantify both ends, choose, and write down what would change your mind. An architecture review (an interview's final round, and your day job) is a room pressure-testing your ADRs. The lab's admit() generates a one-paragraph ADR on every decision — admit or reject — so the platform's choices are auditable by construction. The Consequences line the generator emits ("strongest on security, weakest on reliability — the pillar to revisit") is the single most important sentence: it tells the next engineer where the design is fragile.

Chapter 5: FinOps — The Cost Levers, Quantified

FinOps is treating cloud spend as an engineering discipline: every dollar is attributed, and every optimization is a lever with a number. You cannot be the "cost optimization" pillar's owner without the arithmetic. The levers, with the math:

1. Right-sizing. Match instance size to actual utilization. If a workload uses 20% of a D8s (8 vCPU), it belongs on a D2s (2 vCPU). Saving: \( \text{save} = (\text{vCPU}{old} - \text{vCPU}{new}) \times \text{hours} \times \text{price} \). The number that decides it: the p95 utilization, not the average (you size to peak, not mean, or you die under load).

2. Reservations / Savings Plans. Commit to 1 or 3 years for ~30–60% off on-demand. The decision is a break-even: a 3-year reservation at 55% off pays back if the workload runs more than \( (1 - 0.45) \) of the term — i.e. almost always for steady-state baseline load. The lever's cost is flexibility: you've committed. Rule: reserve the baseline, burst on on-demand/spot.

3. Autoscale + scale-to-zero. Consumption-plan Functions scale to zero when idle — you pay per execution, not per hour. For a spiky workload at 5% duty cycle, that is a ~20× saving over an always-on VM. The cost is cold-start latency; the number that decides it: does the p99 SLO tolerate a ~1–3s cold start? If yes, Consumption; if no, pay for Premium/Flex pre-warming.

4. Storage lifecycle tiers. Hot → Cool → Archive. Hot blob ~\$0.018/GB-month; Archive ~\$0.001/GB-month — ~18× cheaper. A lifecycle policy that moves blobs to Archive after 90 days on a 100 TB cold dataset saves \( 100{,}000 \times (0.018 - 0.001) = \$1{,}700/\text{month} \). The cost: Archive has hours of rehydration latency, so only tier data you won't read soon.

5. Egress avoidance. Inbound is free; egress is ~\$0.087/GB and it ambushes multi-region/active-active designs (every cross-region replication and every user-download is billed). Keep compute next to data; cache at the edge (Front Door); do not chat across regions in the hot path. Egress is the term that dominates a multi-region bill that nobody estimated.

6. Dev/test shutdown. Non-prod that runs 24×7 is paying for 128 idle hours a week. Auto-shutdown nights and weekends: \( 1 - \frac{40}{168} \approx 76% \) saving on dev/test compute. The cheapest lever of all, and the one teams forget.

The principal's habit from Phase 00 carries straight in: find the unused headroom and spend it on cost. Most "expensive" platforms are paying for reliability or freshness no one asked for. Attribute the bill (tag every resource with an owner), make it a map, then pull the lever that moves the dominant term.

Chapter 6: Reliability Math — SLA Composition

Availability composes two ways, and conflating them is a classic interview tell.

Serial dependencies multiply. If a request must traverse every component and any one being down fails the request:

$$ A_{\text{serial}} = \prod_i A_i $$

Three 99.9% services in the path:

$$ 0.999^3 = 0.997002999 \approx 99.7% $$

— roughly three times the downtime of one of them. This is why "add another microservice in the hot path" is never free, and why the reference architecture's serial chain (Front Door → APIM → Functions → Cosmos) lands below its weakest link.

Redundant copies combine. The whole is down only if every copy is down, so the un-availabilities multiply:

$$ A_{\text{redundant}} = 1 - \prod_i (1 - A_i) $$

Two 99% replicas across availability zones:

$$ 1 - (1 - 0.99)^2 = 1 - 0.0001 = 0.9999 = 99.99% $$

— two cheap copies beat one expensive one. This is the math behind availability zones and active-active multi-region.

Nines convert to minutes. The number that makes an SLA real:

$$ \text{downtime}_{\text{min/month}} = (1 - A) \times 43{,}200 $$

Each extra nine is a different architecture, not a config flag — and roughly an order of magnitude more cost. The lab's composite_sla and downtime_minutes_per_month are this table, executable.

Chapter 7: Multi-Region & Disaster Recovery — RPO, RTO, the Spend

Two numbers define a DR posture, and a principal states both before designing:

• RPO (Recovery Point Objective) — how much data you can afford to lose, in time. RPO = 0 means synchronous replication (every write acknowledged in two regions before returning); RPO = 5 min means async replication with a 5-minute window of in-flight writes you'd lose on a sudden failure.
• RTO (Recovery Time Objective) — how long you can be down during failover. RTO = seconds means automated health-probe failover (Front Door); RTO = hours means a human runbook and IaC re-provisioning.

The DR tiers, cheapest to most expensive:

The spend climbs with both lower RPO and lower RTO, and the egress term (Chapter 5) dominates the active-active bill. Azure region pairs (e.g. East US / West US) give platform-coordinated updates and, for some services, geo-replication — but pairing is not failover; you still design the failover. And the principal's non-negotiable: test the failover. An untested DR plan is a hope, not a control — run the game day, fail the primary on purpose, measure the real RPO/RTO against the target.

The decision, as always, is a tradeoff with a number: what does an hour of downtime, or a 5-minute data-loss window, cost this business? A payments ledger buys active-active; an internal reporting tool buys backup/restore and sleeps fine.

Chapter 8: The Reference Architecture — The Whole Track in One Diagram

This is the design every phase has been pointing at. It is the canonical secure, event-driven, serverless platform, and the capstone's admission gate is its control point.

                         ┌─────────── CONTROL PLANE (who/what may exist) ───────────┐
                         │  Landing zone (P05): MG hierarchy, policy baseline        │
                         │  Identity (P03): Entra ID, OAuth2/OIDC, JWT               │
                         │  AuthZ (P04): RBAC (deny wins) + Azure Policy (deny)      │
                         │  IaC (P01/P02) deployed via CI/CD OIDC federation (P08)   │
                         └───────────────────────────┬──────────────────────────────┘
                                                     │ (the admission gate: this lab)
   internet                                          ▼
      │        ┌──────────────────────── DATA PLANE (the request path) ─────────────┐
      ▼        │                                                                     │
 ┌─────────┐   │  ┌──────────┐   ┌────────────┐   ┌────────────────┐   ┌──────────┐ │
 │ Front   │──▶│  │  APIM    │──▶│ Functions /│──▶│ Service Bus /  │──▶│ Cosmos / │ │
 │ Door    │   │  │ (JWT:    │   │ Durable    │   │ Event Grid     │   │ Storage  │ │
 │ (WAF,   │   │  │ aud/iss/ │   │ (P11)      │   │ (P10: peek-lock│   │ via      │ │
 │ global) │   │  │ scp,     │   │            │   │  DLQ, retry)   │   │ Private  │ │
 │ (P06/09)│   │  │ rate-lim)│   │            │   │                │   │ Endpoint │ │
 └─────────┘   │  └──────────┘   └─────┬──────┘   └────────────────┘   └────┬─────┘ │
               │                       │ Managed Identity (P12), no secret  │       │
               │                       ▼                                    │       │
               │                 ┌──────────┐   secrets/keys (P12)          │       │
               │                 │ Key Vault│◀──────────────────────────────┘       │
               │                 │ (RBAC,   │                                        │
               │                 │ purge    │   Observability (P13): App Insights,   │
               │                 │ protect) │   Log Analytics, KQL, alerts            │
               │                 └──────────┘   Reliability (P14): SLA, retry, CB    │
               └─────────────────────────────────────────────────────────────────────┘

Reading it as a principal:

• Identity is the perimeter. Front Door's WAF and APIM's JWT validation (aud/iss/scp) gate every request before compute runs. Compute authenticates to data and Key Vault with a Managed Identity — there is no stored secret anywhere in the picture.
• No public PaaS for PII. Cosmos and Key Vault sit behind Private Endpoints; the landing-zone policy baseline denies public PaaS for classified data — the exact rule the lab's check_policy enforces.
• Async is at-least-once with a safety net. Service Bus peek-lock + DLQ + Event Grid retry give at-least-once delivery; consumers are idempotent for exactly-once-effect. There is no "exactly-once delivery" claim — a principal never makes one.
• Quantify it. Serial SLA (Ch. 6): Front Door 99.99 × APIM 99.95 × Functions 99.95 × Cosmos 99.99 (zone-redundant) ≈ 99.88% — below a 99.95% target, so you either add regional redundancy on the weak link or relax the SLO. Show the math, then decide — and write the ADR.

Every box is an engine you built. The capstone is where they stop being boxes and become a platform you can admit deployments into.

Chapter 9: The Admission Gate — A Review You Can Run

The lab's admit() is the Well-Architected review method (Ch. 2) made executable. A deploy request arrives; the gate runs the steps in order and fails closed:

admit(deployment, platform_config):
   1. check_identity      → 401 if iss/aud/exp invalid          (authenticate)
   2. check_authorization → 403 if RBAC denies or deny-wins      (authorize)
   3. check_policy        → 403 if any `deny` violation          (validate)
   4. score               → composite_sla, security_posture, monthly_cost,
                            well_architected_score                (advise)
   5. ADR                 → generate the one-paragraph record     (record)

The order is not arbitrary: authentication before authorization (you cannot be "unauthorized" before we know who you are — that is why a bad-token and unauthorized request returns 401, not 403). The first three steps are gates (hard reject); the last two are advice (the scorecard and ADR the reviewer reads). A low security score does not reject — a deny policy does. That distinction — gate vs advice — is the heart of platform engineering: enforce the non-negotiables as code, advise on the rest, and let teams self-serve inside the guardrails.

Lab Walkthrough Guidance

Lab 01 — Platform Capstone: the Admission Gate, suggested order (file top-to-bottom):

1. Data model — AvailabilityLink (validate [0,1], non-empty), CostLine (reject negatives), Deployment (validate required fields, normalize tuples).
2. check_identity — iss, then aud (string or list — the confused-deputy guard), then nbf <= now+leeway, then exp > now-leeway. Fail closed; raise only on malformed args, return False on a merely-bad token.
3. check_authorization — deny-wins first (any in-scope deny matches → False), then default-deny over role assignments. Segment-aware scope_contains (the rg/rg2 trap) and *-wildcard action matching.
4. check_policy — walk the condition tree (field equals/notEquals/in/exists under allOf/anyOf/not); collect "<effect>: <name>" violations; has_deny_violation is the reject test.
5. check_security_posture — the weighted rules (MI 30, no-secret 25, private 25, Key Vault 10, CMK 10); build the findings; warn on the public+secret combo.
6. SLA + cost — composite_sla (redundant within a link, serial across links), downtime_minutes_per_month, monthly_cost.
7. well_architected_score — derive each pillar from the engines; clamp; add overall.
8. admit — the gate: run 1→2→3 in order, fail closed with the right status, then build the scorecard and generate the ADR. Test determinism.

Success Criteria

You are ready to call yourself a principal-capable architect when you can, from memory:

1. Run the WAF review method on a cold design prompt in five steps and produce a stage-by-stage scorecard.
2. Compose a serial-and-redundant SLA chain in your head and convert the result to downtime minutes/month (99.9% → 43; 99.99% → 4.3).
3. Name the six FinOps levers and, for each, the single number that decides whether to pull it.
4. State RPO and RTO for the three DR tiers and which workload buys which.
5. Draw the secure event-driven serverless reference architecture from memory, naming the control-plane and data-plane components and which phase built each.
6. Explain why the admission gate returns 401-before-403 and why a low security score is advice but a deny policy is a gate.
7. Write — and defend — the one-paragraph ADR the gate generates.

Interview Q&A

Q: "Design our secure event-driven platform." Where do you start? Not with a service. I start with workload facts — RPS, p99 SLO, availability target, data residency, compliance — because "secure" and "event-driven" are not requirements until they have numbers. Then I split into two planes: control (landing zone, RBAC, policy baseline, IaC + OIDC pipeline, Key Vault) and data (Front Door → APIM → Functions → Service Bus/Event Grid → Cosmos via Private Endpoint). Then I quantify: compose the SLA chain, estimate the monthly bill, set the retry budget. Then I score each pillar and name the one I trade — and write the ADR. The reference architecture is the default; the workload's numbers tell me where to deviate.

Q: Your SLA target is 99.95% and your serial chain composes to 99.88%. What do you do? First, I show the math so everyone sees the gap is real — three nines is the ceiling of a serial chain of four-nines parts. Then I have three honest options: (1) add redundancy on the weakest link (the one contributing most un-availability) — zone or region redundancy turns its term from \(A\) into \(1-(1-A)^2\); (2) remove a serial hop from the request path if one isn't load-bearing; or (3) relax the SLO if the business cost of the gap is lower than the cost of closing it. I pick with the number — what an extra nine costs vs what the downtime costs — and write the ADR. What I do not do is quote 99.95% and hope.

Q: A team wants exactly-once delivery across the whole platform. Your response? "Exactly-once delivery" doesn't exist across a network boundary — what exists is exactly-once-effect: at-least-once delivery (Service Bus retry, Event Grid backoff) plus an idempotent consumer (dedup key, upsert). So "everywhere" is a bigger operability bill than they think, re-earned at every external sink. I'd ask the cost of a duplicate and the cost of a loss per stream, because most don't need it — an analytics event tolerates at-least-once; a balance transfer does not. It's a per-data- product dial, sized by what an error costs, not a platform-wide default that taxes throughput and operability for streams that never needed it.

Q: "Make it cheaper." How do you make that concrete? Attribute first — tag every resource with an owner so the bill is a map, not a number. Then the levers are mechanical and each has a number: right-size to p95 utilization; reserve the steady-state baseline and burst on spot; scale Functions to zero for spiky load (if the p99 SLO tolerates cold start); lifecycle-tier cold storage to Archive (~18× cheaper); kill cross-region egress in the hot path; auto-shutdown non-prod nights and weekends (~76% off dev/test). Crucially I look for unused headroom — reliability or freshness no one asked for — and spend it on cost. Most "expensive" platforms are over-provisioned for a requirement that was never written down.

Q: What is the difference between a senior and a principal architect, in one sentence? A senior builds any one of the components correctly; a principal composes them into a platform, scores the composition against the workload's numbers, defends it in an ADR, and is the person the org trusts to admit what gets deployed into it.

Q: Walk me through what your admission gate does between a deploy request and a created resource. It runs the control-plane request path as code, fail-closed, in order: authenticate the token (iss/aud/exp — 401 if invalid), authorize the action against RBAC with deny-wins (403 if denied), validate the resource against the policy baseline (403 on any deny effect — e.g. public PaaS for PII). Only if all three pass does it score the deployment (composed SLA, security posture, monthly bill, the five pillars) and emit a generated ADR recording the decision and the pillar to watch. The gates are non-negotiable; the scorecard is advice. That is platform engineering: enforce the floor as code, advise on the rest, let teams self-serve inside the guardrails.

References

• Azure Well-Architected Framework — Microsoft Learn, learn.microsoft.com/azure/well-architected/ — the five pillars, the design principles, and the review tooling this phase automates.
• Cloud Adoption Framework (CAF) — Microsoft Learn, learn.microsoft.com/azure/cloud-adoption-framework/ — landing zones, governance, and the operating model the platform lives in (P05).
• FinOps Foundation — finops.org — the discipline, the Framework (Inform → Optimize → Operate), and the cost-lever vocabulary in Chapter 5.
• Azure Architecture Center — learn.microsoft.com/azure/architecture/ — the reference architectures (the event-driven serverless one mirrors Chapter 8) and the reliability/DR design guides (region pairs, RPO/RTO).
• Azure service SLAs — azure.microsoft.com/support/legal/sla/ — the per-service availability numbers you compose in Chapter 6.
• Michael Nygard, Documenting Architecture Decisions (2011) — the ADR pattern.
• The track's system-design/README.md (the four worked designs the capstone composes) and CHEATSHEET.md / GLOSSARY.md.