Phase 09 — RESTful APIs, JSON & Secure AuthN/AuthZ

Difficulty: ⭐⭐⭐☆☆ (the protocol) → ⭐⭐⭐⭐⭐ (the judgment about status codes, the validation-then-authorization order, and where the gateway sits) Estimated Time: 1 week (12–18 hours) Prerequisites: Phase 03 (OAuth2 / OIDC / JWT — this phase consumes the token that phase taught you to mint and validate) and Phase 00 (control plane vs data plane, identity is the perimeter). Comfortable reading JSON and HTTP. No live API Management instance is required for the lab.

Why This Phase Exists

The JD asks for one thing in plain language: "Develop and integrate RESTful APIs using JSON and implement secure authentication and authorization mechanisms (OAuth2, OpenID Connect)." That sentence hides the entire surface area of how the modern web talks to itself — and the part that separates a senior from a principal is not "can you write a controller," it is can you reason about the edge: the gateway that every request crosses before it reaches your code, the contract that says which method is safe to retry, and the exact moment a request becomes a 401 rather than a 403 rather than a 429.

Phase 03 built the token. This phase builds the thing that checks the token on every request and decides what happens next — Azure API Management, modeled as the machine it actually is: a policy pipeline (inbound → backend → outbound → on-error) that matches the route, validates the JWT, authorizes the scope, enforces the rate limit, and only then lets the call through. Almost everyone who has shipped an API can describe this vaguely. Very few can tell you, without hand-waving, why PUT is idempotent and POST is not (and why that decides whether a client may retry on a timeout), why a missing scope is 403 and a missing token is 401 (and why an SDK that confuses them causes a retry storm), or how a token bucket computes the Retry-After you owe a throttled client. Those are not trivia. "We returned 401 on an expired token and the client re-tried the same expired token forever" is a real outage. "Our retry-on-5xx logic double-charged customers because POST /charge isn't idempotent" is a real incident. This phase makes you build the gateway so REST stops being a vibe and becomes a sequence of decisions you can defend line by line.

What "Principal-Level" Means Here

A senior engineer wires an API behind APIM and it works. A principal understands the edge well enough to:

• Use the right method for the right semantics — GET (safe, cacheable), PUT and DELETE (idempotent: same call twice = same state), POST (not idempotent: creates), PATCH (partial, not guaranteed idempotent) — and reason from idempotency to "may the client retry this on a timeout?", which is the whole reason the distinction exists.
• Return the correct status code under pressure — 2xx success, 4xx you (the client) made a mistake, 5xx we (the server) did; and inside 4xx the trio that gets confused: 401 (not authenticated — who are you?), 403 (authenticated, not authorized — not allowed), 429 (authenticated, allowed, but too fast). The wrong code makes the client do the wrong thing.
• Place the gateway correctly and configure its pipeline — products / APIs / operations in APIM; the inbound policies that authenticate (validate-jwt), authorize (claim checks), throttle (rate-limit-by-key), and transform; the backend call; the outbound reshaping; and on-error — and explain why each policy lives in the section it does.
• Separate authentication from authorization at the edge — validate-jwt proves the token is real, current, and for this API (signature / iss / aud / exp); the scope check (scp delegated vs roles app-only) decides whether this caller may invoke this operation. AuthN is 401-or-pass; AuthZ is 403-or-pass. Conflating them is the most common edge bug.
• Do the rate-limit arithmetic — a token bucket with a capacity (burst) and a refill_per_sec (sustained rate); on empty, a 429 with a Retry-After that says exactly how long until one token accrues — and the difference between a rate limit (smooth the burst) and a quota (cap the long-window total for fairness/billing).

Concepts

• REST as resource orientation. A REST API exposes resources (nouns: /users/{id}, /orders) and acts on them with a small fixed set of methods (verbs), rather than RPC-style endpoints (/createUser, /getUserById). The URL names what; the method names the operation; the body and status carry the result. This is why GET /users/42 and DELETE /users/42 are the same resource, two operations.
• The HTTP methods and their semantics. Safe = no observable state change (GET, HEAD, OPTIONS). Idempotent = doing it N times leaves the same state as doing it once (GET, PUT, DELETE, HEAD — and crucially not POST). PATCH is partial update and is not guaranteed idempotent. The payoff: a client may safely retry an idempotent request after a timeout (it might already have succeeded) but must not blindly retry a POST (it might create a second resource) — idempotency is the property that makes the network's "did it happen?" ambiguity survivable.
• Status codes that mean something. 2xx success (200 OK, 201 Created, 204 No Content); 3xx redirect; 4xx client error (400 malformed, 401 unauthenticated, 403 unauthorized, 404 not found, 409 conflict, 415/406 content negotiation, 422 semantic, 429 too many requests); 5xx server error (500, 502, 503, 504). The split matters for retries: a client may retry 5xx (our fault, transient) and 429 (with backoff) but must not retry 4xx like 400/401/403 unchanged — the request itself is wrong.
• 401 vs 403 vs 429 — the three that decide client behavior. 401 Unauthorized is misnamed: it means un-authenticated — no token, or an invalid/expired one; the client should re-authenticate. 403 Forbidden means authenticated but not authorized — a valid token without the required scope; re-authenticating won't help, the client needs more scope/role. 429 Too Many Requests means authenticated and authorized but over budget — the client should back off for Retry-After seconds. Three different fixes; the gateway must return the code that triggers the right one.
• Pagination and content negotiation. Large collections are paged — offset/limit (simple, drifts under writes) or cursor/keyset (stable, the production default) — with a next link/cursor and often a total. Content negotiation lets the client ask for a representation (Accept: application/json) and declare what it sent (Content-Type); a mismatch is 406/415. JSON is the lingua franca; the contract is the schema.
• Azure API Management as the gateway. APIM sits in front of backends and exposes products (a bundle of APIs with a subscription and policies) made of APIs made of operations (method + URL template). Its core is the policy pipeline: every request flows inbound (where you authenticate, authorize, rate-limit, transform, route) → backend (the call to the origin) → outbound (reshape the response) → on-error (catch failures) — a programmable choke point that turns cross-cutting concerns into declarative policy instead of code in every service.
• AuthN vs AuthZ at the edge. validate-jwt authenticates: the signature proves the token wasn't forged, iss proves Entra issued it, aud proves it's for this API (skip this and you're a confused deputy), exp/nbf prove it's current. The scope check authorizes: scp (space-delimited delegated scopes — a user is present) or roles (app roles — a daemon, client-credentials) must contain what the operation requires. AuthN failure → 401; AuthZ failure → 403. Fail closed: anything you can't prove, deny.
• Token-bucket rate limiting. A bucket holds up to capacity tokens and refills at refill_per_sec; each request spends one; an empty bucket yields 429 + a Retry-After of (1 − tokens) / refill_per_sec. capacity is the burst you tolerate; refill_per_sec is the sustained rate you guarantee. A quota is the same idea over a long window (calls/day) for fairness and billing rather than burst smoothing.

Labs

Lab 01 — API Gateway: Routing, Scope Authz & Rate Limiting (flagship, implemented)

→ Lab folder: lab-01-api-gateway/

Integrated-Scenario Suggestions (carried through the whole track)

The phases compound. Keep these in mind — each plugs the gateway into a larger system:

1. SPA → APIM → API → downstream API. A browser app got an access token via auth-code + PKCE (Phase 03); APIM validate-jwt checks aud = your API and scp carries the user's scopes; an insufficient scope is a 403, an expired token a 401. This is Phase 03's validate_jwt running as a policy on every request. (→ P03)
2. Daemon → APIM → API. A nightly job uses client-credentials; its token carries roles, not scp, so the operation authorizes on app roles. The grant choice (P03) determines which claim the gateway authorizes on. (→ P03, P12)
3. Rate limiting as fairness and as defense. rate-limit-by-key per subscription smooths bursts and bulkheads tenants; a quota-by-key caps the monthly total for billing; a flood gets 429 + Retry-After, and a circuit breaker downstream (P14) sheds load when the backend is sick. The gateway is the first place reliability is enforced. (→ P14)
4. Managed Identity behind the gateway. APIM calls a backend that authenticates with a Managed Identity (P12) — no secret on the wire — and the backend reads Key Vault for anything else. The gateway authenticates the caller; MSI authenticates the backend. (→ P12)
5. Observability of the edge. Every 401/403/429/5xx is a signal; APIM emits to Application Insights and Log Analytics, and a KQL query (P13) over the gateway logs is how you find "tenant X is getting 90% 403s — they're missing a scope." (→ P13)

Guides in This Phase

• HITCHHIKERS-GUIDE.md — the 30-minute orientation; read first
• WARMUP.md — the full primer; read slowly
• BROTHER-TALK.md — the candid version

Key Takeaways

• The gateway is a pipeline, and the order is the policy. Match (404) before auth (no operation to authorize), auth (401/403) before rate-limit (429) (don't let probing cost rate budget), then the backend (200). Reorder it and you have a security or a fairness bug, not just a style choice.
• 401 ≠ 403 ≠ 429, and each triggers a different client action. 401 re-authenticate, 403 get more scope, 429 back off. The whole reason status codes are standardized is so a generic client does the right thing without reading your docs — pick the wrong one and you've broken that contract.
• Idempotency decides retryability. PUT/DELETE/GET are idempotent and safe to retry on a timeout; POST is not. Designing your write as idempotent (or giving it an idempotency key) is what makes the network's "did it happen?" ambiguity survivable.
• Authenticate, then authorize, and fail closed. Prove the token is real, current, and for you (validate-jwt); then check the scope. Anything you can't prove denies. Skip aud and you're a confused deputy; conflate 401 and 403 and clients retry forever.
• Rate-limit math is arithmetic, not magic. capacity = burst, refill = sustained, Retry-After = (1 − tokens) / refill. Quota is the same idea over a long window. Put the numbers in the design review, not in the post-incident review.

Deliverables Checklist

• Lab 01 implemented; all 29 tests pass against solution.py and your lab.py
• You can recite the gateway pipeline order and say what each stage's failure code is
• You can state the 401/403/429 rule and the client action each should trigger
• You can list which HTTP methods are idempotent and explain why that decides retryability
• You can implement a token bucket and compute the Retry-After it owes a client
• You can explain scp vs roles authorization and which grant produces each (ties to P03)
• You can place validate-jwt, rate-limit-by-key, and routing in the right APIM pipeline section and say why each belongs there