Phase 11 — Serverless: Functions, Triggers/Bindings & Durable

Difficulty: ⭐⭐⭐☆☆ (the model) → ⭐⭐⭐⭐⭐ (the judgment about scale, cold start, and determinism) Estimated Time: 1 week (12–18 hours) Prerequisites: Phase 10 (Event Grid / Service Bus — the event sources that trigger functions and whose backlog the scale controller reads), Phase 03 (Entra — Managed Identity authenticates a function's outbound calls, P12), and Phase 00 (control plane vs data plane; quantify the tradeoff). No Azure subscription required for the lab.

Why This Phase Exists

Phase 10 built the event plumbing — Service Bus queues, Event Grid delivery. This phase builds the thing on the other end of the wire: the compute that reacts to those events without a server you manage. "Serverless" is a marketing word; the engineering reality is two precise mechanisms, and a principal must own both.

The first is the scale controller. A senior engineer says "Functions autoscales." A principal can tell you how: the controller polls the trigger's event source, reads the backlog (queue depth, Service Bus message count, Event Hub lag), and computes a desired instance count from a per-instance target — then clamps it to the plan's ceiling and ramps it gradually so a burst doesn't stampede the source. And critically, on Consumption it scales to zero when the work drains, which is exactly where cold start comes from: there is no warm instance for the next event, so someone pays the startup latency. "Why is my first request after lunch slow?" is a scale-to-zero question, and the answer (Premium/Flex pre-warmed instances, or keep-alive) is a cost decision you make with numbers.

The second — and the star of this phase — is Durable Functions' replay model. This is the single most misunderstood mechanism in Azure serverless, and the one that separates people who've read about Durable from people who've operated it. A Durable orchestrator is not a long-running process sitting in memory. It is a function that gets replayed from its recorded history every time an event arrives — event sourcing applied to your workflow code. Each replay re-executes the orchestrator from the top; already-completed activities replay their recorded results, and only the next uncompleted step actually runs. The shattering consequence: your orchestrator must be deterministic. A DateTime.Now, a Guid.NewGuid(), a random(), an await httpClient.GetAsync() directly in the orchestrator — anything that returns a different value on the next replay — corrupts the workflow, because the replay no longer matches history. This is the bug that pages people at 2 a.m. and the question that filters principals in interviews. This phase makes you build the replay engine so the determinism rule stops being a warning in the docs and becomes a mechanism you can explain line by line.

What "Principal-Level" Means Here

A senior engineer writes a function that works. A principal understands the runtime well enough to:

• Predict scale and cold start from the trigger. Given "a queue with bursty 10k-message spikes," they can say how many instances the controller will reach, how fast it ramps, where it caps, and whether the workload needs Premium's pre-warmed instances to avoid cold-start latency on the critical path — with numbers, before the load test.
• Pick the hosting plan on sight. Consumption (pay-per-execution, scale-to-zero, cold start) for spiky background work; Premium / Flex Consumption (pre-warmed instances, VNet integration, no cold start) for latency-sensitive or network-isolated work; Dedicated (App Service plan) when you already pay for the compute. Choosing wrong is either a cold- start incident or a bill for idle instances.
• Reason about Durable replay from first principles. They can explain why an orchestrator is replayed, why that forces determinism, what a non-deterministic orchestrator does to a running workflow, and exactly which APIs (context.df.*, current_utc_datetime, new_guid) exist because the raw language primitives are unsafe under replay.
• Choose the right Durable pattern. Function chaining (sequential steps), fan-out/fan-in (parallel work + aggregate), async HTTP (long job + polling status endpoint), monitor (recurring check with a durable timer), and human interaction (wait for an external event with a timeout) — each is a named pattern with a known shape, and naming the right one is the design answer.
• Wire identity and events correctly. A function authenticates outbound with a Managed Identity (P12), is triggered by Service Bus / Event Grid (P10), and is fronted by API Management (P09) when it's an HTTP API — the serverless tier is the integration point of the whole platform, not an island.

Concepts

• The Functions model: trigger → function → bindings. Every function has exactly one trigger — the event that invokes it (httpTrigger, queueTrigger, timerTrigger, eventGridTrigger, serviceBusTrigger, eventHubTrigger). Bindings declaratively wire inputs and outputs (read a blob in, write a Cosmos document out) so you don't write client code for the plumbing. The host/runtime loads your function, invokes it on each trigger event, and applies the bindings.
• The scale controller. A separate component that watches the trigger's event source and decides instance count. Target-based scaling (the modern default for queue, Service Bus, Event Hub, Cosmos, Storage triggers) computes desired = ceil(backlog / target_per_instance) — a per-instance work target — then clamps to the plan's max instances and ramps gradually (a bounded number added per decision) so it doesn't stampede the source. It scales to zero on Consumption when the backlog drains.
• Cold start. Scale-to-zero's cost: when no instance is warm, the next event must spin one up (load the runtime, your code, dependencies), adding latency — typically hundreds of ms to a few seconds depending on language and package size. Premium and Flex Consumption keep pre-warmed instances ready to eliminate it (for a price).
• Hosting plans. Consumption — pay per execution (GB-seconds + executions), scale-to-zero, cold start, ~200 instance cap. Flex Consumption — newer: per-second billing, fast scale, always-ready instances option, VNet integration. Premium (Elastic) — pre-warmed instances, no cold start, VNet, longer runs. Dedicated (App Service) — runs on a plan you already pay for; no scale-to-zero. The plan is the cost/latency/isolation tradeoff.
• Durable Functions = stateful orchestration via event sourcing. An orchestrator coordinates activity functions (the side-effecting units). The Durable extension persists the orchestration's history (an append-only log of TaskScheduled / TaskCompleted events) and replays the orchestrator function from the top on every event — feeding recorded results for completed activities and executing only the next step. Because the orchestrator is re-run, it must be deterministic: no real clock, no randomness, no GUIDs, no I/O outside activities — use the context APIs (call_activity, current_utc_datetime, new_guid, durable timers) instead.
• The Durable patterns. Function chaining (A → B → C, each feeding the next). Fan-out/fan-in (schedule N activities in parallel, await all, aggregate). Async HTTP API (a starter returns 202 with a statusQueryGetUri; the client polls). Monitor (a recurring check on a durable timer until a condition is met). Human interaction (wait for an external event, raced against a durable timer for the timeout). Knowing which to reach for is the design.

Labs

Lab 01 — Functions Runtime: Scale Controller + Durable Replay (flagship, implemented)

→ Lab folder: lab-01-functions-runtime/

Integrated-Scenario Suggestions (carried through the whole track)

The phases compound. Keep these in mind as you build — each wires the serverless tier into the larger platform:

1. Order-processing pipeline (chaining + fan-out). A serviceBusTrigger function starts a Durable orchestrator that validates the order (activity), fans out to reserve inventory across warehouses in parallel (fan-out/fan-in), then charges payment and emits a confirmation event to Event Grid (P10). Every step is an idempotent activity so the orchestration is replay-safe. (→ P10, P12)
2. Async long-running job with polling. An HTTP-triggered starter kicks off a Durable orchestration and returns 202 Accepted with a status URL; the client polls until Completed. API Management (P09) fronts both the start and the status endpoints and validates the caller's JWT (P03). This is the async HTTP API pattern done right — no held connections, no timeouts. (→ P09, P03)
3. Scheduled monitor with human approval. A timerTrigger starts a monitor orchestration that polls a resource on a durable timer; on a threshold breach it waits for an external approval event raced against a 72-hour timeout (human interaction). The function authenticates to Key Vault and the target API via Managed Identity (P12). (→ P12, P10)
4. Event-driven enrichment at scale. An eventHubTrigger function processes telemetry; the scale controller reads partition lag and scales out (capped by partition count) to keep up, then back to zero overnight. You size the max instances, predict the cold-start behavior on the first morning event, and decide whether the latency SLA forces Premium pre-warmed instances. (→ P10, P00 latency budgets)
5. Secure serverless integration. Every function uses a Managed Identity (no stored secret) to read Key Vault (P12), is triggered by Service Bus / Event Grid (P10), and exposes HTTP through API Management (P09) — the serverless tier as the secure integration hub the JD describes ("integrate serverless applications with Azure security services"). (→ P09, P10, P12, P03)

Guides in This Phase

• HITCHHIKERS-GUIDE.md — the 30-minute orientation; read first
• WARMUP.md — the full primer; read slowly
• BROTHER-TALK.md — the candid version

Key Takeaways

• The scale controller turns a backlog into an instance count. desired = ceil(backlog / target_per_instance), clamped to the plan max, ramped gradually. It scales to zero on Consumption — which is where cold start comes from. Premium/Flex pay for pre-warmed instances to remove it. This is arithmetic you do before the load test, not after the incident.
• A trigger invokes; bindings wire. One trigger per function (the reason it runs); bindings are the declarative I/O that keeps plumbing out of your code. Reach for an output binding before you write a client SDK call.
• A Durable orchestrator is replayed, not run. It is re-executed from its event-sourced history on every event; completed activities replay their recorded results, only the next step runs. Internalize this and everything else about Durable follows.
• Replay forces determinism. No clock, no randomness, no GUIDs, no I/O in the orchestrator — use the context APIs (current_utc_datetime, new_guid, call_activity, durable timers) so each replay reproduces the same calls. A non- deterministic orchestrator silently corrupts a running workflow; Durable's non-determinism detection (your DeterminismError) is the guardrail.
• The patterns are the design vocabulary. Chaining, fan-out/fan-in, async HTTP, monitor, human interaction — name the right one and the architecture is half-drawn.

Deliverables Checklist

• Lab 01 implemented; all 22 tests pass against solution.py and your lab.py
• You can compute the scale controller's desired instance count for a given backlog and explain the per-step ramp and the scale-to-zero → cold-start link
• You can pick Consumption vs Premium vs Flex for a workload and justify it with the latency/cost/isolation tradeoff
• You can explain why a Durable orchestrator must be deterministic and what a DateTime.Now inside one does to a replay
• You can name all five Durable patterns and the shape of each
• You can draw fan-out/fan-in and explain how replay rebuilds state without re-running completed activities