Lab 05 — Productionize

Goal. Wrap the agent (Lab 02) in a FastAPI service (async + SSE streaming), store conversations in Cosmos DB (correctly partitioned), containerize with Docker, push to ACR, deploy to Azure Container Apps, and ship it through a CI/CD pipeline with an eval gate. JD4: FastAPI + Docker + Azure Containers + Cosmos DB + CI/CD. Read K07 first.

Stack. FastAPI/Uvicorn · Pydantic · AsyncAzureOpenAI · Cosmos DB (NoSQL) · Docker + ACR + Azure Container Apps · GitHub Actions (or Azure DevOps) · Bicep · App Insights. Local fallback. Cosmos DB emulator + Azurite; run the container locally; a CI workflow that runs on push.

Run it (offline, no Azure)

pip install -r requirements.txt        # fastapi + uvicorn + httpx + pytest
uvicorn app:app --reload               # real API at http://localhost:8000/docs
curl -s localhost:8000/health
curl -s -X POST localhost:8000/chat -H 'content-type: application/json' \
     -d '{"message":"How long to dispute a transaction?","conversation_id":"c1"}'
pytest -q                              # 10 tests (FastAPI TestClient — no server needed)

The FastAPI app is real (async, Pydantic, SSE streaming at /chat/stream, idempotency, health/readiness). The Cosmos store is an in-memory stand-in that models the partition-key + RU mechanics; swap CosmosLikeStore for azure-cosmos and the API is unchanged. The Dockerfile and ci-pipeline.yml show the production container + CI/CD with the AI eval gate.

Code tour

File	Teaches (K07)
app.py	async FastAPI, Pydantic contracts, SSE streaming, idempotency, health/readiness
store.py	Cosmos-shaped store: partition key, RU accounting, point-read vs cross-partition cost
resilience.py	retry w/ backoff honoring Retry-After (429), idempotency cache
Dockerfile	multi-stage, slim, non-root, no secrets, healthcheck
ci-pipeline.yml	GitHub Actions: test → scan → eval gate → ACR → canary deploy, OIDC (no secrets)
test_app.py	success criteria incl. idempotency, streaming, and partition-cost

Steps

API. POST /chat and POST /chat/stream (SSE) wrapping the Lab 02 agent; Pydantic request/response models; GET /health + /ready. Auth via Entra token verification dependency.
Async + resilience. Async clients end-to-end; timeouts; retry with backoff honoring Retry-After on 429; idempotency key on any state-mutating call; graceful degradation to a smaller model on outage.
Cosmos state. Container partitioned by conversationId; each turn a document; point-read the conversation by partition key; TTL for transient data. Justify the partition-key choice (K07 §6).
Docker. Multi-stage, slim base, non-root, no secrets in the image, healthcheck. Build and run locally.
Registry + deploy. Push to ACR; deploy to Azure Container Apps (min 2 replicas across zones, autoscale on concurrency, VNet-integrated, Managed Identity → RBAC to AOAI/Search/Cosmos).
CI/CD. GitHub Actions: lint/type-check → tests (mocked Azure) → Docker build + scan → AI eval gate (Lab 06) → push to ACR → deploy to staging → smoke test → canary to prod (ACA revision traffic split) with rollback. OIDC federation — no stored secrets.
IaC. Provision everything with Bicep (what-if in CI). No click-ops.
Observe. OpenTelemetry → App Insights: per-request trace (prompt, retrieval, tool calls, tokens, latency, cost); dashboards + alerts; PII-redacted logs.

Measurable result

A deployed, streaming /chat endpoint backed by Cosmos, reachable through the pipeline, with: a green CI run that fails on an injected eval regression, a canary rollout, and an App Insights trace showing tokens/latency/cost per request. Show the partition key giving single-partition point reads.

Stretch

Put APIM in front for throttling/audit; add a WAF.
Add Cosmos vector search for conversation memory (K07 §6).
Load-test and size PTUs vs PAYG for a target QPS (K00 §3).

Talking point

"FastAPI's async + SSE is the right fit because LLM calls are slow I/O and chat needs token streaming; I model Cosmos around access patterns (partition by conversationId for single-partition point reads), deploy on Container Apps with min-2 zonal replicas and Managed Identity, and ship through CI/CD that gates on an AI eval — so a quality or safety regression can't reach prod, and every release is reproducible via Bicep with no stored secrets."

Resume bullet

"Productionized an agentic banking assistant as an async FastAPI service with SSE streaming, Cosmos DB conversation state (partition-key-optimized), multi-stage Docker → ACR → Azure Container Apps (zonal HA, Managed Identity, VNet), and a GitHub Actions CI/CD pipeline with image scanning and an AI eval gate, provisioned via Bicep with OIDC (no stored secrets)."

AI Specialist