Concepts Rapid-Fire

~50 screening questions with complete, say-out-loud answers. Read every one aloud tonight. Grouped by theme. Each answer is interview-length (2–5 sentences) — deeper detail lives in the linked knowledge modules.

Table of Contents

A. Azure platform

Q: What is Azure OpenAI Service and how does it differ from OpenAI's API? Same models (GPT-4o, embeddings, etc.) but inside Azure's compliance, networking, and identity boundary, with Microsoft's enterprise data commitments — your data isn't used to train the models, processing stays in-region, you authenticate with Entra ID, and you can lock it behind Private Endpoints. You call a deployment you created, not a raw model name. That envelope is why banks use it.

Q: PAYG vs PTU? PAYG/Standard bills per token on shared capacity with variable latency — good for dev and spiky low volume. PTU (Provisioned Throughput Units) reserves capacity for predictable latency/throughput, billed monthly — the right call for a customer-facing assistant at scale.

Q: How do you authenticate without keys? Managed Identity: the app gets an Entra token automatically (Azure rotates it), and I grant that identity least-privilege RBAC on each resource (e.g. Cognitive Services OpenAI User). Secrets that can't be avoided go in Key Vault. No keys in code — a banking requirement.

Q: What's Azure AI Foundry? Azure's unified platform for building, evaluating, deploying, and governing AI — model catalog, evaluators/observability, prompt flow, and Foundry Agent Service (a managed agent runtime built on the OpenAI Responses API). Rebranded "Microsoft Foundry" in 2026.

Q: You're getting 429s. What's happening and what do you do? You've exceeded the deployment's TPM or RPM quota. Short term: backoff with jitter honoring Retry-After, queue, shed non-critical load. Longer term: raise quota, spread across regions, move hot paths to PTU, and cut tokens via caching/tiering/context-trimming.

Q: How do you meet data-residency requirements? Pin Azure OpenAI to Regional/Data-Zone deployments in the mandated geography, keep Search/Cosmos/Storage in-region, and use the abuse-monitoring opt-out for sensitive data. For a UAE bank that's typically UAE North.

B. GenAI / LLM fundamentals

Q: Why do LLMs hallucinate? They sample the most plausible next tokens from a learned distribution; they have no truth model. When asked for facts the weights don't reliably hold, the most plausible continuation is a fluent fabrication. Fix by grounding (RAG/tools), instructing abstention, verifying groundedness, and constraining output.

Q: RAG vs fine-tuning? RAG/tools inject knowledge at runtime — use it for facts that change or are private (a bank's policies, balances), with citations for audit. Fine-tuning bakes behavior into weights — use it only for stable style/format, never for changing facts.

Q: What's a token and why care? A sub-word unit (~4 chars / ¾ word in English; ~2× more for Arabic). You bill, budget context, and pay latency in tokens — output tokens dominate latency since generation is one token at a time.

Q: What's an embedding? A vector representation of text where semantically similar texts are near each other (cosine similarity). It powers vector/semantic retrieval. text-embedding-3-large is 3072-dim; the dimension sizes your vector index.

Q: Temperature? Controls randomness of sampling. ~0 is near-deterministic — the banking default for extraction/classification/grounded answers. Higher adds diversity you rarely want in a bank.

Q: What is function/tool calling? The model returns a structured request to call one of your functions with arguments it generates; your code executes it and feeds the result back. It's how the model gets live/exact data and takes actions — and the atom of agents.

Q: Structured output — why and how? To get machine-parseable results, constrain the model to a JSON Schema (response_format, strict) and validate with Pydantic; retry on validation failure. Never regex-scrape prose for downstream systems.

C. RAG & Azure AI Search

Q: Walk me through a RAG pipeline. Offline: extract (Document Intelligence for scans) → chunk (~500 tokens, overlap, +metadata) → embed → index in AI Search. Online: embed the query → hybrid retrieve (BM25+vector, RRF) → semantic rerank → top-5 → grounding prompt with citations + abstention → Azure OpenAI → groundedness check → return.

Q: Vector vs keyword vs hybrid? Keyword (BM25) nails exact identifiers but misses paraphrase; vector captures meaning but misses exact tokens. They fail complementarily, so hybrid fuses both with RRF, then a cross-encoder semantic ranker reorders the top candidates. Hybrid+semantic is the enterprise default.

Q: What is RRF? Reciprocal Rank Fusion: each doc scores sum 1/(k+rank) across the keyword and vector result lists. It uses ranks not raw scores, so it fuses the two systems without needing to calibrate their score scales.

Q: Why retrieve-then-rerank? Bi-encoder vectors are cheap and scale to millions but rank approximately; cross-encoders read query+doc together for accurate relevance but are too expensive to run over everything. So retrieve cheaply to ~50, rerank expensively to ~5 — lifting the right chunk into what the LLM reads.

Q: How do you chunk? Structure/recursive-aware ~300–800 tokens with 10–20% overlap, plus metadata (source, page, language, ACL). Too small loses context; too large dilutes embeddings and precision. For statements/forms, chunk by layout/section.

Q: How do you stop a user retrieving documents they can't see? Security-trim at retrieval: ACL labels on each chunk + a filter constraining results to the user's groups. Enforced in the query, never by asking the LLM to "not reveal" — the model is never an access boundary.

Q: HNSW? The ANN graph algorithm AI Search uses for vector search — a multi-layer navigable graph searched greedily, giving near-nearest-neighbors in roughly log time. Tune m/efSearch to trade recall vs latency/memory.

Q: Integrated vectorization? An AI Search indexer + embedding skill that chunks and embeds your data at index time (and embeds the query at search time) — less glue code, fewer moving parts in a regulated pipeline.

D. Agentic AI

Q: What makes something "agentic"? An LLM in a loop that takes actions (tool calls), observes results, and decides the next step until a goal is met — orchestrating a multi-step task, not just answering. ReAct (reason+act) is the canonical pattern.

Q: Why LangGraph over a plain loop? A bare loop has no durable state (crash loses the workflow), no human-in-the-loop, no observability, and can run away. LangGraph models the agent as a graph with shared state, a checkpointer (durable, resumable, auditable), conditional edges/cycles, and interrupts (pause for approval, resume later) — the production/compliance properties a bank needs.

Q: LangChain vs LangGraph? LangChain is the toolkit (model wrappers, loaders, splitters, retrievers, output parsers, simple chains). LangGraph is built on it, adding stateful, cyclic, human-in-the-loop orchestration for real agents.

Q: What is the Microsoft Agent Framework? The 2025 convergence of Semantic Kernel and AutoGen into one open-source SDK + runtime (Python/.NET), Foundry-native — Microsoft's go-forward agent framework. SK and AutoGen are now its predecessors (maintenance mode).

Q: When Foundry Agent Service vs self-hosted LangGraph? Foundry Agent Service for a managed runtime with built-in memory, tools (incl. AI Search grounding), observability, continuous eval, and governance under the Foundry/Entra envelope. Self-hosted LangGraph when I need full control of the loop, custom state backends, or portability. They interop via MCP.

Q: What is MCP? Model Context Protocol — an open client–server standard for exposing tools/resources/prompts to any LLM host. Write a tool (e.g. core-banking lookup) once as an MCP server with central auth; any MCP-aware agent consumes it without rewrites.

Q: How do you keep an autonomous agent safe in a bank? Least-privilege tools that enforce the user's permissions; no money facts from the model; human-in-the-loop on irreversible actions; deterministic guardrails for hard rules; full audit of prompts/thoughts/tool calls; step/cost limits; Content Safety + prompt-injection defense; graceful degradation to human handoff.

E. Document Intelligence & Vision

Q: OCR vs Layout vs Extraction? OCR = text + bounding boxes. Layout adds structure (tables, selection marks, reading order). Extraction adds semantics (named fields + confidence) via prebuilt/custom models. Use Layout for structure-aware RAG chunking; Extraction for the specific fields your system needs.

Q: How do you handle low-confidence extractions in a bank? Risk-weighted confidence thresholds per field; below threshold → human review in a UI highlighting the field on the source image (bounding regions); deterministic validation (IBAN mod-97, balance math); log corrections; track straight-through-processing rate. Never auto-process a low-confidence money/identity field.

Q: When OpenCV vs Azure Vision vs a multimodal LLM? OpenCV for deterministic preprocessing/geometry (deskew, crop, threshold) — cheap, explainable. Azure Vision/Document Intelligence for robust OCR/structure/fields. Multimodal LLM (GPT-4o) when the task needs reasoning/judgment over the image. Combine: clean → extract → reason.

Q: How does YOLO work? Single forward pass of a CNN predicting bounding boxes + class probabilities over a grid, with Non-Max Suppression removing duplicates — hence real-time detection. In banking, fine-tune it to localize custom objects (signature, MICR, tampering) then crop ROIs for OCR.

Q: What changes for Arabic documents? RTL/bidi layout, cursive position-dependent letter shapes and diacritics stress OCR; Eastern Arabic numerals need normalizing; MSA vs dialect varies; multilingual embeddings and an Arabic eval set with native reviewers are required. Test on real samples — English assumptions don't transfer.

F. Prompting & Safety

Q: What's "advanced" prompt engineering? Treating prompts as versioned, tested production artifacts: structured system prompts (identity, grounding rules, scope/refusals, format), few-shot examples, structured output/tool schemas, grounding-with-citations, injection defense, and an eval set that gates changes — not artisanal wording.

Q: How do you stop the bot inventing a fee or rate? Grounding prompt (answer only from sources, never state an absent number, cite, abstain) plus architecture (live figures from tool calls, not the model) plus a groundedness eval gate. Wording + architecture + eval together.

Q: What is prompt injection and how do you defend against it? Hijacking the model via instructions in user input (direct/jailbreak) or in retrieved/uploaded content (indirect). Defenses are layered: system/user trust separation and treating retrieved content as untrusted data; Content Safety Prompt Shields; and least-privilege tools + human approval so an injection can't move money; output filtering; red-team in CI. Architecture limits blast radius — wording alone can't.

Q: Show vs hide chain-of-thought? Use CoT for multi-step reasoning but keep it server-side and logged for audit — don't show raw chain-of-thought to customers; it's unreliable as an explanation and can leak. Return clean answers.

G. Production, Cosmos & CI/CD

Q: Why FastAPI over Flask? LLM calls are slow I/O; FastAPI's async serves many concurrent requests per worker while they await the model and streams tokens (SSE) for responsive chat, with typed Pydantic contracts and auto OpenAPI. Flask/Streamlit are for quick PoCs.

Q: How do you choose a Cosmos partition key? High cardinality, even access, matching the dominant query — usually conversationId (or userId) — so load spreads, reads are single-partition point reads, no hot partition, ≤20GB/partition. It's effectively irreversible, so model access patterns first.

Q: What are RU/s? Request Units per second — Cosmos's throughput/cost currency. Every op costs RUs (a 1KB point read ≈ 1 RU; queries/writes more); exceed your provisioned RU/s and you get 429. Design access patterns (point reads, right indexes, single-partition) to minimize RU.

Q: Cosmos consistency levels? Five from Strong to Eventual. Session (default) gives read-your-own-writes per session — usually right for chat. Strong trades latency for linearizability when correctness demands it.

Q: ACA vs AKS? Container Apps by default — serverless, autoscale (KEDA), managed ingress, VNet-integratable: ideal for an AI microservice. AKS when the bank already runs Kubernetes or you need GPU pools/complex topology, at higher ops cost.

Q: What's in your CI/CD? Short-lived branches → PR with required reviews on protected main. CI: lint/type-check, tests (mocked Azure), Docker build + scan, AI eval gate (groundedness/safety must not regress), Bicep what-if. CD: push to ACR → staging → smoke+eval → canary to prod (ACA revision traffic split) with rollback. OIDC to Azure — no stored secrets; everything IaC.

Q: How do you make it highly available? ≥2 zonal replicas, health probes, autoscale; timeouts, retries with backoff, circuit breakers, idempotency keys for mutating calls; graceful degradation to a smaller model; multi-region + PTUs for model capacity; App Insights to detect and roll back fast.

H. Evaluation

Q: How do you evaluate a RAG assistant? A versioned golden set (representative + edge + out-of-scope + Arabic/English) and an adversarial set. Score retrieval (recall@k, context precision) and generation (groundedness/faithfulness, answer relevance, correctness, citations) with RAGAS/Foundry evaluators, plus safety metrics. Stratify by language/intent; gate in CI; monitor sampled production traffic; feed failures back.

Q: What's groundedness and why is it the headline metric? Whether every claim in the answer is supported by the retrieved context — directly measures hallucination risk, which is the thing a bank most fears. Measured by an LLM-judge / Foundry Groundedness evaluator, gated in CI and monitored online.

Q: LLM-as-judge — risks? Position/verbosity/self-preference bias, and it's not ground truth. Mitigate with rubrics, evidence-required judgments, order randomization, calibration against human labels (report agreement), and sampling. Pair with deterministic checks and human review.

Q: Offline vs online evaluation? Offline gates releases in CI on a fixed set. Online continuously evaluates sampled production traffic (models/docs/inputs drift) and alerts via Azure Monitor. You need both — it's the model-risk monitoring posture a bank expects.

I. Banking & Responsible AI

Q: What makes AI in a bank different? The cost of being wrong (money movement, legal breach, data leakage, fines) forces safety, control, explainability, and auditability over raw capability: money facts from systems of record via tools; access-trimmed retrieval; human approval on high-risk actions; immutable audit; model validation (SR 11-7-style); data residency and PII minimization. AI assists; humans/rules govern decisions.

Q: What is SR 11-7 / model risk management? Federal Reserve guidance on managing the risk that a model is wrong or misused, via development standards, independent validation, and ongoing monitoring (three lines of defense, "effective challenge"). For an LLM app: model card, versioned prompts/models, independent eval set, pre-prod validation, production monitoring.

Q: How do you handle PII? Minimize and classify; detect and redact/tokenize before the model and before logging (Azure AI Language PII / Presidio); pass only what's needed; residency-pinned; encryption + CMK; least-privilege; immutable but PII-redacted audit; retention/erasure per PDPL/GDPR.

Q: A PM wants the agent to autonomously approve fee waivers — your response? Keep the LLM as drafter/recommender; require human (or deterministic-rule) approval for the actual waiver — it's a consequential money decision needing accountability and audit. The agent gathers context, checks policy, proposes with rationale and citations, then pauses for approval (logged). Autonomy only for low-risk, reversible, rule-bounded actions.

Q: How would you localize for the UAE? Arabic+English parity with Gulf-dialect handling and RTL UI; Arabic-capable OCR; multilingual embeddings and Arabic eval sets with native reviewers; in-region residency (UAE North) under CBUAE/PDPL (and DIFC/ADGM if applicable); correct Islamic-banking product vocabulary where relevant; full safety/audit; disclosed AI use and bilingual human escalation.

Q: How do you prove to a regulator why the system did something? End-to-end immutable audit — prompt+version, retrieved chunks+scores, tool calls+args+results, model+params, guardrail triggers, decision+citations, approver — PII-redacted, retained per policy — plus model documentation and eval evidence. I can reconstruct any interaction.