Knowledge 03 — Agentic AI Frameworks

Goal of this module. Go from "an agent is a chatbot that uses tools" to being able to architect a stateful, auditable, banking-grade agent and choose between LangGraph, LangChain, Microsoft Agent Framework, Foundry Agent Service, and MCP with reasons. "Design and implement Agentic AI architectures for banking-grade enterprise applications" is the first "THE MUST" line of JD4 — this is the module the title of the role is named after.

Table of Contents

• 1. What "agentic" actually means
• 2. The agent loop and ReAct
• 3. The five components of an agent
• 4. Why naive agent loops fail in production
• 5. LangGraph: agents as stateful graphs
• 6. LangChain: the toolkit underneath
• 7. Microsoft Agent Framework & Semantic Kernel
• 8. Azure AI Foundry Agent Service
• 9. MCP — the Model Context Protocol
• 10. Multi-agent patterns
• 11. Choosing a framework (the decision table)
• 12. Designing a banking agent safely
• 13. Common misconceptions
• 14. Interview Q&A
• 15. References

1. What "agentic" actually means

A plain LLM call is one-shot: prompt in, text out. An agent is an LLM placed in a loop where it can take actions in the world (call tools/APIs), observe the results, and decide the next step, repeating until a goal is met. The model isn't just answering — it's orchestrating a multi-step task.

The spectrum (know where each banking use case sits):

• Single LLM call — classify an email, summarize a statement.
• Chain — fixed sequence (retrieve → answer). Deterministic, no decisions.
• Router — LLM picks one of N predefined paths.
• Agent — LLM decides which tools to call, in what order, how many times, conditioned on intermediate results. Non-deterministic control flow.
• Multi-agent — several specialized agents collaborate (a planner, a retriever, a checker).

"Agentic AI architectures for banking" means: design systems where the LLM drives a workflow — e.g. handle a disputed transaction: look up the transaction (tool), retrieve the dispute policy (RAG), check eligibility (logic), draft a resolution, pause for human approval, then file it (tool). That is an agent — and the hard parts are state, reliability, approvals, and audit, not the LLM.

2. The agent loop and ReAct

The canonical pattern is ReAct (Reason + Act). Each iteration:

THOUGHT:   (LLM reasons about what to do next)
ACTION:    (LLM emits a tool call, e.g. get_balance(account_id))
OBSERVATION: (your code runs the tool, returns the result into context)
... repeat ...
FINAL:     (LLM produces the answer / completes the task)

Mechanically this is tool calling in a loop (K01 §8): you give the model tools; it either answers or requests a tool call; you execute and feed back the observation; you loop until it answers or hits a limit. "Reasoning" can be explicit (the model writes its thoughts) or internal (o-series reasoning models, or hidden scratchpads).

Why ReAct works: interleaving reasoning with real observations lets the model correct course based on actual data instead of hallucinating a whole plan up front. It grounds each step in tool results.

3. The five components of an agent

Every agent framework is some arrangement of these:

1. Model (the reasoner) — the LLM that decides. Azure OpenAI gpt-4o/4.1 for most steps; an o-series model for hard planning.
2. Tools — typed functions the agent can call: RAG search, account lookup, transaction history, send-email, file-a-ticket, calculator. Each is a JSON schema + your implementation. Tools are the agent's hands; their reliability and permissions bound everything.
3. Memory / State — what persists across steps and turns:
   • Short-term — the current run's scratchpad/message history (in the context window).
   • Long-term — conversation history, user preferences, facts, stored in Cosmos DB or a vector store, retrieved as needed. Foundry Agent Service exposes session / user / procedural memory.
4. Planning — how the agent decomposes a goal: ReAct (interleaved), plan-then-execute (plan up front, then run steps), or a graph you author explicitly (LangGraph).
5. Orchestration / Control flow — the engine that runs the loop, routes between tools/sub-agents, enforces limits, persists state, and handles human-in-the-loop. This is what the frameworks provide.

4. Why naive agent loops fail in production

Write a while True: loop around tool-calling and it demos beautifully and dies in prod. The failure modes (and why frameworks exist):

• No durable state. Process crashes mid-task → the whole multi-step workflow is lost. A bank's "process this loan application" can't restart from scratch. → need checkpointing.
• No human-in-the-loop. High-risk actions (move money, close account) must pause for approval. A bare loop can't suspend and resume days later. → need interrupts/durability.
• Loops and runaway cost. The model calls tools forever or oscillates. → need step limits, timeouts, cycle control.
• No observability. When it does the wrong thing, you can't see the reasoning/tool trace. A bank must audit every step. → need tracing.
• Tool failures. A downstream API 500s; the agent must retry/backoff/degrade, not hallucinate success. → need error handling/retries.
• No determinism where required. Some steps must be fixed business logic, not LLM whim. → need explicit graph edges, not pure free-for-all.

Frameworks (LangGraph, Agent Framework, Foundry Agent Service) exist to provide durable state, interrupts, observability, retries, and explicit control flow — exactly the "production-grade, high availability, security" the JD demands.

5. LangGraph: agents as stateful graphs

What it is. LangGraph models an agent as a directed graph: nodes are functions (call the LLM, call a tool, a decision), edges define control flow (including conditional edges and cycles), and a shared state object flows through and is updated by each node. JD4 names LangGraph specifically — know it cold.

The core concepts:

• State — a typed dict (often a TypedDict) holding everything: messages, retrieved docs, tool results, flags. Each node receives state and returns updates (merged in, e.g. messages appended via a reducer).
• Nodes — (state) -> state_update. E.g. retrieve, call_model, call_tool, human_review.
• Edges — wire nodes. Conditional edges route based on state ("did the model request a tool? → tool node; else → END"). Cycles let it loop (model → tool → model …) — graphs, not just chains.
• Checkpointer — persists state after every node to a backend (in-memory, SQLite, Postgres, Cosmos/Redis). This gives durability (resume after crash), time-travel (replay/inspect), and memory across turns (thread_id).
• Interrupts / human-in-the-loop — pause before a node (e.g. execute_transfer), surface to a human, and resume later from the checkpoint with their decision. This is the killer banking feature.

from langgraph.graph import StateGraph, START, END

def call_model(state):  # node: LLM decides (maybe requests a tool)
    return {"messages": [llm.invoke(state["messages"])]}

def should_continue(state):  # conditional edge
    last = state["messages"][-1]
    return "tools" if last.tool_calls else END

g = StateGraph(AgentState)
g.add_node("agent", call_model)
g.add_node("tools", tool_node)
g.add_node("human_review", human_review_node)   # interrupt point
g.add_edge(START, "agent")
g.add_conditional_edges("agent", should_continue, {"tools": "tools", END: END})
g.add_edge("tools", "agent")                     # cycle
app = g.compile(checkpointer=cosmos_checkpointer, interrupt_before=["human_review"])
# run with a thread_id → state persists, resumes, is auditable

Why a graph and not a chain? Banking workflows have branches (eligible? → path A vs B), loops (retry retrieval, multi-step tool use), and pause points (approval). A graph expresses all three explicitly and durably; a linear chain can't. The explicitness is also what makes it auditable — you can show a regulator the exact path taken.

6. LangChain: the toolkit underneath

LangChain is the broader library: model wrappers (incl. AzureChatOpenAI, AzureAIDocumentIntelligenceLoader, AzureSearch retriever), document loaders, text splitters, output parsers, tool abstractions, and LCEL (a composition syntax for chains). LangGraph is built on top of LangChain's primitives, adding stateful orchestration.

When you use which: LangChain for the building blocks (load → split → embed → retrieve → prompt → parse) and simple chains; LangGraph when you need stateful, cyclic, human-in-the-loop orchestration — i.e. real agents. JD4 lists both; the honest framing in interview: "LangChain for the plumbing and RAG chains, LangGraph for the agent's stateful control flow."

7. Microsoft Agent Framework & Semantic Kernel

Semantic Kernel (SK) — Microsoft's earlier agent/orchestration SDK (C#, Python, Java): "kernel" hosts plugins (skills/functions), planners, memory, and connectors (incl. Azure OpenAI). Strong enterprise/.NET story. As of 2025 it is in maintenance mode, folded into the Agent Framework.

AutoGen — Microsoft Research's multi-agent conversation framework (agents talk to each other to solve tasks). Great for dynamic multi-agent orchestration. Also a predecessor now.

Microsoft Agent Framework — announced public preview Oct 1, 2025: the convergence of Semantic Kernel + AutoGen into one open-source SDK + runtime (Python and .NET). It combines SK's enterprise foundations (type-safe functions, state/threads, filters, telemetry) with AutoGen's multi-agent orchestration patterns, plus explicit workflows for multi-step/human-in-the-loop tasks, and native Azure AI Foundry integration. Strategic message: this is Microsoft's go-forward agent SDK; new MS-stack agent work targets it, not SK/AutoGen directly.

Why JD4 lists these as "good to have": the role centers on LangGraph/LangChain, but a Microsoft-shop bank may standardize on the Microsoft stack. Knowing that Agent Framework = SK + AutoGen converged, Foundry-native is exactly the up-to-date signal that impresses.

8. Azure AI Foundry Agent Service

What it is. A managed runtime for production agents (GA), part of Azure AI Foundry, built on the OpenAI Responses API (wire-compatible with OpenAI agents). Instead of hosting your own loop, you define an agent (model + instructions + tools) and Foundry runs it with:

• Hosted execution and threads/state.
• Built-in tools — file search, code interpreter, Azure AI Search (grounded retrieval), Bing/Function/OpenAPI tools, and MCP tools; plus your custom functions.
• Memory — procedural / user / session.
• Multi-agent orchestration — coordinate specialized agents on long-running tasks.
• Observability & continuous evaluation — traces, and sampled-traffic evaluators surfacing quality/safety/performance to Azure Monitor.
• Governance & identity — Entra ID, content safety, the Foundry compliance envelope; publishable into Teams/M365.

The senior framing: self-host LangGraph in a Container App when you need full control over the loop, custom durability, or portability; use Foundry Agent Service when you want managed memory/observability/governance and tight Azure integration. They're not mutually exclusive — a Foundry agent can call MCP tools your LangGraph services expose, and vice versa.

9. MCP — the Model Context Protocol

The problem it solves. Before MCP, every agent framework had its own way to define tools, so a tool written for LangChain didn't work in Semantic Kernel or Foundry. MCP (open standard, Anthropic, Nov 2024) is a client–server protocol (JSON-RPC) that standardizes how an LLM application discovers and uses external capabilities:

• MCP server — exposes tools (callable functions), resources (readable data/context), and prompts (templates). E.g. a "core-banking MCP server" exposing get_balance, list_transactions.
• MCP host/client — the agent app (LangGraph, Foundry, Claude, etc.) connects to servers, discovers their tools, and calls them.

Why it matters for a bank: write your account-lookup tool once as an MCP server; any MCP-aware agent host can use it — no per-framework rewrites, central governance of what tools exist, and a clean security boundary (the server enforces auth/permissions, not the prompt). It's the USB-C of agent tools. JD4 lists it as "good to have"; naming it correctly (tools/resources/prompts, client–server) is a strong signal.

10. Multi-agent patterns

When one agent's prompt/toolset gets unwieldy, decompose into specialists:

• Supervisor / orchestrator-worker — a supervisor agent routes sub-tasks to worker agents (a "retrieval agent", a "calculation agent", a "compliance-check agent") and synthesizes. Most common enterprise pattern.
• Sequential pipeline — agents in a fixed order (extract → analyze → draft → review).
• Group chat / debate — agents converse to refine an answer (AutoGen-style); a checker agent critiques (reflection).
• Hierarchical — supervisors of supervisors for complex workflows.

Banking caution: more agents = more non-determinism, cost, latency, and audit surface. Use multi-agent only when a single agent genuinely can't, and keep deterministic guardrails and human approval at the boundaries. Interviewers reward "I'd start with one well-scoped agent and add specialists only when the toolset/prompt stops fitting."

11. Choosing a framework (the decision table)

The right interview answer is rarely "always X" — it's "LangGraph for the agent core because I need durable state and human approval; LangChain for the RAG plumbing; expose core-banking via MCP for governance; and I'd consider Foundry Agent Service if the bank wants a managed runtime with built-in observability."

12. Designing a banking agent safely

The non-negotiables an interviewer will check:

• Least-privilege tools — each tool enforces auth (the user's permissions, via Entra/OBO), validates inputs, and is idempotent where it mutates state. The agent can't do what its tools won't let it.
• No money facts from the model — balances/rates/limits come from tool calls to systems of record, never the LLM's "memory" (K01 §7).
• Human-in-the-loop on high-risk actions — transfers, account changes, anything irreversible → pause (LangGraph interrupt) for approval; log the approver.
• Deterministic guardrails — hard business rules as code/edges, not LLM discretion (e.g. "never exceed daily transfer limit" is checked in code).
• Full audit trail — persist every prompt, thought, tool call+args+result, and decision to immutable storage; this is the regulatory requirement.
• Step/cost limits & timeouts — prevent runaway loops.
• Content safety & prompt-injection defense — especially since retrieved docs/customer uploads can carry indirect prompt injection; use Prompt Shields and treat tool outputs/docs as untrusted (K08).
• Graceful degradation — tool down → apologize + offer human handoff, never fabricate.

13. Common misconceptions

• "An agent is just a chatbot with tools." It's a loop with state and control flow; the engineering is in durability, approvals, and audit, not the chat.
• "Use a while loop, it's simple." It dies on crashes, can't pause for approval, isn't observable, and loops forever. Frameworks provide durability/interrupts/tracing for a reason.
• "More agents are better." More agents = more cost/latency/non-determinism/audit surface. Start with one.
• "LangChain and LangGraph are competitors." LangGraph is built on LangChain; LangChain = building blocks, LangGraph = stateful orchestration.
• "Semantic Kernel is the current MS agent SDK." It's in maintenance mode; Microsoft Agent Framework (SK+AutoGen) is the successor.
• "The agent enforces permissions." No — the tools enforce permissions; never let the prompt be your access control.
• "Let the agent move the money autonomously." High-risk actions need human-in-the-loop and deterministic guardrails in a bank.

14. Interview Q&A

Q: Design an agentic workflow to handle a disputed card transaction. A LangGraph agent: nodes = identify_txn (tool: transaction lookup), retrieve_policy (RAG over dispute policy), assess_eligibility (LLM + deterministic rule checks), draft_resolution, human_review (interrupt for an agent's approval on anything refunding money), file_dispute (tool), notify_customer. Shared state carries txn details, policy citations, eligibility, draft. Checkpoint to Cosmos for durability/resume. Every step logged for audit; money-moving step gated by human approval; tools enforce the user's permissions; live amounts from the core-banking tool, never the LLM.

Q: Why LangGraph over a simple loop or a chain? A chain is linear — no branches, cycles, or pauses. A bare loop has no durable state (crash = lost workflow), no human-in-the-loop, no observability, and can run away. LangGraph gives an explicit graph (branches + cycles), a checkpointer (durable, resumable, auditable state), and interrupts (pause for approval, resume later) — exactly the production/compliance properties a bank needs.

Q: When would you use Foundry Agent Service instead of self-hosted LangGraph? When the bank wants a managed runtime with built-in memory, tools (incl. Azure AI Search grounding), observability, continuous evaluation, and governance under the Foundry/Entra compliance envelope — less infra to own. I'd self-host LangGraph when I need full control of the loop, custom durability/state backends, or portability off Azure. They interop via MCP.

Q: What is MCP and why would a bank care? MCP is an open client–server protocol standardizing how agents discover and call tools/resources/prompts. A bank writes its core-banking tools once as an MCP server with centralized auth/governance; any MCP-aware agent host (LangGraph, Foundry, etc.) consumes them without rewrites. It decouples tools from frameworks and gives a clean, governable security boundary.

Q: How do you keep an autonomous agent safe in a bank? Least-privilege tools that enforce the user's permissions and validate inputs; no money facts from the model (tool calls only); human-in-the-loop on irreversible/high-risk actions; deterministic guardrails for hard rules; full immutable audit of prompts/thoughts/tool calls/decisions; step/cost limits; content safety + prompt-injection (Prompt Shields, treating retrieved content as untrusted); graceful degradation to human handoff.

Q: How do you defend against prompt injection in an agent that reads customer-uploaded documents? Treat all retrieved/tool content as untrusted data, not instructions: keep system instructions privileged and separate, use Prompt Shields/indirect-injection detection, constrain tools to least privilege so an injected "transfer funds" can't succeed without the human-approval gate, validate tool arguments, and never let document text alter the agent's authorization. Log and evaluate for injection attempts.

15. References

• ReAct — Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models" (2022).
• Toolformer — Schick et al. (2023); Reflexion — Shinn et al. (2023).
• LangGraph / LangChain — official docs (state, nodes, conditional edges, checkpointers, human-in-the-loop, LCEL).
• Microsoft Agent Framework — Introducing Microsoft Agent Framework (Azure Blog, Oct 2025); Microsoft Learn Agent Framework overview; migration-from-SK/AutoGen guide.
• Semantic Kernel & AutoGen — Microsoft Learn / GitHub docs.
• Azure AI Foundry Agent Service — Announcing GA of Foundry Agent Service; What's new in Foundry Agent Service; built on the OpenAI Responses API.
• Model Context Protocol — modelcontextprotocol.io spec (tools/resources/prompts, JSON-RPC).
• Agentic design patterns — Anthropic, "Building effective agents" (2024); Google/others on multi-agent supervisor patterns.