« Track Overview · Warmup · Lab 01

Phase 15 — Governance, Model Risk & Regulator-Grade Evidence

Answers this JD line: "Harden the platform to meet CBUAE, internal model risk, and Group governance requirements, including auditability, lineage, data residency, model risk controls, third-party model governance, and OWASP LLM Top 10 alignment."

Why this phase exists

Every other phase in this track emits an artifact. This phase is where those artifacts become evidence, and where the difference between the two is made concrete.

An examiner does not ask "do you log?" They ask:

"On 12 March, an agent initiated a payment of AED 250 000 for customer X. Show me: who authorized it, what the agent was permitted to do at that moment, what data it used to decide, which model version produced the decision, which policy version allowed it, and who reviewed it."

A log line answers none of that. Evidence is a linked set of records with shared join keys and a tamper-evident chain — and the only way to have it is for every layer to emit its part as a normal consequence of doing its job. Evidence you have to reconstruct later is evidence you do not have.

The second half is model risk. Banks inherit a framework — most internal standards descend from SR 11-7 — that says models must be inventoried, independently validated, monitored, and governed through a lifecycle. Applying it to an agentic platform raises a question the framework did not anticipate: what, exactly, is the model? The weights are one component. The prompt is another. The retrieval configuration, the tool set and the guardrails all change the output distribution. The defensible answer is that the agent configuration as a whole is the model, which makes prompt and retrieval changes model changes — with everything that implies.

Concept map

• The evidence chain: what each layer emits (channel → session record; control plane → decision + policy version; kernel → execution chain; knowledge → citations + document versions; action gateway → audit record + idempotency key + approvals; model layer → inference record + usage), and the join keys that make them one story.
• Reproducibility: pinned model version, prompt version, retrieval snapshot, policy version, tool versions — the set required to explain a decision six months later.
• Model inventory: every model, prompt, agent and retrieval configuration, with owner, purpose, risk tier, validation status and monitoring.
• Risk tiering: impact-based (financial, customer, regulatory) driving validation depth, approval level, monitoring frequency and autonomy band.
• Independent validation: a function that did not build it, with authority to block; what a validation pack contains for an agentic system (evals, red-team results, scope, limitations, monitoring plan).
• Lineage: the graph from output back to every input — model, prompt, retrieved documents, tool results, policy decisions, identities.
• Data residency and sovereignty: as a provable property of the routing and network topology (Phases 04, 13), with evidence per inference.
• Third-party model governance: data-use terms, sub-processors, region guarantees, deprecation notice periods, and a tested exit plan; concentration risk answered with an architecture.
• Model deprecation and drift: version pinning, evaluation on every version change, and monitoring for silent provider-side updates.
• Regulatory frames: CBUAE (outsourcing/cloud, residency, operational resilience), SR 11-7 (model risk), NIST AI RMF (Govern/Map/Measure/Manage as structuring vocabulary), EU AI Act (as the strictest reference regime), OWASP LLM Top 10 (control mapping from Phase 11).

The lab

124 tests, all green. Test contract: the pack for a given action contains every required artifact or fails with the missing one named; the lineage walk is complete and acyclic; a decision whose model version was not pinned is reported as non-reproducible; an inference outside the permitted region is flagged even when the request was otherwise valid; a tampered record fails chain verification; and an untiered model cannot be promoted to production.

Documents

Deliverables checklist

• You can list the artifacts an evidence pack needs and name the emitting component for each.
• You can argue what "the model" is for an agentic system, and defend it to model risk.
• You can design a risk-tiering scheme and say what each tier changes.
• You can state what reproducibility requires and which pin is usually missing.
• You can explain how residency is proved rather than asserted.
• You can describe a third-party model governance pack and a tested exit plan.
• You can map OWASP LLM Top 10 controls to components and to evidence.

Key takeaways

• Evidence is generated as a by-product of serving. Anything assembled later is a reconstruction, and an examiner can tell.
• Join keys are the design. session_id and context_id on every artifact, from day one.
• The agent configuration is the model. Prompt and retrieval changes are model changes.
• Risk tier drives autonomy. Higher impact means narrower bands and deeper validation.
• Reproducibility needs pins, and the one people forget is the retrieval snapshot.
• Residency is a provable property, not a configuration claim.
• Concentration risk is answered with an architecture, and the answer is only real if the alternative path carries live traffic.