« Track Overview · Warmup · Lab 01
Phase 12 — The Integration Fabric: Core Banking, ISO 20022, Kafka & Data Products
Answers these JD lines: "Lead the integration architecture between the AI Platform and the bank's core estate, including core banking, payments, treasury, credit and risk systems, the enterprise data platform (Azure, Cloudera, Databricks), enterprise APIs, ESB, event streaming (Kafka, Event Hubs), and the data product layer" · "Working knowledge of integration patterns relevant to banking platforms, including event-driven architectures (Kafka, Event Hubs), API gateways, ESB, ISO 20022, payment rails, and core banking integration patterns."
Why this phase exists
An AI platform in a bank is only as useful as the systems it can reach, and those systems are thirty years old, extremely reliable, and were not designed with you in mind.
Three properties of the bank estate shape every integration decision:
- The system of record is slow to change and enormous in blast radius. You do not get to modify core banking to suit an agent. You integrate through mediation, and the mediation layer absorbs the impedance.
- The vocabulary is standardized and the standard is large. ISO 20022 is not a schema you
invent; it is a message catalogue with a shared business model, and getting
pain.001wrong is not a validation error, it is a rejected payment. - Finality is real. A payment past its cut-off cannot be unilaterally reversed. That is not a database property; it is a scheme rule, and it constrains what an agent may safely propose.
The other half of the phase is the modern side: event streaming as the asynchronous substrate, CDC as how a system of record becomes an event source without being modified, the transactional outbox as the fix for the dual-write problem, and data products as the contract-bearing unit the AI platform both consumes and — for traces, evaluations and outcomes — produces.
Concept map
- Core banking integration patterns: never direct; through the ESB, an API gateway, or a mediation service. Read-through, write-behind, and why an agent gets neither without the action gateway (Phase 10).
- ISO 20022: the message families and how to read a name —
pain.001(customer credit transfer initiation),pacs.008(FI-to-FI),camt.053(statement). Mandatory vs optional elements, structured party data, and the amount/currency rules that cause most rejections. - Payment rails and finality: domestic RTGS/ACH, SWIFT, instant schemes; cut-off times; reversal semantics; and mapping "irreversible" as a side-effect class onto scheme reality.
- Event streaming: partitions, offsets, consumer groups; ordering guarantees and what they cost; Kafka and Azure Event Hubs as the same model with different words.
- Delivery semantics: at-least-once as the practical default, exactly-once effects via idempotent consumers, and why exactly-once delivery is not the goal.
- The transactional outbox: writing the domain change and the outbound event in one local transaction, then relaying — the standard fix for the dual-write problem.
- CDC: deriving a stream from the transaction log (Debezium-shaped), and the schema-drift problem it hands you.
- Schema registry and compatibility: backward · forward · full (and transitive variants), and the deploy-order consequence of each — the same who-breaks reasoning as Phase 02.
- Data products and contracts: schema, semantics, quality, freshness, ownership; the AI platform as a consumer of the enterprise data platform (Azure, Cloudera, Databricks) and a producer of trace, evaluation and outcome products.
- Reconciliation: the bank habit worth adopting everywhere — two independent records, compared on a schedule, with a break process.
The lab
| Lab | You build | Proves you understand |
|---|---|---|
| 01 — The Integration Fabric | an ISO 20022 pain.001 parser and validator with mandatory-element, amount/currency and structured-party rules producing a rejection report; a transactional outbox with a relay and an idempotent consumer demonstrating exactly-once effects under duplicate delivery; a partitioned log with consumer groups and offsets showing where ordering holds and where it does not; a schema registry enforcing backward/forward/full compatibility with the deploy-order implication stated; and a data-product contract checker validating schema, freshness and quality with an SLO breach report | that integration in a bank is about contracts and finality, and that "exactly once" is achieved by idempotent handling rather than by clever delivery |
144 tests, all green. Test contract: a pain.001 missing a mandatory element is rejected with
the element named; a duplicate delivery produces exactly one effect; ordering is guaranteed
within a partition and not across; a backward-incompatible schema is refused; and a data product
past its freshness SLO fails its contract check.
Documents
| Document | For |
|---|---|
| WARMUP.md | zero to principal on bank integration — first principles, then the interview answers |
| HITCHHIKERS-GUIDE.md | the fast orientation: what the pieces are and how they fit |
| DEEP-DIVE.md | the mechanisms, in detail, with the failure modes |
| PRINCIPAL-DEEP-DIVE.md | the trade-offs you own at principal level |
| CORE-CONTRIBUTOR.md | what it takes to work on Kafka, Debezium or a schema registry |
| STAFF-NOTES.md | judgment, review signal, war stories |
Deliverables checklist
- You can read an ISO 20022 message name and say what it is for.
- You can explain the dual-write problem and how the outbox solves it.
- You can state what at-least-once plus idempotency buys you and what it does not.
- You can explain the deploy-order consequence of each compatibility mode.
- You can explain finality and how it maps onto side-effect classes.
- You can define a data product with a freshness contract.
- You can describe a reconciliation process and what a "break" is.
Key takeaways
- Never integrate an agent directly with a system of record. Mediation is the layer that absorbs impedance and enforces contracts.
- ISO 20022 is a business model, not a schema. Learn the families; the details are lookups.
- Finality constrains autonomy. An irreversible action needs an approval, not a retry policy.
- Exactly-once effects, not exactly-once delivery. Idempotent consumers, always.
- The outbox is the dual-write fix, and it is boring, which is why it works.
- Compatibility mode determines deploy order. State it, or you will find out during a release.
- The AI platform should publish data products too — traces, evaluations, outcomes — with the same contract discipline it demands of others.