« Phase 07 · Warmup · Track Overview

Principal Deep Dive — Architecture, Tradeoffs & Blast Radius

Table of Contents

• 1. The three tradeoffs
• 2. Where the graph sits in the platform
• 3. Scaling envelope
• 4. Failure modes and blast radius
• 5. Asserted versus derived, and why it is an audit question
• 6. The ontology as an organizational artifact
• 7. Decisions that look wrong but are intentional
• 8. What changes at 10×

1. The three tradeoffs

Tradeoff 1 — materialize vs reason at query time. Forward chaining pays on write and gives free reads; backward chaining pays on every read and is always correct under deletion.

The resolution for a bank is materialize, with the asserted and derived graphs separated. Reads dominate by orders of magnitude, ownership questions are asked constantly, and the deletion problem (§4) is managed by recomputing the derived graph rather than by truth maintenance. The separation is what makes recomputation cheap and makes §5's audit question answerable.

The case for backward chaining is a graph with high churn and low query volume — reference data that changes hourly and is queried daily. Rare in this domain.

Tradeoff 2 — expressivity vs tractability. OWL 2 DL can express a great deal and needs a tableau reasoner with exponential worst cases. OWL 2 RL is rule-based, runs in polynomial time, and covers transitivity, inverses, sub-properties, domain/range — which is everything in §4.3 of the WARMUP.

The resolution: stay in RL unless someone can name the DL construct they need and what it buys. Usually they cannot, and the request is really for a construct RL already has. When it is genuine — complex class expressions for a regulatory definition, say — the answer is often to compute it in a SPARQL query or a SHACL rule rather than to switch reasoning profiles for the whole store.

Tradeoff 3 — one graph vs many. A single graph gives global joins and one place to reason. Many graphs (per domain, per tenant, per classification) give isolation and independent lifecycles.

The resolution uses named graphs — the quad model — rather than separate stores: one store, many named graphs, with queries scoped to the graphs the caller may see. That preserves the ability to reason across domains while keeping the tenant and classification boundaries from Phase 06 enforceable. The asserted/derived split is the same mechanism applied to a different axis.

2. Where the graph sits in the platform

Three roles, and they have different requirements:

The second is the one people miss and it is the highest-value. Using the graph to decide what to retrieve — rather than to answer directly — composes with Phase 06 cleanly, keeps the model out of SPARQL generation, and produces the provenance a graph-derived claim needs.

Do not let the model write SPARQL against a live store. Text-to-SPARQL is fragile, and a generated query with an unbounded property path is a resource-exhaustion incident with no authorization story. The safe shape is parameterized queries as tools: who_controls(entity), obligations_of(entity), path_between(a, b, max_hops) — each with a bounded traversal, each with the caller's entitlements applied, each reviewed once. That is Phase 02's tool-contract discipline applied to a graph.

The authorization consequence: the graph inherits the classification of everything in it. A triple saying ent:Acme bank:involvedIn ent:ProjectFalcon is MNPI as a triple, independent of any document. So graph queries need the same entitlement filtering as retrieval, and named graphs per barrier are the mechanism.

3. Scaling envelope

The one that surprises people: materialized transitive closure is quadratic in chain length. An ownership chain of depth d produces \( d(d-1)/2 \) derived controls triples. A 20-deep chain — which exists in real corporate structures — is 190 derived triples from 19 asserted ones. Across a large corporate register that is a materially larger graph, and it is why some stores offer transitivity as a query-time operator rather than a materialized rule.

The mitigation is to materialize the closure only for the relations you query transitively, and leave the rest as property paths. That is a modelling decision with a storage consequence, and it is exactly the kind of thing an ontology owner should be deciding rather than a developer adding owl:TransitiveProperty because it seemed natural.

Second surprise: join order dominates query latency more than data size. The lab's engine has no planner, and the traced query in DEEP-DIVE §8 would be far cheaper with its patterns reversed. Production engines have cost-based planners with cardinality statistics — and that planner is most of what you are buying when you choose a store.

4. Failure modes and blast radius

Two rows deserve expansion.

The stale closure is this phase's silent failure, and it is the exact analogue of Phase 06's post-hoc filter. A majorityOwns triple is corrected, the derived controls triples are not recomputed, and every ownership answer is wrong. Nothing errors, because a derived triple is indistinguishable from an asserted one in the store. The structural fix — separate named graphs, so recomputation is dropping one and re-running — is cheap if you do it on day one and expensive later.

Graph classification is the row people forget entirely. Retrieval got namespaces and barriers in Phase 06; the graph is treated as "just metadata". But ent:Acme bank:involvedIn ent:ProjectFalcon is the MNPI — the relationship is the sensitive fact, independent of any document containing it. A graph query that ignores barriers leaks exactly what Phase 06's retrieval filter was built to protect.

5. Asserted versus derived, and why it is an audit question

An examiner asks: "On what basis did you conclude that Meridian controls Acme?"

There are three possible answers and they are not equivalent:

1. "A source system told us." An asserted triple, with provenance to a register.
2. "We inferred it" — from majorityOwns assertions plus an ontology rule. Defensible, and it requires you to be able to state which rule and from which assertions.
3. "We don't know which." The answer if asserted and derived triples are in one undifferentiated graph.

Only the third is unacceptable, and it is the default if you do not design against it.

So the architecture:

• Named graphs: bank:asserted and bank:derived, at minimum. Better still, provenance per source system.
• Rule attribution on derived triples, so "which rule produced this" is answerable. Not free — it is essentially truth maintenance — but a lighter version (record the rule name, not the full justification) covers most audit questions cheaply.
• A reproducibility guarantee: given the asserted graph and the ontology version, the derived graph is a pure function of both. Version the ontology, and an old conclusion can be re-derived — which is a Phase 15 requirement.

The same discipline applies to the agent: a graph-derived claim in an answer should carry its path (the lab's expand_neighbourhood returns paths for exactly this reason), so "Meridian was included because Acme is controlledBy Northgate is controlledBy Meridian" is in the evidence rather than implied.

6. The ontology as an organizational artifact

An ontology is not a schema. It is a negotiated agreement about meaning, and its problems are consequently organizational.

It has a blast radius the size of its user base. Changing controls changes every query, every shape and every downstream report. That is Phase 02's who-breaks question with no version pinning available — you cannot have two meanings of controls in one graph.

It attracts scope creep. Every team wants its concepts represented, and each request is individually reasonable. Two years later the ontology has 4 000 classes, no one understands it, and the reasoner takes hours. The discipline is that the ontology models what crosses a boundary; anything used by one team belongs in that team's data, not in the shared vocabulary.

Its owner needs authority to say no. Without that, an ontology becomes the union of everyone's schema — the failure mode from the WARMUP §10, and the one that makes people conclude "semantic technology doesn't work" when what failed was governance.

Adopting FIBO is partly a political move, and worth naming as one. It provides an external authority for definitions, which makes "no, control means what FIBO says it means" a defensible position rather than one team's opinion. That is a real and under-appreciated benefit of adopting a standard vocabulary in a large organization.

The practical governance shape:

7. Decisions that look wrong but are intentional

Materializing rather than reasoning at query time. Looks like it creates a cache-invalidation problem, and it does. Reads dominate by orders of magnitude, and the invalidation problem is managed structurally by separating asserted and derived graphs. The alternative pays on every query forever to avoid a problem that a recompute solves.

Only three indexes, not six. Looks like it will cost you on some access pattern. Three cover every pattern the reasoner and query engine actually issue, and each index is a full copy of the data. Real stores keep six because they must serve arbitrary user queries; a lab with known access patterns should not.

A naive reasoner that re-scans every round. Looks obviously improvable — semi-naive evaluation is a well-known fix. The naive version makes the fixed-point argument visible, and the optimization is a well-named thing to reach for once you understand what it optimizes.

p+ reports that A controls itself in a cycle. Looks like a bug. It is transitive closure over a cyclic graph, which is what was asked for. The decision to exclude self-loops belongs in the query, explicitly, because sometimes you want them (detecting circular ownership is a KYC signal).

SHACL severity is applied at report level. Looks like it complicates a simple boolean. It is what lets you deploy a shape as a warning, measure, and promote — and a shape that blocks ingestion on day one gets disabled rather than fixed.

No SPARQL parser. Looks like an obvious omission for a lab about SPARQL. The parser is the least interesting part; the join semantics, the path walker and the planner-shaped hole are the lessons. Building the parser first is how you spend a week on a grammar and never reach them.

8. What changes at 10×

At 50 000 triples and one ontology module, the lab's design is close to shippable. At 50 million triples across a corporate register:

• Semi-naive evaluation is mandatory — a naive re-scan per round becomes hours.
• A query planner is why you buy a store. Cardinality statistics and join reordering, not storage, are the value.
• Selective materialization: materialize the closure only for relations you query transitively; leave the rest as property paths, because transitivity is quadratic in chain length.
• Named graphs become the primary structure — per source system, per classification, per barrier, plus asserted/derived — and queries are scoped rather than global.
• Entity resolution becomes a first-class problem. Two registers spell the same company differently. owl:sameAs is the construct and it is dangerous: it merges everything said about both IRIs, so a wrong sameAs is a data-quality incident that propagates through the reasoner. Production practice is to keep resolution decisions in their own graph with confidence scores, and materialize merges only above a threshold, reviewably.
• SHACL at ingestion becomes a throughput concern, and shapes get profiled like any other hot path.
• Ontology changes need an impact query: "which queries and shapes reference this class?" — which requires recording usage, which requires deciding to do so early.
• Federation may appear — querying an external register live rather than copying it — with all the availability and latency questions that implies.

Seams to build now: named graphs for asserted vs derived from day one; the ontology version recorded alongside every materialization; parameterized query tools rather than free-form SPARQL; and classification on graphs, not just on documents.