« Phase 15 · Warmup · Track Overview

Core Contributor — Working on the Engines Themselves

What it takes to contribute to OpenLineage, in-toto, Sigstore, or the evidence tooling your bank builds. Read this if you want to understand the systems rather than configure them.

Table of Contents

• 1. Why read the engines
• 2. OpenLineage: the model
• 3. Facets, and contributing one
• 4. Marquez
• 5. in-toto and SLSA
• 6. Sigstore and transparency logs
• 7. WORM storage, mechanically
• 8. Model cards and datasheets
• 9. Building in-house evidence tooling
• 10. Testing evidence code
• 11. Contributing

1. Why read the engines

Because the standards are young and the gaps are real. OpenLineage models data pipelines well and agent runs badly. There is no standard facet for a model configuration, a policy decision or a retrieval snapshot. A bank running agents in production has exactly the experience these specifications are asking for.

Because "tamper-evident" has a precise meaning that only becomes clear from the transparency-log implementations, and the difference between evident and resistant is the difference between a claim and a control.

2. OpenLineage: the model

The standard for lineage, and its data model is three objects:

   Job        ── a process that consumes and produces datasets
   Run        ── one execution of a Job
   Dataset    ── an input or output

with facets — typed, extensible metadata attached to any of the three.

{
  "eventType": "COMPLETE",
  "eventTime": "2026-03-12T09:15:00Z",
  "run":  { "runId": "…", "facets": { "parent": {…}, "nominalTime": {…} } },
  "job":  { "namespace": "ai-platform", "name": "payments-investigator" },
  "inputs":  [ { "namespace": "kb", "name": "case-notes",
                 "facets": { "dataVersion": {…}, "dataQualityMetrics": {…} } } ],
  "outputs": [ { "namespace": "payments", "name": "release-instruction" } ]
}

Three design decisions worth stealing:

Events, not state. A run emits START and COMPLETE events; the graph is derived from the event stream. Which means a consumer that missed an event can be replayed, and it is the same level-triggered idea as Phase 13's reconciliation.

Facets are the extension point, and they are typed with a JSON Schema. So you extend without forking the standard, and a consumer that does not know your facet ignores it rather than failing.

The parent run facet is how a sub-run links to its parent — which is exactly the agent-step nesting this phase needs, and it is the closest the standard comes to modelling an agent run.

Where it fits this phase, and where it does not:

Those four blanks are the contribution opportunity.

3. Facets, and contributing one

A facet is a JSON Schema plus a name. Writing one is not exotic:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/ModelConfigurationRunFacet.json",
  "type": "object",
  "allOf": [{ "$ref": "https://openlineage.io/spec/1-0-5/OpenLineage.json#/$defs/RunFacet" }],
  "properties": {
    "baseModel":            { "type": "string" },
    "baseModelVersion":     { "type": "string" },
    "promptVersion":        { "type": "string" },
    "retrievalSnapshot":    { "type": "string" },
    "toolSetVersion":       { "type": "string" },
    "guardrailVersion":     { "type": "string" },
    "temperature":          { "type": "number" },
    "configFingerprint":    { "type": "string" }
  },
  "required": ["baseModelVersion", "promptVersion", "configFingerprint"]
}

The process: propose it in the OpenLineage repo with a use case, iterate with the community, and it lands as a custom facet before being considered for core. The bar for core is more than one implementer with a real need — which is exactly the position a bank running agents is in.

And the discipline in the meantime, which is the same as Phase 14's: namespace your custom facets (bank_modelConfiguration) so they never collide with a future standard name. Claiming modelConfiguration is how you get a conflict when the spec lands.

4. Marquez

The reference OpenLineage server (MarquezProject/marquez) — Java, Postgres, and small enough to read.

   OpenLineage events ──► API ──► Postgres ──► lineage graph API ──► UI

The schema is the interesting part:

Two things to take from it.

Dataset versions are first-class. Which is exactly the point from §6 of the deep dive: "document 991 was wrong" is not actionable; "version 3 was wrong" bounds the remediation. Marquez models this properly and most home-grown lineage does not.

The raw events are retained alongside the derived graph. So the graph can be rebuilt if the derivation logic changes — which is the difference between a lineage store you can fix and one you have to migrate.

The lineage query itself (/api/v1/lineage) is a bounded-depth graph traversal, and reading it is the fastest way to see why the forward direction needs an index while the backward direction does not.

5. in-toto and SLSA

Supply-chain provenance, and the frameworks transfer to model provenance more directly than they first appear.

in-toto (in-toto/in-toto) defines a layout — the expected steps of a pipeline, who may perform each, and what each consumes and produces — and link metadata signed by each step's performer. Verification checks that the actual links satisfy the layout.

The mapping to this phase is direct:

That last row is a question this phase's inventory answers procedurally and in-toto answers cryptographically, and the difference matters: an inventory can be edited, and signed link metadata cannot.

SLSA (slsa.dev) is the levels framework on top: provenance exists (L1), signed (L2), non-falsifiable and built on hardened infrastructure (L3), two-party reviewed (L4). Applying the vocabulary to models is a live area, and "SLSA L3 for our agent configurations" is a claim a regulator understands more readily than a bespoke description.

Worth reading: the in-toto attestation predicate format, because it is the general envelope everything else (SLSA provenance, SBOMs, VEX) is carried in.

6. Sigstore and transparency logs

The mechanism that turns tamper-evident into tamper-resistant, and it is the piece the lab explicitly cannot provide.

   sign the pack head
     → ephemeral key, OIDC identity, certificate from Fulcio
     → the signature recorded in REKOR, an append-only Merkle log

Rekor (sigstore/rekor) is the transparency log, and it gives two proofs a hash chain cannot:

The second is the one that matters here. A hash chain proves nothing was edited if you trust the chain; a consistency proof shows the log has only ever grown, verifiable by someone who does not trust you. That is the difference between showing an examiner your database and showing them a proof.

The design worth stealing for evidence: publish the pack's chain head to an external append-only log hourly. Then rewriting history requires also rewriting something outside your control, and the claim moves from "we did not edit it" to "we could not have".

Worth reading: pkg/api/entries.go in Rekor for the append path, and Trillian's merkle/ package for the proof construction. And the operational fact from Phase 12: Trillian sequences asynchronously, so an entry is not immediately provable — there is an inclusion delay of seconds, and it must be stated rather than assumed away.

7. WORM storage, mechanically

Write-once-read-many, and the enforcement is what makes it evidence rather than a convention.

The distinction that matters, and it is the one people get wrong:

Governance mode — a privileged user can delete. Useful for policy enforcement, not for evidence, because the control is an access-control decision you would have to demonstrate separately.

Compliance mode — nobody can delete before the retention expires. Not the root account, not the vendor. That is what makes it evidence: the immutability is a property of the storage rather than of your IAM configuration.

Three operational realities:

Locking the policy is irreversible. An Azure immutability policy can be locked, after which nobody can shorten it. Which is the point, and it means a mistake in the retention period is permanent — so lock in a test subscription first.

Legal hold is separate and indefinite. It suspends deletion regardless of the retention period, and it is how litigation hold works. Removing it is an audited action.

Cost is the retention period times the volume. Seven years of evidence at compliance-mode pricing, with no ability to delete early, is a number to compute before choosing what goes in — which is the argument for the retention tiers in §3 of the deep dive.

8. Model cards and datasheets

Two documentation standards that predate this phase and slot into the validation pack.

Model Cards (Mitchell et al., 2019) — intended use, out-of-scope use, factors, metrics, evaluation data, training data, ethical considerations, caveats.

Datasheets for Datasets (Gebru et al., 2018) — motivation, composition, collection process, preprocessing, uses, distribution, maintenance.

Both are worth adopting, and both need adaptation for an agentic system:

And the observation worth making to whoever asks for a model card: a model card for a third-party base model is written by the provider and is not evidence about your system. Your card is about the configuration — the prompt, the retrieval, the tools, the guardrails — which is the thing you control and the thing validation is about (§6 of the warmup).

9. Building in-house evidence tooling

The artifact type requires its join key structurally.

@dataclass(frozen=True)
class Artifact:
    artifact_id: str
    trace_id: str          # ← no default, required in the constructor
    kind: ArtifactKind
    derived_from: Tuple[str, ...]

No default, no Optional. Forgetting it is a type error, which is the only enforcement that survives a deadline.

Emit from ambient context, not an argument. A contextvars-based current-trace, so an emitter cannot pass the wrong one. This is what stops the id being right in the code and wrong at runtime.

The generator distinguishes absent from unavailable.

class ArtifactLookup(NamedTuple):
    found: Tuple[Artifact, ...]
    absent: Tuple[ArtifactKind, ...]      # queried; genuinely not there
    unavailable: Tuple[str, ...]          # the store did not answer

Collapsing those two is the bug that produces a confidently-wrong pack, and it is the first thing to get right in a federated implementation.

Canonical ordering is part of the specification. Sort by (tick, artifact_id) and write it down, or two generators produce different chain heads for the same pack.

Property tests on the invariants:

# the lineage graph is always acyclic after any sequence of valid adds
# ancestors(x) never contains x
# descendants(ancestors(x)) contains x
# the chain head is invariant under the order artifacts were ADDED
# a pack that verifies still verifies after a round trip through storage
# no evidence artifact is ever sampled away
# redacting a pack preserves verification

The fourth is the one that catches the canonical-ordering bug, and Hypothesis finds it in seconds.

Generate the validation pack. Configuration pins, eval results, red-team results, coverage — all of it exists in systems already. A generated pack is never stale, and revalidation becomes a re-run rather than a writing exercise.

10. Testing evidence code

Three that are usually missing.

Contract tests per emitter. For each of the six stores, write a record with a join key, read it back, assert the key survived. It is the only thing that catches a downstream schema silently dropping the column, which is the most common way the chain breaks.

Time-travel tests. Generate a pack for an action whose trace has aged out of the 30-day store. Assert it succeeds with the trace stated as absent by retention policy rather than failing or — worse — silently omitting it. The predicted gap is a design decision; an unpredicted one is a finding.

A mock examination. Take a real past action, give the pack to somebody who was not involved, and ask them the six questions. What they cannot answer is the gap. Twenty minutes, and it finds things no unit test does.

11. Contributing

OpenLineage (OpenLineage/OpenLineage) — Java, Python, spec. The highest-value contribution from this phase is a facet: model configuration, policy decision, retrieval snapshot, residency. All four are genuine gaps, and a bank running agents in production has the use case the community asks for. Start with an issue describing the need.

Marquez (MarquezProject/marquez) — Java + React, moderate size, welcoming. The lineage traversal and the dataset-versioning model are worth reading whether or not you contribute.

in-toto (in-toto/in-toto, in-toto/attestation) — Python and Go, focused, and the attestation predicate format is where model-provenance work would land.

Sigstore (sigstore/rekor) — Go, active. The best way to understand transparency logs is to read this rather than the RFC.

OpenSSF Model Signing (sigstore/model-transparency) — young, directly relevant, and signing model artifacts is exactly the gap between supply-chain provenance and model risk.

NIST AI RMF playbook — not code, and community contributions to the profiles are open. A worked mapping from a real bank's agentic controls to the RMF functions would be genuinely useful and does not exist publicly.

For all of them the useful preparation is the same: implement the mechanism yourself first — the lab is a small version of exactly that — then read theirs and find every place they differ. The differences are where the real engineering is, and in this area they are also where the unsolved problems are.