Warmup Guide — Issue Journey: PMC Level

Zero-to-expert primer for Phase 11: the three problems that define PMC-level judgment — evolving deprecated APIs across an ecosystem, migrating storage formats without losing anyone's data, and driving consensus when committers disagree.

Table of Contents

Chapter 1: What Makes an Issue "PMC-Level"

The earlier phases' issues had a correct answer findable in code. PMC-level issues have three different properties:

  1. The blast radius is the ecosystem, not the codebase: thousands of downstream users with different upgrade cadences, including unmaintained deployments.
  2. Every option is a tradeoff between constituencies: users vs maintainers, fast movers vs stable-branch operators, this project vs sibling projects sharing the format.
  3. The decision needs legitimacy, not just correctness: a technically right call made without process gets relitigated forever; a consensus-built call sticks.

Consequently the artifacts change: not patches but analysis tools (Lab 01), plans (Lab 02), and decision records (Lab 03). The labs build software for each because encoding the reasoning in code forces it to be precise — the same encode-the-process philosophy as Phase 09/10, now at governance altitude.

Chapter 2: Deprecation as a Lifecycle, Not a Label

@Deprecated is the start of a multi-year protocol (Phase 04/10 gave the rules; here's the lifecycle a PMC actually manages):

  1. Replacement exists and is GA — deprecating without a destination is hostile; "no-replacement" deprecations need their own justification class.
  2. Annotate + document: @Deprecated(since, forRemoval) + Javadoc naming the replacement and the earliest removal version. The docs are the contract.
  3. Migrate yourself first: the project's own code and examples stop using it — internal usage is both hypocrisy and a usage-count confounder.
  4. Measure adoption: usage telemetry where it exists; otherwise corpus scans (Maven Central reverse-deps, code search). This is Lab 01's input data.
  5. Escalate signal: release notes → runtime warn-once logging for high-traffic methods → build-breaking only at the major version that removes it.
  6. Remove at a major release, with the migration recipe linked — and only when the measured cost of removal (Chapter 3) clears the bar.

The institutional failure modes: deprecation rot (marked in 2014, never removed — the annotation loses all meaning project-wide) and stealth removal (removed without the lifecycle — trust damage exceeding any cleanup benefit). A PMC's deprecation hygiene is measurable, and Lab 01's analyzer measures it.

Chapter 3: Modeling Migration Urgency

Lab 01's ApiMigrationAnalyzer encodes the judgment as a model — the intellectual move worth internalizing: turn a governance argument into a scoring function with explicit inputs, so disagreement localizes to weights instead of vibes:

  • Inputs per deprecated API: deprecated-since (versions elapsed — staleness), usage count (ecosystem cost of removal), replacement availability (is migration even possible), maintenance burden (what the project pays to keep it — shims, bug surface, blocked refactors).
  • Output: urgency tiers — e.g., REMOVE_NOW (stale + low usage + replacement GA), ACCELERATE (high burden, replacement GA, usage declining — invest in warnings + tooling), HOLD (high usage or no replacement — the analysis says don't remove, whatever the annotation's age), and BLOCKED (no replacement: removal conversation is premature by definition).
  • The model's honest limits: usage counts undercount dark deployments; burden is estimated; weights are policy. The tool's job is to make the inputs auditable and the disagreement explicit — which is precisely what a PMC discussion needs from a staff engineer.

Chapter 4: Storage Format Migration — The Hardest Compatibility Problem

Why formats outrank APIs in difficulty: data outlives code by a decade, and the readers/writers span versions you don't control.

The compatibility vocabulary (precise, because plans depend on it):

  • Backward compatibility: new code reads old data. The non-negotiable baseline — losing it strands every existing byte.
  • Forward compatibility: old code reads new data (perhaps degraded). Needed whenever readers upgrade after writers (mixed clusters, rolling upgrades, sibling projects reading your files).
  • The version-skew matrix: writers {old, new} × readers {old, new} — a migration plan must state the behavior of all four cells explicitly, per phase of the rollout. Most migration postmortems trace to an unstated cell (typically new-writer/old-reader during a partial rollout).

Format-design mechanics that make evolution survivable (and that Lab 02's capability registry models): explicit format version stamps in the data; self-describing schemas (Parquet/ORC footers); field-tagged encodings tolerating unknown fields (protobuf-style) vs positional encodings that can't; and capability negotiation — readers declare what versions they read, writers choose what to emit (Kafka's API-version handshake; HDFS rolling-upgrade machinery).

Chapter 5: Planning a Format Migration

The phase structure every safe migration follows (Lab 02's FormatMigrationPlanner generates instances of it):

  1. Dual-read ships first: all readers understand v2 before any v2 byte exists. Cheap, riskless, and it starts the clock on the skew window.
  2. Dual-write / shadow-write (optional but powerful): write both formats; verify v2 against v1 continuously (the Phase 06 system-design shadow-read pattern, applied to storage). Catches semantic drift while v1 remains the truth.
  3. Default flip: new data writes v2; old data still v1; readers handle both (forever, or until step 5). Reversible by flag — the property to preserve as long as possible.
  4. Background rewrite (only if needed): migrate historical data v1→v2 with checksummed verification, rate-limited, resumable. Many migrations never do this — "old data stays v1, readers keep v1 support" is often the right end state (cost of rewrite vs cost of carrying the reader).
  5. Reader sunset: drop v1 support only at a major version, after telemetry shows v1 reads ≈ 0, with a documented escape (a standalone converter tool).

Each phase has an explicit rollback story; the plan states the skew matrix per phase; and the sibling-project problem (Hive files read by Spark/Impala/Trino) means the ecosystem's slowest reader — not yours — gates the sunset. That last sentence is the PMC-level content: your migration plan is a multi-project treaty.

Chapter 6: Consensus Mechanics When Committers Disagree

The Apache decision stack, from cheapest to heaviest (Phase 04 gave the rules; here's the practice when a real disagreement is live):

  1. Clarify what kind of disagreement: facts (resolvable by benchmark/prototype — go produce one), priorities (a values conversation — name the constituencies), or taste (defer to whoever maintains it). Misdiagnosing the kind wastes months.
  2. Write the options down neutrally — a short doc with each option's costs stated by its opponents' lights (steel-manning as process). Most "deadlocks" dissolve when both options' real costs are on one page.
  3. Seek the synthesis or the experiment: a flag-gated trial, a scoped pilot in one module, a reversible default — anything that converts argument into evidence.
  4. Lazy consensus → explicit [DISCUSS] → [VOTE] only as escalation; a vote on a contested design is a failure to converge, acceptable but costly (the losing side's energy often leaves with the vote).
  5. Vetoes: a justified technical veto on a code change stands until addressed — the system is designed to let one informed objection stop the bus. The senior skill is writing vetoes that are clearly technical and clearly addressable, and receiving them without escalation (Phase 12 Ch. 3).

Chapter 7: Tracking and Closing a Contested Decision

Lab 03's ConsensusEvaluator encodes thread state so that "where are we" is computable:

  • Positions are per-person, current-state (supersession, as in Phase 10's vote parser): the tracker holds each participant's latest stance + its stated rationale class (technical objection / preference / question).
  • Open objections are first-class: a decision isn't "consensus" while a justified technical objection stands unaddressed — the evaluator distinguishes addressed (author responded with change or evidence; objector confirmed) from answered-but- unconfirmed (ping needed) from open.
  • Closure is an artifact: the [RESULT]/decision record states what was decided, the alternatives rejected and why, and who dissented — dissent recorded respectfully is what lets the dissenter commit to the outcome (disagree-and-commit has to be visible to be real).
  • The cultural payload: decision records are how a 20-year project avoids relitigating 2014 every time a new committer arrives. Writing them well is PMC work.

Lab Walkthrough Guidance

Order: Lab 01 (migration analyzer) → Lab 02 (format planner) → Lab 03 (consensus tracker) — API surface, then data, then people; each is a strict superset in blast radius.

  • Lab 01: encode Chapter 3's model; test the tier boundaries (the interesting cases are near thresholds — high-usage-but-ancient, no-replacement-but-zero-usage); write one paragraph defending your weights as if to a skeptical PMC.
  • Lab 02: for each plan the planner emits, verify it states all four skew-matrix cells per phase (make that a unit test — plans missing a cell are invalid, which is the lesson); include one scenario where the right answer is "never rewrite history."
  • Lab 03: model positions/objections/supersession; the key tests are the not-yet-consensus states (open justified objection blocks closure regardless of +1 count) — the negative space is the governance.

Success Criteria

You are ready for Phase 12 when you can, from memory:

  1. Name the three properties that make an issue PMC-level and the artifact each lab produces.
  2. Recite the six-step deprecation lifecycle and both institutional failure modes.
  3. Define backward/forward compatibility and draw the four-cell skew matrix for a mid-rollout mixed cluster.
  4. Walk the five-phase migration plan with each phase's rollback story, and state when background rewrite is correctly skipped.
  5. Distinguish facts/priorities/taste disagreements with the resolution move for each.
  6. State what blocks consensus-closure regardless of vote counts, and what a decision record must contain.

Interview Q&A

Q: You own a deprecated API used by 40% of downstream projects. The maintainers want it gone. Walk me through your decision. Run the Chapter 3 model out loud: usage is high → removal cost is ecosystem-scale; what's the carrying burden (if it's a thin shim, HOLD and re-measure yearly; if it blocks a major refactor, that changes the math); is the replacement truly equivalent (40% usage often means the replacement misses a case — go find it); can tooling (an OpenRewrite recipe) collapse the migration cost? Likely answer: invest in migration tooling + warnings now, set a measurable sunset criterion ("usage <5% or two majors, whichever later"), and publish that criterion — converting a standoff into a schedule. The structure of the answer is the answer.

Q: Design the migration of a widely-read file format where you don't control all readers. Lead with the treaty problem: enumerate reader projects, get dual-read shipped in their release trains first — the slowest sibling gates everything. Then Chapter 5's phases with the skew matrix per phase, shadow-write verification, flag-reversible default flip, and an explicit "v1 readable forever + offline converter" end-state option if sibling sunset never converges. Mention what kills these migrations: unstated new-writer/old-reader cells and assuming your own release cadence applies ecosystem-wide.

Q: Two committers have argued for three weeks about a design. As the senior person arriving fresh, what do you do? Diagnose the disagreement type first (Ch. 6): read the thread and separate factual claims (benchmark them — volunteer to build the prototype), priority conflicts (name the constituencies and force the "for whom" question), and taste (assign to the maintainer-of-record). Produce the one-page neutral options doc with steel-manned costs; propose the smallest reversible experiment that discriminates. Escalate to a vote only if that fails — and if it does, write the decision record so the argument is never re-run from scratch.

References