Hitchhiker's Guide to Phase 11 — PMC-Level Engineering

1. Thirty-Second Mental Model

PMC membership is not a reward for writing good code. It is a responsibility to protect the project's users from decisions that will hurt them for years. That means thinking in terms of contracts (what did we promise?), timelines (how long must we support this?), and consensus (who has to agree before we ship?).

2. Apache Deprecation Lifecycle

STABLE → DEPRECATED → REMOVED

Between DEPRECATED and REMOVED there must be at least two major release cycles by ASF convention (Spark enforces this strictly).

Real Spark example: DataFrame was deprecated in favor of Dataset[Row] in Spark 2.0. The old alias was retained as a type alias, not truly removed, to avoid breaking compiled code. That distinction — source compatible vs. binary compatible — is exactly what Lab 01 models.

3. Storage Format Migration Sequencing

You cannot jump directly from Text to Parquet on a live production table if downstream jobs are writing in Text format. The migration must be staged:

Text → ORC → Parquet

Each step must be:

1. Forward-only readable — new readers can read old data.
2. Rollback-safe — if the new format causes query regression, you can rewrite back without data loss.

Parquet is NOT rollback-safe from ORC because of type representation differences (e.g., TIMESTAMP precision). Lab 02 models this decision.

4. Capability Deltas Between Storage Formats

A migration from ORC to Parquet loses ACID transaction support. This is a PMC-level concern: the release notes must call this out explicitly.

5. Lazy Consensus vs. Explicit Vote

ASF uses two consensus mechanisms on the dev@ mailing list:

Lazy consensus — silence = approval, after a minimum 72-hour window. Used for: code commits, minor config changes, patch approvals.

Explicit vote ([VOTE] thread) — requires ≥ 3 binding +1 and no binding -1. Used for: releases, new committers, PMC membership, major policy changes.

A design proposal (not a release, not a committer vote) typically uses lazy consensus: the proposer sends [DISCUSS] then [VOTE] or just [PROPOSAL], waits 72 hours, and proceeds if no binding objection is raised.

Lab 03 models both paths: a proposal that achieves lazy consensus, and one that is blocked by an explicit objection requiring resolution.

6. What Makes an Objection Blocking?

An objection on the dev@ list is binding when:

1. It is raised by a PMC member (binding voter status).
2. It is a technical objection — not a preference.
3. It is unresolved — the proposer has not addressed the concern.

An objection with a proposed fix that the objector agrees resolves the issue is no longer binding. Lab 03 tracks this resolution state.

7. Interview Q&A

Q: You deprecated an API in version 2.0. A customer is still using it in version 3.5. What do you do?

A: Check whether the removal was announced with sufficient notice (at least two majors). If yes, the contract was honored — help them migrate. If no, extend the deprecation period and file a JIRA to track the delayed removal. Never surprise users.

Q: How do you build consensus across 20 engineers on a design decision?

A: Write a clear proposal with alternatives and explicit tradeoffs. Send [DISCUSS]. Give ≥ 72 hours (more for complex proposals). Address each technical objection in a follow-up. If a blocking objection remains after addressing it, it is time for a formal [VOTE]. If you rush consensus you will get a revert within one release cycle.

Q: What is the difference between lazy consensus and a formal vote?

A: Lazy consensus means no binding objection was raised. It does not mean everyone agreed — it means no one objected. Formal votes require active +1s. Releases, committer additions, and major policy changes require formal votes. Feature proposals generally use lazy consensus.