« All Roles

Apache Committer / PMC Engineer — Complete Apprenticeship Curriculum

Target Roles:

• Apache Committer — Spark, Hadoop, Hive, Tez, Flink, and related ASF projects
• PMC Member — Apache Software Foundation
• Staff / Principal Engineer — Data Infrastructure (Databricks, Cloudera, LinkedIn, Uber, Netflix, Meta, Apple)
• Distributed Systems Lead — JVM ecosystem
• Principal Engineer — Open Source Infrastructure

Duration: 24–36 weeks core — extendable to 18 months for full committer readiness
Goal: Develop the technical depth, community understanding, and engineering discipline required to contribute at the committer and PMC level in major Apache Software Foundation projects.

Why This Curriculum Exists

Most engineers who want to contribute to Apache projects approach it backwards: they submit a patch, wait, get confused by the silence or feedback, and quietly give up.

The reason is not lack of technical skill. The reason is that Apache projects run on a trust model that is almost never explicitly documented. Committers and PMC members are not evaluating your code alone. They are evaluating whether you are the kind of engineer they would want to maintain this system with for the next decade.

This curriculum is built backward from what actually makes someone a trusted Apache committer:

• Deep familiarity with JVM distributed systems and their failure modes
• The discipline to write patches that other engineers can review and maintain
• The ability to navigate unfamiliar codebases at production scale
• An understanding of how Apache releases, governance, and consensus actually work
• Experience reviewing code, writing design docs, and operating these systems — not just using them

Reference Apache Projects and Ecosystems

• Apache Spark → Phases 01–02, 06–07, 09, 11
• Apache Hadoop + YARN → Phases 03, 08, 10
• Apache Hive → Phases 03, 09, 10
• Apache Tez → Phases 03, 09
• ASF Release Process → Phase 04
• Cross-project → Phases 05, 11–12

What You Will Build

By the end of this curriculum you will have produced:

• Contributor landscape maps for real Apache projects with committer journey analysis
• Trust gap audits — structured assessments of what separates contributors from committers
• Annotated codebase traces through real Apache Spark and Hadoop source trees
• Post-mortems of documented Apache ecosystem design failures
• A full release checklist and simulated RC vote thread for a production Apache release
• Code review annotations — reviewer-quality comments on real and simulated patches
• Refactored Java code — taking "clever" production code to maintainable quality with tests
• A graduated build-and-review lab ladder — Raft election and Kafka partitioning, JMH/GC/lock-contention performance work, contribution tooling (diff parser, semver engine, issue state machine), release tooling (checklist engine, compat checker, RC vote parser), up to PMC-level API evolution and consensus labs
• A complete design document at subsystem level with alternatives, tradeoffs, and rollout analysis
• A final committer-simulation package — patch, tests, review response, release note, and reflection

Folder Structure

PMC-Apache-Comitter-Engineer/
├── README.md                                          ← You are here
├── jd.md                                              ← Target role profile
├── phase-01-apache-contributor-foundations/           ← Contributor ladder, trust, stewardship
├── phase-02-jvm-codebase-navigation/                  ← Navigate and trace Apache codebases
├── phase-03-lessons-from-large-systems/               ← Technical and human lessons from Hadoop/Hive/Tez/Spark
├── phase-04-asf-governance-release/                   ← Release lifecycle, signing, japicmp compatibility
├── phase-05-code-quality/                             ← Checkstyle, SpotBugs, refactoring "clever" code
├── phase-06-review-culture/                           ← Reviewing flawed patches, design tradeoffs, patch archaeology
├── phase-07-production-systems/                       ← Raft election, Kafka partitioning, multi-module Maven
├── phase-08-performance-engineering/                  ← JMH, GC/allocation profiling, lock contention
├── phase-09-apache-contribution-mechanics/            ← Diff parsing, semver, issue state machines
├── phase-10-release-engineering/                      ← Release checklist engine, API compat checker, RC vote parser
├── phase-11-issue-journey-pmc/                        ← API migration, format migration, consensus tracking
├── phase-12-final-apprenticeship-milestone/           ← Full contribution cycle capstone
├── interview-prep/                                    ← Committer-track interviews, systems design, behavioral
└── system-design/                                     ← Scheduler, metastore, API evolution walkthroughs

24-Week Core Schedule

Lab Structure

Every lab follows a consistent format:

Code labs ship as complete, tested reference implementations rather than TODO stubs: the committer skill being trained is reading, reviewing, and extending real code under a test harness — run mvn test, then do the extension exercises in each lab README.

Prerequisites

• Java 11+ fluency (read and write production Java; understand generics, concurrency basics, exceptions)
• Maven fundamentals (multi-module builds, mvn compile test package)
• Git (branch, rebase, cherry-pick, bisect)
• Basic distributed systems concepts (RPC, consensus, replication, fault tolerance)
• Comfortable reading JIRA issues and GitHub pull requests
• Familiarity with at least one of: Spark, Hadoop, or Hive as a user