The Project Scaffold

Fifteen projects over 34 months, each starting from an empty directory, is fifteen chances to lay the repository out differently and to forget the notebook, the AI log, or the raw-sample discipline. This generates the structure every project page assumes.

cd scaffold
./new-project.sh <slug> <python|rust|go> [target-dir]

Verified for all three languages: the generated project builds, its tests run, its benchmark emits raw samples, and git is initialised with a first commit.

Table of Contents

• What It Generates
• The Five Make Targets
• Design Decisions
• Why the Placeholder Test Fails
• Language Notes
• What It Deliberately Does Not Do
• Verified Output

What It Generates

<slug>/
  README.md            problem statement, reproduce block, headline result
  RESUME.md            where you stopped + the next 60-second action
  EXIT-CRITERIA.md     what "done" means — filled in BEFORE coding
  REPORT.md            the deliverable, from templates/report.md
  AI-LOG.md            where an assistant materially changed a design
  Makefile             setup / test / bench / check / exit
  src/ (or internal/)  the mechanism. Hand-written
  tests/               correctness. One placeholder that FAILS on purpose
  bench/ (or cmd/)     benchmark driver, importing the shared harness
  tools/               bench.py, metrics.py, roofline.py — copied, not re-invented
  notebook/            000-TEMPLATE.md, 000-EXPERIMENT.md ready to copy
  results/             raw samples. Committed, not ignored

Three placeholders are substituted: __SLUG__, __UNDER__ (underscored, for Python packages and Rust crates), __DATE__. Three and no more — a scaffold with twenty template variables becomes a thing you maintain instead of use.

The Five Make Targets

Identical across all three languages, so muscle memory transfers between projects even when the language does not.

Go additionally has make race, and make check includes it — the race detector finds real bugs in P05–P07 and should never be optional there.

Design Decisions

bench depends on test. The single most consequential line in the Makefile. One of this track's operating rules is no performance work while a correctness test is red, and a rule enforced by a build graph is worth more than a rule written on a page you read once.

results/ is committed, not gitignored. You cannot recover a distribution from a p50. Six months later you will want to re-percentile, check for bimodality, or bootstrap a different statistic, and only the raw samples let you. The generated .gitignore says so explicitly so you do not "helpfully" add it later.

tools/ is copied, never edited in place. All three benchmark drivers import the same harness so that p50/p95/p99 with a bootstrap interval is the default and not a decision. Edits belong upstream in the track's tools/, then re-copied — otherwise fifteen divergent copies of bench.py make fifteen projects incomparable.

RESUME.md ships pre-filled with its own instructions. It is the highest-leverage habit in the program (why) and the easiest to skip on day one, so the file exists before you have a reason to create it.

EXIT-CRITERIA.md includes the eight-item completion gate verbatim, separate from the project-specific criteria, because the gate is not negotiable by scoring well elsewhere. It also has a scope-cuts table — cuts are recorded when taken, since a cut rationalised at the end is a story rather than a decision.

Why the Placeholder Test Fails

Every generated project is red on make test until you delete one test.

A scaffold that is green on creation teaches you that green means nothing. Making it red forces you to open the test file as your first act, which is where the comment lives that says: write a property test, before the code it covers.

$ make test
FAILED tests/test_placeholder.py::test_replace_me - Failed: SCAFFOLD: replace...
1 failed, 1 passed

The one that passes asserts the mechanism is still unimplemented — it documents the state rather than testing behaviour. Both go when you write the first real test.

This was not the first design. The initial version used pytest.mark.xfail(strict=True), Rust's #[should_panic], and Go's t.Skip — and all three reported green, which is precisely the lesson the placeholder is supposed to prevent. Caught by running it.

Language Notes

Python

tests/conftest.py puts src/ and the project root on the path, so there is no packaging step between you and a failing test. This is a research repo, not a distribution.

bench/run.py ships with workload_baseline and workload_candidate that are identical, so a fresh scaffold prints:

  -> OVERLAPPING CIs — no measurable difference

That is the harness demonstrating the behaviour you want from it before you have written anything: two identical functions produce an honest verdict rather than a manufactured 1% win.

Rust

Cargo.toml sets debug = true under [profile.release] so a profiler keeps symbols, and make bench uses --release because a debug-build benchmark is meaningless — bounds checks and absent inlining can cost 10–50×.

The bench driver uses std::hint::black_box, with a comment explaining why. Without it, --release constant-folds the example workload to a literal and the benchmark reports p50=0.0000ms — the "compiler deleted my benchmark" failure from numbers §14. The first version of this template had exactly that bug; it was found by running it, and the fix is now part of the teaching material. With black_box the same workload measures 0.3 µs.

Go

make race exists as its own target and is included in make check. The race detector is the highest-value tool in Stage 3 and costs ~10× runtime, which is why it is separate from the fast make test loop but mandatory before declaring anything done.

Percentiles use nearest-rank in all three languages, so the reported p99 is always an observation that actually occurred rather than an interpolation between two.

What It Deliberately Does Not Do

• It does not scaffold the mechanism. src/ contains one function that raises NotImplementedError. The mechanism is the project; generating it would delete the point.
• No CI configuration. A local make check is sufficient at this scale, and CI for a solo research repo is not yet.
• No Docker, no Nix. Pinned versions and a documented command are what reproducibility means here. Reach for more only after a real reproducibility failure.
• No logging or config framework. Both are peripheral concerns that expand to fill the time available.

Verified Output

Every claim above was executed. Generating all three and running the full cycle: