👨🏻 Brother Talk — Phase 00, Off the Record

No slides, no STAR method. This is me, your brother, telling you the things people only say after the second coffee. Read it once now, and again the week before your interview.

Listen. You've probably been told that becoming a principal data engineer means learning more tools. It doesn't. I've watched brilliant engineers memorise Flink, Spark, Iceberg, Kafka — all of it — and still get stuck at senior, because nobody told them the secret:

The tools are the easy part. The judgment is the job.

Here's the thing that took me too long to learn. When a director says "design our event platform," they are not testing whether you know acks=all. They already assume you do. What they're watching is: do you ask what a duplicate costs before you promise exactly-once? Do you ask the read-to-write ratio before you pick a database? Do you say "a minute of staleness is worth roughly $X to us, so I'll trade freshness for cost here"? That sentence — pricing a tradeoff out loud — is the whole game. Everyone else in the room is reciting features. You're doing engineering.

So let me give you the mindset shifts, brother to brother.

Stop saying "real-time." I mean it. The word is banned for you now. Every time you want to say "real-time," say a number and a percentile instead: "p99 under 60 seconds." The moment you do this, you sound three levels more senior, because you've revealed you know that "real-time" is a budget, not a property. A junior wants real-time everywhere. A principal knows real-time is the most expensive thing on the menu and orders it only where the business pays for it.

Fall in love with the boring force: operability. Everyone optimises latency and throughput because they're sexy and they benchmark well. Nobody brags about "this is easy to debug at 3 a.m." But operability is the force that actually decides whether your platform survives, because a system runs for years and gets debugged thousands of times. The "elegant" exactly-once pipeline that takes a PhD to debug is a liability. The "boring" at-least-once-with-dedup pipeline that any on-call can reason about is an asset. Choose boring on purpose. It's a flex, not a compromise.

Replay is your superpower; protect it. The first time you have an incident — and you will — and you can say "no problem, I'll replay the last six hours from the log and the output will be byte-identical because everything's event-time and idempotent" — that's the moment people start treating you like a principal. So everything you build, ask: can I replay this and get the same answer? If the answer is no, you haven't built a pipeline, you've built a time bomb. The whole reason we obsess over event-time and idempotency in this track is to keep that superpower.

Write the ADR even when nobody asks. Here's a career hack that feels like overhead and pays off enormously: when you make a real decision, write the one-pager — context, decision, alternatives, what would change my mind. Do it even for decisions nobody requested documentation for. Six months later when someone says "why did we do it this way?" you don't defend it from memory and emotion — you point at the doc and say "here were the numbers; have they changed?" You become the person whose decisions are legible. That person gets trusted with bigger decisions. That's the ladder.

The honest truth about this role's difficulty: it's hard because it's wide, not because any one piece is deep beyond reach. Nobody is born knowing Kafka and Spark and Cassandra and Cats. The people who get here didn't have bigger brains; they had a framework (the lifecycle, the five forces, the dials) that let them learn each new tool as "oh, this is the streaming-substrate slot, here's how it answers the same five questions." That framework is this entire phase. Internalise it and the other 16 phases stop being 16 unrelated manuals and become one system you already understand.

One more thing, and it's the most important. You don't have to be the smartest person in the room to be the principal in the room. You have to be the one who turns vague fear ("will this scale? is this correct? what will it cost?") into specific, answerable questions with numbers attached. Calm comes from numbers. When the room is panicking about "will this handle Black Friday," and you say "peak is 4× average, we're sized to 6×, here's the headroom, here's what sheds load first if I'm wrong" — you're not smarter than them. You just did the arithmetic they were too anxious to do. That's leadership in this job. It's learnable. It starts with the lab in this phase.

Go build the modeler. Then come find me in Phase 01 — that's where we make the distributed systems stuff click, the stuff everyone pretends to understand and quietly doesn't.

— your brother 👨🏻