👨🏻 Brother Talk — Phase 14, Off the Record

Security and governance — the phase that's boring until it's a breach or an audit. Let me make you the person who builds it in before either.

Brother, here's the uncomfortable truth about security and governance work: it's invisible when it's done right and career-ending when it's done wrong. Nobody throws a party because PII didn't leak this quarter. But the day a misconfigured bucket exposes customer SSNs, or an auditor asks "prove this analyst never saw payment data" and you can't, it's a very bad day that ends up in the news and sometimes in someone's exit interview. So the move is to build the controls in from the start, quietly, as part of how the platform works — not as a panicked retrofit after the incident. The engineers who do this don't get applause; they get trusted with the sensitive systems, which is its own kind of advancement.

Let me give you the things that actually bite.

The KMS gotcha will eat hours of your life until you internalize it. Someone will have perfect S3 IAM permissions and still get Access Denied, and they'll stare at the IAM policy convinced it's right, because it is. The catch: the object is KMS-encrypted, and who can decrypt is governed by the KMS key policy, separate from S3 IAM. No kms:Decrypt on the CMK, no read — full stop. The first time you debug this it's maddening; after that, "permissions look right but it's denied" should make you immediately check KMS. Knowing this one thing makes you look like a wizard when a teammate is melting down over a permissions mystery.

Deny always wins, and people forget it constantly. The number of times I've seen someone add a broad Allow s3:* and expect it to grant access to a path that a separate statement explicitly denies… it doesn't, and it never will. Explicit Deny beats any Allow, from any policy, in any order. This is good — it means you can put a guardrail Deny on pii/* and know no amount of over-broad granting downstream can punch through it. Use that. A targeted Deny is a seatbelt nobody can unbuckle by accident.

Tag PII at the schema, and I will keep saying this until you do it. I said it in P03 and I'll say it again here because it's that important: the difference between a sane governance program and a nightmare is whether PII is labeled at the source, when the field is defined. If it is, masking, access control, retention, and erasure all have something to hang off of — they're a matter of policy on a tag. If it isn't, you're doing forensic data archaeology across ten thousand columns under a regulatory deadline, and you will miss some, and the ones you miss are the breach. Ten seconds at definition time versus a quarter of misery and risk later. This is the single highest-leverage governance habit, and it costs almost nothing.

The GDPR-erasure-on-a-lakehouse trap is a genuine compliance landmine, and it's subtle. A junior runs DELETE FROM users WHERE id = X, confirms the row is gone, and reports compliance. Wrong — and seriously wrong. On Iceberg/Delta, old snapshots still reference the files containing that row, so time travel resurrects it. Real erasure requires deleting from current and expiring the snapshots that reference the data and removing orphan files — and following the lineage (P13) to wherever that PII flowed (derived tables, the serving store). The same immutability that gives you beautiful time travel makes deletion a multi-step, lineage-aware operation. The person who knows this prevents a compliance incident the rest of the team didn't even know was lurking.

Govern as code, or governance rots. Click-ops security is governance theater: someone makes a change in the console, nobody reviews it, and six months later you have no idea why a bucket is public or a role is over-broad. Terraform turns every security change into a diff in a pull request — a human and a policy-as-code gate (tfsec, OPA) review it before it ships. A bucket going public shows up as a red line someone has to approve. That review trail is worth more than any amount of after-the-fact auditing, because it stops the misconfiguration before it's live. Make the controls themselves code.

On DR and multi-region: the thing nobody tells you is that a disaster recovery plan you've never tested is fiction. Everyone has the doc. Few have actually failed over to the secondary region under realistic conditions and discovered the seventeen things that don't work — the DNS that didn't update, the replica that was hours behind (RPO!), the runbook step that assumed a human who's asleep. Game-day your failover. The teams that survive a regional outage are the ones who practiced, and being the person who runs those drills is how you find the gaps while they're cheap instead of during the actual fire.

Career truth: governance and security maturity is what lets a data platform handle regulated data — finance, health, anything with real PII — and that's where the high-stakes, high-budget work is. The engineer who can say "yes, we can put regulated data on this platform, here's how access, encryption, audit, and erasure are handled, and here's the Terraform that proves it" is the engineer who unlocks the most valuable use cases. It's not flashy, but it's trust, and trust is the currency of the principal level.

Build the kit. Make Deny win. Mask the PII. Audit the access. Then come to P15 — the leadership phase, where you stop building components and start setting the technical direction: ADRs, migrations, build-vs-buy, design reviews, and operating as a principal.

— your brother 👨🏻