Warmup Guide — Security, Privacy & Governance

Zero-to-principal primer for Phase 14: IAM and the Deny-wins evaluation, KMS encryption, network isolation, Lake Formation fine-grained access, PII/privacy/erasure, audit, Terraform/IaC, and multi-region/DR.

Table of Contents

Chapter 1: IAM and the Evaluation Logic

IAM controls who can do what to which resource. The pieces: principals (users, roles), policies (statements of effect + action + resource + optional condition), roles (assumable identities — services and cross-account access use these). The evaluation logic you must know cold (the lab implements it):

  1. Explicit Deny anywhere → denied, full stop. A targeted Deny cannot be overridden by any Allow, however broad.
  2. Otherwise, a matching Allow → allowed.
  3. Otherwise → implicit Deny (default-deny).

So precedence is Deny > Allow > (default) Deny. The classic mistake is assuming a broad Allow s3:* grants access to a path a separate statement Denys — it doesn't. Least privilege is the governing principle: grant only the actions actually used (the lab's unused_grants finds the gap), scope resources tightly (no Resource: *), and prefer roles over long-lived keys. Cross-account access uses AssumeRole + a trust policy on the target role; ABAC grants by tags (principal attributes × resource tags) instead of static ARNs — scalable for many resources.

Chapter 2: Encryption and KMS

  • At rest: S3/EBS/RDS encrypt with a key. SSE-KMS uses envelope encryption — your data is encrypted with a per-object data key, which is itself encrypted by a CMK (customer master key) in KMS. The subtlety principals must hold: who can read the data is governed by the KMS key policy, not only S3/IAM. A principal with s3:GetObject but no kms:Decrypt on the CMK cannot read the object — a common "permissions look right but it's still denied" puzzle.
  • In transit: TLS everywhere; enforce it (bucket policy aws:SecureTransport, broker TLS).
  • Key management: key policies + grants, rotation, separate keys per data classification/ tenant (so revoking a key revokes access to a slice), and kms:ViaService conditions.

Chapter 3: Network Isolation

Defense in depth at the network layer:

  • VPC with public/private subnets; data stores (Cassandra, RDS, MSK, EMR core) live in private subnets — no public IPs.
  • Security groups (stateful instance firewalls) and NACLs (subnet-level) restrict who can reach what.
  • VPC endpoints / PrivateLink: reach AWS services (S3, KMS, Athena) and SaaS privately, so data-plane traffic never traverses the public internet — both a security and a data-exfiltration control (combine with bucket policies that require the endpoint). The rack-management OOB-network idea (separate planes) rhymes here.

Chapter 4: Lake Formation and Glue Governance

  • Glue Data Catalog (P07/P10) is the shared metastore. Lake Formation layers fine-grained access on top: grant table / column / row-level permissions and column masking to principals, centrally, enforced by the query engines (Athena/Trino/ Spark/EMR) at scan time. The lab's apply_column_policy + row_filter are this model in miniature.
  • This is how you let analysts query a table while never seeing the PII columns, and how a tenant sees only their rows — without copying data into separate restricted tables (which forks lineage and multiplies cost). Plus cross-account data sharing via Lake Formation grants, and tag-based (LF-Tags) policies for scale.

Chapter 5: PII, Privacy, and the Right to Erasure

  • Classification at the source: tag fields by sensitivity in the schema/contract (P03) — none / pii / sensitive. The tag then drives downstream masking (Ch. 4), access, and retention. Classifying ten thousand columns after the fact under a compliance deadline is misery; tagging at definition is ten seconds (P03's BROTHER-TALK plea).
  • Masking / tokenization: show masked or tokenized values to uncleared principals; keep the mapping in a separate, tightly-controlled store if reversible tokenization is needed.
  • Retention: delete data past its policy (the lab's records_to_delete); lifecycle policies (P08) automate tiering/deletion.
  • The right to erasure (GDPR/CCPA): deleting a user's data is subtle on a lakehouse — a DELETE removes it from the current snapshot, but old snapshots still reference the files, so time travel can resurrect it (P09). A compliant erasure = delete from current + expire the snapshots that reference it + remove orphan files. Forgetting the second half is a real compliance bug. (This is why retention + snapshot expiry are coupled.)

Chapter 6: Audit, Lineage, and Dataset Certification

  • Audit: log every access and change (CloudTrail for AWS APIs; the lab's AuditLog for data access) — who did what, when, and the decision. Required for compliance and for forensic incident response (P13).
  • Lineage (P13): the field/dataset graph proves where data came from — essential for privacy (where did this PII flow?) and impact analysis.
  • Dataset certification: every production dataset has an owner, SLA, lineage, PII classification, and quality policy (a JD success metric). Certification is the governance contract that makes a dataset safe to depend on (P00's "data product").

Chapter 7: Terraform and Infrastructure as Code

  • Why IaC: infrastructure as versioned, reviewed code — reproducible environments, peer review of security changes (a bucket made public shows up in a diff), and promotion across dev→staging→prod. The opposite of click-ops drift.
  • Terraform model: declarative resources, state (the source of truth of what exists), plan (preview the diff) → apply; modules for reuse; drift detection; remote state with locking. Secrets via a manager (not in code/state).
  • Governance-as-code: encode bucket policies, KMS keys, Lake Formation grants, IAM roles, and Athena workgroup limits in Terraform so the controls themselves are reviewed and versioned. Policy-as-code tools (OPA/Sentinel, tfsec/checkov) gate insecure changes in CI.

Chapter 8: Multi-Region and Disaster Recovery

  • Why: regional outages happen; some data must live in a region (residency, Ch. 5).
  • Replication: S3 cross-region replication; Kafka via MirrorMaker 2 / MSK replication; Cassandra via NetworkTopologyStrategy + per-DC replicas (read/write at LOCAL_QUORUM to avoid cross-region latency, P11); DynamoDB Global Tables.
  • RPO / RTO: Recovery Point Objective (how much data you can lose) and Recovery Time Objective (how fast you recover) — the two numbers that define your DR design and its cost.
  • Active-passive (failover to a standby — cheaper, slower) vs active-active (both serve — costlier, instant, but you must handle multi-region writes/conflicts, P01 vector clocks/LWW). The choice is a P00 dial (cost vs RTO vs consistency), written as an ADR (P15).
  • DR is only real if tested: game-day failovers, not a doc nobody's exercised.

Lab Walkthrough Guidance

Lab 01 — Governance & Security Kit, suggested order:

  1. iam_evaluate — explicit Deny short-circuits; matching Allow; else implicit Deny. Test that Deny wins regardless of statement order.
  2. unused_grants — least-privilege gap.
  3. apply_column_policy + mask_for — mask by class unless cleared.
  4. row_filter / region_filter — row-level security.
  5. AuditLog + audited_access — record every decision; query denials.
  6. retention_expired / records_to_delete — lifecycle/erasure.

Success Criteria

You are ready for Phase 15 when you can, from memory:

  1. State the IAM evaluation order and why a Deny beats any Allow; define least privilege.
  2. Explain KMS envelope encryption and why kms:Decrypt matters beyond s3:GetObject.
  3. Explain VPC endpoints/PrivateLink and private-subnet data stores.
  4. Explain Lake Formation column/row security and why it beats copying into restricted tables.
  5. Explain PII tagging at the schema and a compliant lakehouse erasure (snapshot expiry).
  6. Explain why governance should be Terraform/code and what policy-as-code gates.
  7. Design a multi-region/DR strategy with RPO/RTO and active-active vs active-passive.

Interview Q&A

Q: A principal has s3:GetObject on the bucket but still gets Access Denied. Why? Two usual causes. First, KMS: the object is SSE-KMS encrypted, and the principal lacks kms:Decrypt on the CMK — S3 lets them list/get the ciphertext request but KMS refuses to unwrap the data key, so the read fails. Who can decrypt is the key policy, not just IAM. Second, an explicit Deny: a separate statement (or an SCP/permission boundary, or a bucket policy) denies the path, and Deny beats the Allow. I'd check the KMS key policy and grants first, then look for an explicit Deny in the identity policy, bucket policy, SCP, or Lake Formation. "Permissions look right but it's denied" is almost always KMS or a Deny you didn't see.

Q: Analysts need to query the orders table but must never see the SSN column. How? Lake Formation column-level security (or the query-engine's masking), not a copy. I tag ssn as sensitive PII at the schema (P03), grant analysts table access with the SSN column masked/ excluded via a Lake Formation permission, and the query engine enforces it at scan time — so the analyst can SELECT * and simply never receives SSN, while a cleared role does. The anti-pattern is materializing a separate "orders_no_pii" table: it forks lineage, doubles storage, and drifts. One table, enforced fine-grained access — that's the lab's apply_column_policy.

Q: We got a GDPR erasure request for a user on our Iceberg lake. Walk me through it. DELETE the user's rows from the current table, but that's not sufficient: older Iceberg snapshots still reference the data files containing those rows, so time travel could resurrect them. The compliant flow is delete-from-current → expire the snapshots that still reference those files → remove orphan files (vacuum), so the underlying data is actually gone, then record the erasure in the audit log. This couples retention policy with snapshot expiry (P09). I'd also confirm via lineage (P13) that the PII didn't flow into derived tables or a serving store (P11) that need the same treatment — erasure follows the data's lineage, not just one table.

Q: Why manage security with Terraform instead of the console? Because security changes must be reviewed, versioned, and reproducible. In Terraform, a change that makes a bucket public, widens an IAM role, or alters a KMS key policy shows up as a diff in a pull request that a human (and a policy-as-code gate like tfsec/OPA) reviews before it ships — versus a console click nobody sees until the audit. You also get reproducible environments (dev mirrors prod), drift detection (someone clicked something → the plan shows it), and the ability to roll back infra like code. Governance-as-code turns "we hope it's configured right" into "the configuration is the reviewed, tested artifact."

References