Lab 02 — Seccomp Policy Regression Analyzer

Difficulty: 3/5 | Runs locally: yes

Pairs with the Phase 06 WARMUP Chapter 6 (seccomp-BPF) and the advanced seccomp compiler lab.

Why This Lab Exists (Purpose & Goal)

A seccomp allowlist is not a write-once artifact — every dependency upgrade can change which syscalls a program makes, and the two failure modes are opposite and both bad: too tight and the upgraded service breaks in production; too loose and the sandbox quietly widens. The goal of this lab is to build the analyzer that compares a reviewed allowlist against the calls observed after an upgrade, classifying the difference so a human can make an informed re-approval.

The Concept, In the Weeds

The analyzer sorts post-upgrade syscalls into three buckets:

  • expected additions — calls the upgrade legitimately needs (a new library uses getrandom), candidates for approval;
  • forbidden calls — calls that should never appear (ptrace, mount, raw networking) and signal something is wrong;
  • unexplained policy expansion — additions with no clear justification, which must be reviewed, not rubber-stamped.

The crucial discipline — the same as the Phase 01 syscall-policy lab — is that the tool does not auto-generate or auto-approve the allowlist. Observation is a proposal; approval is a human decision. And the observations themselves must be complete: you must collect traces from normal, error, DNS, TLS, shutdown, and load paths, because a single happy-path trace under-counts the real syscall set and an allowlist built from it will break the moment the service hits an unobserved path.

Why This Matters for Protecting the Company

Seccomp profiles in production rot. Without regression analysis, teams respond to a post-upgrade breakage by loosening the profile (or disabling it) — silently re-opening the kernel attack surface the profile was protecting. A regression analyzer keeps the profile tight and current: it tells you exactly what changed, so you re-approve deliberately instead of reaching for allow-all. This is how a company keeps syscall confinement effective over the lifetime of a service, not just on day one. The "observe → classify → human-approve" loop is also the model for keeping any security baseline current under change.

Build It

Implement the analyzer: compare a reviewed allowlist with calls observed after an upgrade; classify expected additions, forbidden calls, and unexplained expansion. Do not auto-generate the allowlist.

LAB_MODULE=solution pytest -q
# Collect owned Linux traces from normal, error, DNS, TLS, shutdown, and load paths before review.

Validation — What You Should Be Able to Do Now

  • Classify a post-upgrade syscall diff into approved / forbidden / unexplained, and explain why each bucket exists.
  • Explain why you must trace many paths (error/DNS/TLS/load), not one, to build a complete baseline.
  • Explain why "auto-approve observed behavior" and "loosen on breakage" are the two ways seccomp profiles rot, and how regression analysis prevents both.

The Broader Perspective

This lab teaches drift management — the recognition that security baselines decay under change and must be continuously, deliberately re-verified. Every control you build has this property: detections break when telemetry changes (Phase 09), IAM accretes privilege (Phase 07), threat models go stale. The mature pattern is always the same: measure the drift, classify it, and route changes through human review rather than auto-accepting or abandoning the control. A security baseline is a living thing; keeping it alive is as important as setting it up.

Interview Angle

  • "How do you keep a seccomp profile current across upgrades?" — Re-trace across many paths after the upgrade, diff against the reviewed allowlist, classify additions (approved/forbidden/unexplained), and re-approve deliberately — never auto-approve and never loosen to allow-all on breakage.

Extension (Stretch)

Wire it into CI so a profile-affecting dependency bump produces a syscall diff for review before merge.

References