« Phase 09 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Staff Notes

Phase 09 — Core Contributor Notes: Secure Execution: Sandboxing & Capabilities

Our Sandbox is a Python if that models the shape of a capability check. This document is about the three real systems it mirrors — runc/libcontainer, gVisor, and Firecracker — from the angle of someone who has read their source: the setup ordering that matters more than any single flag, the non-obvious decisions, the API evolutions, and the sharp edges a committer knows that the miniature deliberately omits.

runc: the order of operations is the whole security story

When Docker or Kubernetes starts a container, runc (using libcontainer) does not "turn on a sandbox." It executes a precise sequence of syscalls, and the sequence order is load-bearing — get it wrong and the container is either broken or escapable. The essential ordering:

  1. clone/unshare with the namespace flags (CLONE_NEWNS, CLONE_NEWPID, CLONE_NEWNET, CLONE_NEWUTS, CLONE_NEWIPC, CLONE_NEWUSER). The user namespace is special: for rootless containers it must be created first so the process can then create the other namespaces as its own unprivileged "root." This is the mechanism behind rootless Docker — root inside is nobody outside — and getting the UID/GID map written to /proc/PID/uid_map before the child proceeds is a genuine choreography (libcontainer uses a synchronization pipe between parent and child for exactly this handshake).
  2. Mount setup and pivot_root. The container's rootfs becomes the process root; the old root is detached. This is why the container cannot see the host's / — not a check, a change of root. The fs_read/fs_write split in our lab is a userspace model of what pivot_root plus a read-only rootfs plus a tmpfs scratch mount does in the kernel.
  3. cgroup placement — the process is moved into its cgroup with pids.max, memory.max, cpu.max set. On cgroup v2 (the unified hierarchy that has now largely replaced v1) this is a single tree rather than v1's per-controller mounts, which was one of the messier parts of the old API.
  4. Drop capabilities, set no_new_privs, then install the seccomp filter — last. The order here is not negotiable: no_new_privs must be set before seccomp so a setuid binary cannot regain privileges, and seccomp is installed last so the setup syscalls themselves are not blocked by the very filter that forbids them.

The committer's takeaway our lab can't show: security setup is a pipeline where each stage narrows what the next stage — and the workload — may do, and the stages must run in dependency order. Our Sandbox.run charges budgets before dispatching before checking before acting; that "narrow, then proceed" discipline is the same shape runc follows across syscalls.

The runc escape CVEs teach the sharp edge

The instructive runc vulnerabilities all share a pattern: a file descriptor or path that straddles the host/container boundary at the wrong moment. One class (the /proc/self/exe overwrite) let a malicious container overwrite the runc binary on the host by pointing its own entrypoint at the host's view of the runtime. Another class involved a leaked file descriptor or a working directory that resolved to the host filesystem before pivot_root fully sealed the container. The through-line: the escape was never a broken cgroup or a missing namespace — it was a TOCTOU or a boundary-crossing handle in the setup path. Our miniature omits this entire class (it has no file descriptors, no symlinks, no host filesystem), which is exactly the "limits of the miniature" the README flags: a Python if cannot model a race a real kernel boundary must survive.

gVisor: reimplementing Linux in userspace, and where it leaks through

gVisor's central bet is audacious: do not trust the host kernel's ~350-syscall surface with untrusted code — reimplement Linux in a memory-safe userspace process instead. The pieces a contributor thinks in:

  • Sentry is the userspace kernel — a Go program that implements the Linux syscall interface itself. The workload's syscalls are trapped (via the ptrace, KVM, or newer systrap platform) and redirected to Sentry, so the host kernel only ever sees the small, hardened set of syscalls Sentry itself makes. The attack surface shrinks from "all of Linux" to "gVisor's reimplementation."
  • Gofer is a separate process that mediates filesystem access over the 9P protocol, so Sentry itself never holds a host file descriptor directly — another boundary, another process.
  • Netstack is a userspace TCP/IP stack, so the guest's networking does not touch the host's network stack either.

The sharp edges a committer knows: not every syscall is implemented, so exotic or newer syscalls return ENOSYS, and some workloads (heavy io_uring, certain mmap tricks, unusual ptrace use) break or run slowly. And gVisor is not free — every syscall is now a userspace round-trip through Sentry, so a syscall-heavy workload pays real overhead. The design accepts that: gVisor is "container ergonomics, VM-ish isolation" for workloads that can tolerate the syscall tax. It runs OCI images via a runsc runtime, drop-in where runc went, which is the whole ergonomic pitch — swap the RuntimeClass in Kubernetes and the workload does not know.

Our lab's Sandbox denying an op it does not recognize (unknown_op) is a pinhole model of Sentry returning ENOSYS for an unimplemented syscall: the interface is deliberately smaller than the real thing, and calls outside it fail closed rather than falling through to the host.

Firecracker: subtract until only the VMM is left

Firecracker inverts gVisor's approach. Instead of reimplementing the kernel in userspace, it gives each workload a real guest kernel behind a Virtual Machine Monitor stripped to the studs. The contributor-level facts:

  • The VMM is deliberately tiny — on the order of tens of thousands of lines of Rust — because the VMM is the entire attack surface between guest and host, so every line removed is surface removed. There is no BIOS, no PCI, no USB, no legacy device emulation. The device model is only what a serverless workload needs: virtio-net, virtio-block, a serial console, a minimal interrupt controller.
  • The jailer wraps Firecracker itself in namespaces, cgroups, and a chroot, and Firecracker applies a seccomp filter to its own VMM threads — the VMM that isolates the guest is itself sandboxed, defense-in-depth on the isolator.
  • Configuration is over a REST API on a Unix socket, not a giant flag string — you PUT the boot source, drives, network interfaces, and machine config, then issue InstanceStart. Rate limiters on the virtio devices cap the guest's disk and network bandwidth at the VMM, which is where the lab's max_output_bytes idea lives in the real thing.
  • Snapshotting — capture a booted microVM's memory and device state and restore it in single-digit milliseconds — is the feature that makes per-task fresh VMs economical, and it is why platforms can promise a clean environment per request without paying a full guest-kernel boot each time.

Firecracker powers AWS Lambda and Fargate, and in this track it is what Vercel Sandbox runs each execution in — an ephemeral microVM per task. E2B productizes the same shape as disposable cloud sandboxes for agents. When the README says "run untrusted agent code in an ephemeral Firecracker microVM," this is the machine underneath.

seccomp-bpf: why path filtering has to be userspace

The single most important thing a contributor understands about seccomp — and the thing that explains why our lab checks paths in Python and not "in seccomp" — is a hard limitation of the mechanism: a seccomp-BPF filter cannot dereference pointers. The BPF program the kernel runs on every syscall sees the syscall number and the register-level arguments, but it cannot follow a pointer argument to read the string it points at, because the memory could be changed by another thread between the check and the syscall (a TOCTOU the kernel refuses to expose). So seccomp can allow or deny open by flags, but it cannot decide based on the path string — that is a userspace job, which is precisely why filesystem allow-listing lives above the syscall filter, in the mount namespace and in code like our under_allowlist. The filter returns one of a small set of actions — SECCOMP_RET_ERRNO (fail with a chosen errno), SECCOMP_RET_KILL (kill the process), SECCOMP_RET_TRAP, SECCOMP_RET_ALLOW — and Docker's default profile uses these to block ~44 of the ~350 syscalls nobody legitimately needs (mount, reboot, ptrace, kernel-module loading, keyctl).

Docker's default seccomp profile has evolved — syscalls get added to the allow-list as legitimate workloads need them and removed as CVEs demonstrate danger — which is the API-evolution lesson: a security allow-list is a living document maintained against a moving workload-and-threat landscape, not a constant. Our lab's fixed set of op kinds is a frozen snapshot of that living idea.

What the miniature deliberately simplifies

Stated plainly, so the boundary is honest:

  • It is in-process, not a kernel boundary. A Python if is not a security boundary; it models the shape of one. Never run actually-untrusted code against it.
  • No syscall layer, no seccomp. There is no filter on what the workload may ask the runtime to do at all — the op vocabulary is the only interface, which stands in for the syscall surface.
  • No TOCTOU or symlink races. Paths are strings, not inodes or file descriptors, so the entire class of check-then-open races — the class that produced the real runc escapes — is absent.
  • No DNS. net_hosts matches on a hostname string; there is no resolution, so DNS rebinding and IP-literal bypasses (the reason production egress filters must resolve and re-check) are out of scope.
  • No second kernel, no VMM. The isolation is a dictionary and a list, not a namespace or a hypervisor.

What transfers, and it is the whole point, is the invariant shape: check before effect, allow-list over deny-list, fail closed, bound the budget, and — the ordering discipline — narrow first, proceed second. That shape is identical from this if up through runc's setup pipeline, gVisor's Sentry, and Firecracker's minimal VMM. Build the model once and you can read all three systems' configuration and know what each knob buys.