Hitchhiker's Guide — Hardening Native and Systems Code

The operational companion to the Phase 01 WARMUP. The WARMUP explains why memory, integer, and parser bugs exist; this guide is the how — the compiler flags, fuzzing workflow, harness engineering, binary-hardening inspection, and root-cause discipline that turn that understanding into a defensible toolchain. Vulnerable examples exist only to teach root cause and the secure replacement, never to build exploit chains.

Table of Contents

• Compiler and Build Modes
• Attack Fixtures
• A Parser Review, Field by Field
• Hardening Review Checklist
• The Fuzzing Workflow
• Fuzz Harness Engineering
• Debugging to Root Cause
• Inspecting the Final Binary
• Deployment Boundary and Production Defense
• Completion Evidence
• Worked Crash-Triage Session
• Interview Drills
• References

Compiler and Build Modes

For an owned C lab target, the sanitizer/fuzz baseline:

clang -std=c17 -Wall -Wextra -Wpedantic -Wconversion -Werror \
  -fstack-protector-strong -D_FORTIFY_SOURCE=3 -fPIE -pie \
  -fsanitize=address,undefined -fno-omit-frame-pointer parser.c -o parser

Sanitizer builds are test instrumentation, not deployment artifacts. Maintain distinct modes:

Never benchmark sanitizer output as production performance, and never treat a clean release run as a substitute for instrumented testing — a release binary runs the UB silently.

Attack Fixtures

Seed your test corpus with the canonical malformed inputs (Phase 01 WARMUP, Chapters 4–5):

• length larger than remaining input; multiplication overflow before allocation;
• duplicate identity / authorization field; embedded NUL and invalid UTF-8;
• unknown critical record; a valid message plus a trailing second message;
• maximum-size and one-byte-over-limit inputs;
• repeated parse/free cycles and concurrent calls.

Each fixture targets a specific invariant; a parser that accepts any of them has a finding.

A Parser Review, Field by Field

The repeatable review procedure for any length-prefixed/TLV field:

1. Prove the bytes exist (field_len <= remaining).
2. Decode with explicit byte order and width (no host-endianness assumptions).
3. Convert without truncation (watch signed→unsigned, 64→32).
4. Check protocol, memory, and business limits.
5. Prove arithmetic cannot wrap — check operands before the operation:

/* size = count * elem, with no overflow */
if (elem != 0 && count > SIZE_MAX / elem) return ERR_OVERFLOW;
size_t size = count * elem;
if (size > remaining || size > LIMIT) return ERR_BOUNDS;

6. Advance one cursor once (no double-advance, no re-read).
7. Enforce repetition and canonicalization rules (reject duplicate singletons, pick one encoding).
8. Delay side effects until the whole message validates.

Return a typed, immutable value. Never let authorization, logging, or storage reparse the raw bytes with different semantics — that's a parser differential (Phase 01 WARMUP, Chapter 6).

Hardening Review Checklist

• use size_t consistently for sizes; centralize checked add/multiply helpers;
• one owner per allocation; make ownership transfer explicit;
• initialize outputs before all failure paths (no uninitialized reads);
• avoid parsing through packed-struct casts (alignment/aliasing UB);
• cap recursion, nesting, record count, decompression ratio, and total work;
• set O_CLOEXEC on descriptors; clear inherited environment;
• drop privileges before parsing untrusted input where possible;
• log error class and an input hash — never raw secrets or the raw input.

The Fuzzing Workflow

small deterministic harness → seed valid corpus → add dictionary/structure hints
  → run ASan/UBSan → minimize crash → reproduce WITHOUT the fuzzer → root cause
  → patch root cause → add regression input → variant search

Run it concretely (libFuzzer in-process target):

clang -g -O1 -fsanitize=address,undefined,fuzzer harness.c parser.c -o fuzz_parser
./fuzz_parser -dict=tlv.dict corpus/        # explore; crashes saved as crash-*
./fuzz_parser -minimize_crash=1 -runs=100000 crash-abcd   # minimize

Fuzzer runtime is not a quality metric. Track edges/paths covered, execs/sec, corpus size, timeouts, unique root causes (after dedup), and the grammar states never reached.

Fuzz Harness Engineering

A strong harness:

• calls the narrowest stable API (in-process, not a CLI subprocess — orders of magnitude faster and clean sanitizer attribution);
• resets global state and controls clocks/randomness/threads between iterations (no flaky crashes);
• avoids external network and subprocess overhead;
• crashes only for genuine invariant failures (no false oracle where the harness over-reads).

Coverage tells you which grammar states are unreached; a differential disagreement between two implementations is a lead, not proof either is correct. Preserve every fixed input as a small deterministic regression test — fuzzers discover, ordinary tests prevent recurrence on every build.

Debugging to Root Cause

Walk backward from the first invalid operation; the top stack frame is usually the symptom, not the cause:

faulting access ← pointer/size ← decoded field ← attacker bytes
                ← missing invariant ← the API contract that allowed it

Record: allocation and free stacks (for UAF, ASan prints both), thread, object state, the actual integer values, and whether a standalone non-fuzz reproducer reaches the same violation. A UAF crashes at the use; the bug is the premature free plus the surviving pointer — patch the lifetime, not the use site.

Inspecting the Final Binary

Verify the release artifact is actually hardened (Phase 01 WARMUP, Chapter 7) — don't trust the build flags blindly:

readelf -lW parser | grep -E 'GNU_STACK|GNU_RELRO'   # non-exec stack, RELRO present?
readelf -d parser | grep -E 'BIND_NOW|FLAGS'         # full RELRO?
readelf -h parser | grep Type                         # DYN = PIE (ASLR-able)
checksec --file=parser                                # summary: PIE, NX, canary, RELRO, FORTIFY

Confirm: no writable+executable mappings (W^X), PIE present, RELRO and stack canary on, no stray debug paths leaking source layout, and dependency versions match the SBOM.

Deployment Boundary and Production Defense

Run high-risk parsers as isolated services/libraries with least privilege: a dedicated identity, no ambient credentials, bounded descriptors, read-only inputs, no network by default, resource ceilings, and a replaceable (crash-and-restart-safe) process. Memory-corruption mitigations reduce exploitability but never repair the bug — fix the root cause regardless of mitigations.

Completion Evidence

A complete Phase 01 toolkit deliverable includes: the compiler/linker commands; a binary-hardening report; a corpus feature inventory and coverage-gap analysis; a minimized fault; a root-cause data-flow diagram; boundary and variant tests; a release SBOM/provenance; and proof the sandbox leaks no inherited authority (descriptors, environment, network).

Worked Crash-Triage Session

Here is a complete triage from raw ASan output to a verdict — the exact reasoning an interviewer wants to hear. Suppose the fuzzer saves crash-9f2c and a re-run prints:

==12431==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000000d39
WRITE of size 5 at 0x602000000d39 thread T0
    #0 0x4f8b1a in copy_field parser.c:84
    #1 0x4f6022 in parse_record parser.c:142
    #2 0x4f4310 in LLVMFuzzerTestOneInput harness.c:11
0x602000000d39 is located 0 bytes to the right of 9-byte region [0x602000000d30,0x602000000d39)
allocated by thread T0 here:
    #0 0x49a2e8 in malloc
    #1 0x4f8a40 in copy_field parser.c:79

Read it line by line:

1. Class: heap-buffer-overflow, a WRITE — corruption, not just a read. Writes are more severe (they alter adjacent heap state).
2. Magnitude: writing 5 bytes "0 bytes to the right" of a 9-byte region — a small, adjacent overflow. The allocation site (parser.c:79) and the faulting write (parser.c:84) are in the same function, so the bug is local: the buffer was sized for 9 but copy_field wrote 14.
3. Root cause hunt — open parser.c:79–84:

char *buf = malloc(hdr.name_len);        // line 79 — sized from a header field
...
memcpy(buf, src, hdr.name_len + 5);      // line 84 — copies name_len + a 5-byte suffix

The allocation used name_len; the copy used name_len + 5. The fix sizes the allocation to the copy length and validates it against remaining first (Phase 01 WARMUP, Chapter 4):

if (hdr.name_len > SIZE_MAX - 5) return ERR_OVERFLOW;
size_t need = hdr.name_len + 5;
if (need > remaining || need > NAME_MAX) return ERR_BOUNDS;
char *buf = malloc(need);
memcpy(buf, src, need);

4. Impact, stated honestly (the crash→impact chain, Phase 01 WARMUP, Chapter 12): attacker reaches parse_record with any record (reachable); the overflow length is fixed at +5, so controllability of the write size is low, but the content is attacker bytes — a 5-byte adjacent heap write is potentially exploitable depending on the heap layout. I'd report it as a heap overflow with demonstrated memory corruption and inferred (not proven) exploitability, and let the fix stand regardless.
5. Regression: save the minimized crash-9f2c as a unit test that asserts parse_record returns ERR_BOUNDS (fails on the old revision, passes on the patched one).
6. Variant search: grep for the pattern malloc(.*len) followed by a memcpy with a different length expression elsewhere — the same author likely repeated it.

Interview Drills

Practice answering these out loud; they mirror real systems-security loops:

1. "Here's a stack trace ending in __memmove_avx_unaligned — is it exploitable?" Don't answer yes/no. Walk crash→reachability→controllability→boundary→impact: the libc frame is the symptom; ask what called it, whether the length/pointer is attacker-controlled, and whether a trust boundary is crossed. Refuse to overclaim.
2. "Design a safe parser for a length-prefixed format." Lead with the eight rules (WARMUP Chapter 5): bound total input, full fixed header before indexing, len <= max AND len <= remaining, reject duplicate singletons and unknown-critical fields, strict text decode, consume exactly one message, structured errors. Mention fuzzing it with ASan and a differential second implementation.
3. "Why does if (count*size > limit) fail to prevent an overflow?" Because the multiply already wrapped before the comparison; you validated the wrapped value. Show the operand check count > SIZE_MAX / size.
4. "Your coverage plateaued at 40%. What now?" Not "run longer." Diagnose: a checksum/length gate rejecting mutations (recompute in the harness), missing structure (dictionary / structure-aware), a harness that doesn't reach deep code, or a thin corpus. Inspect the coverage map to see which frontier is stuck.
5. "What's the difference between a sanitizer build and a release build, and why can't you ship the sanitizer build?" Sanitizers are test instrumentation (slow, large, shadow memory); release is optimized+hardened. A clean release run proves nothing about UB because the optimizer ran the UB silently — you need the instrumented run to observe it.

References

• SEI CERT C/C++ Coding Standard; Robert Seacord, Secure Coding in C and C++.
• LLVM documentation: AddressSanitizer, UBSan, MSan, TSan, and libFuzzer; AFL++ documentation.
• The Fuzzing Book (Zeller et al.); Google OSS-Fuzz "Ideal integration" guide.
• readelf, checksec, and the ELF specification (binary-hardening inspection).
• _FORTIFY_SOURCE, stack-protector, and RELRO references (glibc / GCC / Clang hardening docs).