Lab 02 — Security Finding Quality Reviewer

Difficulty: 3/5 | Runs locally: yes

Pairs with the Phase 12 WARMUP Chapter 3 (the defensible finding) and Chapter 4 (risk framing).

Why This Lab Exists (Purpose & Goal)

A security finding is a claim you must defend — to the engineer who has to fix it, the executive who has to fund it, and sometimes an auditor or a vendor. A weak finding (vague location, no reproduction, inflated severity) is dismissed; a strong one gets acted on. The goal of this lab is to build the reviewer that enforces what makes a finding defensible, and — critically — that rejects severity claims the demonstrated evidence doesn't support.

The Concept, In the Weeds

A defensible finding contains a specific chain, and the reviewer checks each link:

authorization · evidence · preconditions · demonstrated impact · uncertainty · root cause · remediation · regression test · owner · residual risk

The two highest-value checks encode lessons from across the curriculum:

Demonstrated impact vs. claimed severity. The reviewer rejects severity claims unsupported by the demonstrated boundary (Phase 08's proven-vs-inferred). A "critical" with no demonstrated boundary crossing is not critical — and turning every risk into a critical is the fastest way to be ignored. When everything is critical, nothing is, and engineers stop reading your reports.
Root cause + remediation + regression test. A finding that names the what but not the why, or that has no concrete fix and no test to prevent recurrence, is incomplete — it's a symptom report, not engineering.

The discipline is calibration: severity must reflect demonstrated impact in context (exploitability × impact × asset criticality), with uncertainty stated honestly. A reviewer who calibrates is trusted precisely when a real critical appears.

Why This Matters for Protecting the Company

A security team's entire ability to reduce risk flows through its findings — if engineers don't act on them, nothing improves. Findings get acted on when they are defensible and calibrated: precise enough to fix, honest enough to trust, severe enough to prioritize but not inflated. Severity inflation is a slow poison: it trains the organization to discount security, so the one finding that is critical lands in a muted inbox. A finding-quality bar protects the company by keeping the security signal credible, which is the only way it drives action. This is the program-scale version of the calibration you learned in the Android (Phase 04) and crash-triage (Phase 08) labs.

Build It

Implement the reviewer: check findings for authorization, evidence, preconditions, demonstrated impact, uncertainty, root cause, remediation, regression test, owner, and residual risk. Reject severity claims unsupported by the demonstrated boundary.

LAB_MODULE=solution pytest -q

Secure Implementation Patterns

The anti-pattern. A finding that's a vibe — "critical" with no demonstrated boundary, "sanitize input" as the fix:

severity: critical
demonstrated_impact: "looks exploitable"      # no boundary crossed shown
remediation: "use encryption"                  # non-actionable

The secure pattern — enforce the defensible-finding chain (mirrors solution.py):

REQUIRED = ("authorization", "asset", "preconditions", "evidence", "demonstrated_impact",
            "uncertainty", "root_cause", "remediation", "regression_test", "owner", "residual_risk")

def review(finding: dict) -> tuple[str, ...]:
    issues = []
    for field in REQUIRED:
        if not str(finding.get(field, "")).strip():
            issues.append(f"missing:{field}")                          # the full chain or it's incomplete
    # severity must be backed by DEMONSTRATED impact, not asserted
    if finding.get("severity") == "critical" and \
       "system compromise" not in str(finding.get("demonstrated_impact", "")).lower():
        issues.append("unsupported-critical-severity")
    # remediation must be actionable, not a platitude
    if str(finding.get("remediation", "")).lower() in {"sanitize input", "use encryption", "fix it"}:
        issues.append("non-actionable-remediation")
    return tuple(sorted(issues))

Two checks carry the weight: a critical severity must be backed by demonstrated (not inferred) impact, and the remediation must be concrete enough to act on. Together they enforce calibration — the property that makes findings trusted.

A defensible finding:

authorization: "in-scope per ROE-2026-04"
asset: "tenant billing records"
preconditions: "authenticated low-priv user"
evidence: "request/response showing tenant-2 invoice returned to tenant-1 user (1 record)"
demonstrated_impact: "cross-tenant read of one record; full-table read inferred, not shown"
uncertainty: "exploitability of write paths not tested"
root_cause: "object id trusted without tenant/relationship check on GET /invoices/{id}"
remediation: "enforce object-level authZ: WHERE tenant_id = caller_tenant; return 404 on miss"
regression_test: "test_reader_cannot_read_other_tenant_invoice"
owner: "billing-platform"
residual_risk: "low after fix; monitor cross-tenant access metric"

Production practices to carry forward:

Separate demonstrated from inferred impact — say what you proved; never inflate severity.
Severity inflation is a slow poison — when everything is critical, nothing is, and the real critical lands in a muted inbox.
Every finding has a concrete fix and a regression test so the bug class can't silently return.
Calibrate by exploitability × impact × asset-criticality in context, not a reflexive CVSS number.

Validation — What You Should Be Able to Do Now

Write a defensible finding (location, repro, root cause, demonstrated-vs-inferred impact, calibrated severity, concrete fix, regression test, residual risk).
Reject and re-score severity claims the evidence doesn't support.
Explain why severity inflation destroys credibility and gets real criticals ignored.

The Broader Perspective

This lab crystallizes the professional voice of a security engineer: precise, calibrated, and honest about uncertainty — the qualities that make your assessments trusted and acted upon. It ties together the proven-vs-inferred discipline (Phase 08), the requirement-to-evidence chain (Phase 03/07), and the consequence-based prioritization (Phase 10) into one standard for how you communicate risk. At the staff level, your influence is your impact, and your influence rests on a track record of findings that were right and severities that were calibrated. Crying wolf is not just a style problem — it operationally degrades the company's response to real threats. Carry the calibration habit into every report, memo, and severity call you ever make.

Interview Angle

"Walk me through what makes a finding defensible." — Exact location, reproduction from clean state, root cause (why not just what), demonstrated impact separated from inferred, calibrated severity (exploitability × impact × criticality), a concrete fix with a regression test, honest limitations, and an owner. And never inflate severity — when everything is critical, nothing is.

Extension (Stretch)

Score a corpus of real-style findings, compute a precision metric for your reviewer, and feed accepted findings into the security roadmap prioritizer.

References

Phase 12 WARMUP Chapters 3–4; FIRST CVSS; OWASP risk-rating methodology.

Security Engineer