WARMUP — Phase 12: Reporting, Purple Teaming, and Staff Communication

Operation Cedar Lattice — Phase 12 Pre-Lab Primer This guide builds the conceptual foundation before you touch any code. Read it completely before opening a lab file.

Table of Contents

1. Chapter 1: Engagement Report Structure
2. Chapter 2: Finding Quality Bar
3. Chapter 3: Purple Team Methodology
4. Chapter 4: VECTR Tracking
5. Chapter 5: Executive Communication
6. Chapter 6: Detection Metrics
7. Chapter 7: Misconceptions
8. Lab Walkthrough
9. Success Criteria
10. Common Mistakes
11. Interview Q&A
12. References

Chapter 1: Engagement Report Structure

Why Structure Matters

A red team report is not a dump of terminal output. It is a professional document with a single purpose: to cause organizational change. Every structural decision — ordering, section depth, writing style — is in service of that goal. The Mandiant M-Trends report format (refined over two decades of incident response and red team engagements) provides a proven skeleton. Understanding why each section exists is as important as knowing what goes in it.

The Eight-Section Structure

Engagement Report: Meridian Freight International
Operation Cedar Lattice
──────────────────────────────────────────────────
1. Executive Summary            (1–2 pages)
2. Engagement Overview          (scope, dates, RoE, team)
3. Attack Narrative             (chronological story)
4. Findings                     (prioritized by CVSS + business risk)
5. Detection-Gap Matrix         (which techniques fired vs. went dark)
6. Remediation Roadmap          (30/60/90-day plan)
7. Purple Team Replay           (detection validation results)
8. Appendices                   (raw evidence, logs, screenshots — redacted)
──────────────────────────────────────────────────
Total estimated length: 40–80 pages depending on finding count

Section 1: Executive Summary

Purpose: Give a non-technical reader the five most important facts about the engagement in two pages or fewer.

What it contains:

• One-paragraph business risk statement: what happened, to what systems, with what potential impact
• Top three findings (names only, with severity tags)
• The single most critical attack path (in plain English, no jargon)
• Remediation priority: what leadership needs to authorize this week
• Positive observations (what worked — if anything did)

What it does NOT contain:

• CVE numbers
• Tool names
• Log excerpts
• ATT&CK technique IDs (unless explained inline)

The executive summary is written last, after all findings are complete. It is the only section most executives will read. Every word must justify its existence.

Cedar Lattice example opening:

"During a six-day simulated intrusion against Meridian Freight International, the red team achieved domain controller compromise without triggering a single alert. An attacker replicating this path could gain access to all 42TB of shipping manifest databases — including customer PII — within hours of initial access. Nine of the 23 attack techniques tested produced no detection signal. This report documents the attack path, the detection gaps, and the 90-day remediation plan required to close them."

Section 2: Engagement Overview

Purpose: Establish the rules of the test so readers can evaluate the findings in context.

What it contains:

• Scope: in-scope IP ranges, domains, cloud accounts, and out-of-scope systems
• Engagement window: start date, end date, active testing hours
• Rules of engagement (RoE): what was prohibited, what required advance notification
• Methodology: what framework was followed (MITRE ATT&CK, PTES, custom)
• Team: roles (operator, team lead, reporting lead) — no individual names in client-facing docs unless agreed
• Limitations: what could not be tested and why (e.g., production systems excluded, no physical access)

Section 3: Attack Narrative

Purpose: Tell the story of the engagement chronologically so defenders can map it to their telemetry and understand the sequencing.

What it contains:

• Timeline: Day 1 / Hour 0 through final objective
• Each major action described in one to three sentences
• Technique references in parentheses: (ATT&CK T1190: Exploit Public-Facing Application)
• Lateral movement steps clearly marked
• Pivots identified with the source and destination host (anonymized if needed)

Cedar Lattice narrative excerpt:

Day 1, 09:14: Cedar Lattice identified an unauthenticated Jenkins instance at jenkins.meridianfreight-internal.example exposed via a misconfigured reverse proxy. Exploitation of the Groovy script console (T1190) provided immediate code execution as svc_jenkins on the build server.

Day 1, 11:32: Credential harvesting from Jenkins credential store (T1552.001) yielded twelve service account passwords in plaintext, including svc_deploy with domain administrator privileges.

Day 2, 08:55: Using svc_deploy, the team performed a DCSync attack (T1003.006) against dc01.meridianfreight-internal.example, extracting all domain hashes without requiring interactive logon to the domain controller.

The attack narrative is the section blue teamers use to replay the timeline against their SIEM. It must be precise enough to answer: "at what time, from what IP, to what destination, using what protocol?"

Section 4: Findings

Purpose: Document each discrete vulnerability or security gap as a standalone, actionable artifact.

How findings are ordered:

Findings are NOT sorted by CVSS score alone. They are sorted by a combination of:

1. Severity class (CRITICAL → HIGH → MEDIUM → LOW)
2. Business impact within the same severity class (data exposure > availability > reputational)
3. Exploitability (remotely exploitable, no auth > requires auth > requires local access)

A CVSS 9.8 finding on an internal-only test server ranks lower than a CVSS 7.5 finding on the public-facing booking API that processes credit card data. Context always overrides raw score.

Example table of contents for Findings section:

4. Findings
   4.1 [CRITICAL] Unauthenticated RCE — Jenkins Groovy Console (T1190)
   4.2 [CRITICAL] Domain Credential Exposure — Jenkins Credential Store (T1552.001)
   4.3 [HIGH]    DCSync via Overprivileged Service Account (T1003.006)
   4.4 [HIGH]    LLMNR/NBT-NS Poisoning — No Enforcement Controls (T1557.001)
   4.5 [HIGH]    Kerberoastable Service Accounts — SPN Misconfigurations (T1558.003)
   4.6 [MEDIUM]  Missing MFA on VPN Concentrator (T1078)
   4.7 [MEDIUM]  Cleartext Credentials in Git Repository (T1552.003)
   4.8 [LOW]     Verbose Error Pages — Internal Stack Traces Exposed (T1592)

Each finding is its own complete artifact — see Chapter 2 for the required fields.

Section 5: Detection-Gap Matrix

Purpose: Show, at a glance, which ATT&CK techniques fired a detection and which went dark.

| Technique        | Name                        | Tool Used       | Detected? | Alert Source       |
|------------------|-----------------------------|-----------------|-----------|-------------------|
| T1190            | Exploit Public-Facing App   | Custom HTTP req | NO        | —                 |
| T1552.001        | Credentials in Files        | Mimikatz-like   | NO        | —                 |
| T1003.006        | DCSync                      | Impacket        | NO        | —                 |
| T1557.001        | LLMNR Poisoning             | Responder-like  | NO        | —                 |
| T1558.003        | Kerberoasting               | Custom          | PARTIAL   | AD audit log      |
| T1078            | Valid Accounts              | VPN login       | YES       | SIEM rule: VPN-01 |
| T1021.001        | Remote Desktop Protocol     | Built-in        | YES       | SIEM rule: RDP-02 |

The Detection-Gap Matrix drives the remediation roadmap. Techniques marked "NO" are the prioritized detection gaps.

Section 6: Remediation Roadmap

Purpose: Give a time-boxed, owner-assigned plan that leadership can approve and track.

30-Day Actions (Immediate Risk Reduction):
  - Restrict Jenkins to internal network; enforce authentication [Owner: Platform Eng]
  - Rotate all credentials stored in Jenkins credential store [Owner: Platform Eng]
  - Deploy LLMNR/NBT-NS suppression via GPO [Owner: Active Directory team]

60-Day Actions (Detection Coverage):
  - Deploy Sigma rules for DCSync detection in SIEM [Owner: Security Operations]
  - Enable Advanced Audit Policy for credential access events [Owner: Active Directory team]
  - Enforce MFA on VPN concentrator [Owner: Network Engineering]

90-Day Actions (Systemic Hardening):
  - Implement tiered administration model; remove service accounts from Domain Admins
  - Complete secret rotation cycle (rotate all service account passwords quarterly)
  - Purple team replay: re-execute Cedar Lattice techniques to validate detection coverage

Section 7: Purple Team Replay

Purpose: Document which detections were validated during the purple team exercise.

This section is populated after the purple team exercise (see Chapter 3). It answers: "for each detection gap identified, has a detection now been written and validated?"

Section 8: Appendices

Purpose: Provide evidence without cluttering the main document.

• Appendix A: Screenshots (redacted to remove credentials)
• Appendix B: Tool output excerpts (relevant lines only)
• Appendix C: Raw log samples
• Appendix D: ATT&CK Navigator export (JSON)
• Appendix E: VECTR export

Chapter 2: Finding Quality Bar

The Nine Required Fields

A finding is not complete until all nine fields are present, non-empty, and meet the quality criteria below.

Finding: [Title]
──────────────────────────────────────────
Severity:       [CRITICAL | HIGH | MEDIUM | LOW | INFORMATIONAL]
ATT&CK ID:      [T#### or T####.###]
Precondition:   [What must be true for this to be exploitable]
Evidence:       [Reference to appendix screenshot or log excerpt, ≥ 20 chars]
Impact:         [Specific business impact]
Detection:      [What should have caught this — Sigma rule name or log source + condition]
Remediation:    [Specific, testable action with owner and timeline]
Residual Risk:  [What risk remains after remediation]

Field-by-Field Requirements

Title: Must contain an action verb, an asset, and an implied impact. Not "SQL Injection Found" — that describes a vulnerability class. Instead: "Unauthenticated SQL Injection in Booking API Exposes Full Customer Database."

Checklist:

• Contains an action verb (exploits, exposes, bypasses, enables, allows)
• Names the specific asset (Jenkins Groovy Console, not "CI/CD server")
• Implies or states the impact (Domain Compromise, PII Exposure, Authentication Bypass)
• At least five words (four-word titles are almost always too vague)

Severity: Must be one of CRITICAL / HIGH / MEDIUM / LOW / INFORMATIONAL and must be justified in narrative. The CVSS base score is the starting point, not the final answer. A CVSS 9.8 on an air-gapped system is HIGH at most in context. A CVSS 6.5 on the authentication endpoint of a payment processor may be CRITICAL in context.

ATT&CK ID: Must reference a specific technique or sub-technique. Format: T1234 or T1234.001. Tactic IDs (TA0001) are not valid here — they describe goals, not techniques.

Precondition: What must be true for an attacker to exploit this? Examples:

• "Attacker has network access to port 8080 on the build server (no authentication required)"
• "Attacker has any valid domain user account"
• "Attacker has compromised a single endpoint in the finance VLAN"

Without a precondition, the reader cannot assess exploitability.

Evidence: A reference to a specific appendix item or a quoted log line. Not "we observed this" — "Screenshot B-3: HTTP 200 response from /script endpoint with Groovy output (Appendix B, page 12)." Minimum 20 characters; placeholder strings like "yes" or "see logs" are rejected.

Impact: Specific business impact. Not "attacker can read sensitive data." Instead: "Attacker can read all 42TB of shipping manifest data including customer names, addresses, and shipment contents. This data is subject to GDPR Article 5 and a breach would trigger mandatory notification under Article 33 within 72 hours."

Detection: What specific detection would have caught this? Must reference a rule type: Sigma rule name, Splunk search name, KQL query, auditd rule, Sysmon event ID, CloudTrail event, Zeek script, Falco rule, or SIEM alert name. "Check the logs" is not a detection.

Remediation: Specific, testable action with owner and timeline. Not "fix the configuration." Instead: "Restrict the Jenkins instance to the internal 10.10.0.0/16 network via firewall rule and enable LDAP authentication within 48 hours. Owner: Platform Engineering team. Verify by attempting unauthenticated access from external IP."

Residual Risk: What risk remains after the remediation is applied? Not "no risk." Instead: "Internal threat actors or attackers who have already achieved internal access can still reach Jenkins. Network segmentation and the tiered administration model in the 90-day roadmap are required to close this residual risk."

Complete Finding Example

Finding: Unauthenticated RCE via Exposed Jenkins Groovy Console
──────────────────────────────────────────────────────────────────
Severity:      CRITICAL (CVSS 3.1 Base: 9.8 — AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H)
               Narrative: Remotely exploitable with no authentication; immediate code
               execution as service account; no user interaction required.

ATT&CK ID:     T1190 (Exploit Public-Facing Application)

Precondition:  Jenkins instance at jenkins.meridianfreight-internal.example is
               reachable from the internet via a misconfigured reverse proxy. No
               authentication is required to access the /script endpoint.

Evidence:      Screenshot B-1 (Appendix B, p. 4): HTTP POST to /script returned
               HTTP 200 with Groovy output showing hostname, current user (svc_jenkins),
               and OS version. Log excerpt L-2 (Appendix C, p. 18): web server access
               log confirms unauthenticated POST from external IP.

Impact:        Code execution as svc_jenkins enables credential harvesting from Jenkins
               credential store (see Finding 4.2). This account held 12 service account
               passwords including svc_deploy (Domain Administrator). Full domain
               compromise is achievable within 2 hours of exploiting this finding.
               Meridian Freight's shipping manifest database (42TB, customer PII) is
               accessible from the domain controller.

Detection:     No detection fired. Expected: Sigma rule "Unauthenticated POST to Jenkins
               /script" (detect POST requests to /script without Authorization header in
               web server or reverse proxy logs). Current state: web server logs are not
               forwarded to SIEM.

Remediation:   (1) Restrict Jenkins to internal network 10.10.0.0/16 via firewall ACL
               within 48 hours. Owner: Platform Engineering. (2) Enable LDAP
               authentication on Jenkins within 48 hours. Owner: Platform Engineering.
               (3) Forward web server access logs to SIEM and deploy Sigma rule within
               2 weeks. Owner: Security Operations. Verify by attempting unauthenticated
               HTTP POST to /script from external IP after firewall change.

Residual Risk: Internal users and attackers with internal access can still reach
               Jenkins after network restriction. The 90-day tiered administration
               model and quarterly credential rotation are required to eliminate
               residual risk from an internal threat actor.

Chapter 3: Purple Team Methodology

What Purple Teaming Is

Purple teaming is red and blue working together in the same room (or the same call) to validate whether detections fire against real attack techniques. The goal is not to "win" — it is to produce a measurable improvement in detection coverage before the engagement closes.

Traditional red team: red executes, writes a report, blue reads the PDF three months later and maybe writes a detection rule, never validated against the actual technique.

Purple team: red executes a technique, blue watches the SIEM in real-time, together they determine whether the alert fired within the defined "detected" threshold (TP + alert within 5 minutes), and if it did not, blue writes the detection rule and red re-executes to validate it.

The output is not a PDF finding — it is a detection rule that demonstrably fires against the technique that was just executed.

The Six-Step Purple Team Process

Step 1: Pre-exercise alignment

Before any technique is executed:

• Export the technique list from ATT&CK Navigator as a JSON matrix
• Define "detected" criteria with the blue team: for this exercise, "detected" = TP alert fires within 5 minutes of execution start
• Agree on the environment: production-equivalent staging, or production with monitoring-only
• Confirm VECTR is configured and blue team has access

Step 2: Execute in isolation

Red executes the technique exactly as during the original engagement. Blue does not interfere. The technique runs to completion or until the defined stopping condition.

Step 3: Validate together

Red and blue review the SIEM, EDR console, and any other telemetry source together. For each technique:

• Did an alert fire? (TP or FP?)
• Did it fire within the 5-minute threshold?
• Was the alert actionable (did it contain enough context for an analyst to triage)?

Outcome is one of: DETECTED / NOT DETECTED / DETECTED-PARTIAL (rule fired but insufficient fidelity).

Step 4: Document in VECTR

For each technique, create a VECTR test case entry:

• Technique: ATT&CK ID and name
• Outcome: DETECTED / NOT DETECTED / PARTIAL
• Detection rule name (if fired)
• Notes: what fired, what didn't, what the analyst would have seen

Step 5: Gap-close

For every NOT DETECTED technique:

• Blue writes a detection rule in the SIEM (Sigma rule, KQL query, Splunk search)
• Red and blue review the rule logic together for accuracy
• Blue deploys the rule to the SIEM in audit mode

Step 6: Re-execute and replay

Red re-executes the technique. Blue confirms the new rule fires as a TP. The VECTR entry is updated to DETECTED. This is the only outcome that closes the loop — a detection rule that has not been validated against the technique it is supposed to detect is a hypothesis, not a control.

Traditional Red Team vs. Purple Team

What "Detected" Means

A technique is "detected" if and only if:

1. A TP alert fired (not a FP on similar activity)
2. The alert fired within the defined threshold (5 minutes for Cedar Lattice)
3. The alert contains enough context for a tier-1 analyst to triage without additional investigation

All three conditions must be true. An alert that fires 3 hours after execution is not a detection — it is evidence of poor MTTD. An alert that fires immediately but contains only a raw process hash with no context is not actionable.

Chapter 4: VECTR Tracking

What VECTR Is

VECTR (Visualization, Execution, and Tracking for Testing and Reporting) is an open-source platform for tracking purple team and red team exercise outcomes over time. It provides a structured data model for campaigns, test cases, and outcomes, and generates coverage trend reports across quarters.

VECTR data model:

Organization
└── Assessment (quarterly purple team exercise)
    └── Campaign (e.g., "Cedar Lattice Q2 2026")
        └── Test Case (one per ATT&CK technique tested)
            ├── Technique: T1190
            ├── Outcome: NOT DETECTED
            ├── Detection Rule: (none)
            └── Notes: "Jenkins /script endpoint, no web log forwarding to SIEM"

How VECTR Is Used

During the purple team exercise: One operator keeps VECTR open and logs each technique as it is executed and validated. The test case is not closed until the outcome is confirmed by both red and blue.

After the exercise: VECTR generates a coverage report: of the N techniques tested, what fraction were detected? This is the "detection coverage %" for the quarter.

Across quarters: VECTR's trend view shows detection coverage % by quarter. A mature security program should show monotonic improvement — each quarter's purple team exercise should reveal fewer undetected techniques than the previous quarter's replay.

Scoring methodology:

• Coverage % = (detected + detected-partial×0.5) / total techniques tested
• Partial credit (0.5) for rules that fire but with insufficient fidelity
• Target: >85% coverage for Tier 1 techniques (most common TA TTPs)
• Baseline for new programs: expect 30–50% coverage at start

VECTR vs. ATT&CK Navigator

ATT&CK Navigator shows which techniques you have coverage for in theory (based on tool inventory or policy). VECTR shows which techniques you have validated coverage for in practice (based on purple team execution). Navigator answers "what should detect this?" VECTR answers "what actually detected this when we tested it last quarter?"

Chapter 5: Executive Communication

The Translation Problem

Every technical finding must be translated into business risk before reaching an executive. The translation is not simplification — it is precision at a different abstraction level. The technical fact and the business risk are both true and both precise; they are simply expressed in different vocabularies.

Financial Impact Framework

Use concrete numbers wherever possible. Executives make risk decisions based on expected value, not severity labels.

IBM Cost of a Data Breach 2024: Average total cost: $4.88M per breach. Healthcare: $9.77M. Financial services: $6.08M. For a company of Meridian Freight's profile (logistics, processing customer PII), a realistic breach cost estimate is $3–5M.

GDPR Article 83: Fines up to 4% of global annual turnover or €20M, whichever is higher. For Meridian Freight International with estimated revenue of €800M, maximum exposure is €32M.

Operational disruption: If the domain controller is compromised, how long until logistics operations halt? Cedar Lattice estimate for Meridian: 72–96 hours of partial outage during incident response. Cost estimate based on revenue per day.

Formula for executive brief:

Risk exposure = Likelihood × Impact
Where:
  Likelihood = "demonstrated in 6 days with one entry point"
  Impact = $4.88M breach cost + €32M max regulatory fine + 3 days disruption

Three-Slide Exec Readout Structure

Slide 1: What Happened (in English)

"During six days of simulated attack, our team reached the domain controller — the master key to all of Meridian's systems — without triggering a single alert. This is not a hypothetical risk. We demonstrated it."

No jargon. No ATT&CK IDs. One clear statement of what was achieved and why it matters.

Slide 2: Top Three Risks With Business Impact

1. [CRITICAL] Any internet-facing Jenkins server → full domain access in 2 hours
   Business risk: Complete loss of all shipping manifests + $4.88M breach exposure

2. [CRITICAL] Service account passwords stored in plaintext in build pipeline
   Business risk: Production database credentials accessible to any attacker on internal network

3. [HIGH] No alert fired during 6-day engagement (9 of 23 techniques undetected)
   Business risk: Current SOC cannot detect an active intrusion using common TTPs

Slide 3: What You Need From Leadership

This week (requires CTO + CISO approval):
  - Emergency firewall change: isolate Jenkins from internet
  - Emergency credential rotation: all service accounts

Within 30 days (requires budget approval):
  - Security Operations remediation sprint (see roadmap, Section 6)

Within 90 days (requires board visibility):
  - Purple team replay to validate detection improvements
  - Tiered administration model implementation

Common Exec Questions

"How does this compare to our peers?" Use industry benchmark data: IBM M-Trends 2024 shows mean dwell time of 10 days for red team engagements. Cedar Lattice achieved full domain access in 6 days, which is on the faster end of the distribution. This is not a commentary on Meridian's team — it reflects the specific configuration gaps identified.

"Are we actually being attacked right now?" Answer honestly: "We cannot confirm or deny an active intrusion — that is outside the scope of this engagement. What we can say is that if an attacker used any of these techniques, your current detection stack would not alert. We recommend an immediate threat hunt focused on the techniques in the Detection-Gap Matrix."

"What is the single most important thing to fix?" Cedar Lattice answer: "Isolate Jenkins from the internet today. This one change eliminates the initial access vector that made everything else possible. The rest of the roadmap is important, but this is the one that reduces your risk exposure the most in the least time."

"How confident are you in the severity ratings?" Answer: "Each severity rating is based on CVSS 3.1 base score adjusted for business context — including your data classification, regulatory exposure, and the specific network topology we observed. We are confident in the CRITICAL and HIGH ratings. The MEDIUM and LOW ratings could be re-evaluated if you provide additional context about business criticality of affected systems."

Chapter 6: Detection Metrics

The Confusion Matrix for Security

Every detection rule produces four possible outcomes for each event it evaluates:

                      Reality
                  Attack | No Attack
Alert Fired  TP  |  FP
             ────┼────────────────────
No Alert     FN  |  TN

TP (True Positive): Alert fired AND an attack was happening. The detection worked.

FP (False Positive): Alert fired BUT no attack was happening. Alert fatigue: analysts learn to ignore this rule.

FN (False Negative): Attack happened AND the alert did NOT fire. The dangerous one. The breach you did not see.

TN (True Negative): No attack, no alert. The rule correctly stayed quiet.

Precision

Precision = TP / (TP + FP)

Of all the times this rule fired an alert, what fraction were real attacks?

A rule with precision 0.05 fires 20 false positives for every real attack. Analysts learn to ignore it. When the real attack comes, they dismiss the alert as another false positive.

Precision is about alert quality. Low precision = alert fatigue = missed real attacks.

Recall

Recall = TP / (TP + FN)

Of all the real attacks that occurred, what fraction did the rule catch?

A rule with recall 0.3 misses 70% of real attacks. It might have perfect precision (every alert it fires is a real attack), but it only fires on 30% of the attacks that happen.

Recall is about coverage. Low recall = missed attacks = successful breaches.

F1 Score

F1 = 2 × (Precision × Recall) / (Precision + Recall)

The harmonic mean of precision and recall. Ranges from 0.0 (worst) to 1.0 (perfect).

Why harmonic mean rather than arithmetic mean? If precision is 1.0 and recall is 0.0, arithmetic mean = 0.5 (looks mediocre). Harmonic mean = 0.0 (correctly flags this as a useless rule — it either never fires or never catches anything, depending on which extreme is the issue).

F1 is useful when you want a single number to compare two detection rules. But it hides individual precision and recall values — always report both alongside F1.

Why Recall Matters More Than Precision in Security

The asymmetry of consequences:

• A false positive costs an analyst 5–15 minutes to investigate and close. Over time, too many FPs create fatigue that reduces real alert investigation quality. This is a serious problem, but it is recoverable.

• A false negative means a real attack was missed. The breach progresses. The cost is measured in IBM's $4.88M average — or in Meridian Freight's case, potentially far more given GDPR exposure. This is not recoverable after the fact.

Target posture: High recall, acceptable precision. Tune to reduce FP rate until analysts can investigate every alert the SIEM generates, but never tune recall downward to do it. If recall must be traded for precision, you are operating with incomplete coverage — document it explicitly.

MTTD and MTTR

MTTD — Mean Time to Detect: Time from the moment the attack technique is executed to the moment a TP alert fires in the SIEM. Not "when the analyst saw it" — when the alert fired.

IBM Cost of a Data Breach 2024 benchmark: 194 days average MTTD for breaches not detected immediately. Organizations that detected the breach in under 200 days had costs averaging $3.93M vs. $4.87M for those above 200 days.

Cedar Lattice finding for Meridian: MTTD was effectively infinite (∞) for 9 of 23 techniques — no alert ever fired.

MTTR — Mean Time to Respond: Time from when a TP alert fires to when the threat is contained (attacker evicted, affected systems isolated, credentials rotated). IBM 2024 average MTTR: 73 days after detection.

For purple team exercises, MTTR is measured in minutes during the tabletop phase: "if this alert had fired, how quickly could the SOC have contained the attacker?" Cedar Lattice post-exercise analysis for Meridian: estimated MTTR 4–6 hours for most scenarios, assuming SOC runbooks are followed.

Detection Coverage Percentage

Detection Coverage % = (Techniques with recall > 0) / (Total techniques in scope) × 100

For Cedar Lattice: 14 detected / 23 tested = 60.9% coverage. Industry maturity model:

Meridian Freight at 60.9% is in the "Developing → Defined" transition. The 90-day roadmap targets 80%.

Putting It Together

Cedar Lattice Purple Team Exercise — Meridian Freight
Technique: T1003.006 (DCSync)
─────────────────────────────────────────
tp_count:   0    (rule never fired during 10 executions)
fp_count:   0    (rule never fired at all)
fn_count:  10    (10 attacks, 0 detected)
tn_count:  90    (90 non-attack windows, correctly quiet)

Precision: 0 / (0 + 0) = 0.0 (undefined — no alerts ever fired)
Recall:    0 / (0 + 10) = 0.0 (missed all 10 attacks)
F1:        0.0
FP Rate:   0 / (0 + 90) = 0.0 (no false positives — because the rule never fires)
MTTD:      ∞ (never detected)

This is a "blind spot" detection — perfect TN rate, zero coverage. The FP rate of 0.0 is not a success; it means the rule is silent even when attacks happen.

Chapter 7: Misconceptions

Misconception 1: "High CVSS Score Means Fix It First"

Wrong reasoning: CVSS 9.8 > CVSS 7.5, therefore fix 9.8 first.

Reality: CVSS is a base score computed without knowledge of your environment. It measures exploitability and impact in a generic context. A CVSS 9.8 vulnerability on a test server that has no sensitive data, no connectivity to production, and no path to escalation is less urgent than a CVSS 7.5 vulnerability on the authentication endpoint of your payment processor.

Correct approach: CVSS provides the starting point. Adjust based on:

• Is the affected system reachable from the internet? (Increases urgency)
• What data does it hold or have access to? (Regulatory exposure)
• Is there a compensating control? (Decreases urgency)
• Is there a known exploit in the wild? (Significantly increases urgency)

For Cedar Lattice: Finding 4.8 (verbose error pages, CVSS 5.3) is LOW priority. Finding 4.3 (DCSync via overprivileged service account, CVSS 7.5) is HIGH priority — because it is on the path to full domain compromise that was already demonstrated.

Misconception 2: "Purple Team Is Just Red Team With Blue Team Watching"

Wrong reasoning: We let the blue team watch the red team execute. That is purple team.

Reality: Purple team is collaborative, not observational. Blue team does not just watch — blue team actively engages at each technique to validate, discuss, and improve detection. The distinction is:

• Observation: Blue team watches and takes notes. Red team executes normally. No real-time collaboration.
• Purple team: Red executes. Blue monitors. Both pause after each technique to review telemetry together. Blue writes detection rules in the same session. Red re-executes to validate.

If blue team is passive — if they are "watching" but not writing detection rules during the session — it is not purple team. It is a live red team with an audience.

Misconception 3: "DMARC p=none Is Fine Because We Are Monitoring"

Wrong reasoning: We have DMARC in monitoring mode (p=none). We see all the spoofing attempts. We are covered.

Reality: p=none means the email provider takes no action on DMARC failures. Spoofed emails from your domain are delivered to recipients. Monitoring tells you the attacks are happening — it does not stop them.

The only enforcement states are p=quarantine (spam folder) and p=reject (discard). Until you move to one of these, DMARC is intelligence collection, not protection.

Cedar Lattice finding for Meridian: Meridian's DMARC policy is p=none. Cedar Lattice successfully sent spear-phishing emails from ceo@meridianfreight-spoof.example that passed DMARC alignment checks because they were sent from a legitimate domain that Meridian did not own but that appeared visually similar.

Misconception 4: "We Detected the Attack So We Are Good"

Wrong reasoning: The SIEM fired an alert for DCSync. We are detecting attacks.

Reality: A detection that fires is not the same as a detection that enables timely response. Ask:

• What is the MTTD for this rule? (Does it fire within 5 minutes or 5 hours?)
• What is the precision? (Does the alert contain enough context to triage?)
• What is the workflow after the alert? (Does a tier-1 analyst see it, or does it go into an unreviewed queue?)
• What is the MTTR? (How quickly can the SOC contain the threat after the alert fires?)

An alert that fires 3 hours after DCSync execution — after the attacker already has all domain hashes — is not a useful detection. It is evidence collection for the post-breach forensics team.

The target is MTTD under 5 minutes for Tier 1 techniques, with a documented response playbook that enables containment within 30 minutes.

Misconception 5: "The Report Is Done When the Findings Are Written"

Wrong reasoning: We have documented all the findings. The report is complete.

Reality: A report without validated remediation is a document, not a service. The engagement is complete when:

1. All findings are documented (written)
2. The remediation roadmap is reviewed and accepted by the client (planned)
3. The purple team replay has validated that new detection rules fire (validated)
4. The residual risk acceptance has been signed off by the CISO or equivalent (accepted)

The finding "we deployed a Sigma rule for DCSync" is not remediation. The finding "we deployed a Sigma rule for DCSync AND re-executed the DCSync technique AND the rule fired as a TP within 90 seconds" is remediation.

Misconception 6: "F1 Is the Right Metric for All Detections"

Wrong reasoning: F1 balances precision and recall, so it should be the primary metric for all detection rules.

Reality: F1 assumes that FPs and FNs have equal cost, and that the positive class (attacks) is reasonably common. Both assumptions break in security:

• FN cost >> FP cost (see Chapter 6). A missed breach is far more expensive than a false alarm.
• Attacks are rare events. In a normal enterprise SIEM, the fraction of events that are real attacks is tiny. The denominator in precision calculations is dominated by legitimate activity. In this regime, even a "bad" rule with precision 0.01 catches all real attacks. Recall is the dominant concern.

For rare, high-severity events (e.g., lateral movement to domain controller), optimize for recall first. F1 is a useful tiebreaker when two rules have the same recall — use F1 to choose the one that also has better precision. But never sacrifice recall for F1.

Lab Walkthrough

Lab 1: Report Quality Linter

Goal: Build a linter that evaluates the completeness and quality of red team report findings. Each function checks a different quality dimension of the nine-field finding model.

Key insight: missing_fields checks for presence and non-emptiness. lint_finding checks for quality beyond presence — a field can be present but still fail quality checks (e.g., a title that is present but only 2 words, or an evidence field that is present but only 3 characters).

missing_fields(finding)

Iterate over REQUIRED_FIELDS. For each field, check:

• Not in dict → missing
• Value is None → missing
• Value is "" (empty string) → missing
• Value is [] (empty list) → missing

Return the list of field names that are missing.

def missing_fields(finding: dict) -> list[str]:
    missing = []
    for field in REQUIRED_FIELDS:
        val = finding.get(field)
        if val is None or val == "" or val == []:
            missing.append(field)
    return missing

lint_finding(finding)

Seven checks. Only check quality of a field if the field is present (non-empty). Missing fields are already reported by missing_fields and reported via check 1 in lint.

1. Missing fields: call missing_fields, add "missing required field: {f}" for each
2. Invalid severity: check against VALID_SEVERITIES
3. Vague title: len(title.split()) < 5
4. ATT&CK ID format: regex match against ATTCK_PATTERN
5. No rule type in detection: check for any keyword from RULE_KEYWORDS
6. No timeline in remediation: check for any keyword from TIMELINE_KEYWORDS
7. Evidence too short: len(evidence) < 20

Return sorted(issues).

report_quality_score(findings)

clean = sum(1 for f in findings if not lint_finding(f))
return round(clean / len(findings), 4)

severity_distribution(findings)

Initialize all valid severities to 0. Iterate findings, increment if severity is valid.

prioritized(findings)

Sort key: (severity_order, -completeness_score). Higher completeness breaks ties within severity class.

Lab 2: Purple Team Coverage Scorer

Goal: Compute standard detection metrics from purple team exercise result dicts.

precision(exercise)

tp / (tp + fp) if (tp + fp) > 0 else 0.0

recall(exercise)

tp / (tp + fn) if (tp + fn) > 0 else 0.0

f1(exercise)

Compute p and r first, then 2*p*r/(p+r) if p+r > 0 else 0.0.

detection_coverage(exercises)

Mean of recall(e) for each exercise. Return 0.0 for empty list.

top_gaps(exercises, n=3)

Sort by (recall, technique) ascending. Return first n as dicts with {technique, recall, f1, detection_rule}.

fprate(exercise)

fp / (fp + tn) if (fp + tn) > 0 else 0.0

exercise_summary(exercise)

f"technique: {technique} | recall: {r*100:.1f}% | precision: {p*100:.1f}% | f1: {f:.4f}"

Success Criteria

You have completed Phase 12 when:

• LAB_MODULE=solution pytest -q passes all 12 tests in lab-01 and all 11 tests in lab-02
• You can recite the eight sections of a Mandiant-style engagement report and explain why each exists
• You can state the nine required fields for a complete finding and give a quality criterion for each
• You can explain the purple team six-step loop from memory and distinguish it from traditional red team
• You can calculate precision, recall, F1, and FP rate from a confusion matrix in under 60 seconds
• You can translate any CRITICAL finding from the Cedar Lattice engagement into a one-paragraph business risk statement for a CFO
• You understand why recall matters more than precision in security contexts

Common Mistakes

Mistake 1: Treating p=none as a detection. DMARC monitoring is intelligence. It is not a control. Do not list it as a mitigating control in a finding.

Mistake 2: Empty evidence fields passing the quality bar. The evidence field must contain a specific reference — not a statement that evidence was observed. "Screenshot of Groovy console output (Appendix B, page 4)" passes. "See the appendix" does not.

Mistake 3: Sorting findings by CVSS alone. Always apply business context. A CVSS 7.5 on the payment processor authentication endpoint outranks a CVSS 9.8 on a decommissioned test server.

Mistake 4: Writing remediation without a timeline. "Fix the SQL injection" is not a remediation. "Parameterize all database queries in the booking API using prepared statements within 14 days; Owner: Backend Engineering" is a remediation.

Mistake 5: Treating "we deployed a rule" as "we validated a detection." A rule is not a detection until it has been tested against the technique it is supposed to detect and produced a TP. Purple team re-execution is mandatory for validation.

Mistake 6: F1 optimization at the expense of recall. Never tune a detection rule in a way that improves F1 by reducing recall. F1 improvement must come from precision improvement.

Mistake 7: Using tactic IDs in findings. TA0004 is a tactic (Privilege Escalation). T1068 is a technique (Exploitation for Privilege Escalation). Findings map to techniques or sub-techniques, not tactics.

Interview Q&A

Q1: What makes an executive summary effective for a red team report?

Answer: An effective executive summary communicates the highest-priority business risk in two pages or fewer, using language the executive can act on without technical translation. It contains five elements: (1) a one-paragraph business risk statement describing what was achieved and why it matters in dollars or regulatory exposure terms; (2) the top three findings by name and severity, without technical detail; (3) the single most critical attack path in plain English; (4) what leadership action is required this week; and (5) any positive observations. What it does not contain: CVE numbers, tool names, ATT&CK IDs, or log excerpts. The executive summary is written last, after all findings are complete, because it must accurately represent the full picture. The test for an effective exec summary: give it to a non-technical CFO, and ask "do you know what happened, how serious it is, and what you need to approve?" If the answer is yes to all three, the summary is effective.

Q2: What is the minimum set of fields that makes a finding "complete" for a Mandiant-style report?

Answer: A finding requires nine fields to be complete: (1) title — action verb + specific asset + implied impact, at least five words; (2) severity — CVSS 3.1 base score plus narrative justification for any context adjustment; (3) ATT&CK ID — specific technique or sub-technique (T#### or T####.###), not a tactic ID; (4) precondition — what must be true for the vulnerability to be exploitable; (5) evidence — specific reference to appendix screenshot or log excerpt, minimum 20 characters, not a placeholder; (6) impact — specific business impact naming the data at risk, regulatory exposure, or operational consequence; (7) detection — specific detection reference naming a rule type (Sigma, Splunk, KQL, Sysmon, etc.) that should have fired; (8) remediation — specific, testable action with owner and timeline; (9) residual risk — what risk remains after the remediation is applied. Every field must be present and non-empty. A field that exists but contains a placeholder ("see logs," "yes," "TBD") fails the quality bar even if it is technically present.

Q3: What is the difference between precision and recall in the context of security detections, and which matters more?

Answer: Precision is TP / (TP + FP) — of all alerts fired, what fraction were real attacks. Recall is TP / (TP + FN) — of all real attacks, what fraction triggered an alert. In security, recall matters more because the cost asymmetry is severe: a false positive costs an analyst 5–15 minutes of investigation time; a false negative (missed attack) can cost millions of dollars and months of dwell time. IBM's 2024 data puts average breach cost at $4.88M. The consequence of low precision is alert fatigue, which is a real problem — analysts who see too many false positives learn to dismiss alerts, which eventually causes them to miss real attacks. But the primary optimization target is recall: make sure your detections catch the attacks that happen. Then tune precision to reduce fatigue. Never reduce recall to improve precision. The correct posture is: maximize recall, then tune precision to an acceptable level defined by your SOC's investigation capacity.

Q4: How does purple teaming differ from a traditional red team engagement?

Answer: Traditional red team: the red team executes techniques covertly, writes findings in a PDF, and delivers the report. The blue team reads the report weeks later and maybe writes detection rules. Those rules are never tested against the actual technique that was used. The loop never closes.

Purple team: red and blue work together in real-time, technique by technique. Red executes; blue monitors the SIEM; both review the outcome together. For techniques that went undetected, blue writes a detection rule in the same session and red re-executes to validate it. The output is not a PDF finding — it is a detection rule that demonstrably fires against the technique when tested.

The key distinction is validation closure. In traditional red team, "we identified a gap" is the output. In purple team, "we identified a gap, wrote a rule, validated the rule fires, and updated VECTR" is the output. One produces a list of problems; the other produces working detections.

Q5: How do you translate a technical finding like "LLMNR poisoning" into business risk for a CFO?

Answer: Start with the attacker's position: "An attacker who gets on your office Wi-Fi network — which means any visitor with a conference room code, any compromised guest device, or any attacker who gets past the perimeter — can silently intercept Windows credential challenges from every workstation on the network. The workstations never know they were attacked. The attacker captures password hashes passively, without touching any system, and can then crack or relay those hashes to gain access to other systems as those users."

Then the business impact: "This technique requires no special skills and is widely automated in freely available tools. It has been used in real-world intrusions of companies in the logistics and freight sector. If even one domain administrator's workstation performs an LLMNR lookup during the attacker's window — which happens automatically during normal use — the attacker gets domain administrator credentials silently. From there, our red team demonstrated full domain compromise in under 2 hours."

Then the fix: "The remediation is a Group Policy change that takes less than one hour to deploy and has no business impact on users. We recommend deploying it this week."

Q6: What is VECTR and how is it used in a purple team program?

Answer: VECTR (Visualization, Execution, and Tracking for Testing and Reporting) is an open-source platform for tracking purple team and red team exercise outcomes. Its data model organizes assessments into campaigns and test cases — one test case per ATT&CK technique executed. Each test case records the technique, the outcome (DETECTED / NOT DETECTED / PARTIAL), the detection rule name if it fired, and notes on what the analyst would have seen.

In practice, one operator maintains VECTR open during the purple team exercise and logs each technique as it is executed and validated. After the exercise, VECTR generates a coverage report: of the N techniques tested, what fraction produced a TP alert? This is the detection coverage percentage for the quarter.

The long-term value of VECTR is the trend across quarters. A mature security program uses quarterly purple team exercises to systematically expand coverage. VECTR's trend view shows whether coverage is improving. For Cedar Lattice, Meridian Freight would enter Q2 2026 at 60.9% coverage (14/23 techniques detected) and target 80% coverage (18+ techniques) by Q3 2026 after the detection gap remediation.

Q7: What is the Detection-Gap Matrix and how does it drive the remediation roadmap?

Answer: The Detection-Gap Matrix is a table in the engagement report that shows every ATT&CK technique tested during the engagement, the tool or method used to execute it, whether a detection fired, and which alert source fired (if any). It is the authoritative record of "what went dark" — the techniques for which no alert fired during the engagement.

It drives the remediation roadmap in two ways. First, techniques marked NOT DETECTED in the matrix become the highest-priority items in the 30/60-day detection coverage roadmap. The blue team writes detection rules for each one, in order of the technique's position on the attack path (earlier techniques in the chain — initial access, execution — take priority because detecting them stops the chain). Second, the matrix provides the test list for the purple team replay: every NOT DETECTED technique gets a purple team re-execution session to validate that the new detection rule fires.

For Cedar Lattice, 9 of 23 techniques were NOT DETECTED, including T1190 (initial access via Jenkins), T1552.001 (credentials in files), and T1003.006 (DCSync). These nine entries in the Detection-Gap Matrix map directly to the nine detection rules that Meridian's SOC must write and validate in the 30/60-day plan.

Q8: How do you calculate F1 score and why is it useful for evaluating a detection rule?

Answer: F1 is the harmonic mean of precision and recall:

F1 = 2 × (Precision × Recall) / (Precision + Recall)

Where Precision = TP / (TP + FP) and Recall = TP / (TP + FN).

It uses the harmonic mean (not arithmetic mean) because the harmonic mean penalizes extreme imbalances. If precision is 1.0 and recall is 0.0, arithmetic mean is 0.5 (which sounds mediocre). Harmonic mean is 0.0 (which correctly signals the rule is useless — it either never fires or misses all attacks).

F1 is useful when you want a single number to rank or compare detection rules. When a team has written two candidate Sigma rules for DCSync detection and wants to choose one, F1 provides a principled ranking. It is also useful in VECTR trend reports: if mean F1 across all tested techniques improves from 0.52 to 0.71 quarter over quarter, detection coverage is improving.

The caveat: F1 should not be the primary optimization target in security because it treats FP and FN as equally costly, which they are not. Optimize for recall first, then use F1 to compare rules with equal recall on precision-efficiency grounds.

References

• Mandiant M-Trends 2024 — Annual threat intelligence report with red team engagement statistics and dwell time data. mandiant.com/m-trends
• IBM Cost of a Data Breach 2024 — Annual study of breach costs by industry, region, and detection speed. Average total cost: $4.88M. ibm.com/security/data-breach
• MITRE ATT&CK Navigator — Web-based tool for visualizing ATT&CK technique coverage. Export as JSON for purple team exercise planning. attack.mitre.org/resources/attack-navigator
• VECTR Documentation — Open-source purple team tracking platform documentation and deployment guide. docs.vectr.io
• NIST SP 800-115 — Technical Guide to Information Security Testing and Assessment. Provides methodology foundations for red team reporting. csrc.nist.gov/publications/detail/sp/800-115/final
• Sigma Rule Standard — Generic signature format for SIEM detection rules. Supported by Splunk, Elastic, Microsoft Sentinel, and others. github.com/SigmaHQ/sigma
• CVSS 3.1 Specification — Common Vulnerability Scoring System version 3.1 specification. first.org/cvss/specification-document