Data Retention and Privacy

Phase 14 · Document 02 · Security, Privacy and Governance Prev: 01 — Prompt Injection · Up: Phase 14 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

The moment your LLM app touches personal data — a user's name in a chat, an uploaded document, a support transcript — you've taken on legal obligations (GDPR, CCPA, HIPAA, 07) and a reputational risk that dwarfs most security bugs. And LLM apps leak personal data through more channels than traditional apps: the prompt you send to a third-party provider, the logs you keep, the eval/training sets you build, the vector store you index, and the model weights themselves if you fine-tune (Phase 13.06). The single most-overlooked fact: when you call a hosted model API, you are sending user data to a third party — and whether they train on it or retain it depends on a contract you may not have read. This doc is the privacy lifecycle for LLM systems: what data flows where, how long it lives, who else sees it, and how to honor users' rights over it.

2. Core Concept

Plain-English primer: follow the data, everywhere it goes

Privacy engineering is fundamentally about knowing where personal data flows and controlling each hop. For an LLM app, the data takes a longer journey than people realize:

user input (may contain PII)
   → your app  → [maybe] LOGS  → [maybe] the PROVIDER's API (a third party!)  → provider LOGS/retention
   → [maybe] VECTOR STORE (RAG index)  → [maybe] EVAL sets  → [maybe] FINE-TUNING data → MODEL WEIGHTS
   → model OUTPUT (may regurgitate PII)  → user / downstream systems

Every arrow is a place personal data can be stored, exposed, or sent somewhere it shouldn't be. Privacy work is auditing each hop and applying: do we need this data here? for how long? who can see it? can we delete it?

PII (Personally Identifiable Information) = anything that identifies a person (name, email, phone, address, SSN, IP, device IDs) or, combined, could (quasi-identifiers). Some data is special-category / sensitive (health, biometric, financial, race, religion) with stricter rules (07).

The big one: sending data to a third-party provider

When you call OpenAI/Anthropic/Google/etc., the user's prompt leaves your infrastructure. Two questions decide your exposure:

1. Do they train on your data? For business/API tiers, the major providers contractually do not train on your data by default — but consumer/free tiers often do. This is exactly why "employee pasted source code into ChatGPT" became a famous class of incident. Read the data-use terms for the exact tier/endpoint you use.
2. How long do they retain it? Providers typically retain API inputs/outputs for a limited window (e.g., ~30 days) for abuse monitoring, then delete. Zero-Data-Retention (ZDR) agreements (available to qualifying enterprise customers) mean they don't store your prompts at all — important for regulated data.

The contract that governs all this is the DPA (Data Processing Agreement): it defines them as your processor, what they may do with the data, sub-processors, retention, and deletion. No DPA + personal data = a compliance gap (07).

Self-hosting/local inference (Phase 6) removes the third-party hop entirely — the strongest privacy posture, at the cost of running the model yourself.

Data minimization: the cheapest privacy control

The data you don't collect can't leak. Minimization (a GDPR principle) means: collect only what the task needs, send the model only what it needs, and redact PII before it leaves or lands where it isn't required:

• Scrub/redact PII before logging (logs are the most common accidental PII store, 06).
• Redact or tokenize PII before sending to the provider when the task doesn't need the real values (replace with placeholders, re-insert on the way back).
• Don't index PII in the vector store unless required; if you do, protect it like production data (04).

import re
def scrub_pii(text: str) -> str:
    text = re.sub(r'\b[\w.%+-]+@[\w.-]+\.[A-Za-z]{2,}\b', '[EMAIL]', text)
    text = re.sub(r'\b\d{3}[-.\s]?\d{3}[-.\s]?\d{4}\b', '[PHONE]', text)
    text = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[SSN]', text)
    text = re.sub(r'\b(?:\d[ -]?){13,16}\b', '[CARD]', text)
    return text

(Regex scrubbing is a baseline — real systems add NER/Presidio-style detectors and accept it's lossy; combine with minimization.)

Retention: how long, and then delete

A data retention policy states, per data type, how long you keep it and why, then deletes on schedule. "We keep everything forever" is both a liability and (in the EU) often unlawful (storage limitation). Decide retention for: raw conversations, logs, eval sets, vector indexes, fine-tuning data. Shorter is safer. Automate deletion (TTLs) so it actually happens.

Users' rights over their data (the part engineers forget)

Privacy law grants individuals rights you must be able to honor in code:

• Right to access / portability — produce all their data on request.
• Right to erasure ("right to be forgotten") — delete all their data on request. The trap: deletion must reach every hop — DB, logs, vector store, backups, and the provider (via the DPA), and if you fine-tuned on their data, the weights memorized it (Phase 13.06) and you may need to retrain. Design for deletion before you fine-tune on user data.
• Right to object / restrict — e.g., opt out of having their data used to improve the product.
• Consent — for many uses (especially training on their data), you need a lawful basis/consent.

PII in the model itself (training & output)

• Training: models memorize and regurgitate training data — fine-tuning on un-scrubbed PII risks the model emitting someone's SSN later. Scrub before fine-tuning; consider differential privacy (adds calibrated noise to gradients to bound memorization) for sensitive corpora (Phase 13.06).
• Output: the model can surface PII from its context (RAG docs, history) to the wrong user — control via tenant isolation (04) and output filtering (05).

Anonymization vs pseudonymization

• Pseudonymization — replace identifiers with tokens (reversible with a key). Still "personal data" under GDPR, but lower risk; good for sending to providers.
• Anonymization — irreversibly strip identity so a person can't be re-identified. True anonymization takes data out of privacy law's scope — but it's hard (re-identification via quasi-identifiers is real).

3. Mental Model

   PRIVACY = FOLLOW THE DATA and control every hop:
     user input → LOGS [06] → PROVIDER API (3rd party!) → provider retention → VECTOR STORE → EVAL → FINE-TUNE → WEIGHTS → OUTPUT
     at each hop ask: do we NEED it here? for HOW LONG? WHO sees it? can we DELETE it?

   ★ THIRD-PARTY HOP: calling a hosted API SENDS user data to a 3rd party.
     - TRAIN ON IT? business/API tier = usually NO by default; consumer/free = often YES. READ THE TERMS for your tier.
     - RETENTION? ~limited window for abuse-monitoring; ZDR (enterprise) = they don't store it.
     - DPA = the contract (they're your processor). No DPA + personal data = compliance gap [07].
     - self-host/local [6] removes the hop entirely (strongest posture).

   MINIMIZATION (cheapest control): collect/send/store only what's needed; SCRUB/redact PII before logs/provider/index
   RETENTION POLICY: per-type "how long + why" → DELETE on schedule (TTLs). "forever" = liability + often unlawful (storage limitation)
   USER RIGHTS: access/portability · ERASURE (must reach EVERY hop incl. provider + memorized WEIGHTS [13.06]) · object/restrict · consent
   IN THE MODEL: training memorizes→regurgitates PII (scrub; differential privacy) · output can leak PII (isolation [04] + filtering [05])
   PSEUDONYMIZE (reversible, still personal data) vs ANONYMIZE (irreversible, out of scope but hard)

Mnemonic: follow the data through every hop and minimize, scrub, time-box, and be able to delete it — and never forget that a hosted API call ships user data to a third party governed by a contract (DPA) and a data-use tier you must actually read.

4. Hitchhiker's Guide

What to look for first: does the app touch personal data, and does it call a third-party provider? If yes to both, your two first actions are (1) a DPA + the right data-use tier (no training, known retention/ZDR) and (2) minimization/scrubbing before data leaves.

What to ignore at first: building a bespoke anonymization pipeline. Start with minimization + scrubbing + a retention TTL + a DPA — that covers most of the risk cheaply.

What misleads beginners:

• "The provider doesn't use our data." True only for the right tier with the right contract — consumer/free tiers often train on it. Verify.
• Logging full prompts/responses. Logs are the #1 accidental PII store — scrub before logging (06).
• "Delete" only hitting the database. Erasure must reach logs, vector store, backups, the provider, and fine-tuned weights (Phase 13.06).
• Keeping data forever. Violates storage-limitation and grows breach blast radius — set retention TTLs.
• Fine-tuning on user PII without a deletion plan. The weights memorize it; you may have to retrain to honor erasure — design for it first.

How experts reason: they map the data flow, minimize at every hop (don't collect, don't send, don't store what isn't needed), put a DPA + no-train + known-retention/ZDR contract under every provider call, scrub PII before logs/provider/index, set retention TTLs, and build erasure that reaches every hop — before fine-tuning on user data. For sensitive corpora they consider self-hosting (Phase 6) and differential privacy.

What matters in production: no personal data sent to an unvetted third party; no PII in logs/indexes that don't need it; data deleted on schedule; deletion requests honored end-to-end; lawful basis/consent for training uses.

How to debug/verify: can you answer "where is user X's data?" and delete all of it? Inspect logs/vector store for raw PII; check the provider tier's data-use + retention; verify a test erasure purges every store and that you didn't fine-tune on data you can't delete.

Questions to ask: is there a DPA + no-train tier + known retention/ZDR? what PII flows where, and do we need it there? what are the retention TTLs? can we honor access + erasure across all hops? did we fine-tune on data we'd need to delete? should this be self-hosted?

What silently gets you fined/breached: consumer-tier data use, un-scrubbed logs, no retention limits, deletion that misses a hop, PII memorized in fine-tuned weights, and no DPA.

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• Calling a hosted model API sends user data to a third party — exposure depends on the data-use tier (business=usually no-train; consumer/free=often trains) and retention (limited window, or ZDR); governed by the DPA (07).
• Data flows through many hops — input, logs, provider, vector store, eval sets, fine-tuning data, weights, output — control each one.
• Minimization is the cheapest control — don't collect/send/store/index PII you don't need; scrub before logging and before the provider.
• Set retention TTLs and delete on schedule — "keep forever" is a liability and often violates storage limitation.
• Honor user rights in code: access/portability and especially erasure — deletion must reach every hop incl. the provider and memorized weights (Phase 13.06).
• Models memorize and regurgitate PII — scrub training data; consider differential privacy for sensitive corpora.
• Output can leak PII to the wrong user — control via tenant isolation (04) and output filtering (05).
• Self-hosting/local inference removes the third-party hop — the strongest privacy posture (Phase 6).

9. Observations from Real Systems

• "Employee pasted secrets/source into ChatGPT" became an entire incident class — driven by consumer-tier data use; enterprises responded with no-train API tiers, gateways, and DLP (Phase 8.09).
• ZDR + DPA are standard enterprise asks — regulated industries won't send data to a provider that retains it (07).
• Logs are the most common accidental PII store — teams discover full prompts (with PII) in their observability stack and have to retrofit scrubbing (06).
• Erasure is hard once you've fine-tuned — memorized data in weights means honoring "right to be forgotten" can require retraining; mature teams avoid fine-tuning on un-deletable PII.
• Self-hosting for privacy is a real driver of local/open-weight adoption in healthcare, defense, and finance (Phase 6).

10. Common Misconceptions

11. Engineering Decision Framework

PRIVACY FOR AN LLM APP (follow the data):
 1. INVENTORY: map every hop personal data touches (input→logs→provider→vector store→eval→fine-tune→weights→output).
 2. PROVIDER: sign a DPA; use a NO-TRAIN tier; confirm retention or get ZDR; or SELF-HOST [6] for sensitive data.
 3. MINIMIZE: collect/send/store only what's needed; SCRUB/redact PII before logs [06], before the provider, before indexing.
 4. RETENTION: per-type TTL + reason; automate deletion. Shorter = safer.
 5. RIGHTS: build access + ERASURE that reaches EVERY hop (incl. provider via DPA + memorized weights [13.06]); track consent/lawful basis.
 6. MODEL: scrub training PII (+ differential privacy for sensitive); filter PII in output [05]; isolate tenants [04].
 7. CLASSIFY: special-category data (health/biometric/financial) → stricter controls + [07].

12. Hands-On Lab

Goal

Build a data-flow privacy map for a sample LLM app and implement the core controls: PII scrubbing before logging/provider, a retention TTL, and an end-to-end erasure routine.

Prerequisites

• A sample app that takes user input, calls a provider, logs requests, and indexes some docs in a vector store.

Steps

1. Map the flow: list every hop personal data touches (input, logs, provider call, vector store, any eval/fine-tune set, output).
2. Check the provider: find your tier's data-use (do they train?) and retention terms; note whether you have a DPA/ZDR.
3. Scrub: implement PII scrubbing/redaction before logging and before sending to the provider for fields the task doesn't need raw; verify logs contain no raw PII (06).
4. Retention: add a TTL to stored conversations/logs and an automated deletion job.
5. Erasure: implement "delete everything for user X" that purges DB + logs + vector store entries + (conceptually) the provider via DPA; test it leaves nothing behind.
6. Fine-tune check: decide whether any pipeline would fine-tune on this data — if so, document the deletion implication (retrain) (Phase 13.06).

Expected output

A privacy data-flow map, scrubbing at the log/provider boundaries, a retention TTL with automated deletion, and a working end-to-end erasure routine — the privacy baseline for the app.

Debugging tips

• Grep your logs/vector store for emails/phones/SSNs after a test request — found any? Scrubbing is incomplete.
• If erasure leaves entries in the vector store or provider, it's not compliant.

Extension task

Add NER-based PII detection (e.g., Presidio) and compare catch rate vs regex; implement pseudonymization (tokenize → call provider → de-tokenize).

Production extension

Wire scrubbing/retention/erasure into the request pipeline and the gateway (Phase 8.09); document the DPA/ZDR posture for compliance evidence (07).

What to measure

PII-in-logs (should be 0), scrub catch rate (regex vs NER), retention adherence, erasure completeness across hops.

Deliverables

• A data-flow privacy map + provider data-use/retention findings.
• PII scrubbing at log + provider boundaries.
• A retention TTL + deletion job and a working end-to-end erasure routine.

13. Verification Questions

Basic

1. What happens to user data when you call a hosted model API?
2. What's the difference between a no-train tier, retention window, and ZDR?
3. What is data minimization and why is it the cheapest privacy control?

Applied 4. Why must "right to erasure" reach more than your database? 5. Why is fine-tuning on user PII a deletion problem?

Debugging 6. You find full prompts with emails in your logs. What's the fix? 7. A user requests deletion. List every place you must purge.

System design 8. Design the privacy controls for a healthcare chatbot (sensitive data, EU users).

Startup / product 9. An enterprise prospect asks about your data handling. What do you need in place (DPA, tier, retention, erasure)?

14. Takeaways

1. Follow the data through every hop and at each ask: need it? how long? who sees it? can we delete it?
2. A hosted API call sends user data to a third party — use a no-train tier + DPA + known retention/ZDR, or self-host sensitive data (Phase 6).
3. Minimize and scrub PII before logs/provider/index — the cheapest, highest-leverage control (06).
4. Set retention TTLs and honor user rights in code — especially erasure across all hops, including memorized weights (Phase 13.06).
5. Models memorize PII — scrub training data (consider differential privacy), and filter/isolate PII in output (04/05).

15. Artifact Checklist

• A data-flow privacy map (every hop personal data touches).
• A DPA + no-train tier + retention/ZDR posture per provider (or self-host).
• PII scrubbing/redaction before logging and before the provider.
• A retention policy + automated deletion (TTL).
• An end-to-end erasure routine (DB + logs + vector store + provider + weights plan).

Up: Phase 14 Index · Next: 03 — Secrets and API Keys