« Phase 11 · Warmup · Track Overview

Core Contributor — Working on the Engines Themselves

What it takes to contribute to Presidio, NeMo Guardrails, garak, PyRIT — or to build the guardrail layer your bank runs. Read this if you want to understand the systems rather than configure them.

Table of Contents

• 1. Why read the engines
• 2. Presidio: the architecture
• 3. Writing a recognizer
• 4. NeMo Guardrails: Colang and the rail loop
• 5. garak and PyRIT
• 6. AgentDojo: the benchmark that matters
• 7. CaMeL: reading the reference implementation
• 8. Building an in-house guardrail layer
• 9. Testing guardrails
• 10. Contributing

1. Why read the engines

Because the vendor claims are unfalsifiable from the outside. "Blocks 99% of prompt injections" is meaningless without knowing the corpus, and the only way to form a view is to read how detection is implemented and what it can structurally miss.

Because the interesting work is in containment, and nobody sells it. Every commercial product in this space is a detector. The architecture that actually bounds consequence — taint, capabilities, dual-model separation — is something you build, and the research implementations are the only reference.

2. Presidio: the architecture

Microsoft's PII detection and anonymization toolkit, and the one worth reading in full because it is small enough to hold in your head.

   ┌──────────────────────────────────────────────────────────┐
   │  AnalyzerEngine                                          │
   │    RecognizerRegistry ── PatternRecognizer (regex+score) │
   │                       ── SpacyRecognizer   (NER)         │
   │                       ── ContextAwareEnhancer            │
   │    NlpEngine (spaCy / Stanza / transformers)             │
   └────────────────────────┬─────────────────────────────────┘
                            │ RecognizerResult[]
   ┌────────────────────────▼─────────────────────────────────┐
   │  AnonymizerEngine — replace / redact / mask / hash /      │
   │                     encrypt (reversible)                  │
   └──────────────────────────────────────────────────────────┘

Three design decisions worth stealing:

Recognizers return a score, not a boolean. A regex match on a card-shaped number is 0.5; the same match with a passing checksum is 0.9. Downstream can threshold. A boolean detector cannot express "probably".

The ContextAwareEnhancer boosts a score when supporting words appear nearby. A 16-digit number scores higher next to "card" or "visa". This is a cheap, explainable precision improvement and it is the part most home-grown detectors lack.

Recognition and anonymization are separate engines. You can detect and decide the treatment per-context — mask for the model, redact for the log, encrypt for the pipeline — without re-running detection.

Worth reading in microsoft/presidio:

3. Writing a recognizer

The extension point you will actually use, because no toolkit ships Emirates ID or LEI:

from presidio_analyzer import Pattern, PatternRecognizer

class EmiratesIdRecognizer(PatternRecognizer):
    PATTERNS = [Pattern("Emirates ID (weak)", r"\b784-\d{4}-\d{7}-\d\b", 0.5)]
    CONTEXT = ["emirates", "eid", "identity", "resident"]

    def __init__(self):
        super().__init__(supported_entity="AE_EMIRATES_ID",
                         patterns=self.PATTERNS, context=self.CONTEXT)

    def validate_result(self, pattern_text: str) -> bool | None:
        # Return True  → promote to a high score
        #        False → discard entirely
        #        None  → leave the pattern score alone
        return luhn_ok(pattern_text.replace("-", ""))

The three-valued validate_result is the design worth understanding. False discards — that is the checksum removing a false positive. True promotes — the checksum passed, so this is not a coincidence. None leaves the pattern's own confidence, for classes with no validator.

That single method is where the precision in this whole phase comes from, and it is why the lab threads an optional check through _PATTERNS.

Testing a recognizer — the shape that matters:

# true positives, with context
# true positives, without context (score should be lower)
# near-misses: one digit off (must NOT match)
# format variants: spaces, dashes, none
# adjacent classes: an IBAN containing a card-shaped run

The fourth and fifth rows are where home-grown recognizers fail, and they are the rows that decide whether the control survives contact with a real corpus.

4. NeMo Guardrails: Colang and the rail loop

NVIDIA's guardrails framework. Its distinguishing idea is Colang, a DSL for conversational flows:

define user ask about competitor
  "what do you think of $competitor"
  "is $competitor better"

define flow
  user ask about competitor
  bot refuse to discuss competitors

Under the hood, user utterances are embedded and matched against canonical forms by vector similarity, then the matched flow runs. Which is the thing to understand about it: the rail matching is itself semantic, so it inherits fuzzy behaviour — a paraphrase that lands outside the similarity threshold does not trigger the rail.

The rail types:

Reading NVIDIA/NeMo-Guardrails:

The thing to notice while reading: there is no taint model. Rails act on content at a point in time; nothing marks a chunk as untrusted and follows it into a tool call. That is not a criticism of the framework — it is the gap you fill, and knowing it is the gap is why you read the source.

5. garak and PyRIT

Two red-teaming frameworks with different philosophies, and it is worth running both.

garak (NVIDIA/garak) — a vulnerability scanner. Probes are static or lightly-templated attacks; detectors decide whether the response indicates failure.

   probe (attack corpus) ──► generator (your model) ──► detector ──► report

The architecture to steal is the probe/detector split. A probe knows how to attack; a detector knows what success looks like. They compose, so N probes × M detectors covers more than N hand-built tests. Look at garak/probes/promptinject.py and garak/detectors/.

PyRIT (Azure/PyRIT) — an orchestration framework, and the distinction is real: PyRIT supports multi-turn adversarial attacks where an attacker LLM adapts based on responses.

   attacker LLM ──► target ──► scorer ──► attacker adapts ──► target ──► ...

That matters here because the interesting agent attacks are multi-turn: establish a premise, build trust, exploit at turn eight. A single-turn corpus cannot express them. Read pyrit/orchestrator/red_teaming_orchestrator.py.

Use both: garak for breadth and regression (fast, deterministic, CI-friendly), PyRIT for depth (slow, adaptive, run weekly).

6. AgentDojo: the benchmark that matters

ethz-spylab/agentdojo — the benchmark that made this field measurable, and the one to know by name.

Its contribution is the two-objective structure:

• Utility — did the agent complete the user's task?
• Security — did the injected attacker task succeed?

A defence that blocks everything scores 0% utility and 0% attack success. A model with no defence scores high utility and high attack success. Neither is good, and a single number cannot say so — which is why every credible result in this area now reports the pair.

The environments are realistic agentic settings (a workspace with email and calendar, a banking suite, Slack, a travel agency) with tools that have real side effects, and injections placed in content the agent retrieves rather than in the prompt.

What to take from it even if you never run it:

Report both numbers. "Our guardrail blocks 99% of injections" is meaningless without the utility cost. Ask for the pair, always.

The environments are a template. Build the same thing for your own tools: a set of realistic tasks, a set of injections placed in retrievable content, and a scorer for each objective. That is what a bank-specific red-team suite looks like, and it is more valuable than any public corpus because it exercises your tools.

7. CaMeL: reading the reference implementation

google-research/camel-prompt-injection — the implementation of "Defeating Prompt Injections by Design".

The architecture, and it is the rigorous version of the lab's taint rule:

   P-LLM (privileged)  sees the user's request, never untrusted data
        │  emits a PROGRAM in a restricted Python subset
        ▼
   INTERPRETER  ── every value carries CAPABILITIES (sources, readers)
        │        ── a tool call is refused when its arguments' capabilities
        │           do not permit the effect
        ▼
   Q-LLM (quarantined)  parses untrusted data into TYPED values, calls nothing

Three things to take from the source:

Capabilities are per-value, not per-request. capabilities.py attaches a source set and a reader set to every value. That is the granularity the lab approximates coarsely (§2 of the deep dive), and reading it is the fastest way to see what the coarse version gives up.

The interpreter is the enforcement point, not the model. interpreter/ is ordinary code with ordinary tests — the security property does not depend on model behaviour at all, which is the whole argument.

The Q-LLM returns typed values, never prose. So an injected instruction has no field to occupy. This is the "structured extraction" idea in its strongest form.

And the honest number, worth quoting accurately: 67% of AgentDojo tasks solved with provable security. The remaining third need capability the design does not permit. That is the real frontier — not "is injection solved" but "how much capability does a provable guarantee cost".

8. Building an in-house guardrail layer

Taint is a field on your content type, and it is persisted. Not a wrapper, not a side table. If it can be dropped by a serialization round trip, it will be.

One enforcement point. Every tool call goes through the same check. Two paths means the second one is less careful, and that is the one an incident finds.

Deterministic in the synchronous path. Model-based checks run asynchronously into a signal.

Everything returns evidence, not booleans. GuardrailResult carries the verdict, the reasons, the findings and the control ids. A control that emits no evidence does not exist as far as an auditor is concerned, and cannot be debugged as far as an engineer is concerned.

The coverage matrix is generated and verified. verify_coverage() failing the build when a claimed control's code is absent is the difference between a compliance artifact and a compliance test.

Property tests on the invariants:

# combine() never lowers trust, for any inputs
# combine() never loses a source id
# a side-effecting action with any tainted source and no approval is BLOCKED
# masked output never contains a value that passes its own checksum
# apply_treatment preserves everything outside the finding spans
# normalize() is idempotent

The first and third are the security properties; the others are correctness. Hypothesis will find the combination you did not think of, and for combine it usually does within seconds.

Deterministic tests. No model, injected clock, derived tokens. Every test in the lab runs in under a third of a second and none are flaky — which is not an accident but the direct consequence of keeping the model out of the enforcement path.

9. Testing guardrails

The two most under-used are near-miss corpora and assertion counters.

Near-misses — a number one digit off a valid card, a string that looks like an IBAN, a document that discusses prompt injection without containing one — are what tell you the false-positive rate, and the false-positive rate is what decides whether the control is still enabled in six months.

Assertion counters distinguish "no blocks because nothing hostile happened" from "no blocks because the code path is gone". Count evaluations, not just blocks.

10. Contributing

Presidio (microsoft/presidio) — Python, MIT, active, welcoming. The best first contribution in this space: a recognizer for a national identifier your region needs. Emirates ID, for example, is not shipped. Pattern, context words, checksum, tests — self-contained, reviewable, and immediately useful.

NeMo Guardrails (NVIDIA/NeMo-Guardrails) — Python, Apache 2.0. Entry points: library integrations, Colang standard-library flows, docs. Larger changes touch the Colang runtime and need discussion.

garak (NVIDIA/garak) — Python, Apache 2.0. Very approachable: a new probe is a class with an attack corpus. If you find an attack that works, contributing the probe is a genuine public good.

PyRIT (Azure/PyRIT) — Python, MIT. Entry points: scorers, converters, orchestrators.

AgentDojo (ethz-spylab/agentdojo) — research code. New environments and attack suites are welcome and are how the benchmark stays relevant.

For all of them the useful preparation is the same: implement the mechanism yourself first — the lab is a small version of exactly that — then read theirs and find every place they differ. The differences are where the real engineering is, and in this field they are also where the unsolved problems are.