The catalog pattern: eight entries you can reuse
Lesson 2 of 5 in The Guardrail Catalog: Runtime Checks, Mapped Per Cloud.
Teams discover guardrails one incident at a time. Someone gets burned by a leaked email address, so a PII filter appears in one service; six months later a different team writes a different PII filter with a different threshold, and nobody can answer “what checks does this agent actually have?”
The fix is a catalog: one table, owned centrally, where every entry answers the same four questions.
- Risk addressed — which threat, named in a taxonomy you share (OWASP LLM01 Prompt Injection, ASI02 Tool Misuse, and so on) rather than a vibe like “bad outputs”.
- Check performed — the concrete mechanism. “Classifier”, “schema validation”, “regex allowlist”, “second model call comparing answer to sources”.
- Where it runs — input, tool call, tool response, or output. This is the field teams skip, and it is the one that decides whether the check works at all.
- What it costs — added latency, false-positive rate, and money. A check with no cost entry will be adopted everywhere and then quietly disabled by the first team whose p95 latency doubles.
Here is a starting catalog. Treat the numbers as orders of magnitude for planning, not benchmarks — every one of them depends on your model, provider, region and traffic mix, and you must measure your own.
| Catalog entry | Risk addressed | Check performed | Where it runs | What it costs |
|---|---|---|---|---|
1 · Injection heuristics | Direct and indirect prompt injection (OWASP LLM01) — hidden instructions in retrieved pages, tickets, PR comments, documents. | Classifier plus cheap heuristics: instruction-shaped imperatives in data, invisible text (white-on-white, 1px fonts, zero-width chars), base-64 blobs, “ignore previous” patterns. | Input and — critically — tool response, because indirect injection arrives in tool output, not user text. | One extra model or classifier call per document (tens to hundreds of ms). Meaningful false positives on security content and on documents that legitimately quote instructions. |
2 · PII detection and redaction | Sensitive information disclosure (OWASP LLM02) — personal data reaching logs, third-party tools, or the user’s screen. | Named-entity classifier plus custom regex for your own identifier formats; action is mask or block. | Input (keep it out of context), output (keep it off the screen), and on tool call arguments (keep it out of third-party APIs). | Low latency if a local classifier. False positives mangle legitimate text — order numbers read as card numbers. Masking beats blocking for usability. |
3 · Topic bounds | Off-scope and reputational answers — legal advice from a billing agent, competitor pricing, medical claims. | Denied-topic list or a small classifier per topic, with a fixed refusal response. | Input (refuse early, cheaply) and output (catch drift the input filter missed). | Cheap. The false-positive tax lands on adjacent legitimate questions, which is where user trust erodes fastest. |
4 · Tool-call schema validation | Tool misuse (OWASP LLM06 Excessive Agency / ASI02) — malformed, out-of-range, or scope-escaping arguments. | Deterministic validation against the tool’s JSON schema, plus range, enum, allowlist and cross-field rules (recipient domain in allowlist, amount ≤ limit, path inside the workspace). | Tool call — between the model’s request and execution. | Microseconds, essentially free, near-zero false positives. Best value in the catalog and the first entry to build. |
5 · Grounding check | Misinformation (OWASP LLM09) — confident answers unsupported by the retrieved sources. | Compare the draft answer against its reference passages, claim by claim; block or annotate the unsupported ones. Usually a second model call. | Output, with the retrieval results as the reference. | The most expensive entry: a full extra model call plus the source tokens, often hundreds of ms to seconds. Reserve it for answers people act on. |
6 · Sensitive-action classifier | Irreversible or high-blast-radius actions taken without oversight — bulk deletes, external sends, payments, permission changes. | Classify the proposed action by consequence and route it: allow, require approval, or refuse. Combine with the deterministic rules from entry 4. | Tool call, before execution. | Small latency; the real cost is human — every false positive spends a reviewer’s attention, and reviewer attention is the scarcest thing you have. |
7 · Rate and budget limits | Unbounded consumption (OWASP LLM10) and attacker-induced loops — token spend, tool-call storms, API quota exhaustion. | Hard counters per session and per identity: max tool calls, max tokens, max calls per tool per minute, spend ceiling. Exceed → halt and alert. | Tool call and around the whole loop. | Negligible latency. The honest note: this row is containment wearing a catalog entry — it recognises nothing and bounds everything, which is why it never misses. |
8 · Output sanitisation before rendering | Improper output handling (OWASP LLM05) — model text rendered as markup that fetches attacker URLs, i.e. the exfiltration channel behind most published agent leaks. | Strip or neutralise auto-fetching constructs before rendering — image tags, forms, iframes, CSS url(), link previews — and allowlist any host you still permit. | Output, at the rendering boundary (and again in any channel that renders: chat UI, email, PR comment, Slack message). | Microseconds, no model call, no false positives worth the name. Lesson 5 is the worked example. |
Read down the cost column and the strategy writes itself. Entries 4, 7 and 8 are deterministic, sub-millisecond, and effectively free of false positives — build those first and build them everywhere. Entries 1 and 5 are model calls with real error rates and real latency; place them where the risk justifies the bill. Entries 2, 3 and 6 sit in between, and their true cost is measured in user friction and reviewer attention rather than milliseconds.
A catalog row also needs four operational fields that are not about security at all — they are about the catalog surviving contact with a running organisation.
Owner — a named team, not “platform”
Every entry has a team that tunes its thresholds, triages its false positives, and gets paged when it breaks. Unowned guardrails are not maintained; they are tolerated, until someone silences them during an incident and nobody turns them back on.
Test — a fixture in CI, not a memory of a red-team afternoon
Each entry ships with positive cases (payloads it must catch) and negative cases (legitimate traffic it must not break), and both run in CI on every prompt, model, threshold or provider change. A guardrail with no automated test has an unknown current detection rate — model swaps and provider updates silently move it. This is the hand-off to the evals domain: guardrail fixtures are regression tests, and they belong in the same suite as your quality evals.
Fail mode — what happens when the check itself fails
The classifier times out. Do you fail closed (block the request) or open (let it through and log)? Both are defensible; only one can be the accident. Write it down per entry: grounding checks usually fail open with an annotation, tool-call validation on a payment always fails closed.
Log field — the signal, in your telemetry, on every evaluation
Log every evaluation, not just the blocks: check id, verdict, score, latency, and a pointer to the trace. Without the pass records you cannot compute a false-positive rate, cannot see a detection rate collapse after a model upgrade, and cannot tell a targeted attack from a noisy Tuesday.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.