The Jailbreak Taxonomy
Lesson 3 of 4 in Prompt Injection and Jailbreaks: The Model-Level View.
A Jailbreak is any technique that elicits output the model was trained to refuse. The published attacks sort into a small number of families, and the useful way to learn them is mechanical: not the payload text (which changes weekly and which we do not reproduce beyond what the cited papers themselves publish schematically), but which property of the model each family exploits. Every family below is documented in the security literature this module cites; every one of them is, at bottom, a way of moving the input somewhere the refusal disposition is weak.
Two anchors before the table. Ganguli et al. (2022) had human red teamers attack models conversationally at scale and cataloged the strategies that emerged — the manual families below are the kind of thing that catalog contains. Zou et al. (2023) then showed the search for attacks can be automated: their GCG method optimizes an Adversarial suffix — a short string of tokens, found by gradient-guided search, that raises the probability of a compliant response opening — and, as published, suffixes optimized against open-weight models transferred to other models, including closed ones.
| Family | What the attacker does | What it exploits in the model | Published source |
|---|---|---|---|
Role-play / persona framing | Wraps the request in a fictional frame or asks the model to answer as some persona for whom refusing would be out of character | Conditioning on a persona shifts the distribution the model completes from; refusal training is concentrated on direct requests, and helpfulness-to-the-frame competes with harmlessness | Strategy family surfaced by human red teamers at scale — Ganguli et al. 2022 (arXiv:2209.07858) |
Obfuscation / encoding | Restates the request in an encoding, cipher, uncommon language, or fragments to be reassembled — legible enough to act on, unfamiliar enough to slip past | Safety training generalizes imperfectly off its training distribution; where the model’s ability to decode outruns its refusal training’s coverage, the gap is exploitable | Documented across the red-teaming literature — Ganguli et al. 2022; automated variants in Perez et al. 2022 (arXiv:2202.03286) |
Gradual escalation | A multi-turn sequence of individually innocuous requests, each nudging the conversation closer to the target | Accumulated context shifts what a natural continuation looks like; each turn is judged against a conversation the attacker has already bent | Conversational attacks were the setting of Ganguli et al. 2022, whose red teaming was dialogue-based |
Many-shot pattern pressure | Fills a long context with many fabricated examples of an assistant complying with similar requests before making the real one | In-context learning: the model continues patterns present in its context, and enough compliant examples exert pattern-completion pressure against the refusal disposition | Documented by Anthropic’s “Many-shot Jailbreaking” research (2024) — the technique exploits exactly the long contexts that make models useful |
Optimized adversarial suffixes (GCG) | Appends a machine-found token string — typically gibberish to a human reader — discovered by gradient-guided search to maximize the probability of a compliant opening | Refusal is a continuous disposition in weight space, so optimization can navigate around it; no persuasion involved, and published suffixes transferred across models | Zou et al. 2023 (arXiv:2307.15043) |
Read the middle column again and notice it is one idea five times: move the input off the distribution where refusal training holds. A persona shifts it semantically, an encoding shifts it lexically, escalation shifts it conversationally, many-shot shifts it by in-context pattern, and GCG shifts it by direct optimization. None of these exploits a bug in code — there is no parser to trick, no memory to corrupt. They exploit the fact that the safety behavior is learned, and learned behaviors have a domain of validity that attackers map from the outside.
GCG deserves special weight in your threat model, for two published reasons. It is automated — the attack is a search algorithm, not an artisanal prompt, so patching one suffix does not patch the method. And it is transferable — suffixes optimized where gradients are available (open-weight models) worked, as published, against models whose weights the attacker never saw. The practical implication is stated carefully: attackers do not necessarily need access to your model to search for strings that affect it.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.