Jailbreaking: A Taxonomy

Jailbreaking is the attempt to make a model ignore its safety training and produce content it is supposed to refuse. It is a cousin of prompt injection but aimed at the model's alignment rather than your app's instructions, and understanding the common classes is how builders reason about residual risk — not a how-to. Frontier models in 2026 are trained hard against these patterns, so published one-liners mostly fail on current systems; the value here is recognizing the shapes so you can test your own app and understand why your defenses cannot be a single clever sentence. We sketch each class at a toy level, the way a security course diagrams an attack without shipping a working exploit.

Role-play and framing attacks try to launder a disallowed request through a fictional or hypothetical frame — pretend you are a character with no rules, or we are writing a novel where a character explains the forbidden thing. The mechanism is context-shifting: make the harmful output feel like it belongs to a persona or a story rather than the assistant. Obfuscation and payload-splitting instead hide the request from pattern-matching by encoding it, translating it, spacing letters out, or splitting it across turns so no single message looks bad, then asking the model to reassemble. Both classes exploit the gap between surface form and intent — exactly what alignment training has gotten much better at closing.

Volume and time are the other two axes. Many-shot jailbreaking exploits long context by filling it with dozens or hundreds of fabricated examples of the assistant complying with harmful requests, so the next completion is pulled toward the pattern — a direct consequence of in-context learning working as designed. Multi-turn crescendo attacks start benign and escalate gradually across a conversation, each step a small ask that leans on the prior agreement, until the model is somewhere it would have refused to go in one jump. Both are why single-message content filters aren't enough: the attack lives in the accumulation across the whole conversation, not in any one line.

The takeaway for builders is defense in depth, because no layer is complete alone. Alignment training is the model's built-in resistance, and it is strong but not perfect. Around it you add system-level defenses: input and output classifiers that screen for known attack shapes and harmful content, guardrail models that check requests and responses, conversation-length and context monitoring, and rate limits. Critically, keep the model's capabilities scoped so that even a successful jailbreak of the chat has limited blast radius — a jailbroken model that cannot reach any dangerous tool is a contained incident. Layers compensate for each other's gaps; a single filter never will.