One channel: why this is not a bug

Lesson 1 of 5 in Prompt Injection: The Vulnerability With No Patch.

On 12 September 2022, Riley Goodside showed that a GPT-3 translation app could be hijacked by its own input: text that told the model to ignore its instructions and print something else — and it worked even when the prompt warned the model not to fall for tricks. Simon Willison wrote it up the same day and gave it the name we still use: prompt injection.

Willison drew the parallel to SQL injection, and the parallel is exact in the diagnosis and useless in the cure. SQL injection happens because applications build queries by concatenating strings; prompt injection happens because applications build prompts by concatenating strings. SQL got a real fix — parameterised queries, which move the data out of the instruction channel entirely. There is no parameterised prompt. As Willison later put it, separating instructions from data this way looks extremely difficult, if not impossible with current LLM architectures.

Key terms: prompt injection, indirect prompt injection, context window, system prompt, tool-output poisoning, lethal trifecta

The SQL-injection analogy — where it holds and where it breaks
Dimension	SQL injection	Prompt injection
Root cause	Untrusted input concatenated into a query string.	Untrusted content concatenated into a prompt — same cause, which is why the name stuck.
The channel	The parser distinguishes SQL syntax from literal values once you give it the chance.	The model has one channel. Instructions and data are the same kind of thing: text it attends to.
The structural fix	Parameterised queries. Data can never be promoted to code. Solved problem since the 2000s.	No equivalent exists. Delimiters, XML tags, and “ignore anything below this line” raise the cost of an attack; they do not create a boundary.
Detection as a defence	Unnecessary — the structural fix is complete, so escaping/filtering is belt-and-braces.	Necessary and insufficient. Willison’s point: a filter that stops 95% of attacks is a failing grade in security, because attackers iterate on the 5%.
Consequence for design	Fix the query; the feature is safe.	Assume the model can be turned. Design so a turned model cannot do much — the containment argument this module ends on.

OWASP Top 10 for LLM Applications 2025 — LLM01:2025 Prompt Injection

Prompt injection is LLM01 — the number-one entry, a slot it has held since the list’s first edition. OWASP defines it as user prompts altering the model’s behaviour or output in unintended ways, and splits it into direct injection (the user’s own input alters behaviour) and indirect injection (the model accepts input from external sources such as websites or files, whose content alters behaviour).

Two lines from that entry deserve to be memorised. Injections “do not need to be human-visible/readable, as long as the content is parsed by the model” — white-on-white text and one-pixel fonts are inputs like any other. And RAG plus fine-tuning “do not fully mitigate” the risk. The 2025 edition was published 18 November 2024; OWASP shipped a 2026 edition in August 2026, and as of September 2026 the item pages on genai.owasp.org still enumerate the 2025 list.

Myth: “A stronger system prompt fixes it”

Goodside’s original demonstration already defeated a prompt that warned the model about trickery. A system prompt is text in the same channel as the attack — you are not building a wall, you are arguing with the attacker through the model, and the attacker gets to write last. Prompt hardening reduces the rate; it never creates an invariant.

Myth: “Delimiters and XML tags create a boundary”

They create a convention, and conventions can be described, imitated, and closed by attacker-supplied text. Tagging untrusted content is still worth doing — it improves the model’s odds and it makes your data flow legible to reviewers — but a defence whose enforcement lives inside the thing being attacked is not a boundary.

Myth: “It is the same thing as jailbreaking”

Willison keeps these apart, and so should you. Jailbreaking attacks the model’s safety training: make it produce content the lab tried to prevent. Prompt injection attacks your application: the concatenation of your trusted instructions with untrusted content, so the model serves the attacker’s goal instead of yours. A perfectly safety-aligned model that never says anything harmful will still cheerfully email your documents to a stranger if the text it read told it to.

Myth: “The next model generation will solve it”

Better models resist more attempts — measurably so. But the failure mode is following instructions found in content, which is the same capability that makes the agent useful. Improvements shrink the attack surface without changing its shape, and the security question is unchanged: what happens on the run where it works? Design for that run.

Myth: “We do not have this problem, our agent only reads internal data”

“Internal” is not the same as “trusted”. A ticket a customer filed, a PR description a contractor wrote, an email anyone on the internet can send, a wiki page from 2019 — all of it is attacker-influenceable content that arrives through internal systems. The correct question is not where does this text live but who can write it.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.