Indirect Injection

Lesson 2 of 4 in Prompt Injection and Jailbreaks: The Model-Level View.

Direct injection needs the attacker to be the user. Indirect injection removes even that requirement. The adversarial instructions hide inside content the model is asked to process: a web page your assistant summarizes, a PDF a user uploads, an email an assistant triages, a repository README, a review scraped into a Retrieval-augmented generation (RAG) pipeline. The attacker plants the payload somewhere your Retrieval or ingestion path will eventually fetch it — then waits. Greshake et al. (2023) gave this attack class its name and its threat model, showing that once LLMs are integrated into applications that read external content, the boundary between data and instructions collapses at a distance: anyone who can write to content you read can try to program your model.

The mechanism is the same single-channel fact wearing a different coat. When the fetched page enters the Context window, it is tokens in the same stream as the system prompt and the user’s question. The model has no channel that says these tokens are quoted material — describe them, don’t obey them. Instruction-following was trained as a disposition over the whole stream, so imperative text inside a “document” exerts the same kind of statistical pull as imperative text from the user.

A poisoned document reaches the model

  1. Attacker plants instructions in content

    A public web page, a shared document, an email, a package README — anywhere the application’s ingestion path will eventually read.

  2. User makes an innocent request

    “Summarize this page.” The user is not the attacker — they are the delivery mechanism.

  3. Retrieval pulls the content into context

    RAG, a browsing feature, or a document upload places the poisoned text inside the context window.

  4. Chat template flattens data and instructions

    System prompt, user question, and fetched content become one token stream. Delimiters around the document are just more tokens.

  5. Data — or instructions?

    The model has no provenance bit to consult. Whether the payload is described or obeyed is decided by trained dispositions, per input.

  6. Treated as data

    The summary describes the page, payload included, as text. This is the intended behavior.

  7. Treated as instructions

    The output now serves the attacker: a skewed summary, a planted link, an attempt to exfiltrate context through the response.

What can a followed payload actually accomplish when the model only produces text? More than you might hope. Greshake et al. catalog the model-level consequences: manipulated answers (a summary that quietly advocates for the attacker), planted content (links and recommendations the user believes came from the assistant), and exfiltration through the output channel itself — the paper describes payloads that try to smuggle context contents out through generated artifacts like URLs the application will render or a user will click. Everything the model knows in-context — the system prompt, the user’s question, other retrieved documents — is candidate loot, because it all lives in the same window the payload now influences.

And that is with text only. The moment the model’s output can trigger tools — send the email, call the API, write the file — a followed injection stops being a bad answer and becomes an action, and the attack grows into the injection kill-chain through tool use, Excessive agency, and containment strategy: that whole surface is the sister AI Agent Academy’s territory, and we hand it off there. At this site’s altitude, the lesson is the input side: any content that can reach your context window is part of your attack surface, whoever wrote it.

Why detection-based defenses are probabilistic, not boundaries

The instinctive fix is a detector: run inputs — user messages, retrieved documents — through a classifier trained to flag injection attempts, and block what it catches. Such filters exist, ship in commercial products, and are worth having. What they are not is a security boundary, and the reasons are structural rather than a matter of today’s model quality.

The category has no crisp definition. “Instruction directed at the model” is not a formal property of a string the way <script> is a formal property of HTML. The same sentence can be a quotation, an example, a test case, or an attack depending on intent and context — a classifier is being asked to recover intent from surface text, which is exactly the problem the target model already fails at.

The detector inherits the distribution-shift problem. A classifier is itself a statistical model, strong on the attack styles in its training data and weaker off it. Paraphrase, encoding, splitting a payload across documents, or a genuinely novel framing moves the attack off that distribution — the same mechanism jailbreaks use against refusals (next lesson) works against the filter guarding them.

Adversaries adapt; base rates punish. A deployed detector becomes part of the attack surface: attackers iterate against it until something passes, and published evaluations can only report performance against the attacks someone has already thought of. Meanwhile at production traffic volumes, even a filter with genuinely low error rates faces enough inputs that some attacks pass and some legitimate documents get blocked — and tightening the threshold trades one failure for the other.

The clarifying contrast is SQL injection. That class was effectively closed by parameterized queries — a type-level separation of code from data that the channel itself enforces, with no classifier anywhere. For a single token stream there is no published equivalent; separators and role markers are conventions the model was trained to respect, not structures the runtime can enforce (the full argument is in The Single-Channel Problem). So calibrate the claim: detectors reduce incidence and raise attacker cost, and layered with output-side controls they meaningfully cut risk — but treat any vendor language of “blocks prompt injection” as a probability statement wearing a certainty costume.

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.