One token stream, and now it has hands
Lesson 1 of 5 in The Agent Threat Model.
Classical application security rests on a boundary you have relied on your whole career: code is code, and data is data. The database knows the difference between a query and a string. The CPU knows the difference between an instruction and a buffer. Every injection class in history — SQL, XSS, command injection — is a story about that boundary being blurred, and every fix is a story about restoring it: parameterised queries, escaping, separate channels.
An LLM has no such boundary. Your system prompt, the user’s message, the retrieved document, the tool result, the web page, the email body — all of it is flattened into one sequence of tokens in the context window and attended to by the same weights. The model does not have a privileged instruction channel to hear you on. It has a statistical tendency to obey the text that looks most like an instruction, and no reliable way to ask where any of that text came from.
That is a known and serious property of chatbots. It becomes a security architecture problem when you build an agent, because an agent adds one thing: tool calls. Now the confusion is not merely about what the model says — it is about what the model does. Text the attacker planted in a document becomes a decision, and the decision becomes an API call executed with your credentials.
Hold this sentence, because the rest of the security domain hangs off it: an agent is a system that converts text it reads into actions it takes, and it cannot reliably tell your text from the attacker’s.
Where attacker text enters, and where it lands
- Attacker-controlled content
An email, a public GitHub issue, a CRM lead form, a shared doc, a web page, an MCP server description, another agent’s message. Anything the agent may read that the attacker may write.
- Retrieval / tool result
The content arrives as a tool result or a RAG hit. The runtime knows it is untrusted. The model is about to stop knowing that.
- Flattened into the context window
System prompt, user goal, prior results and attacker text now occupy one token stream with no trust labels attached.
- Model decides the next step
Model-directed control flow: the attacker’s text is now an input to the control-flow decision itself, not just to the wording of an answer.
- Runtime gate: allowed?
Schema validation, permission check, approval gate, egress policy. This is the only layer that can say no with certainty.
- Tool executes with the agent’s credentials
The blast radius is whatever the agent’s identity can reach — not whatever the attacker could reach on their own.
- Data leaves, or state changes
| Property | Chatbot | Agent |
|---|---|---|
What injected text can change | The wording of one answer shown to one user. | The next action. Which tool runs, with which arguments, against which system. |
Who executes the consequence | The human, if they choose to act on the answer. | The runtime, immediately, with the agent’s credentials and no human in the path by default. |
Blast radius | Misinformation, embarrassment, a bad decision downstream. | Everything the agent’s identity can read or write — often far more than the victim user needed for this task. |
Detection surface | The user reads the output and may notice it is odd. | A trace nobody is watching. Zero-click chains complete before a human sees anything. |
Where the fix lives | Output filtering, grounding, better prompts. | Least privilege, egress control, approval gates — the runtime, not the prompt. |
Key terms: prompt injection, indirect prompt injection, lethal trifecta, context window, tool call, least privilege
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.