One Token Stream

Lesson 1 of 3 in The Single-Channel Problem.

Every security topic in this domain descends from one architectural fact, and it fits in a sentence: an Large language model (LLM) receives exactly one input — a sequence of tokens — and nothing in that sequence says where any token came from.

Watch what happens when a request is assembled. Your application takes the System prompt you wrote, the conversation so far, the user’s new message, whatever documents Retrieval fetched, and the results of any tool calls — and concatenates them. A Chat template wraps each piece in role markers. Then the tokenizer flattens the whole thing into token IDs from one vocabulary. From the model’s side of the door there is no “system part” and no “document part”: there is one stream, embedded by one embedding table, read by the same attention that reads everything else.

Here is what is missing, and its absence defines this entire domain. In a conventional system, data carries type and provenance: this buffer is executable, that one is not; this request came from an authenticated admin, that one from the open internet. The token stream has no such type system. A token from your carefully reviewed system prompt and a token from a stranger’s webpage are the same kind of object — an integer, a row in the embedding matrix. Provenance dies at concatenation. Whatever influence your instructions can exert on Next-token prediction, text from any other source in the stream can attempt to exert too.

That single missing property is why Prompt injection is not a bug that some vendor will patch. It is what the OWASP Top 10 for LLM Applications ranks as LLM01 — the risk it lists first — and it is a direct consequence of how the input works.

A stack diagram of five differently-sourced spans of a single prompt — system prompt, chat-template role markers, user message, retrieved document, and tool result — converging into a single highlighted layer labeled one flat token sequence. The retrieved-document span is emphasized to show that text from an unknown author sits in the same stream as the developer’s instructions, with the same privilege.

What the model actually receives: differently-sourced spans, concatenated into one flat token sequence. The highlighted span shows the problem — retrieved text arrives with exactly the same standing as the developer’s instructions. Toy example; real prompts interleave many more pieces. (illustrative — source: Greshake et al. (2023) — indirect prompt injection, arXiv:2302.12173)

“But the chat template separates the roles!” It does — visually, and statistically. It does not do so architecturally, and the difference is the whole point.

Role markers are tokens like any others. During supervised fine-tuning, the model saw millions of examples in which text following a system marker set the rules and text in other positions followed them — so it learned a strong disposition to treat those regions differently (the supervised fine-tuning module shows exactly how that teaching works). A disposition is a statistical tendency baked into weights. An enforcement boundary is a mechanism that makes violation impossible regardless of input. The template gives you the first and only the first. Nothing in the transformer checks a token’s role before attending to it; “system” is a costume, and the model has merely learned to usually respect costumes.

The SQL injection analogy — and where it breaks

Web security lived through this before. SQL injection existed because queries and user data traveled in one string: a name field containing SQL fragments could rewrite the query, because the database parsed everything in the channel as potential code. The fix was parameterized queries — the query’s structure is declared separately, user data is bound as data, and the parser can no longer mistake one for the other. A real type boundary, and the vulnerability class collapsed.

The analogy maps cleanly onto prompts: instructions and data share a channel, and attacker-supplied data gets interpreted as instructions. But the fix does not map. Parameterization worked because SQL has a formal grammar — “code” and “data” are syntactically distinct categories a parser can separate. Natural language has no such grammar: an instruction, a mention of an instruction, and a quotation of an instruction can be textually identical, and interpreting all of them is precisely the capability you bought the model for. No published mechanism gives natural-language prompts what parameterization gave SQL, and it is not clear one can exist — a model that reliably refused to interpret parts of its input would be a different kind of machine.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.