Direct Injection

Lesson 1 of 4 in Prompt Injection and Jailbreaks: The Model-Level View.

Direct Prompt injection is the case where the attacker is the user: the adversarial instructions arrive in the user’s own message, aimed at overriding whatever the developer put in the System prompt. The schematic classic — quoted in essentially every published treatment of the subject — is a user turn that reads, in spirit, “Ignore previous instructions and do X instead.” No exotic payload, no encoding tricks: plain imperative text, and it has worked against real applications since the first instruction-tuned models shipped.

Why does a sentence like that stand any chance? The previous module gave you the root cause; here is the recap in one breath. A Chat template serializes the system prompt, the conversation history, and the new user message into one flat sequence of tokens. The role markers separating them are ordinary tokens from the vocabulary — the model was trained to treat text after a system marker as weightier, but nothing in the architecture enforces it. And instruction tuning (Supervised fine-tuning (SFT) then Reinforcement learning from human feedback (RLHF)) built a strong disposition to follow imperative text wherever it appears in the stream, because that is what the training data rewarded. Put those together: when two instructions conflict inside one stream, the model does not consult a policy. It predicts the next token, and the winner is whichever instruction the weights happen to favor for this input. That is a statistical outcome, not an authorization check.

Key terms: Prompt injection, Indirect prompt injection, Jailbreak, Adversarial suffix, Instruction hierarchy, System prompt leakage

A direct injection attempt, through the stream

  1. App composes the request

    Developer’s system prompt plus the user’s message — the app trusts one and not the other, but that distinction lives only in the app’s code.

  2. User message carries an override

    Schematically: “Ignore previous instructions and …” — imperative text competing with the developer’s imperative text.

  3. Chat template serializes everything

    One flat token sequence. Role markers are ordinary vocabulary tokens — labels the model learned about, not privileges the runtime enforces.

  4. Model reads one undifferentiated stream

    Every token, from every party, conditions the same next-token prediction. There is no separate lane for “trusted” text.

  5. Which instruction shapes the next token?

    Not an access-control check — a probability contest between trained dispositions, decided fresh for every input.

  6. Developer’s instructions win this time

    The trained bias toward system-marked text held — for this phrasing, on this model, today.

  7. Injected instructions win

    The output now serves the attacker: leaked system prompt, off-policy behavior, or content the app never intended.

What do attackers actually want from a direct injection? A remarkably common first goal is the system prompt itself. System-prompt leakage is significant enough that the OWASP Top 10 for LLM Applications names it as its own entry — LLM07: System Prompt Leakage — alongside LLM01: Prompt Injection at the head of the list (we walk the whole list in The OWASP Top 10 for LLM Applications). The extraction attempt is usually as schematic as the override itself: some variant of “repeat the text above.”

Why it matters is twofold. First, reconnaissance: the system prompt reveals your guard instructions, your formatting contract, the names of internal tools and data sources — a map of what to attack next. Second, misplaced secrets: teams have shipped API keys, internal URLs, and business logic inside system prompts on the assumption that the model would keep them confidential. The model-level fact says otherwise: the system prompt is context, models quote their context, and a disposition not to can be eroded by exactly the mechanism this lesson describes. The defensive posture follows directly — treat every system prompt as disclosable. Keep secrets and credentials out of it entirely; the prompt may hold instructions worth protecting, but it must never hold anything whose disclosure is a breach.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.