Walking the List
Lesson 2 of 3 in The OWASP Top 10 for LLM Applications.
The list names risks at the application boundary; the mechanisms behind them are what this site teaches. So the most useful thing this lesson can do is walk the ten entries and, for each one, point at where its mechanism lives — a module here, a sister academy, or a paragraph in this very lesson for the two entries whose natural home is this page. Use the table as your index; the prose after it teaches what only this module teaches.
| Entry | The risk in one line | Where it’s taught | First mitigation |
|---|---|---|---|
LLM01 Prompt Injection | Attacker text — direct or planted in retrieved content — steers the model off its instructions. | Prompt Injection and Jailbreaks, and the root cause in The Single-Channel Problem | Assume injection succeeds: least privilege around the model; instruction-hierarchy training helps but does not close it. |
LLM02 Sensitive Information Disclosure | Training-data memorization, prompt contents, or cross-user context escape through outputs. | Deduplicate and scrub training data; keep secrets out of prompts; isolate tenants. | |
LLM03 Supply Chain | Unvetted weights, datasets, adapters, or serving dependencies enter your stack. | This lesson, below — model-artifact provenance | Pin versions; verify publishers and hashes; inventory model artifacts like dependencies. |
LLM04 Data and Model Poisoning | Manipulated training data plants behaviors or backdoors in the weights. | Data Pipelines and Curation — where training data comes from and how it’s filtered | Data lineage and curation; evaluate models for anomalous behavior before deployment. |
LLM05 Improper Output Handling | Downstream code renders, executes, or queries with model output as if it were trusted. | This lesson, below — the app-side dual of injection | Encode, sanitize, and parameterize model output like any untrusted input. |
LLM06 Excessive Agency | The model can do more than the task needs — permissions, tools, autonomy. | Sister site: AI Agent Academy owns the agent attack surface | Least privilege on tools and scopes; human approval for consequential actions. |
LLM07 System Prompt Leakage | The hidden prompt — and any secrets in it — is extractable through outputs. | No secrets in the system prompt, ever; treat it as public and put enforcement elsewhere. | |
LLM08 Vector and Embedding Weaknesses | The RAG retrieval layer leaks, accepts poison, or skips access control. | Enforce document-level access control at query time, not just at ingest. | |
LLM09 Misinformation | False but plausible output that users and systems act on. | Hallucination — why models confabulate by construction | Ground answers in retrieved sources; measure hallucination; design for abstention. |
LLM10 Unbounded Consumption | Unmetered requests: runaway bills, denial of service, model extraction by mass querying. | Serving in production — quotas and the serving layer | Quotas, rate limits, and per-request token caps on every exposed endpoint. |
LLM03, the paragraph this site owes you. An LLM application consumes artifacts that classical dependency scanning never sees: model weights, Fine-tuning adapters, tokenizer files, embedding models, and the datasets behind all of them. Each is code-equivalent — a fine-tune can change behavior as surely as a patch — yet teams that would never npm install an unreviewed package will happily pull an anonymous community fine-tune from a public hub. The mitigation posture is the one you already know from software supply chains, applied to new artifact types: pin exact versions, verify the publisher and the artifact hash, prefer model cards that document training data and lineage, and keep an inventory so you can answer “which of our systems runs weights derived from X?” The agent-side supply chain — tools, MCP servers, and the plugins an agent loads at runtime — is the sister AI Agent Academy’s territory.
LLM04 connects here too: poisoning is what a compromised data supply chain delivers. The mechanics of where training data comes from — and why curation and Deduplication are your leverage — are taught in the pre-training data pipeline module linked above.
LLM05 is taught here because it is pure application engineering — and it is the entry teams skip. The model’s output can contain anything its context can express: HTML, SQL fragments, shell syntax, markdown with live links, or instructions aimed at the next model in a pipeline. The moment that output crosses into a browser, a database client, an eval harness, or another prompt, it is input again — and every classic input-validation rule applies. Encode for the destination (HTML-escape for pages, parameterize for queries), strip or sandbox active content (scripts, auto-loading resources), and validate structure before parsing model output as data. If your code would not accept a string from an anonymous user, it must not accept the same string from the model — because via injection, an anonymous user may be its true author.
The remaining entries point outward. LLM06 Excessive Agency is the sister AI Agent Academy’s core subject — what happens when outputs become actions, and how to scope tools so a hijacked model has nothing worth spending. LLM08 is the reminder that a Vector database is a database: the Retrieval layer needs query-time access control, or your RAG pipeline becomes a search engine over data the requesting user was never allowed to read. LLM09 is Hallucination wearing a security hat — false-but-plausible output as a risk to the people and systems that act on it, mitigated by Grounding, measurement, and Abstention. And LLM10 is the serving layer’s problem: every public LLM endpoint is a machine that converts requests into cost, so quotas, rate limits, and token caps are security controls, not just budget hygiene.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.