AI Agents Academy glossary

267 terms with primary sources.

agent

A system where a language model directs its own control flow. Your code executes the tool calls, but the model decides which tool, with what arguments, and when to stop. That one property — model-directed control flow — separates agents from chatbots (which only talk) and from workflows (where your code decides every step). It is also the source of every distinctive agent risk: you cannot fully enumerate in advance what the system will do.

agent loop

The core control structure of an agent: send the conversation to the model; if it returns tool calls, execute them, append the results, and go again; if it returns plain text, stop. Everything else in agent engineering — memory, orchestration, guardrails, evals — hangs off this loop. Most production failures are loop failures: no stopping condition, unbounded iterations, or results that poison the next turn.

autonomy

The degree to which a system selects and executes actions without human confirmation. Autonomy is a spectrum: a chatbot has almost none; a workflow has none by construction; an agent slides along it depending on which actions require approval. Where you place each action on that spectrum is a design decision — and in regulated settings, a documented one.

LLM (large language model)

The model at the center of every agent. Three properties matter more than the architecture: LLMs are stateless (they remember nothing between calls — you resend everything), bounded (the context window caps what they can read at once), and non-deterministic (the same input can produce different outputs). Agent engineering is mostly the discipline of working around those three facts.

context window

The maximum input a model processes per call, measured in tokens. Everything the agent "knows" about your task lives here: the system prompt, the conversation, every tool result so far. It fills up fast — one noisy tool result gets re-sent on every later turn — which is why context engineering and result design are core agent skills, not optimizations.

token

The atomic unit of model input and output. Cost, latency, and the context window are all denominated in tokens, so token counts are the agent engineer’s unit of account: a trace that "feels slow" is usually a trace that is token-heavy, and a loop that "costs too much" is a loop that re-sends a bloated history every turn.

non-determinism

LLM sampling is probabilistic: identical inputs can yield different outputs, and low temperature reduces but does not eliminate this. An agent compounds it — a run is a sequence of samples, each conditioned on the last. This is why agent testing is statistical (pass rates over many runs, not single assertions) and why "it worked when I tried it" is not evidence.

tool

The mechanism by which an agent acts on the world. A tool is defined by a contract — name, description, parameter schema — and that contract is the only documentation the model ever reads. The model returns a request to call the tool; your code validates, executes, and returns the result, which becomes the model’s next context. Tool design quality is the single biggest lever on agent reliability.

function calling

The wire-level mechanism under tool use: you send the model a list of function schemas; it may respond with a structured call — a function name plus JSON arguments — instead of prose. Your code executes it and returns the result tagged to the call. Every provider ships a dialect of this; the concept transfers even when the field names differ.

tool call

A single act in an agent run: the model names a tool and supplies JSON arguments; the runtime executes it and appends the result. Traces, permissions, approvals, and audits all operate at tool-call granularity — which is why approval gates must show the exact call parameters, never a summary the model wrote.

stopping condition

What keeps an agent from running forever. The natural stop is the model returning plain text instead of tool calls; production agents layer hard stops on top: maximum iterations, token budgets, wall-clock timeouts, and kill switches. An agent without an explicit stopping condition is an incident waiting for a trigger.

system prompt

The instruction block prepended to every model call — the closest thing an agent has to a job description. For agents it carries four loads: role and voice, hard constraints, tool-use policy (when to use which tool), and stopping guidance. It is also a trust boundary: the model cannot reliably distinguish your instructions from instructions smuggled in through data.

workflow

The disciplined sibling of an agent: the sequence of steps is fixed in code (a chain, a router, a state machine) and the model executes bounded sub-tasks inside it. Workflows trade flexibility for predictability, testability, and cost control — and for a large share of real problems, that trade wins. "Do I need an agent or a workflow?" is the first architecture question, not the last.

orchestration

The layer above the loop: routing work between models, tools, and (sometimes) multiple agents. Patterns range from prompt chaining and routing through supervisor–worker hierarchies. The recurring lesson: orchestrate in code where you can (deterministic, debuggable) and let the model direct flow only where the task genuinely requires it.

multi-agent system

An architecture where several agents — often a supervisor delegating to workers — share a task. It genuinely helps when subtasks need isolated contexts or different capabilities. It hurts everywhere else: more hops, more tokens, compounding non-determinism, and failure modes (deadlocks, contradictory state) that single agents cannot have. Default to one agent; add more only when the context, not the org chart, demands it.

supervisor–worker

The most common multi-agent shape. The supervisor holds the goal and decomposes it; each worker gets a focused brief and a clean context window, and returns results the supervisor integrates. Strengths: context isolation and parallelism. Weaknesses: the supervisor becomes a bottleneck and a single point of misunderstanding — garbage briefs in, garbage work out.

subagent

A delegation device: the parent agent hands a self-contained brief to a child agent that works in an isolated context window and returns only its conclusion. The win is context hygiene — a subagent can read fifty files and return one paragraph. The cost is information loss at the boundary: the child knows only what the brief says.

state machine

An orchestration style where every step and every allowed transition is declared up front, and model calls happen inside states. Graph-based frameworks build on this. You gain replayability, checkpointing, and audits; you give up the open-endedness that makes agents worth building for genuinely exploratory tasks.

memory

Anything that survives the statelessness of the model: the in-run scratchpad, the session transcript, and long-term stores (files, databases, vector indexes) that persist across sessions. Memory is a capability and a liability — stale or poisoned memories keep resurfacing in future contexts. Sometimes the right amount of memory is none.

context engineering

The practice of curating model input: what goes in, what gets summarized, what gets dropped. On long runs it is the difference between an agent that stays sharp and one that degrades — "context rot" — as noise accumulates and gets re-sent every turn. The core moves: compact results at the source, summarize completed work, and isolate side-quests in subagents.

RAG (retrieval-augmented generation)

A retrieval step (keyword, vector, or hybrid search) feeds relevant passages into the context before the model answers. For agents, retrieval usually becomes a tool the agent queries iteratively — agentic search — rather than a fixed pre-processing step. Grounding and citations turn RAG from "plausible answers" into "traceable answers".

human-in-the-loop

The controlled middle of the autonomy spectrum: the agent proposes, a human disposes — for the actions you gated. Good gates are selective (irreversible or high-stakes actions only; gate everything and approvals become rubber stamps) and honest (show the exact tool call, never the model’s own summary of it).

approval gate

The concrete mechanism of human-in-the-loop: intercept a tool call, display precisely what will execute — raw parameters, not a paraphrase — and proceed only on approval. The paraphrase rule is a security property: a compromised agent can lie in a summary, but it cannot lie about the literal call your runtime displays.

MCP (Model Context Protocol)

An open standard for wiring capabilities into AI applications: an MCP server exposes tools, resources, and prompts over a defined protocol; any compliant client (IDE, chat app, agent runtime) can use them without custom integration. It solves the M×N integration problem — but a server you install is code you trust, which makes MCP supply-chain review a security topic, not a convenience topic.

A2A (Agent2Agent protocol)

A protocol for agents built by different teams on different stacks to discover each other and exchange work. The common framing: MCP connects agents to tools; A2A connects agents to agents. Adoption is real but early — treat specific vendor support claims as dated facts to verify, not permanent truths.

structured outputs

Techniques and API features that constrain model output to a schema: JSON modes, schema-enforced decoding, and validate-and-retry loops. The engineering rule: parse, never trust — even schema-enforced output needs validation at the boundary, and your code must have a plan for the day the model returns garbage anyway.

prompt injection

The model reads instructions and data as one token stream, so any text an agent processes — an email, a web page, a tool result — can try to redirect it. Direct injection arrives from the user; indirect injection hides in content the agent retrieves. It is not a bug awaiting a patch; defenses are containment: least privilege, egress control, and gates on consequential actions.

lethal trifecta

Simon Willison’s name (June 2025) for the deadly combination: an agent that can read private data, process untrusted content, and communicate externally can be steered into stealing the data it reads. Remove any one leg and the chain breaks. It is the fastest architecture-review heuristic in agent security: count the legs.

Willison, "The lethal trifecta" (2025)

least privilege

The oldest rule in security, sharpened for agents: because the model can be talked into anything, the runtime must make the dangerous thing impossible — scoped credentials, read-only defaults, egress allowlists, per-tool permissions. Asking the model to behave is a preference; removing the permission is a control.

sandboxing

Isolation for the actions you cannot pre-approve, above all code execution. Containers, ephemeral VMs, and microVM session isolation bound the blast radius: whatever the code does, it does inside walls with no credentials and controlled egress. "Just run the code" is a security decision whether or not you noticed making it.

egress control

Network- or runtime-level restriction of an agent’s outbound communication to an approved list. It is the standard removal of the lethal trifecta’s third leg: an agent that cannot reach attacker-controlled endpoints cannot deliver stolen data, no matter how thoroughly it was fooled. Enforce in infrastructure; never delegate to the model.

tool-output poisoning

A tool result is appended verbatim into the context, so a compromised or malicious tool — a web page, a search hit, a rigged MCP server’s response — can carry instructions the model may follow. This is why tool results from untrusted sources need the same suspicion as user input, and why MCP servers are supply chain, not plumbing.

guardrails

The enforcement layer that runs outside the model: input filters, output classifiers, tool-call validators, PII redaction, grounding checks. Every major cloud ships a managed version. Guardrails are necessary and insufficient — they catch known-bad patterns, while containment (privilege, egress, sandboxes) bounds the unknown ones.

trace

The primary debugging artifact in agent systems: a structured record (typically spans, as in OpenTelemetry) of everything a run did. Reading traces is to agents what reading stack traces is to conventional code — the skill that separates "the agent is flaky" from "step 7 returned an empty result and the model improvised".

span

The unit of tracing: an operation with a start, an end, attributes (model, tokens, cost, status), and a parent — spans nest into the tree that is a trace. OpenTelemetry’s generative-AI semantic conventions standardize span names and attributes so agent traces are portable across observability backends.

eval

A dataset of cases plus a grading method, run repeatedly to measure whether the agent works — and keeps working after every prompt tweak and model upgrade. Two axes matter: outcome (did it get the right answer) and trajectory (did it get there sanely). The discipline: deterministic checks first, LLM judges only where rules cannot reach.

LLM-as-judge

Automated grading by a second model for qualities rules cannot check (helpfulness, tone, groundedness). It works — if you treat the judge as an instrument to calibrate: write concrete rubrics, control for position and verbosity bias, and validate a sample against human judgments before trusting the numbers.

golden dataset

Your ground truth: representative cases — ideally harvested from real traffic and real failures — paired with verified expected outcomes. Quality beats quantity: twenty cases that reflect production reality outperform five hundred synthetic ones. It grows the way a regression suite grows: every production incident becomes a case.

kill switch

The last line of operational defense: a mechanism that halts agent execution now — per agent, per tenant, or globally. It earns its name only if it is fast, authorized, and rehearsed. The AgentOps rule of thumb: if you have never exercised the kill switch, you do not have one; you have a hope.

grounding

The practice of making an agent’s answers rest on retrievable evidence: retrieved passages in context, citations in output, and checks that the cited source actually supports the claim. Grounding converts "the model said so" into "the model said so, per this document" — which is what review, audit, and trust require.

turn

The unit of work in an agent run. Each turn resends the accumulated transcript, so a turn is a full re-read, not an increment — cost and latency grow with run length even when the new instruction is one line. Every budget, gate, and trace boundary you build is measured in turns, and every failure you debug happened inside one.

natural stop

The one way an agent loop is supposed to end — the model decides it is finished and answers. Everything else that ends a run (turn caps, token budgets, timeouts, kill switches) is a backstop bolted on from outside, and the distinction matters operationally: a run that hit a backstop did not finish, it was interrupted, and your metrics and error handling must tell those two apart.

max turns

A counter in your runtime that aborts the loop after N iterations, regardless of what the model wants next. It catches the classic pathology: an agent retrying a broken tool forever, or ping-ponging between two tools while making no progress. Set it from observed traces, not from intuition — if healthy runs finish in four turns, a cap of fifty is not a safety net, it is fifty turns of billing before anyone notices.

token budget

The complement to a turn cap: turns limit how many times the model gets to think, tokens limit how much it gets to read and write. It catches the failure a turn cap misses — three turns that each drag a 200-KB tool result through the context window. Budget the run, not the call, and log how close each run came to the ceiling; runs that repeatedly graze it are telling you the task needs compaction, not a bigger budget.

agent runtime

Everything that is not the model: the loop itself, tool dispatch and validation, retries, budgets, approval gates, logging. The runtime is where control actually lives — the model can only ever request, so any rule you need to hold under adversarial input has to be enforced here. "The prompt tells it not to" is a preference; "the runtime refuses" is a control.

tool contract

The trio of text the model actually sees for each tool. The name routes, the description says when to reach for it and when not to, and the schema constrains the arguments. The model cannot read your code, your API docs, or your wiki — so a tool that gets misused is usually a contract that failed to say something, not a model that failed to reason. Contract quality is the highest-leverage edit in most agent codebases.

parameter schema

The machine-readable half of a tool contract: types, enums, required fields, patterns, and examples. It does double duty — it steers the model and it defends your code. Tight types cut the space of wrong calls (an enum cannot be misspelled, a pattern rejects invented IDs), and the same schema is what your handler validates against before executing, because a schema the provider enforces is still input you did not write.

tool result

The observation half of the loop, and the most under-designed surface in agent engineering. A result is not a return value, it is a prompt fragment: it gets re-sent on every later turn, so a chatty payload taxes the whole run, and an untrusted one can carry instructions the model may follow. Return the fields the model needs, name them clearly, keep errors actionable ("order not found; check the ORD- prefix"), and compact at the source.

tokenizer

The text-to-token converter that sits in front of every model. It explains behaviour that otherwise looks like stupidity: the model never sees letters, so counting characters, reversing strings, or reasoning about exact offsets are tokenizer artifacts, not reasoning failures — put that work in a tool. It also explains your bill, since price, speed, and context limits are all denominated in its output. Working rule: 1 token ≈ ¾ of an English word.

statelessness

A property of the model, not a limitation of your framework: each API call is evaluated from scratch on the tokens you send. Every appearance of memory in an agent is your code resending context — which is why history growth is a cost curve, why a crashed run loses its scratchpad, and why "the model already knows the customer from last week" is the most expensive misconception on a sprint board.

sampling

The model outputs a distribution over the whole vocabulary; the sampler selects one token, which is appended and fed back in. All the model’s randomness lives in this one step — and because each pick conditions the next, a single flipped token can rewrite the rest of a reply. Temperature and top-p are knobs on the pick, not on the thinking.

temperature

A scaling factor on the model’s scores. At 0 the sampler is greedy — the natural setting for tool arguments, extraction, and anything your code parses. Higher values flatten the distribution: more variety, more surprise, more nonsense at the extreme. Two things it does not do: it does not make answers more correct (a confidently wrong model is wrong at every temperature), and temperature 0 is not determinism — floating-point order of operations shifts with batching and hardware, so identical prompts can still diverge.

top-p

A tail-cutting knob. Where temperature reshapes the whole distribution, top-p removes candidates outright — with p = 0.9 the sampler never lands on the long tail of near-zero tokens that produce derailments. Practical guidance for agents: change one knob at a time and record which one, because a run tuned with both moved is a run nobody can reproduce or reason about.

prompt caching

Providers can retain the computed state of a prompt prefix and bill re-reads of it far below normal input rates. Agents are the ideal customer: every turn resends the same system prompt, tool schemas, and early transcript. The engineering consequence is a stability requirement — keep the prefix byte-identical and put volatile content (timestamps, random ids) at the end, or you invalidate the cache on every call. Discount levels, minimum prefix sizes, and cache lifetimes vary by vendor and change often; check the current pricing docs rather than a number you remember.

latency

The delay a user experiences. Single-call thinking misleads here: an agent’s wall-clock time is the sum over turns of prefill, decoding, and tool execution, so a 3-second model call becomes a 40-second wait at twelve turns. That arithmetic is why interactive latency budgets push you toward fewer turns, parallel tool calls, and streaming — and why some products should not put a loop on the hot path at all.

time to first token

The wait before the first output token of a call, driven mostly by prefill: the model reading the transcript you resent. In an agent it degrades as the run lengthens, because turn twelve prefills far more context than turn one. Streaming hides it from users on the final answer only; the intermediate turns nobody sees still spend it, which is why time-to-first-token is a per-call metric and total run time is the one you promise.

scratchpad

The shortest memory horizon: everything accumulating in the context window while a task is in flight. It is where the agent does its thinking, and it is not durable — if the process crashes at turn nine, the plan and the nine turns of observations are gone unless something wrote them out. That is the whole argument for checkpointing long runs: the scratchpad is the most valuable state in the system and the least persistent.

session memory

The middle horizon: state scoped to a conversation or session, resent (or summarized and resent) on each turn so the model appears to remember what was said. It is bounded by the context window, so on long sessions it becomes a curation problem, not a storage problem — you decide what survives summarization. Ending the session should end the memory; if it does not, you have long-term memory and the retention questions that come with it.

long-term memory

The horizon that persists until something explicitly deletes it: files, databases, vector indexes, managed memory services. It buys continuity across sessions and, in exchange, takes on every hard property of a data store — write policy, provenance, expiry, deletion, and disclosure. A store without those is not memory, it is an unbounded log of inferences about your users that resurfaces in future prompts. Sometimes the right long-term memory is none.

memory expiry

A retention policy applied per record: volatile facts get short lifetimes, durable ones longer, and nothing is written as permanent by default. Expiry does two jobs at once — it keeps the store from asserting last quarter’s truth forever, and it bounds the blast radius of every bad write. A poisoned record with a 30-day TTL is a contained incident; the same record stored permanently is an implant that fires until someone happens to notice.

memory provenance

The metadata that makes a memory store accountable. Retrieval uses it to rank — trusted and recent should beat untrusted and old, whatever the similarity score says. Deletion needs it to be real: "forget everything derived from that conversation" is only executable if derived records point back to their source. And incident response uses it as the search key, so that finding every tainted record after a poisoning attempt is a query instead of a guess.

memory poisoning

Prompt injection with persistence. A single injected instruction that reaches the memory store stops being a one-run problem and becomes a standing implant retrieved into future contexts — including sessions with other users, other tasks, and no attacker present. It is why the write path deserves more scrutiny than the read path: gate what qualifies for storage, record provenance, expire aggressively, and make purge a rehearsed operation rather than a hypothesis.

OWASP Top 10 for Agentic Applications — ASI06 Memory & Context Poisoning

privacy accumulation

The failure mode where an agent’s memory store grows into a profile: health hints, family details, and offhand remarks retained because writing them was easier than deciding not to. No single write is the violation; the aggregate is — and it arrives with disclosure, retention, and deletion obligations that nobody scoped. The defence is a write policy with a purpose test, short default lifetimes, and a user-visible answer to "what do you remember about me?"

compaction

What a harness does when a run outgrows the context window: truncate or summarize earlier turns and continue. It keeps long runs alive at a real price — the model silently loses whatever was compacted away, and it cannot tell you what it forgot. So compact deliberately: summarize completed work rather than the current plan, trim tool results at the source before they ever land, and log what was dropped so a confused later turn is debuggable.

model-directed control flow

In a script or a workflow, a human decided the sequence of steps in advance and froze it in code. In an agent, the model chooses the next step based on what it just observed — same LLM, same tools, different holder of the steering wheel. Every distinctive agent capability (handling the case you did not anticipate) and every distinctive agent risk (you cannot enumerate in advance what it will do) follows from this single shift.

autonomy spectrum

A dial rather than a switch, and — the part teams miss — set per action, not per agent. The same agent can draft freely, need approval to refund, and never send external email. Position each action by asking whether a mistake is cheaply reversible and whether it would be noticed quickly. Then keep watching: approval queues decay into rubber stamps, which moves the dial up a tier with nobody deciding to.

compounding errors

Why per-step accuracy is a misleading metric for agents. Chain twenty 95%-reliable steps and the run succeeds roughly 36% of the time (0.95²⁰ ≈ 0.36) — but the arithmetic understates it, because a wrong step feeds a wrong observation into every later decision. Step three misreads a config file and steps four through twenty diligently solve the wrong problem. The practical response: shorten chains, verify at the steps that matter, and measure whole-run pass rates rather than step accuracy.

blast radius

The reach of a mistake: which records, accounts, tenants, or customers a single tool call can touch. It is the sizing input for every control you build — match the friction to the blast radius, so reading a catalogue stays frictionless while issuing a refund or dropping a table does not. It is also the design goal of containment: scoped credentials, sandboxes, egress allowlists, and per-tenant isolation all exist to make the circle smaller before anyone needs it to be.

anti-checklist

The restraint counterpart to a requirements list: fixed steps, a hard latency budget, a compliance need for reproducible decisions, no error tolerance, no eval data. These are disqualifiers, not trade-offs — one clean hit means the loop is the wrong shape for the problem, and the honest move is to ship the workflow. Used early, the anti-checklist is a week of design; used late, it is the postmortem.

over-engineering

The dominant failure mode in this field, and it does not look like failure at first: a loop, five tools, and a supervisor doing what a hundred lines of Python did more cheaply and more predictably. The test is not "could an agent do this?" but "does the agent earn its place?" — against cost, latency, debuggability, and blast radius. When a task genuinely survives that test, an agent is not over-engineering; it is the only design that works.

debugging tax

What you pay forever after shipping an agent: bugs that do not reproduce, fixes that must be verified statistically, and traces to read for every incident. It is why an agent’s true cost is not the build — it is the build plus a permanent share of engineering attention. Budget it explicitly: tracing from day one, a golden dataset, and pass-rate tracking, or you will pay the tax in outages instead of tooling.

rule of least power

A web-architecture principle that transfers exactly to agent design: the weaker the mechanism, the easier it is to analyse, test, and trust. A regex you can reason about beats a model call you cannot; a state machine you can replay beats a loop you can only sample. It is the reason "do I need an agent?" is the first architecture question — power you did not need is complexity, cost, and attack surface you now own.

W3C TAG finding, "The Rule of Least Power" (2006)

pass rate

The headline metric of agent testing: define what pass means (the suite goes green, the refund lands in the right account, a judge scores the trace acceptable), run the task 20 or 100 times, and report the fraction. One successful run is a coin that came up heads once. Progress then reads like an engineering number — passes 87 of 100, up from 74 last release — and regressions become visible instead of anecdotal.

tool overlap

The most common cause of "the agent chose the wrong tool". Given search_orders and find_purchase, the model is not confused about your domain; your contracts failed to draw a boundary, and it is guessing between overlapping descriptions. The fixes are boundary work, not prompt work: merge near-duplicates, state exclusions in each description ("for refunds use issue_refund, not this"), and cut the toolset — every extra tool adds context and one more way to be wrong.

abstraction debt

Frameworks buy speed by hiding the loop, the prompts, and the state handling. The debt comes due at debug time: when the agent goes wrong you must understand the abstraction and the model underneath it, and the hidden scaffolding is usually the least documented part. A library that saves a week in month one can cost that week back every quarter, fighting defaults you never chose — which is why writing one raw loop first is an investment, not a rite of passage.

agent framework

Frameworks package four things: the loop, tool registration, state (ideally with checkpointing), and observability hooks. Choose one for the shape of your problem, not for its star count — a graph library when you need resumable audited runs, a provider SDK when you are single-vendor, nothing at all for a hundred-line loop with three tools. The honest graduation signal: once your own loop grows checkpointing, subagent spawning, and a trace viewer, you have written a framework already — an unmaintained one, with a team of one.

agent harness

The harness is everything you would otherwise write yourself: the loop that keeps calling the model, tool dispatch and result formatting, compaction when the context fills, retries, permission checks, and the trace. Vendors now sell the harness itself rather than just the model — the Claude Agent SDK, for instance, drives the Claude Code harness programmatically. Inheriting a hardened harness is the fastest route to a working agent and the deepest coupling you can take on, because your prompts end up tuned to scaffolding you do not control.

agentic search

Classic RAG retrieves once, before the model answers. Agentic search moves retrieval inside the loop: the model formulates a query, inspects the hits, follows leads across hops, and stops when the evidence is sufficient. That handles the questions one query cannot — comparisons, multi-hop chains, "find the exception" — at the price of unpredictable token spend and latency, so it needs an iteration cap and a budget like any other loop.

automation complacency

A human-factors failure, not a model failure: when the agent has been right ninety-nine times, the hundredth approval gets the same reflex click. Bad gate design manufactures it — gate the read-only queries and the retries and approval requests start arriving faster than judgment can process them, so deny rates collapse toward zero. The defenses are selectivity (gate irreversible, high-blast-radius actions only) and measuring the reviewer, not just the agent: approval latency and deny rate are monitoring signals.

Parasuraman & Manzey, “Complacency and Bias in Human Use of Automation” (2010)

C4 model

Simon Brown’s diagramming convention: each level zooms in one step, and you never mix levels on one diagram. Its most useful property for agent work is social, not notational — most confused design reviews are two people arguing at different zoom levels, and naming the level first ends that. What C4 cannot express is the thing agents are risky for: its boxes are deployables, so “the model chooses which of forty tools to call, and one moves money” has no home in it. Use C4 for the system the agent lives in, and an agent-specific notation for the agent.

c4model.com (Simon Brown)

chain of thought

Eliciting visible steps ("think step by step", or a reasoning model doing it natively) measurably improves multi-step accuracy. For agents it pays twice: the reasoning conditions the next tool choice, and it leaves something legible in the trace. Treat it as a debugging aid, not as evidence — the stated reasoning is generated text, and it can be a plausible story about a decision the model reached on other grounds.

Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” — arXiv:2201.11903

checkpoint

The durable record of where a run stands: message history, tool results, and whatever state the orchestrator tracks, serialized at a step boundary. Checkpoints make pausing for a human, surviving a crash, and rewinding step 7 real rather than aspirational. They are also the cure for context rot: because the model is stateless, the damage lives entirely in the context — restart fresh, re-read the checkpoint, and the run resumes with its knowledge and without its noise.

checkpointing

The mechanism behind resumable agents: after every step, serialize state into a store keyed by thread or session. It is tedious to build and unforgiving to get wrong, which makes it the most common reason teams graduate from a hand-rolled loop to a graph framework or a durable-execution engine. Checkpointing is what turns "the process died" from a lost run into a resumed one — and it is the precondition for audit, because you cannot reconstruct a step you never wrote down.

chunking

The first and most consequential decision in a retrieval pipeline, because no later stage recovers what chunking destroyed — a table row severed from its header, step 4 without steps 1 to 3. Strategies run from fixed windows with overlap to structure-aware splits on headings, sections, or code boundaries. There is no universal right size; there is only the size you measured on your corpus with your questions.

citation

Citations turn "the model said so" into "the model said so, per this passage". They only do work if they are generated from the retrieval record rather than written by the model — a model asked to add references will happily produce plausible ones that lead nowhere. Attach source ids to chunks as they enter the context, render them as links a reviewer can open, and treat an uncheckable citation as a defect rather than a nicety.

compensating action

Borrowed from distributed transactions, and unavoidable in agents because tool calls commit one at a time with no shared rollback. Write the undo before you write the do: for every side-effecting tool, name what reverses it, who may invoke it, and how long the window lasts. Two honest limits — compensation is often partial (a pallet on a truck is recalled, not un-shipped) and it can itself fail, which is what a dead-letter queue plus a named human owner is for. The design goal is not perfect rollback; it is that no failure leaves the world in a state nobody can read. A cheap companion rule: order the effects in a step so the hardest to reverse happens last.

context isolation

The parent hands off a bounded job; the child reads fifty files, burns forty thousand tokens, and returns one paragraph. The parent pays for the conclusion, not for the search — which keeps the main context sharp on long runs and buys a privilege boundary almost for free, since the child can hold the untrusted material while only its verdict comes back. The cost sits at the boundary: the child knows nothing except what the brief says.

context rot

The mechanism is attention, not amnesia — every token is still there, but the model spreads attention across an ever-noisier set, and instruction-following and multi-step reasoning degrade well before needle-in-a-haystack retrieval does. Failed attempts, stale plans, and one 12,000-token grep result get re-sent every turn, so the damage compounds. Rot is recoverable precisely because the model is stateless: checkpoint the durable state, restart, re-read — knowledge kept, noise dropped.

control flow

Control flow is the branching and sequencing of a system. Model-directed control flow is the defining property of an agent — and the source of every risk peculiar to agents, because you cannot enumerate the paths in advance. The discipline that follows: keep control flow in code wherever the task allows (deterministic, testable, auditable), and hand it to the model only where the branches genuinely cannot be written down ahead of time.

critical path

The reason latency work so often achieves nothing. In a fan-out, three fast branches running beside one slow branch contribute zero to wall-clock — so making the loud, obviously-slow dependency instant can save literally nothing, while the quiet branch nobody complains about owns half the run. Read the path from nested trace spans before choosing what to optimise. The same discipline applies to cost with a different shape: cost is a sum over hops rather than a maximum, so cost attribution and latency attribution point at different components and both are worth computing separately.

data-flow diagram (DFD)

The oldest useful drawing for security review, and the direct ancestor of an agent trust-boundary overlay. Four element types — external entities, processes, data stores, data flows — plus the boundary markings that make it a security artifact rather than a description. For agents the adaptation is small and the payoff is large: mark the flows carrying untrusted content inward and the flows carrying your data outward, and the trifecta legs count themselves. If your organisation already threat-models on DFDs, draw the overlay in their notation — matching the house style beats a marginally better diagram.

deadlock

Classic deadlock with no lock manager to detect it: A waits for B’s result, B waits for a clarification A never sends, and the task sits in a pending state looking healthy. No prompt fixes this — it is a runtime problem: timeouts on waiting as well as on acting, a stopping condition for blocked tasks, and escalation to a human after N minutes. A2A gives the state a name you can alert on — a task parked in INPUT_REQUIRED that nobody polls.

A2A Protocol Specification 1.0 — task lifecycle (INPUT_REQUIRED, AUTH_REQUIRED)

delegation brief

A brief that works states the goal, the constraints already discovered, what has been tried and rejected, the output shape expected, and the tools allowed. Unstated context does not degrade gracefully — it vanishes, because a fresh context has no shared history to fill the gap from. Chain briefs (parent to A, A to B) and the loss compounds into the telephone game, which is why delegation chains stay shallow and every hop re-verifies intent instead of assuming it.

divergent state

When several agents write to the same world without one source of truth, you get a hotel booked for a cancelled trip and two "done" reports that disagree. The failure ships success signals — every per-agent health check is green while the composite is broken — so detection has to be system-level: cross-agent invariants, task-level completion checks, and exactly one owner per piece of state. Runtime isolation boundaries beat politer prompts.

durable execution

A durable-execution engine records each completed step, so a process that dies mid-run resumes where it stopped instead of restarting or losing the work. Agents need it for two reasons: runs that wait on a human can stay pending for days, and a retried tool chain must not charge the card twice. The price is a programming model — steps must be deterministic and side effects idempotent, because the engine replays your code to rebuild state. Pydantic AI ships first-party integrations with engines including Temporal, DBOS, Prefect, and Restate (as of September 2026 — check the current docs).

embedding

An embedding model turns a chunk of text into a point in a high-dimensional space, so "how do I get my money back?" lands near "reimbursement procedure" with no shared keywords. A vector index stores those points and returns the nearest ones for an embedded query. Embeddings capture similarity, not correctness — near in meaning includes the confidently wrong neighbour, and a query and its exact negation embed close together, which is why retrieval needs reranking and grounding checks downstream.

episodic memory

One store type in a memory architecture: episodes (transcripts, run summaries, outcomes) retrieved by similarity or recency, distinct from semantic memory (durable facts and preferences) and from the in-run scratchpad. Episodic recall is what makes an agent feel continuous across sessions — and it is where poisoning persists, because a wrong episode retrieved next week is a wrong instruction with a timestamp. The write policy (what gets stored, with what provenance, expiring when) matters more than the database you choose.

escalation

Gates operate on single actions; escalation operates on the task. The triggers are knowable in advance: confidence below a threshold, a policy exception, repeated tool failures, a frustrated user, a value above the agent’s limit. Escalation must arrive with the context attached — the trace, what was tried, the proposed action — or you have moved the work without moving the knowledge. Systems with no escalation path do not get fewer hard cases; they get hard cases handled badly.

evaluator–optimizer

A workflow pattern with two roles, both orchestrated in code: generator and evaluator. It earns its cost when the rubric can be stated explicitly and iteration measurably improves the artifact — translation, ad copy, code that must pass a linter. Without an explicit stopping condition it either never converges or converges on the evaluator’s taste instead of the user’s, so cap the iterations, require a measurable gain per round, and validate the evaluator against a golden dataset before you trust its verdicts.

Anthropic, “Building effective agents” (2024)

external feedback

The difference between a reflection loop that improves and one that spins in place. Same weights, same blind spots: a model critiquing its own reasoning frequently certifies its own error. External feedback breaks the tie — a failing test, a type error, a schema validation failure, a reviewer’s note. Prefer a signal your code can check over a critique your model wrote, and reserve self-critique for the cases where no executable check exists.

Huang et al., arXiv:2310.01798 (limits of self-correction without external feedback)

fan-out

The workhorse delegation pattern: the parent decomposes a question, writes all the briefs up front, spawns the workers in parallel, and synthesizes their compressed reports. It buys parallelism and context isolation, and it is the only multi-agent shape that refunds latency. If worker 2 would work differently after seeing worker 1’s findings, the slices are not independent — that is a pipeline in a fan-out costume, and forcing it parallel makes every worker guess at what a neighbour already knows.

fan-out / fan-in

The mechanical half of parallelization: fan out to N model calls, fan in through an aggregator that combines them. The merge is where the design actually lives — concatenating three independently written summaries produces overlap and contradiction, so aggregation needs its own rules or its own model call with a rubric. Latency is the slowest branch; cost is the sum of all branches — which is why this pattern is a latency trade, never a savings one.

fine-tuning

One of three ways to get knowledge into answers, alongside RAG and long context. Fine-tuning is strongest at form — tone, output structure, a domain’s idiom, a small cheap model matching a big one on one narrow task. It is weakest at freshness: every corpus change means another training run, and the model cannot cite what it absorbed. For agents, tune behaviour and retrieve facts — and budget for redoing the tune every time you change base models.

fresh-eyes review

A subagent pattern where you deliberately send little down: the artifact and the criteria, not the reasoning that produced them. An agent that watched the work happen inherits its assumptions and rubber-stamps them; a fresh context checks the artifact against the spec instead. Independence is a context property, not a prompt property — telling the same conversation to "review critically" does not buy it. One blind spot survives: same model, same training, so orthogonal checks still need a different model or an executable test.

graph orchestration

The state-machine style made concrete: nodes hold the work (often a model call), edges declare which transitions are legal, and shared state flows between them. You get checkpointing, resumability, replay, and a diagram that is the control flow rather than a hopeful picture of it. The cost is that surprises become illegal — anything the graph does not permit cannot happen, which is exactly right for audited pipelines and exactly wrong for genuinely open-ended work.

groundedness

The measurable form of grounding: claim by claim, does the retrieved context entail what the answer says? It catches the failure RAG specifically invites — retrieval worked, then the answer drifted past it. Cloud eval suites ship it as a built-in RAG metric (Microsoft Foundry lists groundedness among its evaluators as of September 2026). Score against the context you actually sent, not against the world: an answer can be true and ungrounded, which is still a defect in a system that promises citations.

hallucination

A model generates the most plausible continuation, which is not the same as the correct one. Agents raise the stakes twice over: a fabricated value becomes a tool argument, and a tight output schema makes it look more legitimate, not less. A required field with no escape hatch is a hallucination generator — the model cannot omit it, so it invents a plausible id that flows into a query and returns "not found" instead of failing loudly. Model unknowns explicitly and resolve identifiers with a tool rather than asking the model to remember them.

handoff

The peer alternative to supervisor–worker: a triage agent decides the question is about refunds and transfers control to the refunds agent, which then talks to the user directly. The structural signature is transferred ownership — control never comes back. The transfer is a model-chosen branch, so treat it with routing-grade suspicion: cap the hops in the runtime (three handoffs, then a human) and eval for overlapping specialist definitions, or two agents will politely bounce a ticket between them while the meter runs.

human-on-the-loop

The act-and-report tier of the autonomy spectrum, distinct from human-in-the-loop, where a human approves before the action fires. It is the right design for high-volume reversible actions, where per-action gates would only manufacture automation complacency. It is oversight only if someone actually reads the reports — an unread digest is autonomy with paperwork — so pair it with anomaly alerts, rate monitoring, and a kill switch for the whole action class.

hybrid search

Vector search finds meaning and misses exact strings; keyword search nails error codes, part numbers, and surnames and misses paraphrase. Hybrid runs both and merges them (reciprocal-rank fusion, or a weighted score), usually with a reranker behind it. It is the sane default for production retrieval, because the queries that embarrass a pure-vector index are exactly the ones users type: "ERR_2481", "invoice 90412", a person’s name.

idempotency

Agents retry constantly: the loop re-plans, a call times out, a durable engine replays the run, a human resumes a paused approval. Without idempotency each of those is a duplicate refund. The mechanism is an idempotency key — a caller-supplied id the downstream service deduplicates on — plus tools designed so a repeat is detectable rather than invisible. Assume every consequential tool call will eventually happen twice, because in a non-deterministic loop with retries it will.

long context

The third route to knowledge, beside RAG and fine-tuning: if the corpus physically fits, retrieval infrastructure is a complication you can skip. Three criteria decide it — corpus size, per-query cost (you pay for the whole corpus on every call), and attention, because a big window does not confer proportionally big attention. A long window is a budget, not a bucket: instruction-following degrades as filler grows, and prompt caching changes the economics without touching the attention problem.

lost in the middle

Models use information at the beginning and end of a long input measurably better than material in the middle. The consequences are structural: put the task, the hard constraints, and the freshest evidence at the edges; restate critical rules near the end rather than trusting a line written 40,000 tokens ago; order retrieved passages by relevance instead of dumping them. Never treat "it is in the context" as "the model will use it" — that assumption is where quiet agent failures live.

Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” — arXiv:2307.03172

orchestrator–worker

The workflow pattern for problems whose subtasks are unknowable until you look — the changelog nobody could template, the edit that touches an unknown number of files. Unlike sectioning, the count and shape of subtasks are decided by a model, which makes this the most agent-like workflow pattern and the one needing the tightest bounds: cap the plan size, cap the worker count, keep briefs non-overlapping. The orchestrator is a single point of misunderstanding for the whole run — bad decomposition yields workers that succeed at the wrong tasks.

Anthropic, “Building effective agents” (2024)

parallelization

Two variants with different purposes: sectioning splits a task into independent pieces run concurrently, and voting runs the same task repeatedly to exploit variance. Both trade tokens for wall-clock time — they cost more, not less. The two traps are hidden dependencies (sections that secretly need each other’s output) and merge design: the aggregation step is real work, and concatenation is not it.

Anthropic, “Building effective agents” (2024)

plan-then-execute

Separating planning from execution buys two things ReAct cannot: a plan a human can approve before anything happens, and cheaper execution, since a small model can run steps a big model planned. The failure mode is ossification — a plan written before the first tool result is a hypothesis, not a schedule — so any serious implementation pairs it with explicit replanning triggers. AutoGPT is the cautionary case: planning without the discipline to abandon a failing plan.

prompt chaining

The simplest workflow: outline, then draft, then check. Each call gets a smaller and better-specified job than one giant prompt would, and the seams between steps are where your gates go. The failure to design against is compounding errors — step 1 is 90% right, step 2 builds on the wrong 10%, and by step 4 nothing is recoverable — which is why a gate has to be able to fail the chain, not just log a warning nobody reads.

Anthropic, “Building effective agents” (2024)

provenance

Provenance is the metadata that answers "where did this come from?" after the fact: tag every retrieved chunk and every memory record with tenant, source, run id, and time. It does three jobs at once — citations for the user, tenant assertions before the model call, and deletion you can actually execute when a record has to go. Untagged context is unauditable context: you cannot cite it, cannot prove it belonged in this run, and cannot remove it on request.

quality gate

The seam-level control in a chained workflow: validate the step output against a schema, a business rule, or a test run, then pass, retry, or halt. It is not an approval gate (that asks a human) and not an LLM judge (that asks a model). A gate that cannot fail the chain is theatre — if your code logs a warning and forwards the bad output anyway, you paid for the check and kept the compounding error.

ReAct

The pattern that made tool-using models work: rather than reasoning in one shot or acting blind, the model alternates thought, action, and observation, so every step is conditioned on a real result. ReAct won by disappearing — what started as a prompting technique is now the default shape of the agent loop and a native API primitive, so most agents you build are ReAct whether anyone says the word or not. Its weakness is myopia: it never commits to a plan you could review before it starts.

Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” — arXiv:2210.03629 (Oct 2022)

reflection

A generate–critique–revise loop, run either inside one agent or split between a generator and a critic. It helps measurably when the critique has something real to bite on: test results, a schema, a rubric, a different model. Same weights, same blind spots — the model that made the error is well placed to miss it again, and research on self-correction without external feedback found revisions can make answers worse. Cap the rounds and require an improvement signal you can measure.

Shinn et al., “Reflexion” — arXiv:2303.11366

replanning

A plan-then-execute agent needs explicit triggers to re-plan: a step failed twice, a tool returned something the plan never anticipated, a new constraint appeared, half the budget is gone with a quarter of the work done. Without them the agent executes a dead plan faithfully to the last step. Make the trigger a rule in code, not a hope in the prompt — and cap the replans, because an agent that re-plans on every surprise never finishes either.

replay

With checkpointed state you can re-execute a run from any step: durable engines replay to rebuild state after a crash, and debuggers replay to reproduce a failure without re-driving the whole conversation. Two constraints make it safe — replayed steps must be deterministic and their side effects idempotent, or replay re-sends the email. Model calls are the awkward part: pin versions and record the exact request, or accept that replay reproduces the path and not the tokens.

reranking

A reranker scores each candidate against the query directly instead of comparing pre-computed vectors: much more accurate, much too slow to run over a whole corpus. Hence the standard shape — fetch wide and cheap, rank hard, then hand the model only what earned its place in the context. The rule that makes the stage matter: no later stage recovers what an earlier one dropped, so a reranker cannot rescue a first-stage retrieval that missed the answer.

routing

One cheap classification step in front of several specialized paths — a small model for FAQs, a large one for escalations, deterministic code for refunds. It often improves quality and cost at once, because each path gets a prompt written for exactly one job. The failure mode is the misclassification cascade: a wrongly routed request is then handled confidently by the wrong specialist. Eval the classifier separately from the paths, and always give it an explicit "unsure" route.

Anthropic, “Building effective agents” (2024)

sectioning

Sectioning works when the pieces do not need each other: summarize twelve documents, screen one input against four unrelated policies, generate five test files. The real design work is the boundaries and the merge, because sections that secretly depend on one another produce confident output built on guesses. The independence test is the one fan-out uses: can every section brief be written before any section starts?

self-critique

Ask a model "is this correct?" about work it just produced and you get another sample from the distribution that produced the mistake. It reliably catches surface problems — missing sections, format violations, flat contradictions — and reliably misses the reasoning error that mattered. Use it as a cheap first filter, never as the last word: pair it with an executable check, a fresh-eyes reviewer in a different context, or a judge validated against human labels.

semantic search

The retrieval mode that lets "how do I get my money back?" find "reimbursement procedure". For agent memory it is how a system recalls the one relevant past episode out of thousands without knowing what to grep for. Similarity is not relevance, and certainly not truth: the near neighbours include the superseded policy and the note that contradicts it, which is why production retrieval layers keyword coverage, reranking, and freshness or provenance filters on top.

sequence diagram

Reach for one when the order is the point: protocol negotiation, a multi-hop handoff, or reconstructing an incident. Its weakness for agent architecture is that every participant gets an identical lane, so “code decides here, the model decides there” has nowhere to live — and on a physical whiteboard adding a participant means redrawing. The practical split worth remembering: shapes for architecture, lanes for time. The incident version is a sequence diagram with the axis rotated and side effects marked below the lanes.

telephone game

Each boundary is a lossy retelling: the parent omits a constraint it thought obvious, agent A paraphrases the goal slightly off, agent B executes the paraphrase perfectly. Loss compounds multiplicatively — five hops at 95% fidelity each preserve intent only about 77% of the time. It is worse than tool failure at the same rate because it fails silently, reporting success on the wrong work. Keep chains shallow and re-verify intent at every hop instead of assuming it survived.

telephone-game brief

The named coordination failure of multi-agent systems: the supervisor’s brief loses a constraint, and the worker does exactly what it was told — "cancelled all subscriptions as requested". Coordination failures ship success signals, so every per-agent health check stays green while the composite does damage. The defenses are structural: shallow chains, briefs that state constraints explicitly, approval gates on consequential actions, and cross-agent invariant checks instead of trust in the relay.

top-k

The knob deciding how much retrieved material reaches the model, and it interacts with everything: chunk size (bigger chunks, smaller k), reranking (fetch a wide k, keep a narrow one), and attention (a large k pushes the good passage toward the middle, where it gets under-weighted). Treat it as a measured number on your own eval set, not a default — and remember that whatever the first stage dropped is gone for the rest of the pipeline.

vector search

A vector index stores chunk embeddings and returns the closest ones for an embedded query, usually through approximate nearest-neighbour search that trades a little recall for a lot of speed. It is one retrieval strategy, not the definition of RAG: superb at paraphrase, weak at exact tokens like error codes, invoice numbers, and surnames. That weakness is why hybrid search plus reranking is the production default and a bare vector index is the prototype.

vendor lock-in

Lock-in here is sneakier than an API signature. Your prompts get tuned to a framework’s hidden scaffolding, your run state persists in its schema, your evals wire into its trace format, your tools speak its dialect. Swapping later means re-tuning behaviour, not renaming imports. So buy deliberately: keep prompts, tool contracts, and eval datasets in your own repo, prefer open boundaries (MCP for tools, OpenTelemetry for traces), and price the exit before you sign.

voting

The parallelization variant that exploits non-determinism: sample N times and take the majority, or demand unanimity before a high-stakes flag. It works when the runs genuinely differ — different prompts, framings, or models. N identical calls to one model at one temperature mostly buy correlated errors, several confident agreements on the same wrong answer, so vary something real and decide the aggregation rule (majority, unanimity, any-flag) before you build the merge.

MCP host

In MCP’s client–host–server architecture the host is the application the user actually runs — an IDE, a chat app, an agent runtime. It coordinates one client connection per server, aggregates their context, and is the only party holding the full conversation: the spec’s design principle is that a server can neither read the whole transcript nor see into other servers. That makes the host the security choke point — consent prompts, tool allowlists, and cross-server policy live there, because no server can be trusted to police itself.

MCP Specification 2026-07-28 — Architecture

MCP client

A protocol-level component, not a product: the host instantiates one client per MCP server and each client keeps a dedicated 1:1 connection to that server. Clients handle transport, capability discovery, and calls like tools/list and tools/call. The distinction matters when you debug: "the client is broken" usually means one connector out of six, and tool-name collisions between servers are the host’s problem to disambiguate because each client only ever sees its own server’s names.

MCP Specification 2026-07-28 — Architecture

MCP server

"Server" here describes the role, not the hosting: a local server is typically a subprocess your host launches over stdio and serves one client, while a remote server speaks Streamable HTTP and serves many. It offers three features to clients — resources (data), prompts (user-invoked templates), and tools (model-invoked functions) — and the controller differs for each. Installing one is a supply-chain decision: an MCP server is code running with your credentials, and its tool descriptions land in your model’s context.

MCP Specification 2026-07-28 — Architecture

JSON-RPC 2.0

A minimal remote-procedure-call convention over JSON — {"jsonrpc":"2.0","method":"tools/call","params":{…},"id":1} and a matching result or error object. MCP’s data layer is JSON-RPC 2.0 and the same message format works across every transport, which is why swapping stdio for HTTP changes your deployment and nothing about your handlers. A2A 1.0 defines JSON-RPC alongside gRPC and HTTP+JSON bindings, with a proto file as the normative definition — so JSON-RPC is A2A’s most common binding, not its foundation. Knowing the envelope pays off in debugging: a protocol error (unknown method, bad params) is a different failure class from a tool that ran and returned isError: true.

JSON-RPC 2.0 Specification; MCP Specification 2026-07-28

stdio transport

One of MCP’s two standard transports. The client spawns the server as a child process and they exchange newline-delimited, UTF-8 JSON-RPC messages over the standard streams — no ports, no TLS, no auth handshake. The authorization spec does not apply here; stdio servers take credentials from their environment instead, which is exactly why a local server inherits whatever your shell and config hand it. Write logs to stderr, never stdout: one stray print corrupts the frame and the connection dies.

MCP Specification 2026-07-28 — Transports

Streamable HTTP

The transport for remote MCP servers, introduced in revision 2025-03-26 to replace the older HTTP+SSE transport (deprecated since then — deprecated, not removed). The shape worth internalising: streaming is a property of an individual request, not a mode the connection is stuck in. As of the 2026-07-28 revision the standalone GET SSE stream and the Mcp-Session-Id header are gone, and servers MUST validate the Origin header — a local server that skips that check can be driven by any web page through DNS rebinding. Check the current spec before you build against transport details; this layer moves.

MCP Specification 2026-07-28 — Streamable HTTP Transport

elicitation

The client feature that lets a server say I need more from the user before I can continue: a deploy tool that needs the target environment asks, rather than inventing a plausible answer. Under revision 2026-07-28 delivery changed — servers never initiate requests, so the server returns an InputRequiredResult carrying inputRequests and the client retries with inputResponses (the Multi Round-Trip Requests pattern, SEP-2322). Treat it as an affordance for accuracy, not an authorization control: the host’s own consent gate is still what stands between a tool and a destructive action.

MCP Specification 2026-07-28 — Elicitation (SEP-2322)

tool annotations

Metadata a server attaches to a tool so hosts can render it honestly and route it through the right gate: is this read-only, is it destructive, is calling it twice the same as once. Tool annotations arrived with revision 2025-03-26, and the spec is blunt about their limits — clients MUST consider tool annotations untrusted unless they come from trusted servers. They are a risk vocabulary, so build your gates on the host’s own policy for that server, never on a flag the server itself supplied. Field names have shifted across revisions; check the current schema before quoting them.

MCP Specification 2026-07-28 — Tools

OAuth 2.1

MCP authorization implements a selected subset of the OAuth 2.1 IETF draft (draft-ietf-oauth-v2-1-13 — a draft, not a published RFC): a protected MCP server acts as an OAuth 2.1 resource server, the client acts as an OAuth 2.1 client, and discovery runs through Protected Resource Metadata (RFC 9728). Two facts save you a wrong design: authorization is OPTIONAL and defined for HTTP transports only — stdio servers take credentials from the environment instead — and as of 2026-07-28 Dynamic Client Registration is deprecated in favour of Client ID Metadata Documents. Check the current authorization spec before you implement; this section changes every revision.

MCP Specification 2026-07-28 — Authorization (OAuth 2.1 draft-ietf-oauth-v2-1-13)

resource indicators

The audience-binding control in MCP authorization. Clients MUST send the resource parameter on both authorization and token requests, and servers MUST validate that an access token was issued specifically for them as the intended audience — and MUST NOT accept or transit any other token. This is the defence against token passthrough and confused-deputy attacks: without it, a malicious or compromised server can collect a token minted for your real API and replay it. Bearer header only; tokens never go in the query string.

RFC 8707 (Resource Indicators for OAuth 2.0); MCP Authorization 2026-07-28

agent card

A2A’s discovery artifact: metadata an A2A server publishes so another agent can decide whether and how to talk to it. The canonical location is https://{domain}/.well-known/agent-card.json, following RFC 8615 well-known URI principles, and the card declares OpenAPI-style security schemes (API key, HTTP auth, OAuth 2, OIDC, mutual TLS) so authentication is negotiated from standard web primitives. The card is self-asserted, which is the engineering consequence: a skill list is a claim, not a guarantee, so treat a remote agent’s advertised capabilities the way you treat any untrusted input.

A2A Protocol Specification — Agent Card; A2A docs, Agent Discovery

supply chain

Agentic supply chain is its own risk category — LLM03 in the OWASP Top 10 for LLM Applications 2025 and ASI04 in the OWASP Top 10 for Agentic Applications — because an agent’s dependencies include things classic review never covered: tool descriptions that enter the model’s context, and MCP servers that can be updated after you approved them. The rug pull is the shape to remember: the npm package postmark-mcp behaved correctly for 15 versions, then version 1.0.16 added one line that BCC’d every email to the author’s own domain (disclosed by Koi Security, September 2025). Pin versions, review diffs, and treat "it worked yesterday" as unrelated to whether it is safe today.

OWASP Top 10 for LLM Applications 2025 (LLM03); OWASP Top 10 for Agentic Applications (ASI04)

session

Also called a thread or a conversation, depending on whose SDK you read. A session holds the ordered messages of one interaction and the id that joins them to traces, billing records, and the deletion request that arrives eight months later. The discipline is keeping it separate from execution: sessions accumulate and outlive processes; runs start, finish, and get retried. Vendor nouns churn — MCP removed protocol-level sessions entirely in revision 2026-07-28 — so name your own types after the concepts and keep each provider’s vocabulary behind one adapter.

run

The unit of agent execution: the agent wakes up, reasons, calls tools, produces output, and reaches a terminal status. The run id is what makes everything else possible — you cannot reconnect to, cancel, trace, or bill work that has no name. A run that needs something from outside (a client-side tool, a human approval) parks in a waiting state and resumes when the caller submits the missing result against that id; that one mechanism is how approval gates and long-running tools work underneath. Only terminal statuses are safe to report to a user.

transcript

What the session actually stores, and what the next run replays as history. Two questions decide your architecture: who holds it — you, or the platform — and how much of it gets resent into the context window each turn. Platform-held transcripts are convenient until you need retention control, tenant isolation, or a deletion guarantee; self-held transcripts are work you own forever. It is also a security surface: a transcript is the highest-value private data an agent touches, and any tool that can read it feeds one leg of the lethal trifecta.

backpressure

A 429, a rising queue depth, or a climbing oldest-message age is capacity information, not a random failure. The agent-specific bite is duration: a run holds resources for minutes, so a small arrival surge becomes an enormous backlog before any dashboard looks alarming. The response is admission control — refuse or defer new work at the edge and reduce concurrency, rather than retrying the same load at the same parallelism, which turns a soft limit into a hard one. Shed load visibly; an unbounded queue does not remove backpressure, it converts it into latency you cannot see.

JSON Schema

The same schema does two jobs at once, and that is the thing to internalise: it teaches the model what a valid call looks like and gives your runtime something to validate against. Every keyword you omit — an enum, a pattern, required, a description on a field — is a guess you have handed to a non-deterministic system. MCP tool input schemas default to draft 2020-12 when no $schema is present, and provider strict-schema modes each support a subset, so exotic oneOf, unbounded regexes, and recursive refs are the usual casualties. Design the schema so illegal calls are unrepresentable, not merely rejected.

JSON Schema 2020-12; MCP Specification 2026-07-28 — Tools

JSON mode

The middle rung of the structured-output ladder: generation is constrained to syntactically valid JSON, so no prose preamble, no code fence, no trailing apology. What it does not promise is your contract — {"answer": "p1 maybe"} is impeccable JSON and useless to your parser. Treat JSON mode as syntax insurance, never as a schema: it removes a whole class of noise, and it leaves every key, type, and required field entirely up to the model. Flag names and availability vary by provider as of September 2026; check the current docs.

constrained decoding

The strongest rung of the structured-output ladder: non-conforming output is not corrected afterwards, it is unrepresentable. Shipped as provider "strict" schema flags and as grammar-constrained samplers in local inference stacks. Two honest caveats survive it: each implementation supports only a subset of JSON Schema, and a run that hits its token limit still ends mid-document — truncated JSON parses no better for having been constrained. So you still validate at the boundary; constrained decoding shrinks the failure set, it does not empty it.

schema validation

The rule is parse, never trust: something already turned tokens into a structure, and nothing has yet checked that the structure is legal — or permitted. Validate arguments at the executor boundary even under constrained decoding, because truncation, refusals, and unsupported schema features all still produce garbage. Keep two checks distinct: validation answers is this a well-formed call, authorization answers is this caller allowed to make it. A path that satisfies your regex can still point at /etc/shadow.

tool choice

The cheapest fix for a whole class of agent misbehaviour, and the one most teams never touch. Four modes recur under different spellings: auto (prose or tools, the model decides), required/any (it must call something), a named tool (it must call that one), and none (prose only). Forcing a call is how you stop an agent narrating instead of acting; forbidding calls is how you make the final summarisation turn safe. The trap in forced modes is that the model must call something even when the honest answer is "not enough information".

parallel tool calls

Models routinely emit multiple calls in a single turn, and that is a latency win: three independent lookups run concurrently instead of costing three round trips. It is also a correctness hazard. Every requested call must come back with a result tagged to its id before the next model turn, or the API rejects the message. And "independent" is your judgement, not the model’s: two writes to the same record are not safely concurrent, so gate non-idempotent tools and serialise them yourself.

streaming tool calls

Streaming a tool call is stranger than streaming prose — for most of the stream the argument JSON is not valid JSON. The name usually arrives whole and early, which is a free two-second win in perceived latency (render "Searching orders…" immediately). The rule that matters: buffer fragments per call id and dispatch only on the completion event, never on "the JSON happens to parse now", which can be true halfway through a nested object. Do that and cancelling mid-stream causes no side effects at all — a real safety property you get purely from where the dispatch sits.

tool-use id

The join key of the agent loop. The model emits a call with an id; your result must come back carrying that exact id, because with parallel calls position tells the model nothing and a missing or mismatched id is a hard API error rather than a soft degradation. It is also the anchor for everything you build around a call: the trace span, the approval record, the idempotency key, and the audit line that says which specific request a human signed off on.

tool retrieval

RAG with tools in place of documents, adopted when a catalog is too large or too volatile to hand-maintain. The arithmetic is real: 200 contracts riding on every call becomes 8. The cost is a silent failure mode — when the right tool is not in the top k, nothing errors; the model reasons competently over a menu that lacks the correct move, which is indistinguishable from a model that had the tool and chose badly. So: recall@k is the governing metric (not precision), a floor set of always-present tools cannot be filtered away, hybrid retrieval beats pure semantic because tool names are short exact strings, and the exposed tool set must be logged on every call or every selection bug is unfalsifiable.

tool sprawl

Nobody writes forty tools; a host connects to nine servers that each export a handful. Sprawl expands three surfaces at once. Routing: every additional near-neighbour makes the model’s choice harder. Context: unused contracts are paid for on every call. Prompt security: each connected server contributes names and descriptions that land in your context as instructions, so every new server is a new party with write access to your prompt — the mechanism behind description poisoning and rug pulls. The distinguishing test against genuine scale: have you deleted or merged anything in the last quarter? An unpruned catalog is sprawl, and every mechanism you layer on top will faithfully preserve its duplicates while adding a failure mode.

progressive disclosure

The interface-design idea applied to tool catalogs: a handful of always-present tools, plus two meta-tools — one that lists capabilities in a domain, one that invokes by name. Context cost becomes roughly constant in catalog size, and the hot tools stay in the cacheable prompt prefix. Its costs are specific and worth stating: an extra round trip before any work begins, weaker argument validation at the outer boundary (the invoke tool takes a name and a payload, so it cannot be schema-checked as tightly as a first-class contract), and models that under-use discovery — guessing a plausible name instead of listing first. Mitigate the last one by prompting for discovery explicitly and making a failed invoke return the available names rather than a bare error. Best fit: a clear stable core with a long rarely-used tail.

agent identity

A principal in your directory that represents an agent, so its actions can be authorized and audited as its own — not laundered through a shared service account or a developer’s credentials. Every managed platform now ships some version of it: AWS implements agent identities as workload identities with a distinct ARN in a central directory, and Microsoft Foundry gives each hosted agent a dedicated Entra identity. This is the least portable layer you will build: the identity plane is wired to one directory and one token-exchange model, which is why it usually decides the platform rather than the reverse.

Entra Agent ID

Microsoft Entra Agent ID extends Entra to agents. An agent identity has an object ID, a display name, and an optional human sponsor — a named person or group accountable for it — and it holds no credentials of its own: the agent identity blueprint it was created from acquires tokens on its behalf. It speaks OAuth 2.0, MCP, and A2A, and reaches non-Microsoft platforms through workload identity federation or an auth sidecar. Launch stages here move: as of September 2026 at least one Entra Agent ID page still carried a preview banner, so check the current status before you write "GA" in a design doc.

Microsoft Learn — What is Microsoft Entra Agent ID?

workload identity

The non-human end of identity: a principal for a process, container, or agent, authenticated by what it is rather than by a secret someone typed. AgentCore implements agent identities as workload identities with specialized attributes, and exchanges a validated user JWT for a workload access token so the agent can act downstream with brokered credentials. The engineering payoff shows up at incident time: two agents serving two users produce four distinct authority contexts, each traceable to a specific workload identity instead of one shared role you cannot untangle.

managed identity

The Azure answer to "where do we keep the credential": the platform creates and rotates an Entra identity bound to a resource, and your code asks for a token instead of holding a key. For agents it is one of several documented auth options on Foundry tool and MCP connections — alongside a user Entra token and the agent’s own identity — and the choice matters: a project managed identity means every agent in the project shares that authority, while an agent identity keeps authorization and audit per agent. Prefer the narrowest principal that still works.

on-behalf-of (OBO)

Delegated authority. When a user token is present, the agent trades it for a downstream token in which the subject is the user and the actor is the agent — so the agent reaches exactly what that user may reach, and the audit log records both parties. Without a user token the agent falls back to acting as itself, with whatever standing permissions it holds. The distinction is the whole security argument: OBO makes an over-privileged agent harmless for a low-privileged user, while app-token paths need their own scoping because there is no user ceiling to inherit.

OAuth 2.0 On-Behalf-Of flow (Microsoft Entra)

RBAC (role-based access control)

Permissions are attached to named roles, and principals get roles scoped to a resource, group, or subscription. An agent identity is worthless as a control until you assign it roles: give the agent’s principal the narrowest role at the narrowest scope that still lets the task finish, and per-agent audit becomes per-agent accountability. The common failure is scope, not role — a reader role at subscription scope reads everything you own.

managed runtime

The layer that runs your agent process for you — session lifecycle, scale-to-zero, per-session isolation, identity injection, and telemetry — so you are not operating a container fleet to run a loop. All three major platforms host popular frameworks, which makes the loop the portable part: what you gain is operational leverage, and what you owe is a careful read of the session model, because the timeouts and isolation guarantees differ per platform and per substrate.

session isolation

The containment property that makes it safe to run model-chosen code on shared infrastructure. AWS documents a dedicated microVM per session with isolated CPU, memory, and filesystem, destroyed and sanitized at termination to prevent cross-session contamination; Foundry documents a per-session VM-isolated sandbox whose filesystem persists so an idle session can resume statefully. Those are different guarantees, and the difference matters: ask each platform what the boundary actually is rather than assuming the phrase means the same thing everywhere.

Amazon Bedrock AgentCore Developer Guide — How AgentCore Runtime works

tool gateway

A gateway sits between the agent and its tools and centralizes four jobs: ingress auth (who may call), egress auth (which credential reaches the backend), discovery, and policy enforcement. AgentCore Gateway converts APIs, Lambda functions, OpenAPI schemas, and remote MCP servers into MCP tools behind one endpoint, and Foundry Toolbox groups curated tools behind a single managed MCP-compatible endpoint with centralized credential injection and versioning. The trap is ownership: registering tools in a gateway makes the gateway the source of truth, so keep the tool contracts in your repo and treat the gateway as a deployment target.

undifferentiated heavy lifting

The plumbing that is identical across everyone solving your problem: session hosting, credential brokering, trace collection, sandbox isolation. None of it is your product. The discipline is to sort each layer honestly into yours (the prompt, the tool contracts, the golden dataset, the definition of done), theirs (isolation, token vaults, span plumbing), and boundary decisions where buying quietly hands over your data model — memory record shapes and policy languages sit there. Getting that sort wrong in either direction is expensive: build the plumbing and you burn quarters, buy the boundary layers unexamined and you cannot leave.

agent versioning

An agent is a configuration, and configurations need release engineering. Foundry snapshots agent versions automatically as you iterate, serves an administrator-selected active version behind a stable endpoint, and supports version-selector rules that split traffic by percentage across versions. That is what makes a prompt change reviewable rather than ambient: you can canary it, compare it against the previous version on the same task set, and revert without redeploying the callers. An agent you cannot name a version of is an agent you cannot roll back.

private endpoint

The network half of a boundary requirement: the service gets an address in your virtual network, resolved through private DNS, and public access can be switched off. For Foundry agents this is not a late-stage toggle — bring-your-own network requires a delegated subnet and private endpoints, and network injection must be set when the account is created and cannot be added or changed afterward. It also has consequences downstream: because Microsoft 365 cannot reach private endpoints, publishing an agent to Copilot or Teams from a private project needs an explicit, source-IP-filtered public route for just that endpoint.

customer-managed keys

You supply the key in your own key-management service; the platform encrypts your data with it. The technical delta over platform-managed encryption is modest — the governance delta is the point: you hold the audit trail, the rotation schedule, and the ability to revoke. Availability is per service and per configuration tier, not per cloud: Foundry supports it on Standard setups, Google lists CMEK as supported for most Agent Platform services but not all, and AWS uses KMS customer-managed keys for the AgentCore Identity token vault and Agent Registry. Check the current support matrix per component, because one unsupported component can sink a compliance claim.

data residency

A contractual or regulatory constraint on where data lives, and for agents it binds far more surfaces than teams expect: conversation history, extracted long-term memories, uploaded files, vector indexes, and the trace store all contain customer data. Treat it as a disqualifying constraint you evaluate first — it is binary, it is per component, and no amount of preference outranks it. Standard Foundry setups keep agent data in your own storage, search, and database resources for exactly this reason; on other platforms, map every component before you promise a region.

data gravity

Agents are mostly retrieval plus judgment, so the corpus they read is the heaviest thing in the design. Moving it costs egress, time, a sync story, and a second copy to secure — which is why the cloud your primary corpus already lives in usually wins the platform decision, ahead of SDK preference and ahead of rate cards. The useful test is not "where is our data" but "where is the data this agent actually needs, and what breaks if it is a replica".

OTel (OpenTelemetry)

The instrumentation standard agent observability converged on: spans with agreed names and attributes, plus W3C trace-context propagation across hops. It is the closest thing to a portability guarantee in this whole stack — AgentCore emits OTel-format telemetry to CloudWatch and Foundry stores agent traces in Application Insights using the OTel generative-AI semantic conventions, so instrumenting in OTel keeps your dashboards and trace-based evals from being welded to one backend. Verify the wire format per platform before assuming a collector can read it; not every documented path is OTLP.

OpenTelemetry semantic conventions for generative AI

Application Insights

Connect an Application Insights resource to a Foundry project and agent traces land there — viewable in the portal’s Traces view or in Azure Monitor, and queryable alongside the rest of your Azure telemetry. Two details change how you use it: capturing chat message content is opt-in, because prompts and tool results routinely contain personal data; and agents hosted outside Foundry can be registered — a preview capability as of September 2026 — so their spans, tagged with a matching agent id, feed the same dashboards and trace-based evaluations.

Microsoft Learn — Set up tracing in Microsoft Foundry

Prompt Shields

Two detectors at two intervention points. User prompt attacks (jailbreaks) are scanned at user input; document attacks — hidden instructions embedded in third-party content, i.e. indirect injection — are scanned at both user input and the tool-response point, which is the one that matters for agents, since tool output is where retrieved content enters context. Responses come back as annotations with detected and filtered flags. Like any classifier it has a false-negative rate, so it bounds the known-bad and never replaces least privilege and egress control.

Microsoft Learn — Prompt Shields in Microsoft Foundry

XPIA (cross-prompt injection attack)

The attack where the payload arrives in data — a document, a web page, a calendar invite, an MCP server’s response — and the model treats it as instruction because it is all one token stream. It is the dominant agent threat precisely because the attacker never needs to talk to your user. Defence is layered: scan at the tool-response intervention point, mark third-party content as untrusted, and then assume some of it gets through, which is why the containment layers below it exist. A filter reduces frequency; only removed capability changes the worst case.

cost per resolved task

Cost per run flatters a cheap agent that fails often: the retries, the escalations, and the human who cleans up afterwards are all real spend that per-run accounting hides. Measure on a frozen golden set with a matched model class, count every attempt, exclude free tiers, and report p50 and p95 latency next to the number. Done that way it is the one cost comparison that means anything across platforms — because published compute rates sit in the same order of magnitude and the dominant line on an agent bill is usually inference plus retrieval, not the runtime.

lock-in

Not a yes/no property of a platform but a map of what you would have to rebuild. The loop is the cheap part: every platform hosts the same popular frameworks. Stickiness accumulates in four seams — the shape of your extracted long-term memories, the gateway holding your tool registrations and backend credentials, the language your policy rules are written in, and the trace schema every dashboard and trace-based eval was built on. Keep the loop, the tool contracts, and the golden set in your repo, instrument in OTel, and write down the conditions that would make you revisit the decision.

indirect prompt injection

The attacker never talks to your agent. They plant instructions where the agent will later read them, and the model treats retrieved text as part of the same instruction stream as your system prompt. This is what makes zero-click attacks possible: EchoLeak needed only an email in the victim’s mailbox, ShadowLeak only an email the Deep Research agent would open. The engineering consequence is blunt — every retrieval path is an untrusted input boundary, and "we do not accept user input here" is never a defence.

Greshake et al., "Not what you’ve signed up for" (arXiv:2302.12173, 2023)

excessive agency

The reach half of the agent threat model. Injection supplies the intent; excessive agency supplies the blast radius — the agent held org-wide scope, the tool accepted any recipient, nobody capped the spend. Unlike injection it is not an attack but a design defect, which means you can audit for it before anyone exploits it: enumerate the tools, the credentials they hold, and the network they reach. Remediation is subtraction, not more prompt instructions.

OWASP Top 10 for LLM Applications 2025 — LLM06:2025 Excessive Agency

exfiltration

The third leg of the lethal trifecta in action. In the published agent disclosures the channel is almost never malware: it is a markdown image the client auto-fetches, a "click to reauthenticate" link, a domain still sitting on someone’s CSP allowlist, or a pull-request body. The exfil channel is usually your own renderer or your own allowlist, which is why egress control and output sanitisation cut more chains than any detection filter — and why an outbound path counts as a capability when you review an agent.

confused deputy

The classic access-control failure: the deputy holds real authority and is talked into spending it on someone else’s behalf. Agents make it routine, because their instructions arrive as text and they read text from places you do not control — tickets, web pages, retrieved documents, tool results. Prompt injection is a confused-deputy attack delivered through content. The fix is never a smarter deputy: it is narrow, per-request authority, so the authority the deputy can be confused into spending is not worth stealing.

tool-description poisoning

A tool’s description is the only documentation the model ever reads, so whoever writes it steers the agent. The MCP specification is explicit that tool descriptions and annotations should be treated as untrusted unless they come from a trusted server — they are data from a third party, not part of your prompt. The operational consequence: review descriptions and schemas at approval time, hash them, and diff them on every version bump, because nothing in the protocol stops a server changing its description tomorrow.

MCP Specification 2026-07-28 — Security and Trust & Safety

rug pull

The supply-chain attack that defeats one-time review. The npm package postmark-mcp worked legitimately for fifteen versions; version 1.0.16 added a single line that BCC’d every email sent through the server to the developer’s own domain (Koi Security, 25 September 2025). Publisher verification does not help here — the same verified publisher ships the poisoned version. What helps is pinning an exact version or digest, hashing tool names, descriptions and schemas at approval time, and treating a version bump as a change that needs re-review.

supply-chain attack

OWASP lists it twice for good reason: LLM03:2025 Supply Chain in the LLM Top 10 and ASI04 Agentic Supply Chain Vulnerabilities in the Top 10 for Agentic Applications. Agents raise the stakes because installed code inherits the agent’s credentials and its reach: the Nx s1ngularity packages (August 2025) even used the AI CLIs already on the machine to help with reconnaissance, per Wiz’s analysis. An agent runtime’s dependency list is a permission list — review it like one, and pin what you approve.

OWASP Top 10 for LLM Applications 2025 — LLM03:2025 Supply Chain

computer use

The capability that lets an agent operate software with no API: it looks at the screen and moves the mouse. The screen becomes the API, and the whole desktop becomes the permission surface — there is no parameter schema to constrain, no allowlist of endpoints, and every pixel of rendered content is a potential injection channel. Which is why computer use belongs in a disposable, credential-free environment with controlled egress, and why the session recording matters as much as the result.

containment

The strategic move that agent security turns on. Prompt injection cannot be reliably detected, so the design assumption becomes assume the model can be turned, and make sure a turned model cannot do much: read-only defaults, parameter constraints enforced in code, scoped credentials, egress allowlists, sandboxes, per-tool rate limits, approval gates on the irreversible calls. Detection still earns its place — it buys you signal and time — but containment is what holds when detection misses.

defense in depth

The old principle with a sharp new test for agents: layers only count as layers if they fail differently. Prompt hardening is advisory, guardrail classifiers are probabilistic, tool-boundary constraints and credential scopes are deterministic, a sandbox is physical, a human gate is out-of-band. Stack three probabilistic text filters and you have one layer with a nicer dashboard; stack a filter, a scoped token, and an egress allowlist and you have three genuine chances to survive the same injection.

rule of two

Published by Meta on 31 October 2025 as the "Agents Rule of Two": until prompt injection can be reliably detected and refused, an agent session should satisfy no more than two of [A] processes untrustworthy inputs, [B] has access to sensitive systems or private data, [C] can change state or communicate externally. If a task genuinely needs all three, it should not run autonomously — it needs human-in-the-loop approval or another dependable validation mechanism. It maps closely onto the lethal trifecta, reframed from a diagnosis into a design budget you can enforce per session.

Meta AI, "Agents Rule of Two: A Practical Approach to AI Agent Security" (31 October 2025)

output sanitization

The cheapest control in the catalog and the one whose absence produced the largest run of published agent data-leak disclosures. Markdown and HTML contain constructs that make the rendering client fetch a URL automatically, so if an agent can be induced to put attacker-chosen text into that URL, displaying the answer performs the exfiltration — no click, no download. GitHub’s eventual fix for CamoLeak was to disable image rendering in Copilot Chat entirely, which tells you how hard the channel is to sanitise halfway.

OWASP Top 10 for LLM Applications 2025 — LLM05:2025 Improper Output Handling

permission surface

Not what the system prompt says the agent may do — the union of everything its registered tools can reach. Teams routinely audit the prompt and never audit the surface, which is how an agent described as "read-only summariser" turns out to hold a token with write scope on the whole workspace. Reviewing it is mechanical: list the tools, the credential behind each, the resources that credential opens, and the endpoints the runtime can dial. That list, not the prompt, is your threat model.

rate limit

Agents fail at machine speed and in volume: the difference between a bad email and a reputational event is a thousand iterations nobody stopped. Per-tool, per-run, and per-tenant caps make the worst case arithmetic instead of a story — twenty messages, not twenty thousand. Enforce them in the runtime or gateway, never as an instruction in the prompt, and alert on the cap being hit, because a tripped limit is one of the earliest honest signals that a loop has gone wrong.

scoped credentials

Standing ambient authority is what turns a single injection into a tenant-wide incident, because whatever the credential can reach, injected instructions can reach too. Scoping inverts the default: the credential exists for this run, this user, this resource, and expires before the incident review starts. In practice that means issuing per task rather than per service, letting the scope be the intersection of the agent’s rights and the caller’s, and treating any long-lived broad token in an agent as a finding.

delegation

The authorization model agents actually need. A delegated token names two parties — subject = who this is for, actor = who is doing it — which is what lets you cap the agent below the user’s own rights, revoke the agent without disabling the user, and answer "was that Dana or Dana’s agent?" six months later. OAuth 2.0 calls the mechanism token exchange; Microsoft calls the pattern On-Behalf-Of. Collapse it into passing the raw user token through and it still works on day one, having destroyed attribution forever.

OAuth 2.0 token exchange — subject and actor claims

impersonation

The shortcut that looks like delegation and is not: forward the raw user token and every downstream log records the human. You lose three things at once — the ability to scope the agent below the user, the ability to revoke the agent without disabling the person, and the ability to prove afterwards who actually acted. The log will blame a human for an agent’s action, which is exactly the repudiation failure OWASP names as Identity Spoofing and Impersonation in its agentic threat taxonomy.

OWASP Agentic AI — Threats and Mitigations (Identity Spoofing & Impersonation)

service principal

A per-agent principal is the precondition for everything else in agent security: you cannot scope, revoke, or attribute what has no identity of its own. Share one API key across five agents and a permission grant is a grant to all five, a revocation is an outage for all five, and the audit log cannot tell them apart. Microsoft Entra Agent ID models agent identities as a special kind of service principal; AWS and Google express the same idea as workload identities with their own ARNs and IAM identities (checked September 2026 — verify current naming in the vendor docs).

Microsoft Learn — Agent identities in Microsoft Entra Agent ID (checked 2026-09)

downscoped token

The per-call discipline that makes delegation safe. You exchange the broad inbound token for the narrowest one that completes this call: one API as the audience, the scopes that call needs, a lifetime measured in minutes, and never more than the intersection of the agent’s rights and the user’s. A broad token handed to a tool is a broad token available to whatever text talks the agent into using it — downscoping is how you stop an injected instruction inheriting the whole session’s authority.

short-lived credential

Agents leak credentials in ways humans do not: into traces, logs, summaries, memory stores, and the context window itself. Short TTLs are the control that survives the leak — the token expires faster than the incident escalates — and audience restriction is its partner, so a leaked token works against one API rather than everything the agent can see. The operational cost is a refresh path; the alternative is a long-lived secret sitting in an artefact you cannot fully enumerate.

secrets management

The rule for agents is one line: the agent references a credential, it never holds one. Anything that enters the context window can be summarised, logged, echoed into an answer, or exfiltrated by an injection, so a secret in the prompt is a secret already halfway out. In practice a vault holds the material and a gateway or tool wrapper attaches it at the boundary — AWS describes AgentCore Identity’s token vault storing OAuth tokens, API keys and client secrets encrypted with KMS keys, and the same pattern assembles from your existing secret manager (checked September 2026).

AWS Security Blog — Securing AI agents with Amazon Bedrock AgentCore Identity (checked 2026-09)

audit trail

An agent audit trail has to answer a question ordinary logs never had to: which identity acted, and on whose behalf. That means recording the actor (the agent principal), the subject (the human authority), the exact tool call and parameters, the approval if there was one, and the outcome — append-only, retained on a schedule you chose deliberately. Get it wrong and you hit what OWASP calls Repudiation and Untraceability: an incident nobody can reconstruct, and a change nobody can be held to.

OWASP Agentic AI — Threats and Mitigations (Repudiation & Untraceability)

non-repudiation

The property agents break by default. Shared API keys, forwarded user tokens, and one service account for the whole fleet all produce logs in which any actor can be argued to be any other actor — and "the agent did it" becomes unfalsifiable in both directions. Restoring it needs three things together: a distinct identity per agent, delegation that preserves the actor claim end to end, and an append-only record with retention that outlasts your dispute window.

OWASP Agentic AI — Threats and Mitigations (Repudiation & Untraceability)

trust boundary

The single most useful line to draw on an agent architecture. Everything inside runs under your identity, in your network, with your assumptions; everything crossing it is either untrusted content coming in or your data going out. A boundary without a named mechanism is a wish: “inside” means nothing unless you can say identity, network, or sandbox — which of the three actually holds it. Agents make boundaries unusually hard to place because the model reads content from outside and then chooses actions inside, so the boundary runs through the loop rather than around it. Every crossing is where least privilege, egress control, and output sanitization have to live.

alert fatigue

Every agent signal is noisy: pass rates wobble run to run, judges disagree with themselves, latency spikes because an upstream API had a bad afternoon. Alert on that raw noise and you teach your on-call to dismiss the page without opening the trace — which is worse than having no alert, because now you also believe you are covered. The fix is to alert on sustained, action-linked changes: a rolling window that breaches a threshold, attached to a named response. If nobody can say what they would do when it fires, it is not an alert, it is a dashboard.

batch inference

Providers sell the same tokens far cheaper when you let them schedule the work: you upload a batch of requests, wait, and collect results later. For agents this is the wrong tool for the live loop and the right tool for everything around it — backfilling evals, re-grading a golden dataset, bulk classification, nightly summarization. The design consequence is architectural: if a step does not need an answer while a human waits, it does not belong on the interactive path at interactive prices. Discounts and turnaround windows are vendor specifics that change, so check the current rate card before you build a budget on them.

blind holdout

When you tune prompts against a visible eval set, you eventually optimize for that set — the agent-era version of overfitting, done by hand. A blind holdout is a reserved slice of cases that nobody prompts against and nobody reads failures from during iteration; you run it rarely, at release gates. A gap between your visible score and your holdout score is the measurement of how much you have fooled yourself. Keep it locked, keep it representative, and rotate cases in only when you can afford to burn them.

canary release

The standard way to ship a change you cannot fully test offline: route 1–5% of traffic to the new prompt, model, or tool set, compare it against the incumbent on live signals, then ramp or roll back. Agents make this mandatory rather than nice-to-have, because offline evals cannot cover the input distribution production actually sends you. Two rules make a canary real: pre-declare the metrics and the abort threshold before you start, and wire the rollback so it is one action, not an emergency deploy.

cost per successful run

Take every dollar spent in a window, including retries, abandoned runs, and the runs a human had to redo, and divide by the count of runs that reached a correct outcome. This metric is honest in the way average cost per call is not: an optimization that halves token spend while dropping the pass rate makes this number worse, and the naive metric better. It is also the number to put next to the manual cost of the same task, which is the comparison anyone funding the agent is actually making.

cost per task

The unit of economics for agents is the task, not the token or the request, because a single task may fan out into a dozen model calls and forty tool calls. Instrument it at the trace level and you can immediately answer the questions that matter: which task types are unaffordable, which tools are cost hotspots, and whether the cost distribution has a long tail of runaway runs hiding behind a comfortable average. Watch the p95, not the mean — the mean is set by easy tasks and hides the loop that ran ninety turns.

distribution shift

Your golden dataset froze a snapshot of the traffic you had when you wrote it. Real usage moves: new customer segments, new document formats, a marketing campaign that sends questions nobody anticipated. A green suite over a stale distribution is not evidence that the agent works, only that it still handles last quarter. The counter-practice is a feedback loop — sample real traffic periodically, diff it against the categories in your suite, and promote what is missing into cases.

drift

The umbrella failure of agents in production: nothing in the repo changed and behaviour changed anyway. Three sources, and you need to be able to tell them apart: input drift (users ask new things), model drift (the provider updated the endpoint behind your model name), and prompt or memory drift (accumulated edits and stale stored context). Detection needs a fixed reference — a pinned model plus an unchanged eval set — because you cannot measure movement without something that is not moving.

flake budget

Agent evals are statistical: run the same suite twice with no changes and the score moves. A flake budget is the explicit answer to "how much movement is normal?" — measured by running your suite several times against an unchanged agent and recording the spread. Anything inside that band is noise and must not gate a deploy; anything outside it is a signal and must. Without one, teams oscillate between chasing phantom regressions and rubber-stamping real ones, and either way they stop trusting the suite.

graceful degradation

Agents depend on services that time out, rate-limit, and occasionally go down, and on budgets that genuinely run out mid-run. Graceful degradation is deciding in advance what the reduced-service version looks like: fall back to a smaller model, serve cached or partial results, skip the enrichment tool, or stop and hand the task to a human with everything gathered so far. The failure mode to design against is the confident one — an agent that silently proceeds without the tool result and invents the answer is far worse than one that says it could not finish.

ground truth

The reference a grader scores against: the expected answer, the expected tool trajectory, or the assertion that must hold. Its quality caps the quality of every metric built on it, and it has a shelf life — policies change, prices change, APIs change, and yesterday’s correct answer becomes today’s false failure. Treat each expected answer as an artifact with an owner and a verification date, and note the structural consequence: evaluators that need ground truth can only run against curated sets, which is why live-traffic monitoring must use reference-free measures instead.

inter-rater agreement

Measured between two humans, it tells you whether your rubric is well defined: if two competent reviewers disagree half the time, the rubric is ambiguous and no model will fix that. Measured between a human and your judge on the same sample, it tells you whether the judge can be trusted to scale. Report a chance-corrected statistic such as Cohen’s kappa rather than raw percent agreement, because on a skewed set two graders who both say "pass" for everything look 90% aligned and have measured nothing. Re-run it after every rubric or judge-model change.

judge drift

A judge is an instrument, and an uncalibrated instrument invalidates the history it produced. Scores shift when the judge model is updated behind an alias, when someone reworded the rubric, or when the examples in the judge prompt changed. Defend the comparability of your metrics: pin the judge model version, version the rubric like source code, and keep a small human-graded calibration set you re-run on every judge change. If yesterday’s 0.82 and today’s 0.79 came from different judges, you have not measured a regression — you have measured two different rulers.

loop depth

Count the iterations of the agent loop per run and plot the distribution. Healthy work clusters in a narrow band; the tail is where your incidents live — retry storms, two tools undoing each other, a model re-reading the same file because a result came back empty. Loop depth is worth alerting on precisely because it is cheap to compute and correlates with everything expensive: cost, latency, and context exhaustion all rise together with it.

model pinning

Aliases like "latest" are a convenience that hands a third party write access to your agent’s behaviour. Pin the dated or numbered version, treat a version bump as a code change with its own pull request, and run your regression suite against the new version before it reaches users — then keep the old pin available so rollback is a config revert. The same discipline applies to judge models, embedding models, and any framework or MCP server in the loop: unpinned dependencies turn "we changed nothing" into a sentence you cannot verify.

model routing

Most steps in an agent run are not hard: classifying an intent, extracting a field, summarizing a tool result, deciding whether to continue. Route those to a small model and reserve the frontier model for planning and synthesis, and cost falls by a large factor with no user-visible quality loss — if you gate each route with its own eval set. Routing without per-route measurement is how a silent quality regression enters a system nobody thinks they changed. The router itself must be cheap and deterministic where possible; a model call to decide which model to call is a tax.

observability

Monitoring tells you a number crossed a line; observability lets you ask "why did this run cost nine dollars and end in the wrong refund?" and get an answer without shipping new code. For agents the primary artifact is the trace — the full tree of model calls, tool calls, arguments, results, tokens, and timings — because the interesting failures are compositional and invisible in aggregate metrics. Instrument for it from the first prototype: retrofitting traces onto a live agent is the most common reason teams cannot debug the incident they are currently having.

online evaluation

Offline evals measure the cases you thought of; online evaluation measures what users actually sent this morning. It works by sampling real traces at some rate and scoring them, which imposes one hard constraint: live traffic has no expected answer, so online evaluation can only use reference-free evaluators — groundedness, tool-selection sanity, goal-success judgements, refusal rates. Anything needing ground truth stays in the offline suite. That split is the load-bearing design decision: CI proves you did not regress, online evaluation tells you what reality is doing to you.

OpenTelemetry

The instrumentation standard most agent observability tooling now speaks: you emit OTel spans and any compliant backend can ingest them, which keeps your traces portable across vendors. Its generative-AI semantic conventions name the spans and attributes agents need — model calls, invoke agent, plan, execute tool — and as of September 2026 those conventions carry Status: Development, not stable, and live in a dedicated open-telemetry/semantic-conventions-genai repository. Instrument through OTel anyway, but expect attribute names to move and check the current spec before you build dashboards on exact field names.

OpenTelemetry GenAI semantic conventions (Status: Development, checked 2026-09)

outcome eval

The first eval you should write, because it measures what the user cares about and is often checkable deterministically: was the refund $40, does the file compile, was the record created with these fields. Outcome evals are cheap, stable, and hard to argue with. Their blind spot is the reason trajectory evals exist: an agent can reach the right answer through eight wasteful or dangerous steps, and an outcome eval will award it full marks. Start with outcomes, add trajectory checks where the path itself carries risk or cost.

position bias

Show a judge two answers and it will systematically prefer one slot. Left uncontrolled, this means your pairwise comparisons partly measure the order you happened to write the prompt in. The standard control is cheap: run every comparison both ways and keep only the verdicts that survive the swap, treating flips as ties. The cost is a doubled judge bill; the return is comparisons that mean something, which is the whole point of running them.

prefill

Inference has two phases with different economics: prefill processes all input tokens in parallel, then decode emits output tokens one at a time. Prefill drives your time-to-first-token and scales with prompt size, which is why a bloated context hurts latency before it hurts your bill, and why prompt caching — reusing the computed prefix — is the highest-leverage latency fix available to an agent that resends a growing history every turn. Decode, by contrast, is bound by output length, so cutting verbosity is the other lever.

redaction

Agent traces are the richest debugging artifact you have and, for the same reason, the most dangerous data you hold: full prompts, tool arguments, retrieved documents, and API responses, retained for months in a third-party observability backend. Redact at the instrumentation boundary so the sensitive value never reaches storage in the first place — masking after ingestion is a cleanup, not a control. Then face the tradeoff honestly: over-redact and the trace is useless for debugging, so redact values while keeping shapes, lengths, and hashes that still let you reason about what happened.

regression suite

A curated, versioned set of cases plus graders, wired into CI so that prompt edits, tool changes, and model bumps all have to pass it. It differs from a unit-test suite in two ways that shape everything: results are statistical, so a threshold plus a flake budget replaces a binary assertion, and it costs real money to run, so you tier it — a fast subset per pull request, the full suite plus blind holdout at release. It grows the only way that works: every production failure becomes a case before the fix is merged.

resend tax

The model remembers nothing between calls, so turn 20 ships the system prompt, the full history, and every tool result accumulated so far. Input cost therefore grows quadratically with loop depth, not linearly — the single most under-estimated line item in agent budgets, and the reason one chatty tool result is a recurring charge rather than a one-off. The three fixes attack it directly: cache the stable prefix, compact tool results at the source, and summarize or drop completed work instead of carrying it forever.

rubric

A judge is only as good as the instructions it grades against, and most judge failures are rubric failures. Three rules do most of the work: score one quality per rubric (bundling "helpful and accurate and safe" into one number produces a number that means nothing), anchor every level to something observable ("cites a source for each factual claim", not "good grounding"), and require the judge to quote the evidence for its score so a human can audit the verdict. Version rubrics in the repo — a silently edited rubric invalidates every score before it.

self-preference

Models tend to favour text that looks like text they would produce — the same phrasing habits, structure, and hedging patterns. That makes a same-family judge a poor referee for a model-selection bake-off, because part of the score measures stylistic kinship rather than quality. Use a judge from a different family than the models you are comparing, and validate a sample against human grades before you believe the ranking. Where the stakes are high, run two judges from different families and treat disagreement as a case for human review.

shadow eval

Mirror live requests to the new prompt, model, or tool set, score what it would have done, and throw the output away. Shadow evaluation buys you the one thing offline suites cannot — the real input distribution — without exposing a single user to a regression. Two limits to plan for: you pay double inference for the shadowed share of traffic, and shadow runs must not execute side-effecting tools, so anything that writes needs a sandbox or a dry-run mode before you can shadow it at all.

speculative execution

When one tool call is highly likely on the next turn — fetch the user record, list the directory, run the search — you can launch it in parallel with the model call and have the result waiting. This trades wasted work for wall-clock latency, which is often the right trade when tools are cheap and slow. Never speculate on anything with side effects: prefetching a read is an optimization, pre-executing a write is an unauthorized action you may then have to undo, and there is no clean rollback for a sent email.

streaming

Streaming does not make an agent faster; it makes it feel faster by collapsing perceived latency to time-to-first-token. For agents the same principle extends past text: stream the step narration too — "searching the knowledge base", "drafting the reply" — because a run that legitimately takes ninety seconds is tolerable when visibly working and abandoned when silent. Streaming is a UX control, not a performance fix: measure and optimize time-to-done separately, or you will ship an agent that looks responsive and still takes two minutes.

time to done

Per-call latency is the wrong unit for agents: a run is a chain of model calls and tool calls, and time-to-done is their sum plus every retry and every wait for a human approval. Optimize it by attacking the chain, not the calls — cut loop iterations, parallelize independent tool calls, and route easy steps to faster models — because shaving 200ms off one call is irrelevant next to a turn you did not need to take. Report it as a distribution: the p95 run is the one that generates the support ticket.

trajectory eval

Trajectory evals read the trace and score the process: was the right tool selected, were the arguments correct, was the order sane, did it stop when it should. They catch the failures outcome evals cannot see — the run that got the right answer by calling the search tool eleven times, or by reading a record it had no business reading. Matching should be as loose as the task allows: require an exact tool sequence and every legitimate refactor breaks the suite, so prefer "these calls happened, in this partial order" over transcript equality.

verbosity bias

Given two answers of equal quality, judges reliably prefer the longer one, and given a wrong long answer against a right short one they are less reliable than you would like. This matters because agents optimized against a verbose-biased judge learn to pad — and padding costs output tokens on every run forever. Counter it in the rubric: state explicitly that length is not a quality signal, penalize unsupported claims, and log answer length alongside the score so you can check whether your metric is quietly tracking word count.

AgentOps maturity

A ladder of operational capability, not a badge. The rungs are concrete and answerable yes or no: can you reproduce any run from its trace, does a golden dataset gate your deploys, can you revert a prompt change in minutes, has anyone actually pulled the kill switch in a drill. Maturity is measured by what you can do during an incident, not by how good your model is — a team on a frontier model with no traces is less mature than a team on a small model with replay, gates, and drills.

auto-rollback

You declare the metric, the threshold, the observation window, and the revert action; the deploy system watches and reverts on breach. For agents the watched signals have to be agent-specific — escalation rate, tool-error rate, refusal rate, cost per resolved task, turns per task — because HTTP 5xx stays perfectly flat while an agent quietly gets worse at its job. Auto-rollback changes the version; it does not stop the system, so you still need a kill switch for the failure the previous version would also have produced.

canary deployment

The standard progressive-delivery move: 1% of traffic, then 5, then 25, then everything, with a defined observation window and abort criteria at each step. Agents complicate the arithmetic twice — a run costs minutes and dollars, so a small slice takes a long time to say anything statistically, and an averaged canary metric hides per-cohort damage (2% of traffic can be 100% of one tenant). Pair percentage canaries with named-cohort feature flags so you choose who is exposed, not just how many.

circuit breaker

Three states: closed (calls pass), open (calls fail immediately), half-open (a single probe decides). It exists so that retries cannot turn a slow dependency into a total outage. In agents, wire the breaker per tool, not per process, because the model will happily retry a broken tool for as long as its turn budget allows and every one of those attempts is a full model call — an unbroken circuit burns tokens as well as time. When a breaker opens, hand the model an explicit "tool unavailable" result so it can route around the hole instead of guessing.

Nygard, "Release It!" (2007)

dead-letter queue

Standard messaging plumbing (SQS, Service Bus, and Pub/Sub all ship one): messages a consumer could not process after its retry limit land somewhere you can read them. For agents the queue is also an evidence store — a failed run belongs somewhere you can replay it, not in a log line you cannot — so park the full input, the trace id, and the failure reason, then fix the cause and re-drive the case. A dead-letter queue nobody watches is a data-loss mechanism with extra steps, so alarm on its depth and give it an owner.

deadline propagation

Instead of each layer starting a fresh timeout, the caller sends a deadline — a point in time — and every downstream hop budgets against it. Without propagation, nested timeouts multiply: a 30-second tool inside a retry inside a 60-second agent turn can burn minutes past the moment the user gave up. In agents the deadline must reach both the model call and the tool call, and the loop should know how much budget remains so it can pick a cheaper path or return partial work instead of being cut off mid-plan.

degraded mode

The answer to "what happens after we pull the kill switch?" A good degraded mode is designed in advance and visible to users: read-only answers with no writes, suggest-only with a human approving every action, or a straight handoff to a queue. The failure mode you have not designed is the one you get — teams without a degraded mode either keep a broken agent running because switching it off breaks the product, or take the whole feature down. Say plainly which capability is off; a silently dumber agent erodes trust faster than an honest "this is limited right now".

escalation rate

Runs handed to a person (transfer, approval queue, abandonment) over total runs, sliced by intent and by tenant. It needs no labels and it moves before accuracy metrics do, which makes it the best early warning you have: a prompt or model change that quietly hurts quality shows up as escalations first. Read it in both directions — a falling rate can mean the agent got better, or that it stopped escalating things it should have.

exponential backoff

Retry after one second, then two, then four, then eight, with a cap and a maximum attempt count. The point is not politeness: synchronised immediate retries are how a brief blip becomes an outage, because every client piles on at the same instant. Agents need backoff at two layers — the HTTP client retrying a 429 from the model provider, and the loop deciding whether the model may attempt a failed tool again — and the second layer is the expensive one, since each agent-level retry is a fresh model call. Always pair backoff with jitter and a retry budget.

AWS Architecture Blog, "Exponential Backoff And Jitter" (2015)

fallback chain

Degradation you declared in advance: primary model, then a second model or region, then a cheaper deterministic path, then a person. Two rules keep it honest. A fallback that silently changes behaviour is an incident you cannot see — emit which link served each request and alarm on the mix, because a chain quietly running on link three for a week is a quality regression nobody filed. And exercise the links: an untested fallback is usually a second outage, since the standby model may not honour your tool schemas or your prompt at all.

feature flag

The unit of exposure control. For agents, flag more than the feature: the model version, each individual tool, the autonomy level (suggest-only versus act), memory writes. That granularity is what makes a real incident survivable — you can disable the one tool that is misbehaving instead of taking the whole agent down — and it is what lets you expose named design-partner tenants deliberately rather than hoping a percentage canary happens to include them. Flags are also debt: each one is a branch you have to test, so give every flag an owner and an expiry date.

jitter

Pure exponential backoff still synchronises: everyone who failed at the same moment retries at the same moment, so the herd arrives together and takes the dependency down again. Adding a random component — full jitter picks the delay uniformly between zero and the current cap — spreads that load out. Jitter is the cheapest reliability fix in the retry stack, a few lines that turn a thundering herd into a trickle, and it matters more for agent fleets than for web requests: a thousand agents retrying one rate-limited model provider is exactly a synchronised herd.

AWS Architecture Blog, "Exponential Backoff And Jitter" (2015)

microVM

Stronger isolation than a container (a real hypervisor boundary and its own kernel) at close to container startup cost, which makes it practical to give every agent session its own machine and then throw it away. As of September 2026 Amazon Bedrock AgentCore Runtime does exactly that: each session runs in a dedicated microVM with isolated CPU, memory, and filesystem, and on termination the microVM is destroyed and its memory sanitized. Session-scoped microVMs stop "the agent wrote a file" from being a cross-tenant problem — but they isolate compute only; shared memory stores, shared credentials, and shared indexes are separate layers you still have to scope.

Amazon Bedrock AgentCore Developer Guide — How Runtime works (microVMs)

model drift

Three distinct failures wear this one name: the provider updates or retires the model version you were implicitly relying on, your traffic shifts toward inputs the prompt was never tuned for, or the data behind your tools changes shape. All three are invisible without a regression suite that runs on a schedule, not only on your commits — drift arrives on the provider’s calendar, not yours. The defences are pinning explicit model versions, re-running a golden dataset weekly, and watching a label-free proxy such as escalation rate between eval runs.

multi-tenancy

The efficiency case is obvious (one fleet, one upgrade path); the risk is that every shared component becomes a candidate cross-tenant leak. Agents add shared surfaces classic SaaS does not have: memory stores, vector indexes, tool credentials, prompt caches, and the context window itself. Multi-tenancy is not one property but a checklist per layer — compute, storage, memory, identity, telemetry — and a platform can be immaculate at one layer while wide open at the next.

namespace

Logical partitioning: keys, paths, or collections prefixed by tenant or actor so a query only touches one slice. AgentCore Memory works this way, scoping events by actor and session and retrieving long-term records through namespace paths. The catch is that a namespace is a convention enforced by whoever writes the query — one retrieval call with the prefix omitted, or templated from model output, crosses the boundary silently. Namespaces therefore belong behind a layer that injects the scope, never at the discretion of each call site.

Amazon Bedrock AgentCore Developer Guide — Memory

noisy neighbor

The classic multi-tenant failure: a single heavy user exhausts a resource the others depend on. Agents sharpen it because one run is unbounded work — a task that decides to call a tool two hundred times can drain an entire provider rate limit by itself, and the noisy neighbor is usually a bug or an injection, not your biggest customer. The controls are per-tenant quotas and concurrency caps enforced at admission, plus per-run turn and token budgets so no single agent can consume the fleet.

promotion gate

The written contract for "is this good enough to go further": named checks with thresholds, evaluated automatically, blocking by default. For agents the gate is a bundle rather than a single number — outcome pass rate on the golden dataset, 100% on the safety slice, cost and latency inside budget, no unreviewed change to tool permissions. A gate anyone can wave through is documentation, not a control, so record every override and its reason; that override log is the most informative artefact a maturing AgentOps practice produces.

retry budget

Per-call retry limits still permit a system-wide storm: if every caller retries three times during an outage, offered load quadruples exactly when the dependency is weakest. A budget makes retries a scarce global resource — say, at most 10% of in-flight requests may be retries — and sheds the rest. Agents need a second budget inside the loop, because a model re-deciding to call a failed tool is a retry that costs a full model call and no HTTP client is counting those.

right to erasure

Article 17 of the GDPR gives data subjects the right to erasure of their personal data in defined circumstances. The engineering problem is that agents copy personal data into places nobody inventories: long-term memory records, summaries derived from those records, traces and spans, eval cases harvested from production, prompt caches. If you cannot enumerate every store a run wrote to, you cannot honour an erasure request — which is why per-actor namespaces, retention limits on traces, and redaction before an eval case is saved are compliance features, not hygiene.

GDPR Article 17 (right to erasure)

rollback

Reverting code is easy; reverting an agent is not. The unit of rollback is the whole versioned bundle — prompt, model version, tool schemas, memory schema — because a prompt that only works against the new tool contract breaks the moment you revert one half. And the model already acted: a rolled-back deployment leaves real side effects behind (emails sent, tickets created, records written), so the plan needs compensating actions and an idempotency story, not just the previous image tag.

schema version

An agent has more contracts than it looks: the tool argument schemas the model was prompted against, the shape of memory records written last month, the structured output your downstream code parses. Version each and support N and N-1 at once, because during a canary both versions are live and a run that started under the old schema will finish under the new deployment. Additive changes (a new optional field) are usually safe; renames and type changes break the model’s learned behaviour as much as your parser.

shadow mode

The candidate sees live inputs, its tool calls are stubbed or forced read-only, and nothing it produces reaches a user. You get production-distribution evidence at zero user risk, which no canary can give you — a canary is already acting. The limits are real: shadow mode cannot test anything downstream of a write, so a run that depends on the record it would have created diverges immediately, and you pay full inference cost for traffic that serves nobody. Use it to build the comparison set a promotion gate then judges.

tenant isolation

The property multi-tenancy is supposed to preserve, checked layer by layer: compute (a session per tenant), storage and memory (scoped namespaces and keys), identity (per-tenant credentials, never one shared master key), telemetry (traces a support engineer can read without seeing another tenant’s content). Isolation is only as strong as its weakest layer — a per-run microVM buys you nothing if the agent inside it queries a global memory store with a global API key. Test it adversarially: a prompt-injection case whose goal is another tenant’s data belongs in your eval suite.

timeout

Agents need timeouts at three levels, and having one is not having the others: per tool call (a hanging HTTP request), per model call (a stalled stream), and per run (wall clock across the whole loop). The run-level budget is the one teams forget and the one that stops a task from grinding for an hour. A timeout without a defined result is just a different bug — decide what the model sees when a tool times out, because an explicit "timed out, unknown whether it applied" beats silence: otherwise the model assumes failure and retries a call that may already have succeeded.

tool allowlist

Least privilege applied to the agent’s action surface: name the tools this agent, in this environment, for this tenant is permitted to invoke, and reject the rest at the runtime boundary. It beats prompt instructions because the model cannot be talked out of a list it does not control, and it makes the agent auditable — a diff to the allowlist is a change in blast radius and deserves review as one. Keep it per environment (production rarely needs what staging needs) and pair it with argument validation, because an allowed tool with unconstrained arguments is still a wide door.

trajectory

The other half of agent quality. Two runs can return the same correct answer while one took three tool calls and the other took thirty-one, called a write tool twice, and arrived by luck. Trajectory evals grade that path from the trace: tool choice, order, redundant calls, whether the run stopped for the right reason. A right answer reached by a wrong path is a latent failure — it will not survive a slightly different input, and it is usually where a regression surfaces first after a model upgrade.