Agentic AI: governing systems that act

Lesson 4 of 5 in Third-Party AI, Generative AI, Agentic AI, and Frontier Governance.

Everything so far governed AI that produces outputs a human then uses. Agentic AI removes that human from the middle of the sentence: the system plans multi-step tasks, calls tools and APIs, holds memory across steps, and executes — booking, purchasing, filing, emailing, writing and running code. The foundations path gave you the autonomy spectrum; this lesson gives you the control set for its far end.

The governance shift is precise: with a chatbot, the worst case is a wrong sentence that a human might still catch; with an agent, it is a wrong action — possibly the fourth action in a chain no human reviewed, taken with credentials you issued, at machine speed. Wrongness compounds: a 95%-reliable step, chained ten times, completes correctly less than 60% of the time. And the prompt-injection exposure you just studied becomes kinetic — an injected instruction to a text generator produces bad text; the same instruction to an agent with a payment tool produces a payment.

Five control families define current practice:

  1. Least-privilege tool permissioning. The agent gets the minimum tool set for its task — scoped, revocable, and its own. An agent that only needs to read the knowledge base cannot email anyone, no matter what an attacker injects. Permissions are the one defense that does not depend on the model behaving.
  2. Budgets and rate limits. Hard caps on spend, action counts, and loop iterations per task and per day. Budgets convert "unbounded failure" into "bounded, priced failure".
  3. Human-approval checkpoints on consequential and irreversible actions. Payments above thresholds, external communications, deletions, contract commitments — the agent proposes, a human disposes. Design these against rubber-stamping: batch approvals of 40 agent actions in one click recreates the automation-bias trap at higher speed.
  4. Agent identity and authentication. Agents act as distinct principals with their own credentials — never as an invisible extension of a human’s login. Without separate identity there is no attribution, no revocation, and no honest audit trail; with shared logins, "who did this?" becomes unanswerable at exactly the moment a regulator asks.
  5. Comprehensive action logging and replay. Every tool call, input, output, and decision point captured — so an incident can be reconstructed step by step. For agents, logs are not a compliance nicety; they are the only forensic record of a decision process nobody watched.

The agent permission model — rings of trust, gates on the way out

  1. Autonomous zone — Read-only, reversible, in-sandbox

    Actions the agent may take freely: reading permitted data sources, drafting content, running code in a sandbox, querying internal tools. Low blast radius, fully logged. Most of an agent’s useful work should live here — that is a design goal, not an accident.

  2. Guarded zone — Bounded writes under budget

    Actions with real but capped consequences, allowed under budgets, rate limits, and tight scoping: small payments under threshold, updates to non-critical records, sending messages to pre-approved internal recipients. Exceeding any cap escalates to the gate.

  3. Human gate — Consequential or irreversible — agent proposes, human approves

    External communications, payments above threshold, deletions, contractual commitments, anything touching production systems or personal data at scale. The approval interface must show the human what the agent intends and why — and approval rates near 100% at reflex speed mean the gate has failed, not succeeded.

  4. Forbidden zone — No tool exists — not "please don’t"

    Actions the agent must never take are enforced by absence of capability, not by instructions. No credential, no API route, no tool registration. Instructions can be injected away; a missing tool cannot. This ring is the difference between security and etiquette.

Two frontier-of-practice problems deserve a flag even though the playbooks are young.

Multi-agent interactions. When agents call other agents — yours calling a vendor’s, or several of yours negotiating — failure modes appear that no single-agent review catches: feedback loops, emergent coordination, responsibility laundering ("our agent only did what their agent asked"). Current best practice is conservative: treat every agent-to-agent boundary as an external trust boundary with its own authentication, logging, and budget.

Tool-calling supply chains. Agents discover and call tools through connector protocols (the Model Context Protocol pattern), which imports a classic software supply-chain problem: a malicious or compromised tool server can feed poisoned data (indirect injection again) or misdescribe its own capabilities. Governance response: an allow-listed tool registry with an approval process — tools are procured artifacts like any other dependency, with an owner, a review, and a version pin.

Key terms: agentic AI, least privilege, human-in-the-loop, action logging, agent identity, mcp

Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.