The mapping that matters: your practices are the evidence

Lesson 2 of 3 in Where Agents Meet the Law.

Now the good news, and it is genuinely good: you have already built most of this. Not because governance frameworks are toothless, but because the engineering that keeps a non-deterministic system from hurting you is the same engineering that demonstrates control over it.

Read the matrix in both directions. Left to right, it tells you what each practice is for outside your team. Right to left, it tells you what you would have to build if a reviewer asked and you had nothing — which is the retrofit you are avoiding by reading this now.

Course practice → governance expectation. Deliberately generic: the expectation names a shape of obligation that recurs across instruments, not a clause in any one of them.
Practice this course taught you	Governance expectation it answers	What a reviewer actually asks for	Where you learned it
Traces and spans, retained and searchable	Record-keeping and logging — the system’s behaviour must be reconstructable after the fact.	"Show me everything this agent did on 14 March for customer 8812: which tools, which inputs, which outputs, under whose credentials."	→ agentops: monitoring and incident response
Evals on a golden dataset, gated per release	Testing and validation evidence — accuracy and robustness claims must be supported before deployment.	"What did you test before shipping, on what data, at what pass rate, and where is that result tied to the exact version now in production?"	→ the evals domain
Approval gates with reviewer identity, review time, rejection rate	Human oversight — a person must be able to understand, intervene in, and refuse the system’s actions.	"Who can say no, do they have enough on screen to mean it, and what is your evidence the gate is not a reflex click?"	→ architectures: human in the loop
Kill switches, rehearsed, with a measured time-to-stop	Incident controls — the ability to halt the system and respond when something goes wrong.	"How is it stopped, in how long, by whom, and when did you last actually test that instead of documenting it?"	→ agentops: rollout and kill switches
The version tuple: prompt, toolset, pinned model and parameters, runtime build	Change management and configuration control — you must know what is running and what changed.	"The system we assessed in June — is that the system running today? Prove the difference."	→ agentops: deploying and versioning
least privilege tool scopes, sandboxing, egress control	Technical risk controls proportionate to the risk — the boundary on what the system can do at worst.	"What is the maximum reach of this agent if it behaves badly or is manipulated by injected content, and what enforces that limit?"	→ the security domain
A named owner, an on-call rotation, an escalation path	Accountability and defined roles — a human, not a team abstraction, answers for the system.	"Who is accountable for this agent, by name, and who is on the hook at 03:00?"	→ agentops: org patterns
An agent inventory with per-agent capability and data notes	Documentation and registration — a maintained record of what exists and what it touches.	"List every agent in production, what each can do, and what data each processes."	→ agentops: org patterns

Three ways good instrumentation still fails as evidence

It is not retrievable. Traces sampled at 1%, retained for seven days, and queryable only by the two engineers who know the log syntax are excellent for debugging and useless for answering a question about last quarter. Retention windows and query paths are governance decisions disguised as cost decisions.

It is not attributable. A pass rate with no version stamp, or a trace with no version tuple attached, cannot answer "was this the system you assessed?" Evidence that floats free of the thing it describes proves nothing — this is why the version tuple belongs in the trace and on the eval report.

It is contradicted by your own metrics. Claiming meaningful human oversight while your gate shows a three-second median review and a 0.2% rejection rate is worse than claiming nothing, because you have now documented a control that your telemetry disproves. Measure the control, then describe what you measured.

There is also a real tension here, and pretending otherwise makes you look naive in the room: record-keeping pulls toward capturing everything, data protection pulls toward capturing the minimum. A full trace of a support agent contains the customer’s message, the retrieved account record, and whatever the model repeated back — personal data, sometimes special-category, now duplicated into an observability system with different access controls than the source database.

You do not resolve that trade-off alone, and you should not try to. What you do is surface it early with options attached: redaction at the collection boundary, field-level hashing, shorter retention for payloads than for the decision skeleton, tighter access on the trace store. Bringing three implementable options to governance is a five-minute conversation; bringing the raw dilemma is a three-week one.

Interactive sorting exercise: A reviewer asks for evidence in five shapes. Sort each artifact into the expectation it primarily satisfies — and notice how many are things you already produce for your own debugging.

Interactive flashcard deck.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.