Learning from incidents — and ending systems well
Lesson 5 of 5 in Human Oversight, Post-Deployment Monitoring, and Incident Response.
Aviation became safe because every crash — and every almost-crash — feeds a shared learning system. AI is building the same infrastructure, and a working governance program plugs into it in both directions: mining others’ incidents for risks to test against, and treating its own as data, not embarrassments.
AI Incident Database (AIID) — the crash registry
Run by the Responsible AI Collaborative, the AIID catalogues public reports of AI harms — roughly 1,700 incidents and counting as of late 2026 — indexed and taxonomised. Its governance uses: pre-deployment horizon scanning (“what has gone wrong with systems like ours?”), red-team scenario seeding, and evidence that a risk you dismissed as theoretical has already happened to someone else.
OECD AI Incidents Monitor (AIM) — the policy lens
The OECD’s monitor tracks AI incidents and hazards from global news coverage, feeding the OECD’s work on incident-reporting frameworks and giving regulators a shared evidence base. Where AIID is practitioner-oriented and submission-driven, AIM aims at cross-country comparability — the beginnings of an international common operating picture for AI harm.
MIT AI Risk Repository — the risk taxonomy
A living meta-catalogue distilling hundreds of identified AI risks from the research literature into a structured taxonomy (by domain and by cause). Less an incident record than a checklist generator: teams use it to test the completeness of their risk assessments — the question is not “did we think of risks?” but “which catalogued risks did we consciously rule out, and why?”
Your internal registry — near misses and blameless postmortems
The external databases only see what became public. Your richest data is the incident that almost happened: the near miss a reviewer caught, the drift alert that fired a week before harm. Capturing near misses requires blameless postmortems — analysis focused on how the system of controls allowed the failure path, not on who to punish — because the moment reporting a near miss becomes career-dangerous, your near-miss data dries up and your next incident arrives unannounced.
The lifecycle’s last governance act is decommissioning, and it is not “turn it off”. A model woven into business processes needs: a planned degradation path (what replaces its decisions — a human process, a rules baseline, a successor model that must first pass the release gate?); dependency mapping (which downstream systems consume its outputs and will silently break?); artefact archiving — the model version, training-data snapshot, documentation, and logs retained per legal holds and retention duties (remember: technical documentation ten years, logs at least six months, litigation holds potentially longer), because decommissioned systems get litigated too; and registry closure, so the AI inventory reflects reality. The kill switch you designed in the oversight lesson is this process compressed into an emergency: if you have never rehearsed operating without the model, you do not actually have the option of turning it off.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.