Systemic risks, the accountability gap, and keeping score

Lesson 6 of 6 in Why AI Needs Governance: A Taxonomy of Harms and Risks.

The last harm family is the hardest to see because it has no single victim. Systemic risks damage the structure everything else runs on:

Concentration. Frontier-scale AI needs frontier-scale compute, and a handful of firms control it. Concentration is a harm multiplier: one company’s design choice, safety failure, or price change propagates through every product built on its models.

Monoculture. When thousands of downstream systems share one foundation model, they share its blind spots — and can fail simultaneously, in the same direction. Finance learned this lesson with correlated risk models in 2008; AI is rebuilding the same correlation at the cognitive layer.

Model collapse. As generated content floods the web, future models increasingly train on the outputs of past models. Research suggests recursive training degrades quality and diversity — the data commons polluted by its own consumers.

Loss of control. The frontier-safety concern: systems capable and agentic enough that human intervention becomes unreliable. Whatever probability you assign it, the governance machinery it motivated is concrete and present: capability evaluations, frontier safety frameworks, compute-threshold rules, safety institutes, and international summit commitments from Bletchley (2023) onward.

And beneath all seven families sits the meta-harm: the accountability gap. AI harm is distributed harm — the “many hands” problem. Data collectors, model developers, fine-tuners, deployers, and users each contributed, so each can point elsewhere. Victims often cannot detect the algorithm’s role, cannot afford to prove it, and meet humans who defer to the machine at every appeal. Closing that gap — with role-based duties, documentation trails, disclosure rights, and contestability — is, in one sentence, what the rest of this curriculum teaches.

The field’s answer to scattered, deniable harm is systematic memory. The AI Incident Database (AIID) has catalogued thousands of incidents since 2020; the OECD’s AI Incidents Monitor tracks them for policymakers; the MIT AI Risk Repository synthesises hundreds of risk taxonomies into one; NIST’s GenAI Profile enumerates generative-specific risk categories. These catalogs matter for a practical reason: shared taxonomies are what turn anecdotes into regulation — and into your risk register. When frameworks require you to identify “reasonably foreseeable” risks, foreseeability is defined by exactly these public records. An incident type already catalogued a hundred times is foreseeable by definition — and so is your liability for ignoring it.

Now prove you can run the taxonomy. Classify each real incident below by its primary harm family — the lens under which the case became famous.

Interactive sorting exercise: Drag each documented incident to its primary harm family.

Key terms: systemic risk (GPAI), monoculture risk, model collapse, many hands problem, AI Incident Database, contestability

Tool: AI Incident Tabletop — Test the taxonomy under pressure: run a live incident tabletop and classify, escalate, and respond as the harms unfold.