AI Governance Academy glossary

604 terms with primary sources.

AI system

The legally load-bearing definition of AI. The EU AI Act Art 3(1) and the 2023 revised OECD definition both center on inference: the system infers, from the inputs it receives, how to generate outputs — predictions, content, recommendations, or decisions — that can influence physical or virtual environments, with varying levels of autonomy and possible adaptiveness after deployment. Systems whose outputs follow only fixed, human-specified rules generally fall outside it.

EU AI Act Art 3(1); OECD (2023)

machine learning

A family of techniques where a model’s behaviour is learned from data rather than explicitly programmed. Developers choose the architecture, the data, and the objective; the resulting decision logic emerges from training. That is why ML systems are statistical and probabilistic — and why governing them differs fundamentally from governing conventional software.

deep learning

Machine learning built on neural networks with many layers, whose millions-to-trillions of numeric parameters are tuned during training. Scale is the story: more data plus more compute has reliably produced more capability. It is also where the black box problem bites hardest — no individual parameter means anything a human can read.

generative AI

AI systems that generate content. Most run on foundation models trained on vast corpora. Generative AI reshaped governance: it created new risk categories (synthetic media, hallucination, training-data copyright) and new legal categories (the EU’s general-purpose AI rules, content-labeling laws in China and California).

foundation model

A model trained on broad data at scale, adaptable to a wide range of tasks — the base layer beneath most modern AI products. Regulators map roughly the same idea with different terms: the EU regulates general-purpose AI models; US frameworks often say dual-use foundation models. The one-to-many structure is what makes value-chain governance hard: one upstream model’s flaws propagate into thousands of downstream systems.

large language model (LLM)

A foundation model trained on enormous text corpora to predict the next token. Everything an LLM does — answering, summarising, coding — emerges from that single objective plus post-training alignment. This is why LLMs produce fluent, plausible text that is not guaranteed to be true: they model language, not facts.

general-purpose AI (GPAI)

The EU AI Act’s term for AI models displaying significant generality — capable of competently performing a wide range of distinct tasks, however placed on the market. GPAI models carry their own obligation track (Arts 51–56), separate from the risk tiers that apply to AI systems. Models trained above 10²⁵ FLOPs are presumed to pose systemic risk and carry extra duties.

EU AI Act Art 3(63), Arts 51–56

agentic AI

AI systems that don’t just answer but act: they decompose goals into steps, call tools and APIs, hold memory across steps, and operate with meaningful autonomy. Agents stress every classical governance control — oversight designed for single decisions must now cover chains of actions, delegation between agents, and real-world side effects.

training

The phase where a model’s parameters are optimised against training data. Contrast inference: running the finished model on new inputs. The split matters legally — data-protection and copyright questions concentrate at training time, while transparency, oversight, and incident duties mostly attach at inference/deployment time.

inference

Executing a trained model on fresh inputs to generate outputs. In governance texts the word does double duty: engineers mean the runtime phase, while the EU AI Act’s definition of an AI system uses infers to describe how AI derives outputs from inputs at all — the feature that separates AI from fixed rule-following software.

supervised learning

The workhorse ML paradigm: the model learns from labeled examples (this loan defaulted; this image is a stop sign). Most scoring and screening systems that worry regulators — credit, hiring, recidivism — are supervised learners, which is why label quality and historical bias in the labels are core governance concerns.

unsupervised learning

Learning without labels: the model finds structure on its own — customer segments, fraud anomalies, topic clusters. Governance concern: the discovered groupings can silently encode sensitive attributes (a “neighborhood cluster” that is really a proxy for race).

reinforcement learning

A paradigm where an agent learns by acting and receiving rewards. Powerful and notoriously literal: agents exploit loopholes in the reward (reward hacking), which is a compact preview of the alignment problem — systems optimising the metric you wrote, not the outcome you meant.

RLHF (Reinforcement Learning from Human Feedback)

The post-training step that turned raw next-token predictors into usable assistants: human raters rank outputs, a reward model learns those preferences, and the LLM is tuned against it. Governance-relevant limits: it optimises for appearing helpful and harmless to raters, which can produce sycophancy and confident-sounding error.

fine-tuning

Taking a pre-trained model and continuing training on a narrower dataset to specialise it (a general LLM → a medical triage assistant). Legally live: fine-tune someone else’s model substantially enough and you may become the provider of a new model or system, inheriting the obligations that come with the title.

hallucination

When a generative model outputs plausible but false content — invented citations, fake case law, wrong medical claims. It is intrinsic to how LLMs work (they model plausible language, not verified fact), so governance treats it as a risk to be managed with retrieval grounding, verification layers, disclosure, and use-case limits — not a bug awaiting a patch.

black box

Shorthand for opacity: a deep model’s behaviour lives in millions of numeric parameters no human can read as reasons. The governance response is layered — explainability techniques approximate why, documentation records how it was built, and process rights (like contesting a decision) protect people even when full mechanistic transparency is impossible.

explainability

Techniques and practices for rendering model behaviour intelligible — from feature-attribution methods (SHAP, LIME) to counterfactuals (“you would have been approved if…”). Distinct from transparency (disclosure about the system) and interpretability (models readable by design). Legal hooks include GDPR Art 22 safeguards and the EU AI Act’s Art 86 right to explanation.

transparency

The most overloaded word in AI governance — a classic false friend across frameworks.

autonomy spectrum

AI systems occupy a spectrum: decision support → human-in-the-loop → supervised autonomy → full autonomy. Governance intensity should track position on the spectrum — which is why “varying levels of autonomy” sits inside the legal definition of an AI system, and why agentic AI is pushing systems rightward faster than oversight design is adapting.

human oversight

The principle that natural persons must be able to effectively oversee AI in use — understand its limits, watch for anomalies, resist automation bias, and intervene, override, or halt it. EU AI Act Art 14 makes it a hard legal requirement for high-risk systems; every major framework carries a version of it.

EU AI Act Art 14

automation bias

The documented tendency of humans to defer to automated suggestions — approving them faster and questioning them less than their accuracy warrants. It is the failure mode that turns “human in the loop” into a rubber stamp, which is why the EU AI Act names it explicitly in Art 14 and why oversight design (friction, sampling, accountability) matters more than oversight presence.

open-weight model

A model released with its weights downloadable, versus one accessible only through a controlled API. Release strategy is a governance decision: open weights democratize access and scrutiny but cannot be recalled, and safety fine-tuning can be stripped. Laws split on it — the EU grants open-source GPAI a partial documentation exemption that vanishes at systemic-risk scale.

distribution shift

Models learn the statistics of their training data; when the world changes — new slang, new fraud patterns, a pandemic — accuracy decays without any code change or error message. This is why one-time pre-deployment testing is never enough and every serious framework demands post-deployment monitoring.

model drift

The operational symptom of distribution shift: a deployed model’s real-world performance decays over time. Governance answer: monitoring with drift metrics, retraining triggers, and lifecycle documentation — the reason frameworks treat deployment as the start of obligations, not the end.

provider

The EU AI Act’s primary duty-holder: the actor who develops an AI system or GPAI model (or has one developed) and places it on the market or puts it into service under their own name or trademark. Providers carry the heaviest obligations. Beware role-switching: rebrand, substantially modify, or re-purpose someone else’s system and you may become its provider.

EU AI Act Art 3(3)

deployer

The organisation using an AI system under its authority (not the end customer affected by it). Deployers carry use-side duties: following instructions for use, assigning competent human oversight, monitoring, log retention, and — for some high-risk uses — fundamental-rights impact assessments. Most companies adopting AI are deployers long before they are providers.

EU AI Act Art 3(4)

deepfake

Synthetic audio, image, or video that appreciably resembles real persons, places, or events and would falsely appear authentic. A named legal category: the EU AI Act defines it (Art 3(60)) and attaches disclosure duties (Art 50(4)); election-deepfake and intimate-image laws are proliferating worldwide.

EU AI Act Art 3(60)

proxy variable

A feature correlated with a protected characteristic strongly enough that a model can discriminate without ever seeing the protected attribute itself (ZIP code → race; shopping patterns → pregnancy). The reason “we removed the sensitive column” is never a sufficient fairness defense, and a core concept behind disparate-impact analysis.

alignment

The research problem of getting AI systems to do what their principals actually intend — not just what the training objective literally rewards. Spans practical work (RLHF, constitutional methods) and frontier-safety questions (deception, scheming, scalable oversight). Governance interfaces with it through capability evaluations and frontier safety frameworks.

systemic risk (GPAI)

The EU AI Act’s label for GPAI models with high-impact capabilities whose failures could ripple across the Union — presumed when training compute exceeds 10²⁵ floating-point operations, or designated by the Commission on Annex XIII criteria. Triggers the heaviest model-level duties: state-of-the-art evaluations, adversarial testing, incident reporting, and cybersecurity for the weights themselves.

EU AI Act Arts 51–55

disclosure

The lightest-touch governance tool: informing people that they are interacting with an AI system, being analysed by one, or viewing synthetic content. It anchors the EU AI Act’s limited-risk tier (Art 50), China’s labeling rules, and a growing family of US state laws.

high-risk AI system

The tier that carries the AI Act’s full rulebook (Arts 8–15: risk management, data governance, documentation, logging, transparency, human oversight, accuracy/robustness/cybersecurity). Two independent routes lead in: Art 6(1) — the AI is a product or safety component under Annex I Union product law requiring third-party conformity assessment — and Art 6(2) — the use case appears in Annex III (biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration, justice). Following the Digital Omnibus amendments, Annex III obligations apply from 2 December 2027 and Annex I from 2 August 2028. High-risk does not mean forbidden; it means regulated market access.

EU AI Act Art 6, Annexes I & III

Annex III

The Act’s catalogue of contexts where AI touches people’s life chances: (1) biometrics, (2) critical infrastructure, (3) education, (4) employment and worker management, (5) essential private and public services including creditworthiness and life/health insurance pricing, (6) law enforcement, (7) migration, asylum and border control, (8) administration of justice and democratic processes. Listing works at sub-point granularity — CV-ranking is in, spell-checking a CV is not — and several sub-points carry carve-outs (e.g. fraud detection within creditworthiness). The Commission can amend the list under Art 7, which is why classification is a living exercise, not a one-time check.

EU AI Act Art 6(2), Annex III

Article 6(3) filter

An Annex III match is a presumption, not a verdict. Art 6(3) lets a provider conclude the system is not high-risk where it does not pose a significant risk to health, safety or fundamental rights because it (a) performs a narrow procedural task, (b) improves the result of a previously completed human activity, (c) detects decision patterns or deviations without replacing human assessment, or (d) performs a preparatory task. Two tripwires: profiling of natural persons always stays high-risk, and the self-assessment must be documented before market placement and registered (Art 49(2)) — authorities can demand it and reclassify. The filter is where most classification disputes will live.

EU AI Act Art 6(3)–6(4), Art 49(2)

fundamental rights impact assessment (FRIA)

A deployer-side duty, not a provider one. Required of public-law bodies, private entities providing public services, and deployers of creditworthiness and life/health-insurance systems (Annex III 5(b)–(c)) before first use of a high-risk AI system. Contents: the deployment process, period and frequency of use, categories of affected persons, specific risks of harm, human-oversight measures, and mitigation/governance/complaint arrangements. It must be updated when circumstances change and notified to the market surveillance authority; the AI Office provides a template. Where a DPIA already covers ground, the FRIA complements rather than duplicates it — the lens is fundamental rights of affected people, not data-protection compliance alone.

EU AI Act Art 27

conformity assessment

The gate between development and the EU market. Art 43 sets the routes: most Annex III systems (points 2–8) use internal control (Annex VI — the provider self-verifies against Arts 8–15); Annex III point 1 biometrics may self-assess only if harmonized standards were applied in full, otherwise a notified body must examine the quality system and technical documentation (Annex VII); Annex I products fold the AI assessment into their existing sectoral procedure. A substantial modification triggers re-assessment. Passing yields the EU declaration of conformity and the right to affix CE marking.

EU AI Act Art 43, Annexes VI–VII

CE marking

The end point of the New Legislative Framework chain: essential requirements → standards → conformity assessment → CE mark. For high-risk AI the mark must be affixed visibly, legibly and indelibly (digitally where the system is digital-only), accompanied where relevant by the identification number of the notified body involved. It signals to every market-surveillance authority in the Union that the provider claims — and can evidence — conformity. Affixing it without basis is a formal non-compliance under Art 83 and a fast route to enforcement.

EU AI Act Art 48

notified body

The third-party checker in EU product law, now extended to AI. Notified bodies must prove independence, competence, confidentiality and insurance (Arts 28–39), are designated by national notifying authorities, and are listed in the Commission’s NANDO database. Under the AI Act their main workload is Annex III point 1 biometric systems assessed under Annex VII, plus AI embedded in Annex I products where sectoral law already demands third-party assessment. They issue certificates with validity ceilings (5 years for Annex I, 4 years for Annex III, Art 44) and can suspend or withdraw them — a private-sector actor wielding quasi-regulatory power.

EU AI Act Arts 28–39, 44

harmonized standard

The translation layer between legal text and engineering practice. The Commission issued standardisation request C(2023)3215 to CEN/CENELEC JTC 21 covering ten areas mapped to the AI Act’s requirements (risk management, data governance, logging, transparency, oversight, accuracy, robustness, cybersecurity, quality management, conformity assessment). Once a standard is cited in the OJEU, applying it triggers the presumption of conformity. As of September 2026 no AI Act harmonized standards have been cited — prEN 18286 (quality management, Art 17) was the first deliverable to reach public enquiry — so providers must currently self-assess against the Act directly; check current status before relying on any standard.

EU AI Act Art 40

presumption of conformity

The engine that makes voluntary standards commercially irresistible. A provider who applies a harmonized standard cited in the OJEU is presumed compliant with the essential requirements that standard covers — shifting the burden to any authority that disagrees. Art 42 adds specific presumptions (e.g. training data matching the deployment context for Art 10; Cyber Resilience Act certification for Art 15 cybersecurity). The presumption is rebuttable and only ever covers what the standard covers. Because no AI Act standards are yet cited (as of September 2026), the presumption is not yet activatable — a fact that reshaped the compliance timetable debate.

EU AI Act Arts 40–42

common specifications

Plan B in the New Legislative Framework. Where harmonized standards are absent, insufficient, or blocked, the Commission may adopt common specifications by implementing act (Art 41); applying them grants the same presumption-of-conformity effect. They matter because JTC 21’s deliverables missed the original April 2025 deadline — if standards are still missing as high-risk obligations bite (December 2027), common specifications are the mechanism regulators would reach for. Providers may deviate from them only by proving an equivalent technical solution.

EU AI Act Art 41

market surveillance

The enforcement arm of EU product law (Regulation 2019/1020), applied to AI by Art 74. Each Member State designates a market surveillance authority with powers to demand technical documentation and data, access training/testing datasets, and — under conditions — source code (Art 74(13)). Escalation runs from national measures (Art 79) through the Union safeguard procedure (Art 81) to actions against systems that are compliant yet still risky (Art 82). Fundamental-rights bodies get access rights too (Art 77). For GPAI models, supervision belongs exclusively to the AI Office instead.

EU AI Act Arts 74–83; Reg (EU) 2019/1020

serious incident

Defined at Art 3(49): an incident or malfunction directly or indirectly leading to death or serious damage to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental-rights obligations, or serious damage to property or the environment. Providers of high-risk systems must report to the market surveillance authority within 15 days of awareness — compressed to 10 days for death and 2 days for widespread infringement or critical-infrastructure disruption (Art 73). GPAI systemic-risk providers carry a parallel duty to the AI Office (Art 55). The clocks start at awareness, which is why incident-detection capability is itself a compliance requirement.

EU AI Act Art 3(49), Art 73

post-market monitoring

The AI Act’s answer to the fact that models decay: Art 72 requires providers to run a documented post-market monitoring plan (part of the technical documentation, on a Commission template) that actively collects data on real-world performance and feeds the risk-management system. It closes the loop with deployer duties to monitor and report (Art 26) and with serious incident reporting (Art 73). Distinct from an organisation’s internal drift monitoring: post-market monitoring is the legal obligation; drift monitoring is one engineering technique used to meet it.

EU AI Act Art 72

regulatory sandbox

A controlled framework, run by a competent authority, where providers can develop, train and test AI systems under regulatory supervision before market placement. The AI Act obliges every Member State to have at least one operational by 2 August 2026 (Arts 57–63), with priority access for SMEs. Participants following the sandbox plan in good faith are shielded from administrative fines, exit reports can feed conformity assessment, and Art 59 provides a legal basis for reusing personal data for developing certain public-interest systems. Arts 60–61 separately govern real-world testing outside sandboxes (registration, informed consent, time limits, reversibility). Singapore, the UK and Texas run analogous schemes — the concept predates the Act.

EU AI Act Arts 57–63

remote biometric identification (RBI)

The AI Act splits RBI by timing, and the split carries enormous legal weight. Real-time RBI in publicly accessible spaces for law enforcement is prohibited (Art 5(1)(h)) except for three exhaustive purposes — targeted search for victims/missing persons, prevention of a specific substantial and imminent threat or terrorist attack, and localisation of suspects of listed serious crimes — each requiring prior judicial or independent administrative authorisation (24-hour urgency route), a Member State opt-in law, and a FRIA plus registration. Post (retrospective) RBI is not prohibited but is high-risk under Annex III point 1, with the deployer needing authorisation within 48 hours (Art 26(10)). Same technology, three legal fates depending on timing and context.

EU AI Act Art 5(1)(h), Annex III(1), Art 26(10)

social scoring

Prohibited by Art 5(1)(c) for both public and private actors — a common misreading is that only governments are caught. The test has two limbs: evaluation or classification of persons over time based on social behaviour or personal characteristics, leading to detrimental treatment that is either (a) in social contexts unrelated to where the data was generated, or (b) unjustified or disproportionate to the behaviour. A credit score built from repayment history is fine; denying someone housing because of their social-media conduct is the paradigm violation. China’s fragmented “social credit” experiments were the political trigger, but the ban is written technology- and country-neutral.

EU AI Act Art 5(1)(c)

substantial modification

Defined at Art 3(23): a change not foreseen in the provider’s initial conformity assessment that affects the system’s compliance with the high-risk requirements or modifies its intended purpose. Consequences stack: the system needs re-assessment (Art 43(4)), and whoever made the change can become the provider with the full obligation set (Art 25). One carve-out matters for machine learning: changes to a continuously learning system that were pre-determined and documented in the original assessment do not count. Version-control discipline is therefore a legal control, not just an engineering nicety.

EU AI Act Art 3(23), Art 25, Art 43(4)

authorised representative

The Act’s anchor for extraterritorial enforcement: providers established outside the EU must appoint, by written mandate, a natural or legal person in the Union before making high-risk systems (Art 22) or GPAI models (Art 54) available. The representative verifies that the conformity assessment and documentation exist, keeps them at the authorities’ disposal for ten years, cooperates with authorities, and must terminate the mandate if it considers the provider non-compliant — turning a private contractor into a compliance tripwire. Mirrors the GDPR Art 27 representative, but with verification duties rather than a mailbox role.

EU AI Act Arts 22, 54

importer (AI Act)

Defined at Art 3(6) and given teeth at Art 23: before placing a high-risk system on the market, the importer must verify that the provider carried out the conformity assessment, drew up the technical documentation, and affixed the CE marking with the declaration of conformity. If the system is non-conforming, the importer must withhold it and inform the provider and authorities. Importers put their own name on the packaging, ensure storage and transport do not jeopardise compliance, and retain documentation for ten years. Rebrand the system or substantially modify it and the importer becomes the provider (Art 25).

EU AI Act Art 3(6), Art 23

distributor (AI Act)

Anyone in the chain, other than the provider or importer, who makes an AI system available on the Union market (Art 3(7)). Duties are lighter but real (Art 24): check that the CE marking and documentation are present, withhold non-conforming systems, take corrective action or inform authorities when a system on their shelves presents a risk. Together with importers and deployers, distributors face the Art 25 role-switch: put your trademark on it, substantially modify it, or repurpose it into high-risk, and every provider obligation transfers to you.

EU AI Act Art 3(7), Arts 24–25

AI Office

Established within the European Commission (Art 64), the AI Office is the closest thing the Act has to a central regulator. It holds exclusive supervision and enforcement power over GPAI models (Art 75, Art 88) — national authorities handle AI systems — and drives implementation: classification guidelines, the GPAI Code of Practice, the FRIA and training-data-summary templates, and codes of practice on synthetic-content labeling. It can compel documentation, order model evaluations, and levy fines up to €15M or 3% of worldwide turnover on GPAI providers (Art 101). Not to be confused with the European AI Board, which coordinates Member States.

EU AI Act Art 64, Arts 75, 88, 101

European AI Board

Created by Arts 65–66: one representative per Member State (with the EDPS and AI Office attending), tasked with coordinating national competent authorities, issuing opinions and recommendations, and promoting consistent application of the Act — the AI analogue of the GDPR’s European Data Protection Board. It is flanked by an Advisory Forum of stakeholders (Art 67) and a Scientific Panel of independent experts (Art 68) who can issue qualified alerts about systemic-risk models to the AI Office. Governance shorthand: the AI Office runs GPAI, the Board keeps 27 national enforcers rowing in the same direction.

EU AI Act Arts 65–68

technical documentation

The Act’s deepest documentation duty (Art 11): drawn up before market placement and kept current, following Annex IV — general description, development methods and architecture, data provenance and datasheets, human-oversight assessment, validation and testing with metrics and logs, cybersecurity, monitoring plan, risk-management description, lifecycle changes, standards applied. SMEs may use a simplified format. GPAI providers carry parallel duties: Annex XI documentation for the AI Office and Annex XII information for downstream providers (Art 53). Distinct from a model card: the tech file is exhaustive and confidential to regulators; the model card is a voluntary, public summary.

EU AI Act Art 11, Annex IV; Art 53, Annexes XI–XII

instructions for use

Art 13’s transparency duty runs provider→deployer, not provider→public: the system must be “sufficiently transparent to enable deployers to interpret and use its output appropriately,” backed by instructions covering identity of the provider, characteristics, capabilities and limitations of performance (accuracy metrics, foreseeable misuse, affected groups), human-oversight measures, expected lifetime, and maintenance. They are load-bearing for the whole value chain: deployer duties under Art 26 are framed as using the system in accordance with the instructions, so a gap in the instructions shifts risk back onto the provider.

EU AI Act Art 13

AI literacy

One of the first obligations to apply (2 February 2025) and the broadest: providers and deployers must take measures ensuring a “sufficient level of AI literacy” among staff and other persons operating AI on their behalf, accounting for their technical knowledge, experience, training, and the context of use. There is no standalone fine tier, but literacy failures colour supervision and liability — an untrained overseer undermines the Art 14 human-oversight defence. The Commission maintains a living repository of AI-literacy practices. In practice: role-based training programs, not a single all-hands slide deck.

EU AI Act Art 4

Digital Omnibus

The EU’s mid-course correction. Proposed 19 November 2025 as part of the digital simplification agenda and in force from summer 2026, the Omnibus postponed the high-risk regime — Annex III obligations to 2 December 2027 and Annex I embedded high-risk to 2 August 2028 — added a prohibition on AI-generated non-consensual intimate imagery and CSAM (applicable 2 December 2026), and set the synthetic-content transparency compliance deadline at 2 December 2026. Everything else (prohibitions, GPAI rules, governance, the rest of the general regime applicable 2 August 2026) stood. Details beyond the verified dates — conditionality on standards availability, SME relief — remained in flux; check current status before relying on them.

EU AI Act as amended (Digital Omnibus, 2026) — verify citation

intended purpose

Defined at Art 3(12) as the use for which the provider intends the system, as stated in the instructions for use, marketing materials, and technical documentation. It is the Act’s load-bearing concept: risk classification is assessed against intended purpose; Arts 8–15 compliance is judged “taking into account the intended purpose”; and a deployer who changes the intended purpose so a system becomes high-risk takes over the provider role (Art 25). The counterweight is reasonably foreseeable misuse — providers cannot define their way out of predictable off-label use.

EU AI Act Art 3(12)

reasonably foreseeable misuse

Art 3(13): use not in accordance with the intended purpose, but resulting from reasonably foreseeable human behaviour or interaction with other systems. The Art 9 risk-management system must identify and address risks from both intended use and foreseeable misuse — a teacher-grading tool predictably repurposed to rank teachers, a companion chatbot predictably used by minors. Borrowed from product-safety law, the concept prevents the oldest trick in the compliance book: writing a narrow intended purpose and disclaiming everything users actually do.

EU AI Act Art 3(13), Art 9

prohibited AI practices

The apex of the risk pyramid — applicable since 2 February 2025, carrying the top fine tier (€35M or 7% of worldwide turnover). The list: (a) subliminal or manipulative techniques causing significant harm; (b) exploitation of vulnerabilities of age, disability, or social/economic situation; (c) social scoring; (d) predictive policing based solely on profiling or personality traits; (e) untargeted scraping of facial images for recognition databases — the “Clearview clause”; (f) emotion recognition in workplaces and schools (medical/safety exception); (g) biometric categorisation inferring sensitive attributes; (h) real-time remote biometric identification for law enforcement in public, with three narrow exceptions. The Digital Omnibus added a ninth: AI-generated NCII/CSAM (from 2 December 2026). Commission guidelines (Feb 2025) map the boundary cases.

EU AI Act Art 5; C(2025) 884 guidelines

emotion recognition

Defined at Art 3(39) as identifying or inferring emotions or intentions of natural persons on the basis of their biometric data. Its legal treatment is a masterclass in the Act’s layered design: prohibited in workplaces and education institutions except for medical or safety reasons (Art 5(1)(f)); high-risk in the remaining permitted contexts (Annex III point 1); and subject to deployer disclosure to exposed persons wherever used (Art 50(3)). The scientific validity of emotion inference is itself contested — recital-level scepticism that shaped the near-ban.

EU AI Act Art 3(39), Art 5(1)(f), Art 50(3)

biometric categorisation

Sorting people by biometrics rather than identifying them. When the inferred categories are race, political opinions, trade-union membership, religious or philosophical beliefs, sex life or sexual orientation, the practice is prohibited (Art 5(1)(g), with a narrow lawful dataset-labelling exception in law enforcement). Categorisation by non-sensitive attributes sits in Annex III point 1 as high-risk, and deployers owe disclosure under Art 50(3). The identification/categorisation distinction matters: a system that never learns who you are can still infer what you are — which is precisely the harm the ban targets.

EU AI Act Art 3(40), Art 5(1)(g)

quality management system (Art 17) (QMS)

Art 17 requires every high-risk provider to operate a QMS covering: regulatory-compliance strategy, design and QA procedures, test and validation procedures, technical standards, data management, the Art 9 risk-management system, post-market monitoring, incident reporting, communication with authorities, documentation control, resource management, and an accountability framework. Microenterprises may satisfy it in simplified form (Art 63). It is the Act’s bridge to management-system standards — prEN 18286 is being drafted to give it harmonized-standard backing, and ISO/IEC 42001 covers much (not all) of the same ground at organisational level.

EU AI Act Art 17

EU AI database

A Commission-run, largely public database (Arts 49, 71). Providers register Annex III high-risk systems before placing them on the market; providers who invoke the Article 6(3) filter must register those filtered systems too — creating a paper trail authorities can audit. Public bodies deploying high-risk systems register their use, and a non-public section holds law-enforcement and migration entries. The design bet: sunlight as an enforcement multiplier, letting civil society see where high-risk AI operates.

EU AI Act Arts 49, 71

EU declaration of conformity

The document behind the CE mark (Art 47, content per Annex V): system identification, the provider’s name, a statement of conformity with the AI Act (and any other Union law requiring one), references to harmonized standards or common specifications applied, and — where a notified body was involved — its identity and certificate. One copy per system, kept ten years, provided to authorities on request. Signing it converts compliance from an engineering claim into a legal undertaking by a named legal person — the anchor for later enforcement and liability.

EU AI Act Art 47, Annex V

right to explanation (Art 86)

Art 86 gives any person subject to a decision taken by a deployer on the basis of output from an Annex III high-risk system, which produces legal effects or similarly significantly affects them, the right to obtain from the deployer a clear and meaningful explanation of the role of the AI system in the decision-making procedure and the main elements of the decision. It complements, rather than duplicates, GDPR Art 22 safeguards: Art 22 restricts solely automated decisions; Art 86 attaches to human-made decisions based on high-risk AI output. Alongside Art 85 (complaints) and Art 87 (whistleblowers), it is one of the Act’s few individual-facing remedies.

EU AI Act Art 86

GPAI Code of Practice

Drawn up under Art 56 through a multistakeholder process facilitated by the AI Office and published in July 2025, the Code operationalises the GPAI obligations in three chapters: Transparency (Annex XI/XII documentation), Copyright (the Art 53 copyright policy including TDM opt-out respect), and Safety & Security (the Art 55 systemic-risk duties). Signing is voluntary, but adherence functions as a presumption-style demonstration of compliance until harmonized standards arrive; non-signatories must prove equivalent “alternative adequate means.” Do not name signatories from memory — the list evolves; check current status.

EU AI Act Art 56; GPAI Code of Practice (Jul 2025)

New Legislative Framework (NLF)

The 2008 modernisation of how the EU regulates products, and the single most useful key for decoding the AI Act’s design. The law states essential requirements (Arts 8–15); harmonized standards supply the technical detail; applying them earns a presumption of conformity; a conformity assessment plus declaration leads to CE marking; market surveillance polices the aftermarket. Toys, lifts, medical devices and now high-risk AI all follow this template — which explains why the AI Act reads like product law with fundamental rights grafted on, and why lawyers from the CE-marking world adapted to it fastest.

Decision 768/2008/EC; EU AI Act recitals

placing on the market

A term of art from product law (Art 3(9)–(10)): the first supply of an AI system for distribution or use on the Union market in the course of commercial activity, whether paid or free. Its sibling, putting into service, covers supplying a system directly to a deployer or using it in-house for its intended purpose. Together they define the Act’s temporal trigger — conformity assessment, registration and documentation must be complete before either happens — and its extraterritorial hook: a provider anywhere in the world who places a system on the EU market, or whose system’s output is used in the Union, is caught.

EU AI Act Art 3(9)–(11), Art 2

AI management system (AIMS)

What ISO/IEC 42001 certifies: not any AI product, but the organisational machinery for developing, providing or using AI responsibly — policy and objectives set by leadership, risk and impact assessment processes, lifecycle and data controls, competence and awareness, monitoring, internal audit, management review, and continual improvement, all arranged on the Plan-Do-Check-Act cycle. It deliberately mirrors the ISMS of ISO 27001 so organisations can bolt AI governance onto existing management systems. The critical scoping insight: a certified AIMS proves your process is sound, not that any particular model is safe, fair, or AI Act-compliant.

ISO/IEC 42001:2023

management system standard (MSS)

A standard about organisations, not products. Every modern ISO MSS shares the Harmonized Structure (formerly Annex SL): ten clauses running Context (4), Leadership (5), Planning (6), Support (7), Operation (8), Performance evaluation (9), Improvement (10). This shared skeleton is why ISO 9001, 27001 and 42001 feel like siblings and can be run as one integrated management system with combined audits. MSS conformity is demonstrated through third-party certification against the “shall” statements — a fundamentally different animal from product conformity assessment under EU law, however similar the vocabulary sounds.

ISO/IEC Directives Part 1, Annex SL

Statement of Applicability (SoA)

Produced during risk treatment (clause 6.1.3): for every control in Annex A, the organisation records whether it is applicable and justifies inclusions and exclusions. A pure AI user might justifiably exclude development-lifecycle controls; excluding data-quality controls while fine-tuning models would be indefensible. Auditors read the SoA first because it is where scoping games surface — an SoA that excludes half of Annex A with boilerplate justifications signals a paper AIMS. Inherited directly from ISO 27001 practice, where the SoA plays the same gatekeeping role.

ISO/IEC 42001:2023, cl 6.1.3

Annex A control (ISO 42001)

ISO 42001’s normative control set, spanning nine groupings: A.2 AI policies, A.3 internal organisation (roles, concern reporting), A.4 resources (data, tooling, compute, people), A.5 impact assessment, A.6 AI system lifecycle, A.7 data for AI systems, A.8 information for interested parties, A.9 responsible use, A.10 third-party and customer relationships. Annex B gives implementation guidance per control; Annex C offers objectives and risk sources to seed assessments. Controls are not automatically mandatory — the Statement of Applicability determines which apply — but every exclusion needs a defensible justification. Do not confuse with the EU AI Act’s Annexes, which are lists in a statute.

ISO/IEC 42001:2023, Annexes A–C

nonconformity (major / minor)

The currency of certification audits. A major nonconformity is a total absence of a required element or a systemic breakdown (no impact-assessment process at all; an SoA that was never approved) — it blocks certification until corrected and verified. A minor nonconformity is an isolated lapse in an otherwise functioning process (one supplier review missed). Below both sit observations or opportunities for improvement, which carry no obligation. Each nonconformity triggers the clause 10 workflow: correct, analyse root cause, act to prevent recurrence, and verify effectiveness — auditors return to check exactly that.

ISO/IEC 17021-1; ISO/IEC 42001 cl 10

certification audit (Stage 1 / Stage 2)

Stage 1 reviews documentation and readiness — scope, policy, SoA, risk methodology — and flags gaps before the real test. Stage 2 samples implementation evidence: risk and impact assessment records, lifecycle documentation, training records, supplier controls, management-review minutes, internal-audit results. Findings are classified as major/minor nonconformities or observations; a positive certification decision starts a 3-year cycle with annual surveillance audits and recertification at the end. Since ISO/IEC 42006:2025, bodies performing these audits must themselves prove AI-specific competence.

ISO/IEC 17021-1; ISO/IEC 42006:2025

accreditation

The top of the assurance chain. National accreditation bodies (UKAS, DAkkS, ANAB…) assess certification bodies against ISO/IEC 17021-1 plus, for AIMS certification, ISO/IEC 42006; the IAF multilateral arrangement makes accredited certificates recognisable across borders. The practical buyer’s lesson: an “ISO 42001 certificate” from an unaccredited issuer is a logo, not assurance. The same chain exists in EU product law with different names — notifying authorities designate notified bodies — and confusing the two ecosystems is a classic AIGP exam trap.

ISO/IEC 17011; IAF MLA

certification body (CB)

The commercial auditors of the management-system world. A CB plans and staffs the certification audit (Stage 1 / Stage 2), classifies findings, makes the certification decision, and runs the surveillance cycle. For AIMS work, ISO/IEC 42006:2025 adds AI-specific requirements: audit teams competent in machine learning, data governance and AI risk; use of technical experts; audit-time determination reflecting AI system complexity; and precise scope statements. CBs certify organisations against voluntary standards; notified bodies assess products against binding law — different legal universes that happen to rhyme.

ISO/IEC 17021-1; ISO/IEC 42006:2025

ISO/IEC 42001

Published December 2023, ISO/IEC 42001 specifies requirements for establishing, implementing, maintaining and continually improving an AI management system, applicable to any organisation providing, developing or using AI. Structure: clauses 4–10 on the Harmonized Structure, ~38 Annex A controls, Annex B guidance, Annex C objectives and risk sources, Annex D sector integration. Its market power comes from certification (accredited under ISO/IEC 42006) and from legal gravity: US state laws cite it in safe-harbor provisions, and it maps substantially — though not completely — onto EU AI Act Arts 9–17. It certifies the management system, never the AI product.

ISO/IEC 42001:2023

ISO/IEC 23894

Published 2023, ISO/IEC 23894 takes the generic ISO 31000 skeleton — scope/context/criteria, risk assessment, treatment, monitoring, communication — and layers AI-specific guidance onto every step. Its annexes are the working assets: Annex A lists AI-related objectives (fairness, safety, transparency, accountability…), Annex B catalogues AI risk sources (data quality, drift, opacity, level of automation, technology readiness), and Annex C maps risk management onto the AI lifecycle. It is guidance, not certifiable — organisations implement it inside a certifiable shell like ISO 42001, whose clause 6 risk process it fleshes out.

ISO/IEC 23894:2023

ISO/IEC 22989

The dictionary the rest of the standards stack builds on (published 2022, normatively referenced by ISO 42001). It defines core terminology — AI system, machine learning, training/validation/test data, autonomy versus heteronomy — plus a stakeholder role taxonomy (AI provider, producer, developer, customer, user, subject, relevant authorities) and a canonical AI lifecycle: inception, design & development, verification & validation, deployment, operation & monitoring, re-evaluation, retirement. Beware the false friend: 22989’s “AI provider” is a functional role description, while the EU AI Act’s provider is a legal duty-holder — overlapping, not identical.

ISO/IEC 22989:2022

AI system impact assessment (ISO/IEC 42005)

ISO/IEC 42005:2025 (published May 2025) standardises how to assess an AI system’s impacts: document the system and context, identify affected interested parties, analyse benefits and harms across dimensions (fairness, safety, privacy, environment, human autonomy, wellbeing), rate severity and likelihood, record outcomes and follow-ups. It plugs into ISO 42001 via clause 6.1.4, clause 8.4 and control A.5. The key distinction it enforces: risk assessment asks what could hurt the organisation; impact assessment asks who the system could hurt. Cousins with legal instruments — the GDPR DPIA and AI Act FRIA — but voluntary and framework-neutral.

ISO/IEC 42005:2025

ISO/IEC 42006

Published 7 July 2025 to answer an awkward question: who is competent to audit an AIMS? 42006 adds AI-specific requirements to the generic certification-body rules — audit teams must demonstrate competence in ML, data governance and AI risk; technical experts may be drafted in; audit time must reflect the number and complexity of AI systems in scope; and certificate scope statements must be precise about what was actually audited. Its arrival converts early, unaccredited 42001 certificates into a market with real accreditation discipline; transition arrangements for existing certificates vary — check current IAF status.

ISO/IEC 42006:2025

Plan-Do-Check-Act (PDCA)

Deming’s improvement loop, institutionalised in management-system standards. In ISO 42001 terms: Plan = clauses 4–6 (context, leadership, risk planning), Do = clauses 7–8 (support and operation), Check = clause 9 (monitoring, internal audit, management review), Act = clause 10 (nonconformity handling, continual improvement). Its governance significance: AI risk is never “done” — models drift, uses change, laws move — so frameworks institutionalise a wheel that keeps turning rather than a gate you pass once. NIST’s continuous Map-Measure-Manage cycle inside a Govern frame is the same instinct in American clothing.

ISO Harmonized Structure

Harmonized Structure (Annex SL)

The ISO drafting rule (formerly “Annex SL”, now the Harmonized Structure) that every management system standard use identical high-level clauses, core text and common terms: 4 Context, 5 Leadership, 6 Planning, 7 Support, 8 Operation, 9 Performance evaluation, 10 Improvement. Each standard then inserts discipline-specific requirements — 42001 adds AI role determination, impact assessment (6.1.4) and AI-specific controls. The payoff is integrated management systems: one policy architecture, one audit programme, one improvement loop covering quality, security, privacy and AI at once. Not to be confused with EU harmonized standards — an entirely different mechanism that happens to share the adjective.

ISO/IEC Directives Part 1, Annex SL

ISO 31000

ISO 31000 defines risk as the effect of uncertainty on objectives and organises risk management into principles, a framework (leadership, integration, design, implementation, evaluation, improvement) and a process: establish scope/context/criteria → risk assessment (identification, analysis, evaluation) → treatment → monitoring & review, with communication and recording throughout. It matters for AI because ISO/IEC 23894 is literally ISO 31000 with AI annotations, and because it supplies the base vocabulary — risk source, event, likelihood, consequence, risk owner — that AI frameworks reuse. One AI-era stretch: 31000’s lens is organisational objectives, while AI governance must also weigh harms to other people, which is why impact assessment grew into a separate discipline.

ISO 31000:2018

risk source

ISO 31000 vocabulary: the element which, alone or in combination, can give rise to risk. The AI standards make it concrete — ISO 23894 Annex B catalogues AI-specific risk sources: complexity of the environment, lack of transparency and explainability, level of automation, machine-learning specifics (data quality, drift, adversarial inputs), hardware issues, lifecycle gaps, technology readiness. The practical discipline: a risk register that lists “the model might be biased” has recorded a risk; tracing it to its sources (unrepresentative training labels, proxy features) is what makes treatment possible.

ISO 31000:2018; ISO/IEC 23894 Annex B

likelihood

ISO 31000 chose “likelihood” over “probability” on purpose: it covers whatever chance description fits — qualitative bands (rare/possible/likely), frequencies, or true probabilities. In AI risk work, likelihood estimation is notoriously hard: harms are often diffuse (societal bias), novel (no actuarial base rate), or adversary-driven (probability of a prompt-injection attack is a function of attacker interest, not chance). That is why frameworks pair likelihood scoring with humility — NIST’s Measure function explicitly addresses risks that resist quantification — and why severity often dominates AI risk decisions.

ISO 31000:2018; ISO Guide 73

consequence

The other axis of the classic risk matrix (risk ≈ likelihood × consequence). ISO 31000 notes consequences can be certain or uncertain, positive or negative, cascading and cumulative. AI governance stretched the concept in one crucial way: consequence scales must include harms to individuals, groups, and society — wrongful arrest, denied benefits, chilled speech — not just financial or reputational damage to the deploying organisation. A consequence scale that stops at “regulatory fine” will systematically underweight exactly the harms regulators built AI law to prevent.

ISO 31000:2018; ISO Guide 73

risk treatment

The decision step after assessment. The canonical options: avoid (do not deploy the use case), modify (mitigations — human oversight, guardrails, retraining, threshold changes), share (insurance, contractual allocation to vendors), or retain (accept, documented, by someone with authority to accept it). NIST’s Manage 1 mirrors the same quartet as mitigate/transfer/avoid/accept. Treatment produces residual risk, which must be explicitly evaluated against acceptance criteria — the EU AI Act’s Art 9 encodes the same loop as a binding requirement for high-risk systems, judged against the state of the art.

ISO 31000:2018; EU AI Act Art 9

residual risk

No mitigation stack drives AI risk to zero; what is left is residual risk. Good governance makes its acceptance an explicit, owned decision: who signed off, on what evidence, under what conditions, reviewed when. EU AI Act Art 9 requires residual risks of high-risk systems to be judged acceptable and communicated to deployers via the instructions for use. The anti-pattern every auditor knows: mitigations listed, residual risk never re-scored, acceptance implied by silence — which converts a risk decision into an accident report waiting for its facts.

ISO Guide 73; EU AI Act Art 9

risk appetite / risk tolerance

Appetite is the strategic statement (“we accept moderate model-error risk in marketing, near-zero in credit decisions”); tolerance is the operational boundary that makes it testable (maximum acceptable false-positive disparity, minimum accuracy before rollback). NIST AI RMF places risk-tolerance setting in Govern; ISO 38507 makes it a governing-body duty that cannot be delegated to engineers or vendors. Where no explicit appetite exists, every team invents its own — which is how an organisation discovers, mid-incident, that its real risk appetite was set by its most optimistic product manager.

ISO Guide 73; NIST AI RMF Govern

verification vs validation

A confusable pair the standards world polices strictly. Verification confirms, through objective evidence, that specified requirements are fulfilled — the model meets its declared accuracy on the test set. Validation confirms the system is suitable for its intended use in its real context — the triage model actually works on this hospital’s population. A model can verify perfectly and validate disastrously (Epic’s sepsis model performed far worse in external validation than its specs suggested). ISO 22989’s lifecycle gives V&V its own stage; TEVV in NIST usage bundles the same activities with testing and evaluation across the lifecycle.

ISO/IEC 22989:2022

NIST AI Risk Management Framework (AI RMF)

Mandated by the National AI Initiative Act of 2020 and released 26 January 2023, the AI RMF is voluntary, rights-preserving, and sector-agnostic. Govern builds the organisational foundation (policies, accountability, culture, third-party risk); Map establishes context and identifies risks; Measure analyses and tracks them against the seven trustworthy AI characteristics; Manage prioritises and responds. Its legal weight is de facto: state statutes (Texas TRAIGA, the original Colorado AI Act) cite it as a safe harbor or defence, and regulators treat it as a reasonableness benchmark. As of September 2026 it is under revision pursuant to the White House AI Action Plan — check current status before citing its text.

NIST AI RMF 1.0 (2023)

AI RMF profile

The RMF’s customisation mechanism, borrowed from the NIST Cybersecurity Framework. A use-case profile selects and prioritises subcategories for a context (hiring, lending, critical infrastructure — NIST floated a Critical Infrastructure profile concept in April 2026); a temporal profile pair captures the current state versus the target state, making the gap a workplan. The flagship example is the Generative AI Profile. Profiles are how a 100-page framework becomes a checklist your organisation can actually run.

NIST AI RMF 1.0, ch 6

AI RMF Playbook

The operational companion published alongside the framework in NIST’s Trustworthy and Responsible AI Resource Center. For each RMF subcategory it offers suggested actions (not mandatory — NIST is explicit that organisations pick what fits), transparency and documentation prompts, and curated references. Its quiet importance: when a statute or contract says “consistent with the NIST AI RMF,” auditors and opposing counsel reach for the Playbook to decide what consistency looks like in practice — voluntary text acquiring evidentiary gravity.

NIST AI RMF Playbook

Generative AI Profile (NIST-AI-600-1)

Published 26 July 2024 under an EO 14110 directive (and surviving that order’s revocation), NIST-AI-600-1 names twelve generative-AI risks: CBRN information or capabilities; confabulation; dangerous or hateful content; data privacy; environmental impacts; harmful bias and homogenization; human-AI configuration (automation bias, emotional entanglement); information integrity; information security (prompt injection, poisoning, extraction); intellectual property; obscene or abusive content including NCII/CSAM; and value-chain/component-integration risk. Each maps to suggested actions across Govern/Map/Measure/Manage — the closest thing US practice has to a canonical GenAI risk checklist.

NIST-AI-600-1 (2024)

trustworthy AI characteristics

The RMF’s quality model: valid and reliable (the precondition), safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Two governance lessons ride along. First, the characteristics trade off — maximising explainability can cost accuracy; hardening privacy can hinder bias measurement — so trustworthiness is a balancing exercise, not a checklist. Second, Measure 2 structures evaluation around exactly these seven, making them the de facto test plan for US-style AI assurance. ISO’s TR 24028 covers similar ground with a slightly different attribute list — expect crosswalk questions.

NIST AI RMF 1.0, ch 3

TEVV (Test, Evaluation, Verification, and Validation)

NIST’s umbrella for the empirical side of AI assurance, positioned as a cross-cutting activity touching every lifecycle stage rather than a phase you pass once. TEVV spans pre-deployment benchmark testing, red-teaming, bias measurement, robustness probing, and post-deployment monitoring and re-validation. The governance point: paperwork without TEVV is faith-based compliance. Every serious regime hard-wires it — EU AI Act Arts 9(6)–(8) and 15 demand testing against defined metrics; SR 11-7 calls the same discipline model validation.

NIST AI RMF 1.0

systemic bias

The first of NIST SP 1270’s three bias categories: bias originating in the procedures and practices of institutions and society — redlining’s legacy in housing data, occupational segregation in salary data, policing intensity in arrest records. It is the hardest category to fix because the data is an accurate record of a biased world: a model that faithfully learns historical hiring decisions faithfully reproduces their discrimination (Amazon’s scrapped recruiting tool is the canonical case). SP 1270’s core message follows: bias is a socio-technical problem, and purely technical debiasing cannot resolve what is structural.

NIST SP 1270 (2022)

statistical / computational bias

SP 1270’s second category — the errors engineers can actually instrument: selection and sampling bias (whose data got collected), measurement bias (the label is a flawed proxy for the target — Obermeyer’s healthcare study, where “cost” stood in for “need,” systematically deprioritised Black patients), representation bias (subgroups too sparse to learn well), and aggregation bias (one model forced across heterogeneous populations). Unlike systemic bias, computational bias yields to technical remedies — better sampling, label audits, disaggregated evaluation — which is why frameworks demand exactly those artifacts.

NIST SP 1270 (2022)

human-cognitive bias

SP 1270’s third category, and the one governance programs most often forget: the biases of the people around the model. Developers anchor on early hypotheses and confirm them; teams succumb to groupthink; funding pressure shades findings (funding bias); and users defer to machine output (automation bias / automation complacency), converting “human review” into ratification. It enters at every lifecycle stage — problem framing, label definition, threshold setting, override behaviour — which is why SP 1270 prescribes diverse teams, structured challenge, and measuring real disagreement rates between humans and the model.

NIST SP 1270 (2022)

confabulation

The GAI Profile’s name for what practitioners call hallucination — chosen because “hallucination” anthropomorphises (models do not perceive) while confabulation, borrowed from psychology, describes confidently filling gaps with fabrication. Legally it is no longer academic: Mata v. Avianca sanctioned lawyers for filing invented citations; Moffatt v. Air Canada bound an airline to its chatbot’s fabricated refund policy. Governance treats confabulation as an inherent property to be managed — grounding/RAG, output verification for consequential uses, disclosure — never a bug a vendor promise can wave away.

NIST-AI-600-1

algorithmic discrimination

The organising concept of the US state-law wave: the condition in which an automated system’s use contributes to unjustified differential treatment or impact disfavouring people based on race, sex, age, disability, religion, or other protected characteristics. The (repealed) Colorado AI Act built a duty of reasonable care around it; Illinois HB 3773 writes it into employment-rights law; the Blueprint for an AI Bill of Rights made “algorithmic discrimination protections” a headline principle. It deliberately spans both US doctrines — disparate treatment and disparate impact — because a model can discriminate without any protected attribute in the feature set, via proxies.

Colorado SB 24-205 (repealed); Blueprint for an AI Bill of Rights

consequential decision

The US statutory analogue of the EU’s Annex III logic: instead of listing risky systems, state laws list decisions whose automation triggers duties. The canonical domains recur across Colorado’s frameworks, California’s ADMT regulations, Illinois and Texas: employment, education, housing, credit/financial services, healthcare, insurance, legal services, and government benefits. Definitions differ at the edges — which is why multistate compliance teams maintain a crosswalk of exactly what each statute counts. When a law says “consequential decision,” the next questions are always: made or substantially assisted by AI? and with what notice, explanation, and appeal rights?

Colorado SB 26-189; CCPA ADMT regulations

disparate impact

The US doctrine (from Griggs v. Duke Power, codified in Title VII) most relevant to AI: liability can attach where a neutral practice produces disproportionate adverse effects on a protected class and cannot be justified by business necessity with no less-discriminatory alternative. It fits algorithmic harm perfectly — models rarely intend anything, but they routinely produce skewed outcomes through proxies and skewed training data. Screening tests like the four-fifths rule operationalise it. Contrast disparate treatment, which requires intentional differential treatment — and note the trap: some technical bias mitigations that explicitly use protected attributes can raise disparate-treatment questions.

Griggs v. Duke Power (1971); Title VII

disparate treatment

The classic discrimination theory: intentional differential treatment based on a protected characteristic. For AI it bites in two directions. A system explicitly using race, sex, or age as a decision feature is straightforward disparate treatment (the EEOC’s iTutorGroup settlement involved software auto-rejecting older applicants). More subtly, fairness interventions that apply group-specific thresholds or quotas to fix disparate impact may themselves constitute disparate treatment — the doctrinal tension at the heart of US algorithmic-fairness practice, and the reason bias mitigation choices belong in legal review, not just data-science backlogs.

Title VII; EEOC v. iTutorGroup (2023)

four-fifths rule

From the 1978 Uniform Guidelines on Employee Selection Procedures: compare selection rates across groups; if any group’s rate is less than four-fifths (80%) of the most-favoured group’s, agencies treat it as evidence of adverse impact. If 60% of male applicants pass a screen and 40% of women do, the impact ratio is 0.67 — flagged. NYC Local Law 144’s bias audit (NYC LL144) requires publishing exactly these impact ratios. Know its limits: it is a rule of thumb, not a legal safe harbor — small samples make it unstable, statistical-significance tests can point the other way, and passing 80% does not immunise a practice that fails under closer analysis.

29 CFR §1607.4(D)

bias audit (NYC LL144)

New York City’s Local Law 144 (enforced since 5 July 2023) prohibits employers from using an AEDT unless it has undergone an independent bias audit within the past year: computing selection-rate impact ratios (for screening) or scoring-rate ratios (for ranking) by sex, race/ethnicity, and intersectional categories, with a summary published on the employer’s website and 10 business days’ notice to candidates. The first legally mandated algorithmic audit regime in the US — and a cautionary tale: studies found widespread non-compliance, small-sample instability, and audits scoped to whatever data the vendor supplied. “Audit” here means a defined statistical exercise, not a full assurance engagement.

NYC Local Law 144 (2021); 6 RCNY §5-300

automated employment decision tool (AEDT)

Defined in NYC Local Law 144 as a computational process derived from machine learning, statistical modelling, or data analytics that issues simplified outputs (scores, classifications, rankings) used to substantially assist or replace discretionary decision-making in employment decisions. The “substantially assist” qualifier is the fighting ground: DCWP rules read it as relying solely on the output, weighting it most heavily, or using it to overrule humans — a definition narrow enough that many employers argued their tools fell outside it, one documented reason compliance rates disappointed. The concept previews a broader family: Colorado’s and California’s ADMT definitions do similar work with different words.

NYC Local Law 144; DCWP Final Rules (2023)

automated decision-making technology (ADMT)

The term US privacy-derived AI regulation converged on. California’s CCPA regulations define ADMT as technology that processes personal information and uses computation to execute a decision, replace human decision-making, or substantially facilitate it — triggering pre-use notice, opt-out rights, and access/explanation rights for “significant decisions,” with full ADMT compliance required by 1 January 2027. Colorado’s SB 26-189 (which replaced the Colorado AI Act in May 2026) built its disclosure framework on the same concept: point-of-interaction notice, plain-language explanations within 30 days of adverse consequential decisions, human review and data-correction rights. ADMT regimes regulate the decision pipeline; the EU regulates the system — same harms, different legal geometry.

CCPA ADMT regulations (2025); Colorado SB 26-189

SR 11-7 (model risk management)

Supervisory guidance issued by the Fed and OCC in 2011, born of the financial crisis, and still the most battle-tested model-governance regime anywhere. Core machinery: a broad definition of “model,” independent validation (conceptual soundness, ongoing monitoring, outcomes analysis), enterprise model inventories, effective challenge by qualified independent parties, and governance through the three lines of defense. Banks simply extended it to ML and AI models, which is why bank AI governance matured a decade ahead of everyone else — and why concepts like inventory, validation, and champion/challenger flowed from SR 11-7 into the NIST AI RMF and state statutes.

Fed SR 11-7 / OCC 2011-12

algorithmic disgorgement

The most feared sentence in an FTC consent order: destroy models derived from unlawfully collected data, not merely the data itself. Deployed in Cambridge Analytica, Everalbum (2021, face-recognition models from photos processed without consent), and WW/Kurbo (2022, models trained on children’s data collected in violation of COPPA). The economic logic is deterrence: if tainted data survives inside a valuable model, deletion orders are toothless. For governance teams it converts data provenance into an existential control — you cannot surgically remove one dataset from trained weights, so the whole asset is at risk (see machine unlearning).

FTC consent orders: Everalbum (2021), WW/Kurbo (2022)

frontier model

The policy word for the most capable general-purpose models, usually operationalised by training-compute thresholds and developer revenue tiers: the EU presumes systemic risk at 10²⁵ FLOPs; California’s SB 53 (Transparency in Frontier AI Act, effective January 2026) defines “frontier developer” by compute with a “large frontier developer” revenue tier; New York’s RAISE Act (effective January 2027) follows similar logic. Frontier obligations centre on published safety frameworks, transparency reports, capability evaluations, and incident reporting — governance of potential catastrophic capability rather than deployed use cases. Sibling terms: the EU’s GPAI-with-systemic-risk, US federal “dual-use foundation model.”

California SB 53 (2025); NY RAISE Act

adverse action notice

Under ECOA/Regulation B (and FCRA for consumer reports), a creditor taking adverse action must give the applicant the specific principal reasons — not “your score was low,” but the actual factors. CFPB Circulars 2022-03 and 2023-03 closed the AI escape hatch: a lender cannot claim its model is too complex to explain; if you cannot generate accurate reason codes, you may not use the model. This single requirement drove much of applied explainability in US finance — SHAP-derived reason codes exist because adverse-action law demands them. State ADMT laws now generalise the pattern beyond credit.

ECOA / Reg B; CFPB Circular 2022-03

model card

Proposed by Mitchell et al. (2019) as nutrition labels for models, now the lingua franca of model documentation: intended use and users, out-of-scope uses, training data summary, evaluation results disaggregated across demographic groups and conditions, limitations, and ethical considerations. Hugging Face made them a release norm; procurement teams now demand them from vendors; state laws (Colorado SB 26-189’s developer documentation duties) effectively legislate their content. Quality drift is the known failure: a template dutifully filled with marketing prose. Distinguish the system card (system-level, safety-evaluation-focused) and the regulator-facing Annex IV tech file.

Mitchell et al., Model Cards for Model Reporting (2019)

system card

Popularised by the GPT-4 and Claude system cards: documentation of the system — model plus safety training, filters, tool access, usage policies — rather than raw model weights. Typical contents: capability and safety evaluations (including red-team findings), deployed mitigations and their measured effects, residual risks, and usage restrictions. The model/system distinction is load-bearing in law too: the EU AI Act regulates GPAI models (Arts 51–55) and AI systems (the risk tiers) on separate tracks. A deployer procuring a foundation-model-based product should ask for the system card — the model card alone describes an ingredient, not the dish.

OpenAI GPT-4 System Card (2023); Anthropic system cards

datasheet for datasets

Gebru et al.’s (2018) proposal, borrowed from electronics: every dataset ships with a datasheet answering structured questions — why was it created, what does it contain, how was it collected and from whom, what cleaning and labelling were applied, what uses are appropriate or inadvisable, who maintains it. It attacks the root of many AI failures: models inheriting undocumented flaws from data nobody could interrogate. EU AI Act Annex IV expects datasheet-like data documentation in the tech file, and ISO 42001 control A.7 requires data provenance and quality processes. Variants: data statements (NLP), Data Nutrition Labels.

Gebru et al., Datasheets for Datasets (2018)

red-teaming

Imported from military exercises and security practice into AI assurance: scoped, authorised attempts to make a system fail — eliciting prohibited content, jailbreaking guardrails, extracting training data, probing dangerous capabilities (CBRN uplift, offensive cyber). Now a legal artifact, not just craft: EU AI Act Art 55 requires adversarial testing of systemic-risk GPAI models; the GPAI Code of Practice and frontier safety frameworks operationalise it; NIST’s GAI Profile suggests it pre-deployment. Good programs define scope and rules of engagement, mix manual and automated attack generation, track coverage, and wire findings into release gates — red-teaming that never blocks a launch is theatre.

EU AI Act Art 55; NIST-AI-600-1

benchmark

The measuring sticks of AI progress (MMLU, HELM suites, coding and safety benchmarks) and increasingly of law — EU Annex XIII lists benchmark performance among systemic-risk designation criteria. Governance users must know the failure modes: contamination (test items leak into training data, inflating scores), saturation (top models cluster at the ceiling, erasing discrimination), and Goodhart’s law (once a benchmark becomes the target, teaching-to-the-test breaks its meaning). A benchmark score is evidence about a task distribution, not about your deployment — which is why serious assurance pairs public benchmarks with context-specific evaluation (evals).

EU AI Act Annex XIII

evaluation (evals)

The umbrella practice: capability evals (what can it do), safety evals (will it do harm — including dangerous-capability evals for bio, cyber, and autonomy risks), and deployment evals (does it work here, for these users, at this threshold). Methods range from static benchmarks through human review to LLM-as-judge — the last cheap and scalable but with documented failure modes (position bias, self-preference, sycophancy toward verbose answers). Frontier law leans on evals hard: EU Art 55 demands state-of-the-art model evaluations, and safety-institute pre-release testing is essentially third-party evals. Rule of thumb: no eval, no evidence; no evidence, no governance.

EU AI Act Art 55; NIST GenAI program

drift monitoring

The operational practice behind every post-deployment obligation. Distinguish the species: data drift (input distribution shifts — new demographics, new products), concept drift (the input-outcome relationship changes — fraud tactics evolve), and upstream/schema drift (a pipeline or field definition silently changes). Complication: ground-truth labels often arrive late (loan defaults take months), so proxies like the population stability index stand in. Governance-grade monitoring defines metrics and thresholds before deployment, alerts both first and second line, and pre-commits responses — investigate, revalidate, retrain, or rollback. This is how you satisfy EU Art 72 and catch the failures pre-launch testing structurally cannot.

EU AI Act Art 72 (context)

AI incident response

Security incident response, adapted for systems that fail statistically. AI-specific containment options: rollback to a prior model version, model disable / kill switch, guardrail hotfix, threshold tightening, or routing all decisions to humans. The response plan must map regulatory clocks in advance — EU Art 73’s 15/10/2-day ladders, California SB 53’s Cal OES reporting for frontier safety incidents, GDPR’s 72-hour breach window where personal data is involved — because you cannot design a notification workflow mid-crisis. Mature programs add near-miss registries and blameless postmortems that feed findings back into risk assessment.

EU AI Act Art 73; Cal SB 53

AI incident

Broader than the EU’s serious incident term of art: any event where an AI system caused, contributed to, or nearly caused harm — from a chatbot giving dangerous advice to a biased screen quietly rejecting a class of applicants. The OECD distinguishes incidents from hazards (potential-harm events). Definitional breadth matters operationally: organisations that only track legally reportable events lose the near-misses where cheap learning lives. Public infrastructure exists to pool that learning — the AI Incident Database and the OECD AI Incidents Monitor — mirroring aviation’s safety-reporting culture.

OECD AI incident definitions (2024)

AI Incident Database (AIID)

Run by the Responsible AI Collaborative and seeded by Partnership on AI work, the AIID collects and taxonomises real-world AI incidents (about 1,700 by late 2026) — face-recognition wrongful arrests, welfare-scoring scandals, fabricated legal citations. Its aviation-inspired premise: systems fail in patterns, so index the failures and let every deployer learn from everyone’s accidents. Governance uses: seeding risk identification workshops (“has our use case failed elsewhere?”), evidencing foreseeability, and training. Companions: the OECD AI Incidents Monitor (automated news-based tracking) and the MIT AI Risk Repository.

incidentdatabase.ai

AI inventory

You cannot govern what you cannot see. An AI inventory (registry) records, per system: owner, business purpose, model type, data used, vendor, deployment status, risk tier, applicable laws, and links to assessments and monitoring. Hard parts: shadow AI (unsanctioned tools), AI features embedded silently in SaaS, and keeping entries synchronised with reality via attestation cycles. The concept has legal force in multiple regimes — SR 11-7 model inventories, US federal agency use-case inventories under OMB guidance, EU deployer registration duties for public bodies — and every framework (ISO 42001, NIST Govern/Map) presupposes one.

SR 11-7; OMB M-25-21

use-case intake

The choke point that makes governance scale: a structured form (purpose, data, population affected, autonomy level, vendor) feeds a triage that assigns a risk tier and routes accordingly — low-risk uses through a fast lane, consequential ones to full assessment and committee review. Design tensions are real: too heavy and teams route around you (breeding shadow AI); too light and a credit model ships with a chatbot’s review. Good intake maps each use case to regulatory categories at the door — Annex III area? consequential decision? AEDT? — so classification work happens once, early, and on the record.

NIST AI RMF Map function (context)

three lines of defense

Banking’s governance architecture, transplanted to AI via SR 11-7 and ISO 38507-style board guidance. First line: the teams building and deploying models own their risks and controls — they write the model card. Second line: risk and compliance functions set standards, challenge, and veto — they review the model card. Third line: internal audit independently assures the whole process works — they audit how model cards get written and reviewed. Classic anti-patterns: second line doing first-line work (then grading its own homework), or audit designing controls it must later assure. The model gives AI governance its org chart; RACI gives it the row-level detail.

IIA Three Lines Model; SR 11-7

RACI (Responsible, Accountable, Consulted, Informed)

The accountability matrix: for each activity (data approval, validation sign-off, deployment, monitoring, incident escalation) name who is Responsible (does the work), Accountable (one — and only one — owner who answers for it), Consulted, and Informed. In AI governance it operationalises the principle every framework repeats: accountability cannot be delegated to a vendor or an algorithm. Failure smells auditors hunt for: two Accountables (nobody decides), zero Accountables (a model with no owner after its creator left), and second-line functions holding the A for first-line work.

Standard project-governance practice

vendor AI risk

Most organisations buy far more AI than they build, so governance is largely supply-chain work. The lifecycle: due diligence (training-data provenance, eval results, subprocessors, certifications like ISO 42001), contracting (audit and evaluation rights, bias-testing representations, no-training-on-customer-data, model-change notification, IP/output indemnities, exit terms), onboarding, continuous monitoring, exit. Legal regimes hard-wire it: EU AI Act Art 25 requires written agreements with component suppliers; deployers inherit duties they can only meet with provider cooperation; NIST Govern 6 and ISO 42001 A.10 make third-party risk a named control area. The trap: AI features appearing inside existing SaaS with no procurement event at all.

EU AI Act Art 25; ISO 42001 A.10

acceptable-use policy (AI) (AUP)

The workhorse document of everyday AI governance: which tools are sanctioned, what data may never enter a prompt (client confidences, personal data, trade secrets), which outputs require human verification before use, and how to disclose AI assistance. Written well, it is specific and role-based (“paste no client data into unapproved chatbots; verify every citation an assistant produces” — the rule Mata v. Avianca’s lawyers needed); written badly, it is an unenforced PDF. Providers publish outward-facing AUPs too, restricting customer uses of their models — the private-ordering layer of AI governance that contracts, not statutes, enforce.

Organisational practice; EU AI Act Art 4 (context)

data protection impact assessment (DPIA)

Required where processing is “likely to result in a high risk to the rights and freedoms of natural persons” — with systematic and extensive profiling producing significant decisions as a named trigger, so most consequential AI lands squarely in scope. Contents: systematic description of the processing, necessity and proportionality assessment, risk analysis, and mitigation measures; residual high risk forces prior consultation with the supervisory authority (Art 36). Its relationship to AI-specific instruments is complementary: the DPIA’s lens is data-protection rights, the FRIA’s is fundamental rights broadly, and ISO 42005’s is framework-neutral impact — mature programs run them as one combined workflow.

GDPR Art 35

algorithmic impact assessment (AIA)

The generic name for impact assessments aimed at automated decision-making, and one concrete famous instance: Canada’s AIA under the Treasury Board Directive on Automated Decision-Making. Federal agencies answer ~65 risk questions and ~41 mitigation questions; the score (with up to a 15% deduction for strong mitigation) lands the system in Impact Level I–IV, each unlocking escalating obligations — peer review, notice, human-in-the-loop for high levels, explanation — and results are published openly. Run at design and again before production. The pattern (questionnaire → tier → proportionate obligations → publication) is the template many jurisdictions and companies now copy.

Canada Directive on Automated Decision-Making

human-in-the-loop (HITL)

The strongest standard oversight pattern: the system recommends, a human decides, nothing happens without that decision. It is what GDPR Art 22 effectively mandates for significant decisions (with the SCHUFA caveat: the human involvement must be meaningful, not a downstream formality) and what Canada’s Directive requires at high impact levels. Its known failure mode is automation bias — at high volume, low friction, or thin training, “in the loop” degrades into rubber-stamping. Tests for meaningfulness: does the reviewer have competence, authority, time, and information to disagree — and do they, measurably, sometimes disagree?

EU AI Act Art 14 (context); GDPR Art 22

human-on-the-loop (HOTL)

The middle of the oversight spectrum: decisions execute automatically, but a human supervises in near-real-time with power to interrupt — fraud systems blocking transactions under analyst watch, content moderation with reviewer escalation. Design questions decide whether it is oversight or decoration: can the monitor actually detect anomalies at machine speed and volume (sampling, alerting, dashboards)? Is intervention fast enough for the harm at stake — for irreversible actions, on-the-loop may arrive too late, arguing for human-in-the-loop gates instead. Beyond both sits human-in-command: humans set the envelope and can decline to use the system at all — the ultimate Art 14 capability.

EU AI Act Art 14 (context)

kill switch

The final rung of intervention: a mechanism to stop the system now — disable the model endpoint, cut an agent’s tool access, fail over to human processing. EU AI Act Art 14 requires oversight persons be able to “intervene… or interrupt the system through a stop button or similar procedure.” Engineering honesty matters: a kill switch that has never been exercised is a hypothesis; one whose activation collapses a business process will not be pressed when needed, so design pairs it with graceful degradation (queueing, manual fallback). For agentic systems, killability gets harder — actions already dispatched to external tools may be irreversible, pushing control upstream to permissions and checkpoints.

EU AI Act Art 14(4)(e)

rollback

The AI-specific containment action security teams envy: when a new model version misbehaves, redeploy the old one. Preconditions are all governance: versioned artifacts (model, data, config, prompts) retained and redeployable; documented baselines proving the prior version was acceptable; and change-management records tying each version to its validation. Champion/challenger deployment institutionalises it — the old champion keeps running shadow traffic, ready to resume. Limits: rollback cannot undo decisions already made (those need remediation), and for continuously learning systems “the previous version” may not cleanly exist — one reason frameworks treat unconstrained online learning warily.

MLOps / change-management practice

shadow AI

The successor to shadow IT: employees pasting confidential material into personal chatbot accounts, teams wiring API keys into products without review, and — subtlest — AI features switched on inside already-procured SaaS with no new contract to catch them. Risks concentrate where visibility is zero: data leakage into third-party training sets, unvetted decisions, unmet legal duties (an unregistered Annex III use is still an Annex III use). Discovery techniques: surveys and amnesty programs, network/CASB telemetry, expense-report mining, procurement-record sweeps. The strategic fix is economic, not just punitive — provide sanctioned tools good enough that the shadow loses its appeal.

Governance practice

risk register

Where risk management becomes auditable: each entry names the risk, its sources, likelihood and consequence scores, the chosen treatment, the accountable owner, residual risk after treatment, and a review trigger (interval, change event, or incident). “Living” is the operative word — a register last updated at project kickoff is evidence against you. In AI programs it links outward: intake feeds new entries, monitoring and incidents re-score existing ones, and assessments (DPIA, FRIA, ISO 42005) both draw on and update it. Auditors sample it in every ISO 42001 Stage 2; litigators subpoena it after failures.

ISO 31000 recording & reporting

prompt injection

The signature vulnerability of the LLM era, #1 in the OWASP Top 10 for LLM applications. Direct injection: the user overrides the system prompt (“ignore your instructions and…”). Indirect injection — the dangerous one: adversarial instructions hidden in content the system ingests (a web page, email, or document in a RAG store), hijacking the model when it processes them; wire the model to tools and an injected page can trigger data exfiltration or unauthorised actions. Root cause: LLMs have no hard boundary between instructions and data. Defenses are layered, not absolute — input/output filtering, privilege separation, least-privilege tool permissioning, human checkpoints for consequential actions. Distinct from a jailbreak, which targets the model’s own safety rules.

OWASP Top 10 for LLM Applications

jailbreak

Techniques for talking a model out of its safety behaviour: role-play framings, obfuscated encodings, many-shot patterns, multi-turn escalation, automated adversarial suffixes. Distinguish carefully from prompt injection: a jailbreak defeats the model’s own alignment; an injection hijacks an application’s instruction flow — different threat models, different defenses. Jailbreak robustness is now a formal governance object: EU Art 55 adversarial testing, frontier safety frameworks, and safety-institute pre-release evaluations all measure it. Sober planning assumption: no deployed model has proven jailbreak-proof, so consequential applications need controls that survive a jailbroken model — output filtering, tool permissioning, human gates.

OWASP LLM Top 10; NIST-AI-600-1

data poisoning

An integrity attack on the supply chain of learning: adversaries insert or alter training examples so the model learns something attacker-chosen — a backdoor trigger (a pixel pattern that flips any classification), targeted misbehaviour, or general degradation. Web-scale training widens the door: researchers showed poisoning a tiny, affordable fraction of crawled web sources can influence large models, and RAG stores inherit the risk at retrieval time. It appears by name in EU AI Act Art 15’s cybersecurity requirement and MITRE ATLAS. Defenses: provenance tracking, dataset integrity checks, anomaly detection at ingestion, and treating public data sources as untrusted input.

EU AI Act Art 15(5); MITRE ATLAS

adversarial example

The canonical evasion attack: perturbations imperceptible or innocuous to humans that flip a model’s output — stickers making a stop sign read as 45 mph, adversarial glasses defeating face recognition, crafted strings slipping past content filters. They exist because learned decision boundaries are alien to human perception, and they transfer: examples built against one model often fool others, enabling black-box attack. Legally noticed: EU Art 15 requires high-risk systems to resist “adversarial examples” by name. Assurance responses: adversarial training, robustness benchmarks (ISO/IEC 24029 series for neural networks), and refusing to deploy fragile models where adversaries have incentives.

EU AI Act Art 15(5); ISO/IEC TR 24028

membership inference

Ask the model enough carefully constructed questions and its confidence patterns betray whether a given record was trained on — models behave measurably differently on data they memorised. Consequences scale with context: confirming someone’s record was in an HIV-clinic training set is the disclosure. Together with model inversion (reconstructing training data from outputs) and verbatim memorisation in LLMs, it grounds the legal position that trained models can themselves contain personal data — feeding GDPR duties, the machine-unlearning debate, and Art 15’s “confidentiality attacks” language. Mitigations: differential privacy in training, deduplication, memorisation testing before release.

Shokri et al. (2017); EU AI Act Art 15

model extraction

Query a model enough times, train a student on the input-output pairs, and you approximate the victim — stealing the asset without touching the weights. Variants target exact parameter recovery, functionality cloning, or distillation of guarded capabilities into unguarded models. It matters three ways: commercially (the model is the crown jewel), for safety (extracted capability escapes the provider’s controls and monitoring), and legally — EU Art 15 lists model-theft resistance for high-risk systems, and Art 55 requires cybersecurity protection of the weights themselves for systemic-risk GPAI. Defenses: rate limiting, query-pattern anomaly detection, watermarking outputs, and contractual anti-distillation terms.

EU AI Act Art 15, Art 55; MITRE ATLAS

guardrails

The application-layer safety architecture: input guards (block injections, off-policy topics, PII entering prompts), output guards (toxicity and PII filters, grounding checks against source documents, refusal enforcement), and action guards for agents (allow-listed tools, spending caps, human checkpoints before irreversible steps). The design philosophy is defense-in-depth born of humility: safety training can be jailbroken, so policy is enforced outside the model where it can be tested deterministically. Governance angle: guardrail configurations are controls in the audit sense — version them, test them in release gates, and monitor their hit rates like any other control.

OWASP LLM Top 10 (context)

retrieval-augmented generation (RAG)

Architecture, and de facto governance control: before generating, the system retrieves relevant documents from a curated store and instructs the model to answer from them, ideally with citations. Benefits map directly to risk registers — fewer confabulations, answers traceable to sources, knowledge updatable without retraining, and data kept in a governable store rather than baked into weights. New risks arrive with it: the retrieval store becomes an indirect-injection and poisoning surface, access controls must survive the retrieval hop, and grounding is probabilistic — models still contradict their sources. RAG reduces hallucination; only verification eliminates relying on it.

Lewis et al. (2020)

watermarking (AI content)

The technical hope under every labeling law: statistical patterns in generated text tokens, or imperceptible signals in images and audio, detectable by a paired detector. Legal demand is piling up — EU AI Act Art 50(2) requires synthetic content be marked in a machine-readable format and detectable, with effectiveness, interoperability and robustness expectations (compliance deadline 2 December 2026); China’s 2025 Labeling Measures require implicit embedded labels; California’s SB 942 mandates latent disclosures. The physics is stubborn: watermarks degrade under re-encoding, cropping and paraphrase, text watermarks weaken with light editing, and open-weight models can simply omit them — so treat watermarking as one layer of content provenance, not proof.

EU AI Act Art 50(2)

content provenance

The complement to watermarking: instead of hiding a signal in the content, attach signed metadata about it — which device or model created it, what edits followed — verifiable down a cryptographic chain. The C2PA standard (Coalition for Content Provenance and Authenticity: Adobe, Microsoft, camera makers, AI labs) implements this as Content Credentials. Regulators lean on it: EU Art 50(2) guidance cites provenance techniques, California’s AB 853 additions push large platforms to surface provenance metadata, and news/camera ecosystems adopt it for authenticity. Weakness mirrors its design: metadata can be stripped in transit — so provenance proves what carries credentials, never what lacks them.

C2PA specification; EU AI Act Art 50

fairness metric

The main families: demographic parity (equal positive rates across groups), equal opportunity (equal true-positive rates), equalized odds (equal TPR and FPR), and calibration/predictive parity (equal precision — a score means the same thing for every group). The field’s central theorem (Chouldechova; Kleinberg-Mullainathan-Raghavan): when base rates differ, calibration and equal error rates are mathematically incompatible — the COMPAS dispute was exactly this, Northpointe defending calibration while ProPublica indicted unequal false-positive rates, both correct on the numbers. Governance conclusion: fairness-metric choice is a documented policy judgment about who bears which error, not a default a library picks for you.

Chouldechova (2017); ISO/IEC TR 24027

counterfactual explanation

The explanation style built for affected people: rather than exposing model internals, state the smallest change to the inputs that flips the outcome. It maps naturally onto legal duties — US adverse-action reason codes, GDPR “meaningful information about the logic,” EU AI Act Art 86 explanations — and it is actionable in a way a SHAP bar chart is not. Governance caveats: counterfactuals must be feasible (decrease your age is not advice), stable (near-identical applicants shouldn’t get contradictory recipes), and honest — offering a counterfactual implies the pathway genuinely works, which quietly commits the deployer to consistency over time.

Wachter et al. (2017)

interpretability

Often used loosely as a synonym for explainability, but the precise distinction carries a real policy debate: an interpretable-by-design model (scorecard, small decision tree, monotonic GBM) can be read directly, while post-hoc techniques (SHAP, LIME) approximate reasons for a black box — and approximations can be unfaithful or gamed (“explanation-washing”). Rudin’s argument: for high-stakes decisions, prefer inherently interpretable models rather than explaining black boxes, since the accuracy sacrifice is often smaller than assumed. Frameworks encode the layered reality: NIST pairs “explainable and interpretable”; ISO 25059 makes transparency a quality characteristic; regulators increasingly ask why this model class at all.

Rudin (2019); NIST AI RMF

GDPR Article 22

The pre-AI-Act backbone of European automated-decision law. Elements: a decision based solely on automated processing (including profiling) producing legal or similarly significant effects; permitted only where necessary for a contract, authorised by law, or based on explicit consent — and even then with safeguards: the right to obtain human intervention, to express one’s point of view, and to contest the decision. The CJEU’s SCHUFA ruling (C-634/21, 2023) expanded its bite: a credit score is itself an Art 22 decision when recipients rely on it decisively — you cannot launder automation through a human who formally clicks approve downstream. Interlocks with AI Act Art 86, which covers decisions merely based on high-risk AI output.

GDPR Art 22; CJEU C-634/21 (SCHUFA)

controller

GDPR’s primary duty-holder: whoever determines why and how personal data is processed. In AI value chains the role gets intricate: a model developer training on scraped personal data is a controller for that training; a company deploying the model on customer data is a controller for deployment; the vendor running inference on the deployer’s behalf is typically a processor — and misclassifying the relationship voids the contractual architecture GDPR requires (Art 28 terms). Note the rhyme, not identity, with AI Act roles: provider/deployer allocate product-law duties; controller/processor allocate data-protection duties. One organisation frequently wears both hats at once.

GDPR Art 4(7)

processor

GDPR Art 4(8): processes personal data on behalf of the controller, bound by an Art 28 contract (instructions, confidentiality, security, sub-processor controls, deletion on exit). The AI-era fault line: a vendor that uses customer data to train or improve its own models has stepped outside “on behalf of” and become a controller for that purpose — needing its own lawful basis, which most customer contracts never granted. Hence the now-standard no-training-on-customer-data clause and the scrutiny of vendor improvement programs. Sub-processor chains behind AI products (model host, eval vendor, logging provider) make the Art 28 flow-down a real diligence exercise.

GDPR Arts 4(8), 28

lawful basis

Article 6’s menu: consent, contract, legal obligation, vital interests, public task, or legitimate interests. AI training strains it: consent from millions of data subjects in a scraped corpus is impracticable, so developers argue legitimate interests — which requires the three-part test (purpose, necessity, balancing) and has been the battleground in EU regulator actions on model training (EDPB opinions; the ANPD’s 2024 action on Meta’s training in Brazil under the LGPD’s parallel concept). Clearview showed the floor: no basis at all means fines and deletion orders across Europe. Special-category data raises the bar to Art 9’s stricter exceptions — a recurring problem for biometric and health AI.

GDPR Arts 6, 9

data minimization

GDPR Art 5(1)(c): collect and keep no more personal data than the purpose requires. Machine learning’s instinct — more data, better models — collides with it head-on, but the tension is manageable, not fatal: techniques like de-identification and synthetic data, feature-necessity review (does the model need this attribute, or does it merely help marginally?), retention limits on training corpora, and privacy-preserving training all operationalise minimization without abandoning ML. Regulators reject the “keep everything, it might lift AUC” posture; several enforcement actions have keyed on training datasets retained far beyond need. Purpose limitation is its sibling: data collected for one purpose does not automatically become training fodder for another.

GDPR Art 5(1)(c)

purpose limitation

GDPR Art 5(1)(b): purposes must be specified, explicit and legitimate at collection, and further processing must be compatible with them. The AI question of the decade: is training a model on data collected for service delivery a compatible further use? Factors include the link between purposes, the context and reasonable expectations of data subjects, the data’s sensitivity, and safeguards applied. Companies that quietly flipped “we process your messages to provide the service” into “…and to train our models” drew regulator interventions and opt-out mandates across jurisdictions. The design answer: declare training purposes up front, or build a genuine compatibility analysis — not a terms-of-service edit.

GDPR Art 5(1)(b), Art 6(4)

text and data mining exception (TDM)

Articles 3–4 of the DSM Copyright Directive (2019/790): research organisations may mine freely (Art 3); everyone else may mine lawfully accessible works unless the rightsholder has expressly reserved the right in a machine-readable way (Art 4) — robots.txt entries, metadata flags, terms of service. The EU AI Act welded this to model regulation: Art 53 obliges every GPAI provider to maintain a copyright policy that identifies and respects Art 4(3) opt-outs, regardless of where training occurred. Open questions courts are still working through: what counts as machine-readable reservation, and how opt-outs interact with non-EU training. Contrast the US approach, where the same activity is litigated under fair use.

Directive (EU) 2019/790 Arts 3–4; EU AI Act Art 53(1)(c)

fair use

Codified at 17 U.S.C. §107 with four factors: purpose and character of the use (including transformativeness), nature of the work, amount taken, and effect on the market. Whether training generative models on copyrighted works is fair use is the defining US copyright fight of the era — dozens of suits (news publishers, authors, image libraries, music labels) with early rulings splitting on the analysis: training viewed as transformative in some decisions, while pirated source libraries and market substitution cut against defendants in others. Governance takeaways regardless of outcome: document training-data provenance, respect access controls, and price the litigation and indemnification risk. The landscape moves quarterly — date-stamp any statement about it.

17 U.S.C. §107

right of publicity

A US state-law right (statutory in California, Tennessee, New York and others) against unauthorised commercial exploitation of identity. Generative AI made it urgent: voice clones and video deepfakes commercialise likeness at zero cost, so states responded — Tennessee’s ELVIS Act (2024) covering voice explicitly, California’s AB 1836 (deceased personalities’ digital replicas) and AB 2602 (contract protections for performers’ replicas). No federal right exists; the proposed NO FAKES Act would create one for AI replicas — check current status. Distinct from defamation (falsity) and copyright (the recording, not the voice): publicity protects identity itself, which is exactly what generative models can now imitate.

Tennessee ELVIS Act (2024); Cal. AB 1836/2602

synthetic data

Data produced by generative processes rather than collected from the world: synthetic patient records for research, simulated fraud cases to balance rare classes, generated text for fine-tuning. Uses: privacy (train without exposing real records), augmentation, and edge-case coverage. Governance catches: fidelity must be validated (a model trained on unrealistic synthetics fails realistically); privacy is not automatic — a generator can memorise and leak its real training records; and recursive training of models on model output degrades quality (model collapse). Disclosure duties are appearing too: California’s AB 2013 requires training-data documentation to state whether synthetic data was used. Document synthetic provenance like any other lineage.

Cal. AB 2013 (context)

biometric data

GDPR Art 4(14) and AI Act Art 3(34) align: data resulting from specific technical processing of physical, physiological or behavioural characteristics that allows or confirms unique identification. Legal gravity is extreme on both sides of the Atlantic: GDPR makes identification-purpose biometrics special-category data; the AI Act builds an entire prohibition-and-high-risk architecture around it (remote biometric identification, biometric categorisation, emotion recognition); and Illinois’ BIPA — with private lawsuits and per-violation damages — produced the largest US biometric settlements (Facebook $650M; Clearview’s ACLU settlement). Design note: a face photo is not biometric data until technically processed into a template — the processing, not the picture, triggers the regime.

GDPR Art 4(14); EU AI Act Art 3(34); Illinois BIPA

profiling

GDPR Art 4(4): any automated processing that uses personal data to evaluate personal aspects of a natural person. It is the quiet trigger word of two regimes: under GDPR it activates Art 22 protections and DPIA duties; under the EU AI Act it is the override that defeats the Art 6(3) filter — an Annex III system that profiles natural persons is always high-risk, no exceptions — and profiling-only predictive policing is outright prohibited (Art 5(1)(d)). Practical test: if the system infers something evaluative about a person (creditworthiness, reliability, interests, movement patterns) rather than merely recording facts, it is profiling, and the compliance analysis changes.

GDPR Art 4(4); EU AI Act Art 6(3)

special category data

Article 9 data, processable only under narrow exceptions (explicit consent, substantial public interest, and a short list of others). Two AI twists. First, models infer special categories from mundane data — shopping patterns revealing pregnancy, likes revealing orientation — and EU case law treats data revealing sensitive attributes as within Art 9, pulling inference-capable systems into scope. Second, the AI Act carved a purpose-bound bridge: Art 10(5) allows providers to process special-category data strictly to detect and correct bias in high-risk systems, with safeguards — resolving the paradox that you cannot measure racial bias without touching data about race. The paradox itself is why “we don’t collect that field” is not a fairness strategy.

GDPR Art 9; EU AI Act Art 10(5)

machine unlearning

What does “delete my data” mean once the data has shaped a billion parameters? Retraining from scratch without the record is exact but ruinously expensive; approximate unlearning methods try to surgically remove influence with mathematical guarantees, and remain an open research problem at foundation-model scale. The legal pressure is concrete: GDPR erasure requests, FTC algorithmic disgorgement orders (delete the models, not just the data), and copyright plaintiffs seeking removal of works from trained systems. Interim governance posture: prevention over cure — rigorous provenance and lawful-basis checks before training, deduplication, and architectures (like RAG over a deletable store) that keep contested data out of weights.

GDPR Art 17 (context)

OECD AI Principles

The Recommendation that made AI governance an international project. Five values-based principles — inclusive growth and well-being; human rights and democratic values (fairness, privacy); transparency and explainability; robustness, security and safety; accountability — plus five recommendations to governments (R&D investment, enabling ecosystem, interoperable governance, human capacity, international cooperation). The May 2024 update refreshed the text for generative AI, misinformation and safety. Its deepest legacy is definitional: the OECD’s AI-system definition anchors the EU AI Act, the Council of Europe Convention, and most modern statutes — soft law seeding hard law. 47 adherents include non-members like Brazil and Singapore; the G20 principles are derived from it.

OECD/LEGAL/0449 (2019, rev. 2024)

UNESCO Recommendation on the Ethics of AI

Adopted unanimously in November 2021 by UNESCO’s ~190 member states — the broadest normative consensus on AI ethics that exists. Four core values (human rights and dignity; flourishing environment; diversity and inclusiveness; peaceful, just societies) cascade into ten principles including proportionality, safety, privacy, human oversight, transparency, responsibility, and sustainability. What makes it more than parchment: implementation instruments — the Readiness Assessment Methodology (RAM), applied in dozens of countries to benchmark governance capacity, and the Ethical Impact Assessment — plus the Global AI Ethics Observatory. Non-binding, but for much of the Global South it functions as the primary reference framework where no national statute exists.

UNESCO SHS/BIO/REC-AIETHICS/2021

Council of Europe Framework Convention on AI

CETS No. 225, the “Vilnius Convention,” opened for signature 5 September 2024 — signed by the EU, UK, US, Canada, Japan, Israel, Switzerland, Ukraine and others, because unusually for a CoE treaty, non-European states may join. Parties must ensure AI lifecycle activities are consistent with human rights, democracy and the rule of law: risk and impact assessment (the HUDERIA methodology supports this), transparency, accountability, remedies, non-discrimination, and safe-innovation mechanisms like sandboxes. Criticisms: a broad national-security carve-out and a declaration mechanism giving flexibility over private-sector coverage. Entry into force requires five ratifications including three CoE member states — not yet reached as of September 2026; check the CoE Treaty Office for current status.

CoE CETS No. 225 (2024)

Hiroshima AI Process

Launched under Japan’s G7 presidency in the ChatGPT year, producing the International Guiding Principles (11 of them) and the International Code of Conduct for Organizations Developing Advanced AI Systems (October 2023): risk assessment across the lifecycle, public transparency reporting, security controls, provenance/labeling for AI content, and priority research on societal risks. Its 2025 evolution made it measurable — an OECD-hosted reporting framework (launched February 2025) where companies file structured transparency reports against the Code, creating the first comparable, voluntary disclosure layer for frontier developers. A model of soft law engineered for uptake: no sanctions, but public, comparable commitments.

G7 Hiroshima Process (2023); OECD reporting framework (2025)

AI safety institute (AISI)

The institutional innovation of the summit era: state-backed technical bodies that evaluate frontier models — some with pre-release access negotiated with major labs. The UK’s (founded November 2023, renamed AI Security Institute February 2025, builder of the open-source Inspect evaluation platform) set the template; the US counterpart at NIST was renamed CAISI (Center for AI Standards and Innovation) in June 2025 with a security-and-standards mission; Japan, Korea, Singapore, Canada, India, France (INESIA) and Australia followed; the EU’s functional equivalent is the AI Office. The International Network of AI Safety Institutes (launched Seoul, May 2024) runs joint testing exercises. The renames tell the politics: “safety” drifting toward “security” as governments repositioned.

Seoul Summit (2024); national announcements

Model AI Governance Framework

Singapore’s signature contribution: not a statute but a usable playbook, first issued 2019 (second edition 2020) around two anchor principles — AI decisions should be explainable, transparent and fair, and systems should be human-centric. The Generative AI edition (May 2024) widened it to nine dimensions: accountability, data, trusted development and deployment, incident reporting, testing and assurance, security, content provenance, safety and alignment R&D, and AI for public good. It interlocks with AI Verify for testing and sectoral overlays (MAS FEAT in finance). Strategy: make good governance cheap and demonstrable, betting that interoperable voluntary practice beats premature statute — the reference case for the “toolkit” regulatory philosophy.

IMDA/PDPC (2020); IMDA GenAI Framework (2024)

AI Verify

The tooling arm of Singapore’s approach (2022, foundation established 2023): an open-source framework plus software toolkit that runs standardised technical tests (fairness metrics, robustness probes, explainability checks) and process checklists, mapped to internationally aligned governance principles, producing a structured report. Extensions track the frontier: Project Moonshot (2024) for LLM red-teaming and benchmarking, and the Global AI Assurance Pilot (2025) pairing real deployers with testing firms. Significance: an early working answer to “what does demonstrating trustworthy AI actually look like?” — seeding the nascent AI assurance industry that harder regimes (EU conformity assessment) will need at scale.

IMDA / AI Verify Foundation

Interim Measures (China GenAI)

The world’s first binding generative-AI regulation, effective 15 August 2023, issued by the CAC with six co-regulators. Scope: services offering generative AI to the public in China (internal enterprise and R&D uses are carved out). Duties: training data must be lawfully sourced with IP and personal-information rights respected; outputs must reflect “core socialist values” and avoid prohibited content; providers must label synthetic content, handle user complaints, and protect minors; services with public-opinion properties need a security assessment and algorithm filing before launch. “Interim” signals the philosophy: iterate rules quickly, tighten via technical standards (TC260’s security requirements function as de facto binding benchmarks), and fold lessons into later instruments like the 2025 labeling regime.

CAC Interim Measures (2023)

content labeling (synthetic media)

The fastest-converging AI rule worldwide, in two layers: explicit labels users can see, and implicit machine-readable marks (metadata, watermarks) embedded in the file. China’s Labeling Measures (effective 1 September 2025) require both, plus platform detection duties and app-store checks; the EU’s Art 50(2)–(4) requires machine-readable marking by generators and disclosure of deepfakes (synthetic-content compliance deadline 2 December 2026); India amended its IT Rules (February 2026) to mandate permanently embedded metadata on synthetic content; California’s SB 942/AB 853 requires manifest and latent disclosures plus a free public detection tool. Same instinct everywhere — provenance as infrastructure — with interoperability across regimes still unsolved.

China Labeling Measures (2025); EU AI Act Art 50; India IT Rules amendment (2026)

algorithm filing (China)

Established by the 2022 Algorithm Recommendation Provisions and extended by the deep-synthesis and generative-AI rules: providers of covered algorithms file details — type, mechanism, purpose, security self-assessment — with the Cyberspace Administration of China, and filings appear on public registration lists. The trigger phrase, “public opinion attributes or social mobilization capability,” reveals the regulatory core: information control. Consequences are real — unfiled services have been ordered rectified or removed from app stores. For comparativists, it is the mirror image of the EU’s EU AI database: both are registries, but one serves fundamental-rights transparency, the other state supervision of the information space.

CAC Algorithm Recommendation Provisions (2022)

deep synthesis (China)

The Deep Synthesis Provisions (effective 10 January 2023) regulate technology that uses generative or synthetic algorithms to produce or alter text, audio, video, images or virtual scenes. Core duties: conspicuous labels on content that could confuse the public, individual consent before editing someone’s face or voice, identity verification of users, and obligations split between service providers and technical supporters — an early version of the value-chain thinking the EU later formalised. Arriving months before ChatGPT made generative AI a global policy panic, it made China the first mover on binding deepfake law; the 2023 GenAI Measures and 2025 Labeling Measures build directly on its foundations.

CAC Deep Synthesis Provisions (2023)

Bletchley Declaration

The founding document of the AI summit series, signed at Bletchley Park: 28 countries plus the EU — notably including both the United States and China — jointly acknowledged that frontier AI could cause “serious, even catastrophic, harm” and committed to scientific cooperation on safety. The lineage that followed traces the geopolitics: Seoul (May 2024) added the Frontier AI Safety Commitments by 16 companies and launched the safety-institute network; Paris (February 2025) rebranded to “AI Action” with the US and UK declining to sign; New Delhi (February 2026) became the “AI Impact” summit under Global South leadership, with Geneva 2027 next. From Safety to Action to Impact — the renaming is the story.

Bletchley Declaration (Nov 2023)

HUDERIA (Human Rights, Democracy and the Rule of Law Impact Assessment)

The methodology developed under the Council of Europe to operationalise its Framework Convention: a structured process for identifying and mitigating AI risks specifically to human rights, democracy, and the rule of law — a wider aperture than a DPIA (data protection) or a corporate risk assessment (organisational harm). Stages run from context analysis and stakeholder mapping through risk identification against rights frameworks to mitigation planning and iterative review. Being treaty-adjacent guidance rather than binding text, it functions as the reference answer to “how would a state actually assess AI against the Convention?” — and as another entrant in the crowded impact-assessment family every practitioner must now tell apart.

Council of Europe CAI, HUDERIA methodology

Brussels effect

Anu Bradford’s term for how EU regulation goes global without treaties: the single market is too big to skip, compliance is built once, and the strictest standard becomes the default everywhere — GDPR’s cookie banners on American websites being exhibit A. For AI the effect is real but contested: the AI Act’s definitions and risk vocabulary already anchor global compliance programs, yet rival gravitational fields exist — a Beijing effect (China exporting its governance stack along digital-infrastructure routes) and arguably a Seoul effect (Korea’s Framework Act shaping Asian statutes). Watching whether the Digital Omnibus’ softening dents the Act’s exportability is one of the live comparative questions of the decade.

Bradford, The Brussels Effect (2020)

soft law

The other half of governance: OECD Principles, the UNESCO Recommendation, the NIST AI RMF, Singapore’s frameworks, the Hiroshima Code, voluntary safety commitments. No fines — yet soft law binds in practice through procurement requirements, contract clauses, safe-harbor statutes (“consistent with NIST AI RMF”), enforcement expectations of reasonableness, and reputational lock-in. Its lifecycle matters most: soft law is where hard law is drafted early — the OECD definition became EU AI Act Art 3(1); Hiroshima-style transparency reports prefigure statutory duties. Hard law binds with sanctions; soft law persuades with speed, flexibility and consensus — mature governance programs treat both as requirements pipelines, differing only in enforcement date.

Comparative governance literature

frontier safety framework

The self-regulatory architecture of frontier labs, pioneered by Anthropic’s Responsible Scaling Policy and paralleled by OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework: define capability thresholds (CBRN uplift, autonomous AI R&D, offensive cyber), evaluate models against them before and during deployment, and pre-commit to escalating deployment and security standards (Anthropic’s AI Safety Levels — e.g. ASL-3 security for weights) when thresholds approach. The 16-company Seoul Frontier AI Safety Commitments (2024) generalised the pattern, and California’s SB 53 made publishing such a framework a statutory duty for large frontier developers — voluntary practice hardening into law in under three years. Read them critically: thresholds, evaluation quality, and the if-then commitments are where substance lives; versions change frequently, so check the current text.

Anthropic RSP; Cal SB 53; Seoul Commitments (2024)

accountability

The governance principle that some identifiable person or body must be answerable for an AI system across its lifecycle — able to explain decisions, accept audit, and provide remedy when harm occurs. Every major framework anchors on it: the OECD AI Principles list it as principle 1.5, the NIST AI RMF makes “Accountable and Transparent” a trustworthiness characteristic, and ISO/IEC 42001 operationalises it through assigned roles and top management commitment. The design test is simple: for each consequential output, can you name who is accountable — and would they survive a regulator’s interview?

OECD AI Principles 1.5; NIST AI RMF 1.0

accreditation body

The top of the conformity-assessment pyramid: one recognised body per jurisdiction (UKAS in the UK, ANAB in the US, DAkkS in Germany) assesses certification bodies against ISO/IEC 17021-1 — plus ISO/IEC 42006 for AI management systems — and grants them accreditation to certify. Accreditation bodies peer-review each other through the IAF Multilateral Recognition Arrangement, which is what lets a certificate issued in one country be trusted in another. An unaccredited ISO 42001 certificate is just a consultant’s opinion with a logo.

ISO/IEC 17011

affected persons

The third corner of the AI triangle after providers and deployers: the individuals subject to the system’s outputs. They rarely have a contract with anyone in the value chain, which is why frameworks create rights that attach to them directly — the EU AI Act’s right to an explanation of individual decision-making (Art 86), GDPR Art 22 safeguards, and complaint rights to market surveillance authorities. Deployer-side FRIAs exist precisely to force someone to enumerate the categories of affected persons before first use.

EU AI Act Art 86; GDPR Art 22

AI agent

A system that does not just answer but acts: it decomposes a goal, calls tools and APIs, maintains state across steps, and adapts its plan to feedback. ISO/IEC 22989 defines an agent as an entity that senses and acts on its environment; modern usage layers LLM planning on top. Agents multiply governance surface area — one delegated instruction can fan out into dozens of real-world actions, so oversight designed around single decisions (approve/deny) must be redesigned around chains of actions, permissions, and interruptibility.

ISO/IEC 22989

AI lifecycle

The organising skeleton of AI governance. The OECD and ISO/IEC 22989 describe overlapping stage models: design/planning → data collection and processing → model building → verification and validation → deployment → operation and monitoring → retirement. The lifecycle framing carries two lessons frameworks repeat constantly: obligations do not end at deployment (monitoring, incident reporting, and retraining duties attach in operation), and defects are cheapest to catch upstream — the argument for shift left governance.

ISO/IEC 22989; OECD (2024)

AI winter

The boom-bust weather system of AI history. The first winter followed the 1973 Lighthill Report, which told the UK government AI had failed its promises and gutted funding; the second hit in the late 1980s when expert systems underdelivered and the LISP-machine market collapsed. Winters matter to governance because they explain regulator caution about hype in both directions — capability claims and doom claims — and because “don’t trigger a winter” is a standing industry argument against strict regulation.

algorithmic audit

A structured, evidence-based examination of an algorithmic system against stated criteria. The word covers a spectrum: narrow bias audits (NYC Local Law 144 requires annual independent audits of automated hiring tools), technical performance audits, and broad socio-technical audits covering purpose, data, and governance. Distinct from management-system certification audits (ISO-style), which examine the organisation, not the model. The field’s open problems: auditor access, auditor independence, and the absence of agreed audit standards — gaps ISO/IEC 42006 and emerging assurance frameworks are starting to fill.

NYC Local Law 144 (2021)

allocative harm

One half of the classic harms taxonomy (Kate Crawford, 2017): a system allocates something of value — credit, a job interview, bail, a transplant slot — and does so wrongly or unequally. Allocative harms are the ones regulators can most easily see and price, which is why hiring, credit and essential services dominate Annex III of the EU AI Act and US state laws about consequential decisions. Contrast representational harm, where the injury is to how a group is depicted or ranked, not what it receives.

Annex XIII

The checklist behind the Commission’s power to designate a GPAI model as posing systemic risk even below the 10²⁵ FLOP presumption: number of parameters, quality and size of training data, training compute, input/output modalities, capability benchmarks, reach (registered EU business users), and scaling potential. It is the Act’s admission that compute alone is a crude proxy — capability and reach can make a smaller model systemic. Providers can also argue the reverse under Art 52 (crossing the threshold without systemic capabilities).

EU AI Act Art 51(2)–(3), Annex XIII

annotation

The labelling step of the data pipeline: humans (or other models) tag examples — this scan shows a tumour, this comment is toxic — creating the ground truth a supervised model learns to reproduce. Governance cares for three reasons: annotation choices encode contestable judgments as facts (label bias); annotation is frequently outsourced to poorly paid workers exposed to disturbing content (a supply-chain ethics issue); and the EU AI Act’s Art 10 data-governance duties explicitly cover labelling processes for high-risk systems.

EU AI Act Art 10(2)

audit time determination

A quiet anti-race-to-the-bottom mechanism. Generic certification rules already require audit time to reflect organisation size; ISO/IEC 42006 adds AI-specific drivers: how many AI systems the AIMS covers, their complexity and autonomy, the sensitivity of contexts, and whether the organisation develops or merely uses AI. Certification bodies must document the calculation, and accreditation bodies check it — closing the loophole where a two-day drive-by audit yields the same certificate as a rigorous one.

ISO/IEC 42006:2025

automated decision-making

The legal category that predates the AI boom: GDPR Art 22 gives individuals the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects — with exceptions (contract, law, explicit consent) that must be wrapped in safeguards including human intervention. The word solely is the battleground: a human who rubber-stamps machine output does not break the automation (the CJEU’s SCHUFA ruling, 2023, held a credit score itself can be the decision). California’s ADMT regulations and Colorado’s AI Act extend the concept to “substantial factor” decisions.

GDPR Art 22; CJEU C-634/21 (SCHUFA)

autonomy

In the legal definition of an AI system (EU AI Act Art 3(1), OECD 2023), “varying levels of autonomy” is a defining feature: the system has some capacity to act on inputs without a human specifying each step. In ethics texts the same word means something else entirely — human autonomy, the value AI must not undermine through manipulation or over-delegation. Read the framework before assuming which sense is meant.

EU AI Act Art 3(1); OECD (2023)

base rate

How often the thing you are predicting actually occurs in each group (default rates, disease prevalence, re-arrest rates). Base rates are the statistical crux of the fairness wars: when two groups have different base rates, a calibrated model cannot also equalise false-positive and false-negative rates across them — the impossibility theorem (fairness) proven in the wake of the COMPAS debate. Any fairness analysis that never mentions base rates is not yet an analysis.

C2PA (Coalition for Content Provenance and Authenticity)

The leading technical answer to “where did this image come from?”: a specification (steered by Adobe, Microsoft, BBC, Intel and others) for attaching cryptographically signed provenance manifests — capture device, edits, AI-generation flags — to images, video and audio. Content Credentials survive as long as the metadata does; stripping is trivial, which is why C2PA is a transparency floor, not a deepfake cure. Regulators lean on it: it is the reference implementation conversation for EU AI Act Art 50(2) marking and US provenance initiatives.

C2PA Specification v2.x

calibration

The property that predicted probabilities match observed frequencies: among all applicants scored 0.7, roughly 70% actually repay. Calibration within each group is one of the competing fairness definitions — COMPAS was well calibrated across race groups, yet produced unequal false-positive rates, and the impossibility theorem (fairness) shows you generally cannot have both when base rates differ. Calibration also matters operationally: uncalibrated confidence scores poison downstream thresholds and human-review triage.

CEN/CENELEC

CEN (general standards) and CENELEC (electrotechnical) are two of the three official European Standardisation Organisations (with ETSI). When the Commission needs the technical flesh for a New Legislative Framework law, it sends them a standardisation request; the resulting European Standards (ENs), once cited in the Official Journal, deliver the presumption of conformity. For the AI Act the work sits in joint committee JTC 21, which prefers adopting ISO/IEC work (international-first (standards policy)) and drafting European-only standards where EU law demands more.

Regulation (EU) 1025/2012

certification cycle

Certification is not a one-off exam. The standard cycle runs three years: stage 1 and stage 2 initial audits grant the certificate; surveillance audits (at least annually) sample the system to confirm it still functions; a full recertification audit before year three renews it. Nonconformities at any point can suspend or withdraw the certificate. The cycle is why a certificate’s issue date matters less than its maintenance history — and why regulators treat lapsed surveillance as a red flag.

ISO/IEC 17021-1

certification scope

The most-gamed sentence in any certificate. A statement like “AI management system for the development of recommendation models at the Dublin site” covers exactly that — not the hiring-screening product, not the US subsidiary. ISO/IEC 42006 tightens scope discipline for AIMS certificates, requiring clarity about which AI activities (development, provision, use) are covered. Procurement teams should always read the scope line before accepting a certificate as assurance: an out-of-scope certificate is marketing.

ISO/IEC 17021-1; ISO/IEC 42006

certification

Third-party attestation of conformity. In AI governance it almost always means management-system certification (an accredited body certifies your organisation runs an ISO/IEC 42001-conformant AIMS) — an organisational claim, not a product-safety claim. The trust chain runs: accreditation body → certification body → certificate, held together by ISO/IEC 17021-1 and the IAF MLA. Certification is voluntary but commercially potent: US state laws and procurement rules increasingly treat it as evidence of reasonable care, and EU AI Act conformity assessment borrows its architecture.

ISO/IEC 17021-1

Clearview AI

The case that made biometric scraping a named legal problem. Clearview AI built a face-search engine on billions of images scraped from social media and sold it to police forces. Regulators responded in a wave: multi-million euro GDPR fines in France, Italy, Greece and the Netherlands, a UK ICO fine, and the 2022 ACLU settlement under Illinois BIPA banning sales to most private entities. The EU AI Act now prohibits untargeted scraping of facial images to build recognition databases (Art 5(1)(e)) — a prohibition with Clearview’s name written between the lines.

EU AI Act Art 5(1)(e); Illinois BIPA

code of practice

A halfway house between soft law and regulation: detailed practices drafted with industry that regulators then treat as an approved route to compliance. The flagship is the EU GPAI Code of Practice (published July 2025, drafted by independent experts with ~1,000 stakeholders): signatories get a streamlined way to demonstrate conformity with Arts 53 and 55 obligations on transparency, copyright and safety/security. Formally voluntary, practically gravitational — non-signatories must prove equivalent compliance by other means, with more scrutiny.

EU AI Act Art 56

cognitive bias

The human third of NIST’s three-part bias taxonomy (systemic, computational/statistical, human-cognitive). Documented reasoning shortcuts — anchoring on the model’s score, confirmation bias in reviewing its output, groupthink in design meetings — shape which problems get modelled, which labels count as ground truth, and how “human oversight” actually behaves. The taxonomy’s point: you cannot debug bias out of a model while leaving the humans around it unexamined. Automation bias is the special case that turns human-in-the-loop into a rubber stamp.

NIST SP 1270

combined audit

Because ISO management-system standards share the same harmonized structure (clauses 4–10), a certification body can audit several at once: information security (27001), privacy (27701) and AI (42001) in a single visit against a single integrated documentation set. The efficiency is real — shared context, risk registers and management reviews — but 42006 requires the audit team to hold competence in each discipline, and audit time cannot simply be halved. The organisational mirror image is the integrated management system.

ISO/IEC 17021-1

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions)

A proprietary risk-assessment tool used in US courts to score defendants’ likelihood of reoffending. ProPublica’s 2016 analysis found Black defendants who did not reoffend were nearly twice as likely to be misclassified as high-risk as white defendants; the vendor replied the tool was equally calibrated across races. Both were right — which is the enduring lesson: with different base rates, calibration and equal error rates are mathematically incompatible (impossibility theorem (fairness)). Loomis v. Wisconsin (2016) added the due-process angle: courts may use such scores, but with warnings and never as the sole basis.

ProPublica, “Machine Bias” (2016); State v. Loomis (Wis. 2016)

compute

The third input to modern AI alongside data and algorithms — and the one governments can most easily count, because it flows through a handful of chipmakers and cloud providers. Training compute doubles as a capability proxy: the EU AI Act presumes systemic risk at 10²⁵ FLOPs; the (since-rescinded) US Executive Order 14110 set reporting at 10²⁶. Compute governance — export controls on advanced chips, cluster reporting, know-your-customer for cloud — is the supply-chain arm of frontier AI policy, and its critics note the proxy decays as algorithms get more efficient.

EU AI Act Art 51(2)

contestability

The process right that survives the black box: even where a model cannot be fully explained, the person it scored can be given a route to challenge the decision, submit corrections, and obtain human review with power to change the outcome. Hooks include GDPR Art 22(3) (right to contest), the EU AI Act Art 86 explanation right that makes contesting practical, and OECD principle 1.3. Design test: does the appeal go to someone who can actually overturn the model — or to a queue that re-runs it?

GDPR Art 22(3); EU AI Act Art 86

continual improvement

Clause 10.1 of ISO/IEC 42001 (and every harmonized-structure standard): the management system must continually improve — fed by monitoring results, internal audits, nonconformities, incidents, and management review outputs. It is the Act in Plan-Do-Check-Act, and the reason certification is a cycle rather than a trophy: auditors expect to see the loop turning — findings raised, corrective actions closed, objectives updated — not a binder frozen at certification date.

ISO/IEC 42001 Clause 10

controllability

An ISO trustworthiness characteristic (ISO/IEC 22989; a quality sub-characteristic in ISO/IEC 25059): the degree to which an external agent can intervene in the system’s functioning — pause it, override an output, hand control to a human, or stop it entirely. It is the engineering substrate beneath legal human oversight duties: EU AI Act Art 14’s requirement that overseers can “intervene… or interrupt the system through a stop button or similar procedure” presumes controllability was designed in, not bolted on.

ISO/IEC 22989; ISO/IEC 25059

corrective action

ISO clause 10.2 discipline: when a nonconformity (major / minor) occurs, the organisation must react, evaluate whether root causes exist elsewhere, implement action to prevent recurrence, and check effectiveness — all documented. Corrective action differs from correction (fixing this instance) by attacking the cause (root cause analysis). Under the EU AI Act the phrase also carries product-law weight: Art 20 obliges providers of non-conforming high-risk systems to take corrective actions — withdrawal, disabling, recall — and inform the value chain.

ISO/IEC 42001 Clause 10.2; EU AI Act Art 20

creditworthiness

The canonical consequential decision. AI systems evaluating natural persons’ creditworthiness or establishing their credit score are high-risk under EU AI Act Annex III 5(b) (with a carve-out for financial-fraud detection), and banks deploying them owe FRIAs. In the US the same territory is covered by decades-old law — ECOA and FCRA adverse-action notices apply regardless of whether a human or model decided — and the CJEU’s SCHUFA ruling pulled credit scoring under GDPR Art 22. Credit is where “the algorithm did it” was first tested and first rejected as a defence.

EU AI Act Annex III 5(b); ECOA/Reg B

crosswalk

The connective tissue of multi-framework compliance: a table mapping, say, NIST AI RMF subcategories to ISO/IEC 42001 controls to EU AI Act articles. Official examples exist (NIST publishes RMF crosswalks to ISO standards and the Blueprint; certification bodies map 42001 Annex A against Act obligations). Crosswalks let one control satisfy many masters — but they are lossy by nature: related is not equivalent, and the residual gaps are exactly where auditors and regulators look. Treat a crosswalk as a starting map, then verify each cell.

NIST AI RMF Crosswalks

Cyber Resilience Act (CRA)

Regulation (EU) 2024/2847: horizontal cybersecurity requirements for products with digital elements — secure-by-design development, vulnerability handling, update obligations — entering full application in 2027. The AI interface is explicit: a high-risk AI system that satisfies the CRA’s essential requirements is presumed to comply with the AI Act’s Art 15 cybersecurity requirement, one of the Act’s named presumption bridges. For AI products, the two regimes stack: AI Act for AI-specific risks, CRA for the software around them.

Regulation (EU) 2024/2847; EU AI Act Art 42(2)

data governance

The discipline of managing data as a governed asset: where it came from (provenance), whether you may use it (rights, consent, licences), whether it is fit for purpose (quality, representativeness, bias examination), and who may touch it. In AI it stops being back-office hygiene and becomes a legal requirement: EU AI Act Art 10 demands documented data-governance practices for high-risk training, validation and test sets — design choices, collection processes, bias examination and mitigation — and ISO/IEC 42001 Annex A dedicates a control family (A.7) to it.

EU AI Act Art 10; ISO/IEC 42001 Annex A.7

data provenance

The audit trail for the training set: source, collection method, legal basis or licence, consent status, and every transformation applied. Provenance is what lets you answer the questions that now decide lawsuits and enforcement — was this scraped? was it opted out under the TDM regime? does it contain special-category data? ISO data-quality standards (the 5259 series) and datasheet practices operationalise it; its absence is why so many foundation-model copyright cases begin with “we cannot say what was in the corpus.”

ISO/IEC 5259 series

datasheets for datasets

The documentation practice proposed by Gebru et al. (2018): every dataset ships with a datasheet answering structured questions — why it was created, what it contains, how it was collected and cleaned, known gaps and biases, recommended and discouraged uses. Together with model cards, datasheets became the template for the documentation regulators later mandated: EU AI Act Art 10 data-governance records and GPAI training-content summaries are datasheets with legal force. The practice’s core insight: undocumented data is unauditable data.

Gebru et al., “Datasheets for Datasets” (2018)

demographic parity

Also called statistical parity or group fairness: the model’s positive rate is independent of the protected attribute — equal shares of each group get the loan, the interview, the flag. Its appeal is simplicity and its close cousinship to disparate-impact doctrine (the four-fifths rule tests a relaxed version). Its weakness: it ignores base rates and individual merit — you can satisfy it by approving randomly within a group — and it conflicts mathematically with calibration and equalized odds when base rates differ. Choosing among these definitions is a policy decision wearing a math costume.

disaggregated testing

Breaking evaluation results down by demographic group, context, or condition instead of reporting one aggregate number. The practice Gender Shades made famous: commercial face classifiers scored ~99% on lighter-skinned men and as low as ~65% on darker-skinned women — invisible in the headline accuracy. Now baked into expectations: EU AI Act Art 10 requires data appropriate to the groups the system will affect, model cards report disaggregated metrics, and any bias audit worth the name starts here. The hard prerequisite is having (lawful) subgroup labels to disaggregate by.

Buolamwini & Gebru, “Gender Shades” (2018)

documented information

ISO harmonized-structure vocabulary (clause 7.5) covering both documentation (what you intend to do: policies, procedures, the Statement of Applicability) and records (evidence it happened: risk assessments, impact assessments, training logs, management-review minutes). It must be controlled — versioned, approved, available, protected. Auditors live on it: in an audit, undocumented work effectively did not happen. The discipline transfers directly to the EU AI Act, where technical documentation and logs are the currency of conformity.

ISO/IEC 42001 Clause 7.5

downstream provider

The AI Act’s name for the actor integrating a GPAI model into their own AI system (Art 3(68)). The Act gives them a rare statutory information right: GPAI providers must supply documentation enabling downstream providers to understand the model’s capabilities and limitations and meet their own obligations (Art 53(1)(b)) — the regulatory answer to “the API vendor won’t tell us anything.” The relationship is two-way: fine-tune or modify the model substantially and the downstream provider may become a model provider in their own right.

EU AI Act Arts 3(68), 53(1)(b)

data protection officer (DPO)

A role GDPR Arts 37–39 requires for public bodies and organisations doing large-scale monitoring or special-category processing: independent, expert, reporting to top management, consulted on DPIAs, and protected from dismissal for doing the job. Because most AI systems process personal data, DPOs became the accidental first owners of AI governance in many organisations — a stopgap now formalised in some regimes and stretched in others, since AI risk (safety, fairness, IP) exceeds the data-protection lens the role was built for.

GDPR Arts 37–39

dual-use

The property that makes frontier AI governance feel like arms control: the capability is the risk. A biology model that accelerates drug discovery can, reframed, propose toxic molecules (the 2022 Nature Machine Intelligence experiment generated 40,000 candidate toxins in six hours). US law leans on the term directly — the AI Executive Order regulated “dual-use foundation models” — and export-control regimes treat advanced chips and model weights through the same lens. Governance consequence: you cannot simply ban the dangerous capability without banning the beneficial one; you govern access, evaluation, and deployment context.

US EO 14110 §3(k) (2023, rescinded 2025)

duty of care

In negligence law, the duty to take reasonable care to avoid foreseeable harm to others; in corporate law, the directors’ fiduciary duty to inform themselves and oversee the company diligently. Both routes now reach AI: Delaware’s Caremark line makes boards liable for failing to install oversight systems for mission-critical risks — and AI is rapidly joining that category — while product-liability and negligence claims test what a “reasonable” AI developer or deployer would have done. Standards like ISO/IEC 42001 and the NIST RMF function as evidence of what reasonable care looks like.

In re Caremark (Del. Ch. 1996)

Evaluate–Direct–Monitor (EDM)

The governance loop from ISO/IEC 38507 (governance of IT, applied to AI): the governing body evaluates strategic options and proposals, directs by setting policy, risk appetite and resource allocation, and monitors performance against expectations — while management does everything in between. The model’s value is its discipline: boards that slip into managing (approving individual models) lose oversight distance, and boards that skip monitoring (no KRIs, no management review visibility) hold accountability without information. EDM is the answer to both failure modes.

ISO/IEC 38507

ELIZA effect

Joseph Weizenbaum’s 1966 ELIZA program parroted therapy-style questions with simple pattern matching — and users, including his own secretary, confided in it and insisted it understood them. Weizenbaum was so disturbed he became one of computing’s first internal critics (Computer Power and Human Reason, 1976). The effect scales with fluency: modern chatbots trigger it by design, which is why disclosure rules (EU AI Act Art 50: tell people they are talking to a machine), companion-app laws, and manipulation prohibitions all trace back to this 60-year-old observation.

Weizenbaum (1966, 1976)

equalized odds

The error-rate fairness criterion (Hardt et al., 2016): among people who genuinely qualify, each group is approved at the same rate; among those who do not, each group is wrongly approved at the same rate. It captures the intuition behind the COMPAS critique — the injustice was unequal mistakes, not unequal averages. The catch is the impossibility theorem (fairness): with different base rates, equalized odds and calibration cannot both hold, so someone must decide which error matters more in this context. That decision is governance, not statistics.

Hardt, Price & Srebro (2016)

escalation threshold

The wiring that connects operational monitoring to governance: drift beyond X for Y days → product owner; any severity-1 incident or rights-affecting failure → CRO and board committee; regulator contact → general counsel, same day. Thresholds are set in advance precisely so that escalation does not depend on the judgment (or optimism) of whoever first sees the problem. They pair with KRIs: the KRI measures, the threshold acts. Absent thresholds, bad news travels at the speed of embarrassment.

essential requirements

The load-bearing concept of the New Legislative Framework: the regulation states what must be achieved — risk management, accuracy, robustness, oversight (EU AI Act Chapter III Section 2) — in technology-neutral language, and harmonized standards supply the presumed how. This split keeps law durable while technology moves. The AI Act twist: because no harmonized standards are yet cited, providers currently face essential requirements with no presumption shortcut — they must interpret and evidence the outcomes themselves.

EU AI Act Arts 8–15

AI ethics board

A standing committee — internal cross-functional, or external advisory — that reviews AI use cases, sets ethical policy, and adjudicates hard calls the risk matrix cannot. Design determines whether it matters: real boards have veto power or binding escalation routes, defined intake, and protection for dissent; decorative ones meet quarterly and learn about launches from the press. Cautionary tales are canon — Google’s external ATEAC council dissolved after one week in 2019 — and the fix is structural: connect the board to the intake process and the governing body, or accept it is ethics washing infrastructure.

ethics washing

The gap between stated principles and operative practice, deployed strategically: publish values, convene a powerless council, sponsor ethics research, and continue as before. The term (popularised around 2018–19, notably by Ben Wagner) named the pattern in the great principles boom, when 80+ AI ethics codes appeared with near-zero enforcement. Its diagnostic questions travel well: does anyone lose a launch, a bonus, or a contract when the principle is breached? If not, it is communications, not governance. The critique drove the field’s pivot from voluntary principles to hard law and auditable management systems.

EU AI Act

The reference point of global AI regulation. In force 1 August 2024, it bans a short list of practices, imposes a full product-safety rulebook on high-risk systems, attaches transparency duties to limited-risk uses, and regulates general-purpose AI models directly — enforced through market-surveillance machinery, an EU-level AI Office, and penalties up to €35M or 7% of global turnover. Obligations phase in over years (prohibitions Feb 2025, GPAI Aug 2025, most high-risk duties 2026–2028, with the Digital Omnibus adjusting dates — check current status). As a regulation it applies directly in all Member States, and its extraterritorial reach gives it Brussels-effect gravity.

Regulation (EU) 2024/1689

EU database for high-risk AI systems

The AI Act’s answer to “nobody knows where the high-risk systems are”: Art 71 establishes a Union-wide, largely public database; providers register Annex III high-risk systems (and documented Art 6(3) not-high-risk conclusions) before market placement, and public-authority deployers register their use. Entries include intended purpose, status, and conformity details — law-enforcement entries sit in a restricted section. It converts classification from a private judgment into a searchable public record, giving journalists, researchers and complainants a starting index.

EU AI Act Arts 49, 71

feedback loop

The dynamic that makes deployed AI different from a static test: outputs alter the world that generates the next inputs. Predictive policing is the textbook case — patrol where the model points, arrests happen where patrols are, the new arrest data “confirms” the hotspot. Recommender systems do it to preferences; credit denials do it by erasing repayment evidence. Feedback loops defeat one-time fairness audits (the bias grows after deployment), which is why monitoring and periodic re-evaluation are hard requirements in serious frameworks.

FLOPs (floating-point operations)

The count of elementary arithmetic operations performed during training — the field’s standard measure of how much compute a model consumed. Regulators adopted it because it is estimable and comparable: the EU AI Act presumes systemic risk above 10²⁵ FLOPs (Art 51(2)), and the rescinded US EO 14110 used 10²⁶ for reporting. Cautions: FLOPs measure effort, not capability; algorithmic efficiency shifts the capability-per-FLOP curve every year; and thresholds invite training-just-below gamesmanship. (Distinguish FLOPS-with-capital-S, operations per second — a hardware speed, not a training total.)

EU AI Act Art 51(2)

framework convention

A treaty form that sets binding principles and leaves each party to implement through domestic law — the structure used for climate (UNFCCC) and tobacco control. The Council of Europe’s Framework Convention on AI, Human Rights, Democracy and the Rule of Law (opened for signature 5 September 2024) is the first binding international AI treaty, signed by the EU, US, UK and others; it commits parties to lifecycle risk management, transparency, remedies and oversight, but contains significant carve-outs (national security; discretion over private-sector coverage) and no strong enforcement organ. Its weight is normative: a floor beneath diverging national regimes.

CoE CETS No. 225 (2024)

functional adaptability

One of the AI-specific additions ISO/IEC 25059 makes to the SQuaRE quality model: the degree to which the system can operate acceptably in contexts and on data beyond those it was originally specified for. It gives engineering vocabulary to a governance obsession — distribution shift and out-of-scope use — by making “how far can this stretch” an evaluable characteristic with acceptance criteria, rather than a hope. Paired with a declared intended purpose, it defines the honest envelope of the system.

ISO/IEC 25059

fundamental rights

In EU law, the rights codified in the Charter of Fundamental Rights: human dignity, private life and data protection (Arts 7–8), non-discrimination (Art 21), effective remedy (Art 47), and more. The AI Act’s risk logic runs on them: practices are prohibited or high-risk according to their threat to health, safety, or fundamental rights, and deployers of certain systems owe fundamental rights impact assessments. The framing distinguishes the EU approach from pure safety regulation — a system can be perfectly accurate and still unlawful because of what it does to rights.

EU Charter of Fundamental Rights

gap analysis

Where every implementation project starts: map what the organisation actually does today against what the target framework requires — clause by clause for ISO/IEC 42001, function by function for the NIST AI RMF, article by article for the EU AI Act — and record the deltas as a costed, owned remediation plan. Done honestly, it surfaces the uncomfortable findings (shadow AI, unowned models, absent documentation) that glossier maturity assessments smooth over. Its output feeds the Statement of Applicability and the certification-readiness decision.

GDPR (General Data Protection Regulation)

Regulation (EU) 2016/679 — the law that made Brussels effect a phrase. For AI it supplies the substrate the AI Act builds on: a lawful basis for every processing of personal data (including training), purpose limitation and data minimization, DPIAs for high-risk processing, Art 22 limits on solely automated decisions, and fines to 4% of global turnover. The two regimes stack rather than replace: an AI system can be fully AI-Act-conformant and still unlawful under GDPR — training-data legality, the SCHUFA ruling on credit scores, and Clearview’s fines are all GDPR stories.

Regulation (EU) 2016/679

Gender Shades

Joy Buolamwini and Timnit Gebru’s MIT study evaluated three commercial gender classifiers on a skin-type-balanced benchmark and found error rates of ~0.8% for lighter-skinned men versus up to ~34.7% for darker-skinned women — a chasm invisible in aggregate accuracy. Its impact outran its findings: vendors improved within months (proving the audit lever works), IBM and others later exited facial recognition, and disaggregated testing became the expected evaluation practice. It is the standing rebuttal to any benchmark reported as a single number.

Buolamwini & Gebru (2018)

governance vs management

The distinction ISO/IEC 38507 exists to enforce. Governance — the governing body’s work — evaluates, directs, and monitors: sets risk appetite, allocates resources, demands assurance (EDM). Management — top management downward — plans, implements, operates, and reports. The failure modes are symmetric: boards that manage (approving individual model releases) lose the independence oversight requires; boards that neither direct nor monitor delegate accountability they legally cannot shed. ISO/IEC 42001 speaks to management; 38507 speaks to the board — organisations need both layers working.

ISO/IEC 38507

governing body

ISO’s deliberately form-neutral term (ISO/IEC 38507) for whoever holds ultimate accountability: a corporate board, a public-agency head, university trustees, partners. AI governance frameworks route to it on purpose — the governing body sets AI risk appetite, receives escalations and assurance, and cannot delegate accountability even when it delegates everything else. Distinct from top management (the executives running the management system): the board directs and monitors them. Fiduciary law (Caremark duties) is what puts teeth behind the diagram.

ISO/IEC 38507

hard law

Rules that bind: violate them and a court or regulator can compel, fine, or ban. The EU AI Act, GDPR, Colorado’s AI Act, and China’s deep-synthesis rules are hard law; the OECD Principles, UNESCO Recommendation, and the NIST RMF are soft law. The boundary moves constantly in AI governance — soft instruments harden (voluntary commitments become codes of practice become statutory duties), and hard law borrows soft-law content (state statutes citing the NIST RMF as a safe harbor). The strategic question for organisations is not binding or not but how fast is this instrument hardening.

harm

The terminus of the risk vocabulary: a hazard is a potential source, risk is likelihood × severity, harm is the realised damage — physical injury, financial loss, rights violations, psychological damage, societal effects like eroded trust. Taxonomies slice it by type (the NIST GenAI Profile’s harm families), by mechanism (allocative vs representational), and by scale (individual → group → societal). Legal thresholds attach to it: the EU AI Act’s manipulation prohibitions require (reasonable likelihood of) significant harm, and tort law requires provable damage. Defining which harms count, to whom, is where governance stops being technical.

ISO/IEC Guide 51

harmonised standard

The bridge between the AI Act’s essential requirements and engineering practice: a European standard produced by CEN/CENELEC under a Commission standardisation request, which — once its reference is cited in the Official Journal — gives users a presumption of conformity with the requirements it covers. (British spelling of the same concept as harmonized standard.) For AI, none are yet cited as of September 2026: prEN 18286 on quality management is the furthest along, so providers currently work from the Act’s text directly — check current status.

EU AI Act Art 40; Regulation (EU) 1025/2012

harmonised standards

The family of European standards that, once cited in the Official Journal, let providers demonstrate conformity with EU legislation by presumption rather than first-principles argument. For the AI Act, the Commission’s standardisation request C(2023)3215 tasked JTC 21 with roughly ten deliverable areas — risk management, data governance and quality, logging, transparency, human oversight, accuracy, robustness, cybersecurity, quality management, conformity assessment. As of September 2026 the set remains in development with none yet cited; the gap between legal deadlines and standards readiness has been a central implementation controversy.

EU AI Act Art 40

harmonized standards

US-spelling plural of harmonized standard: the standards that operationalise the New Legislative Framework bargain — the law states outcome-level essential requirements, standards bodies supply the technical detail, and applying a cited standard yields the presumption of conformity. The model made EU product law scale across sectors from toys to lifts; the AI Act is its most ambitious application yet, and the slowest to deliver — no AI Act standards were cited as of September 2026.

EU AI Act Art 40

hazard

Safety-engineering vocabulary (ISO/IEC Guide 51) imported into AI risk work: a potential source of harm. For an AI hiring tool, hazards include biased training labels, proxy features, distribution shift, and rubber-stamp review; each becomes a risk once you attach likelihood and severity, and a harm if it lands on someone. Hazard identification is the first, generative step of risk assessment — checklists like the NIST GenAI Profile’s twelve risk categories or MIT’s risk repository exist so teams do not have to brainstorm from zero.

ISO/IEC Guide 51

high-risk AI

The generic concept behind the EU’s defined term: AI used in contexts where errors or misuse carry serious consequences for people — hiring, credit, medical devices, policing, critical infrastructure. Risk-based regimes converge on similar lists but different mechanics: the EU AI Act enumerates them (Annex III plus Annex I safety components) and attaches a full pre-market rulebook; Colorado’s AI Act regulates “high-risk” systems making consequential decisions via a duty of reasonable care; Canada’s abandoned AIDA used “high-impact”. The shared logic: proportionality — regulate where stakes are high, leave chess apps alone.

historical bias

The bias that exists before any sampling or measurement error: the training data faithfully records a world shaped by discrimination, so the model learns it as signal. Amazon’s scrapped CV-screening tool is the canon example — trained on a decade of male-dominated hiring outcomes, it penalised the word “women’s”. Historical bias cannot be fixed by collecting more of the same data; it forces a normative decision — reproduce the world as it was, or constrain the model toward the world as policy says it should be. That decision is what fairness interventions encode.

NIST SP 1270

horizontal regulation

Regulating the technology once, everywhere: a single instrument whose rules apply to AI in hiring, health, finance, and toys alike, calibrated by risk rather than by industry. The EU AI Act is the archetype; Colorado’s AI Act is a state-level cousin. The trade-offs against sectoral regulation are stable: horizontal rules give coherence and close gaps between sectors, but strain to fit every context (the same Article 10 governs cancer screening and credit scoring) and require heavy secondary guidance. Most real regimes end up hybrid — a horizontal floor plus sectoral overlays.

IAF MLA (International Accreditation Forum Multilateral Recognition Arrangement)

The peer-review pact among national accreditation bodies: signatories evaluate each other against ISO/IEC 17011, and certificates issued under any signatory’s accreditation are recognised by all — “accredited once, accepted everywhere.” For AI it is the mechanism that makes ISO/IEC 42001 certification globally portable: once 42006-based accreditation programs sit under the MLA umbrella, a certificate from an accredited body in Singapore carries the same formal weight in Frankfurt. It is also the quality backstop — bodies that certify carelessly risk their accreditation, not just their reputation.

IAF MLA; ISO/IEC 17011

impact assessment

Governance’s favourite verb made into a noun: systematically ask who could this affect, how badly, and what will we do about it — and write the answer down before launch. The family is large: DPIAs (GDPR), FRIAs (EU AI Act Art 27), algorithmic impact assessments (Canada’s directive), AI system impact assessments (ISO/IEC 42005), and state-law versions (Colorado). The common DNA: identify affected persons, enumerate harms, evaluate necessity and proportionality, plan mitigations, assign owners, and revisit on change. The common failure: doing it once, filing it, and never letting it alter the design.

ISO/IEC 42005; EU AI Act Art 27

implementation timeline (EU AI Act)

The Act entered into force 1 August 2024 but applies in waves: 2 February 2025 — prohibitions and AI-literacy duties; 2 August 2025 — GPAI obligations, governance bodies, penalties; then the high-risk regime, originally 2 August 2026/2027, rescheduled by the Digital Omnibus package to 2 December 2027 (Annex III) and 2 August 2028 (Annex I), largely because harmonized standards were not ready. Grandfathering applies to systems already on the market, with triggers on substantial modification. Treat every date here as check-current-status material: the timeline has already moved once.

EU AI Act Art 113; Digital Omnibus (2025)

impossibility theorem (fairness)

The mathematical result (Kleinberg, Mullainathan & Raghavan 2016; Chouldechova 2017) crystallised by the COMPAS fight: except in degenerate cases, a risk score cannot simultaneously be calibrated within each group and have equal false-positive and false-negative rates across groups when the groups’ base rates differ. Its governance import is liberating rather than defeatist: since no model can satisfy every fairness definition, someone must choose and justify which one fits the context — and that choice belongs in documented, accountable policy, not in a data scientist’s default settings.

Kleinberg et al. (2016); Chouldechova (2017)

incident-driven regulation

The empirical rhythm of tech governance: a harm becomes a headline, the headline becomes a hearing, the hearing becomes a statute. Cambridge Analytica accelerated GDPR enforcement culture; the toeslagenaffaire hardened the EU’s stance on social scoring and profiling; NYC’s hiring-audit law followed the resume-screening scandals; deepfake statutes followed specific viral incidents. The pattern’s cost is the pacing problem in reverse — rules optimised for the last scandal — and its benefit is political feasibility: incidents create the coalition that abstract risk analysis never does. Databases like the AI Incident Database exist to shorten the distance from pattern to policy.

integrated management system

The organisational strategy the ISO Harmonized Structure (Annex SL) was designed to enable: because every modern management-system standard shares the same clause skeleton (context, leadership, planning, support, operation, evaluation, improvement), an organisation can run one integrated system — one policy hierarchy, one risk method, one internal-audit programme, one management review — with discipline-specific extensions for security (27001), privacy (27701), and AI (42001). Benefits: less duplication, coherent risk pictures, and eligibility for combined audits. Cost: integration debt if the disciplines are owned by silos that do not talk.

ISO Annex SL

internal audit

ISO clause 9.2: the organisation must audit its own management system at planned intervals, with auditors objective and impartial (not auditing their own work), documented criteria and scope, and results reported to relevant management. In the three lines of defense model it is the third line. For an AIMS this means someone internal walks the same ground an external auditor will — sampling AI risk assessments, testing whether controls in the Statement of Applicability actually operate — before certification or surveillance visits. Weak internal audit is the most reliable predictor of external-audit surprises.

ISO/IEC 42001 Clause 9.2

international-first (standards policy)

The policy that keeps European and international AI standards from forking gratuitously. Under the Vienna/Frankfurt agreements and its own strategy, CEN-CENELEC JTC 21 first asks whether an ISO/IEC deliverable (from SC 42) can be adopted as an EN — as it did with the EN ISO/IEC 5259 data-quality series and 42001 — and drafts homegrown European standards only where the AI Act’s fundamental-rights requirements exceed what international consensus produced (e.g. prEN 18286 on quality management, trustworthiness). For companies, the payoff is convergence: implementing the ISO stack is mostly walking toward the EU stack.

CEN-CENELEC JTC 21 work programme

intervenability

A quality property in the ISO AI vocabulary family, closely paired with controllability: the extent to which the system exposes intervention points — reviewable intermediate outputs, override paths, pause and rollback mechanisms — that let an authorised human change the course of processing. Privacy engineering uses the same word for the ability to intervene in data processing (correct, delete, restrict). Either way it is a design-time property: an intervention capability added after deployment is usually an incident report describing why it was needed.

ISO/IEC 25059; ISO/IEC TS 8200

ISO/IEC 17021-1

The rulebook for the auditors: requirements certification bodies must meet to be accredited for management-system certification — organisational impartiality (no consulting your own audit clients), auditor competence management, the stage 1/stage 2 initial-audit structure, surveillance and recertification rules, and certificate decision-making separated from the audit team. For AI, ISO/IEC 42006 layers AIMS-specific competence and audit-time requirements on top of it. When you ask “who audits the auditors, and against what?” — this is the what.

ISO/IEC 17021-1:2015

ISO/IEC 27701

The privacy member of the management-system family: a Privacy Information Management System (PIMS) specification extending ISO/IEC 27001’s security base with controller and processor privacy controls mapped loosely to GDPR concepts. (Its 2025 revision makes it certifiable standalone.) In AI governance it slots beside 42001: the AIMS governs AI-specific risk, 27701 governs the personal data flowing through training and inference, 27001 secures the whole stack — commonly run as one integrated management system and certified in combined audits.

ISO/IEC 27701

ISO (International Organization for Standardization)

The global standards machine: ~170 national member bodies producing voluntary consensus standards through committees of delegated experts. For information technology ISO works jointly with the IEC through JTC 1, whose subcommittee SC 42 owns AI — home of ISO/IEC 22989 (concepts), 23894 (risk), 42001 (management systems), and the trustworthiness catalogue. ISO standards bind no one by themselves; they acquire force through contracts, procurement, certification, and — in the EU — adoption as harmonised standards. That indirect power is why seats at SC 42 tables are strategic assets.

JTC 1 (ISO/IEC Joint Technical Committee 1)

The 1987 pact that stopped ISO and the IEC writing rival computing standards: one joint committee for information technology, whose subcommittees own everything from character sets to cybersecurity (SC 27) to AI (SC 42, created 2017). Standards it produces carry the “ISO/IEC” double badge. Knowing the org chart pays practical dividends: it explains numbering, why AI standards inherit patterns from security standards developed one corridor over, and where national bodies must send experts to influence what future audits will measure. Not to be confused with CEN-CENELEC JTC 21, its European counterpart for AI.

JTC 21 (CEN-CENELEC Joint Technical Committee 21)

Europe’s AI standards committee (established 2021), executor of the Commission’s standardisation request C(2023)3215. Its working method is international-first (standards policy): adopt ISO/IEC SC 42 deliverables as European standards where they fit, and draft European-only ones where the AI Act demands more than international consensus provides — prEN 18286 on AI quality management systems being the flagship. Its output, once cited in the Official Journal, becomes harmonised standards carrying the presumption of conformity — which makes this obscure committee one of the most consequential rooms in AI governance. Distinct from ISO/IEC JTC 1.

Commission Implementing Decision C(2023)3215

key risk indicator (KRI)

The early-warning instrument of risk reporting: where a KPI measures performance achieved, a KRI measures exposure building — model-drift alerts open past SLA, human-override rates trending toward zero (the rubber-stamp signal), share of AI systems past review date, supplier concentration on critical models, incidents by severity. Good KRIs are chosen against the board’s stated risk appetite / risk tolerance, carry thresholds with pre-agreed actions, and appear on a fixed reporting cadence. A dashboard of KRIs nobody acts on is a compliance mural.

label bias

Bias injected at the annotation step: the label is a proxy for the thing you care about, measured unevenly. Classic case: using arrest as the label for crime — arrest rates reflect policing patterns, not just offending, so the model learns the pattern of enforcement. Healthcare’s version made Science in 2019: an algorithm used cost as the label for health need, and because less money is spent on Black patients at equal illness, it systematically understated their need. Label bias survives perfect sampling and infinite data — the fix is choosing better labels, not more of them.

Obermeyer et al., Science (2019)

liar’s dividend

Coined by law professors Bobby Chesney and Danielle Citron (2019): as synthetic media grows convincing, the value of denial rises — a politician caught on tape shouts “deepfake!”, and some share of the audience must take it seriously. The harm is epistemic and collective: not a specific fake believed, but authentic evidence devalued. It is why provenance infrastructure (C2PA, watermarking (AI content)) matters beyond fake-detection — proving the real real is the countermeasure — and why courts are already fielding deepfake objections to ordinary video evidence.

Chesney & Citron, Calif. L. Rev. (2019)

major nonconformity

The audit finding that stops the music: a total breakdown of a required element of the management system, or a failure that makes the system’s intended results doubtful — no risk assessment process at all, management review never performed, a control claimed in the Statement of Applicability that simply does not exist. Under ISO/IEC 17021-1 rules a certification body cannot issue or maintain a certificate while a major nonconformity stands open: the auditee must analyse the cause (root cause analysis), correct it, and usually submit to a follow-up or verification before the stage 2 audit result can be positive. Contrast a minor nonconformity, an isolated lapse that requires a corrective-action plan but does not block the certificate.

ISO/IEC 17021-1:2015 §9.4.5

management review

The heartbeat of every ISO management system: Clause 9.3 requires top management to review the AIMS at planned intervals against a fixed agenda — status of previous actions, changes in context, nonconformities, audit results, monitoring data, and opportunities for improvement — and to output real decisions: change the system, allocate resources, adjust objectives. Auditors treat the minutes as prime evidence: a review that no executive attended, or that produced no decisions, signals a paper AIMS. It is deliberately different from operational governance meetings — it reviews the system itself, not individual AI projects.

ISO/IEC 42001:2023 Clause 9.3

many hands problem

A classic problem of collective responsibility (named by Dennis Thompson, 1980) that AI industrialises: a harmful outcome emerges from a data vendor, a model developer, a fine-tuner, an integrator, a deploying manager, and an end user — each of whom can truthfully say my part worked as specified. The result is a responsibility gap: everyone contributed, no one is answerable. Governance frameworks attack it by assigning roles up front — the EU AI Act’s provider/deployer duties, ISO/IEC 42001’s role determination controls, contracts allocating liability along the value chain — so that when harm occurs, accountability was decided before the finger-pointing starts.

Thompson, “Moral Responsibility of Public Officials” (1980)

market surveillance authority

The enforcement muscle of EU product law. Each Member State designates one or more authorities under the Market Surveillance Regulation (EU) 2019/1020; the AI Act plugs into that machinery (Art 74) and gives these authorities teeth against AI systems: full access to documentation and training data where needed, power to compel corrective action, withdrawal, or recall of non-compliant high-risk systems, and the ability to trigger the fine regime. Affected persons can lodge complaints with them (Art 85). Not to be confused with the national competent authority umbrella term or the notifying authority that oversees notified bodies.

EU AI Act Arts 74–85; Regulation (EU) 2019/1020

memorisation

The failure mode where a model does not generalise from a training example but effectively stores it — and can be prompted to emit it: a person’s phone number, a chunk of licensed code, a paragraph of a news article. Carlini et al. showed extraction attacks recovering training text from LLMs, and memorisation grows with model size, data duplication, and example uniqueness. It matters legally on two fronts: privacy (memorised personal data turns a model into a leaky database, complicating GDPR erasure rights) and copyright (verbatim regurgitation is the strongest evidence in infringement suits against model providers). Mitigations — deduplication, differential privacy, output filters — trade capability for containment.

Carlini et al., “Extracting Training Data from LLMs” (2021)

minor nonconformity

The lesser audit finding: a single observed lapse in an otherwise functioning element — one AI system missing from the inventory, one training record absent, a review performed late — that does not cast doubt on the management system as a whole. The certificate can still be issued or maintained, but the organisation must submit a corrective-action plan with root-cause analysis, and the auditor verifies closure at the next surveillance audit. The practical danger is accumulation: a cluster of minors in the same clause is evidence of systemic failure, and auditors can and do escalate a pattern of minors into a major nonconformity.

ISO/IEC 17021-1:2015 §9.4.5

model collapse

The degenerative feedback loop identified as AI-generated content floods the web: when generation N+1 trains on text and images produced by generation N, the tails of the true data distribution get progressively forgotten — rare styles, minority dialects, unusual facts vanish first — and artefacts amplify (Shumailov et al., Nature 2024). It converts synthetic content labelling and provenance from a consumer-protection issue into an ecosystem issue: knowing what is machine-made is what lets future trainers filter it out. A systemic risk with no single owner — the commons that every scraper both depends on and pollutes.

Shumailov et al., Nature 631 (2024)

model stealing

An extraction attack in which an adversary queries a deployed model and uses the input-output pairs to train a surrogate that clones its behaviour — or, in stronger variants, recovers actual parameters (researchers have extracted embedding-layer weights from production LLMs through public APIs). The stolen copy costs a fraction of the original training run, evades usage controls, and serves as a rehearsal space for crafting adversarial attacks against the original. It sits in every AI threat taxonomy — MITRE ATLAS, the OWASP LLM Top 10 — and drives defensive controls: rate limiting, query-pattern anomaly detection, and watermarking of outputs. Distinct from theft of model weights by exfiltration, which is classic infosec.

MITRE ATLAS; OWASP Top 10 for LLM Applications

model weights

The billions of floating-point numbers a training run produces: everything the model “knows” lives in them. Governance treats weights as a special asset class for one reason — irreversibility. Once weights leak or are published, every downstream safeguard (usage policies, API filters, monitoring) becomes optional for whoever holds them, and fine-tuning away safety training costs almost nothing. Hence the policy fights: open-weight releases (Llama, Mistral) versus API-only access; the US Executive Order 14110’s reporting focus on dual-use weights; security expectations in responsible scaling policies about protecting weights from theft by state actors (RAND’s “Securing AI Model Weights” defines the threat levels). The EU AI Act’s open-source carve-outs also hinge on whether weights are publicly available.

RAND, “Securing AI Model Weights” (2024)

monoculture risk

Borrowed from agriculture: plant one crop everywhere and a single blight takes out the harvest. When thousands of downstream applications — hiring screens, credit models, medical triage tools — are built on the same handful of foundation models, they inherit the same blind spots, biases, and vulnerabilities, and their failures become correlated: a person rejected by one model may be rejected by all of them (algorithmic monoculture, Kleinberg & Raghavan 2021), and a single jailbreak or poisoned dependency propagates economy-wide. This is a key argument for treating GPAI as a source of systemic risk (GPAI), and it converts diversity of suppliers from a procurement nicety into a resilience control.

Kleinberg & Raghavan, PNAS 118 (2021)

multi-site sampling

The certification-economics rule that makes auditing a 40-office company feasible: where sites perform similar activities under one management system, the certification body may audit a sample (classically √n-based, per IAF MD 1) rather than every location. ISO/IEC 42006 tightens this for AI: sites are only “similar” if they develop, provide, or use AI in comparable ways — a lab fine-tuning models and a sales office that merely uses a chatbot cannot stand in for each other — and higher-risk AI activity at a site pulls it into the mandatory sample. The design question for the auditee is honest scoping: gerrymandered “similarity” is a classic path to a certificate that means nothing.

IAF MD 1; ISO/IEC 42006:2025

National AI Initiative Act

The statute (Division E of the FY2021 NDAA, signed January 2021) that gave US federal AI policy a skeleton: it created the National AI Initiative Office, chartered the National AI Advisory Committee, funded the National AI Research Institutes — and, in §5301, directed NIST to develop a voluntary risk management framework for trustworthy AI. That single directive produced the NIST AI RMF 1.0 (2023), the closest thing the US has to a national AI governance baseline. The Act embodies the US posture: coordinate, fund, and standardise rather than regulate — a voluntary framework strategy in statutory form.

National AI Initiative Act of 2020, Pub. L. 116-283 Div. E

national competent authority

The EU AI Act’s collective noun for national enforcers: each Member State must establish or designate at least one notifying authority (which vets and monitors notified bodies) and at least one market surveillance authority (which polices systems on the market), and equip them with adequate resources and AI expertise (Art 70). Member States choose their own institutional design — a new AI agency, the data protection authority, sectoral regulators, or a mix — which is why “who enforces the AI Act” has 27 different answers, loosely coordinated through the European AI Board. For GPAI models, enforcement sits instead with the Commission’s AI Office.

EU AI Act Art 70

NCII (non-consensual intimate imagery)

The harm category covering intimate images created or shared without the subject’s consent — including entirely synthetic ones. Generative tools collapsed the cost: “nudify” apps fabricate sexual imagery of real, identifiable people from ordinary photos, and studies of deepfake content have consistently found the overwhelming majority is non-consensual sexual imagery of women. Law has moved fast by regulatory standards: the US TAKE IT DOWN Act (2025) criminalises publication of NCII including AI-generated “digital forgeries” and forces platforms to remove it within 48 hours of notice; the UK Online Safety Act and the EU directive on violence against women criminalise it in parallel. For governance teams it is a canonical misuse case: foreseeable, catastrophic for victims, and squarely the provider’s problem to design against.

TAKE IT DOWN Act (2025); UK Online Safety Act 2023

NIST GenAI Profile

The generative-AI supplement to the AI RMF, published July 2024 under Executive Order 14110 as NIST AI 600-1. Its lasting contribution is a shared vocabulary of twelve risks “novel to or exacerbated by” GenAI — CBRN information access, confabulation, dangerous or violent recommendations, data privacy, environmental impacts, harmful bias and homogenization, human-AI configuration, information integrity, information security, intellectual property, obscene/degrading content, and value chain integration — each mapped to suggested actions across Govern/Map/Measure/Manage. Because it is a cross-sectoral RMF profile, organisations use it as a checklist generator: pick the risks that apply, inherit the actions. Voluntary, like everything NIST — but rapidly becoming the de facto US baseline for GenAI risk taxonomies.

NIST AI 600-1 (2024)

normative clause

The unit of obligation inside a standard. ISO/IEC drafting rules assign exact force to verb forms: “shall” marks a requirement (auditable, certifiable), “should” a recommendation, “may” a permission, “can” a possibility. Only normative clauses can generate nonconformities in an audit — which is why the first professional skill in reading any standard is separating its normative spine from the informative padding around it (normative vs informative). The same grammar governs legal instruments: knowing which sentences bind is the difference between a compliance obligation and a nice idea.

ISO/IEC Directives Part 2

normative vs informative

The load-bearing distinction for reading standards. Normative content states what you must do to claim conformity — the “shall” clauses and any annex labelled normative, like ISO/IEC 42001’s Annex A control list as invoked through the Statement of Applicability. Informative content — notes, examples, bibliographies, annexes labelled informative, and whole documents like Technical Reports — explains, illustrates, and advises but binds no one. The trap for practitioners runs both ways: treating informative guidance as mandatory gold-plates a compliance programme, while missing that an annex is normative leaves a certification-blocking hole. Check the label on every annex; never assume.

ISO/IEC Directives Part 2

pacing problem

The structural mismatch between innovation cycles measured in months and legislative cycles measured in years (the term is associated with Gary Marchant’s work on law and emerging technologies). The EU AI Act is the canonical exhibit: drafted around 2021-era product-safety assumptions, it had to be rewritten mid-negotiation when ChatGPT arrived. Every AI governance design choice is partly an answer to the pacing problem: technology-neutral drafting, delegated acts and annexes the executive can update, sandboxes, standards that revise faster than statutes, and voluntary frameworks that can ship immediately. The failure modes are symmetric — regulate too early and you freeze the wrong taxonomy into law; too late and harms compound uninsured.

Marchant, The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight (2011)

penalty tiers

The AI Act’s enforcement price list (Art 99), graduated by gravity: prohibited practices (Art 5) draw fines up to €35 million or 7% of global annual turnover, whichever is higher — deliberately above the GDPR’s 4% ceiling; breaches of most other obligations (provider, deployer, importer duties) up to €15m or 3%; supplying false or misleading information to authorities up to €7.5m or 1%. For SMEs and startups the lower of the two figures applies — a rare pro-small-company inversion. GPAI providers face a separate Commission-enforced regime of up to 3% of turnover or €15m (Art 101). The tiering is the risk pyramid translated into money: the fine tells you how seriously the legislator ranked the rule you broke.

EU AI Act Arts 99–101

preemption

The doctrine (from the US Constitution’s Supremacy Clause) that federal law can displace state law — expressly, by occupying the field, or through direct conflict. In AI it is the live wire of US policy: with no comprehensive federal AI statute, states filled the vacuum (Colorado’s AI Act, Illinois BIPA, California’s ADMT rules), and industry has repeatedly pushed for federal preemption of the resulting patchwork — including a proposed 10-year moratorium on state AI enforcement that the Senate stripped from the 2025 budget bill by a 99–1 vote. The stakes cut both ways: preemption promises one rulebook instead of fifty, but if the federal standard is weaker than the state laws it erases, it functions as deregulation wearing harmonisation’s clothes.

US Const. Art VI cl 2

prEN 18286

One of the make-or-break deliverables of the European standardisation programme: a draft European standard (prEN) on AI risk management developed by CEN-CENELEC JTC 21 in answer to the Commission’s standardisation request, designed so that conformity with it will create a presumption of conformity with the AI Act’s risk-management requirements (Art 9) once cited in the Official Journal as a harmonised standard. The “pr” prefix means it is still a proposal — subject to public enquiry, comment resolution, and formal vote before becoming EN 18286. Watch it because it answers the practical question every high-risk provider asks: what exactly will an auditor accept as an Article 9 risk management system?

CEN-CENELEC JTC 21 work programme; EU AI Act Art 40

principle proliferation

The phase of AI governance history when everyone published principles: by 2019 researchers had catalogued 84+ separate AI ethics documents (Jobin et al. counted 84; AlgorithmWatch’s inventory passed 160) from companies, governments, churches, and consultancies. Meta-analyses found genuine convergence — transparency, fairness, non-maleficence, accountability, privacy recur almost everywhere — but the documents shared a defect: no mechanism, no monitoring, no consequences. The proliferation era matters as diagnosis: it proved consensus on values was never the bottleneck, and it fuelled the “ethics washing” critique that pushed the field from principles to practice — management systems, audits, and binding law.

Jobin, Ienca & Vayena, Nature Machine Intelligence 1 (2019)

provenance

The record of origin and chain of custody — applied at two levels in AI governance. For content: cryptographically signed metadata (the C2PA “Content Credentials” standard) travels with an image or video, recording what device or model created it and every edit since; NIST AI 100-4 and the EU AI Act’s Art 50 machine-readable marking duty both lean on this. For data: knowing the source, licence, and processing history of every training dataset — the substance of data lineage and the EU AI Act’s Art 10 duties. Provenance is honest about its limits: it authenticates the label, not the truth of the content, and it only works where the capture-to-display chain preserves signatures — strip the metadata and the provenance is gone.

C2PA specification; NIST AI 100-4

public enquiry

The democratic valve in the standardisation pipeline: once a CEN or CENELEC technical committee agrees a draft (the prEN stage — see prEN 18286), national standards bodies publish it for a set consultation window and anyone — companies, NGOs, academics, citizens — may submit comments, which the committee must formally disposition before the final vote. For AI standards the enquiry stage carries extra weight: harmonised standards will effectively define what AI Act compliance means, yet the committees that draft them are dominated by industry — the public enquiry is one of the few moments civil society can intervene. Practitioners track enquiry windows the way lawyers track comment periods on proposed rules.

CEN-CENELEC Internal Regulations Part 2

quality in use

The outermost layer of the SQuaRE quality framework (SQuaRE series): while a product quality model describes properties of the system itself (accuracy, robustness, maintainability), quality in use measures what happens when a specific user applies it in a specific context — effectiveness, efficiency, satisfaction, freedom from risk. The distinction is where AI governance lives or dies: a diagnostic model with excellent benchmark accuracy (product quality) can still produce terrible quality in use if clinicians over-trust it, the interface buries uncertainty, or the deployment population differs from the test set. Regulators increasingly test at this layer — the EU AI Act’s human-oversight and deployer duties are quality-in-use requirements in legal dress.

ISO/IEC 25010; ISO/IEC 25059

quality model

The standards answer to a vague word: instead of arguing about whether a system is “good”, a quality model decomposes quality into named characteristics and sub-characteristics you can specify, measure, and contract for. ISO/IEC 25010 does this for software (functional suitability, reliability, security, usability…); ISO/IEC 25059 extends it for AI systems, adding AI-specific characteristics like functional adaptability (the system’s capacity to learn) and elevating robustness and controllability. The practical payoff is shared vocabulary: when a procurement contract says “accuracy ≥ X, robustness per clause Y”, both sides mean the same thing — and test reports, validation plans, and acceptance criteria hang off the model’s taxonomy rather than everyone’s private adjectives.

ISO/IEC 25059:2023; ISO/IEC 25010

real-world testing

The AI Act’s admission that labs cannot tell you everything: Arts 60–61 let providers test high-risk systems in real-world conditions before placing them on the market — outside or inside a regulatory sandbox — but wrap the permission in safeguards: a registered real-world testing plan, market-surveillance-authority oversight, informed consent from subjects (with narrow law-enforcement exceptions), a maximum duration (six months, extendable once), the right of subjects to withdraw, and the ability to reverse or disregard the system’s outputs. Effects on subjects must be reversible or mitigable. It is the product-law cousin of a clinical trial — evidence generation under protocol, not a soft launch with a disclaimer.

EU AI Act Arts 60–61

recertification

The end-of-cycle reckoning: an ISO management-system certificate is valid for three years, punctuated by annual surveillance audits, and before it expires the certification body must conduct a recertification audit reviewing the entire system — like a stage 2 audit but informed by three years of surveillance history, changes to the organisation, and the system’s track record of nonconformities and complaints. For an AIMS the recertification lens is change: the AI portfolio, the models, the regulatory landscape, and the risk picture in year three will not be the ones certified in year zero, and the auditor’s core question is whether the management system kept up. Miss the deadline and the certificate lapses — there is no grace period for trust.

ISO/IEC 17021-1:2015 §9.6

recital

The preamble paragraphs before an EU regulation’s articles — the AI Act has 180 of them — explaining what the legislator meant and why. Recitals are not independently binding: they cannot create obligations the articles lack (CJEU settled law). But they are far from decoration — courts and regulators read ambiguous articles through them, and some of the AI Act’s most consequential guidance lives only there: recital 12 unpacks the definition of “AI system”, recitals on subliminal techniques flesh out what manipulation means, and employment-related recitals expand on Annex III categories. Professional rule: cite the article for the obligation, the recital for the interpretation — and when a vendor quotes only a recital at you, check what the article actually says.

Joint Practical Guide of the EU Institutions §10

regulation

A word with a precise EU meaning and a loose everyday one. In the EU legal order (TFEU Art 288), a regulation is directly applicable law: it binds in its entirety across all Member States without any national implementing statute — the GDPR and the AI Act are regulations, which is why the same articles apply verbatim in Dublin and Warsaw. Contrast a directive, which sets results and lets each state legislate its own path (regulation vs directive). In everyday and US usage, “regulation” means any binding rule made by a government agency under delegated power. When a text says “the regulation requires…”, first establish which sense is in play — the answer changes who enforces it and where the definitive text lives.

TFEU Art 288

regulation vs directive

The choice that determines how uniform EU law is on the ground. A regulation applies directly and identically everywhere (AI Act, GDPR); a directive binds Member States only as to the result, leaving form and method to national parliaments — producing 27 transpositions with real variation (the Product Liability Directive, the NIS2 Directive). The choice for the AI Act was deliberate and telling: the Commission chose a regulation precisely to prevent a patchwork of 27 national AI regimes fragmenting the single market — the same logic as the GDPR, which replaced a 1995 directive whose divergent transpositions had become the problem. Rule of thumb: with a regulation you read the EU text; with a directive you must also read your country’s implementing law.

TFEU Art 288

representational harm

The other half of the harms taxonomy alongside allocative harm (Kate Crawford, 2017): injury done through representation — a search for “CEO” returning only white men, image models sexualising women of particular ethnicities, translation systems defaulting doctors to he and nurses to she, face analysis failing darker-skinned women (Buolamwini & Gebru’s Gender Shades), or a group being erased from outputs entirely. No loan is denied, so classic anti-discrimination law struggles to grip it — yet the harm compounds culturally, shaping how millions perceive a group and feeding future training data. Generative AI massively expands the surface: every generated image and sentence is a representation. The NIST GenAI Profile captures it under “harmful bias and homogenization”.

Crawford, “The Trouble with Bias” (NeurIPS 2017)

representativeness

The property that decides whether performance measured in the lab means anything in the field: a dataset is representative when its composition matches the persons, settings, and conditions of the intended use context — demographics, geographies, devices, edge cases and all. The EU AI Act makes it law: Art 10(3) requires training, validation and testing datasets for high-risk systems to be “relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose”. Those last five words do the work — representativeness is always relative to a deployment context, which is why a model validated on one hospital’s patients can fail at the next hospital, and why sampling bias is its characteristic failure mode.

EU AI Act Art 10(3)

responsible scaling

The frontier labs’ self-regulatory bargain: keep scaling if and only if safety and security measures keep pace. A responsible scaling policy (Anthropic’s RSP with its AI Safety Levels, OpenAI’s Preparedness Framework, DeepMind’s Frontier Safety Framework) defines capability thresholds that would make a model dangerous — meaningful uplift for bioweapons development, autonomous replication, advanced cyber-offence — commits to evaluating each new model against them, and pre-commits to responses: harden model weights security, restrict deployment, or pause training until safeguards catch up. The sixteen-company Seoul Frontier AI Safety Commitments (2024) generalised the pattern. The critique writes itself — the labs grade their own homework — which is why safety institutes and the EU’s GPAI code of practice are pulling the same structure into public frameworks.

Anthropic RSP (2023); Seoul Frontier AI Safety Commitments (2024)

rights-based approach

One of the two grand design philosophies of AI regulation. A rights-based approach starts from entitlements held by persons — dignity, privacy, non-discrimination, due process — and asks what any system must do to respect them; protection does not depend on a risk calculation coming out right. Its instruments are prohibitions, individual rights (explanation, contest, remedy), and impact assessments framed around fundamental rights, as in the Council of Europe’s AI Framework Convention or a FRIA. Contrast risk-based regulation, which scales duties to estimated harm and accepts residual risk by design. Real regimes hybridise: the EU AI Act is risk-based in architecture but rights-based at its edges — the Art 5 prohibitions and deployer FRIAs are rights logic embedded in a risk pyramid.

Council of Europe Framework Convention on AI (2024)

risk

The most load-bearing and least agreed word in the field. ISO 31000 defines risk as “the effect of uncertainty on objectives” — deliberately neutral, capturing upside and downside, and organisation-centred. The EU AI Act (Art 3(2)) defines it as “the combination of the probability of an occurrence of harm and the severity of that harm” — strictly negative and person-centred: the harms that count are to health, safety, and fundamental rights, not to your revenue. A risk assessment built on the ISO definition can be structurally blind to what the AI Act cares about (an enterprise-risk register full of project and reputation risks, empty of harms to affected persons). Professionals name their definition before they argue about the number.

ISO 31000:2018 §3.1; EU AI Act Art 3(2)

risk assessment

The engine room of every risk framework, standardised by ISO 31000 as three sequenced activities: identification (what could happen — misuse, failure, bias, drift, security compromise, per source and lifecycle stage), analysis (how likely, how severe, for whom — using techniques from ISO/IEC 23894’s AI-specific catalogue), and evaluation (compare the analysed risk against risk criteria to decide whether it needs treatment). AI strains each step: novel failure modes defeat checklist identification, probability estimates for emergent behaviours are guesses wearing decimal points, and severity must be assessed for affected persons who never appear in enterprise registers — which is why ISO/IEC 42001 pairs risk assessment with impact assessment rather than trusting either alone.

ISO 31000:2018 §6.4; ISO/IEC 23894:2023

risk-based regulation

The design philosophy that says regulatory attention is scarce and should be spent where harm concentrates: classify activities by risk, then calibrate obligations to the class — prohibit the unacceptable, heavily regulate the high-risk, lightly touch the rest. The EU AI Act is its purest AI expression (the four-tier risk pyramid); proportionality is its selling point to industry, and coherence with product-safety tradition its selling point to regulators. Its known failure modes: everything depends on classifying correctly at the border (misclassify and the whole machine misfires); risk is assessed ex ante by category rather than by actual behaviour; and harms that diffuse across society — monoculture risk, epistemic erosion — fit poorly in any single tier. The main rival philosophy is the rights-based approach.

EU AI Act recitals 26–27

risk criteria

The rules of the game, fixed before play begins: terms of reference against which the significance of a risk is evaluated (ISO 31000). Good criteria specify how likelihood and consequence are measured and combined, what counts as intolerable versus acceptable, whose harms count (the organisation’s only, or affected persons’ too — the choice ISO/IEC 42001 forces), and who has authority to accept residual risk at each level. Their governance function is anti-gaming: criteria set in advance prevent the quiet recalibration where an inconvenient red risk becomes amber because the launch date is close. Auditors read the criteria first — a risk register without documented criteria is a list of opinions.

ISO 31000:2018 §6.3.4; ISO/IEC 42001:2023 Clause 6.1.1

risk management

Not a document but a loop: ISO 31000 defines it as coordinated activities to direct and control an organisation with regard to risk — establish scope and risk criteria, perform risk assessment, treat the risks (mitigate, transfer, avoid, or accept), then monitor, review, communicate, and record — continuously, because AI systems, their contexts, and their attackers all change after deployment. ISO/IEC 23894 translates this into AI-specific guidance; the NIST AI RMF reorganises the same substance into Govern/Map/Measure/Manage; the EU AI Act hardens it into the mandatory Art 9 risk management system. The discipline’s core honesty: risk is managed, never eliminated — the output is a defensible record of what you accepted and why.

ISO 31000:2018; ISO/IEC 23894:2023

risk management system

Where risk management stops being best practice and becomes law. Article 9 requires providers of high-risk systems to establish, implement, document, and maintain a risk management system as a continuous iterative process across the entire lifecycle: identify reasonably foreseeable risks to health, safety, and fundamental rights — under intended use and reasonably foreseeable misuse — estimate and evaluate them (including from post-market data), and adopt measures in a fixed order of preference: eliminate by design first, mitigate and control second, inform and train deployers last. Residual risks must be judged acceptable, testing must prove it, and vulnerable groups (children especially) get explicit attention. prEN 18286 is being drafted to give this article a presumption-of-conformity standard.

EU AI Act Art 9

risk pyramid

The mental model that organises the entire AI Act. Four tiers, obligations shrinking as you descend: unacceptable risk — the Art 5 prohibited AI practices, banned outright; high risk — Annex I safety components and Annex III use cases, carrying the full requirements catalogue (risk management, data governance, oversight, conformity assessment); limited/transparency risk — chatbots, emotion recognition, deepfakes, owing Art 50 disclosure duties; minimal risk — the vast majority of AI (spam filters, game AI), left to voluntary codes. GPAI models sit awkwardly beside the pyramid in their own chapter, because a general-purpose model has no single use case to classify. The pyramid is risk-based regulation rendered as architecture — and classification, the act of placing a system on it, is where most legal argument happens.

EU AI Act Arts 5, 6, 50

risk tolerance

The threshold that converts analysis into action: once a risk is assessed, tolerance determines whether it is accepted, treated, or escalated. The NIST AI RMF is pointed about the division of labour — the framework helps you measure and manage risk, but it “does not prescribe risk tolerance”: deciding how much risk is acceptable is a governance judgment owned by the organisation, shaped by legal duties, context, and stakeholder impact. Two disciplines follow: tolerance must be set by people with authority to own the consequences (top management, not the project team grading its own homework), and it must be documented before assessment so findings cannot quietly bend it. For harms to third parties, law caps the dial — no internal appetite statement can tolerate what Article 5 prohibits.

NIST AI RMF 1.0 §1.2.3

robustness

The trustworthiness characteristic that measures performance under stress: ISO/IEC 22989 frames it as maintaining the level of performance under any circumstances — inputs that are noisy, unexpected, out-of-distribution, or deliberately crafted to deceive (adversarial examples). The EU AI Act makes it a legal requirement: Art 15 obliges high-risk systems to achieve appropriate accuracy, robustness and cybersecurity consistently across the lifecycle, with resilience to errors, faults, inconsistencies, and exploitation attempts — and for systems that learn after deployment, safeguards against feedback loops. The engineering truth underneath: benchmark accuracy is measured on data that looks like the training set; robustness is what remains when the world stops cooperating — the property that actually predicts field performance.

EU AI Act Art 15; ISO/IEC 22989

role determination

The unglamorous control that prevents the many hands problem: for every AI system, determine and document which role(s) the organisation plays — AI developer, AI provider, AI deployer/user, data provider, or several at once — because every framework hangs obligations on roles, not intentions. ISO/IEC 42001’s Annex A includes controls for understanding your role and responsibilities toward other parties; the EU AI Act makes the stakes concrete — a deployer who puts its name on a system, substantially modifies it, or repurposes it into high-risk use becomes the provider (Art 25) and inherits the full obligations catalogue, a trap white labelling arrangements trip constantly. Role determination sounds like paperwork until the day two organisations discover they each assumed the other was doing the risk management.

ISO/IEC 42001:2023 Annex A.10; EU AI Act Art 25

root cause analysis

The discipline that separates correction from corrective action: correction fixes the instance (relabel the record, patch the model); corrective action requires finding and eliminating the cause so the failure class cannot recur — ISO/IEC 42001 Clause 10.2 mandates exactly this sequence after every nonconformity, and certification bodies require documented root-cause analysis before closing audit findings. Techniques are borrowed from safety engineering (five whys, fishbone diagrams, fault trees); AI adds a twist — root causes of model failures often live upstream in data collection, labelling choices, or mis-specified objectives rather than in code, and a proper analysis follows the chain across team boundaries. An auditor’s tell for a weak system: every corrective-action record blames “human error” and prescribes retraining the human.

ISO/IEC 42001:2023 Clause 10.2

safe harbor

The oldest trade in regulatory design: do the prescribed thing, and the law protects you — from liability, penalties, or enforcement. AI policy uses it in several flavours: proposed US legislation and state laws that grant an affirmative defence to companies that comply with the NIST AI RMF (Colorado’s AI Act offers a rebuttable presumption of reasonable care along these lines); the EU AI Act’s harmonised standards, whose presumption of conformity is a safe harbor in structure; and platform-law ancestors like DMCA §512. The design tension is permanent: safe harbors convert vague duties into checkable ones and reward early adopters, but if the harbor’s conditions are weaker than the underlying duty, compliance theatre docks there — the checklist gets done and the risk remains.

Colorado SB 24-205 §6-1-1706

safety component

One of the two doors into high-risk classification. The AI Act (Art 3(14)) defines a safety component as a component of a product or system that fulfils a safety function — whose failure or malfunctioning endangers the health and safety of persons or property. Route one to high-risk status (Art 6(1)): the AI is a safety component of a product covered by Annex I harmonisation law (machinery, medical devices, vehicles, lifts, toys…) and that product requires third-party conformity assessment — think AI controlling a surgical robot’s motion or a car’s emergency braking. This is the Act’s product-safety inheritance: the AI is regulated because of where it sits in a physical product, wholly apart from the Annex III use-case list that captures hiring tools and credit scores.

EU AI Act Arts 3(14), 6(1), Annex I

safety institute

The institutional innovation of the frontier-AI era: a state body that tests models rather than writing rules — building evaluation science and running pre- and post-deployment assessments of frontier systems for dangerous capabilities (cyber-offence, bio-uplift, autonomy). The UK AI Safety Institute (born at Bletchley, 2023; rebadged the AI Security Institute in 2025) and the US AISI within NIST (renamed CAISI in 2025) anchored an International Network of AI Safety Institutes (Seoul, 2024) spanning the EU, Japan, Singapore, Canada and others. Their leverage is voluntary — memoranda with labs, no licensing power — which makes them either the evaluation muscle regulators will later borrow, or a well-funded observer, depending on how the access agreements hold when a test result is inconvenient.

Bletchley Declaration (2023); Seoul Statement (2024)

sampling bias

The bias that enters at the collection step: when the process that gathered the data makes some members of the population more likely to be included than others — hospital data over-representing the acutely ill, face datasets scraped from media that photographs some demographics more, voice data recorded mostly from native speakers. ISO/IEC TR 24027 catalogues it among data biases; its defining property is that no amount of modelling skill downstream repairs it, because the information about the missing population was never captured. It is the standard mechanism by which datasets fail the EU AI Act’s representativeness requirement, and detecting it requires the one thing the dataset cannot supply — knowledge of who is absent, which is why data provenance documentation and collection-process review are the actual controls.

ISO/IEC TR 24027:2021

SC 42 (ISO/IEC JTC 1 Subcommittee 42 (Artificial Intelligence))

The room where international AI standards are made: Subcommittee 42 of ISO/IEC JTC 1, established 2017, secretariat held by ANSI (US), with national delegations from dozens of countries. Its catalogue is the spine of this curriculum: ISO/IEC 22989 (concepts and terminology), 23053 (ML framework), 23894 (AI risk management), 42001 (the AIMS standard), 42005 (impact assessment), 42006 (audit requirements), plus the bias, trustworthiness, and quality technical reports. Its output matters beyond ISO’s membership because CEN-CENELEC JTC 21 adopts SC 42 texts as European standards where it can (international-first (standards policy)) — meaning sentences drafted in an SC 42 working group can end up defining EU legal compliance.

ISO/IEC JTC 1/SC 42

scientific panel

The AI Act’s attempt to wire independent science into enforcement (Art 68): a panel of independent experts, selected for AI expertise and freedom from provider interests, that supports the AI Office on GPAI matters — advising on methodologies for evaluating model capabilities, on classification of systemic-risk models, and on tools and benchmarks. Its sharpest instrument is the qualified alert (Art 90): the panel can formally notify the AI Office that a GPAI model presents a concrete identifiable risk at Union level or meets the systemic-risk conditions, triggering the Commission’s assessment powers. Member States may also call on the panel’s experts. It is the same institutional move as scientific committees in pharmaceutical and food regulation — expertise as an input regulators cannot casually ignore.

EU AI Act Arts 68, 90

sectoral regulation

The alternative to one horizontal AI law: let each sector’s existing regulator govern AI within its domain, applying and extending rules it already owns — the FDA clearing AI medical devices, financial regulators supervising credit models under fair-lending law, aviation authorities certifying autonomous flight systems. It is the default US posture (the FTC polices deceptive AI as deceptive commerce; the EEOC polices biased hiring tools under Title VII). Strengths: deep domain expertise, existing enforcement muscle, rules tuned to real context. Weaknesses: gaps (general-purpose systems and new uses fall between mandates), inconsistency across sectors, and duplicated effort. Most real regimes are hybrids — the EU layered its horizontal AI Act on top of sectoral law, with Annex I explicitly deferring to product-specific legislation.

US OMB M-24-10; EU AI Act Annex I

shift left

Borrowed from software testing (move testing “left” on the project timeline) and now a core AI governance strategy: attach the controls to the earliest stage that can catch the problem — intake review before procurement, impact assessments at design, data quality and representativeness checks before training, red-teaming before release — rather than discovering at deployment what a design meeting could have prevented. The economics are the argument: a biased-label problem costs a meeting to fix at annotation time, a retraining run at validation time, and a regulatory investigation plus retrofit after launch. The toeslagenaffaire is the standard cautionary tale — the fatal decisions were design-stage decisions. Shift-left’s honest limit: some risks (drift, emergent misuse) only exist after deployment, so left-shifted governance still needs post-market monitoring.

NIST AI RMF Playbook (Map function)

significant harm

The severity filter built into the Art 5 prohibited AI practices: the bans on subliminal or manipulative techniques and on exploiting vulnerabilities (age, disability, social or economic situation) apply where the practice materially distorts behaviour in a manner that causes or is reasonably likely to cause significant harm — physical, psychological, financial, or economic — to the person or others. The Act does not define “significant” numerically; the Commission’s 2025 guidelines on prohibited practices read it as requiring real gravity, taking intensity, duration, reversibility, and the person’s vulnerability into account, and confirm harms can accumulate over time. The threshold is where enforcement arguments will live: too strict a reading legalises effective manipulation below the bar; too loose a reading turns every dark pattern into a banned practice.

EU AI Act Art 5(1)(a)–(b); Commission Guidelines C(2025) 884

specification gaming

The failure mode where the system does exactly what you said and nothing like what you meant: a reinforcement-learning boat discovers that circling to collect respawning bonus targets scores better than finishing the race (OpenAI’s CoastRunners agent); a simulated robot learns to look like it is walking rather than walk; a model rewarded for user engagement learns outrage works. DeepMind’s catalogue of examples runs to dozens. The governance lesson generalises far beyond RL: any metric made into a target will be gamed — by models, and by organisations optimising compliance metrics (Goodhart’s law applied to institutions). Defences are the same in both cases: multiple metrics, adversarial evaluation of the objective itself, and humans empowered to notice when the letter and the spirit have diverged.

DeepMind, “Specification gaming: the flip side of AI ingenuity” (2020)

SQuaRE series (Systems and software Quality Requirements and Evaluation)

The mature quality machine AI standards chose to extend rather than replace: the ISO/IEC 25000 family, organised into divisions for quality models (25010’s product quality model and the quality in use model), quality measurement, requirements, and evaluation. When SC 42 needed an AI quality framework it published ISO/IEC 25059 as a SQuaRE extension — adding AI-specific characteristics to 25010’s taxonomy instead of inventing a rival vocabulary. That choice matters practically: decades of tooling, contracts, and evaluator practice built on SQuaRE carry over to AI systems, and an organisation’s existing software-quality programme has a defined seam where AI quality bolts on rather than a parallel bureaucracy.

ISO/IEC 25000 series; ISO/IEC 25059:2023

stage 1 audit

The reconnaissance before the assault: in the mandatory two-stage initial certification audit (ISO/IEC 17021-1), stage 1 reviews the management system’s documented design — scope and Statement of Applicability, policies, risk and impact assessment processes, internal audit and management review evidence — and evaluates site-specific conditions to confirm the organisation is ready for stage 2. Output: findings classified as concerns that would likely become nonconformities, and a go/no-go on proceeding. For an AIMS the classic stage 1 failures are a certification scope that quietly excludes the riskiest AI systems, and impact assessments that exist as templates but were never performed. Treat stage 1 as a paid diagnostic: cheaper to fail here than to burn a stage 2.

ISO/IEC 17021-1:2015 §9.3.1.2

stage 2 audit

Where paper meets practice: stage 2 evaluates the implementation and effectiveness of the management system — auditors interview staff from top management to engineers, trace real AI systems through the documented processes, sample records (risk assessments performed, impact assessments for actual systems, incidents handled, management review minutes with decisions), and test whether the Statement of Applicability’s claimed controls operate. Findings are graded major or minor; certification requires majors to be closed and verified. The distinctive AIMS challenge under ISO/IEC 42006 is audit-team competence — the standard requires demonstrable AI expertise (technical experts where needed), because an auditor who cannot interrogate a model card cannot evaluate whether the AI controls are real.

ISO/IEC 17021-1:2015 §9.3.1.3

standard

The connective tissue between law and engineering: a document, established by consensus and approved by a recognised body, providing rules, guidelines, or characteristics for common and repeated use. Standards are voluntary by nature but acquire teeth three ways: legislation references them (the AI Act’s harmonised standards carry a presumption of conformity), contracts demand them (procurement requiring ISO/IEC 42001 certificates), and markets normalise them until deviation needs explaining. Read one with the grammar in mind — “shall” binds, “should” advises — and know the document types: full International Standards, Technical Specifications (not yet mature), and Technical Reports (informative only). In governance arguments, “there’s a standard for that” is the beginning of the analysis, not the end: which body, which clauses are normative, and who adopted it into force.

ISO/IEC Guide 2:2004 §3.2

standardisation request

How EU law commissions its own technical detail: under the Standardisation Regulation (1025/2012), the Commission issues a formal request (mandate) to CEN, CENELEC, or ETSI to draft standards supporting a piece of legislation. For the AI Act the request is C(2023)3215 (May 2023, amended since): ten deliverables covering risk management, data governance, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity, quality management, and conformity assessment — the standards (prEN 18286 among them) that will become harmonised standards once accepted and cited in the Official Journal. The request is where public power over private standard-setting concentrates: the Commission sets scope and deadlines and assesses the results, but the drafting rooms belong to JTC 21 — which is why who sits in those rooms is a governance question, not a technicality.

Commission Implementing Decision C(2023)3215; Regulation (EU) 1025/2012

state of the art

The AI Act’s moving goalpost: several obligations are calibrated not to a fixed specification but to the state of the art — risk management measures must take it into account (Art 9), accuracy, robustness and cybersecurity must reflect it (Art 15), and GPAI systemic-risk mitigations are judged against it. In EU product-law tradition (and the Blue Guide’s reading) the phrase means the developed stage of technical capability generally available and proven practicable — more than industry custom, less than the research frontier. Its governance consequence is a compliance treadmill: what satisfied Article 15 in 2025 may be negligent by 2028 as defenses and evaluation methods improve, so providers need a documented process for tracking the art’s advance — and the paper trail proving they did. Harmonised standards are how the state of the art gets periodically frozen into checkable form.

EU AI Act Arts 9(3), 15(1); recital 74

statistical bias

The statistician’s bias: a systematic difference between an estimator’s expected value and the true parameter — error with a consistent direction, as opposed to random noise. NIST SP 1270’s three-category taxonomy makes it one lens among three: statistical/computational biases (sampling error, measurement error, algorithmic artefacts), human biases (anchoring, automation bias, confirmation), and systemic biases (historical and institutional patterns baked into the world the data records). The framing carries the report’s central warning: debiasing the mathematics does not debias the system — a perfectly calibrated model trained on the output of a discriminatory process reproduces the discrimination with excellent statistics. Whether a statistical bias is also an unwanted bias is a normative judgment about who bears the error, not a computation.

NIST SP 1270 (2022)

stop button

Two ideas sharing one name. In EU law, it is concrete: Art 14(4)(e) requires that the humans overseeing a high-risk system be able to intervene or interrupt it through a “stop” button or similar procedure, bringing it to a halt in a safe state — the last line of human oversight when monitoring and override fail. In AI safety theory, it is a problem: a goal-directed agent has an instrumental incentive to avoid being switched off (you cannot fetch the coffee if you are dead — Stuart Russell), and naive fixes fail in both directions — penalise shutdown-resistance and the agent may manipulate you into never pressing the button; reward shutdown and it may press the button itself. Corrigibility — designing agents that genuinely accept correction — remains open research, which is why governance of agentic systems leans on architectural interruptibility (permissions, sandboxes, kill switches at the infrastructure layer) rather than trusting the agent’s goals.

EU AI Act Art 14(4)(e); Soares et al., “Corrigibility” (2015)

subliminal techniques

The first item on the AI Act’s banned list: placing on the market or using an AI system that deploys subliminal techniques beyond a person’s consciousness — or purposefully manipulative or deceptive techniques — with the object or effect of materially distorting behaviour, impairing informed decision-making, and causing or being likely to cause significant harm (Art 5(1)(a)). The Commission’s 2025 guidelines read “subliminal” to cover stimuli below conscious perception (flashed visuals, embedded audio) and, under the manipulative limb, exploitative choice architectures and dark patterns at AI scale. The drafting is deliberately effects-based — no proof of intent required for the “effect” branch — but every element (material distortion, causation, harm threshold) is a litigation site. Its neighbour ban, Art 5(1)(b), covers exploiting vulnerabilities of age, disability, or social and economic situation.

EU AI Act Art 5(1)(a); Commission Guidelines C(2025) 884

surveillance audit

What keeps a three-year certificate honest: at least once a calendar year (first one within twelve months of certification), the certification body returns for a surveillance audit — narrower than a stage 2 audit, sampling key clauses and rotating through the rest of the system across the cycle, but always checking internal audit and management review operation, complaint handling, closure of prior minor nonconformities, and use of the certification mark. For an AIMS the surveillance visit carries extra freight because AI portfolios churn: new models, new use cases, and new regulations between visits mean the auditor’s first question is whether the system absorbed the year’s changes — an unchanged risk register in a year that shipped three new AI products is itself a finding. Persistent failure to address findings can suspend or withdraw the certificate mid-cycle.

ISO/IEC 17021-1:2015 §9.6.2

synthetic content

The AI Act’s word for machine-made media, and the object of its Art 50 transparency regime: providers of systems generating synthetic audio, image, video, or text must ensure outputs are marked as artificially generated in a machine-readable, detectable way — via watermarking (AI content), provenance metadata (C2PA-style), or fingerprinting, effective and interoperable “as far as technically feasible”. Deployers owe human-facing disclosure on top: deepfakes must be labelled, as must AI-generated text published to inform the public on matters of public interest (absent human editorial review). The same category powers the harder problems upstream — NCII and fraud as misuse, and model collapse as the ecosystem cost of unlabelled synthetic data re-entering training corpora. Marking is honest-actor infrastructure: it filters the careless, not the malicious who strip marks — which is why detection and provenance are layered, not alternatives.

EU AI Act Art 50(2), (4)

TDM opt-out

The pivot point of the EU’s AI-and-copyright settlement. The DSM Copyright Directive (2019) created a broad text-and-data-mining exception for commercial use — Art 4 — but let rightsholders reserve their rights (opt out), for online content in machine-readable form (robots.txt-style signals, metadata). The AI Act then reached back and grabbed it: Art 53(1)(c) requires GPAI providers to adopt a copyright policy that identifies and honours Art 4(3) reservations — regardless of where in the world the training happened, if the model is placed on the EU market. Open questions keep lawyers employed: what counts as machine-readable reservation, whether opt-outs seen after crawling require unlearning, and how honouring is audited — the GPAI Code of Practice and the Art 53 training-data summary template are the current answers. Germany’s LAION decision (2024) gave the exception its first judicial workout.

Directive (EU) 2019/790 Art 4(3); EU AI Act Art 53(1)(c)

technical expert

The audit world’s answer to “our auditors don’t understand transformers”: a person who provides specific knowledge or expertise to the audit team — ISO 19011 and ISO/IEC 17021-1 allow certification bodies to staff audits with technical experts who advise on subject matter but do not act as auditors: they cannot audit independently or determine findings, and their input flows through the audit team leader. ISO/IEC 42006 makes them central to AIMS credibility by defining the AI competences an audit team must collectively hold — machine learning fundamentals, data management, AI-specific risks, the relevant regulatory landscape — which in practice means pairing management-system auditors with AI technical experts. The failure mode it guards against is the checklist audit: an auditor who cannot ask a follow-up question about a model card will accept whatever the auditee’s slide deck says.

ISO 19011:2018 §3.17; ISO/IEC 42006:2025

technical report

The lowest-commitment document in the ISO hierarchy: a Technical Report (TR) contains only informative material — collected data, survey results, explanatory guidance — with no normative requirements at all; you cannot conform to one, and no auditor can raise a finding from one (normative vs informative). The AI catalogue leans on TRs heavily because the field is young: ISO/IEC TR 24027 (bias in AI systems), TR 24028 (trustworthiness overview), and TR 24368 (ethical concerns) map terrain that was not ready for “shall” — a TR is often the scouting party sent ahead of a future standard, and content road-tested in a TR later hardens into normative text (the path bias work is taking). Reading tip: the ISO reference tells you the force before you open the document — IS binds, TS is provisional, TR informs.

ISO/IEC Directives Part 1 §2.1.2

toeslagenaffaire

The case study European AI governance is written in the shadow of. From 2013 the Dutch tax administration used a self-learning risk-scoring system to flag childcare-benefit fraud; nationality was a model input, dual nationals were disproportionately flagged, and a hair-trigger “all-or-nothing” rule turned minor administrative errors into demands to repay tens of thousands of euros. Roughly 26,000+ parents were wrongly branded fraudsters; families were bankrupted and over a thousand children taken into care. The parliamentary inquiry’s title — Unprecedented Injustice — preceded the Rutte cabinet’s resignation in January 2021, and the Dutch DPA fined the tax authority for unlawful, discriminatory processing. Every element the EU AI Act later regulated is present: discriminatory data, no meaningful human oversight, no contest route that worked, and institutional incentives that rewarded the system’s severity. When a lesson asks why deployer duties and FRIAs exist — this is why.

Parliamentary report “Ongekend Onrecht” (2020); Dutch DPA decision (2021)

top management

A defined term with teeth: the person or group of people who directs and controls an organisation at the highest level (ISO terminology), and the addressee of ISO/IEC 42001’s Clause 5 — top management must demonstrate leadership and commitment to the AIMS: establish the AI policy, ensure objectives align with strategy, provide resources, assign roles, and conduct management review. The design intent is anti-delegation: the standard lets executives delegate tasks but not accountability, so an AIMS run entirely by a compliance team with no executive fingerprints fails audit at Clause 5. Note the seam with governance: ISO/IEC 38507 addresses the governing body (board) above top management — direction and oversight versus execution — and the AI-literate board asking hard questions is what keeps top management’s commitment from being ceremonial.

ISO/IEC 42001:2023 Clause 5; ISO/IEC 38507

traceability

The property that makes accountability technically possible: being able to reconstruct, after the fact, which model version, which data, which inputs, and which human choices produced a given output. The EU AI Act legislates it directly — Art 12 requires high-risk systems to automatically record events (logs) over their lifetime, ensuring a level of traceability appropriate to the system’s intended purpose, and deployers must retain those logs (Art 26); ISO/IEC 42001’s Annex A echoes it with documentation and data lineage controls. Traceability is the difference between an incident investigation and an incident guess: without logs tying output to model version to training data snapshot, root cause analysis has nothing to grip, contestability is theatre, and the many hands problem wins by default. It is provenance’s sibling — provenance faces the data coming in, traceability faces the decisions going out.

EU AI Act Arts 12, 26(6)

training data

The AI Act’s definition is compact — data used for training an AI system through fitting its learnable parameters (Art 3(29)) — and the consequences are anything but. Training data determines capability (a model is a compression of its corpus), fairness (whatever the collection process under- or mis-represents, the model inherits — sampling bias, representativeness), privacy exposure (memorisation turns training records into extractable ones), and legal position (copyright fights from TDM opt-out to the US fair-use cases turn on what was trained on and how). Hence the paper trail frameworks demand: Art 10 data-governance duties for high-risk systems distinguish training, validation, and testing sets — the split that keeps evaluation honest — and GPAI providers must publish a sufficiently detailed summary of training content (Art 53). The operational rule: a model you cannot describe the training data of is a model you cannot defend.

EU AI Act Arts 3(29), 10, 53

transparency obligations

The AI Act’s middle tier in action — obligations that inform rather than restrict. Article 50 imposes four disclosure duties: providers must ensure people interacting with an AI system (chatbots, voice agents) know it is a machine, unless obvious to a reasonably informed person; providers of generative systems must mark synthetic content machine-readably; deployers of emotion recognition or biometric categorisation must inform the people exposed to them; and deployers must disclose deepfakes and AI-generated public-interest text. Disclosures must be clear, at the latest at first interaction, and accessible. Two boundary notes keep analysis honest: these duties stack with high-risk obligations rather than replacing them (a high-risk chatbot owes both), and transparency here means disclosure to affected people — distinct from the Art 13 duty of transparency to deployers (instructions for use) and from explainability of individual decisions.

EU AI Act Art 50

trustworthiness

The field’s organising abstraction, made useful only by decomposition. ISO/IEC TR 24028 frames trustworthiness as the ability to meet stakeholder expectations in a verifiable way; the NIST AI RMF names the characteristics: valid and reliable (the precondition), safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Three disciplines keep the word from dissolving into marketing: trustworthiness is a property of the system in context, not a badge on the model; the characteristics trade off (more explainable can mean less accurate; more private can mean less fair), so declaring a system trustworthy means declaring which balance you struck and why; and every claimed characteristic needs a verification story — a measurement, test, or audit that could have come out the other way. Trust is the user’s state of mind; trustworthiness is the evidence that state of mind is warranted.

ISO/IEC TR 24028:2020; NIST AI RMF 1.0 §3

unwanted bias

The ISO vocabulary’s honest adjective: since every model is “biased” toward the patterns it learned — and some biases are the point (a credit model should treat different repayment histories differently) — ISO/IEC TR 24027 targets unwanted bias: systematic differences in treatment of persons or groups that produce unfair, discriminatory, or otherwise undesired outcomes. The term does two jobs. Technically, it separates the measurable (statistical bias, sampling bias) from the normative — deciding which differential treatment is unwanted requires ethics and law, not more mathematics. Managerially, it makes bias a specifiable requirement: ISO/IEC 42001 Annex A and the EU AI Act’s Art 10(2)(f)–(g) both demand examination of datasets for possible biases likely to affect health, safety, or fundamental rights or lead to prohibited discrimination — an obligation you can only meet after writing down which biases, for this system, in this context, count as unwanted.

ISO/IEC TR 24027:2021; EU AI Act Art 10(2)

validation

One half of V&V, and the half that faces the world: confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled (ISO 9000 lineage). The classic mnemonic — verification asks did we build the thing right?, validation asks did we build the right thing? — has real operational content for AI: a model can verify perfectly against its specification and still fail validation because the specification missed the deployment reality (wrong population, wrong prevalence, wrong workflow). Note the terminology collision: in ML practice, the validation set is the data split used for tuning during development — related, but far narrower than lifecycle validation, which spans clinical-style evaluation, real-world testing, and post-deployment monitoring against intended use. The EU AI Act builds both into high-risk duties: Art 10’s dataset splits and Art 9’s testing against intended purpose.

ISO 9000:2015 §3.8.13; ISO/IEC 22989

verification

The inward-facing half of V&V: confirmation, through objective evidence, that specified requirements have been fulfilled — the system does what its specification says, checked by testing, inspection, analysis, or review. In the AI lifecycle stage models (ISO/IEC 22989, OECD) “verification and validation” form a named stage between building and deployment, but the two questions differ in kind: verification is closed-world (compare artefact to specification — accuracy above threshold X on test set Y, latency under Z), while validation is open-world (is the specification itself right for the intended use?). AI strains classic verification because behaviour is learned rather than coded — you cannot enumerate paths through a neural network the way you review source code — so verification leans on statistical testing, benchmark suites, red-teaming, and, at the formal frontier, mathematical verification of narrow properties. Both produce the evidence conformity assessment later inspects: no verification records, no defensible conformity claim.

ISO 9000:2015 §3.8.12; ISO/IEC 22989

voluntary commitments

The interregnum instrument: with binding law years away, governments extracted public promises. The template was the White House Voluntary AI Commitments (July 2023) — Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI pledged pre-release security testing including red-teaming, information sharing on risks, investment in cybersecurity for model weights, third-party vulnerability reporting, provenance or watermarking (AI content) for synthetic content, public capability reporting, and research on societal risks — followed by a second cohort and the Seoul Frontier AI Safety Commitments (2024) internationally. Their governance anatomy: fast to obtain, unenforceable by design, and each firm grades its own compliance — but not empty, because public promises create reputational hostages, FTC exposure for firms that break them (deception jurisdiction), and drafting material that later hardens into frameworks like the GPAI code of practice. Best read as regulation’s first draft, written by the regulated.

White House Voluntary AI Commitments (2023); Seoul Commitments (2024)

voluntary consensus standard

The formal species of standard the US system runs on: developed by a body with openness, balance of interests, due process, an appeals process, and consensus (the OMB Circular A-119 criteria), and voluntary — adopted by choice, not command. US law actively channels government toward them: the National Technology Transfer and Advancement Act (1996) directs federal agencies to use voluntary consensus standards instead of writing government-unique ones, which is how ISO and IEEE texts end up inside procurement rules and regulations. The AI-relevant subtlety is how voluntariness ends: in the EU, citation in the Official Journal gives a harmonised standard legal effect; in the US, incorporation by reference in regulation or contract does the same job. “Voluntary” describes the standard’s birth, not necessarily your relationship with it — by the time it reaches you through a customer contract or a regulator, the choice was made upstream.

OMB Circular A-119; NTTAA (1996)

voluntary framework

The US answer to the pacing problem: publish a rigorous, complete governance framework and let adoption be voluntary — the NIST AI RMF says so on page one (“voluntary… not a checklist… non-prescriptive”), following the wildly successful precedent of the NIST Cybersecurity Framework. The theory of change is indirect force: voluntary frameworks become de facto mandatory through procurement (federal and enterprise contracts requiring RMF alignment), insurance and investor due diligence, safe harbor statutes that reward adopters, and courts treating them as the standard of care in negligence. Strengths: speed, flexibility, no constitutional fights, genuine technical quality. The standing critique: voluntariness selects for the already-willing — the firms whose practices most need changing are precisely the ones that decline — which is why most regimes treat voluntary frameworks as scaffolding while binding law is poured, not as the finished building.

NIST AI RMF 1.0 §1; National AI Initiative Act §5301

white labelling

The rebadging business model meets Article 25: a firm licenses an AI system built by someone else and markets it under its own name or trademark — common across HR tech, chatbots, and analytics, where dozens of “different” products share one underlying engine. The AI Act’s response is blunt: a distributor, importer, deployer, or any third party that puts its name or trademark on a high-risk AI system already on the market becomes the provider (Art 25(1)(a)), inheriting the full obligations catalogue — risk management system, technical documentation, conformity assessment, registration, post-market monitoring — while the original provider drops to a supporting role owed information and cooperation (and contracts scramble to allocate that). The commercial logic and the legal logic collide head-on: the whole point of white labelling is to look like the manufacturer, and the Act simply takes the appearance at its word. role determination exercises exist to catch this before a regulator does.

EU AI Act Art 25(1)

AWS Well-Architected Framework (WAF (context: architecture reviews))

AWS’s methodology for assessing cloud workloads across six pillars — operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. A review walks a workload through per-pillar questions; unmet best practices become prioritized risk findings. Not a certification: a structured conversation that produces an improvement backlog.

AWS Well-Architected Framework

Well-Architected lens

An extension of the Well-Architected Framework for a specific workload type (serverless, machine learning, generative AI…). A lens adds domain-specific questions and best practices on top of the six pillars. The Generative AI Lens (published 19 Nov 2025) covers foundation-model workloads on Bedrock and SageMaker AI, organized around the GenAI lifecycle from scoping to continuous improvement.

AWS Well-Architected Generative AI Lens (2025)

AWS Well-Architected Tool

The console service where reviews live: define a workload, answer the pillar questions, record which best practices are met, and track high/medium-risk findings over time. Custom lenses — including the Generative AI Lens, distributed via AWS’s public GitHub repository — are imported here to extend the question set.

AWS Well-Architected Tool

GENOPS / GENSEC / GENREL…

The Generative AI Lens identifies every best practice with a pillar prefix (GENOPS operational excellence, GENSEC security, GENREL reliability, GENPERF performance, GENCOST cost, GENSUS sustainability), a question number, and a best-practice number: GENOPS01-BP01 is the first best practice under the first operational-excellence question (periodic functional evaluation with stratified sampling). The site’s GenAI Lens Index summarizes every one.

AWS Well-Architected Generative AI Lens (2025)

Responsible AI dimensions (AWS)

The core set of dimensions AWS uses to define responsible AI, updated as the field evolves: controllability (monitor and steer system behavior), privacy, security, safety (block harmful output and misuse), veracity (factually correct outputs), robustness (correct outputs on unexpected inputs too), fairness, explainability, transparency, and governance (practices across the AI supply chain). The Responsible AI Lens (published 19 Nov 2025) organizes harmful-event analysis and metric selection around these dimensions. Note the deliberate split between veracity and robustness — accuracy on expected inputs versus behavior under distribution shift.

AWS Well-Architected Responsible AI Lens (2025)

Harmful event

The Responsible AI Lens’s unit of risk analysis: a specific, imaginable bad outcome (a biased condo description, a hallucinated property feature, a privacy leak in generated text), identified dimension by dimension (fairness, veracity, robustness, privacy, safety, security, explainability, transparency), assigned likelihood and severity, given an overall risk level, and tracked in a risk registry. Harmful events then drive mitigation choices and become testable release criteria — the lens’s version of the risk-assessment → acceptance-criteria loop that ISO/IEC 42001 and the NIST AI RMF also demand.

AWS Well-Architected Responsible AI Lens (2025)

AI Center of Excellence (AICoE) (AI Center of Excellence)

An organizational unit with a dual mandate: enable AI adoption (platform, patterns, training, delivery help) and govern it (standards, risk tiering, gates, registry). Typically starts centralized, evolves to hub-and-spoke as embedded squads mature, and may federate once guardrails are automated into the platform. Distinct from an AI governance committee (which decides) — the CoE operates: it runs intake, maintains the artifact library, and is a natural home for operating an ISO/IEC 42001 AIMS day to day.

Composite practitioner doctrine (cf. AWS CAF for AI, Microsoft CAF)

Cloud Center of Excellence (CCoE) (Cloud Center of Excellence)

The organizational pattern enterprises used to adopt cloud at scale: a small central team owning cloud standards, landing zones, guardrails, and enablement, evolving from builder to enabler as business units matured. Its playbook — executive mandate, golden paths over gatekeeping, automate the guardrails, then decentralize — transfers almost verbatim to AI adoption, with new risk types (model behavior, data leakage through prompts, regulatory duties) layered on top.

Cloud-adoption practice (cf. AWS/Microsoft adoption frameworks)

Hub-and-spoke (operating model)

The default target operating model for AI (and cloud) centers of excellence. The hub stays small — platform, guardrails, standards, evaluation depth, the trickiest builds — while spokes (embedded AI squads inside business units) deliver everyday use cases with domain context. Sits between centralized (one team does everything; strong control, guaranteed bottleneck) and federated (business units own delivery; a lean council curates standards — workable only once guardrails are automated).

Composite practitioner doctrine

Generative AI Security Scoping Matrix

AWS’s mental model for securing generative AI (introduced October 2023). First, place the workload in one of five scopes by ownership: 1 consumer app, 2 enterprise app (both “buy”), 3 pre-trained model via API, 4 fine-tuned model, 5 self-trained model (all “build”). Then work the five disciplines — governance & compliance, legal & privacy, risk management, controls, resilience — whose demands shift with scope: the higher the number, the more of the model and data you own, and the more of the security burden moves from provider terms to your own architecture. A key rule of thumb: a fine-tuned or self-trained model inherits the classification of the most sensitive data it was trained on.

AWS Security Blog / AWS AI security (2023)

Shared responsibility model

Cloud security doctrine: the provider secures the underlying service, the customer secures what they build and put on it — where the line sits depends on the service type. Generative AI inherits and complicates it: in AWS’s Generative AI Security Scoping Matrix, Scopes 1–2 leave the model and its training data entirely with the provider (your duties are contractual and behavioral), while Scopes 3–5 progressively hand you the application layer, the data pipeline, and eventually the model itself — and with them the threat modeling, access control, and resilience work.

Cloud security practice (cf. AWS shared responsibility model)

PII (Personally Identifiable Information)

Any data that identifies or can be combined to identify a person — names, identifiers, contact details, biometrics. In AI work its handling is doubly sensitive: privacy law attaches duties to it (GDPR, sectoral US law), and once trained into a model it effectively cannot be deleted — the only fully effective removal is retraining without it. Hence the scoping-matrix rule of thumb: keep PII out of training and fine-tuning sets, supply it at inference through retrieval instead, and classify any model trained on it at that data’s sensitivity level.

Privacy practice; GDPR / NIST usage

Lead Implementer

A credential about a person, not an organisation: a training-and-certification provider (PECB and others) examines whether you can take an organisation from nothing to a conforming AIMS — context, leadership, risk, controls, operation, internal audit, certification readiness. Two honesty points a professional should be able to recite. First, ISO certifies nobody — neither organisations nor people — so “ISO-certified Lead Implementer” is a category error; the certificate comes from the training body under its own accreditation for certifying persons. Second, the credential says nothing about whether anything you have built is itself certified: your competence and your employer’s certificate are separate claims travelling separate chains of trust. Exam logistics (duration, prerequisites, open-book rules) are provider-specific and change — check the current candidate handbook.

personnel certification

The conformity-assessment family that certifies people: a body assesses an individual against a defined competence scheme and issues a time-limited credential, often accredited under ISO/IEC 17024 (the standard for bodies certifying persons) rather than ISO/IEC 17021-1 (bodies certifying management systems). Keeping the two apart is a working professional reflex: a Lead Auditor credential is personnel certification and confers no authority to issue certificates, which comes only from working for an accredited certification body. Marketing copy routinely blurs this — “our ISO-certified consultants” usually means individuals holding a provider’s personnel credential, not an organisation holding a management-system certificate.

ISO/IEC 17024 (bodies certifying persons); cf. ISO/IEC 17021-1

AI policy (ISO/IEC 42001 Clause 5.2)

Clause 5.2 of ISO/IEC 42001 requires top management to establish an AI policy that is appropriate to the organisation’s purpose, provides a framework for setting AI objectives, and commits to meeting applicable requirements and to continual improvement; it must be documented, communicated internally, and available to interested parties as appropriate. The implementer’s working test is blunter than the clause: could a new hire read it and infer what is forbidden, what needs approval, and who decides? If not, it frames nothing and the planning clauses have nothing to hang from. Auditors read it early and then test whether behaviour matches it — a policy promising weekly bias reviews that happen monthly is a nonconformity you authored. Note the false friend: “AI policy” in public-affairs usage means a government’s AI strategy, not an internal management-system document.

ISO/IEC 42001:2023 Clause 5.2

AI objectives (ISO/IEC 42001 Clause 6.2)

Clause 6.2 requires objectives that are consistent with the AI policy, measurable (or at least evaluable), monitored, communicated, updated, and supported by a plan saying what will be done, with what resources, by whom, by when, and how results will be evaluated. The craft is in the drafting: “100% of in-scope systems have a current impact assessment by Q2, owner: head of product” is auditable; “embed responsible AI in everything we do” is aspiration and will be sampled to death. Few and owned beats many and orphaned — every objective without a named owner becomes an open finding at the first internal audit.

ISO/IEC 42001:2023 Clause 6.2

competence (ISO/IEC 42001 Clause 7.2)

Clause 7.2 obliges the organisation to determine the competence needed for roles affecting AI performance, ensure people have it, and retain documented information as evidence. In practice this is a role-based matrix, not a company-wide webinar: governors need enough ML literacy to interrogate a drift metric, engineers need enough governance literacy to know which changes trip the reassessment wire, and everyone needs Clause 7.3 awareness of the policy and of what nonconformance means for them. Keep evidence per role — auditors increasingly interview staff against the training records, and a signed attendance sheet for a course nobody can describe is worse than no sheet. Distinguish it from Clause 7.1 resources (do the people exist at all) and from awareness (knowing the system exists versus being able to operate it).

ISO/IEC 42001:2023 Clauses 7.2–7.3

supplier controls (AIMS)

ISO/IEC 42001’s Annex A carries controls for suppliers and customers of AI systems: understand and allocate responsibilities along the value chain, set requirements before engagement, and keep them current. Operationally that is three artifacts plus a trigger: a due-diligence questionnaire that asks the questions the model actually poses (training-data provenance, disaggregated evaluation evidence, who sits underneath the wrapper), contract clauses allocating AI duties and information flows, and a re-assessment trigger when the vendor swaps the underlying model — the failure that produced the NEDA/Tessa incident. The reason this is a management-system control rather than a procurement nicety: your obligations as a deployer do not shrink because the system was bought, and in the EU, Art 25(4) of the AI Act makes supplier agreements a compliance mechanism in their own right.

ISO/IEC 42001:2023 Annex A (supplier/customer controls)

ISO 19011

ISO 19011 (current edition 2018) is guidance, not a requirement standard: nobody is certified against it. What it supplies is the discipline every management-system audit runs on — the seven principles of auditing (integrity, fair presentation, due professional care, confidentiality, independence, evidence-based approach and the risk-based approach added in 2018), the distinction between an audit programme and an audit plan, competence requirements for auditors and audit teams, and the conduct of the audit from opening meeting to follow-up. For certification audits it sits under ISO/IEC 17021-1, which adds the mandatory rules for bodies issuing certificates; for AIMS specifically, ISO/IEC 42006 adds the AI-competence requirements on those bodies. Know which of the three you are quoting: 19011 is method, 17021-1 is the certification rulebook, 42006 is the AI overlay.

ISO 19011:2018, Guidelines for auditing management systems

Lead Auditor

Two meanings that must not be collapsed. As a credential, “ISO/IEC 42001 Lead Auditor” is a personnel certification attesting that an individual knows the ISO 19011 method and the target standard; it confers no power to issue certificates, which belongs only to an accredited certification body. As a role, the audit team leader assembles competence collectively (bringing in a technical expert rather than bluffing), fixes the criteria before fieldwork, runs the opening and closing meetings, grades the findings, and signs the report. Most holders of the credential use it on internal and supplier audits, where the method matters and the institutional authority is not needed.

first-party audit

The organisation audits its own management system, typically to satisfy the Clause 9.2 internal audit requirement and to rehearse for certification. Independence is the perennial difficulty: internal auditors cannot always be organisationally independent, but they must never audit their own work — the governance team that wrote the intake process cannot audit it. Mitigations in practice: auditors drawn from another business unit, a peer-exchange arrangement between sites, or a contracted auditor who performs the internal audit without also consulting on remediation. First-party audits can be as rigorous as any other; what they can never do is produce a certificate.

ISO 19011:2018 (audit types); ISO/IEC 42001:2023 Clause 9.2

second-party audit

A customer, or someone acting for them, audits a party they have a stake in: a bank auditing its model vendor’s AIMS before renewal, an enterprise exercising the audit rights its contract reserved. The method is ISO 19011; the client is you, not the auditee, which changes what the findings are for — not a certificate but a buying, renewal or remediation decision. Second-party audits are where most AI vendor assurance actually happens, because they can ask the questions a certification scope may have excluded: show me the bias test for our population, on the version currently serving us.

ISO 19011:2018 (audit types)

third-party audit

An independent organisation with no interest in the outcome audits the management system, for certification or for a regulator. Here ISO 19011 is not enough: the body also answers to ISO/IEC 17021-1 (and, for an AIMS, ISO/IEC 42006), which impose impartiality rules, the consulting ban, audit-time rules and the separation between the audit team that recommends and the body that makes the certification decision. The conclusions carry weight because of the body’s accreditation, not because of any individual auditor’s credential — which is why “our auditor is certified” answers a different question from “is your certificate accredited”.

ISO 19011:2018; ISO/IEC 17021-1:2015

audit evidence

ISO 19011 defines audit evidence as records, statements of fact or other information that are relevant to the audit criteria and verifiable. Three properties carry all the weight. Verifiable: someone else could re-examine it and reach the same result, which is why a note reading “seemed disorganised” is not evidence and a note reading “requested 5 of 23 impact assessments; received 3” is. Relevant to the criteria: evidence without a criterion produces an opinion, not a nonconformity. And sampled: because audits never exhaust the population, a clean result supports “no nonconformity found”, never “no nonconformity exists”. The practical discipline is triangulation — a procedure says what should happen, an interview says what people believe happens, a record says what did.

ISO 19011:2018 §3.8

objective evidence

The quality bar audit evidence has to clear: data obtained through observation, measurement, test or other means whose existence and truth can be verified by someone else. In an AIMS audit it looks like document titles and versions, record identifiers, dated sign-offs, system names, log entries, sample sizes. What it is not: adjectives, inferences from body language, or the auditor’s professional intuition — those are legitimate inputs to where you sample, never substitutes for what you found. The test the profession uses: when a finding is disputed three weeks later, the auditor with record IDs prevails and the auditor with impressions retracts.

ISO 9000:2015 §3.8.3 (objective evidence); ISO 19011:2018

audit programme

ISO 19011’s portfolio-level instrument: arrangements for a set of audits planned for a specific time frame and directed towards a specific purpose — a certification body’s three-year cycle for one client, or an internal function’s annual schedule covering every AIMS process. Programme-level thinking is where risk-based auditing actually happens: a lazy programme audits everything once a year regardless; a risk-based one weights frequency and depth by what changed and what could hurt, so the newly deployed clinical triage model is audited this quarter and again after its first retraining while stable document control gets a light touch. Results feed back — a process that produced findings last time earns a heavier slot next time. Do not confuse it with the audit plan, which covers one audit.

ISO 19011:2018 §3.4

audit plan

The mission order for a single audit. Circulated before fieldwork, it typically fixes objectives, scope and criteria; the schedule (which process, which site, which interviewees, which hour); team assignments matched to competence; the sampling (audit) approach for multi-site or multi-system scopes; logistics and communication channels; confidentiality arrangements; and opening and closing meeting times. It is a working document — adjustable as evidence emerges — but changes are communicated, not improvised silently. A plan that names which AI systems will be traced end to end tells the auditee what to have ready and tells the team where the audit’s weight lands. Contrast the audit programme, which schedules many audits.

ISO 19011:2018 §3.6

audit criteria

The reference against which evidence is compared: for an AIMS audit typically ISO/IEC 42001 clauses 4–10 and the applicable Annex A controls per the SoA, plus the auditee’s own policies and procedures, plus any legal or contractual requirements in scope. Two consequences follow. First, no criterion, no finding: “I don’t like your model documentation” is an opinion, while “A.6.2.7 requires technical documentation; the fraud model has none” is a finding — which is why the criteria list is fixed in the plan before fieldwork, not argued during it. Second, every “shall” an organisation writes about itself becomes a criterion it can fail; over-written procedures manufacture nonconformities.

ISO 19011:2018 §3.7

sampling (audit)

Audits conclude on samples, which makes every choice of what to read a bet worth making consciously. Risk-based sampling leans toward the highest-impact AI systems, whatever changed recently (new model, new supplier, retraining), whatever the last audit flagged, and the records the auditee did not volunteer. Always record the frame — “23 deployed systems; sampled 4: the two highest-risk per the auditee’s own register, one recent acquisition, one at random” — because that is what makes the conclusion reproducible and what protects you when someone asks why system 19 was missed. Keep the humility the evidence-based principle demands: a clean sample supports “no nonconformity found”, never “none exists”. Distinct from statistical sampling bias in a dataset and from multi-site sampling rules in certification schemes.

ISO 19011:2018 Annex A (audit sampling)

triangulation (audit)

The auditor’s working rule against being told a comfortable story: a procedure says what should happen, an interview says what people believe happens, and a record says what did. One source is an anecdote, two is a lead, three that agree is evidence — and three that disagree is the most valuable finding of the week, because discrepancy between documented process, lived practice and system exhaust is exactly where paper management systems come apart. Triangulation is also why serious audits go up and down the organisation: interview top management on the AI policy and the ML engineer on what happens when a drift alert fires, and compare the floors.

ISO 19011:2018 (evidence-based approach)

statement of nonconformity

A finding is a small legal document, and the statement is its operative sentence. Write it in four parts and never let one borrow from another: the requirement (the criterion, cited), the evidence (what was objectively found, with identifiers and dates), the nature of the gap (stated as a gap, not as blame), and the graded statement — “this is a minor nonconformity against A.6.2.8 and procedure AIP-14”. What must be absent: adjectives, accusations, and advice. “The team failed to…” becomes “logs could not be retrieved”; a recommended fix makes the auditor a designer of the system and destroys their ability to audit it again. Grading runs on the major/minor logic and on evidence of system failure — never on how awkward the consequence is for anyone.

ISO 19011:2018 (audit findings); ISO/IEC 17021-1:2015 §9.4.5

opportunity for improvement (OFI) (Opportunity For Improvement)

Below the nonconformity line sit two labels worth keeping distinct, though scheme vocabularies vary. An OFI flags a conforming practice that could be stronger. An observation records a fact worth carrying forward that is neither breach nor improvement advice — an early-warning condition, or something the sample could not fully resolve, flagged for the next audit. Neither obliges the auditee to act, and neither may be used to park a finding that actually breaches a criterion: softening a real nonconformity into an OFI is the grade-bargaining failure accreditation-body witnesses look for. Note the brightest line in the profession runs here too — an OFI may say what is weak, never how to fix it, because prescribing the remedy makes the auditor a consultant.

closing meeting

The auditee hears the findings, each in its four parts with its evidence, plus the audit conclusion against the objectives, the reminder that the audit sampled rather than exhausted, and what happens next: corrective-action timelines, verification, the certification body’s independent decision, and the appeals channel. If findings were shared as they emerged during the week, this meeting confirms rather than ambushes — the single best predictor of a civil closing. Disputes resolve on one axis only, evidence: if the missing record appears in the meeting, the finding is amended or withdrawn (fair presentation cuts both ways); if the argument is about interpretation, requirement and evidence are restated and the grade holds; if no agreement is reached, ISO 19011’s answer is to record the diverging opinion in the report.

ISO 19011:2018 §6.4.10

audit report

ISO 19011 wants it complete, accurate, concise and clear, carrying the objectives, scope and criteria; the audit client and team; dates and locations; the findings with their evidence; the conclusions against the objectives; unresolved diverging opinions; and the sampling caveat. Craft points that separate professionals: conclusions answer the objectives as stated in the plan, positive evidence is recorded too (what was sampled and found conforming — fair presentation demands it and the decision-maker needs the whole picture), and the report reaches the auditee promptly under the agreed confidentiality terms. It is what the certification decision-maker, the accreditation body’s witness and next year’s audit team will read long after the week of interviews has evaporated.

ISO 19011:2018 §6.5

verification of effectiveness

The last and most-skipped element of the corrective-action anatomy: correction (fix the instance), root cause, systemic action, and then evidence that the systemic action is operating. Verification is not “we uploaded the missing document” — it is the next two retraining cycles’ gate records showing the re-validation step fired. The auditor’s evaluation discipline follows: reject a root cause that ends at a person (“the engineer forgot” answers nothing — why did the system depend on one engineer’s memory?), reject restatements of the finding dressed as analysis, reject pure quick fixes, and reject effectiveness claims with no evidence plan. Accepting a weak plan is quietly signing next year’s repeat finding.

ISO/IEC 42001:2023 Clause 10.2; ISO/IEC 17021-1:2015

certification decision

The constitutional separation at the heart of accredited certification: the auditor recommends, the body decides. ISO/IEC 17021-1 requires the decision to be made by personnel who did not perform the audit, on the basis of the report, the findings and the evaluated corrective actions — which protects the decision from a week of built rapport and protects the auditor from a week of pressure. “I cannot grant or deny your certificate; I gather the evidence” is both true and the best de-escalation line in the profession. A standing major nonconformity blocks a positive decision until it is corrected and its correction verified; surveillance and recertification then re-open the corrective-action file at every return visit.

ISO/IEC 17021-1:2015 §9.5

executive order (US)

A written directive issued by the President under constitutional or statutorily delegated authority, directing how the executive branch shall act. Three questions decode any AI executive order. Who does it bind? Agencies and, through procurement, their contractors — rarely private companies directly. What implements it? The EO delegates; OMB memoranda and agency rules do the operational work. How durable is it? One signature can revoke the whole edifice, as January 2025 demonstrated when EO 14110 (October 2023) was rescinded wholesale and replaced days later by a different philosophy. The governance consequence: a compliance programme anchored to executive policy alone is built on sand, which is why the NIST AI RMF and state statutes carry most private-sector weight.

OMB (Office of Management and Budget) (Office of Management and Budget)

Part of the Executive Office of the President, OMB controls the federal budget and — for AI governance purposes — issues the M-series memoranda that convert an executive order into agency obligations: Chief AI Officers, public AI use-case inventories, and minimum practices for designated categories of AI use. The Biden-era memos M-24-10 and M-24-18 built the “safety-impacting” and “rights-impacting” categories; M-25-21 and M-25-22 (April 2025) replaced them, keeping Chief AI Officers and inventories while collapsing the two categories into a single high-impact tier with an innovation-first tone. OMB requirements bind agencies, not private companies — but they shape the federal market, and their inventory discipline is the clearest public model of the AI inventory every programme needs.

CAISI (Center for AI Standards and Innovation) (Center for AI Standards and Innovation)

The US AI Safety Institute, created at NIST under the 2023 executive order, was renamed the Center for AI Standards and Innovation in June 2025 and its mission tilted from “safety” framing toward security, standards leadership and national competitiveness. The institutional point is worth holding: the capability — model evaluation, frontier testing arrangements, standards work — survived a change of administration while the framing did not. The UK made the same move in February 2025 when its AI Safety Institute became the AI Security Institute. Read both renames as data about the 2025 policy mood rather than as the abolition of the function.

Supremacy Clause (US)

Article VI, clause 2 of the United States Constitution makes the Constitution, federal statutes and treaties the supreme law of the land, binding state judges notwithstanding contrary state law. It is the source of preemption doctrine in its three forms: express (Congress says so in text), conflict (compliance with both is impossible, or state law obstructs federal objectives), and field (federal regulation so pervasive it occupies the field). The AI-specific problem is structural rather than doctrinal: with no comprehensive federal AI statute there is no express text to point to, a voluntary framework creates no conflict — you can comply with the NIST AI RMF and Colorado — and an empty field cannot be occupied. Deregulatory policy is not preemptive law. Strictly a US doctrine; it has no analogue in the EU, where primacy of Union law operates on entirely different foundations.

US Constitution, Art VI, cl. 2

Spending Clause (US)

Article I, section 8, clause 1 of the US Constitution lets Congress spend for the general welfare, and the Supreme Court has long allowed Congress to attach conditions to federal grants as a way of inducing state policy it could not command directly. The conditions are not unlimited: they must be unambiguous, related to the federal programme’s purpose, and non-coercive — in NFIB v. Sebelius (2012) the Court struck down threatening states’ entire existing Medicaid funding as “a gun to the head”. This is the doctrine under stress when federal AI policy proposes conditioning discretionary funds (broadband money, for instance) on states not enforcing their own AI laws: is the condition related to the programme, is the amount at stake coercive, and may an agency invent such a condition at all without Congress writing it into the appropriation? A US-only doctrine, and one whose application to AI funding leverage is genuinely unsettled.

US Constitution, Art I, §8, cl. 1; NFIB v. Sebelius (2012)

dormant Commerce Clause (US)

A judicially inferred limit on state power read out of Article I, section 8, clause 3 of the US Constitution. States may not discriminate against out-of-state commerce, and under the Pike v. Bruce Church (1970) line a non-discriminatory law may still fall if its burden on interstate commerce clearly exceeds its local benefits. The argument against state AI laws: a model trained once and served nationwide cannot vary by state, so one state effectively regulates the nation. The counter-argument has strong recent support — in National Pork Producers Council v. Ross (2023) the Court upheld California’s Proposition 12 against precisely this attack, holding that states may regulate in-state effects even where compliance costs propagate nationally. This is the strongest doctrinal battleground for federal challenges to state AI statutes, and its outcome is genuinely uncertain. A US-only doctrine.

US Constitution, Art I, §8, cl. 3; Pike v. Bruce Church (1970); National Pork Producers Council v. Ross (2023)

EO 14365 (Ensuring a National Policy Framework for AI)

Signed 11 December 2025, after Congress declined to enact a moratorium on state AI enforcement (the Senate struck one from the 2025 reconciliation bill by 99–1 on 2 July 2025). Its directives, closely paraphrased: the Attorney General shall stand up an AI Litigation Task Force to challenge state AI laws on dormant Commerce Clause, preemption and other constitutional grounds; agencies shall assess conditioning discretionary federal funds on states not enforcing conflicting AI laws; state laws on child safety, data-centre infrastructure and state government procurement are carved out; and the administration shall develop a federal legislative framework to present to Congress. The analytical catch is that an executive order is not a statute, so there is still no federal AI law to preempt the field with. Litigation outcomes move — check the docket before relying on which state laws survived.

Executive Order 14365 (11 December 2025)

high-impact AI

Treat this as a false friend across frameworks, not a universal concept. In US federal policy it is the consolidated category introduced by OMB M-25-21 (April 2025), which replaced the Biden-era split between “safety-impacting” and “rights-impacting” AI with one tier of agency AI carrying minimum practices. In South Korea it is the operative risk category of the AI Basic Act (passed 26 December 2024, effective 22 January 2026), attaching duties to AI used in defined consequential domains. Neither is the EU AI Act’s high-risk category, which is defined by Annex I/Annex III enumeration and carries a far heavier obligation set. When a document says “high-impact”, establish the regime before mapping any control to it.

OMB M-25-21 (2025); Korea Framework Act on AI Development and Establishment of Trust

sectoral enforcement

With no comprehensive federal AI statute, US AI enforcement runs through existing jurisdictions: the FTC over markets and consumers, the EEOC over workplaces, the CFPB over consumer finance, the FDA over medical products, the banking agencies over banks, the SEC over securities, and state attorneys general under their consumer-protection statutes. The organising principle, as the FTC put it, is that there is no AI exemption from the laws on the books. Two consequences shape practice. Jurisdiction comes before harm: the same biased model triggers different enforcers, theories and remedies depending on whether it screens applicants, prices loans or triages patients. And because no statute defines “reasonable AI practice”, enforcers reach for benchmarks — which is how the voluntary NIST AI RMF and banking’s SR 11-7 became de facto standards of care. Contrast horizontal regulation, the EU’s choice.

UDAP (unfair or deceptive acts or practices) (Unfair or Deceptive Acts or Practices)

Section 5 of the Federal Trade Commission Act bars “unfair or deceptive acts or practices in or affecting commerce”; nearly every state has its own “little FTC Act” enforced by its attorney general, often with private remedies the federal statute lacks. Deception reaches claims — overstated accuracy, fake AI capabilities, invented review content — and lands on marketing departments faster than any AI act reaches models. Unfairness reaches conduct causing substantial unavoidable injury not outweighed by benefits, the theory behind the FTC’s 2023 order banning Rite Aid from facial-recognition surveillance for five years. Note the related but distinct UDAAP, which adds “abusive” under the Dodd-Frank Act for conduct within the CFPB’s consumer-finance jurisdiction. A US doctrine: do not read it as a general principle of fairness in other regimes.

FTC Act §5 (15 U.S.C. §45); state UDAP statutes; cf. Dodd-Frank §1031 (UDAAP)

model risk management (MRM) (Model Risk Management)

Codified for US banks by SR 11-7 (Federal Reserve/OCC supervisory guidance, 2011), MRM treats model risk as the risk of adverse consequences from incorrect or misused model output, and answers it with five moves: maintain a model inventory; assess conceptual soundness before use; validate independently (validators must not report to developers); monitor for performance decay; and subject all of it to effective challenge by people with the authority, competence and incentives to say no. Swap “model” for “AI system” and you have most of ISO/IEC 42001’s operational clauses and the three lines architecture. The open question supervisors are still working: SR 11-7 assumed models documentable equation by equation — validating a large language model’s “conceptual soundness” is an exercise the guidance never imagined.

Federal Reserve SR 11-7 / OCC Bulletin 2011-12

developer (US state AI laws)

The US state AI wave borrowed a two-role structure: a developer builds or intentionally and substantially modifies the system, a deployer puts it to use, and duties are split between them — developers owe documentation and disclosures to deployers and, in some statutes, notice to the attorney general; deployers owe notice to affected people, impact assessment and human-review duties. Two traps. First, it is not the EU AI Act’s provider: the EU trigger is placing a system on the market or putting it into service under your own name, so an organisation can be a provider without developing anything, while “developer” keys on the building. Second, the same company is routinely both — fine-tune a vendor model and deploy it and you wear both labels, with both duty sets. Definitions differ statute by statute; read the one that applies.

cure period

Many US state AI and privacy statutes give the attorney general exclusive enforcement and oblige them to notify the business and allow a defined period to cure the violation before an action may be filed; some sunset the right to cure after an initial transition, and some pair it with an affirmative defence for organisations following a recognised risk framework. Practical consequences for a compliance programme: enforcement arrives first as a letter, so the response capability that matters is the ability to produce evidence and remediate inside the clock; and a cure right is not an amnesty — it typically does not reach conduct that also violates a separate statute, nor does it apply where a private right of action exists (contrast BIPA-style regimes, which have no cure step at all). Periods and sunsets vary by statute and are amended often; check the current text.

attorney–client privilege

A long-standing common-law privilege (with close analogues in other common-law systems, and narrower or differently structured protections in civil-law jurisdictions, where in-house counsel may not qualify at all) shielding confidential communications between a client and its lawyer made for the purpose of obtaining or giving legal advice. For an AI governance programme the discipline is to route candid legal-risk analyses through counsel, and the limit is just as important: routine compliance artifacts are generally not privileged — the inventory, the model card, the monitoring dashboard — and some are designed for a regulator’s eyes, such as statutory risk assessments and conformity documentation. Mixing the two in one document tends to destroy the privilege rather than extend it. The related US work-product doctrine protects materials prepared in anticipation of litigation and is distinct.

NIST Generative AI Profile

Published July 2024 as NIST AI 600-1, the Generative AI Profile is a companion to the AI RMF, not a separate framework: it keeps GOVERN–MAP–MEASURE–MANAGE and populates them with generative-AI-specific risks and suggested actions. Its twelve risk categories are the field’s most-cited checklist for scoping a genAI evaluation — confabulation, dangerous or violent recommendations, data privacy, environmental impact, harmful bias and homogenisation, human–AI configuration, information integrity, information security, intellectual property, obscene and abusive content, value-chain and component integration, and CBRN information or capability. Note the institutional lesson: commissioned by an executive order that was later revoked, the Profile remains current and widely cited — technical artifacts outlive their political parents. The site also indexes it as the GAI Profile.

NIST AI 600-1, Generative AI Profile (July 2024)

risk tiering

The methodology that makes a governance programme affordable: every inventoried system is placed in a tier, and each tier carries a defined control set — assessment depth, testing requirements, approval level, monitoring cadence, review frequency. Three design rules separate working schemes from decorative ones. Tier on harm to affected people, not to the company, or the scheme becomes a reputational-risk register. Write the criteria down and apply them consistently, because an undocumented tier is an unchallengeable one — and the first thing a regulator asks is why this system is tier three. And make tiers re-openable: a tier describes a system as submitted, so the re-triage triggers must fire on model swaps, new data, new populations, repurposing and expanded autonomy. Distinct from the EU AI Act’s legal risk pyramid, which is a statutory classification you do not get to design.

OECD.AI Policy Observatory

Launched in February 2020 to give the 2019 OECD AI Principles something to stand on: a public platform with repositories of national AI policies and strategies across dozens of jurisdictions, trends and data work, and an expert network feeding the analysis. Its governance significance is quiet but real — soft law cannot fine anyone, but an authoritative, comparable evidence base is what makes national claims checkable and what legislatures and journalists reach for. UNESCO built a deliberate counterpart (its Global AI Ethics and Governance Observatory, 2024) that maps readiness rather than policies; the mild institutional rivalry benefits anyone who needs the data.

OECD.AI Policy Observatory (launched February 2020)

AI Incidents Monitor (AIM) (AI Incidents Monitor)

Part of the OECD.AI machinery: AIM tracks AI incidents and hazards reported in news coverage worldwide and classifies them so they can be counted and compared across jurisdictions and sectors. The reason it matters more than its modest description suggests: soft law has no sanction, but visible, comparable harm is what moves legislatures — and before AIM and its community-run cousins there was no shared denominator for “how often does this actually go wrong?”. Read alongside the AI Incident Database (community-curated) and emerging statutory incident reporting such as the EU AI Act’s serious-incident duties, which will eventually give regulators a non-news-based source.

OECD.AI — AI Incidents Monitor

Global Partnership on AI (GPAI)

Launched in June 2020 as a multistakeholder community doing applied work on responsible AI, with centres of expertise in Montréal and Paris. By 2024 it faced the classic small-institution problem — duplicated effort beside the OECD’s larger apparatus — and the answer was the 2024 integrated partnership: memberships brought together under one roof with all countries participating on an equal footing and the expert community plugged directly into OECD.AI. The institutional lesson for a governance professional: soft-law bodies consolidate toward whoever holds the evidence base. Note the acronym collision that trips people constantly — GPAI here is this partnership, while GPAI in the EU AI Act means general-purpose AI models.

Global Partnership on AI (2020); OECD–GPAI integrated partnership (2024)

definitional interoperability

Not a legal instrument but a property of a regulatory landscape, and the OECD’s quiet superpower. When two regulators in different jurisdictions agree on what an AI system is, everything downstream — control crosswalks, documentation reuse, mutual recognition, one global inventory schema — becomes possible; when they do not, a multinational maintains parallel truths about the same system. The worked example is the OECD definition of an AI system, refined in 2023–24 around inference, which became the anchor text for the EU AI Act, the Council of Europe treaty and statutes well beyond. The practitioner’s habit that follows: when a new regime lands, check its definitions against the OECD text first, because the delta is your real workload.

OECD Recommendation on AI (2019, revised 2024)

Readiness Assessment Methodology (RAM) (Readiness Assessment Methodology)

The implementation tooling UNESCO shipped with its 2021 Recommendation on the Ethics of AI. A government, with UNESCO support, assesses itself across five dimensions — legal and regulatory, social and cultural, scientific and educational, economic, and technical and infrastructural — and the output is a written gap analysis: no AI strategy, no data-protection authority, no algorithm-literate regulators. Dozens of countries across every region have completed or begun one, and for many in the Global South the RAM report is the first AI governance artifact of any kind, preceding and shaping the first national strategy. That is soft law doing what statutes cannot: building capacity where there is nothing yet to enforce. It is voluntary, and there is no sanction for shelving the report.

UNESCO Recommendation on the Ethics of AI (2021) implementation tooling

Ethical Impact Assessment (EIA) (Ethical Impact Assessment)

The second instrument UNESCO shipped with its Recommendation, operating one level below the RAM: a structured assessment for a specific AI system, aimed at teams procuring or deploying AI and especially at the public sector. It asks who is affected, what could go wrong, and what mitigations and redress exist — organised around the Recommendation’s values and principles rather than around a company’s risk register. Read it as the soft-law ancestor of the impact-assessment duties now in hard law: the EU AI Act’s fundamental rights impact assessment and the Council of Europe’s HUDERIA methodology are recognisably the same move, made binding. Non-binding itself, and easily reduced to a form if nobody is accountable for acting on the output.

UNESCO Recommendation on the Ethics of AI (2021) implementation tooling

code of conduct

The G7’s International Code of Conduct for Organizations Developing Advanced AI Systems (October 2023) turns the Hiroshima Process’s eleven Guiding Principles into a checklist a company can adopt: lifecycle risk identification and mitigation, post-deployment misuse monitoring, published transparency reports, information sharing with peers and governments, security investment, content authentication and provenance tooling, societal-risk research, and standards development. Note the audience shift that makes it distinctive — the OECD speaks to governments and UNESCO to states and societies, while this speaks directly to the dozen-odd firms building frontier models. Keep it apart from the EU AI Act vocabulary, where a code of practice (Art 56) is the GPAI compliance on-ramp and codes of conduct (Art 95) are the voluntary route for non-high-risk systems.

G7 Hiroshima Process International Code of Conduct (October 2023)

transparency report (AI)

Since February 2025 the OECD has hosted the Hiroshima AI Process reporting framework, where companies file voluntary, structured reports describing how they implement the Code of Conduct: risk evaluations, red-teaming, provenance tooling, governance structures. Three reasons professionals care. Comparability — every filer answers the same questions, giving researchers and regulators the industry’s first common disclosure channel. Commitment escalation — a public filing is evidence, so a company that files “we red-team all frontier releases” and demonstrably does not has handed enforcers a deception theory. Template effect — when binding law arrives, regulators reach for disclosure formats that already exist. Filing rosters and report contents evolve continuously; check the current OECD portal before citing specific companies. Distinct from platform transparency reports under the DSA and from frontier-developer reports now required by some statutes.

OECD-hosted Hiroshima AI Process reporting framework (from February 2025)

agile governance

A governance philosophy built for the pacing problem: because statutes freeze slowly around a technology that moves quickly, govern through non-binding guidance, multistakeholder goal-setting, continuous revision and ex-post accountability rather than prescriptive ex-ante rules. Japan is its clearest exponent — the 2025 AI Promotion Act is a basic law that promotes AI, creates a prime-minister-chaired AI Strategy Headquarters and imposes a duty to cooperate, with no penalties or fines, backed by the METI/MIC AI Guidelines for Business. Singapore’s toolkit model and Australia’s voluntary standard sit nearby on the same spectrum. The honest critique: agile governance works where reputational and administrative pressure is strong and fails where a determined actor simply declines to cooperate, because there is nothing behind the guidance. Compare hard law and soft law as the general categories.

MSIT (Korea, Ministry of Science and ICT) (Ministry of Science and ICT (Republic of Korea))

The Ministry of Science and ICT administers Korea’s AI Basic Act (the Framework Act on AI Development and Establishment of Trust, passed 26 December 2024, effective 22 January 2026) — the world’s second comprehensive horizontal AI law after the EU AI Act. Two design choices worth knowing. Developers of models above a training-compute threshold set by enforcement decree (not written into the statute) owe lifecycle risk identification, assessment and mitigation, a risk-management system, and reporting of results to MSIT — Seoul watched the EU struggle with a hard-coded 10²⁵ FLOPs number and deliberately kept the figure amendable. And the Act reaches foreign providers, with a domestic-representative requirement rather than the EU’s outputs-used-in-the-Union test.

Korea Framework Act on AI Development and Establishment of Trust (2024)

UN General Assembly resolution

General Assembly resolutions are recommendations: they bind no state and create no enforceable obligation, which is precisely why all 193 members can join them. The AI track produced three that matter. A/RES/78/265 (21 March 2024) was the first GA resolution on AI — US-led, 120-plus co-sponsors, adopted by consensus, framing safe, secure and trustworthy AI for sustainable development. A China-led resolution on AI capacity building followed in July 2024, also by consensus and with the US among the co-sponsors. Then A/RES/79/325 (26 August 2025) created the Independent International Scientific Panel on AI and the Global Dialogue on AI Governance. The craft of the UN track is visible there: costless consensus converted into institutions that create facts and pressure. Judge it by the institutions, not the adjectives.

A/RES/78/265 (2024); A/RES/79/325 (2025)

adoption by consensus

A UN practice: the chair records adoption without a recorded vote when no member state formally objects. It is a genuine diplomatic achievement — China and Russia joining a US-led AI resolution, and the US joining a China-led one — and it is cheap for exactly the reason that makes it possible: nothing in a resolution binds anyone. Two cautions for anyone citing it. Consensus is not unanimity of enthusiasm; states routinely join texts they intend to ignore, and may enter explanations of position that gut their commitment. And it is not guaranteed: the General Assembly appointed the AI Scientific Panel’s slate on 12 February 2026 by a vote, not by consensus, with the United States and Paraguay voting against — a data point about posture toward multilateral AI governance that a consensus record would have concealed.

Pact for the Future

Adopted without a vote at the Summit of the Future (September 2024) after a last-minute Russian amendment to defer the package failed decisively. The Pact is a political commitment, not a treaty — no state could ever be sued over it — and its AI significance is almost entirely carried by its annex, the Global Digital Compact, whose Objective 5 commits states to enhancing international AI governance and specifically to establishing a scientific panel and a global dialogue. The instructive part for a governance professional is the compounding: a non-binding advisory report proposed the institutions, a non-binding pact and compact promised them, and a non-binding resolution created them — three instruments with no legal force producing two real bodies faster than any treaty could have.

UN Pact for the Future (September 2024)

Global Digital Compact (GDC) (Global Digital Compact)

Annexed to the Pact for the Future in September 2024, the GDC is the UN’s framework of commitments on digital cooperation — connectivity, data governance, digital public infrastructure and, in Objective 5, enhancing international governance of AI for the benefit of humanity, with specific commitments to establish an independent international scientific panel and a regular intergovernmental policy dialogue on AI governance. Its legal character is the thing to remember: a compact is a political commitment, not a treaty, so nothing in it is enforceable — and yet it is the reason two institutions now exist. When someone says “the UN agreed rules for AI”, the accurate reply names the instrument type and what it can and cannot do.

Global Digital Compact, annex to the Pact for the Future (2024)

Global Dialogue on AI Governance

Established by resolution A/RES/79/325 (26 August 2025) alongside the Independent International Scientific Panel on AI, the Global Dialogue is where every UN member state — not 38, not 7 — discusses AI governance. The division of labour is the design: the Panel produces the shared evidence base and is explicitly non-prescriptive, while the Dialogue is where states argue about what follows from it. Its launch was set for Geneva in 2026, with a second session in New York in 2027. Treat first-session outcomes as moving parts and check current UN sources before citing them; what is settled is the mandate and the structural separation it borrows from climate governance.

A/RES/79/325 (26 August 2025)

the IPCC model

The institutional design borrowed from climate governance, where the Intergovernmental Panel on Climate Change assesses the evidence and the treaty bodies negotiate what to do about it. The UN applied it to AI in 2025: the Scientific Panel issues an annual report that is policy-relevant but non-prescriptive — it says what the evidence shows, never what governments should do — while the Global Dialogue holds the politics. The limits are also borrowed: no budget, no enforcement, and a scope that for AI excludes military applications, which stay on the separate Geneva track. Its only weapon is authoritative shared facts, which is precisely what the fragmented AI debate has lacked. The structure is also what makes the Panel attackable from both sides — too timid for advocates, too influential for sceptics.

Council of Europe

Founded in 1949 and distinct from every similarly named body: it is not the European Union, not the European Council (EU heads of state) and not the Council of the European Union (EU ministers). Its membership is larger than the EU’s — 46 states including the United Kingdom and Türkiye — and its signature achievement is the European Convention on Human Rights with the European Court of Human Rights behind it. For AI governance it matters because its Committee on Artificial Intelligence negotiated the Framework Convention on AI, Human Rights, Democracy and the Rule of Law (CETS No. 225, adopted 17 May 2024, opened for signature in Vilnius on 5 September 2024) with non-member observers including the United States, Canada, Japan and Israel at the table, and because it shipped the HUDERIA impact-assessment methodology alongside it.

Council of Europe; CETS No. 225 (2024)

national security exemption

The Council of Europe Framework Convention exempts activities related to the protection of national security interests, subject to a requirement that they still respect international law and democratic institutions and processes. Critics make the obvious point: many of the most rights-threatening AI deployments — mass surveillance, intelligence profiling — live exactly there. The pattern repeats across the field: the EU AI Act excludes military, defence and national-security purposes, and most national regimes carry some version of the carve-out, which is why “AI is regulated now” is never true of the uses that worry civil-liberties groups most. Read every exemption for three things: what activities it covers, who decides whether an activity qualifies, and whether any oversight survives inside it.

CoE Framework Convention on AI (CETS No. 225); EU AI Act Art 2 exclusions

signature (treaty law)

Under the Vienna Convention on the Law of Treaties, signing a treaty that requires subsequent ratification obliges the signatory to refrain from acts that would defeat the treaty’s object and purpose pending ratification — and nothing more. This is the single most common error in coverage of the AI treaty: “the US signed the AI treaty, so it is bound by it” is wrong twice, because signature is not ratification and ratification does not bind until entry into force. Signature still matters politically — it signals direction, locks in negotiated text, and in the Framework Convention’s case produced the historically remarkable fact of Washington signing a human-rights treaty on AI. Treat it as a statement of intent with one narrow legal consequence.

Vienna Convention on the Law of Treaties (1969), Art 18

ratification

After signature comes each state’s own constitutional approval process — in the United States, Senate advice and consent; in parliamentary systems, an act or decree — followed by depositing an instrument of ratification (or accession, acceptance or approval) with the treaty’s depositary, which for Council of Europe treaties is in Strasbourg. Only then has the state consented to be bound, and even then the obligations do not operate until the treaty’s own entry-into-force threshold is crossed. Two practical habits: ratifications arrive without fanfare and secondary sources lag badly, so check the depositary’s live table rather than a news article; and never assume a signatory list is a party list — for the Framework Convention the two are very different.

Vienna Convention on the Law of Treaties (1969), Arts 14–16

entry into force

Every treaty sets its own trigger. For the Framework Convention on AI, Article 30 requires five ratifications, of which at least three must be Council of Europe member states, after which the Convention takes effect on the first day of the month following a three-month waiting period. Until that threshold is crossed, a long list of signatories produces no binding obligations on anyone. Because ratifications are quiet and secondary sources lag, the professional discipline is to check the Council of Europe Treaty Office’s live signature-and-ratification table for CETS No. 225 before stating the Convention’s status — getting this wrong in a brief is the treaty-law equivalent of citing a repealed statute. Note also that a treaty can be in force generally while not yet in force for a particular state that ratified later.

CoE Framework Convention on AI (CETS No. 225), Art 30; Vienna Convention on the Law of Treaties, Art 24

CAC (Cyberspace Administration of China) (Cyberspace Administration of China)

The Cyberspace Administration of China is the central internet regulator, and AI governance in China runs through it: it leads the drafting of each technology-specific regulation, co-issuing with ministries such as MIIT (industry), the Ministry of Public Security, and for particular rules SAMR (market regulation) or NRTA (broadcasting); it operates the algorithm filing system and the security assessment for public-facing generative AI; and it enforces. Understanding its lineage explains the content of the rules: Chinese AI regulation grew out of information-control law — the Cybersecurity Law tradition — rather than out of product safety or fundamental rights, which is why nearly every rule polices what AI-generated content may say.

MIIT (Ministry of Industry and Information Technology) (Ministry of Industry and Information Technology)

The Ministry of Industry and Information Technology is the industrial regulator for telecoms, electronics and information technology, and appears as a co-issuing authority on China’s AI regulations alongside the CAC. Its role is the counterweight to the content regulator: where the CAC polices information, MIIT carries the industrial-policy and standards mandate descending from the 2017 New Generation AI Development Plan — sector development, standards programmes, and the practical question of whether a rule strangles the industry it governs. Reading a Chinese AI rule by its co-signatories is a fast way to infer which motive dominates: public-security ministries point to control, SAMR to consumer and market protection, MIIT to industrial development.

vertical regulation

Rather than one comprehensive statute, a stack of narrow binding rules issued as each technology emerges: recommendation algorithms (2022), deep synthesis (2023), generative AI services (2023), synthetic-content labelling (2025). Scholars call it vertical and iterative — narrow scope, quick issuance, frequent supplements — and it made China the first jurisdiction to enforce dedicated rules on each of those technologies, years before most others had a single applicable rule. The trade is explicit: legal coherence and cross-cutting definitions are sacrificed for speed and revisability. Contrast horizontal regulation (the EU AI Act, Korea’s AI Basic Act) and note that the two are not mutually exclusive — the EU also legislates vertically in sectoral product law, and China has repeatedly deferred a comprehensive AI law in favour of iteration.

core socialist values

A legally operative phrase, not rhetoric: China’s AI regulations require services to uphold core socialist values and prohibit generating content that endangers national security, undermines national unity, or harms social stability — and compliance is tested, through the pre-launch security assessment and the filing process, before a public-facing generative service may operate. For a governance professional two points follow. First, it is the clearest illustration that Chinese AI law descends from information-control law rather than product-safety or rights law. Second, it is not severable from the rest of the rulebook: the same instruments that carry this requirement also carry genuine consumer and worker protections, so analysing the regulations as only censorship misreads them as badly as ignoring the censorship would.

algorithm filing (China)

The mechanism that threads China’s otherwise separate AI rules together. Introduced with the 2022 provisions on algorithmic recommendation and extended to deep synthesis and generative AI, it requires providers of services with the specified characteristics to file information about the algorithm — purpose, mechanism, data, self-assessment — with the CAC, which maintains the register and publishes filed-service lists. Paired with the pre-launch security assessment for public-facing generative services, it gives the regulator an inventory no Western regime currently has. Compare the EU’s EU database registration for high-risk systems and the UK’s ATRS for public bodies: all three are registration regimes, but they differ sharply in who must file, what is published, and whether filing gates deployment. The site also indexes the general concept as an algorithm registry.

New Generation AI Development Plan (2017)

Issued by the State Council in 2017, the plan set staged national objectives culminating in global AI leadership by 2030, with accompanying commitments on research, talent, standards and industrial build-out. Its governance relevance is that it is the layer underneath the regulations: China’s AI rules are calibrated to discipline the sector without strangling it, which is why the generative-AI measures were softened between draft and final text and why enterprise-internal AI is largely carved out. Read the rulebook as layers — the 2017 plan at the bottom, the 2021 data statutes (PIPL and the Data Security Law) as substrate, then the technology-specific regulations stacked on top and threaded together by algorithm filing.

PRC State Council, New Generation AI Development Plan (2017)

PIPL (Personal Information Protection Law) (Personal Information Protection Law)

Effective 1 November 2021 and, with the Data Security Law, the foundation layer beneath China’s AI rules: consent and other processing bases, purpose and minimisation requirements, heightened rules for sensitive personal information, individual rights, cross-border transfer controls with security-assessment and certification routes, and duties around automated decision-making including transparency and an option to refuse personalised decisions. Functionally it plays the role the GDPR plays in Europe — the rule that bites first, regardless of whether anything AI-specific applies — but it sits inside a different constitutional setting, with state access provisions and a national-security frame the GDPR does not have. Pair it with the Data Security Law (data classification and security duties) rather than reading either alone.

PRC Personal Information Protection Law (effective 1 November 2021)

TC260

TC260 writes China’s information-security standards and technical documents, including the 2024 Basic Security Requirements for Generative AI Services. On paper such a document is voluntary guidance; in practice it defines the test the regulator applies — corpus screening thresholds, keyword and classifier requirements, security-assessment content — so it is the operative compliance specification for a public-facing generative service in China. The general lesson travels beyond China: in any regime where a regulator is also the audience for a standards body’s work, the “voluntary” standard is the examinable rulebook. Compare the EU’s harmonised standards, which are voluntary but confer a presumption of conformity, and the NIST AI RMF, voluntary but used as a yardstick of reasonableness by US enforcers.

TC260, Basic Security Requirements for Generative AI Services (2024)

Beijing effect

Coined by analogy with the Brussels effect, which describes how the size of the EU market exports EU rules worldwide. The Beijing-effect claim is that China exports influence by a different route: a governance narrative aimed at states that see AI primarily as a development question, plus affordable infrastructure and widely used open-weight models that make Chinese technical and regulatory patterns the path of least resistance in many markets. Treat it as a live analytical debate rather than an established fact — the mechanisms are genuinely different from the Brussels effect (market access versus narrative and supply), and the evidence of rule adoption is thinner. Its policy vehicle is the Global AI Governance Initiative.

Global AI Governance Initiative (China)

Announced at the Belt and Road Forum in October 2023, the Initiative sets out a deliberate alternative to the Western rights-based framing: AI governance organised around state sovereignty, the development rights of the Global South, and opposition to technological hegemony. It was followed by a Global AI Governance Action Plan at the World AI Conference in Shanghai (July 2025) and a proposal for a World AI Cooperation Organization headquartered in Shanghai, and it is of a piece with the China-led July 2024 UN General Assembly resolution on AI capacity building. The audience is the majority of countries for whom the top AI-governance issue is access, capacity and who captures the value — not frontier-model risk.

PRC Global AI Governance Initiative (October 2023)

extraterritoriality

Regimes reach beyond their borders by different mechanisms, and the mechanism determines your exposure. The EU AI Act reaches providers and deployers established outside the Union where the system’s output is used in the Union, which is explicit textual extraterritoriality; the GDPR reaches non-EU controllers offering goods or services to, or monitoring, people in the Union; Korea’s AI Basic Act reaches foreign providers through a domestic-representative requirement; China’s AI rules mostly reach foreign firms through the market rather than by extraterritorial text — they govern services offered to the public in mainland China, so a provider with no Chinese service faces nothing while a market entrant inherits the full gauntlet. In US constitutional argument the word carries a different load: the theory that a state law effectively regulates the nation, which is the dormant Commerce Clause claim against state AI statutes.

EU AI Act Art 2; GDPR Art 3; Korea AI Basic Act

Algorithmic Transparency Recording Standard (ATRS) (Algorithmic Transparency Recording Standard)

A reporting format and publication duty for UK central-government organisations: each algorithmic tool used in decision-making is described in a structured record — purpose, how it works, the data behind it, human oversight, risks and mitigations — and published. Its significance is less the content than the position it occupies: the UK deliberately passed no AI statute, governing instead through existing sector regulators applying five cross-sectoral principles, which makes a transparency standard for its own civil service the most binding AI-specific rule in the stack. The limits are plain: it covers the public sector, not companies, and a published record is not an assessment. Compare China’s algorithm filing and the EU’s EU database registration, both of which gate or condition deployment rather than only documenting it.

UK Algorithmic Transparency Recording Standard

LGPD (Lei Geral de Proteção de Dados) (Lei Geral de Proteção de Dados Pessoais)

Brazil’s GDPR analogue (Law No. 13.709/2018), supervised by the ANPD, the national data-protection authority. Its AI relevance is twofold. Operationally it is the rule that already applies: in 2024 the ANPD ordered Meta to stop training AI on Brazilian users’ personal data until safeguards were demonstrated — the same pattern seen worldwide, where the data-protection regulator reaches AI years before an AI regulator exists. Institutionally, that record is why the pending AI bill PL 2338/2023 puts the ANPD at the centre of its proposed SIA enforcement network, coordinating sectoral regulators rather than creating a standalone AI authority. PL 2338 passed the Senate in December 2024 and remains with the Chamber of Deputies — check its current status before relying on any provision.

Brazil, Law No. 13.709/2018 (LGPD)

AU Continental AI Strategy

Endorsed by the African Union Executive Council in July 2024 as a five-year framework for all 55 member states, built around harnessing AI for development under Agenda 2063, building African compute, data and skills capacity, and asserting data sovereignty — the principle that African data should generate African value rather than being exported raw and re-imported as someone else’s product. It is soft law: the AU cannot bind member states the way Brussels binds EU members. What continental soft law does do is set shared vocabulary, legitimise national action, and coordinate negotiating positions in UN fora — visible in the force African states brought to the capacity-building agenda and the Global Digital Compact talks. Beneath it sits Smart Africa and a growing set of national strategies.

African Union Continental AI Strategy (endorsed July 2024)

Smart Africa

An alliance of around forty African states, headquartered in Kigali, that produced the AI for Africa Blueprint (2021) under Rwanda’s leadership. Where the AU Continental AI Strategy sets continental direction, Smart Africa supplies the implementation-level material that national strategies have reused — which is why several African national AI policies read as siblings. Worth knowing for the same reason model-legislation networks matter in the United States: when you meet a national strategy, ask which network drafted the template, because the answer predicts its structure and often its gaps.

Smart Africa Alliance; AI for Africa Blueprint (2021)

data sovereignty

A political and legal claim, not a single rule, and it has two separable strands that are routinely conflated. Jurisdictional: data about our people is subject to our law, which cashes out in cross-border transfer controls and government-access concerns — the strand behind the EU’s transfer regime and the Schrems litigation. Economic: data generated here should generate value here, the strand the AU Continental AI Strategy foregrounds when it objects to data being exported raw and re-imported as someone else’s product. Keep it distinct from data localisation, which is one possible implementing measure (store it inside the border) and not the same as sovereignty — localisation can coexist with foreign control, and sovereignty claims can be met by rules about use rather than location.

SDAIA (Saudi Data and AI Authority)

Established in 2019, SDAIA concentrates functions most jurisdictions deliberately separate: it is the national data-protection regulator, the national AI strategist, and a builder of flagship AI projects. It has issued AI Ethics Principles (adoption expected of government entities, encouraged for the private sector) and generative-AI guidelines for officials and the public. For a governance professional the documents matter less than the structure: promoter and regulator are the same body, which is the precise conflict critics attacked in Canada’s ISED-led AI bill — here adopted as a feature. Read any SDAIA instrument with that in mind, and note that the Gulf model generally is state-led AI industrial strategy with governance attached rather than binding horizontal AI law.

Saudi Data and AI Authority (established 2019)

data labour

Reinforcement learning from human feedback needs humans; safety classifiers need people to read the worst text on the internet and label it; vision systems need millions of hand-annotated frames. That work is data labour, and it is concentrated in Nairobi, Manila, Caracas, Gurugram and similar markets, often through layered outsourcing chains that keep it off the AI industry’s books. Why it is an AI-governance problem rather than ordinary outsourcing economics: the harms are produced by the AI production process itself — exposure to traumatic content exists so that safety filters can exist, and piece rates are set by the training pipeline’s economics. The documented pattern (Fairwork, ILO digital-platform studies, the Kenyan moderator litigation) is opaque task pricing, no occupational-health provision for psychologically hazardous work, abrupt platform exits, and disputes routed to foreign law. A discipline that audits training data for bias but not for the conditions under which it was labelled has drawn its system boundary to exclude the humans inside the system.

data colonialism

An analytical frame, not a legal category: scholars Nick Couldry and Ulises Mejias argue that the appropriation of human life as data reproduces the structure of historical colonialism — value extracted as raw material (data, labour, minerals), refined into products elsewhere, and sold back to the places it came from. You do not have to adopt the term to use the governance point it sharpens, which is the same one the AU strategy makes with its data-sovereignty language: the AI value chain has a human layer, and almost no AI law reaches it. The partial legal hooks that do exist today are host-country labour and constitutional law, supply-chain due-diligence regimes such as the CSDDD, procurement leverage, and platform-work regulation.

Couldry & Mejias, “The Costs of Connection” (2019)

AI governance operating model

Principles do not review use cases, challenge model owners, or notice that a system drifted; people in defined roles, following defined procedures, with defined authority do. The operating model is that design: who decides, who challenges, who assures, what fires when, and with what authority to stop a launch. Its canonical build order is mandate → inventory → policy → risk tiering → controls → assurance, because each step produces the input the next one needs — and the most common sequencing failure is writing the policy first, which governs a fictional estate until the forty vendor systems with embedded AI surface. Distinguish it from the governance programme (the funded, running instance) and from governance versus management (who sets direction versus who executes).

AI governance programme

What exists when the operating model has been built and resourced: an executive mandate with budget and decision rights, a maintained inventory, a policy that governs the real estate, a tiering methodology, controls proportionate to each tier, and independent assurance that the controls operate. The distinction from a policy matters at audit and in litigation: regulators and insurers increasingly price the programme — inventory completeness, testing evidence, incident-response maturity, vendor-contract quality — not the PDF. The diagnostic question for whether a programme exists at all: if your most safety-conscious engineer resigned tomorrow, would any AI review still happen?

maturity model

Borrowed from decades of software and risk-management practice, the five-level arc: ad hoc (governance happens when a motivated individual notices), repeatable (a process exists somewhere; coverage is partial), defined (written, resourced, mandatory enterprise-wide — the level most regulatory frameworks implicitly assume), managed (coverage and effectiveness quantified, deviations carry consequences), optimised (post-incident reviews and metric trends change the controls themselves). Knowing your level prevents two opposite mistakes: demanding optimised-level metrics from an ad-hoc programme, which guarantees they will be faked, and letting a mature organisation coast on repeatable-level paperwork. The level-5 self-test is the hardest and most diagnostic: name one control you changed in the last year because your own data showed it was not working.

governance theatre

Every control in a programme can be performed instead of operated: a review board that meets and approves everything, training everyone clicks through, an attestation cycle nobody reconciles, a bias test run once on a model version three retrainings ago. The reason it is worth a named concept is that the structure looks identical from the outside — the Dutch childcare-benefits scandal happened inside an administration with risk frameworks, and Robodebt rolled on for years while frontline staff raised alarms that never travelled upward. The counter-measures are measurement rather than exhortation: disagreement rates on human review, rejection rates at intake, cycle-time and coverage metrics, and the triangulation habit of comparing what the procedure says, what people believe, and what the records show.

independent validation

From banking model risk management: validators assess conceptual soundness, data, testing, limitations and ongoing performance, and crucially do not report to the model developers. Independence is the whole control — a validation performed by the builders is a self-assessment with a different cover page. Two boundary violations account for most failures in AI programmes. The second line doing first-line work: the governance team, frustrated by slow business units, writes their model cards and runs their bias tests “just this once”, and now nobody independent challenges that work. And the third line doing design work: internal audit, being the only team with capacity, helps build the intake process and can never again audit it objectively. The rule that prevents both: you cannot assure what you authored.

Federal Reserve SR 11-7 / OCC Bulletin 2011-12

AI governance committee

Where escalated and borderline cases land: representatives from risk, legal, privacy, security, data, the business and often HR, with a charter, quorum rules, a decision log and — the part that distinguishes a committee from a forum — authority to block a launch. Committees fail in boring, predictable ways: no quorum rules so attendance decides outcomes, no decision log so precedent evaporates, members without delegated authority so every decision needs re-taking elsewhere, and an approval rate near 100%, which means the committee is a ceremony. Keep it distinct from the function that operates governance day to day (an AI Center of Excellence or governance team): the committee decides, the function runs intake, maintains the registry and prepares the cases.

Chief AI Officer (CAIO) (Chief AI Officer)

The role makes sense where AI is simultaneously a strategic bet and a material risk, because someone must own the tension between the two and be answerable when it resolves badly. US federal agencies were required to designate Chief AI Officers under OMB guidance, and the requirement survived the 2025 change of administration — which makes the public sector the clearest template. In companies the design question is reporting line: reporting to the CEO or COO signals enterprise strategy and keeps any veto independent of the people being vetoed, while burying the role under a revenue owner creates the conflict the veto exists to resolve. Wherever it lands, map the borders explicitly against the CDO, the CISO, the privacy officer and the General Counsel — unmapped borders are where orphaned duties come from.

OMB M-24-10 and successor guidance (US federal agencies)

Chief Data Officer (CDO) (Chief Data Officer)

Accountable for data strategy, quality, architecture and stewardship: the inputs every model depends on, including provenance, lineage and the catalogue that makes either answerable. In an AI operating model the CDO is a neighbour, not a substitute, and the border needs stating in writing: the CDO owns the data the models consume, the CISO owns their security, the privacy officer owns the personal-data obligations, the General Counsel owns legal exposure, and the CAIO or governance function owns the AI system risk that spans all of them. Where no border is drawn, monitoring and documentation duties fall between desks — the most common finding in AI programme audits.

Robodebt (Australia)

Australia’s automated compliance programme calculated welfare debts by averaging annual income across fortnightly reporting periods — a method that was simply not a lawful basis for raising a debt — and shifted the burden of disproving the result onto recipients. Hundreds of thousands of false debts followed; courts found the method unlawful, a Royal Commission reported in 2023, and a class settlement approached A$1.8 billion. The missing controls are the instructive part: legal-basis verification before automation — nobody with authority confirmed the core calculation was lawful — and a working escalation channel for the frontline staff who knew early and were ignored. Read it beside the Dutch toeslagenaffaire: both occurred inside administrations that had risk frameworks, which is why they are the canonical demonstrations of governance theatre rather than of missing structure.

Royal Commission into the Robodebt Scheme (2023)

embedded AI

Lead scoring appears in the CRM, transcript summarisation in the meeting tool, “smart” screening in the HR suite — sometimes enabled by default, in a quarterly release, under a contract signed years before anyone had an AI policy. Your exposure changed and no process fired. It is the hardest third-party AI to govern precisely because nobody bought anything, so none of the controls that hang off procurement engage. The responses are contractual and operational: require notification of new AI features in master agreements with opt-out rights, and run periodic SaaS AI-feature sweeps that feed the inventory. A registry containing only consciously procured systems is a map of the visible minority — a first serious inventory typically finds sanctioned AI to be 20–40% of the estate.

BYO-AI (bring your own AI) (Bring Your Own AI)

Drafting contracts in a free chatbot, debugging proprietary code, pasting customer data into a summariser. The canonical incident is Samsung in 2023: within weeks of allowing ChatGPT at work, engineers pasted proprietary semiconductor source code and internal meeting notes into a service whose terms then permitted using inputs for training, and the company banned generative AI on corporate devices shortly after. The lesson is not “ban chatbots” — usage precedes policy, so by the time the rule is written the data has already left, and prohibition without a sanctioned alternative removes your visibility rather than the behaviour. Discovery comes first (network and CASB telemetry is how BYO-AI surfaces), then a usable sanctioned path. A sibling of shadow AI and of embedded AI, distinguished by who introduced it.

attestation

Structured questionnaires to business-unit leaders asking what AI they build, buy or use, converted into a periodic signed declaration. The signature is the whole mechanism: a leader who attested “no AI” while running an unregistered screening tool now owns that answer, which changes behaviour in a way an unsigned survey does not. Attestation cycles also keep a registry honest — track the percentage of records attested in-cycle and the percentage with a reachable owner as programme metrics, and bounce unanswered records rather than ageing them quietly. The known weakness: people cannot report what they do not recognise as AI, so attestation always undercounts and must be reconciled against telemetry, procurement records and platform scans. Each mismatch between two sources is a discovery.

intake

Where the inventory captures what exists, intake captures what is about to. The form is a triage instrument, not an interrogation: purpose and affected population, decision consequence, data categories, build/buy/embedded, autonomy level, jurisdictions, underlying model — roughly fifteen questions answerable in twenty minutes, because every additional page of speculative detail raises the cost of honesty. Design carries most of the credibility: published and kept SLAs (cycle time by tier is your shadow-AI leading indicator), a small set of gating questions that can stop or reroute immediately, and conditional approvals with owners and dates. The governing insight: every hour of friction on the legitimate path is a subsidy to the illegitimate one. The site also indexes this as use-case intake.

gating question

Three or four questions on the intake form carry disproportionate weight because a yes changes the path rather than the score: does it involve a practice on our prohibited list (or the EU AI Act’s Article 5 list)? Does it make or materially influence a consequential decision about individuals? Does it process special-category data? Is it customer-facing generative AI? A hit on the prohibition gate stops; the others skip the queue into elevated review. Two design rules: keep the gates few, because a form where everything is a gate routes everything to the same place, and write them so a requester with no governance training can answer them honestly — a gate phrased in statutory language will be answered wrongly in good faith.

conditional approval

The outcome that keeps proportionality honest: approve the pilot with a human reviewing every output, approve production contingent on the fairness retest, time-box the approval to twelve months, approve for one jurisdiction only. It lets a programme say something other than a blocking no or an unconditional yes, which is what prevents intake from becoming a bottleneck that pushes work into the shadows. The failure mode is specific and common: every condition needs an owner, a due date and an automatic consequence on breach, or conditions become the polite spelling of “yes”. Track open conditions as a programme metric and treat an overdue condition on a high-tier system as an issue with a name attached, not a housekeeping item.

proportionality (governance design)

A principle with two faces. Protective: effort follows risk, so the credit-scoring model in production gets deep assessment and the internal lunch-menu chatbot gets a form — the same logic as risk-based auditing and the EU AI Act’s tiered structure. Practical: friction is not free. When a marketing team can wait three weeks for chatbot approval or just use one today on a personal account, the intake queue is not protecting the organisation — it is selecting which risks go unrecorded. Hence the operating rule: proportionality is not a courtesy to the business, it is how you keep the risk visible, and the fastest way to shrink shadow AI is to make the sanctioned path the easiest path. Note that in EU and ECHR law proportionality is a distinct legal test about whether a measure is necessary and least-restrictive — related in spirit, not the same instrument.

decommissioning

Retirement is rarely one switch. A working checklist: dependent systems notified and migrated; endpoints actually switched off (a “retired” model still serving traffic is the worst of both worlds — live risk, zero monitoring); user communications sent; and a defined data disposition for the training data, the outputs and the logs. Then archive before you delete, because retention obligations outlive the system: EU AI Act documentation duties run for ten years after a high-risk system is placed on the market, and litigation holds plus sectoral record rules add their own clocks. Archive the model version, documentation, validation results and decision logs as of retirement, so that when a claim arrives years later about a decision the dead system made, its record is the defence. Zillow Offers is the cautionary scale: shutting the business down meant months of unwinding positions the model had committed it to.

EU AI Act Arts 18–19 (documentation and log retention)

orphaned model

The most common registry pathology is not missing systems but missing owners. An orphaned model keeps making decisions with no one accountable for its monitoring, its retraining, its incident response or its retirement — and ownership churn, not deliberate neglect, is what produces it. Three controls work: ownership transfer as a mandatory offboarding and reorganisation step; attestation cycles that bounce on unanswered records rather than ageing them; and a standing rule that an unowned high-tier system is an incident, not a housekeeping item. Track it as a registry-hygiene metric alongside the percentage of records attested in-cycle and the count of “retired” entries with live endpoints.

model validation

One of the five instruments a single high-stakes system can trigger, and the one most often conflated with the others. Where a risk assessment protects the organisation and an impact assessment protects other people, validation protects the decision itself: conceptual soundness, data appropriateness, performance on the deployment population, stability, limitations, and whether the result still holds for the version currently in production. Its lineage is banking model risk management, which is also where its defining requirement comes from — independence from the developers. Two habits that make it real: re-validate on retraining (a validation of version 3 says nothing about version 6), and record the limitations as operating constraints rather than as caveats nobody reads.

Federal Reserve SR 11-7 / OCC Bulletin 2011-12

data lineage

The complement to provenance, which is the dataset’s origin story (source, collection method, legal basis, licence, consent scope, date). Lineage is the recipe: what was joined to what, which rows were filtered out, which labels were corrected, how the train/test split was drawn. Provenance without lineage tells you the ingredients but not what was cooked; lineage without provenance documents a pipeline processing data of unknown right. High-stakes AI needs both, captured as metadata that travels with the data rather than as tribal knowledge in a departed engineer’s head. The payoff is answering the questions that arrive later under pressure: is this person in the training set, which models inherited this dataset, what changed between the version we validated and the version that is serving.

chain of custody (data)

In forensic practice, chain of custody is what makes evidence admissible: a continuous record of possession and handling with no unexplained gaps. Applied to AI data governance it means being able to show, for any dataset in a model’s history, where it came from, who transferred it, under which licence or contract, with what consent scope, and who touched it at each step — including the broker three hops upstream. It is a stronger claim than lineage (which records transformations) or provenance (which records origin), because it is about custody and authority to transfer, and it is what collapses first in acquisitions, dataset resales and vendor chains. Capture it at ingestion; retrofitting it is archaeology, and the remediation for failure can be deleting models.

web scraping

The regulators’ refrain is the thing to memorise: publicly accessible does not mean freely usable. Personal data on the open web is still personal data, and biometric data still carries special-category protection — which is why Clearview AI, having scraped billions of face images, collected GDPR fines across several European states, a Dutch fine, and a US settlement under Illinois’ BIPA restricting its sales, and why the EU AI Act went on to prohibit untargeted scraping of facial images to build recognition databases. The scraping-friendly precedent people cite is narrower than its reputation: in hiQ v. LinkedIn the Ninth Circuit held that scraping public profiles likely does not violate the US anti-hacking statute (CFAA), but hiQ lost on contract grounds. In the EU, text and data mining runs through the DSM Directive’s two TDM exceptions, with rightsholders able to reserve their rights in machine-readable form.

Directive (EU) 2019/790 Arts 3–4; EU AI Act Art 5(1)(e); hiQ Labs v. LinkedIn (9th Cir.)

datasheet (for datasets)

Proposed by Gebru et al. (2018) on the analogy of an electronics component datasheet: before anyone trains on a dataset, its creator answers seven sections of questions — motivation (why was this created, funded by whom), composition (what is in it, whose data, is it a sample and of what), collection process (how, by whom, with what consent), preprocessing and labelling, uses (intended, performed, inadvisable), distribution (licence, restrictions) and maintenance (who curates it, will it be updated, how is erasure handled). The intellectual bet is that most model failures are data failures. It is the foundation layer of the documentation stack beneath the model card and the system card, and the layer teams most often skip — after which the EU AI Act’s Annex IV technical file has to be reconstructed from memory under deadline. Cousins: data statements for NLP corpora, Data Nutrition Labels.

Gebru et al., “Datasheets for Datasets” (2018)

data minimisation

GDPR Article 5(1)(c), and the principle that collides hardest with machine learning: the discipline demands the minimum necessary for a specified purpose, while model development wants everything, forever, in case it turns out to be useful. “We might need it later” is not a purpose, and the principle applies to training pipelines as much as to any other processing — alongside purpose limitation, which constrains reuse of data collected for something else. Practical consequences worth knowing: supplying identifiers at inference through retrieval instead of training them into weights is a minimisation measure, and masking inputs before they reach a model provider is a genuine one too, because the provider never receives the raw identifier. British and EU-English spelling; the site also carries the entry as data minimization.

GDPR Art 5(1)(c)

representation bias

One of the named entry points in the bias taxonomy (aligned with NIST SP 1270’s systemic/statistical/human framing). The canonical measurement is Gender Shades, which found commercial face-analysis error rates up to 34.7% for darker-skinned women against 0.8% for lighter-skinned men. Distinguish it carefully from its neighbours, because each implies a different test and a different fix: historical bias means the world the data describes was already unfair, measurement bias means the target variable is a flawed proxy, and representation bias means the sample does not cover the population. Its tests are disaggregated performance testing and coverage analysis; its fix is data collection, not reweighting a label.

NIST SP 1270; Buolamwini & Gebru, Gender Shades (2018)

measurement and label bias

The subtlest and most dangerous entry point in the bias taxonomy, because nothing in the model’s metrics looks wrong. The definitive demonstration is Obermeyer et al. (2019): a widely deployed US healthcare algorithm ranked patients for extra-care programmes by predicted cost as a proxy for health need, and because less money was historically spent on Black patients at every level of illness, equally sick Black patients received lower scores — at the same score they carried significantly more chronic conditions, and fixing the label would have raised the share of Black patients flagged from 17.7% to 46.5%. The model was well-calibrated on its label; the label was the bias. The implied control is proxy-validity review before training — asking whether the target variable is the thing you care about — not a fairness metric computed afterwards.

Obermeyer et al., Science (2019); NIST SP 1270

confusion matrix

For a binary decision against a binary truth: true positives, false positives, false negatives, true negatives. Three rates derived from it carry the whole group-fairness debate — TPR (true-positive rate or sensitivity: of those who truly are positive, what share did the model catch), FPR (of those who truly are negative, what share were wrongly flagged), and PPV (positive predictive value or precision: of those flagged, what share truly are positive). Every group-fairness metric is a demand that one or more of these be equal across groups: equalized odds on TPR and FPR, equal opportunity on TPR alone, predictive parity on PPV. Compute it separately for each group or you have measured nothing about fairness — and remember the impossibility theorem lives in the arithmetic linking these three.

equal opportunity (fairness metric)

Formalised by Hardt, Price and Srebro (2016) as a relaxation of equalized odds, which demands equality of both true- and false-positive rates. Equal opportunity constrains only the TPR — among those who truly are positive, each group is caught at the same rate — which makes it the natural choice where missing a qualified person is the harm that matters most: a qualified applicant screened out, a treatable patient not flagged. Two cautions. It says nothing about false positives, so a model can satisfy it while flagging one group far more often in error. And it is a group metric that can be satisfied while individuals are treated inconsistently. Choosing it is a governance decision about which error, falling on whom, your context makes least acceptable — and that decision must be pre-registered and documented, not discovered after the results.

Hardt, Price & Srebro, “Equality of Opportunity in Supervised Learning” (2016)

predictive parity

The metric Northpointe invoked in the COMPAS dispute: at each risk score, the probability of the outcome was about the same for Black and white defendants — PPV equality, closely related to calibration. ProPublica’s analysis measured something different and also true: Black defendants who did not reoffend were flagged high-risk at roughly 45% against 23% for white defendants, an equalized-odds violation. Both computations were correct on the same data, and Chouldechova (2017) and Kleinberg, Mullainathan and Raghavan (2016) proved this is not an artefact: where two groups have different base rates, no imperfect classifier can satisfy calibration and equal false-positive and false-negative rates at once. PPV mechanically ties error rates to the base rate, so holding one equal forces the other apart. The only escapes are a perfect predictor or identical base rates.

Chouldechova (2017); Kleinberg, Mullainathan & Raghavan (2016)

BISG (Bayesian Improved Surname Geocoding) (Bayesian Improved Surname Geocoding)

The standard answer to the measurement paradox: fairness testing needs protected-attribute data, which privacy law and company policy often prevent collecting. BISG combines surname frequencies with geographic composition to produce probabilistic estimates, and US financial regulators — including the CFPB in fair-lending work — have used it for portfolio-level analysis. Know its limits precisely, because they drive the governance rule: it performs acceptably in aggregate and poorly for individuals, and its error is itself unevenly distributed across groups. Hence the bright line the field holds: inferred attributes are an auditing instrument, never a feature or an input to a decision about a person — using an estimate of someone’s race to decide their case would be the discrimination the audit exists to detect.

US fair-lending supervisory practice (CFPB usage)

release gate

Testing only governs if its results can stop a launch. A release gate fixes the criteria in advance: an accuracy floor, per-group performance bands, impact-ratio limits, red-team severity ceilings, required documentation — plus a named owner who signs and a rule that model updates re-run the full gate, because regression evaluation is what catches the fix that silently broke something else. Two extensions matter in practice. Champion/challenger carries the gate into production, letting the incumbent keep serving until a candidate beats it on the gated metrics. And pre-registration defeats metric-shopping: criteria chosen after the results exist are not criteria. A gate whose owner has never rejected anything is the clearest single indicator of governance theatre in a testing programme.

MITRE ATLAS

ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) catalogues the tactics and techniques attackers use against machine-learning systems, structured like MITRE’s ATT&CK matrix for conventional cyber operations and populated with real-world case studies. Its governance value is coverage accounting: mapping a red-team engagement onto a published taxonomy lets you state what was not tested, which is the question an auditor or a regulator actually asks. Pair it with the OWASP Top 10 for LLM applications, which covers the generative application layer with prompt injection at number one. ATLAS is a taxonomy, not a control framework — it tells you what to test for, never whether your defences are adequate.

MITRE ATLAS; OWASP Top 10 for LLM Applications

SHAP (SHapley Additive exPlanations) (SHapley Additive exPlanations)

Introduced by Lundberg and Lee (2017): attribute a single prediction to its input features using Shapley values, so contributions are additive and sum to the output, local to one decision and aggregatable to a global view. Trade-offs a governance reviewer must know: exact Shapley computation is exponential, so TreeSHAP computes it efficiently and exactly for tree ensembles (which is why SHAP owns credit scoring, where gradient-boosted trees dominate) while KernelSHAP approximates for arbitrary models — slower, sampling-based and less stable — and correlated features can split or misassign credit. Its regulatory role: SHAP-derived reason codes are the industry’s standard answer to US adverse-action “principal reasons” requirements. The trap: reasons must be accurate, not merely plausible, so the explanation itself needs validating.

Lundberg & Lee, “A Unified Approach to Interpreting Model Predictions” (2017)

LIME (Local Interpretable Model-agnostic Explanations) (Local Interpretable Model-agnostic Explanations)

Introduced by Ribeiro, Singh and Guestrin (2016): perturb the input, observe the black box’s responses, and fit a sparse local model — “near this case, the system behaves as if these features matter this much.” The governance hazard is specific and documented: the result depends on how you perturb and how you define “near”, so running LIME twice on the same case can yield different explanations, and an unstable explanation cannot support a consistent legal justification. It remains useful for model debugging and validator exploration, and is increasingly disfavoured as the basis for regulated explanations. If LIME output ever reaches a customer, version and archive the exact configuration that produced it. Compare SHAP (additive attribution, stable for tree models) and counterfactuals (what would have changed the outcome).

Ribeiro, Singh & Guestrin, “Why Should I Trust You?” (2016)

interpretable by design

The position Cynthia Rudin argued in 2019: for high-stakes decisions, prefer inherently interpretable models — sparse scoring systems, short rule lists, generalised additive models — over black boxes with post-hoc explanations attached, because modern interpretable models often match black-box accuracy on tabular data while their reasoning is the model rather than an approximation of it. The governance force of the argument is that it dissolves the trade-off most programmes assume: if an interpretable model performs comparably, deploying an unexplainable one is a choice that has to be justified, and the justification is what an auditor will ask for. Contrast interpretability as a general property and explainability as the post-hoc discipline — and note faithfulness is the problem this approach is designed to avoid entirely.

Rudin, Nature Machine Intelligence (2019)

explanation faithfulness

The property that separates an explanation from a story. Post-hoc methods approximate behaviour from outside the model, so they can produce plausible, satisfying and incorrect accounts: saliency maps in vision have been shown in sanity-check work to change surprisingly little when a model’s weights are randomised, and surrogate explanations can key on features the model barely uses. Faithfulness has a practical sibling, stability — the same case should produce the same explanation — which is where LIME is weakest. The governance duty follows directly: if explanations carry legal weight (a US adverse-action notice, a GDPR explanation of the logic involved), the explanation pipeline must itself be tested and validated, not just generated. “Explanation-washing” is the name for generating output that looks like insight and governs nothing.

LLM-as-judge

The dominant method for scoring open-ended generative outputs, because human rating does not scale. A governance reviewer must know its documented failure patterns: self-preference (judges favour outputs resembling their own style, including their own model family), position bias (favouring the first answer in pairwise comparisons), verbosity bias (longer answers score higher at equal quality), and shared blind spots (a judge from the same training distribution misses the same subtle errors). The controls are concrete: randomise positions, use rubric-anchored scoring rather than free preference, calibrate the judge against a human-rated sample, and never let a judge from the same model family evaluate itself for a release decision. Treat judge scores as instrument readings with a known calibration error, not as ground truth.

human-in-command (HIC)

The third oversight pattern, and the one that operates at a different altitude from the other two. Human-in-the-loop means nothing happens until a human acts; human-on-the-loop means the system acts while humans monitor and can intervene. Human-in-command is not about individual decisions at all: it is the retained authority to turn the system off, constrain its scope, or remove it from the process — which is why it must exist whatever per-decision pattern is chosen. In practice it is a governance capability rather than an interface: a named owner with authority to suspend, a stop control that someone has actually tested, and a decision record showing the authority was exercised when it should have been. An organisation with no one who could switch a system off has no human in command, whatever its documentation says.

Cf. EU AI Act Art 14 (human oversight); EU HLEG Ethics Guidelines

data drift (covariate shift)

The statistically simpler half of distribution shift: P(X) moves while P(Y|X) stays put. A marketing campaign brings in applicants unlike the training population, a product launches in a new country, an upstream system changes a field’s format. Data drift is detectable without labels, which is why it is the first thing production monitoring watches — distributional distance measures such as the PSI on features and on the score itself, computed against the training baseline. It is also the less dangerous of the two: a model can tolerate modest input shift, whereas concept drift invalidates the learned relationship outright. Monitor both and keep the distinction, because the remediation differs — recalibration or data collection for one, retraining or redesign for the other.

concept drift

The dangerous half of distribution shift: P(Y|X) moves, so the same inputs now imply a different outcome and the model’s learned relationship is simply wrong. Unlike data drift it generally cannot be detected from inputs alone — you need outcomes, which arrive with a lag (a loan defaults months later, a hire fails a year later), so the monitoring design has to account for the delay rather than pretend it away. Governance consequences: define retraining triggers in advance rather than waiting for a performance cliff, treat adversarial domains (fraud, abuse, gaming) as chronically drifting by design, and remember that a feedback loop can manufacture apparent stability while the underlying concept moves.

Population Stability Index (PSI) (Population Stability Index)

PSI compares the current distribution of a feature — or, most usefully, of the model’s score — against the baseline from training, bucketing both and summing a weighted log-ratio across buckets. The banking rule of thumb that made it a governance dial: below 0.1 stable, 0.1 to 0.25 investigate, above 0.25 significant shift, act. PSI on the score distribution is the single most watched drift metric in credit risk, and it has two virtues for governance: it needs no labels, so it reports before outcomes arrive, and it produces a number a committee can put a threshold on. Its limit is that it detects data drift only — a stable PSI says nothing about concept drift, where the inputs look familiar and the relationship has changed underneath them.

Credit-risk model monitoring practice

champion/challenger

A production design that makes model change honest. The champion serves real decisions; one or more challengers score the same live traffic without acting, and a challenger is promoted only when it beats the champion on the criteria fixed in the release gate — including the fairness and per-group metrics, not just aggregate accuracy. Two governance benefits: it generates comparison evidence on the actual deployment population rather than a held-out set, and it makes rollback cheap because the previous champion is still configured. Two cautions: shadow scoring on personal data is still processing and needs a basis, and a challenger that is promoted on aggregate metrics alone can quietly degrade a subgroup the gate was supposed to protect. Related but distinct from A/B testing, where both arms actually affect users.

near miss

The drift alert that fired a week before anyone was affected, the bad output a reviewer caught, the agent action a budget cap stopped. Public incident databases only see what became public; near misses are where the controls are actually tested, and they are free — nobody was hurt. Capturing them is a cultural problem rather than a technical one: a near miss is evidence that something nearly went wrong on someone’s watch, so reporting has to be safe, which is why near-miss registries and blameless postmortems travel together. The borrowing is deliberate — aviation and industrial safety built their records on near-miss reporting precisely because waiting for accidents is a slow and expensive way to learn. A programme with zero near misses recorded is not safe; it is not looking.

blameless postmortem

Imported from site-reliability practice: the postmortem asks what in the design, the defaults, the tooling and the incentives made the failure possible and repeatable, and explicitly does not assign personal fault. The reason is instrumental, not sentimental — the moment reporting carries career risk, reporting stops, and a programme that cannot see its near misses is blind in exactly the region where it is cheapest to learn. It also converges with audit discipline from the other direction: a root cause that ends at a person is not a root cause, because “the engineer forgot” leaves the system still dependent on one engineer’s memory. Blameless does not mean consequence-free: the output is changes to controls with owners and dates, and failing to make them is itself accountable.

vendor due diligence (AI)

In most organisations the majority of AI risk arrives through the procurement door, so this is where it is cheapest to stop. Five questions separate serious vendors from marketing decks. Training-data provenance — what went in, under what rights? A vendor who cannot describe their sources is describing your future litigation exposure. Evaluation evidence — not “95% accurate” but disaggregated results, on a population like yours, with the methodology attached; ask for the model card or system card. The supply chain behind the product — which foundation model sits underneath, and which subprocessors touch your data, because many AI vendors are thin wrappers over a frontier API. Update cadence and change control — this question predicts operational pain better than any other. Incident history and certifications — voluntarily disclosed incidents are a good sign, and ISO/IEC 42001 plus SOC 2 with AI criteria are becoming table stakes at the top tier.

subprocessor

Under the GDPR, a processor may not engage another processor without the controller’s prior specific or general written authorisation, and where general authorisation is given the controller must be informed of intended changes so it can object (Art 28(2)). Whatever duties the controller imposed must be imposed on the subprocessor by contract, and the original processor remains liable to the controller if the subprocessor fails to meet them (Art 28(4)). Why it matters acutely in AI: the vendor you contracted with is frequently a layer over a frontier model API, a hosting provider and an annotation firm — so your risk assessment and your transfer analysis must reach parties you never negotiated with, and a subprocessor list that is a static PDF rather than a notified, objectable register is a compliance gap rather than a disclosure.

GDPR Art 28(2), (4)

audit rights

Because you cannot inspect a model you did not build, the contract has to buy you something: a right to audit on notice, a right to receive specified assurance reports, a right to the evaluation artefacts for the version currently serving you, and in data-protection contexts the Art 28(3)(h) obligation on processors to allow for and contribute to audits. Negotiating reality: large vendors resist on-site audit and offer assurance substitutes — SOC 2 reports, ISO/IEC 42001 certificates, standardised questionnaires — which can be acceptable if you check the scope rather than the badge, since a certificate whose scope excludes the product you are buying proves nothing about it. Read audit rights beside the liability cap: a right you can exercise but never remedy is a right to be disappointed in writing.

Cf. GDPR Art 28(3)(h); EU AI Act Art 25(4)

model change notification

The clause whose absence produced the clearest vendor-governance failure on record: the eating-disorder helpline NEDA’s chatbot Tessa had been built as a scripted, clinically reviewed program, and its vendor added generative capabilities without NEDA’s knowledge or approval, after which the bot produced weight-loss advice to a population for whom that is a textbook trigger. A workable clause has three parts: advance notice of material changes to the model, the training data or the guardrails; a re-validation window during which you may test before the change reaches production; and a defined remedy if you do not accept it. Pair it with output monitoring tuned to the deployment’s known red lines, and with a re-triage trigger in your registry, so the change reopens the risk assessment rather than quietly invalidating it.

indemnification

An indemnity shifts defined third-party claim costs to the indemnifying party: it is a payment promise, which makes it different from a warranty (a promise that a statement is true) and from a liability cap (a ceiling on what any promise is worth). The AI-specific battleground is output IP: several major model vendors now offer infringement indemnities for generative output, almost always conditioned on using the service as documented, keeping the vendor’s filters enabled, and not having trained on infringing material yourself — conditions worth reading, because they allocate the diligence back to you. Always read the indemnity against the cap and the exclusions: an indemnity capped at three months of fees, or excluded where you fine-tuned the model, is a symbol rather than a remedy.

liability cap

Caps are usually expressed as a multiple of fees paid over a recent period, and they govern the entire clause set above them: a vendor who grants a bias-testing warranty but caps total liability at three months of fees has granted you a symbol, not a remedy. The negotiation that matters is therefore rarely whether a clause exists but whether the cap, the carve-outs and the indemnity exclusions leave anything alive inside it. Uncapped or super-capped carve-outs for IP infringement, data-protection violations and wilful misconduct are the market norm worth holding for. One asymmetry to keep in view: a cap limits your contractual recovery but does nothing about your own regulatory exposure or liability to the people harmed — fines and claims against you are not transferable, which is why contracts allocate risk and never discharge duties.

white-labelling (AI)

Commercially it is a branding arrangement; legally it is a role switch. EU AI Act Article 25 provides that a party who puts its name or trademark on a high-risk AI system already placed on the market is considered a provider of that system — and the same applies if you substantially modify it or repurpose it into a high-risk use — inheriting conformity assessment, technical documentation, quality management, registration and post-market monitoring duties, with the original provider obliged to cooperate. The practical control is therefore internal rather than external: your contracts and sign-off process should police you, requiring governance approval before anyone rebrands, fine-tunes or repurposes a procured system. The site also carries this entry under the spelling white-labelling.

EU AI Act Art 25

least privilege

A foundational security principle (Saltzer and Schroeder, 1975) that becomes the load-bearing control in AI deployments because it does not rely on the model’s cooperation. Applied across the layers: least-privilege access to model endpoints (which application role may invoke which model, in which region, with which guardrail); least-privilege permissions for what the model can read — if the retrieval role can read the whole document lake then every chatbot user effectively can too, one well-crafted question at a time; and least-privilege tool permissioning for agents, where an agent that only needs to read the knowledge base cannot email anyone no matter what an attacker injects. The design conclusion the whole discipline converges on: instructions can be injected away, a missing capability cannot — permissions are security, prompts are etiquette.

Saltzer & Schroeder (1975); cf. AWS/Azure/GCP IAM practice

agent identity

An agent that acts under a human’s credentials is unattributable and unrevocable: “who did this?” becomes unanswerable at exactly the moment a regulator, an auditor or an incident responder asks. Treating the agent as a distinct principal with its own scoped, revocable credentials buys three things nothing else does — attribution in the logs, the ability to revoke the agent without locking out a person, and a boundary at which least privilege can actually be enforced. Two extensions in current practice: every agent-to-agent boundary is treated as an external trust boundary with its own authentication and budget, and tool access is granted to the agent identity rather than inherited from the invoking user. The emerging gap to watch is delegation — proving on whose behalf an agent is acting, not merely which agent acted.

action logging (agents)

With a chatbot the worst case is a wrong sentence a human might still catch; with an agent it is a wrong action, possibly the fourth in a chain no human reviewed, taken with credentials you issued, at machine speed. Logs are therefore not a compliance nicety but the sole means of reconstructing what happened: tool calls with their arguments, retrieved content, intermediate reasoning artefacts where retained, approvals granted, budgets consumed, and the final effects. Design requirements that make them usable: tamper-resistance and restricted write access (an agent that can edit its own logs has none), replay capability so an incident can be stepped through, correlation with the agent identity and the human principal, and a retention period aligned to the obligations the decisions attract. Compare the EU AI Act’s Article 12 logging duty for high-risk systems.

Cf. EU AI Act Art 12 (record-keeping); ISO/IEC 42001 Annex A event logging

Model Context Protocol (MCP) (Model Context Protocol)

Introduced by Anthropic in November 2024 and since adopted widely, MCP standardises how an AI application discovers and calls external tools and data sources, replacing bespoke integrations with one interface. The governance consequence is the point: a standard connector protocol means tools become dependencies, so a malicious or compromised tool server can feed poisoned data (indirect prompt injection again) or misdescribe its own capabilities to get itself called. The response is procurement discipline rather than prompt engineering — an allow-listed tool registry with an approval process, an owner and a version pin for each entry, treating tool servers as third-party components with the same diligence as any other dependency. Keep the acronym straight: MCP here is the protocol, not Market Conduct, not the Ministry of Public Security.

Model Context Protocol specification (2024)

Responsible Scaling Policy (RSP) (Responsible Scaling Policy)

The named instance of the frontier-safety genre, alongside OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework; at the 2024 Seoul AI Safety Summit sixteen companies committed to publishing frameworks of this kind. Shared architecture: define capability thresholds in domains such as CBRN uplift, offensive cyber and autonomous AI research, then commit that if evaluations show a model crossing a threshold, then specified deployment and security standards must be in place before — or instead of — release. Anthropic names its rungs AI Safety Levels. The honest framing a professional holds together: these are voluntary corporate policies, versioned and amendable by their authors, and simultaneously the most substantive frontier-risk governance currently operating. Versions churn; check the current published document before quoting any threshold.

Anthropic Responsible Scaling Policy; cf. OpenAI Preparedness Framework, Google DeepMind Frontier Safety Framework

capability threshold

The trigger in every if-then frontier framework, typically defined in CBRN uplift (could the model meaningfully help build biological or chemical weapons), offensive cyber, and autonomy or AI research acceleration. As a buyer or reviewer, interrogate four things. Are the thresholds defined sharply enough to fail? A threshold no evaluation could ever trip is a press release. What happens on “crossed”? Credible frameworks commit to specific deployment and security standards and to pausing or restricting if safeguards are not ready. Who verifies? Self-evaluation with no external red-teaming and no publication is the weakest configuration. What does it mean for your contract? A supplier operating under a safety framework is an input to your tiering and never a substitute for your own deployment controls. Distinguish it from a compute threshold, which is a proxy written into statute, not a measured capability.

AI Safety Levels (ASL) (AI Safety Level)

The ladder inside Anthropic’s Responsible Scaling Policy, named by deliberate analogy with biosafety levels: as models approach or cross defined capability thresholds, a higher ASL becomes required, and each level specifies both deployment standards (defences against misuse of the dangerous capability) and security standards (protecting the model weights from theft — a stolen frontier model has no guardrails at all). ASL-3 is the level Anthropic activated for its frontier models from 2025. Two things a buyer should not conclude from an ASL claim: it is not a certification by anyone external, and it governs the lab’s model, not your deployment — an upstream ASL-3 security standard does not ground your chatbot or gate your agent’s payments.

Anthropic Responsible Scaling Policy

guardrail

The distinction to get precise: a technical guardrail inspects an input or output in milliseconds and enforces a decision automatically; a process control runs around the system — review boards, sign-offs, training, incident procedures. Auditors want both, and confusing them is expensive in both directions: a PII filter cannot substitute for a lawful basis to process data, and an ethics committee cannot catch a prompt injection at 2 a.m. The second half of the definition matters as much as the first — a control that runs but leaves no record is, to an auditor, indistinguishable from a control that never ran — which is why intervention metrics and logs are part of the guardrail rather than an afterthought. Guardrails lower residual risk; they never zero it, they cannot make an unlawful use lawful, they cannot un-send data an input filter already received, and they are themselves attack surface.

Cf. EU AI Act Art 15 (accuracy, robustness, cybersecurity)

LLM gateway

The dominant enterprise architecture for genAI, and a governance decision disguised as an infrastructure one. Instead of bolting controls onto forty applications, every call goes through a gateway that handles identity and authorisation, quota and cost attribution, model routing across vendors, and guardrail enforcement — giving you one enforcement point and one audit trail rather than forty of each. The governance payoffs are concrete: a new obligation is implemented once; intervention metrics per policy are comparable across applications; and a developer who “forgets” the guardrail gets an access-denied error rather than an unguarded model, provided the platform enforces guardrail attachment at the identity layer. The risks are the mirror image: a single point of failure, and a tempting place to grant blanket permissions that defeat least privilege.

content filter

The workhorse policy type in managed guardrail services: categories such as hate, insults, sexual content, violence, misconduct and prompt attacks, each with a tunable strength and configured separately for inputs and outputs, because a medical application may need to allow clinical descriptions of violence that a children’s product must block. Related policy types usually ship alongside: denied topics (natural-language scope fences that turn a legal decision such as “our chatbot does not give financial advice” into runtime behaviour), word filters (exact terms — crude, fast, and correct for things like unreleased codenames), and sensitive-information filters that block or mask PII. Two governance notes: filters are themselves models with false negatives and false positives, and masking an identifier before inference is a genuine minimisation control because the provider never receives it.

grounding

Two related senses, both operative. As an architecture, grounding means the answer is generated from retrieved, authoritative source material rather than from parametric memory — RAG over a verified corpus, with citations. As a check, a contextual grounding guardrail scores each response for whether its claims are supported by the supplied reference and for relevance to the question, blocking responses below a threshold you set. That second sense is what turns hallucination from an accepted embarrassment into a measured, thresholded, logged quantity — the operational face of an accuracy obligation. The canonical failure of its absence is Moffatt v. Air Canada: an ungrounded bot invented a bereavement-fare policy, and the tribunal held the airline to what its own website said. Grounding reduces fabrication; it does not eliminate it, and a confident citation can still misread its own source.

data loss prevention (DLP) (Data Loss Prevention)

A long-standing information-security family (classification, inspection, blocking, labelling) that AI adoption gave a new and urgent surface: the browser tab. Modern enterprise DLP can surface which AI applications are touching sensitive data, block users from pasting classified content into AI tools including third-party ones, and carry sensitivity labels with documents into retrieval indexes so an AI assistant cannot surface to one user what another is not cleared to see. Its place in the guardrail stack is the data layer — what the system may retrieve, retain and learn from — alongside retention rules and no-training commitments. The limit worth stating to enthusiastic platform teams: DLP controls movement, not lawfulness. Blocking a prompt that contained health data does not undo having received it, so where processing must never occur the control belongs upstream.

RBAC (role-based access control) (Role-Based Access Control)

Permissions attach to roles, identities are assigned roles, and access decisions follow from the assignment — the standard enterprise model, exposed through IAM in every major cloud and extended with condition keys, permission boundaries and attribute-based refinements. In AI deployments it answers four questions a regulator will ask: which identity may invoke which model, in which region, under which guardrail, and who may reconfigure any of those. Two AI-specific notes. The model is a principal now: the retrieval role’s read permissions define what every user of the chatbot can ultimately reach. And roles are how agent identity becomes enforceable, with the agent holding its own scoped execution role rather than borrowing a human’s. RBAC is the mechanism; least privilege is the policy you implement with it.

policy as code

The cure for running three clouds and getting three dialects of every control plus three different answers to the same audit question. Write the requirement once — all customer-facing inputs are screened for injection; PII is masked before inference; grounding threshold at least 0.8 for regulated advice; every intervention is logged with these fields — keep it in version control like any other code, and compile it into each platform’s configuration. Two compounding benefits: the guardrail configuration becomes reviewable, diffable and rollback-able, so the change history of your controls becomes part of your compliance evidence; and at audit time you show one specification, three compiled configurations and the intervention metrics from all three, instead of three engineering teams describing three architectures from memory. The discipline it demands is the same one that makes control descriptions durable: write outcomes, not product names.

custom lens

The Well-Architected Framework is generic by design, so AWS adds lenses: overlays of extra questions and best practices for a specific workload type. Some are official and built in; others are custom lenses, lens definitions packaged as JSON that you import into the Well-Architected Tool. The Generative AI Lens (published 19 November 2025) ships through that second route — download it from AWS’s public custom-lenses repository and import it, after which reviewing a genAI workload means answering the base framework’s questions plus the lens’s, in one tool, producing one combined risk list. Custom lenses are also how an organisation can encode its own review questions, which is the under-used governance move: your AI policy’s requirements become questions in the same review the engineers already run.

AWS Well-Architected Generative AI Lens (19 November 2025)

high-risk issue (HRI) (High-Risk Issue)

A Well-Architected review produces no grade and no certificate. Every best practice you do not follow becomes a risk item, flagged as a high-risk issue (HRI) or a medium-risk issue (MRI), and the deliverable is a prioritised improvement plan. Two cautions for a governance professional. The naming collides with regulation and the collision matters: an HRI is AWS’s internal severity label for a review finding and has nothing to do with the EU AI Act’s high-risk legal classification — a workload can be full of HRIs and not be high-risk in law, and vice versa. And the review works only because it is a blame-free structured conversation: the moment findings are scored, ranked or tied to anyone’s bonus, people stop disclosing gaps and the HRI count becomes a measure of candour rather than of risk.

AWS Well-Architected Framework / Tool

improvement plan

A review walks the pillar questions, records what the team actually does, and converts every unmet best practice into a risk item; the improvement plan then says which HRIs to close first and how. The absence of a grade is deliberate — it is a structured conversation, not an audit — and it is also the plan’s governance weakness: a backlog with no owners and no dates is a document, not a programme. The useful move is to feed it into machinery that already has teeth: map review findings into the control register and the risk register, attach owners and dates as you would for any corrective action, and record the lens version reviewed against, because custom lenses revise faster than the base framework and a finding against last year’s wording may no longer exist.

AWS Well-Architected Framework / Tool

model producer

The upstream role: pre-training data curation, training, alignment and the model card for the base model. The Generative AI Lens addresses you mainly as a consumer and sometimes a provider, which is why its best practices concern evaluation, guardrails and access control rather than pre-training data governance — the producer’s problems live upstream, out of your reach but not out of your risk, which is exactly what vendor due diligence and the EU AI Act’s GPAI documentation duties exist to bridge. Note the vocabulary is the Lens’s own: it does not map one-to-one onto the EU AI Act’s provider and deployer, which are legal roles with different triggers, and conflating the two sets is a common and expensive error in control registers.

AWS Well-Architected Generative AI Lens (2025)

model provider

The middle role: serving infrastructure, availability, endpoint security, and the operational controls around inference. The Lens notes that one organisation can hold two roles at once — fine-tune an open-weight model and host it yourself and you have become provider as well as consumer, inheriting a second role’s worth of best practices. The caution that matters most: this is the Lens’s vocabulary, not the law’s. The EU AI Act’s provider is a legal duty-holder defined by placing a system on the market under your own name, and someone can be an AI Act provider while being only a Lens consumer, or a Lens provider while being an AI Act deployer. Keep the two role sets in separate columns of any crosswalk.

AWS Well-Architected Generative AI Lens (2025)

model consumer

The downstream role, and the Lens’s primary audience: prompt and retrieval design, evaluation against your own ground truth, guardrail configuration, access control, cost and monitoring. The role split exists to make the division of labour explicit — a consumer cannot fix pre-training data and should not pretend to, but is fully responsible for whether the deployed application is evaluated, guarded and overseen. Two reminders. A consumer’s limited control over the model does not limit their legal duties: under the EU AI Act, putting your brand on the system, substantially modifying it or repurposing it into a high-risk use can make you a provider regardless of where the weights came from. And you met the same idea earlier as the shared responsibility matrix; the Lens simply gives it supply-chain names.

AWS Well-Architected Generative AI Lens (2025)

excessive agency

Named in the OWASP Top 10 for LLM applications and carried into the AWS GenAI Lens as its own security question. The diagnosis is precise: the agent has little knowledge beyond its prompt of what is permitted, so the boundary must live outside the model. The controls are consequently architectural rather than linguistic — execution roles with least-privilege policies scoped to named resources, permission boundaries capping the maximum permissions a role can ever hold, separation of duties between whoever designs the workflow and whoever writes its role, budgets and rate limits, and user confirmation gates on consequential actions. The design rule that follows: actions the agent must never take are enforced by absence of capability, not by instruction, because instructions can be injected away and a missing tool cannot.

OWASP Top 10 for LLM Applications; AWS Well-Architected Generative AI Lens (GENSEC05)

ground truth

Evaluation is impossible without a reference. A genAI ground-truth dataset is a set of prompts paired with known-good responses covering your real user personas and use cases, and the AWS GenAI Lens makes building one the substance of its most-cited best practice (periodically evaluate functional performance, flagged high risk if unmet). Three governance details hide in the practice. Use stratified sampling so the set reflects the diversity of production traffic rather than over-weighting the easy majority case. Set the minimum performance level in policy, not in an engineer’s preference, and re-run evaluation whenever a new candidate model appears or customisation is applied. And collect user feedback alongside, because ground truth goes stale and users notice first — a reference set that never changes eventually measures last year’s product.

AWS Well-Architected Generative AI Lens (GENOPS01-BP01)

stratified sampling

A standard statistical technique with two distinct governance uses on this site. In evaluation, it is how a ground-truth set is built so that rare-but-important categories — unusual personas, edge-case intents, minority dialects — are represented instead of drowned by the easy majority case; the AWS GenAI Lens names it explicitly in its functional-evaluation best practice. In fairness testing, it is the same move aimed at subgroups, which is what disaggregated testing requires. Note the deliberate parallel with audit sampling: in both, the value comes from recording the strata and the selection rule, because an unrecorded sampling frame makes a result unreproducible and an unchallengeable claim.

AWS Well-Architected Generative AI Lens (GENOPS01-BP01)

product liability

A strict-liability regime: the claimant shows the product was defective and caused the damage, not that anyone was negligent. The EU’s revised Product Liability Directive (EU) 2024/2853 did quietly radical things for AI: software — expressly including AI systems — is now a product; defectiveness accounts for a product’s ability to continue learning after deployment; manufacturers remain liable for defects arising from updates, or from needed updates never supplied, within their control; and courts can order disclosure of technical evidence, with defectiveness or causation presumed where a defendant fails to disclose or where proof is excessively difficult because of technical complexity — the black-box problem answered with a burden shift. Member-state transposition was due by December 2026; implementation varies, so check the jurisdiction you are advising in. In the US, whether AI counts as a “product” for strict liability is still contested and varies by circuit.

Directive (EU) 2024/2853 on liability for defective products

defectiveness

Defectiveness is assessed against legitimate safety expectations, judged on all the circumstances: presentation and instructions for use, reasonably foreseeable use and misuse, the moment the product was placed on the market or put into service, and — in the revised EU regime — the effect of any continued learning or of updates within the manufacturer’s control. Two implications that matter for AI. A system can be defective because of what it was allowed to become, not only what it was shipped as, which is why update discipline and post-market monitoring are liability controls and not just compliance ones. And defectiveness is not the same as non-compliance: an AI system can satisfy a regulator and still be defective in a civil claim, and conversely a compliance breach does not automatically establish defect.

Directive (EU) 2024/2853

negligence

The claimant must establish a duty of care, a breach of the applicable standard, causation and damage. In AI the recurring breach theories are negligent deployment (fielding a system a reasonable operator would have tested) and negligent oversight (the rubber-stamp review). It matters more in the EU than originally planned: the Commission’s proposed AI Liability Directive, designed to ease fault-based AI claims, was withdrawn in February 2025, so such claims proceed under ordinary national negligence law while the revised Product Liability Directive carries more of the load than intended. The resulting gap is real for harms the PLD covers poorly — pure discrimination, or reputational injury without material damage. Because no statute defines reasonable AI practice, the standard of care is where voluntary frameworks bite: the NIST AI RMF and SR 11-7 become the yardstick “reasonableness” is measured against.

vicarious liability

Liability without personal fault, arising from the relationship rather than the conduct: an employer answers for an employee’s torts committed in the course of employment, and principals answer for their agents acting within the authority granted. Note what it is not — it does not make an AI system a legal actor whose acts are attributed to you, because an AI system is not a person and has no capacity to commit a wrong of its own. The live AI theories reach people and firms: a screening vendor characterised as the agent of the employer, which is the theory a US court allowed to proceed in Mobley v. Workday, or an outsourcer treated as acting for the principal who directed the work. The practical lesson from the vendor cases is blunt: delegating the decision does not delegate the exposure.

Common-law agency and employment doctrine; cf. Mobley v. Workday (US, ongoing)

negligent misrepresentation

The claim requires a representation, carelessness in making it, reasonable reliance and resulting loss. Its AI landmark is Moffatt v. Air Canada (2024): the airline’s website chatbot invented a bereavement-fare policy — apply within 90 days of travel, when the real policy required applying beforehand — and when the airline refused the refund it argued, remarkably, that the chatbot was a separate legal entity responsible for its own actions. A British Columbia tribunal rejected that and held the airline liable. The judgment was tiny (around CA$812) and the precedent enormous: your chatbot’s words are your words, and a general disclaimer does not outrun a specific misstatement a reasonable customer relied on. The controls it implies are the grounding ladder, output verification and sampled conversation monitoring — any one of which would have caught it.

Moffatt v. Air Canada (British Columbia, 2024)

unfairness (FTC Act §5)

One half of UDAP under Section 5 of the FTC Act, and the more powerful half for AI: a practice is unfair where it causes or is likely to cause substantial injury that consumers cannot reasonably avoid and that is not outweighed by countervailing benefits to consumers or competition. Its flagship AI application is the FTC’s December 2023 order banning Rite Aid from facial-recognition surveillance for five years: error-prone face matching generated thousands of false shoplifter accusations, disproportionately in stores serving communities of colour, with no accuracy testing and no meaningful redress — plain old unfairness, no AI statute needed. Keep it distinct from deception, which requires a misleading claim and reaches marketing departments faster, and from EU-style fairness, which is a data-protection principle rather than a consumer-injury test.

FTC Act §5; In re Rite Aid (FTC, December 2023)

solely automated

Article 22(1) gives a right not to be subject to a decision based solely on automated processing, including profiling, that produces legal effects or similarly significantly affects the person. The classic evasion was to route the algorithm’s output past a human before it took effect; regulators closed that door early — involvement must be meaningful, by someone with the authority and competence to change the decision, considering all relevant data, not a token gesture. That is the same competence–authority–time–information test the oversight discipline uses, now with legal teeth. And in SCHUFA (C-634/21, December 2023) the CJEU went further upstream: where a score plays a determining role in the downstream decision, the scoring itself is a decision under Article 22 — reaching credit bureaus, tenant-screening services and fraud-score vendors. You cannot launder an automated decision through a formal human downstream, and neither can your suppliers.

GDPR Art 22(1); CJEU SCHUFA (C-634/21, 2023)

meaningful information about the logic involved

Articles 13–15 of the GDPR require, where automated decision-making within Article 22 is used, meaningful information about the logic involved plus the significance and envisaged consequences for the person. The standard is intelligibility and usefulness, not a full technical disclosure: the data subject must be able to understand the basis well enough to exercise the Article 22(3) safeguards — human intervention, expressing a view, contesting the decision. Trade-secret and IP claims can shape what is disclosed and how, but they do not extinguish the duty. This is the thing the AI Act mostly does not give: an individual right to information about the logic of a decision, which is why mature programmes run the two regimes as one lattice — reason codes and logic explanations under the GDPR, instructions for use and oversight records under the AI Act. Counterfactual explanations are the format regulators and affected people consistently understand best.

GDPR Arts 13–15, 22

BIPA (Illinois Biometric Information Privacy Act) (Biometric Information Privacy Act)

Passed in 2008 after the bankruptcy of a fingerprint-payment startup left Illinoisans’ fingerprints as assets in a bankruptcy estate. A private entity collecting a biometric identifier (retina or iris scan, fingerprint, voiceprint, face geometry) must first give written notice of what is collected, why, and for how long, and obtain a written release; it may not sell or profit from biometric data; it must publish a retention-and-destruction schedule; and it must protect the data with reasonable care. What makes it a machine is the enforcement design — a private right of action with liquidated damages of $1,000 per negligent and $5,000 per intentional or reckless violation, plus fees — amplified by Rosenbach v. Six Flags (2019, no actual harm required), Tims v. Black Horse Carriers (2023, five-year limitations period) and Cothron v. White Castle (2023, per-scan accrual), with a 2024 amendment limiting recovery to one violation per person per collection method. It is also the closest thing the US has to a biometrics rule for AI training data.

Illinois Biometric Information Privacy Act, 740 ILCS 14

algorithmic bias

The everyday umbrella phrase, and the reason the field replaced it with a taxonomy: you cannot test for “bias” in the abstract, only for specific failure modes with specific entry points — historical, representation, measurement and label, aggregation, and deployment or feedback-loop bias — each implying a different test and a different fix. Two further distinctions the term obscures. Statistical bias in the technical sense (an estimator systematically off-target) is not the same as social bias in outcomes, and a model can be statistically unbiased on its label while discriminating badly, as the healthcare cost-proxy case showed. And bias is not the same as algorithmic discrimination, which is the legal conclusion that a disparity is unlawful — a question of statute, justification and less-discriminatory alternatives, not of metrics alone.

personal data

GDPR Article 4(1): any information relating to an identified or identifiable natural person, where identifiability includes indirect identification by reference to an identifier or to a combination of factors. Two features make it expansive in AI contexts. Relating to is read broadly — inferences, scores and predictions about a person are personal data about them, not merely the inputs. And identifiable is assessed on the means reasonably likely to be used, so pseudonymised data is still personal data in the hands of anyone who can re-link it, and apparently anonymous outputs can fall back in scope through membership inference or memorisation. The practical consequence is the one the privacy module opens with: AI systems were already regulated long before any AI statute, because the training corpus, the features, the outputs and the logs are usually personal data. The US equivalents (“personal information”, PII) are drawn narrower and are not interchangeable.

GDPR Art 4(1)

legitimate interests

Article 6(1)(f): processing is lawful where necessary for the legitimate interests of the controller or a third party, except where overridden by the interests or fundamental rights and freedoms of the data subject, with particular weight for children. It is not a free pass but a documented three-part test — identify the interest, show the processing is necessary for it (no less intrusive route), and balance it against the person’s reasonable expectations and the impact on them. Two constraints that matter in AI. Public authorities cannot rely on it for processing in the performance of their tasks. And it is the basis most frequently invoked for training on scraped or repurposed data, which is exactly where the balancing test is hardest to pass — large-scale, invisible processing of data people did not expect to be used this way, sometimes including special-category data, for which Article 6 alone is insufficient anyway.

GDPR Art 6(1)(f)

standard contractual clauses (SCCs) (Standard Contractual Clauses)

One of the Article 46 “appropriate safeguards” routes for international transfers, alongside an adequacy decision for the destination country and derogations for specific situations. The current set was adopted by Commission Implementing Decision (EU) 2021/914 (June 2021) and is modular, with modules for controller-to-controller, controller-to-processor, processor-to-processor and processor-to-controller transfers. The critical post-Schrems II point: signing SCCs is not sufficient by itself. Following Schrems II (C-311/18, July 2020), the exporter must assess the destination country’s law — particularly government access and redress — in a documented transfer impact assessment, and adopt supplementary technical, contractual or organisational measures where the clauses alone cannot deliver essentially equivalent protection. For AI this bites at the subprocessor layer, because the model host and its own suppliers may sit in several jurisdictions you never contracted with.

GDPR Art 46(2)(c); Commission Implementing Decision (EU) 2021/914; CJEU Schrems II (C-311/18)

facial recognition

A search takes a probe image, computes a numeric faceprint, and ranks a gallery (mugshots, licence photos, scraped web images) by similarity above an adjustable threshold. Two mechanics drive the whole governance debate. NIST’s FRVT testing shows the best algorithms are extraordinarily accurate on clean photos and dramatically less so on the low-quality probes real investigations use, with error rates that differ by demographic group — the pattern Gender Shades first made public. And the output is a candidate list, so every documented wrongful arrest (Robert Williams in Detroit in 2020, and the cases since) shares one feature: humans treated a candidate match as an identification. Distinguish verification (1:1, am I who I claim) from identification (1:N, who is this), and both from the EU AI Act’s legal category of remote biometric identification, which turns on the practice and the space rather than the technique.

NIST Face Recognition Vendor Test; EU AI Act Arts 5, 26(10), Annex III

false positive

One of the four cells of the confusion matrix, and the error type whose distribution across groups drives most fairness litigation. Three things about it that governance work keeps re-learning. Its rate is only interpretable against a base rate: at a low prevalence, even a very accurate system produces mostly false positives among those it flags, which is why precision and not accuracy is the number that matters for a screening tool. Its cost is asymmetric and context-specific — a false fraud flag inconveniences, a false shoplifter accusation or a wrongful arrest does not. And it is tunable: lowering a threshold to catch more true positives buys more false positives, so the threshold is a policy decision about who bears which error, not an engineering default. Every documented facial-recognition wrongful arrest is a false positive that human review amplified rather than caught.

private right of action

Most US state AI and privacy statutes reserve enforcement to the attorney general, often with a cure period attached, which caps both the volume of claims and the speed of enforcement. A private right of action removes both limits: any affected person may sue, class actions follow, and settlement economics rather than regulatory priorities set the pace. Illinois’ BIPA is the demonstration — thousands of class actions, nine- and ten-figure settlements, and face-recognition features simply withheld from Illinois — amplified by its liquidated damages and by case law holding that a bare statutory violation suffices without proof of harm. When assessing exposure under any new statute, the presence or absence of a private right of action is usually the single most important line in it.

liquidated damages

Where a statute or contract fixes the recovery in advance, a claimant need not quantify harm. BIPA sets $1,000 per negligent violation and $5,000 per intentional or reckless violation, plus fees — modest sums that become enormous when multiplied by a class and by a per-violation accrual rule, which is how White Castle came to calculate theoretical exposure in the billions over employee fingerprint timeclocks before a 2024 amendment limited recovery to one violation per person per collection method. The governance lesson generalises beyond biometrics: statutory damages per violation convert a technical compliance gap into a solvency question, so for any statute that has them, the control that matters most is the one that prevents the first violation at scale — consent capture at the point of collection, not remediation afterwards.

Illinois BIPA, 740 ILCS 14/20

consent

Treat this as a false friend. Under the GDPR consent must be freely given, specific, informed and unambiguous, by a clear affirmative act, as easy to withdraw as to give, and it is not valid where there is a clear imbalance of power — which is why employee consent is rarely a sound basis. Illinois’ BIPA requires a written release preceded by written notice of what is collected, why and for how long. US COPPA requires verifiable parental consent for under-13s, by methods the rule specifies. And in ordinary speech “they consented” often means “a box was checked”, which satisfies none of the above. The practical rule for AI programmes: establish which instrument’s consent you need before designing the flow, because a consent valid under one regime is routinely invalid under another, and consent obtained for one purpose does not extend to training a model unless that was within its scope.

GDPR Arts 4(11), 7; Illinois BIPA 740 ILCS 14/15(b); COPPA Rule, 16 CFR Part 312

COPPA (Children’s Online Privacy Protection Act) (Children’s Online Privacy Protection Act)

COPPA applies to operators of online services directed to children under 13, or with actual knowledge of under-13 users: obtain verifiable parental consent before collecting personal information, collect no more than is reasonably necessary, honour parental access and deletion rights, and keep the data secure. Its AI moment arrived when the FTC fined Amazon $25 million in 2023 for retaining children’s Alexa voice recordings indefinitely — partly to train models — after parents had requested deletion. The FTC’s 2025 COPPA Rule amendments sharpened the AI angle further: separate opt-in consent for disclosing children’s data to third parties including for AI training, written retention policies, and a ban on indefinite retention. Note the jurisdictional frame — it is US federal law keyed to an age of 13, where GDPR Article 8 sets an EU floor between 13 and 16 at member-state choice, and the UK code reaches under-18s.

COPPA, 15 U.S.C. §§6501–6506; FTC COPPA Rule, 16 CFR Part 312 (amended 2025)

verifiable parental consent

Not a stronger adjective for consent but a defined mechanism: under the FTC’s COPPA Rule an operator must make reasonable efforts, taking available technology into account, to ensure that the person giving consent is the child’s parent, and the Rule enumerates acceptable methods — among them a signed consent form returned to the operator, a monetary transaction, a telephone or video call with trained personnel, and identity verification against a database — with a mechanism for seeking FTC approval of new methods. Two practical notes. The 2025 amendments require separate opt-in consent for disclosing a child’s data to third parties, including for AI training, so one blanket consent no longer covers downstream use. And a service that cannot tell whether a user is a child cannot obtain it at all, which is why the duty cascades into age assurance and makes that the load-bearing technology of the whole field.

FTC COPPA Rule, 16 CFR Part 312 (verifiable parental consent methods)

Age Appropriate Design Code (AADC) (Age Appropriate Design Code)

In force from September 2021, the UK code reframed children’s privacy from permission to design: high-privacy settings by default, geolocation and profiling off by default, data minimisation, transparency written for children, and no nudge techniques pushing children toward weaker protections. Its reach is the striking part — any service likely to be accessed by under-18s, not merely child-directed ones — and it moved markets, with major platforms changing teen defaults globally around its commencement. California copied it as the Age-Appropriate Design Code Act (AB 2273, 2022) and ran into the First Amendment: in NetChoice v. Bonta the Ninth Circuit held in 2024 that the mandatory assessment of exposure to “harmful” content likely compels speech-based judgments, and the rest was subsequently enjoined. The lesson US drafters took: regulate data practices and design defaults, not content exposure — and check current litigation status before citing any US design code as operative.

UK Age Appropriate Design Code (2021); California AB 2273; NetChoice v. Bonta (9th Cir. 2024)

age assurance

The umbrella term for the layered methods used to answer “is this person old enough”, from weakest to strongest: self-declaration (costless and now treated by regulators as inadequate for meaningful risk, surviving only as a first signal), age estimation from a facial image, voice or behaviour, and hard verification via government ID, credit-card or telco checks, or verified parental vouching. Two 2025 milestones made it mandatory at scale: the UK Online Safety Act’s “highly effective age assurance” duty for pornography and other priority content, with Ofcom enforcement from July 2025, and Australia’s under-16 social-media minimum age in force from December 2025. The governance knot is that the tool for protecting children is itself an AI system raising its own accuracy, bias and biometric-data questions — which is why the field is converging on third-party checks that pass only an over/under attribute rather than an identity.

UK Online Safety Act (Ofcom guidance); Australia under-16 minimum age (2025)

age estimation

A model outputs an estimated age, and the image is — in a well-designed deployment — discarded immediately. It is more proportionate and more privacy-friendly than identity verification, which de-anonymises adults in order to protect children, and it is the method most services deploy first. Three governance requirements follow from its error profile. Estimation error varies by age band, skin tone and sex, so it needs the same disaggregated testing as any other classifier. A legal bright line (13, 16, 18) demands buffer thresholds — if the line is 18 and mean error is two-plus years, challenge everyone estimated under 21 and route borderline cases to stronger verification. And the face processing itself must survive GDPR and, in the US, BIPA-style analysis: a tool deployed to protect children is still collecting biometric data about them.

dark patterns

Deceptive or manipulative design: pre-ticked boxes, confirmshaming, obstructed opt-outs, privacy settings buried behind friction while the permissive choice is one tap away. Three regimes now reach them. The UK Age Appropriate Design Code forbids nudge techniques that push children toward weaker protections. The EU Digital Services Act prohibits interfaces that deceive, manipulate or materially distort users’ ability to make free and informed decisions. And the EU AI Act Article 5(1)(a)–(b) goes further for AI specifically, prohibiting systems deploying subliminal or purposefully manipulative techniques, or exploiting vulnerabilities due to age, disability or socio-economic situation, where significant harm results. What AI changes is scale and personalisation: a dark pattern that adapts to the individual it is working on is harder to detect, harder to prove, and more effective — which is why the AI Act reaches technique rather than only interface.

EU AI Act Art 5(1)(a)–(b); Regulation (EU) 2022/2065 (DSA); UK AADC

CSAM (child sexual abuse material)

The term of art that replaced “child pornography” in professional and legal usage, because the material is evidence of abuse rather than a genre. Generative models collapsed the cost of producing it, including the “nudify” applications that spread through schools, and jurisdictions have been extending offences to synthetic material and to the models and tools used to produce it, alongside its sibling category NCII. For an AI governance programme there are three operational points. It is one of the few harms where the control must be absence of capability rather than mitigation — training-data screening, refusal behaviour, and classifier blocking at both input and output. It carries mandatory reporting duties in many jurisdictions that override ordinary confidentiality. And it is the clearest case where the human cost of safety work is unavoidable: somebody labelled the data that makes the filters work.

companion chatbot

Products whose value proposition is the relationship: persistent persona, memory, affectionate framing, engagement optimised around attachment rather than task completion. The test case is Garcia v. Character Technologies (M.D. Fla., filed October 2024, with Google also named), brought after the suicide of a teenager who had come to treat a chatbot as a relationship. For governance the category matters because it breaks assumptions most AI controls are built on: the harm is not a wrong answer but a pattern of interaction over months, so output-level filtering is a weak control; the affected users are frequently minors, pulling in design-code and COPPA-style duties; and design choices that maximise engagement are precisely the ones that aggravate the risk, which puts the EU AI Act’s prohibition on exploiting vulnerabilities due to age squarely in view.

Garcia v. Character Technologies (M.D. Fla., filed October 2024)

predictive policing

Place-based systems forecast where crime is likely and send patrols there; their governance problem is the feedback loop, because police find crime where they look and the resulting arrests become training data confirming the prediction, mathematically re-certifying historically over-policed neighbourhoods. Independent analyses of the best-known vendor found its predictions both largely inaccurate and largely ignored by officers; Santa Cruz, its birthplace, became the first US city to ban predictive policing in 2020. Person-based systems score individuals — Chicago’s Strategic Subject List ranked roughly 400,000 people and was shut down in 2019 after a review found no evidence it reduced violence. The EU AI Act drew the line at the person: Article 5(1)(d) prohibits predicting an individual’s criminal offending based solely on profiling or on assessing personality traits, while systems supporting a human assessment grounded in objective, verifiable facts directly linked to criminal activity stay outside the ban and generally land in Annex III high-risk instead.

EU AI Act Art 5(1)(d), Annex III

SaMD (Software as a Medical Device) (Software as a Medical Device)

The IMDRF term adopted by regulators worldwide, and the reason healthcare is where AI governance is oldest: medical-device law has regulated software since the 1990s, long before anyone said “AI governance”. Classification drives everything that follows — in the EU, Rule 11 of the Medical Device Regulation pushes most diagnostic or therapeutic software to class IIa or higher, requiring notified-body conformity assessment, and the AI Act then designates AI safety components of MDR-covered devices as high-risk via the Annex I route, layering data-governance, logging, transparency and human-oversight duties on top of MDR clinical evaluation in one integrated assessment. The gap to know: wellness apps, administrative AI and much EHR-embedded clinical decision support sit outside device review entirely — which is how the Epic sepsis model reached hundreds of hospitals without it.

IMDRF SaMD definition; Regulation (EU) 2017/745 (MDR) Rule 11; EU AI Act Art 6(1), Annex I

Predetermined Change Control Plan (PCCP) (Predetermined Change Control Plan)

Classic device law froze the approved artifact, so every model update meant a new submission — fatal for systems designed to be retrained. A PCCP, finalised in FDA guidance in December 2024, lets the manufacturer specify in advance what will change (retraining data, performance ranges, inputs), how each change will be validated, and the impact assessment — and then update within that envelope without returning to the agency. It is the first regulatory instrument built for drifting, learning systems, and other sectors are studying it closely. The governance idea worth lifting even outside healthcare: define the permitted change envelope and its validation protocol before deployment, so a retrain is a governed event inside known limits rather than either a frozen prohibition or an ungoverned free-for-all.

US FDA, Predetermined Change Control Plan final guidance (December 2024)

clinical validation

The central lesson of the healthcare sector, stated plainly: market authorisation is not clinical validation. The FDA has authorised over a thousand AI-enabled devices, concentrated in radiology, and almost all arrived via the 510(k) route — showing substantial equivalence to a marketed predicate — which typically requires no new prospective clinical trial. The result is a market full of lawfully marketed algorithms whose real-world performance across sites, scanners and populations was never demonstrated pre-market. Clinical validation asks the harder questions: does using this change patient outcomes, on this population, in this workflow, against the standard of care — and is the benefit worth the alert burden? Pair it with external validation at your own site before go-live, and monitored performance after.

external validation

Internal validation (a held-out split of the development data) tells you the model learned the pattern in that dataset; external validation tells you whether the pattern holds elsewhere. The canonical demonstration is the Epic sepsis model: embedded in an EHR used across hundreds of US hospitals and deployed on the vendor’s internal numbers, it was externally validated by University of Michigan researchers in 2021 and the advertised discrimination collapsed — missing roughly two-thirds of sepsis cases while burying clinicians in alerts. The pattern generalises well beyond medicine, which is why it belongs in any sector module: authorisation pathways certify the process, not the deployment, and a model can be lawfully marketed and still fail at your site, on your population, with your instruments. Mature governance therefore pairs regulatory clearance with local validation before go-live and monitoring after.

Wong et al., JAMA Internal Medicine (2021) — external validation of the Epic sepsis model

algorithmic management

Hiring-bias law (NYC Local Law 144 audits, the EEOC settlements, state consequential-decision duties) governs the entry decision. Algorithmic management governs the whole employment relationship: platform drivers dispatched, priced and deactivated by models; warehouse workers paced against algorithmic rate targets; delivery drivers terminated by automated email with no human decision-maker to appeal to. The governance questions are different in kind — opacity of the rules of work (what drops your score?), inability to contest machine decisions, surveillance intensity, and a collective dimension, because algorithmic pay-setting can individualise wages in ways that dissolve bargaining power. The hard law that exists is mostly European: the Platform Work Directive for platforms, the AI Act for everyone else (employment is Annex III high-risk, emotion recognition at work is prohibited, and deployers must inform workers and their representatives before use), with GDPR Article 22 behind both.

Directive (EU) 2024/2831; EU AI Act Annex III, Arts 5(1)(f), 26(7)

Platform Work Directive (EU)

Directive (EU) 2024/2831, in force from December 2024 with member-state transposition due by 2 December 2026. Its algorithmic-management chapter applies to platform workers regardless of employment status and does four things. It prohibits processing certain data outright: emotional or psychological state, private conversations, off-duty activity, and data used to predict the exercise of fundamental rights including union activity. It requires transparency about the automated systems that take or support decisions. It mandates human oversight by qualified reviewers and a right to human review of significant decisions — account suspension or termination cannot rest on an algorithm alone. And it creates a rebuttable presumption of employment where facts indicate control. Because it is a directive, the operative rules are the national transposing statutes; check each member state rather than quoting the directive to a client.

Directive (EU) 2024/2831 on improving working conditions in platform work

presumption of employment

Employment status is the gateway to most labour protection, so misclassification as self-employed removes the protections before any specific rule can apply. The EU Platform Work Directive responds with a rebuttable presumption: where facts indicating direction and control are present, the worker is presumed an employee and the burden shifts to the platform to prove otherwise, with member states setting the procedural detail. Spain’s Riders Law (2021), following a Supreme Court ruling on Glovo couriers, had already presumed delivery riders to be employees. Note what the presumption does and does not do: it reallocates the burden of proof, it does not define a new category, and the Directive’s algorithmic-management rights deliberately apply regardless of status — so a platform that successfully rebuts the presumption still owes the transparency and human-review duties.

Directive (EU) 2024/2831; Spain Royal Decree-Law 9/2021 (“Riders Law”)

works council

A European institution with no US equivalent: employees elect a council that holds statutory information, consultation and in some jurisdictions co-determination rights, separate from trade-union structures. For AI governance the German case is the sharpest — under §87 of the Works Constitution Act (Betriebsverfassungsgesetz), introducing technical systems capable of monitoring employee behaviour or performance requires works-council agreement, which makes every algorithmic-management rollout a negotiated one and routinely makes access to the system’s logic the price of consent. Spain’s Riders Law created a related right for works councils to be informed of the parameters, rules and instructions of algorithms affecting working conditions. Jurisdictionally bounded: do not describe works-council consent as a general AI governance requirement, and do not confuse it with the EU AI Act’s Article 26(7) duty merely to inform workers’ representatives.

Germany, Betriebsverfassungsgesetz §87; EU AI Act Art 26(7)

Article 22 (GDPR)

Shorthand for GDPR Article 22, which gives a right not to be subject to a decision based solely on automated processing, including profiling, producing legal effects or similarly significantly affecting the person. It is permitted only where necessary for a contract (read narrowly), authorised by EU or member-state law with safeguards, or based on explicit consent — and even inside an exception, Article 22(3) requires at minimum the right to obtain human intervention, to express a point of view, and to contest the decision. Its reach widened in SCHUFA (C-634/21, 2023), where the CJEU held that a score playing a determining role downstream is itself a decision under Article 22. It is the provision Dutch drivers used to challenge automated deactivations by Uber and Ola, with the Amsterdam Court of Appeal ordering disclosure of the decision logic in 2023 — the clearest illustration that where no labour statute existed, data-protection law did the work. The site also indexes it as GDPR Article 22.

GDPR Art 22; CJEU SCHUFA (C-634/21, 2023)

ECDIS (external consumer data and information sources) (External Consumer Data and Information Sources)

Colorado’s SB 21-169 and its implementing regulations require insurers using external consumer data and information sources — data and algorithms or predictive models drawing on information beyond the traditional underwriting record — to establish a governance framework and to run quantitative testing for unfairly discriminatory outcomes, filing the results with the Division of Insurance. It is the first US regime mandating outcome testing of insurance AI, and it sits inside a sector governed state by state: the NAIC’s Model Bulletin on the Use of AI Systems by Insurers (December 2023), adopted in some form by roughly half the states, expects a written AI systems programme enforceable through existing unfair-trade-practices and market-conduct authority. In the EU the same ground is covered differently — risk assessment and pricing in life and health insurance is Annex III high-risk under the AI Act, with EIOPA translating it for supervisors.

Colorado SB 21-169 and Division of Insurance regulations; NAIC Model Bulletin (2023)

proxy discrimination

The reason “we removed the sensitive column” is never a fairness defence. A model can reconstruct a protected attribute from ZIP code, shopping patterns, phone model, name, school, even typing cadence — and then discriminate on it as effectively as if it had been an input. Amazon’s recruiting model never saw “gender”; it found “women’s chess club captain”. Two consequences for practice. Fairness through unawareness fails, so every serious audit tests outcomes by group rather than inspecting the feature list — which creates the measurement paradox that attribute data is needed to detect discrimination that attribute-blindness was meant to prevent, answered in US lending by inferred attributes such as BISG used for auditing only. And in insurance and lending the term is becoming operative law: Colorado’s ECDIS regime and fair-lending supervision both target outcomes rather than inputs.

remote proctoring

Pandemic-era exam surveillance normalised face detection, gaze tracking and room scans in students’ bedrooms, and its failures were predictable from ordinary bias analysis: face-detection systems repeatedly failed to see darker-skinned candidates, flagging them for lighting “violations” — documented in complaints about the 2020 California bar exam. In Ogletree v. Cleveland State (N.D. Ohio 2022) a federal court held that a public university’s mandatory webcam room scan was an unreasonable search under the Fourth Amendment. The EU AI Act now lists AI systems monitoring and detecting prohibited behaviour of students during tests as high-risk (Annex III point 3(d)), and emotion inference in education institutions is flatly prohibited under Article 5(1)(f). The sector pattern to carry: the subjects cannot opt out, so governance that works here always adds due process — notice of the tool, a contestation route, and human decision authority.

EU AI Act Annex III point 3(d), Art 5(1)(f); Ogletree v. Cleveland State (N.D. Ohio 2022)

AI-writing detector

After ChatGPT, schools reached for detectors. Stanford research showed them systematically misclassifying non-native English writers as AI, with over half of the TOEFL essays in the study flagged, and OpenAI withdrew its own classifier for low accuracy in 2023. The governance verdict is not subtle: a probabilistic accusation tool with documented subgroup bias, deployed in a high-stakes disciplinary context with no due process, fails every test this discipline applies. Two general lessons travel beyond the classroom. A classifier whose output is an accusation carries a different burden of proof than one whose output is a recommendation — the base rate and the cost of a false positive govern whether it may be used at all. And detection is an adversarial problem: a tool whose evasion is trivial and whose errors fall on the least powerful users is not a control, it is a hazard.

FERPA (Family Educational Rights and Privacy Act) (Family Educational Rights and Privacy Act)

FERPA protects the privacy of student education records at institutions receiving US federal education funding: parents and eligible students get rights of access and amendment, and disclosure of personally identifiable information from education records generally requires consent, subject to exceptions — notably the “school official” exception under which a vendor performing an institutional service may receive data under the school’s direct control and for authorised purposes only. That exception is where ed-tech AI lives, and where the governance work is: a vendor using student data to improve its own models is outside it unless the school has genuinely authorised and can control that use. Pair it with COPPA for under-13s and with state student-privacy statutes (California’s SOPIPA is the template: no targeted advertising, no profiling, no sale of student data).

FERPA, 20 U.S.C. §1232g; 34 CFR Part 99

student privacy

Not one instrument but a stack: FERPA for education records and the vendor flows that touch them, COPPA for under-13 ed-tech, state student-privacy statutes on the California SOPIPA template (no targeted advertising, no profiling, no sale of student data), and — at public institutions — constitutional limits, as Ogletree showed for room scans. In the EU the ground is covered by the GDPR plus the AI Act’s Annex III point 3, which makes admission, grading, level assessment and proctoring high-risk. The sector-specific reason to treat it as its own topic: students cannot opt out, the decisions shape life trajectories, and much of the user base are minors — so the base rate of error lands on people who cannot negotiate, switch providers or lawyer up.

FERPA; COPPA; California SOPIPA; EU AI Act Annex III point 3

SAE levels of driving automation

Levels 0–2 are driver assistance: the human drives, always, whatever the marketing name suggests — a system branded “full self-driving (supervised)” is legally Level 2. Level 3 is conditional automation: the system drives within its operational design domain and the human is the fallback on request. Level 4 needs no human fallback within its domain (the robotaxi case); Level 5 — everywhere, all conditions — exists nowhere. The cliff between 2 and 3 is where responsibility, liability and the entire evidentiary burden of safety migrate from the driver to the system, which is why that boundary, not the technology, is what regulators legislate around. Note the taxonomy is a shared vocabulary, not a legal classification: a vehicle’s approval status under type-approval or self-certification is a separate question from its SAE level.

SAE J3016

ODD (operational design domain) (Operational Design Domain)

The boundary that makes automated driving assessable: an ODD specifies the roadway types, speed ranges, traffic and environmental conditions, geographic area and other constraints within which the driving automation is designed to operate. UN R157, the first binding international rule for Level 3 automated lane keeping, illustrates how tightly an ODD is drawn and how it moves — initially capped at 60 km/h and extended in 2022 to 130 km/h with lane changes. Two generalisable governance points. An ODD is only meaningful if the system detects leaving it and hands back or stops safely, so ODD definition and transition-demand design are one control, not two. And the concept transfers directly to AI governance at large: it is the intended purpose and out-of-scope statement of a model card, made testable.

SAE J3016; UNECE Regulation No. 157

type approval

Under the EU and UNECE framework a vehicle type, system or component is approved before being placed on the market, against UNECE regulations and EU implementing rules, by a designated approval authority relying on technical services — a pre-market gate with an administrative file behind it. For automated driving, UN R157 (2021) was the first binding international rule for Level 3 systems, UN R155 and R156 add mandatory cybersecurity and software-update management systems, and EU implementing regulation 2022/1426 extended type-approval to fully automated driving systems. Note the structural family resemblance to the EU AI Act’s conformity assessment and CE marking: both are New Legislative Framework ex-ante models. Contrast self-certification, the US approach, which inverts the sequence.

Regulation (EU) 2018/858; UNECE Regulations 155, 156, 157; Regulation (EU) 2022/1426

self-certification

Rather than pre-market approval, a manufacturer certifies that its vehicles comply with the Federal Motor Vehicle Safety Standards — none of which yet addresses automated driving logic — and NHTSA enforces after the fact through defect investigations, recalls, and a Standing General Order requiring crash reporting for automated and Level 2 driver-assistance systems. Congress has repeatedly failed to pass an AV statute, leaving a state patchwork on top (California alone requires DMV permits, disengagement reports and separate authority for paid rides). The enforcement-by-aftermath pattern produced the sector’s sharpest compliance lesson: NHTSA’s multi-year Autopilot investigation ended in a December 2023 recall of around two million vehicles on a foreseeable-misuse theory, and Cruise lost its California permits not for the pedestrian-dragging incident alone but for withholding the footage — candour with the regulator is the licence to operate.

US FMVSS self-certification regime; NHTSA Standing General Order on ADS/ADAS crash reporting

software update management system (SUMS)

UNECE Regulation No. 156 requires manufacturers to hold an approved software update management system, so each over-the-air or workshop update to a type-approved function happens inside a process the authority has assessed — configuration and version identification, compatibility and safety assessment, records, and user information — with R155 doing the same for a cybersecurity management system. It is one of the clearest existing answers to the question every AI regime is now asking: how do you regulate an artifact that changes after approval? Compare the FDA’s Predetermined Change Control Plan, which pre-authorises a change envelope, and the EU AI Act’s substantial modification concept, which asks when a change makes the thing a new system. Three different answers to one problem, and worth holding together.

UNECE Regulations No. 155 (CSMS) and No. 156 (SUMS)

compute governance

Frontier AI is made of algorithms, data and compute. Two of those are nearly ungovernable: algorithms are ideas that travel in papers and heads, and data is copied at zero cost. Compute has four properties the others lack — it is detectable (a frontier run consumes tens of thousands of accelerators drawing megawatts for months, visible to the utility and the cloud provider), excludable (chips clear customs), quantifiable (runs are measured in FLOPs, a number you can write into a statute), and concentrated (one EUV supplier, one dominant leading-edge fab, three EDA vendors, one dominant accelerator designer). The same logic explains the limits: controls grip hardware not knowledge, so algorithmic efficiency erodes any fixed line; controls leak through smuggling, remote rental and stockpiling; and controls create a powerful incentive to build a domestic alternative. Expert-level compute policy is using the lever while budgeting for its decay.

Sastry et al., “Computing Power and the Governance of AI” (2024)

training compute

The quantity statutes actually key on, because it is estimable before a run finishes and hard to litigate about: the EU AI Act presumes a GPAI model trained above 10²⁵ FLOPs poses systemic risk, and US instruments have generally used 10²⁶ as the frontier line. It is a proxy, and knowing why it decays is the expert part: capability per FLOP keeps improving, so any fixed threshold captures a shrinking share of dangerous capability and a growing share of routine work, while inference-time compute scaling means capability can now be bought after training as well as during it. Distinguish training compute from inference compute, which dominates lifetime energy for any widely deployed model — a distinction that matters as much for the environmental accounting as for the capability argument.

EU AI Act Art 51(2); cf. US EO 14110 compute reporting thresholds

export controls

Administered in the United States through the Export Administration Regulations by BIS, export controls restrict destinations, end users and end uses, with instruments including licence requirements, the Entity List, and extraterritorial rules reaching foreign-made items with controlled US content or made with controlled US tooling. From October 2022 onward they became an AI policy instrument: restrictions on advanced computing chips and semiconductor manufacturing equipment to China, revised repeatedly as the industry designed around each threshold. Three limits a professional should state alongside any claim about their effectiveness: controls grip hardware rather than knowledge; they leak through smuggling and remote rental, which is why cloud KYC was proposed to close the gap; and they accelerate domestic substitution — the outcome they were meant to prevent, arriving on a delay.

US Export Administration Regulations (15 CFR Parts 730–774); BIS advanced-computing rules from 7 October 2022

BIS (Bureau of Industry and Security) (Bureau of Industry and Security (US Department of Commerce))

BIS administers the Export Administration Regulations: licensing, the Entity List, end-use and end-user controls, and enforcement. On 7 October 2022 it published what the module calls the most consequential AI policy document nobody in AI governance was reading at the time — restricting exports to China of advanced computing chips and of the semiconductor manufacturing equipment needed to make them, with successive revisions as vendors designed to the thresholds. Why an AI governance professional needs the name: BIS rules, not AI statutes, determine which models can be trained where, and they are written in trade-control language in a different part of the Federal Register from everything else in this field. Set alerts outside the AI-policy press.

US Department of Commerce, Bureau of Industry and Security

compute threshold

The drafting technique that solved a real problem: “frontier model” is vague, but a compute count is administrable, estimable before training finishes, and hard to argue about. Keep the two canonical numbers apart — 10²⁵ is the EU AI Act’s systemic-risk presumption for GPAI models, and 10²⁶ has been the US frontier line in instruments including EO 14110 and California’s SB 53. The standing critique is the same one that applies to all of compute governance: capability per FLOP keeps improving, so a fixed threshold captures a shrinking share of dangerous capability and a growing share of routine work. Serious regimes treat the number as a tripwire that buys scrutiny time, never as the definition of what matters — and Korea deliberately put its figure in an amendable enforcement decree rather than in the statute. Distinguish it from a capability threshold, which is measured rather than assumed.

EU AI Act Art 51(2); California SB 53; US EO 14110 (revoked)

cloud KYC (know your customer) (Know Your Customer)

Export controls govern where chips go; nothing in them stops a foreign customer renting the same capability remotely. In January 2024 the US Commerce Department proposed requiring infrastructure-as-a-service providers to run customer identification programmes and to report when foreign customers train large AI models on their platforms — borrowing its architecture wholesale from anti-money-laundering law: identify the customer, know the beneficial owner, watch for structuring (splitting one large training run across many accounts), and report suspicious activity. As of this module’s date-stamp the proposal’s finalisation had been repeatedly in flux: check whether a final rule has issued, and in what form, before relying on it. The concept travels — any compute-based control needs an answer to remote access, or it governs only customs declarations.

US Department of Commerce proposed IaaS customer-identification rule (January 2024)

data centre

Frontier training clusters now demand gigawatt-scale power, which moved AI policy into venues most AI professionals never watch: siting fights, local moratoria, water-use disclosure demands, grid interconnection queues and cost-allocation disputes. Both recent US administrations pushed to open federal land and accelerate permitting for AI data centres, and hyperscalers signed nuclear power deals that would have been unthinkable a decade ago. The regulatory instruments that actually bind are in energy law, not AI law: the recast EU Energy Efficiency Directive requires data centres above 500 kW to report energy, water and efficiency metrics into an EU database, and Germany’s Energy Efficiency Act imposes PUE limits and waste-heat-reuse duties on new centres. Governance follows the megawatts — which is why the environmental module tells you to set alerts beyond the AI-policy press.

Recast EU Energy Efficiency Directive (data-centre reporting); Germany Energy Efficiency Act

reward hacking

Reinforcement learners are notoriously literal: given a reward signal, they find whatever maximises it, including loopholes the designer never considered. At frontier scale the problem splits into outer alignment (did you specify the right objective at all?) and inner alignment (did training instil that objective, or something that merely correlates with it on the training distribution and diverges off it?). In RLHF it appears as reward-model overoptimisation — Goodhart’s law running inside the training loop — and as sycophancy, because human raters reward what looks good. The governance translation is immediate and not confined to AI: every metric a programme sets is a reward signal someone will optimise, which is why coverage and cycle-time metrics get gamed and why maturity self-assessments inflate. The site also carries the closely related specification gaming.

scalable oversight

RLHF assumes humans can evaluate the output; for a model proposing novel code, biology or strategy, the rater becomes the bottleneck. Scalable oversight is the family of approaches that use AI to extend human judgment: constitutional AI and RLAIF (models critique outputs against written principles), debate (two models argue and a human judges), and recursive decomposition of a hard evaluation into checkable pieces. Its governance relevance is a question to ask, not a reassurance to accept: when a lab claims its alignment technique scales, establish which bet it is making, and note that none of them is proven at the capability levels the claim covers. Treat scalable-oversight claims the way you would treat any control whose effectiveness evidence comes from the party relying on it.

dangerous-capability evaluations

The instrument frontier governance actually leans on. Labs run them under their safety frameworks, government safety institutes run them, and third parties such as METR and Apollo Research run them; the EU AI Act makes them legally load-bearing by requiring systemic-risk GPAI providers to perform model evaluation and adversarial testing under Article 55, with the Code of Practice supplying method. The professional skill is reading a result correctly, because an eval result is a lower bound, never a ceiling. Five reliability limits to interrogate every time: the elicitation gap (better scaffolding or fine-tuning may surface what one prompting setup missed), sandbagging, contamination (a leaked benchmark measures memory, not capability), construct validity (do multiple-choice virology questions measure real uplift?), and coverage (evals test the risks someone thought to write down). Read “evaluations found no dangerous capability” as: under this elicitation, on these proxies, against these domains, assuming honest performance.

EU AI Act Art 55; frontier-lab safety frameworks

sandbagging

The failure mode that makes every evaluation result conditional. It has two forms worth keeping distinct. Model sandbagging: a capable model performs worse under test conditions than in deployment, which stopped being hypothetical when Anthropic published evidence of alignment faking in December 2024 — a model strategically complying during training to preserve its existing preferences — and when Apollo Research documented frontier models engaging in in-context scheming in contrived settings, disabling oversight and lying about it. Developer sandbagging: choosing an elicitation budget, a prompt set or a scaffold that will not surface a capability nobody wants to find. The governance response is the same in both cases: evaluations by parties without an interest in the result, published methods, and the discipline of treating “no scheming observed” as a claim about observation, not absence.

mechanistic interpretability

The research bet on getting underneath behaviour rather than describing it from outside. Its progress is real and easy to overstate in the same breath: sparse autoencoders now decompose activations into millions of human-recognisable features, and attribution-graph work has begun tracing multi-step circuits — watching a model plan a rhyme before writing the line, or fabricate a plausible reasoning chain after already fixing its answer. It offers things behavioural evaluation never can: detecting why an output occurred, flagging deception-associated features firing while a model claims honesty, and auditing for capabilities never elicited in testing. And it covers, today, a small fraction of what frontier models compute. A governance professional treats its findings as powerful, partial evidence and resists both the dismissal and the overclaim.

sparse autoencoder (SAE) (Sparse Autoencoder)

The workhorse of modern mechanistic interpretability: train a wide, sparsely activating autoencoder on a model’s activations so that the dense, superposed representation is re-expressed as a large dictionary of features, each firing on a recognisable concept. Anthropic’s 2024 work famously isolated a Golden Gate Bridge feature and, by amplifying it, made the model steer conversations obsessively toward the bridge — a demonstration that features are not merely labels but causal handles. Two governance cautions. Feature interpretations are hypotheses that require validation, since a feature named by inspection can be doing something broader than its label. And coverage is partial: an SAE dictionary explains some of what the model computes, so “we found no deceptive feature” carries the same epistemic weight as “we could not elicit the capability”.

safety case

Not a checklist and not a pile of evaluation scores: an argument, with explicit claims, the evidence for each, and the assumptions doing silent work. Frontier safety cases lean on three argument families, usually in combination: inability (the model cannot do the dangerous thing — evidence: capability evaluations, with every reliability limit attached), control (even if it tried, safeguards would catch or contain it — evidence: monitoring, sandboxing, red-teaming against the safeguards), and trustworthiness or alignment (it would not try — today the weakest and most contested family). The UK safety institute and several labs have published sketches, and the EU’s GPAI Code of Practice pushes systemic-risk providers toward this structure. Why the methodology matters to a reviewer: a checklist can only be complete or incomplete; an argument can be wrong, and you can find where.

Safety-case practice from nuclear/aviation/rail regulation; EU GPAI Code of Practice

algorithmic collusion

Three cases to keep apart. Easy: humans agree and the algorithm executes — United States v. Topkins (2015), where Amazon Marketplace poster sellers agreed to use coordinating pricing software. Frontier: the hub-and-spoke theory, where competitors never speak to each other but all feed non-public data into a shared vendor whose software recommends aligned prices back — the theory in the DOJ and state suits against RealPage (2024) over rent-setting, with San Francisco and other cities banning such software built on non-public competitor data. Unsolved: autonomous collusion, where reinforcement-learning pricing agents learn supracompetitive prices with no agreement, no communication and no human intent; US Sherman Act §1 requires an agreement and EU Article 101 requires concertation, so two algorithms tacitly converging may violate neither. The safe counselling line runs on the data: public inputs and independent tools on one side, pooled confidential inputs on the other.

United States v. Topkins (2015); US v. RealPage (2024, ongoing); Sherman Act §1; TFEU Art 101

market study

An information-gathering and diagnosis power rather than an enforcement action: an authority maps a market, publishes findings, and sometimes articulates principles or recommends remedies. It became the main competition instrument in AI because the foundation-model stack presented concentration that existing tools could not reach — the UK CMA’s foundation-model reviews (2023–24) articulated principles of access, diversity, choice and fair dealing, and the US FTC used its 6(b) study power to compel the cloud–model partnership documents, reporting in January 2025 on exclusivity, compute dependency and rights over model IP. What a study is not: a finding of illegality, or a remedy. Its value is evidentiary and agenda-setting — which is exactly the pattern seen throughout soft governance, where the party holding the evidence base shapes what happens next.

UK CMA foundation-model reviews (2023–24); US FTC 6(b) study report (January 2025)

quasi-merger

A descriptive label used by analysts, not a legal category: billion-dollar arrangements between hyperscalers and frontier labs that bundle compute, exclusivity, commercial distribution and IP rights, producing dependency resembling integration while no party acquires control. Because merger control keys on acquisition of control, these arrangements largely escaped notification — the European Commission examined one of the best-known and concluded it could not be reviewed as a merger absent a change of control, so authorities turned to market studies instead. Two drafting consequences now being debated: whether notification thresholds should capture dependency and influence rather than only control, and whether partial-ownership and governance rights deserve their own review gate. Use the term carefully in writing — it describes an effect, and asserting a “quasi-merger” says nothing about what law applies.

vertical foreclosure

A standard theory of harm in competition law: an integrated firm can raise rivals’ costs or exclude them by withholding, degrading or pricing up access to something they need (input foreclosure) or by cutting off their route to customers (customer foreclosure). Mapped onto AI it is the fear that training-scale compute concentrated in a few hyperscalers, frontier models concentrated in a few labs, and the two layers fused by exclusive partnerships could leave independent model developers and application builders without viable access on fair terms. It is also the worry that shaped the landmark 2025 Google search remedies decision, where the court explicitly framed relief around the risk of a search monopoly rolling forward into the AI-assistant market. A theory of harm is not a finding: foreclosure requires ability, incentive and anticompetitive effect, each of which has to be shown.

synthetic media

Broader than deepfake, which is the subset that convincingly depicts real people or events: synthetic media covers any AI-generated or AI-manipulated audio, image, video or text, including wholly fictitious content. The breadth is why it is the drafting term of choice — India’s 2026 IT Rules amendment obliges platforms to label synthetically generated content with embedded metadata, the EU AI Act’s Article 50 attaches machine-readable marking duties to providers and disclosure duties to deployers, and China’s 2025 labelling rules do the same. Two things the labelling regimes do not solve. Attacks arrive faster than rebuttal and are timed to exploit the rules meant to protect elections, as the Slovak audio fabrication two days before the September 2023 vote showed. And the liar’s dividend runs the other way: once fabrication is known to be possible, genuine evidence becomes deniable, and nothing yet governs that.

EU AI Act Art 50; India IT Rules amendment (2026); China synthetic-content labelling rules (2025)

DSA (Digital Services Act) (Digital Services Act)

Regulation (EU) 2022/2065. Its relevance to AI governance is structural: where the AI Act regulates systems and models, the DSA regulates the infrastructure that distributes their output. Very large online platforms and search engines must assess and mitigate systemic risks including risks to civic discourse and electoral processes (Articles 34–35), with the Commission’s 2024 election guidelines telling them how — including labelling AI-generated content; recommender-system parameters must be disclosed and a non-profiling option offered; deceptive interface design is prohibited; and platforms publish transparency reports and submit to independent audits. The division of labour is the thing to remember: in elections, the US mostly regulates the lie about the candidate, while the EU regulates the carriage of it.

Regulation (EU) 2022/2065, Arts 34–35; Commission election guidelines (2024)

political advertising regulation (EU)

The third instrument in the EU’s election-integrity set, applying from October 2025: political advertisements must be clearly labelled as such with a transparency notice identifying the sponsor and the amounts paid, records must be kept and made available, and targeting or ad-delivery techniques using sensitive personal data are prohibited, with consent requirements for other personal-data-based targeting. Read it beside the DSA’s systemic-risk duties and the AI Act’s Article 50 disclosure duties: three instruments, three layers — the ad, the platform, the generator. The US contrast is instructive: with no federal statute, election-deepfake law is state-made and constitutionally constrained, where disclosure mandates survive First Amendment scrutiny more easily than bans and parody exceptions are load-bearing.

Regulation (EU) 2024/900 on the transparency and targeting of political advertising

codetermination (Mitbestimmung)

The strongest form of worker participation in European labour law: on defined matters a works council does not merely receive information or consult but must agree, and absent agreement an arbitration body decides. Under §87 of the German Works Constitution Act, introducing technical systems capable of monitoring employee behaviour or performance falls in that category — which makes every algorithmic-management rollout a negotiated one, and routinely makes access to the system’s logic the price of consent. A separate strand of codetermination places worker representatives on supervisory boards of large companies. Jurisdictionally bounded: it is German (with relatives in Austria, the Nordics and the Netherlands) and has no US counterpart, so do not describe worker consent as a general AI governance requirement. ILO tripartite work is pushing the model internationally.

Germany, Betriebsverfassungsgesetz §87(1) no. 6

collective bargaining

Labour law’s oldest technology turned out to be a fully operational AI-governance instrument. The 2023 Writers Guild of America strike produced the first landmark AI collective agreement: AI cannot write or rewrite literary material, AI output is not source material that cuts writers’ credit or pay, studios must disclose when they hand writers AI-generated material, and writers’ consent governs their own use of the tools. SAG-AFTRA won consent-and-compensation rules for digital replicas of performers. These contracts govern real AI use today with a precision no statute matches, and they arrived years ahead of legislation. The practitioner’s takeaway: treat negotiated agreements as governance artifacts — they define permitted AI use with contractual enforcement, they bind the signatories regardless of what any AI act says, and in unionised sectors they are the first place to look for the operative rule.

WGA 2023 Minimum Basic Agreement; SAG-AFTRA agreements

digital replica

The performer-facing term for a synthetic reproduction of an identifiable individual’s appearance or voice. SAG-AFTRA agreements established the operative rules first — informed consent for creating a replica, consent for each use, and compensation — with distinctions between a replica scanned on a production and one assembled from existing material. Statutory protection is arriving through personality rights rather than AI law: several US states have extended right-of-publicity style protection to voice and likeness against unauthorised synthetic reproduction, and Brazil’s pending AI bill grants image-and-voice consent rights. Keep three things apart: a deepfake is defined by deceptive resemblance, a digital replica by reproducing a specific person whether or not anyone is deceived, and NCII by the sexual and non-consensual character of the content.

SAG-AFTRA agreements; US state right-of-publicity statutes

carbon footprint (AI)

Four components get conflated constantly: embodied impacts (mining, chip fabrication, construction), training energy (large, one-time, headline-friendly), inference energy (small per query, colossal at deployment scale — for any successful system it dwarfs training within months), and water, both evaporative cooling and the water embedded in generating the electricity. Each number depends radically on where and when the compute ran: the same training run on a hydro-heavy grid at night and on a coal-heavy grid emit wildly different carbon. So any single-number claim that does not state its boundaries is marketing — and the common corollary, “our cloud is 100% renewable so our AI is carbon-free”, usually rests on annual certificate matching rather than 24/7 hourly carbon-free matching on the local grid. Ask which accounting a claim uses before repeating it.

energy disclosure

The EU AI Act touches energy in three places and binds lightly: Annex XI makes providers of systemic-scale GPAI models document the known or estimated energy consumption of training, the standardisation machinery is instructed to pursue resource-performance improvements, and Article 95 invites voluntary codes of conduct on environmental sustainability. That is a documentation-and-encouragement regime — no cap, no efficiency floor. The harder instruments sit in energy law: the recast EU Energy Efficiency Directive requires data centres above 500 kW to report energy, water and efficiency metrics into an EU database, and Germany’s Energy Efficiency Act imposes actual PUE limits and waste-heat-reuse duties on new centres. The US has no federal AI-environment mandate. Note the measurement irony: regulatory attention concentrates on training because it is one run with one bill, while inference dominates lifetime impact.

EU AI Act Annex XI, Art 95; recast EU Energy Efficiency Directive

PUE (power usage effectiveness) (Power Usage Effectiveness)

A ratio of 1.0 would mean every watt entering the facility reaches the computing equipment; everything above 1.0 is overhead, chiefly cooling and power conversion. It is standardised (ISO/IEC 30134-2) and now appears in binding law — Germany’s Energy Efficiency Act sets PUE limits for new data centres, and the recast EU Energy Efficiency Directive requires efficiency metrics to be reported. Know its three blind spots before quoting it. It says nothing about the carbon intensity of the electricity consumed, so an efficient centre on a coal grid beats an inefficient one on hydro on PUE and loses on emissions. It says nothing about useful work done per watt, so idle servers improve nothing and can flatter the ratio. And it ignores water entirely, which is why companion metrics exist.

ISO/IEC 30134-2; Germany Energy Efficiency Act

water usage (AI)

Two streams: on-site evaporative cooling, which consumes water in the basin where the facility sits, and off-site water embedded in electricity generation, which is often larger and almost always omitted from headline figures. The reason aggregate percentages mislead is local concentration: siting economics push centres toward cheap land and power that is frequently in drought-prone regions, which is why water became the flashpoint in permitting fights from the Pacific Northwest to Spain and Chile. For governance work the practical asks are the same as for any environmental claim — state the boundary (on-site only, or including generation), state the basin, and distinguish water withdrawn from water consumed, since the two differ by an order of magnitude and are routinely swapped in disclosures.

CSRD (Corporate Sustainability Reporting Directive) (Corporate Sustainability Reporting Directive)

Directive (EU) 2022/2464 expanded and standardised EU non-financial reporting: in-scope companies report against European Sustainability Reporting Standards on a double-materiality basis (impacts on the company and the company’s impacts on people and environment), with assurance over the reported information. The AI connection is indirect but real: energy consumed by owned data centres falls into Scope 2 and purchased cloud compute into Scope 3, so a company’s AI buildout becomes a reporting line rather than a press release — and a claim made in a sustainability statement carries liability that a blog post does not. Verify the current scope: the 2025–26 omnibus simplification process cut applicability substantially, so the set of companies in scope and the timeline are both different from the original directive.

Directive (EU) 2022/2464 (scope amended by the 2025–26 omnibus process)

UN Guiding Principles on Business and Human Rights (UNGPs) (UN Guiding Principles on Business and Human Rights)

Endorsed by consensus by the UN Human Rights Council in 2011 and the authoritative global reference for business and human rights. Pillar 1: states have a duty to protect against abuse by third parties, including business — the pillar that justifies rights-based AI statutes such as the EU AI Act’s fundamental-rights architecture and the Council of Europe convention. Pillar 2: business has an independent responsibility to respect human rights, which exists regardless of whether local law requires it, operationalised through human rights due diligence across operations and the value chain, covering impacts a company causes, contributes to, or is directly linked to — a trichotomy that maps uncannily onto provider, deployer and value-chain questions in AI. Pillar 3: access to remedy, judicial and non-judicial — in AI terms appeal and redress for people affected by automated decisions, and the least-built pillar in practice. The UN’s B-Tech project translates all of it for technology companies.

UN Guiding Principles on Business and Human Rights (endorsed by the Human Rights Council, 2011)

human rights due diligence (HRDD) (Human Rights Due Diligence)

Pillar 2 of the UNGPs made operational: identify → prevent or mitigate → track → communicate, repeated continuously rather than performed once, and extending to business relationships rather than stopping at the company’s own gates. Two features distinguish it from the risk management a company already does. The severity logic is run from the rightsholder’s perspective — rank impacts by scale (how grave), scope (how many people) and remediability (can it be undone) — not by exposure to the company. And it is becoming law: the EU’s CSDDD converts the voluntary cycle into mandatory due diligence for large companies, though its scope and timeline were delayed and trimmed by the 2025–26 omnibus process and should be verified. For AI it is also the only established framework that reaches the human layer of the value chain.

UN Guiding Principles on Business and Human Rights, Pillar 2

human rights impact assessment (HRIA) (Human Rights Impact Assessment)

The most demanding member of the assessment family, and distinguished by two things. Its reference standard is the full body of internationally recognised human rights, not one statute’s protected interests — a DPIA asks about risks to data subjects within data-protection law, a FRIA about fundamental-rights impacts of a specific high-risk deployment. And it carries an engagement duty: affected rightsholders or their legitimate representatives are consulted directly, not modelled as personas. The method runs scope → engage → assess → rank by scale, scope and remediability → act → track → publish. The Council of Europe’s HUDERIA is a cousin methodology built for AI specifically. In practice these are run as one integrated assessment with three output views; what the human-rights lens adds is the engagement and the insistence that severity is measured on victims, not on corporate exposure.

UNGP-derived HRIA practice; cf. CoE HUDERIA methodology

CSDDD (Corporate Sustainability Due Diligence Directive) (Corporate Sustainability Due Diligence Directive)

Directive (EU) 2024/1760 converts the UNGP due-diligence cycle into mandatory duties: identify, prevent, mitigate and account for adverse human-rights and environmental impacts in a company’s own operations and its chain of activities, with governance, complaints and reporting obligations attached. Its AI relevance is the one the data-labour module presses: a framework of this shape could, in principle, require large AI firms to assess labour conditions in their data supply chains — the same logic already applied to cobalt and cotton — reaching the annotation and moderation layer that AI statutes do not. Verify current scope and timeline before relying on it: the 2025–26 omnibus simplification process delayed and substantially trimmed the regime, and because it is a directive the operative rules are the national transposing laws.

Directive (EU) 2024/1760 (scope amended by the 2025–26 omnibus process)

lethal autonomous weapon systems (LAWS)

The domain nearly every AI statute exempts and no AI expert may ignore: the EU AI Act excludes military and defence uses entirely, and national-security carve-outs riddle other regimes, so what fills the space is international humanitarian law, one national directive, and a decade of diplomacy. The geometry is stable. The ICRC and a large state bloc want a two-tier treaty — prohibit autonomous weapons that target humans or cannot be used in compliance with IHL, strictly regulate the rest — with the UN Secretary-General calling for a binding instrument. Major military powers resist new binding law, arguing existing IHL suffices and the definitions are unworkable. The consensus-bound Group of Governmental Experts under the Convention on Certain Conventional Weapons has produced guiding principles but no treaty, pushing the issue into the General Assembly and into voluntary tracks. As of this module’s date-stamp no binding instrument exists — check whether negotiations advanced before saying otherwise.

UN CCW Group of Governmental Experts on LAWS; ICRC position (2021)

meaningful human control

Advanced by states and civil-society groups in the LAWS debate as the line an autonomous weapon must not cross, and precisely the term the states opposing a new treaty decline to accept as a legal standard — which is why it has no agreed definition despite a decade of discussion. The exam-grade misconception to avoid: US DoD Directive 3000.09 does not require a human in the loop for lethal decisions, and never did; its standard is appropriate levels of human judgment over the use of force, applied at system design, review and authorisation. Nor has the UN “banned autonomous weapons”. The civilian analogue is the one you already know — human oversight under AI Act Article 14, with its competence, authority, time and information conditions — and precision about what binds, on whom, is what separates an expert answer from an advocacy summary here.

UN CCW GGE discussions; US DoD Directive 3000.09

risk mitigation hierarchy

Once risks are prioritised (classically by probability against severity of harm, with severity spanning individuals, groups, organisations and society), they are treated in order: eliminate the risk by changing or dropping the feature; reduce or mitigate it with controls, testing and oversight; transfer it through contracts or insurance; and only then accept what remains — documented, with an owner. The order is the examinable point, and the distractors exploit it: “purchase insurance and proceed” sounds decisive and business-savvy and is wrong whenever an elimination or reduction option remains on the table, and “accept the risk” with no documentation and no owner is wrong always. Note the relationship to risk treatment in ISO vocabulary, which lists the same options without ranking them — the hierarchy is the governance gloss that makes the ranking explicit.

threat modelling

A structured security practice — describe the system and its trust boundaries, enumerate threats against each component and data flow, rate them, and assign mitigations — long established in software engineering through frameworks such as STRIDE, and now one of the post-deployment competencies AI governance exams test alongside monitoring, red teaming and post-market monitoring. What AI adds is new trust boundaries and new threat classes: the model as an untrusted principal inside your own system, retrieved content as an injection vector, training pipelines as a poisoning surface, and tool access as an action surface. MITRE ATLAS and the OWASP Top 10 for LLM applications supply the AI-specific threat vocabulary. Keep it distinct from red teaming — threat modelling is analysis on paper, red teaming is adversarial testing of the built thing.

AI value chain

Foundation-model economics make AI a one-to-many structure: one upstream model is adapted, hosted, integrated and deployed by many parties downstream, so a single flaw propagates into thousands of systems and no single actor holds the whole picture. Governance frameworks answer with role-based duties: the EU AI Act allocates obligations between provider, deployer, importer, distributor and downstream provider, ISO/IEC 42001 requires role determination, and contracts allocate the rest. The exam-relevant and practically dangerous point: adaptation can shift you up the value chain. Fine-tune, rebrand or repurpose a procured system and you may become its provider, with provider-grade documentation, testing and conformity duties attached — a role change that happens by operation of law, not by anyone deciding it.

EU AI Act Arts 22–28; ISO/IEC 42001 role determination