AI Governance Academy glossary

365 terms with primary sources.

AI system

The legally load-bearing definition of AI. The EU AI Act Art 3(1) and the 2023 revised OECD definition both center on inference: the system infers, from the inputs it receives, how to generate outputs — predictions, content, recommendations, or decisions — that can influence physical or virtual environments, with varying levels of autonomy and possible adaptiveness after deployment. Systems whose outputs follow only fixed, human-specified rules generally fall outside it.

EU AI Act Art 3(1); OECD (2023)

machine learning

A family of techniques where a model’s behaviour is learned from data rather than explicitly programmed. Developers choose the architecture, the data, and the objective; the resulting decision logic emerges from training. That is why ML systems are statistical and probabilistic — and why governing them differs fundamentally from governing conventional software.

deep learning

Machine learning built on neural networks with many layers, whose millions-to-trillions of numeric parameters are tuned during training. Scale is the story: more data plus more compute has reliably produced more capability. It is also where the black box problem bites hardest — no individual parameter means anything a human can read.

generative AI

AI systems that generate content. Most run on foundation models trained on vast corpora. Generative AI reshaped governance: it created new risk categories (synthetic media, hallucination, training-data copyright) and new legal categories (the EU’s general-purpose AI rules, content-labeling laws in China and California).

foundation model

A model trained on broad data at scale, adaptable to a wide range of tasks — the base layer beneath most modern AI products. Regulators map roughly the same idea with different terms: the EU regulates general-purpose AI models; US frameworks often say dual-use foundation models. The one-to-many structure is what makes value-chain governance hard: one upstream model’s flaws propagate into thousands of downstream systems.

large language model (LLM)

A foundation model trained on enormous text corpora to predict the next token. Everything an LLM does — answering, summarising, coding — emerges from that single objective plus post-training alignment. This is why LLMs produce fluent, plausible text that is not guaranteed to be true: they model language, not facts.

general-purpose AI (GPAI)

The EU AI Act’s term for AI models displaying significant generality — capable of competently performing a wide range of distinct tasks, however placed on the market. GPAI models carry their own obligation track (Arts 51–56), separate from the risk tiers that apply to AI systems. Models trained above 10²⁵ FLOPs are presumed to pose systemic risk and carry extra duties.

EU AI Act Art 3(63), Arts 51–56

agentic AI

AI systems that don’t just answer but act: they decompose goals into steps, call tools and APIs, hold memory across steps, and operate with meaningful autonomy. Agents stress every classical governance control — oversight designed for single decisions must now cover chains of actions, delegation between agents, and real-world side effects.

training

The phase where a model’s parameters are optimised against training data. Contrast inference: running the finished model on new inputs. The split matters legally — data-protection and copyright questions concentrate at training time, while transparency, oversight, and incident duties mostly attach at inference/deployment time.

inference

Executing a trained model on fresh inputs to generate outputs. In governance texts the word does double duty: engineers mean the runtime phase, while the EU AI Act’s definition of an AI system uses infers to describe how AI derives outputs from inputs at all — the feature that separates AI from fixed rule-following software.

supervised learning

The workhorse ML paradigm: the model learns from labeled examples (this loan defaulted; this image is a stop sign). Most scoring and screening systems that worry regulators — credit, hiring, recidivism — are supervised learners, which is why label quality and historical bias in the labels are core governance concerns.

unsupervised learning

Learning without labels: the model finds structure on its own — customer segments, fraud anomalies, topic clusters. Governance concern: the discovered groupings can silently encode sensitive attributes (a “neighborhood cluster” that is really a proxy for race).

reinforcement learning

A paradigm where an agent learns by acting and receiving rewards. Powerful and notoriously literal: agents exploit loopholes in the reward (reward hacking), which is a compact preview of the alignment problem — systems optimising the metric you wrote, not the outcome you meant.

RLHF (Reinforcement Learning from Human Feedback)

The post-training step that turned raw next-token predictors into usable assistants: human raters rank outputs, a reward model learns those preferences, and the LLM is tuned against it. Governance-relevant limits: it optimises for appearing helpful and harmless to raters, which can produce sycophancy and confident-sounding error.

fine-tuning

Taking a pre-trained model and continuing training on a narrower dataset to specialise it (a general LLM → a medical triage assistant). Legally live: fine-tune someone else’s model substantially enough and you may become the provider of a new model or system, inheriting the obligations that come with the title.

hallucination

When a generative model outputs plausible but false content — invented citations, fake case law, wrong medical claims. It is intrinsic to how LLMs work (they model plausible language, not verified fact), so governance treats it as a risk to be managed with retrieval grounding, verification layers, disclosure, and use-case limits — not a bug awaiting a patch.

black box

Shorthand for opacity: a deep model’s behaviour lives in millions of numeric parameters no human can read as reasons. The governance response is layered — explainability techniques approximate why, documentation records how it was built, and process rights (like contesting a decision) protect people even when full mechanistic transparency is impossible.

explainability

Techniques and practices for rendering model behaviour intelligible — from feature-attribution methods (SHAP, LIME) to counterfactuals (“you would have been approved if…”). Distinct from transparency (disclosure about the system) and interpretability (models readable by design). Legal hooks include GDPR Art 22 safeguards and the EU AI Act’s Art 86 right to explanation.

transparency

The most overloaded word in AI governance — a classic false friend across frameworks.

autonomy spectrum

AI systems occupy a spectrum: decision support → human-in-the-loop → supervised autonomy → full autonomy. Governance intensity should track position on the spectrum — which is why “varying levels of autonomy” sits inside the legal definition of an AI system, and why agentic AI is pushing systems rightward faster than oversight design is adapting.

human oversight

The principle that natural persons must be able to effectively oversee AI in use — understand its limits, watch for anomalies, resist automation bias, and intervene, override, or halt it. EU AI Act Art 14 makes it a hard legal requirement for high-risk systems; every major framework carries a version of it.

EU AI Act Art 14

automation bias

The documented tendency of humans to defer to automated suggestions — approving them faster and questioning them less than their accuracy warrants. It is the failure mode that turns “human in the loop” into a rubber stamp, which is why the EU AI Act names it explicitly in Art 14 and why oversight design (friction, sampling, accountability) matters more than oversight presence.

open-weight model

A model released with its weights downloadable, versus one accessible only through a controlled API. Release strategy is a governance decision: open weights democratize access and scrutiny but cannot be recalled, and safety fine-tuning can be stripped. Laws split on it — the EU grants open-source GPAI a partial documentation exemption that vanishes at systemic-risk scale.

distribution shift

Models learn the statistics of their training data; when the world changes — new slang, new fraud patterns, a pandemic — accuracy decays without any code change or error message. This is why one-time pre-deployment testing is never enough and every serious framework demands post-deployment monitoring.

model drift

The operational symptom of distribution shift: a deployed model’s real-world performance decays over time. Governance answer: monitoring with drift metrics, retraining triggers, and lifecycle documentation — the reason frameworks treat deployment as the start of obligations, not the end.

provider

The EU AI Act’s primary duty-holder: the actor who develops an AI system or GPAI model (or has one developed) and places it on the market or puts it into service under their own name or trademark. Providers carry the heaviest obligations. Beware role-switching: rebrand, substantially modify, or re-purpose someone else’s system and you may become its provider.

EU AI Act Art 3(3)

deployer

The organisation using an AI system under its authority (not the end customer affected by it). Deployers carry use-side duties: following instructions for use, assigning competent human oversight, monitoring, log retention, and — for some high-risk uses — fundamental-rights impact assessments. Most companies adopting AI are deployers long before they are providers.

EU AI Act Art 3(4)

deepfake

Synthetic audio, image, or video that appreciably resembles real persons, places, or events and would falsely appear authentic. A named legal category: the EU AI Act defines it (Art 3(60)) and attaches disclosure duties (Art 50(4)); election-deepfake and intimate-image laws are proliferating worldwide.

EU AI Act Art 3(60)

proxy variable

A feature correlated with a protected characteristic strongly enough that a model can discriminate without ever seeing the protected attribute itself (ZIP code → race; shopping patterns → pregnancy). The reason “we removed the sensitive column” is never a sufficient fairness defense, and a core concept behind disparate-impact analysis.

alignment

The research problem of getting AI systems to do what their principals actually intend — not just what the training objective literally rewards. Spans practical work (RLHF, constitutional methods) and frontier-safety questions (deception, scheming, scalable oversight). Governance interfaces with it through capability evaluations and frontier safety frameworks.

systemic risk (GPAI)

The EU AI Act’s label for GPAI models with high-impact capabilities whose failures could ripple across the Union — presumed when training compute exceeds 10²⁵ floating-point operations, or designated by the Commission on Annex XIII criteria. Triggers the heaviest model-level duties: state-of-the-art evaluations, adversarial testing, incident reporting, and cybersecurity for the weights themselves.

EU AI Act Arts 51–55

disclosure

The lightest-touch governance tool: informing people that they are interacting with an AI system, being analysed by one, or viewing synthetic content. It anchors the EU AI Act’s limited-risk tier (Art 50), China’s labeling rules, and a growing family of US state laws.

high-risk AI system

The tier that carries the AI Act’s full rulebook (Arts 8–15: risk management, data governance, documentation, logging, transparency, human oversight, accuracy/robustness/cybersecurity). Two independent routes lead in: Art 6(1) — the AI is a product or safety component under Annex I Union product law requiring third-party conformity assessment — and Art 6(2) — the use case appears in Annex III (biometrics, critical infrastructure, education, employment, essential services, law enforcement, migration, justice). Following the Digital Omnibus amendments, Annex III obligations apply from 2 December 2027 and Annex I from 2 August 2028. High-risk does not mean forbidden; it means regulated market access.

EU AI Act Art 6, Annexes I & III

Annex III

The Act’s catalogue of contexts where AI touches people’s life chances: (1) biometrics, (2) critical infrastructure, (3) education, (4) employment and worker management, (5) essential private and public services including creditworthiness and life/health insurance pricing, (6) law enforcement, (7) migration, asylum and border control, (8) administration of justice and democratic processes. Listing works at sub-point granularity — CV-ranking is in, spell-checking a CV is not — and several sub-points carry carve-outs (e.g. fraud detection within creditworthiness). The Commission can amend the list under Art 7, which is why classification is a living exercise, not a one-time check.

EU AI Act Art 6(2), Annex III

Article 6(3) filter

An Annex III match is a presumption, not a verdict. Art 6(3) lets a provider conclude the system is not high-risk where it does not pose a significant risk to health, safety or fundamental rights because it (a) performs a narrow procedural task, (b) improves the result of a previously completed human activity, (c) detects decision patterns or deviations without replacing human assessment, or (d) performs a preparatory task. Two tripwires: profiling of natural persons always stays high-risk, and the self-assessment must be documented before market placement and registered (Art 49(2)) — authorities can demand it and reclassify. The filter is where most classification disputes will live.

EU AI Act Art 6(3)–6(4), Art 49(2)

fundamental rights impact assessment (FRIA)

A deployer-side duty, not a provider one. Required of public-law bodies, private entities providing public services, and deployers of creditworthiness and life/health-insurance systems (Annex III 5(b)–(c)) before first use of a high-risk AI system. Contents: the deployment process, period and frequency of use, categories of affected persons, specific risks of harm, human-oversight measures, and mitigation/governance/complaint arrangements. It must be updated when circumstances change and notified to the market surveillance authority; the AI Office provides a template. Where a DPIA already covers ground, the FRIA complements rather than duplicates it — the lens is fundamental rights of affected people, not data-protection compliance alone.

EU AI Act Art 27

conformity assessment

The gate between development and the EU market. Art 43 sets the routes: most Annex III systems (points 2–8) use internal control (Annex VI — the provider self-verifies against Arts 8–15); Annex III point 1 biometrics may self-assess only if harmonized standards were applied in full, otherwise a notified body must examine the quality system and technical documentation (Annex VII); Annex I products fold the AI assessment into their existing sectoral procedure. A substantial modification triggers re-assessment. Passing yields the EU declaration of conformity and the right to affix CE marking.

EU AI Act Art 43, Annexes VI–VII

CE marking

The end point of the New Legislative Framework chain: essential requirements → standards → conformity assessment → CE mark. For high-risk AI the mark must be affixed visibly, legibly and indelibly (digitally where the system is digital-only), accompanied where relevant by the identification number of the notified body involved. It signals to every market-surveillance authority in the Union that the provider claims — and can evidence — conformity. Affixing it without basis is a formal non-compliance under Art 83 and a fast route to enforcement.

EU AI Act Art 48

notified body

The third-party checker in EU product law, now extended to AI. Notified bodies must prove independence, competence, confidentiality and insurance (Arts 28–39), are designated by national notifying authorities, and are listed in the Commission’s NANDO database. Under the AI Act their main workload is Annex III point 1 biometric systems assessed under Annex VII, plus AI embedded in Annex I products where sectoral law already demands third-party assessment. They issue certificates with validity ceilings (5 years for Annex I, 4 years for Annex III, Art 44) and can suspend or withdraw them — a private-sector actor wielding quasi-regulatory power.

EU AI Act Arts 28–39, 44

harmonized standard

The translation layer between legal text and engineering practice. The Commission issued standardisation request C(2023)3215 to CEN/CENELEC JTC 21 covering ten areas mapped to the AI Act’s requirements (risk management, data governance, logging, transparency, oversight, accuracy, robustness, cybersecurity, quality management, conformity assessment). Once a standard is cited in the OJEU, applying it triggers the presumption of conformity. As of September 2026 no AI Act harmonized standards have been cited — prEN 18286 (quality management, Art 17) was the first deliverable to reach public enquiry — so providers must currently self-assess against the Act directly; check current status before relying on any standard.

EU AI Act Art 40

presumption of conformity

The engine that makes voluntary standards commercially irresistible. A provider who applies a harmonized standard cited in the OJEU is presumed compliant with the essential requirements that standard covers — shifting the burden to any authority that disagrees. Art 42 adds specific presumptions (e.g. training data matching the deployment context for Art 10; Cyber Resilience Act certification for Art 15 cybersecurity). The presumption is rebuttable and only ever covers what the standard covers. Because no AI Act standards are yet cited (as of September 2026), the presumption is not yet activatable — a fact that reshaped the compliance timetable debate.

EU AI Act Arts 40–42

common specifications

Plan B in the New Legislative Framework. Where harmonized standards are absent, insufficient, or blocked, the Commission may adopt common specifications by implementing act (Art 41); applying them grants the same presumption-of-conformity effect. They matter because JTC 21’s deliverables missed the original April 2025 deadline — if standards are still missing as high-risk obligations bite (December 2027), common specifications are the mechanism regulators would reach for. Providers may deviate from them only by proving an equivalent technical solution.

EU AI Act Art 41

market surveillance

The enforcement arm of EU product law (Regulation 2019/1020), applied to AI by Art 74. Each Member State designates a market surveillance authority with powers to demand technical documentation and data, access training/testing datasets, and — under conditions — source code (Art 74(13)). Escalation runs from national measures (Art 79) through the Union safeguard procedure (Art 81) to actions against systems that are compliant yet still risky (Art 82). Fundamental-rights bodies get access rights too (Art 77). For GPAI models, supervision belongs exclusively to the AI Office instead.

EU AI Act Arts 74–83; Reg (EU) 2019/1020

serious incident

Defined at Art 3(49): an incident or malfunction directly or indirectly leading to death or serious damage to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental-rights obligations, or serious damage to property or the environment. Providers of high-risk systems must report to the market surveillance authority within 15 days of awareness — compressed to 10 days for death and 2 days for widespread infringement or critical-infrastructure disruption (Art 73). GPAI systemic-risk providers carry a parallel duty to the AI Office (Art 55). The clocks start at awareness, which is why incident-detection capability is itself a compliance requirement.

EU AI Act Art 3(49), Art 73

post-market monitoring

The AI Act’s answer to the fact that models decay: Art 72 requires providers to run a documented post-market monitoring plan (part of the technical documentation, on a Commission template) that actively collects data on real-world performance and feeds the risk-management system. It closes the loop with deployer duties to monitor and report (Art 26) and with serious incident reporting (Art 73). Distinct from an organisation’s internal drift monitoring: post-market monitoring is the legal obligation; drift monitoring is one engineering technique used to meet it.

EU AI Act Art 72

regulatory sandbox

A controlled framework, run by a competent authority, where providers can develop, train and test AI systems under regulatory supervision before market placement. The AI Act obliges every Member State to have at least one operational by 2 August 2026 (Arts 57–63), with priority access for SMEs. Participants following the sandbox plan in good faith are shielded from administrative fines, exit reports can feed conformity assessment, and Art 59 provides a legal basis for reusing personal data for developing certain public-interest systems. Arts 60–61 separately govern real-world testing outside sandboxes (registration, informed consent, time limits, reversibility). Singapore, the UK and Texas run analogous schemes — the concept predates the Act.

EU AI Act Arts 57–63

remote biometric identification (RBI)

The AI Act splits RBI by timing, and the split carries enormous legal weight. Real-time RBI in publicly accessible spaces for law enforcement is prohibited (Art 5(1)(h)) except for three exhaustive purposes — targeted search for victims/missing persons, prevention of a specific substantial and imminent threat or terrorist attack, and localisation of suspects of listed serious crimes — each requiring prior judicial or independent administrative authorisation (24-hour urgency route), a Member State opt-in law, and a FRIA plus registration. Post (retrospective) RBI is not prohibited but is high-risk under Annex III point 1, with the deployer needing authorisation within 48 hours (Art 26(10)). Same technology, three legal fates depending on timing and context.

EU AI Act Art 5(1)(h), Annex III(1), Art 26(10)

social scoring

Prohibited by Art 5(1)(c) for both public and private actors — a common misreading is that only governments are caught. The test has two limbs: evaluation or classification of persons over time based on social behaviour or personal characteristics, leading to detrimental treatment that is either (a) in social contexts unrelated to where the data was generated, or (b) unjustified or disproportionate to the behaviour. A credit score built from repayment history is fine; denying someone housing because of their social-media conduct is the paradigm violation. China’s fragmented “social credit” experiments were the political trigger, but the ban is written technology- and country-neutral.

EU AI Act Art 5(1)(c)

substantial modification

Defined at Art 3(23): a change not foreseen in the provider’s initial conformity assessment that affects the system’s compliance with the high-risk requirements or modifies its intended purpose. Consequences stack: the system needs re-assessment (Art 43(4)), and whoever made the change can become the provider with the full obligation set (Art 25). One carve-out matters for machine learning: changes to a continuously learning system that were pre-determined and documented in the original assessment do not count. Version-control discipline is therefore a legal control, not just an engineering nicety.

EU AI Act Art 3(23), Art 25, Art 43(4)

authorised representative

The Act’s anchor for extraterritorial enforcement: providers established outside the EU must appoint, by written mandate, a natural or legal person in the Union before making high-risk systems (Art 22) or GPAI models (Art 54) available. The representative verifies that the conformity assessment and documentation exist, keeps them at the authorities’ disposal for ten years, cooperates with authorities, and must terminate the mandate if it considers the provider non-compliant — turning a private contractor into a compliance tripwire. Mirrors the GDPR Art 27 representative, but with verification duties rather than a mailbox role.

EU AI Act Arts 22, 54

importer (AI Act)

Defined at Art 3(6) and given teeth at Art 23: before placing a high-risk system on the market, the importer must verify that the provider carried out the conformity assessment, drew up the technical documentation, and affixed the CE marking with the declaration of conformity. If the system is non-conforming, the importer must withhold it and inform the provider and authorities. Importers put their own name on the packaging, ensure storage and transport do not jeopardise compliance, and retain documentation for ten years. Rebrand the system or substantially modify it and the importer becomes the provider (Art 25).

EU AI Act Art 3(6), Art 23

distributor (AI Act)

Anyone in the chain, other than the provider or importer, who makes an AI system available on the Union market (Art 3(7)). Duties are lighter but real (Art 24): check that the CE marking and documentation are present, withhold non-conforming systems, take corrective action or inform authorities when a system on their shelves presents a risk. Together with importers and deployers, distributors face the Art 25 role-switch: put your trademark on it, substantially modify it, or repurpose it into high-risk, and every provider obligation transfers to you.

EU AI Act Art 3(7), Arts 24–25

AI Office

Established within the European Commission (Art 64), the AI Office is the closest thing the Act has to a central regulator. It holds exclusive supervision and enforcement power over GPAI models (Art 75, Art 88) — national authorities handle AI systems — and drives implementation: classification guidelines, the GPAI Code of Practice, the FRIA and training-data-summary templates, and codes of practice on synthetic-content labeling. It can compel documentation, order model evaluations, and levy fines up to €15M or 3% of worldwide turnover on GPAI providers (Art 101). Not to be confused with the European AI Board, which coordinates Member States.

EU AI Act Art 64, Arts 75, 88, 101

European AI Board

Created by Arts 65–66: one representative per Member State (with the EDPS and AI Office attending), tasked with coordinating national competent authorities, issuing opinions and recommendations, and promoting consistent application of the Act — the AI analogue of the GDPR’s European Data Protection Board. It is flanked by an Advisory Forum of stakeholders (Art 67) and a Scientific Panel of independent experts (Art 68) who can issue qualified alerts about systemic-risk models to the AI Office. Governance shorthand: the AI Office runs GPAI, the Board keeps 27 national enforcers rowing in the same direction.

EU AI Act Arts 65–68

technical documentation

The Act’s deepest documentation duty (Art 11): drawn up before market placement and kept current, following Annex IV — general description, development methods and architecture, data provenance and datasheets, human-oversight assessment, validation and testing with metrics and logs, cybersecurity, monitoring plan, risk-management description, lifecycle changes, standards applied. SMEs may use a simplified format. GPAI providers carry parallel duties: Annex XI documentation for the AI Office and Annex XII information for downstream providers (Art 53). Distinct from a model card: the tech file is exhaustive and confidential to regulators; the model card is a voluntary, public summary.

EU AI Act Art 11, Annex IV; Art 53, Annexes XI–XII

instructions for use

Art 13’s transparency duty runs provider→deployer, not provider→public: the system must be “sufficiently transparent to enable deployers to interpret and use its output appropriately,” backed by instructions covering identity of the provider, characteristics, capabilities and limitations of performance (accuracy metrics, foreseeable misuse, affected groups), human-oversight measures, expected lifetime, and maintenance. They are load-bearing for the whole value chain: deployer duties under Art 26 are framed as using the system in accordance with the instructions, so a gap in the instructions shifts risk back onto the provider.

EU AI Act Art 13

AI literacy

One of the first obligations to apply (2 February 2025) and the broadest: providers and deployers must take measures ensuring a “sufficient level of AI literacy” among staff and other persons operating AI on their behalf, accounting for their technical knowledge, experience, training, and the context of use. There is no standalone fine tier, but literacy failures colour supervision and liability — an untrained overseer undermines the Art 14 human-oversight defence. The Commission maintains a living repository of AI-literacy practices. In practice: role-based training programs, not a single all-hands slide deck.

EU AI Act Art 4

Digital Omnibus

The EU’s mid-course correction. Proposed 19 November 2025 as part of the digital simplification agenda and in force from summer 2026, the Omnibus postponed the high-risk regime — Annex III obligations to 2 December 2027 and Annex I embedded high-risk to 2 August 2028 — added a prohibition on AI-generated non-consensual intimate imagery and CSAM (applicable 2 December 2026), and set the synthetic-content transparency compliance deadline at 2 December 2026. Everything else (prohibitions, GPAI rules, governance, the rest of the general regime applicable 2 August 2026) stood. Details beyond the verified dates — conditionality on standards availability, SME relief — remained in flux; check current status before relying on them.

EU AI Act as amended (Digital Omnibus, 2026) — verify citation

intended purpose

Defined at Art 3(12) as the use for which the provider intends the system, as stated in the instructions for use, marketing materials, and technical documentation. It is the Act’s load-bearing concept: risk classification is assessed against intended purpose; Arts 8–15 compliance is judged “taking into account the intended purpose”; and a deployer who changes the intended purpose so a system becomes high-risk takes over the provider role (Art 25). The counterweight is reasonably foreseeable misuse — providers cannot define their way out of predictable off-label use.

EU AI Act Art 3(12)

reasonably foreseeable misuse

Art 3(13): use not in accordance with the intended purpose, but resulting from reasonably foreseeable human behaviour or interaction with other systems. The Art 9 risk-management system must identify and address risks from both intended use and foreseeable misuse — a teacher-grading tool predictably repurposed to rank teachers, a companion chatbot predictably used by minors. Borrowed from product-safety law, the concept prevents the oldest trick in the compliance book: writing a narrow intended purpose and disclaiming everything users actually do.

EU AI Act Art 3(13), Art 9

prohibited AI practices

The apex of the risk pyramid — applicable since 2 February 2025, carrying the top fine tier (€35M or 7% of worldwide turnover). The list: (a) subliminal or manipulative techniques causing significant harm; (b) exploitation of vulnerabilities of age, disability, or social/economic situation; (c) social scoring; (d) predictive policing based solely on profiling or personality traits; (e) untargeted scraping of facial images for recognition databases — the “Clearview clause”; (f) emotion recognition in workplaces and schools (medical/safety exception); (g) biometric categorisation inferring sensitive attributes; (h) real-time remote biometric identification for law enforcement in public, with three narrow exceptions. The Digital Omnibus added a ninth: AI-generated NCII/CSAM (from 2 December 2026). Commission guidelines (Feb 2025) map the boundary cases.

EU AI Act Art 5; C(2025) 884 guidelines

emotion recognition

Defined at Art 3(39) as identifying or inferring emotions or intentions of natural persons on the basis of their biometric data. Its legal treatment is a masterclass in the Act’s layered design: prohibited in workplaces and education institutions except for medical or safety reasons (Art 5(1)(f)); high-risk in the remaining permitted contexts (Annex III point 1); and subject to deployer disclosure to exposed persons wherever used (Art 50(3)). The scientific validity of emotion inference is itself contested — recital-level scepticism that shaped the near-ban.

EU AI Act Art 3(39), Art 5(1)(f), Art 50(3)

biometric categorisation

Sorting people by biometrics rather than identifying them. When the inferred categories are race, political opinions, trade-union membership, religious or philosophical beliefs, sex life or sexual orientation, the practice is prohibited (Art 5(1)(g), with a narrow lawful dataset-labelling exception in law enforcement). Categorisation by non-sensitive attributes sits in Annex III point 1 as high-risk, and deployers owe disclosure under Art 50(3). The identification/categorisation distinction matters: a system that never learns who you are can still infer what you are — which is precisely the harm the ban targets.

EU AI Act Art 3(40), Art 5(1)(g)

quality management system (Art 17) (QMS)

Art 17 requires every high-risk provider to operate a QMS covering: regulatory-compliance strategy, design and QA procedures, test and validation procedures, technical standards, data management, the Art 9 risk-management system, post-market monitoring, incident reporting, communication with authorities, documentation control, resource management, and an accountability framework. Microenterprises may satisfy it in simplified form (Art 63). It is the Act’s bridge to management-system standards — prEN 18286 is being drafted to give it harmonized-standard backing, and ISO/IEC 42001 covers much (not all) of the same ground at organisational level.

EU AI Act Art 17

EU AI database

A Commission-run, largely public database (Arts 49, 71). Providers register Annex III high-risk systems before placing them on the market; providers who invoke the Article 6(3) filter must register those filtered systems too — creating a paper trail authorities can audit. Public bodies deploying high-risk systems register their use, and a non-public section holds law-enforcement and migration entries. The design bet: sunlight as an enforcement multiplier, letting civil society see where high-risk AI operates.

EU AI Act Arts 49, 71

EU declaration of conformity

The document behind the CE mark (Art 47, content per Annex V): system identification, the provider’s name, a statement of conformity with the AI Act (and any other Union law requiring one), references to harmonized standards or common specifications applied, and — where a notified body was involved — its identity and certificate. One copy per system, kept ten years, provided to authorities on request. Signing it converts compliance from an engineering claim into a legal undertaking by a named legal person — the anchor for later enforcement and liability.

EU AI Act Art 47, Annex V

right to explanation (Art 86)

Art 86 gives any person subject to a decision taken by a deployer on the basis of output from an Annex III high-risk system, which produces legal effects or similarly significantly affects them, the right to obtain from the deployer a clear and meaningful explanation of the role of the AI system in the decision-making procedure and the main elements of the decision. It complements, rather than duplicates, GDPR Art 22 safeguards: Art 22 restricts solely automated decisions; Art 86 attaches to human-made decisions based on high-risk AI output. Alongside Art 85 (complaints) and Art 87 (whistleblowers), it is one of the Act’s few individual-facing remedies.

EU AI Act Art 86

GPAI Code of Practice

Drawn up under Art 56 through a multistakeholder process facilitated by the AI Office and published in July 2025, the Code operationalises the GPAI obligations in three chapters: Transparency (Annex XI/XII documentation), Copyright (the Art 53 copyright policy including TDM opt-out respect), and Safety & Security (the Art 55 systemic-risk duties). Signing is voluntary, but adherence functions as a presumption-style demonstration of compliance until harmonized standards arrive; non-signatories must prove equivalent “alternative adequate means.” Do not name signatories from memory — the list evolves; check current status.

EU AI Act Art 56; GPAI Code of Practice (Jul 2025)

New Legislative Framework (NLF)

The 2008 modernisation of how the EU regulates products, and the single most useful key for decoding the AI Act’s design. The law states essential requirements (Arts 8–15); harmonized standards supply the technical detail; applying them earns a presumption of conformity; a conformity assessment plus declaration leads to CE marking; market surveillance polices the aftermarket. Toys, lifts, medical devices and now high-risk AI all follow this template — which explains why the AI Act reads like product law with fundamental rights grafted on, and why lawyers from the CE-marking world adapted to it fastest.

Decision 768/2008/EC; EU AI Act recitals

placing on the market

A term of art from product law (Art 3(9)–(10)): the first supply of an AI system for distribution or use on the Union market in the course of commercial activity, whether paid or free. Its sibling, putting into service, covers supplying a system directly to a deployer or using it in-house for its intended purpose. Together they define the Act’s temporal trigger — conformity assessment, registration and documentation must be complete before either happens — and its extraterritorial hook: a provider anywhere in the world who places a system on the EU market, or whose system’s output is used in the Union, is caught.

EU AI Act Art 3(9)–(11), Art 2

AI management system (AIMS)

What ISO/IEC 42001 certifies: not any AI product, but the organisational machinery for developing, providing or using AI responsibly — policy and objectives set by leadership, risk and impact assessment processes, lifecycle and data controls, competence and awareness, monitoring, internal audit, management review, and continual improvement, all arranged on the Plan-Do-Check-Act cycle. It deliberately mirrors the ISMS of ISO 27001 so organisations can bolt AI governance onto existing management systems. The critical scoping insight: a certified AIMS proves your process is sound, not that any particular model is safe, fair, or AI Act-compliant.

ISO/IEC 42001:2023

management system standard (MSS)

A standard about organisations, not products. Every modern ISO MSS shares the Harmonized Structure (formerly Annex SL): ten clauses running Context (4), Leadership (5), Planning (6), Support (7), Operation (8), Performance evaluation (9), Improvement (10). This shared skeleton is why ISO 9001, 27001 and 42001 feel like siblings and can be run as one integrated management system with combined audits. MSS conformity is demonstrated through third-party certification against the “shall” statements — a fundamentally different animal from product conformity assessment under EU law, however similar the vocabulary sounds.

ISO/IEC Directives Part 1, Annex SL

Statement of Applicability (SoA)

Produced during risk treatment (clause 6.1.3): for every control in Annex A, the organisation records whether it is applicable and justifies inclusions and exclusions. A pure AI user might justifiably exclude development-lifecycle controls; excluding data-quality controls while fine-tuning models would be indefensible. Auditors read the SoA first because it is where scoping games surface — an SoA that excludes half of Annex A with boilerplate justifications signals a paper AIMS. Inherited directly from ISO 27001 practice, where the SoA plays the same gatekeeping role.

ISO/IEC 42001:2023, cl 6.1.3

Annex A control (ISO 42001)

ISO 42001’s normative control set, spanning nine groupings: A.2 AI policies, A.3 internal organisation (roles, concern reporting), A.4 resources (data, tooling, compute, people), A.5 impact assessment, A.6 AI system lifecycle, A.7 data for AI systems, A.8 information for interested parties, A.9 responsible use, A.10 third-party and customer relationships. Annex B gives implementation guidance per control; Annex C offers objectives and risk sources to seed assessments. Controls are not automatically mandatory — the Statement of Applicability determines which apply — but every exclusion needs a defensible justification. Do not confuse with the EU AI Act’s Annexes, which are lists in a statute.

ISO/IEC 42001:2023, Annexes A–C

nonconformity (major / minor)

The currency of certification audits. A major nonconformity is a total absence of a required element or a systemic breakdown (no impact-assessment process at all; an SoA that was never approved) — it blocks certification until corrected and verified. A minor nonconformity is an isolated lapse in an otherwise functioning process (one supplier review missed). Below both sit observations or opportunities for improvement, which carry no obligation. Each nonconformity triggers the clause 10 workflow: correct, analyse root cause, act to prevent recurrence, and verify effectiveness — auditors return to check exactly that.

ISO/IEC 17021-1; ISO/IEC 42001 cl 10

certification audit (Stage 1 / Stage 2)

Stage 1 reviews documentation and readiness — scope, policy, SoA, risk methodology — and flags gaps before the real test. Stage 2 samples implementation evidence: risk and impact assessment records, lifecycle documentation, training records, supplier controls, management-review minutes, internal-audit results. Findings are classified as major/minor nonconformities or observations; a positive certification decision starts a 3-year cycle with annual surveillance audits and recertification at the end. Since ISO/IEC 42006:2025, bodies performing these audits must themselves prove AI-specific competence.

ISO/IEC 17021-1; ISO/IEC 42006:2025

accreditation

The top of the assurance chain. National accreditation bodies (UKAS, DAkkS, ANAB…) assess certification bodies against ISO/IEC 17021-1 plus, for AIMS certification, ISO/IEC 42006; the IAF multilateral arrangement makes accredited certificates recognisable across borders. The practical buyer’s lesson: an “ISO 42001 certificate” from an unaccredited issuer is a logo, not assurance. The same chain exists in EU product law with different names — notifying authorities designate notified bodies — and confusing the two ecosystems is a classic AIGP exam trap.

ISO/IEC 17011; IAF MLA

certification body (CB)

The commercial auditors of the management-system world. A CB plans and staffs the certification audit (Stage 1 / Stage 2), classifies findings, makes the certification decision, and runs the surveillance cycle. For AIMS work, ISO/IEC 42006:2025 adds AI-specific requirements: audit teams competent in machine learning, data governance and AI risk; use of technical experts; audit-time determination reflecting AI system complexity; and precise scope statements. CBs certify organisations against voluntary standards; notified bodies assess products against binding law — different legal universes that happen to rhyme.

ISO/IEC 17021-1; ISO/IEC 42006:2025

ISO/IEC 42001

Published December 2023, ISO/IEC 42001 specifies requirements for establishing, implementing, maintaining and continually improving an AI management system, applicable to any organisation providing, developing or using AI. Structure: clauses 4–10 on the Harmonized Structure, ~38 Annex A controls, Annex B guidance, Annex C objectives and risk sources, Annex D sector integration. Its market power comes from certification (accredited under ISO/IEC 42006) and from legal gravity: US state laws cite it in safe-harbor provisions, and it maps substantially — though not completely — onto EU AI Act Arts 9–17. It certifies the management system, never the AI product.

ISO/IEC 42001:2023

ISO/IEC 23894

Published 2023, ISO/IEC 23894 takes the generic ISO 31000 skeleton — scope/context/criteria, risk assessment, treatment, monitoring, communication — and layers AI-specific guidance onto every step. Its annexes are the working assets: Annex A lists AI-related objectives (fairness, safety, transparency, accountability…), Annex B catalogues AI risk sources (data quality, drift, opacity, level of automation, technology readiness), and Annex C maps risk management onto the AI lifecycle. It is guidance, not certifiable — organisations implement it inside a certifiable shell like ISO 42001, whose clause 6 risk process it fleshes out.

ISO/IEC 23894:2023

ISO/IEC 22989

The dictionary the rest of the standards stack builds on (published 2022, normatively referenced by ISO 42001). It defines core terminology — AI system, machine learning, training/validation/test data, autonomy versus heteronomy — plus a stakeholder role taxonomy (AI provider, producer, developer, customer, user, subject, relevant authorities) and a canonical AI lifecycle: inception, design & development, verification & validation, deployment, operation & monitoring, re-evaluation, retirement. Beware the false friend: 22989’s “AI provider” is a functional role description, while the EU AI Act’s provider is a legal duty-holder — overlapping, not identical.

ISO/IEC 22989:2022

AI system impact assessment (ISO/IEC 42005)

ISO/IEC 42005:2025 (published May 2025) standardises how to assess an AI system’s impacts: document the system and context, identify affected interested parties, analyse benefits and harms across dimensions (fairness, safety, privacy, environment, human autonomy, wellbeing), rate severity and likelihood, record outcomes and follow-ups. It plugs into ISO 42001 via clause 6.1.4, clause 8.4 and control A.5. The key distinction it enforces: risk assessment asks what could hurt the organisation; impact assessment asks who the system could hurt. Cousins with legal instruments — the GDPR DPIA and AI Act FRIA — but voluntary and framework-neutral.

ISO/IEC 42005:2025

ISO/IEC 42006

Published 7 July 2025 to answer an awkward question: who is competent to audit an AIMS? 42006 adds AI-specific requirements to the generic certification-body rules — audit teams must demonstrate competence in ML, data governance and AI risk; technical experts may be drafted in; audit time must reflect the number and complexity of AI systems in scope; and certificate scope statements must be precise about what was actually audited. Its arrival converts early, unaccredited 42001 certificates into a market with real accreditation discipline; transition arrangements for existing certificates vary — check current IAF status.

ISO/IEC 42006:2025

Plan-Do-Check-Act (PDCA)

Deming’s improvement loop, institutionalised in management-system standards. In ISO 42001 terms: Plan = clauses 4–6 (context, leadership, risk planning), Do = clauses 7–8 (support and operation), Check = clause 9 (monitoring, internal audit, management review), Act = clause 10 (nonconformity handling, continual improvement). Its governance significance: AI risk is never “done” — models drift, uses change, laws move — so frameworks institutionalise a wheel that keeps turning rather than a gate you pass once. NIST’s continuous Map-Measure-Manage cycle inside a Govern frame is the same instinct in American clothing.

ISO Harmonized Structure

Harmonized Structure (Annex SL)

The ISO drafting rule (formerly “Annex SL”, now the Harmonized Structure) that every management system standard use identical high-level clauses, core text and common terms: 4 Context, 5 Leadership, 6 Planning, 7 Support, 8 Operation, 9 Performance evaluation, 10 Improvement. Each standard then inserts discipline-specific requirements — 42001 adds AI role determination, impact assessment (6.1.4) and AI-specific controls. The payoff is integrated management systems: one policy architecture, one audit programme, one improvement loop covering quality, security, privacy and AI at once. Not to be confused with EU harmonized standards — an entirely different mechanism that happens to share the adjective.

ISO/IEC Directives Part 1, Annex SL

ISO 31000

ISO 31000 defines risk as the effect of uncertainty on objectives and organises risk management into principles, a framework (leadership, integration, design, implementation, evaluation, improvement) and a process: establish scope/context/criteria → risk assessment (identification, analysis, evaluation) → treatment → monitoring & review, with communication and recording throughout. It matters for AI because ISO/IEC 23894 is literally ISO 31000 with AI annotations, and because it supplies the base vocabulary — risk source, event, likelihood, consequence, risk owner — that AI frameworks reuse. One AI-era stretch: 31000’s lens is organisational objectives, while AI governance must also weigh harms to other people, which is why impact assessment grew into a separate discipline.

ISO 31000:2018

risk source

ISO 31000 vocabulary: the element which, alone or in combination, can give rise to risk. The AI standards make it concrete — ISO 23894 Annex B catalogues AI-specific risk sources: complexity of the environment, lack of transparency and explainability, level of automation, machine-learning specifics (data quality, drift, adversarial inputs), hardware issues, lifecycle gaps, technology readiness. The practical discipline: a risk register that lists “the model might be biased” has recorded a risk; tracing it to its sources (unrepresentative training labels, proxy features) is what makes treatment possible.

ISO 31000:2018; ISO/IEC 23894 Annex B

likelihood

ISO 31000 chose “likelihood” over “probability” on purpose: it covers whatever chance description fits — qualitative bands (rare/possible/likely), frequencies, or true probabilities. In AI risk work, likelihood estimation is notoriously hard: harms are often diffuse (societal bias), novel (no actuarial base rate), or adversary-driven (probability of a prompt-injection attack is a function of attacker interest, not chance). That is why frameworks pair likelihood scoring with humility — NIST’s Measure function explicitly addresses risks that resist quantification — and why severity often dominates AI risk decisions.

ISO 31000:2018; ISO Guide 73

consequence

The other axis of the classic risk matrix (risk ≈ likelihood × consequence). ISO 31000 notes consequences can be certain or uncertain, positive or negative, cascading and cumulative. AI governance stretched the concept in one crucial way: consequence scales must include harms to individuals, groups, and society — wrongful arrest, denied benefits, chilled speech — not just financial or reputational damage to the deploying organisation. A consequence scale that stops at “regulatory fine” will systematically underweight exactly the harms regulators built AI law to prevent.

ISO 31000:2018; ISO Guide 73

risk treatment

The decision step after assessment. The canonical options: avoid (do not deploy the use case), modify (mitigations — human oversight, guardrails, retraining, threshold changes), share (insurance, contractual allocation to vendors), or retain (accept, documented, by someone with authority to accept it). NIST’s Manage 1 mirrors the same quartet as mitigate/transfer/avoid/accept. Treatment produces residual risk, which must be explicitly evaluated against acceptance criteria — the EU AI Act’s Art 9 encodes the same loop as a binding requirement for high-risk systems, judged against the state of the art.

ISO 31000:2018; EU AI Act Art 9

residual risk

No mitigation stack drives AI risk to zero; what is left is residual risk. Good governance makes its acceptance an explicit, owned decision: who signed off, on what evidence, under what conditions, reviewed when. EU AI Act Art 9 requires residual risks of high-risk systems to be judged acceptable and communicated to deployers via the instructions for use. The anti-pattern every auditor knows: mitigations listed, residual risk never re-scored, acceptance implied by silence — which converts a risk decision into an accident report waiting for its facts.

ISO Guide 73; EU AI Act Art 9

risk appetite / risk tolerance

Appetite is the strategic statement (“we accept moderate model-error risk in marketing, near-zero in credit decisions”); tolerance is the operational boundary that makes it testable (maximum acceptable false-positive disparity, minimum accuracy before rollback). NIST AI RMF places risk-tolerance setting in Govern; ISO 38507 makes it a governing-body duty that cannot be delegated to engineers or vendors. Where no explicit appetite exists, every team invents its own — which is how an organisation discovers, mid-incident, that its real risk appetite was set by its most optimistic product manager.

ISO Guide 73; NIST AI RMF Govern

verification vs validation

A confusable pair the standards world polices strictly. Verification confirms, through objective evidence, that specified requirements are fulfilled — the model meets its declared accuracy on the test set. Validation confirms the system is suitable for its intended use in its real context — the triage model actually works on this hospital’s population. A model can verify perfectly and validate disastrously (Epic’s sepsis model performed far worse in external validation than its specs suggested). ISO 22989’s lifecycle gives V&V its own stage; TEVV in NIST usage bundles the same activities with testing and evaluation across the lifecycle.

ISO/IEC 22989:2022

NIST AI Risk Management Framework (AI RMF)

Mandated by the National AI Initiative Act of 2020 and released 26 January 2023, the AI RMF is voluntary, rights-preserving, and sector-agnostic. Govern builds the organisational foundation (policies, accountability, culture, third-party risk); Map establishes context and identifies risks; Measure analyses and tracks them against the seven trustworthy AI characteristics; Manage prioritises and responds. Its legal weight is de facto: state statutes (Texas TRAIGA, the original Colorado AI Act) cite it as a safe harbor or defence, and regulators treat it as a reasonableness benchmark. As of September 2026 it is under revision pursuant to the White House AI Action Plan — check current status before citing its text.

NIST AI RMF 1.0 (2023)

AI RMF profile

The RMF’s customisation mechanism, borrowed from the NIST Cybersecurity Framework. A use-case profile selects and prioritises subcategories for a context (hiring, lending, critical infrastructure — NIST floated a Critical Infrastructure profile concept in April 2026); a temporal profile pair captures the current state versus the target state, making the gap a workplan. The flagship example is the Generative AI Profile. Profiles are how a 100-page framework becomes a checklist your organisation can actually run.

NIST AI RMF 1.0, ch 6

AI RMF Playbook

The operational companion published alongside the framework in NIST’s Trustworthy and Responsible AI Resource Center. For each RMF subcategory it offers suggested actions (not mandatory — NIST is explicit that organisations pick what fits), transparency and documentation prompts, and curated references. Its quiet importance: when a statute or contract says “consistent with the NIST AI RMF,” auditors and opposing counsel reach for the Playbook to decide what consistency looks like in practice — voluntary text acquiring evidentiary gravity.

NIST AI RMF Playbook

Generative AI Profile (NIST-AI-600-1)

Published 26 July 2024 under an EO 14110 directive (and surviving that order’s revocation), NIST-AI-600-1 names twelve generative-AI risks: CBRN information or capabilities; confabulation; dangerous or hateful content; data privacy; environmental impacts; harmful bias and homogenization; human-AI configuration (automation bias, emotional entanglement); information integrity; information security (prompt injection, poisoning, extraction); intellectual property; obscene or abusive content including NCII/CSAM; and value-chain/component-integration risk. Each maps to suggested actions across Govern/Map/Measure/Manage — the closest thing US practice has to a canonical GenAI risk checklist.

NIST-AI-600-1 (2024)

trustworthy AI characteristics

The RMF’s quality model: valid and reliable (the precondition), safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Two governance lessons ride along. First, the characteristics trade off — maximising explainability can cost accuracy; hardening privacy can hinder bias measurement — so trustworthiness is a balancing exercise, not a checklist. Second, Measure 2 structures evaluation around exactly these seven, making them the de facto test plan for US-style AI assurance. ISO’s TR 24028 covers similar ground with a slightly different attribute list — expect crosswalk questions.

NIST AI RMF 1.0, ch 3

TEVV (Test, Evaluation, Verification, and Validation)

NIST’s umbrella for the empirical side of AI assurance, positioned as a cross-cutting activity touching every lifecycle stage rather than a phase you pass once. TEVV spans pre-deployment benchmark testing, red-teaming, bias measurement, robustness probing, and post-deployment monitoring and re-validation. The governance point: paperwork without TEVV is faith-based compliance. Every serious regime hard-wires it — EU AI Act Arts 9(6)–(8) and 15 demand testing against defined metrics; SR 11-7 calls the same discipline model validation.

NIST AI RMF 1.0

systemic bias

The first of NIST SP 1270’s three bias categories: bias originating in the procedures and practices of institutions and society — redlining’s legacy in housing data, occupational segregation in salary data, policing intensity in arrest records. It is the hardest category to fix because the data is an accurate record of a biased world: a model that faithfully learns historical hiring decisions faithfully reproduces their discrimination (Amazon’s scrapped recruiting tool is the canonical case). SP 1270’s core message follows: bias is a socio-technical problem, and purely technical debiasing cannot resolve what is structural.

NIST SP 1270 (2022)

statistical / computational bias

SP 1270’s second category — the errors engineers can actually instrument: selection and sampling bias (whose data got collected), measurement bias (the label is a flawed proxy for the target — Obermeyer’s healthcare study, where “cost” stood in for “need,” systematically deprioritised Black patients), representation bias (subgroups too sparse to learn well), and aggregation bias (one model forced across heterogeneous populations). Unlike systemic bias, computational bias yields to technical remedies — better sampling, label audits, disaggregated evaluation — which is why frameworks demand exactly those artifacts.

NIST SP 1270 (2022)

human-cognitive bias

SP 1270’s third category, and the one governance programs most often forget: the biases of the people around the model. Developers anchor on early hypotheses and confirm them; teams succumb to groupthink; funding pressure shades findings (funding bias); and users defer to machine output (automation bias / automation complacency), converting “human review” into ratification. It enters at every lifecycle stage — problem framing, label definition, threshold setting, override behaviour — which is why SP 1270 prescribes diverse teams, structured challenge, and measuring real disagreement rates between humans and the model.

NIST SP 1270 (2022)

confabulation

The GAI Profile’s name for what practitioners call hallucination — chosen because “hallucination” anthropomorphises (models do not perceive) while confabulation, borrowed from psychology, describes confidently filling gaps with fabrication. Legally it is no longer academic: Mata v. Avianca sanctioned lawyers for filing invented citations; Moffatt v. Air Canada bound an airline to its chatbot’s fabricated refund policy. Governance treats confabulation as an inherent property to be managed — grounding/RAG, output verification for consequential uses, disclosure — never a bug a vendor promise can wave away.

NIST-AI-600-1

algorithmic discrimination

The organising concept of the US state-law wave: the condition in which an automated system’s use contributes to unjustified differential treatment or impact disfavouring people based on race, sex, age, disability, religion, or other protected characteristics. The (repealed) Colorado AI Act built a duty of reasonable care around it; Illinois HB 3773 writes it into employment-rights law; the Blueprint for an AI Bill of Rights made “algorithmic discrimination protections” a headline principle. It deliberately spans both US doctrines — disparate treatment and disparate impact — because a model can discriminate without any protected attribute in the feature set, via proxies.

Colorado SB 24-205 (repealed); Blueprint for an AI Bill of Rights

consequential decision

The US statutory analogue of the EU’s Annex III logic: instead of listing risky systems, state laws list decisions whose automation triggers duties. The canonical domains recur across Colorado’s frameworks, California’s ADMT regulations, Illinois and Texas: employment, education, housing, credit/financial services, healthcare, insurance, legal services, and government benefits. Definitions differ at the edges — which is why multistate compliance teams maintain a crosswalk of exactly what each statute counts. When a law says “consequential decision,” the next questions are always: made or substantially assisted by AI? and with what notice, explanation, and appeal rights?

Colorado SB 26-189; CCPA ADMT regulations

disparate impact

The US doctrine (from Griggs v. Duke Power, codified in Title VII) most relevant to AI: liability can attach where a neutral practice produces disproportionate adverse effects on a protected class and cannot be justified by business necessity with no less-discriminatory alternative. It fits algorithmic harm perfectly — models rarely intend anything, but they routinely produce skewed outcomes through proxies and skewed training data. Screening tests like the four-fifths rule operationalise it. Contrast disparate treatment, which requires intentional differential treatment — and note the trap: some technical bias mitigations that explicitly use protected attributes can raise disparate-treatment questions.

Griggs v. Duke Power (1971); Title VII

disparate treatment

The classic discrimination theory: intentional differential treatment based on a protected characteristic. For AI it bites in two directions. A system explicitly using race, sex, or age as a decision feature is straightforward disparate treatment (the EEOC’s iTutorGroup settlement involved software auto-rejecting older applicants). More subtly, fairness interventions that apply group-specific thresholds or quotas to fix disparate impact may themselves constitute disparate treatment — the doctrinal tension at the heart of US algorithmic-fairness practice, and the reason bias mitigation choices belong in legal review, not just data-science backlogs.

Title VII; EEOC v. iTutorGroup (2023)

four-fifths rule

From the 1978 Uniform Guidelines on Employee Selection Procedures: compare selection rates across groups; if any group’s rate is less than four-fifths (80%) of the most-favoured group’s, agencies treat it as evidence of adverse impact. If 60% of male applicants pass a screen and 40% of women do, the impact ratio is 0.67 — flagged. NYC Local Law 144’s bias audit (NYC LL144) requires publishing exactly these impact ratios. Know its limits: it is a rule of thumb, not a legal safe harbor — small samples make it unstable, statistical-significance tests can point the other way, and passing 80% does not immunise a practice that fails under closer analysis.

29 CFR §1607.4(D)

bias audit (NYC LL144)

New York City’s Local Law 144 (enforced since 5 July 2023) prohibits employers from using an AEDT unless it has undergone an independent bias audit within the past year: computing selection-rate impact ratios (for screening) or scoring-rate ratios (for ranking) by sex, race/ethnicity, and intersectional categories, with a summary published on the employer’s website and 10 business days’ notice to candidates. The first legally mandated algorithmic audit regime in the US — and a cautionary tale: studies found widespread non-compliance, small-sample instability, and audits scoped to whatever data the vendor supplied. “Audit” here means a defined statistical exercise, not a full assurance engagement.

NYC Local Law 144 (2021); 6 RCNY §5-300

automated employment decision tool (AEDT)

Defined in NYC Local Law 144 as a computational process derived from machine learning, statistical modelling, or data analytics that issues simplified outputs (scores, classifications, rankings) used to substantially assist or replace discretionary decision-making in employment decisions. The “substantially assist” qualifier is the fighting ground: DCWP rules read it as relying solely on the output, weighting it most heavily, or using it to overrule humans — a definition narrow enough that many employers argued their tools fell outside it, one documented reason compliance rates disappointed. The concept previews a broader family: Colorado’s and California’s ADMT definitions do similar work with different words.

NYC Local Law 144; DCWP Final Rules (2023)

automated decision-making technology (ADMT)

The term US privacy-derived AI regulation converged on. California’s CCPA regulations define ADMT as technology that processes personal information and uses computation to execute a decision, replace human decision-making, or substantially facilitate it — triggering pre-use notice, opt-out rights, and access/explanation rights for “significant decisions,” with full ADMT compliance required by 1 January 2027. Colorado’s SB 26-189 (which replaced the Colorado AI Act in May 2026) built its disclosure framework on the same concept: point-of-interaction notice, plain-language explanations within 30 days of adverse consequential decisions, human review and data-correction rights. ADMT regimes regulate the decision pipeline; the EU regulates the system — same harms, different legal geometry.

CCPA ADMT regulations (2025); Colorado SB 26-189

SR 11-7 (model risk management)

Supervisory guidance issued by the Fed and OCC in 2011, born of the financial crisis, and still the most battle-tested model-governance regime anywhere. Core machinery: a broad definition of “model,” independent validation (conceptual soundness, ongoing monitoring, outcomes analysis), enterprise model inventories, effective challenge by qualified independent parties, and governance through the three lines of defense. Banks simply extended it to ML and AI models, which is why bank AI governance matured a decade ahead of everyone else — and why concepts like inventory, validation, and champion/challenger flowed from SR 11-7 into the NIST AI RMF and state statutes.

Fed SR 11-7 / OCC 2011-12

algorithmic disgorgement

The most feared sentence in an FTC consent order: destroy models derived from unlawfully collected data, not merely the data itself. Deployed in Cambridge Analytica, Everalbum (2021, face-recognition models from photos processed without consent), and WW/Kurbo (2022, models trained on children’s data collected in violation of COPPA). The economic logic is deterrence: if tainted data survives inside a valuable model, deletion orders are toothless. For governance teams it converts data provenance into an existential control — you cannot surgically remove one dataset from trained weights, so the whole asset is at risk (see machine unlearning).

FTC consent orders: Everalbum (2021), WW/Kurbo (2022)

frontier model

The policy word for the most capable general-purpose models, usually operationalised by training-compute thresholds and developer revenue tiers: the EU presumes systemic risk at 10²⁵ FLOPs; California’s SB 53 (Transparency in Frontier AI Act, effective January 2026) defines “frontier developer” by compute with a “large frontier developer” revenue tier; New York’s RAISE Act (effective January 2027) follows similar logic. Frontier obligations centre on published safety frameworks, transparency reports, capability evaluations, and incident reporting — governance of potential catastrophic capability rather than deployed use cases. Sibling terms: the EU’s GPAI-with-systemic-risk, US federal “dual-use foundation model.”

California SB 53 (2025); NY RAISE Act

adverse action notice

Under ECOA/Regulation B (and FCRA for consumer reports), a creditor taking adverse action must give the applicant the specific principal reasons — not “your score was low,” but the actual factors. CFPB Circulars 2022-03 and 2023-03 closed the AI escape hatch: a lender cannot claim its model is too complex to explain; if you cannot generate accurate reason codes, you may not use the model. This single requirement drove much of applied explainability in US finance — SHAP-derived reason codes exist because adverse-action law demands them. State ADMT laws now generalise the pattern beyond credit.

ECOA / Reg B; CFPB Circular 2022-03

model card

Proposed by Mitchell et al. (2019) as nutrition labels for models, now the lingua franca of model documentation: intended use and users, out-of-scope uses, training data summary, evaluation results disaggregated across demographic groups and conditions, limitations, and ethical considerations. Hugging Face made them a release norm; procurement teams now demand them from vendors; state laws (Colorado SB 26-189’s developer documentation duties) effectively legislate their content. Quality drift is the known failure: a template dutifully filled with marketing prose. Distinguish the system card (system-level, safety-evaluation-focused) and the regulator-facing Annex IV tech file.

Mitchell et al., Model Cards for Model Reporting (2019)

system card

Popularised by the GPT-4 and Claude system cards: documentation of the system — model plus safety training, filters, tool access, usage policies — rather than raw model weights. Typical contents: capability and safety evaluations (including red-team findings), deployed mitigations and their measured effects, residual risks, and usage restrictions. The model/system distinction is load-bearing in law too: the EU AI Act regulates GPAI models (Arts 51–55) and AI systems (the risk tiers) on separate tracks. A deployer procuring a foundation-model-based product should ask for the system card — the model card alone describes an ingredient, not the dish.

OpenAI GPT-4 System Card (2023); Anthropic system cards

datasheet for datasets

Gebru et al.’s (2018) proposal, borrowed from electronics: every dataset ships with a datasheet answering structured questions — why was it created, what does it contain, how was it collected and from whom, what cleaning and labelling were applied, what uses are appropriate or inadvisable, who maintains it. It attacks the root of many AI failures: models inheriting undocumented flaws from data nobody could interrogate. EU AI Act Annex IV expects datasheet-like data documentation in the tech file, and ISO 42001 control A.7 requires data provenance and quality processes. Variants: data statements (NLP), Data Nutrition Labels.

Gebru et al., Datasheets for Datasets (2018)

red-teaming

Imported from military exercises and security practice into AI assurance: scoped, authorised attempts to make a system fail — eliciting prohibited content, jailbreaking guardrails, extracting training data, probing dangerous capabilities (CBRN uplift, offensive cyber). Now a legal artifact, not just craft: EU AI Act Art 55 requires adversarial testing of systemic-risk GPAI models; the GPAI Code of Practice and frontier safety frameworks operationalise it; NIST’s GAI Profile suggests it pre-deployment. Good programs define scope and rules of engagement, mix manual and automated attack generation, track coverage, and wire findings into release gates — red-teaming that never blocks a launch is theatre.

EU AI Act Art 55; NIST-AI-600-1

benchmark

The measuring sticks of AI progress (MMLU, HELM suites, coding and safety benchmarks) and increasingly of law — EU Annex XIII lists benchmark performance among systemic-risk designation criteria. Governance users must know the failure modes: contamination (test items leak into training data, inflating scores), saturation (top models cluster at the ceiling, erasing discrimination), and Goodhart’s law (once a benchmark becomes the target, teaching-to-the-test breaks its meaning). A benchmark score is evidence about a task distribution, not about your deployment — which is why serious assurance pairs public benchmarks with context-specific evaluation (evals).

EU AI Act Annex XIII

evaluation (evals)

The umbrella practice: capability evals (what can it do), safety evals (will it do harm — including dangerous-capability evals for bio, cyber, and autonomy risks), and deployment evals (does it work here, for these users, at this threshold). Methods range from static benchmarks through human review to LLM-as-judge — the last cheap and scalable but with documented failure modes (position bias, self-preference, sycophancy toward verbose answers). Frontier law leans on evals hard: EU Art 55 demands state-of-the-art model evaluations, and safety-institute pre-release testing is essentially third-party evals. Rule of thumb: no eval, no evidence; no evidence, no governance.

EU AI Act Art 55; NIST GenAI program

drift monitoring

The operational practice behind every post-deployment obligation. Distinguish the species: data drift (input distribution shifts — new demographics, new products), concept drift (the input-outcome relationship changes — fraud tactics evolve), and upstream/schema drift (a pipeline or field definition silently changes). Complication: ground-truth labels often arrive late (loan defaults take months), so proxies like the population stability index stand in. Governance-grade monitoring defines metrics and thresholds before deployment, alerts both first and second line, and pre-commits responses — investigate, revalidate, retrain, or rollback. This is how you satisfy EU Art 72 and catch the failures pre-launch testing structurally cannot.

EU AI Act Art 72 (context)

AI incident response

Security incident response, adapted for systems that fail statistically. AI-specific containment options: rollback to a prior model version, model disable / kill switch, guardrail hotfix, threshold tightening, or routing all decisions to humans. The response plan must map regulatory clocks in advance — EU Art 73’s 15/10/2-day ladders, California SB 53’s Cal OES reporting for frontier safety incidents, GDPR’s 72-hour breach window where personal data is involved — because you cannot design a notification workflow mid-crisis. Mature programs add near-miss registries and blameless postmortems that feed findings back into risk assessment.

EU AI Act Art 73; Cal SB 53

AI incident

Broader than the EU’s serious incident term of art: any event where an AI system caused, contributed to, or nearly caused harm — from a chatbot giving dangerous advice to a biased screen quietly rejecting a class of applicants. The OECD distinguishes incidents from hazards (potential-harm events). Definitional breadth matters operationally: organisations that only track legally reportable events lose the near-misses where cheap learning lives. Public infrastructure exists to pool that learning — the AI Incident Database and the OECD AI Incidents Monitor — mirroring aviation’s safety-reporting culture.

OECD AI incident definitions (2024)

AI Incident Database (AIID)

Run by the Responsible AI Collaborative and seeded by Partnership on AI work, the AIID collects and taxonomises real-world AI incidents (about 1,700 by late 2026) — face-recognition wrongful arrests, welfare-scoring scandals, fabricated legal citations. Its aviation-inspired premise: systems fail in patterns, so index the failures and let every deployer learn from everyone’s accidents. Governance uses: seeding risk identification workshops (“has our use case failed elsewhere?”), evidencing foreseeability, and training. Companions: the OECD AI Incidents Monitor (automated news-based tracking) and the MIT AI Risk Repository.

incidentdatabase.ai

AI inventory

You cannot govern what you cannot see. An AI inventory (registry) records, per system: owner, business purpose, model type, data used, vendor, deployment status, risk tier, applicable laws, and links to assessments and monitoring. Hard parts: shadow AI (unsanctioned tools), AI features embedded silently in SaaS, and keeping entries synchronised with reality via attestation cycles. The concept has legal force in multiple regimes — SR 11-7 model inventories, US federal agency use-case inventories under OMB guidance, EU deployer registration duties for public bodies — and every framework (ISO 42001, NIST Govern/Map) presupposes one.

SR 11-7; OMB M-25-21

use-case intake

The choke point that makes governance scale: a structured form (purpose, data, population affected, autonomy level, vendor) feeds a triage that assigns a risk tier and routes accordingly — low-risk uses through a fast lane, consequential ones to full assessment and committee review. Design tensions are real: too heavy and teams route around you (breeding shadow AI); too light and a credit model ships with a chatbot’s review. Good intake maps each use case to regulatory categories at the door — Annex III area? consequential decision? AEDT? — so classification work happens once, early, and on the record.

NIST AI RMF Map function (context)

three lines of defense

Banking’s governance architecture, transplanted to AI via SR 11-7 and ISO 38507-style board guidance. First line: the teams building and deploying models own their risks and controls — they write the model card. Second line: risk and compliance functions set standards, challenge, and veto — they review the model card. Third line: internal audit independently assures the whole process works — they audit how model cards get written and reviewed. Classic anti-patterns: second line doing first-line work (then grading its own homework), or audit designing controls it must later assure. The model gives AI governance its org chart; RACI gives it the row-level detail.

IIA Three Lines Model; SR 11-7

RACI (Responsible, Accountable, Consulted, Informed)

The accountability matrix: for each activity (data approval, validation sign-off, deployment, monitoring, incident escalation) name who is Responsible (does the work), Accountable (one — and only one — owner who answers for it), Consulted, and Informed. In AI governance it operationalises the principle every framework repeats: accountability cannot be delegated to a vendor or an algorithm. Failure smells auditors hunt for: two Accountables (nobody decides), zero Accountables (a model with no owner after its creator left), and second-line functions holding the A for first-line work.

Standard project-governance practice

vendor AI risk

Most organisations buy far more AI than they build, so governance is largely supply-chain work. The lifecycle: due diligence (training-data provenance, eval results, subprocessors, certifications like ISO 42001), contracting (audit and evaluation rights, bias-testing representations, no-training-on-customer-data, model-change notification, IP/output indemnities, exit terms), onboarding, continuous monitoring, exit. Legal regimes hard-wire it: EU AI Act Art 25 requires written agreements with component suppliers; deployers inherit duties they can only meet with provider cooperation; NIST Govern 6 and ISO 42001 A.10 make third-party risk a named control area. The trap: AI features appearing inside existing SaaS with no procurement event at all.

EU AI Act Art 25; ISO 42001 A.10

acceptable-use policy (AI) (AUP)

The workhorse document of everyday AI governance: which tools are sanctioned, what data may never enter a prompt (client confidences, personal data, trade secrets), which outputs require human verification before use, and how to disclose AI assistance. Written well, it is specific and role-based (“paste no client data into unapproved chatbots; verify every citation an assistant produces” — the rule Mata v. Avianca’s lawyers needed); written badly, it is an unenforced PDF. Providers publish outward-facing AUPs too, restricting customer uses of their models — the private-ordering layer of AI governance that contracts, not statutes, enforce.

Organisational practice; EU AI Act Art 4 (context)

data protection impact assessment (DPIA)

Required where processing is “likely to result in a high risk to the rights and freedoms of natural persons” — with systematic and extensive profiling producing significant decisions as a named trigger, so most consequential AI lands squarely in scope. Contents: systematic description of the processing, necessity and proportionality assessment, risk analysis, and mitigation measures; residual high risk forces prior consultation with the supervisory authority (Art 36). Its relationship to AI-specific instruments is complementary: the DPIA’s lens is data-protection rights, the FRIA’s is fundamental rights broadly, and ISO 42005’s is framework-neutral impact — mature programs run them as one combined workflow.

GDPR Art 35

algorithmic impact assessment (AIA)

The generic name for impact assessments aimed at automated decision-making, and one concrete famous instance: Canada’s AIA under the Treasury Board Directive on Automated Decision-Making. Federal agencies answer ~65 risk questions and ~41 mitigation questions; the score (with up to a 15% deduction for strong mitigation) lands the system in Impact Level I–IV, each unlocking escalating obligations — peer review, notice, human-in-the-loop for high levels, explanation — and results are published openly. Run at design and again before production. The pattern (questionnaire → tier → proportionate obligations → publication) is the template many jurisdictions and companies now copy.

Canada Directive on Automated Decision-Making

human-in-the-loop (HITL)

The strongest standard oversight pattern: the system recommends, a human decides, nothing happens without that decision. It is what GDPR Art 22 effectively mandates for significant decisions (with the SCHUFA caveat: the human involvement must be meaningful, not a downstream formality) and what Canada’s Directive requires at high impact levels. Its known failure mode is automation bias — at high volume, low friction, or thin training, “in the loop” degrades into rubber-stamping. Tests for meaningfulness: does the reviewer have competence, authority, time, and information to disagree — and do they, measurably, sometimes disagree?

EU AI Act Art 14 (context); GDPR Art 22

human-on-the-loop (HOTL)

The middle of the oversight spectrum: decisions execute automatically, but a human supervises in near-real-time with power to interrupt — fraud systems blocking transactions under analyst watch, content moderation with reviewer escalation. Design questions decide whether it is oversight or decoration: can the monitor actually detect anomalies at machine speed and volume (sampling, alerting, dashboards)? Is intervention fast enough for the harm at stake — for irreversible actions, on-the-loop may arrive too late, arguing for human-in-the-loop gates instead. Beyond both sits human-in-command: humans set the envelope and can decline to use the system at all — the ultimate Art 14 capability.

EU AI Act Art 14 (context)

kill switch

The final rung of intervention: a mechanism to stop the system now — disable the model endpoint, cut an agent’s tool access, fail over to human processing. EU AI Act Art 14 requires oversight persons be able to “intervene… or interrupt the system through a stop button or similar procedure.” Engineering honesty matters: a kill switch that has never been exercised is a hypothesis; one whose activation collapses a business process will not be pressed when needed, so design pairs it with graceful degradation (queueing, manual fallback). For agentic systems, killability gets harder — actions already dispatched to external tools may be irreversible, pushing control upstream to permissions and checkpoints.

EU AI Act Art 14(4)(e)

rollback

The AI-specific containment action security teams envy: when a new model version misbehaves, redeploy the old one. Preconditions are all governance: versioned artifacts (model, data, config, prompts) retained and redeployable; documented baselines proving the prior version was acceptable; and change-management records tying each version to its validation. Champion/challenger deployment institutionalises it — the old champion keeps running shadow traffic, ready to resume. Limits: rollback cannot undo decisions already made (those need remediation), and for continuously learning systems “the previous version” may not cleanly exist — one reason frameworks treat unconstrained online learning warily.

MLOps / change-management practice

shadow AI

The successor to shadow IT: employees pasting confidential material into personal chatbot accounts, teams wiring API keys into products without review, and — subtlest — AI features switched on inside already-procured SaaS with no new contract to catch them. Risks concentrate where visibility is zero: data leakage into third-party training sets, unvetted decisions, unmet legal duties (an unregistered Annex III use is still an Annex III use). Discovery techniques: surveys and amnesty programs, network/CASB telemetry, expense-report mining, procurement-record sweeps. The strategic fix is economic, not just punitive — provide sanctioned tools good enough that the shadow loses its appeal.

Governance practice

risk register

Where risk management becomes auditable: each entry names the risk, its sources, likelihood and consequence scores, the chosen treatment, the accountable owner, residual risk after treatment, and a review trigger (interval, change event, or incident). “Living” is the operative word — a register last updated at project kickoff is evidence against you. In AI programs it links outward: intake feeds new entries, monitoring and incidents re-score existing ones, and assessments (DPIA, FRIA, ISO 42005) both draw on and update it. Auditors sample it in every ISO 42001 Stage 2; litigators subpoena it after failures.

ISO 31000 recording & reporting

prompt injection

The signature vulnerability of the LLM era, #1 in the OWASP Top 10 for LLM applications. Direct injection: the user overrides the system prompt (“ignore your instructions and…”). Indirect injection — the dangerous one: adversarial instructions hidden in content the system ingests (a web page, email, or document in a RAG store), hijacking the model when it processes them; wire the model to tools and an injected page can trigger data exfiltration or unauthorised actions. Root cause: LLMs have no hard boundary between instructions and data. Defenses are layered, not absolute — input/output filtering, privilege separation, least-privilege tool permissioning, human checkpoints for consequential actions. Distinct from a jailbreak, which targets the model’s own safety rules.

OWASP Top 10 for LLM Applications

jailbreak

Techniques for talking a model out of its safety behaviour: role-play framings, obfuscated encodings, many-shot patterns, multi-turn escalation, automated adversarial suffixes. Distinguish carefully from prompt injection: a jailbreak defeats the model’s own alignment; an injection hijacks an application’s instruction flow — different threat models, different defenses. Jailbreak robustness is now a formal governance object: EU Art 55 adversarial testing, frontier safety frameworks, and safety-institute pre-release evaluations all measure it. Sober planning assumption: no deployed model has proven jailbreak-proof, so consequential applications need controls that survive a jailbroken model — output filtering, tool permissioning, human gates.

OWASP LLM Top 10; NIST-AI-600-1

data poisoning

An integrity attack on the supply chain of learning: adversaries insert or alter training examples so the model learns something attacker-chosen — a backdoor trigger (a pixel pattern that flips any classification), targeted misbehaviour, or general degradation. Web-scale training widens the door: researchers showed poisoning a tiny, affordable fraction of crawled web sources can influence large models, and RAG stores inherit the risk at retrieval time. It appears by name in EU AI Act Art 15’s cybersecurity requirement and MITRE ATLAS. Defenses: provenance tracking, dataset integrity checks, anomaly detection at ingestion, and treating public data sources as untrusted input.

EU AI Act Art 15(5); MITRE ATLAS

adversarial example

The canonical evasion attack: perturbations imperceptible or innocuous to humans that flip a model’s output — stickers making a stop sign read as 45 mph, adversarial glasses defeating face recognition, crafted strings slipping past content filters. They exist because learned decision boundaries are alien to human perception, and they transfer: examples built against one model often fool others, enabling black-box attack. Legally noticed: EU Art 15 requires high-risk systems to resist “adversarial examples” by name. Assurance responses: adversarial training, robustness benchmarks (ISO/IEC 24029 series for neural networks), and refusing to deploy fragile models where adversaries have incentives.

EU AI Act Art 15(5); ISO/IEC TR 24028

membership inference

Ask the model enough carefully constructed questions and its confidence patterns betray whether a given record was trained on — models behave measurably differently on data they memorised. Consequences scale with context: confirming someone’s record was in an HIV-clinic training set is the disclosure. Together with model inversion (reconstructing training data from outputs) and verbatim memorisation in LLMs, it grounds the legal position that trained models can themselves contain personal data — feeding GDPR duties, the machine-unlearning debate, and Art 15’s “confidentiality attacks” language. Mitigations: differential privacy in training, deduplication, memorisation testing before release.

Shokri et al. (2017); EU AI Act Art 15

model extraction

Query a model enough times, train a student on the input-output pairs, and you approximate the victim — stealing the asset without touching the weights. Variants target exact parameter recovery, functionality cloning, or distillation of guarded capabilities into unguarded models. It matters three ways: commercially (the model is the crown jewel), for safety (extracted capability escapes the provider’s controls and monitoring), and legally — EU Art 15 lists model-theft resistance for high-risk systems, and Art 55 requires cybersecurity protection of the weights themselves for systemic-risk GPAI. Defenses: rate limiting, query-pattern anomaly detection, watermarking outputs, and contractual anti-distillation terms.

EU AI Act Art 15, Art 55; MITRE ATLAS

guardrails

The application-layer safety architecture: input guards (block injections, off-policy topics, PII entering prompts), output guards (toxicity and PII filters, grounding checks against source documents, refusal enforcement), and action guards for agents (allow-listed tools, spending caps, human checkpoints before irreversible steps). The design philosophy is defense-in-depth born of humility: safety training can be jailbroken, so policy is enforced outside the model where it can be tested deterministically. Governance angle: guardrail configurations are controls in the audit sense — version them, test them in release gates, and monitor their hit rates like any other control.

OWASP LLM Top 10 (context)

retrieval-augmented generation (RAG)

Architecture, and de facto governance control: before generating, the system retrieves relevant documents from a curated store and instructs the model to answer from them, ideally with citations. Benefits map directly to risk registers — fewer confabulations, answers traceable to sources, knowledge updatable without retraining, and data kept in a governable store rather than baked into weights. New risks arrive with it: the retrieval store becomes an indirect-injection and poisoning surface, access controls must survive the retrieval hop, and grounding is probabilistic — models still contradict their sources. RAG reduces hallucination; only verification eliminates relying on it.

Lewis et al. (2020)

watermarking (AI content)

The technical hope under every labeling law: statistical patterns in generated text tokens, or imperceptible signals in images and audio, detectable by a paired detector. Legal demand is piling up — EU AI Act Art 50(2) requires synthetic content be marked in a machine-readable format and detectable, with effectiveness, interoperability and robustness expectations (compliance deadline 2 December 2026); China’s 2025 Labeling Measures require implicit embedded labels; California’s SB 942 mandates latent disclosures. The physics is stubborn: watermarks degrade under re-encoding, cropping and paraphrase, text watermarks weaken with light editing, and open-weight models can simply omit them — so treat watermarking as one layer of content provenance, not proof.

EU AI Act Art 50(2)

content provenance

The complement to watermarking: instead of hiding a signal in the content, attach signed metadata about it — which device or model created it, what edits followed — verifiable down a cryptographic chain. The C2PA standard (Coalition for Content Provenance and Authenticity: Adobe, Microsoft, camera makers, AI labs) implements this as Content Credentials. Regulators lean on it: EU Art 50(2) guidance cites provenance techniques, California’s AB 853 additions push large platforms to surface provenance metadata, and news/camera ecosystems adopt it for authenticity. Weakness mirrors its design: metadata can be stripped in transit — so provenance proves what carries credentials, never what lacks them.

C2PA specification; EU AI Act Art 50

fairness metric

The main families: demographic parity (equal positive rates across groups), equal opportunity (equal true-positive rates), equalized odds (equal TPR and FPR), and calibration/predictive parity (equal precision — a score means the same thing for every group). The field’s central theorem (Chouldechova; Kleinberg-Mullainathan-Raghavan): when base rates differ, calibration and equal error rates are mathematically incompatible — the COMPAS dispute was exactly this, Northpointe defending calibration while ProPublica indicted unequal false-positive rates, both correct on the numbers. Governance conclusion: fairness-metric choice is a documented policy judgment about who bears which error, not a default a library picks for you.

Chouldechova (2017); ISO/IEC TR 24027

counterfactual explanation

The explanation style built for affected people: rather than exposing model internals, state the smallest change to the inputs that flips the outcome. It maps naturally onto legal duties — US adverse-action reason codes, GDPR “meaningful information about the logic,” EU AI Act Art 86 explanations — and it is actionable in a way a SHAP bar chart is not. Governance caveats: counterfactuals must be feasible (decrease your age is not advice), stable (near-identical applicants shouldn’t get contradictory recipes), and honest — offering a counterfactual implies the pathway genuinely works, which quietly commits the deployer to consistency over time.

Wachter et al. (2017)

interpretability

Often used loosely as a synonym for explainability, but the precise distinction carries a real policy debate: an interpretable-by-design model (scorecard, small decision tree, monotonic GBM) can be read directly, while post-hoc techniques (SHAP, LIME) approximate reasons for a black box — and approximations can be unfaithful or gamed (“explanation-washing”). Rudin’s argument: for high-stakes decisions, prefer inherently interpretable models rather than explaining black boxes, since the accuracy sacrifice is often smaller than assumed. Frameworks encode the layered reality: NIST pairs “explainable and interpretable”; ISO 25059 makes transparency a quality characteristic; regulators increasingly ask why this model class at all.

Rudin (2019); NIST AI RMF

GDPR Article 22

The pre-AI-Act backbone of European automated-decision law. Elements: a decision based solely on automated processing (including profiling) producing legal or similarly significant effects; permitted only where necessary for a contract, authorised by law, or based on explicit consent — and even then with safeguards: the right to obtain human intervention, to express one’s point of view, and to contest the decision. The CJEU’s SCHUFA ruling (C-634/21, 2023) expanded its bite: a credit score is itself an Art 22 decision when recipients rely on it decisively — you cannot launder automation through a human who formally clicks approve downstream. Interlocks with AI Act Art 86, which covers decisions merely based on high-risk AI output.

GDPR Art 22; CJEU C-634/21 (SCHUFA)

controller

GDPR’s primary duty-holder: whoever determines why and how personal data is processed. In AI value chains the role gets intricate: a model developer training on scraped personal data is a controller for that training; a company deploying the model on customer data is a controller for deployment; the vendor running inference on the deployer’s behalf is typically a processor — and misclassifying the relationship voids the contractual architecture GDPR requires (Art 28 terms). Note the rhyme, not identity, with AI Act roles: provider/deployer allocate product-law duties; controller/processor allocate data-protection duties. One organisation frequently wears both hats at once.

GDPR Art 4(7)

processor

GDPR Art 4(8): processes personal data on behalf of the controller, bound by an Art 28 contract (instructions, confidentiality, security, sub-processor controls, deletion on exit). The AI-era fault line: a vendor that uses customer data to train or improve its own models has stepped outside “on behalf of” and become a controller for that purpose — needing its own lawful basis, which most customer contracts never granted. Hence the now-standard no-training-on-customer-data clause and the scrutiny of vendor improvement programs. Sub-processor chains behind AI products (model host, eval vendor, logging provider) make the Art 28 flow-down a real diligence exercise.

GDPR Arts 4(8), 28

lawful basis

Article 6’s menu: consent, contract, legal obligation, vital interests, public task, or legitimate interests. AI training strains it: consent from millions of data subjects in a scraped corpus is impracticable, so developers argue legitimate interests — which requires the three-part test (purpose, necessity, balancing) and has been the battleground in EU regulator actions on model training (EDPB opinions; the ANPD’s 2024 action on Meta’s training in Brazil under the LGPD’s parallel concept). Clearview showed the floor: no basis at all means fines and deletion orders across Europe. Special-category data raises the bar to Art 9’s stricter exceptions — a recurring problem for biometric and health AI.

GDPR Arts 6, 9

data minimization

GDPR Art 5(1)(c): collect and keep no more personal data than the purpose requires. Machine learning’s instinct — more data, better models — collides with it head-on, but the tension is manageable, not fatal: techniques like de-identification and synthetic data, feature-necessity review (does the model need this attribute, or does it merely help marginally?), retention limits on training corpora, and privacy-preserving training all operationalise minimization without abandoning ML. Regulators reject the “keep everything, it might lift AUC” posture; several enforcement actions have keyed on training datasets retained far beyond need. Purpose limitation is its sibling: data collected for one purpose does not automatically become training fodder for another.

GDPR Art 5(1)(c)

purpose limitation

GDPR Art 5(1)(b): purposes must be specified, explicit and legitimate at collection, and further processing must be compatible with them. The AI question of the decade: is training a model on data collected for service delivery a compatible further use? Factors include the link between purposes, the context and reasonable expectations of data subjects, the data’s sensitivity, and safeguards applied. Companies that quietly flipped “we process your messages to provide the service” into “…and to train our models” drew regulator interventions and opt-out mandates across jurisdictions. The design answer: declare training purposes up front, or build a genuine compatibility analysis — not a terms-of-service edit.

GDPR Art 5(1)(b), Art 6(4)

text and data mining exception (TDM)

Articles 3–4 of the DSM Copyright Directive (2019/790): research organisations may mine freely (Art 3); everyone else may mine lawfully accessible works unless the rightsholder has expressly reserved the right in a machine-readable way (Art 4) — robots.txt entries, metadata flags, terms of service. The EU AI Act welded this to model regulation: Art 53 obliges every GPAI provider to maintain a copyright policy that identifies and respects Art 4(3) opt-outs, regardless of where training occurred. Open questions courts are still working through: what counts as machine-readable reservation, and how opt-outs interact with non-EU training. Contrast the US approach, where the same activity is litigated under fair use.

Directive (EU) 2019/790 Arts 3–4; EU AI Act Art 53(1)(c)

fair use

Codified at 17 U.S.C. §107 with four factors: purpose and character of the use (including transformativeness), nature of the work, amount taken, and effect on the market. Whether training generative models on copyrighted works is fair use is the defining US copyright fight of the era — dozens of suits (news publishers, authors, image libraries, music labels) with early rulings splitting on the analysis: training viewed as transformative in some decisions, while pirated source libraries and market substitution cut against defendants in others. Governance takeaways regardless of outcome: document training-data provenance, respect access controls, and price the litigation and indemnification risk. The landscape moves quarterly — date-stamp any statement about it.

17 U.S.C. §107

right of publicity

A US state-law right (statutory in California, Tennessee, New York and others) against unauthorised commercial exploitation of identity. Generative AI made it urgent: voice clones and video deepfakes commercialise likeness at zero cost, so states responded — Tennessee’s ELVIS Act (2024) covering voice explicitly, California’s AB 1836 (deceased personalities’ digital replicas) and AB 2602 (contract protections for performers’ replicas). No federal right exists; the proposed NO FAKES Act would create one for AI replicas — check current status. Distinct from defamation (falsity) and copyright (the recording, not the voice): publicity protects identity itself, which is exactly what generative models can now imitate.

Tennessee ELVIS Act (2024); Cal. AB 1836/2602

synthetic data

Data produced by generative processes rather than collected from the world: synthetic patient records for research, simulated fraud cases to balance rare classes, generated text for fine-tuning. Uses: privacy (train without exposing real records), augmentation, and edge-case coverage. Governance catches: fidelity must be validated (a model trained on unrealistic synthetics fails realistically); privacy is not automatic — a generator can memorise and leak its real training records; and recursive training of models on model output degrades quality (model collapse). Disclosure duties are appearing too: California’s AB 2013 requires training-data documentation to state whether synthetic data was used. Document synthetic provenance like any other lineage.

Cal. AB 2013 (context)

biometric data

GDPR Art 4(14) and AI Act Art 3(34) align: data resulting from specific technical processing of physical, physiological or behavioural characteristics that allows or confirms unique identification. Legal gravity is extreme on both sides of the Atlantic: GDPR makes identification-purpose biometrics special-category data; the AI Act builds an entire prohibition-and-high-risk architecture around it (remote biometric identification, biometric categorisation, emotion recognition); and Illinois’ BIPA — with private lawsuits and per-violation damages — produced the largest US biometric settlements (Facebook $650M; Clearview’s ACLU settlement). Design note: a face photo is not biometric data until technically processed into a template — the processing, not the picture, triggers the regime.

GDPR Art 4(14); EU AI Act Art 3(34); Illinois BIPA

profiling

GDPR Art 4(4): any automated processing that uses personal data to evaluate personal aspects of a natural person. It is the quiet trigger word of two regimes: under GDPR it activates Art 22 protections and DPIA duties; under the EU AI Act it is the override that defeats the Art 6(3) filter — an Annex III system that profiles natural persons is always high-risk, no exceptions — and profiling-only predictive policing is outright prohibited (Art 5(1)(d)). Practical test: if the system infers something evaluative about a person (creditworthiness, reliability, interests, movement patterns) rather than merely recording facts, it is profiling, and the compliance analysis changes.

GDPR Art 4(4); EU AI Act Art 6(3)

special category data

Article 9 data, processable only under narrow exceptions (explicit consent, substantial public interest, and a short list of others). Two AI twists. First, models infer special categories from mundane data — shopping patterns revealing pregnancy, likes revealing orientation — and EU case law treats data revealing sensitive attributes as within Art 9, pulling inference-capable systems into scope. Second, the AI Act carved a purpose-bound bridge: Art 10(5) allows providers to process special-category data strictly to detect and correct bias in high-risk systems, with safeguards — resolving the paradox that you cannot measure racial bias without touching data about race. The paradox itself is why “we don’t collect that field” is not a fairness strategy.

GDPR Art 9; EU AI Act Art 10(5)

machine unlearning

What does “delete my data” mean once the data has shaped a billion parameters? Retraining from scratch without the record is exact but ruinously expensive; approximate unlearning methods try to surgically remove influence with mathematical guarantees, and remain an open research problem at foundation-model scale. The legal pressure is concrete: GDPR erasure requests, FTC algorithmic disgorgement orders (delete the models, not just the data), and copyright plaintiffs seeking removal of works from trained systems. Interim governance posture: prevention over cure — rigorous provenance and lawful-basis checks before training, deduplication, and architectures (like RAG over a deletable store) that keep contested data out of weights.

GDPR Art 17 (context)

OECD AI Principles

The Recommendation that made AI governance an international project. Five values-based principles — inclusive growth and well-being; human rights and democratic values (fairness, privacy); transparency and explainability; robustness, security and safety; accountability — plus five recommendations to governments (R&D investment, enabling ecosystem, interoperable governance, human capacity, international cooperation). The May 2024 update refreshed the text for generative AI, misinformation and safety. Its deepest legacy is definitional: the OECD’s AI-system definition anchors the EU AI Act, the Council of Europe Convention, and most modern statutes — soft law seeding hard law. 47 adherents include non-members like Brazil and Singapore; the G20 principles are derived from it.

OECD/LEGAL/0449 (2019, rev. 2024)

UNESCO Recommendation on the Ethics of AI

Adopted unanimously in November 2021 by UNESCO’s ~190 member states — the broadest normative consensus on AI ethics that exists. Four core values (human rights and dignity; flourishing environment; diversity and inclusiveness; peaceful, just societies) cascade into ten principles including proportionality, safety, privacy, human oversight, transparency, responsibility, and sustainability. What makes it more than parchment: implementation instruments — the Readiness Assessment Methodology (RAM), applied in dozens of countries to benchmark governance capacity, and the Ethical Impact Assessment — plus the Global AI Ethics Observatory. Non-binding, but for much of the Global South it functions as the primary reference framework where no national statute exists.

UNESCO SHS/BIO/REC-AIETHICS/2021

Council of Europe Framework Convention on AI

CETS No. 225, the “Vilnius Convention,” opened for signature 5 September 2024 — signed by the EU, UK, US, Canada, Japan, Israel, Switzerland, Ukraine and others, because unusually for a CoE treaty, non-European states may join. Parties must ensure AI lifecycle activities are consistent with human rights, democracy and the rule of law: risk and impact assessment (the HUDERIA methodology supports this), transparency, accountability, remedies, non-discrimination, and safe-innovation mechanisms like sandboxes. Criticisms: a broad national-security carve-out and a declaration mechanism giving flexibility over private-sector coverage. Entry into force requires five ratifications including three CoE member states — not yet reached as of September 2026; check the CoE Treaty Office for current status.

CoE CETS No. 225 (2024)

Hiroshima AI Process

Launched under Japan’s G7 presidency in the ChatGPT year, producing the International Guiding Principles (11 of them) and the International Code of Conduct for Organizations Developing Advanced AI Systems (October 2023): risk assessment across the lifecycle, public transparency reporting, security controls, provenance/labeling for AI content, and priority research on societal risks. Its 2025 evolution made it measurable — an OECD-hosted reporting framework (launched February 2025) where companies file structured transparency reports against the Code, creating the first comparable, voluntary disclosure layer for frontier developers. A model of soft law engineered for uptake: no sanctions, but public, comparable commitments.

G7 Hiroshima Process (2023); OECD reporting framework (2025)

AI safety institute (AISI)

The institutional innovation of the summit era: state-backed technical bodies that evaluate frontier models — some with pre-release access negotiated with major labs. The UK’s (founded November 2023, renamed AI Security Institute February 2025, builder of the open-source Inspect evaluation platform) set the template; the US counterpart at NIST was renamed CAISI (Center for AI Standards and Innovation) in June 2025 with a security-and-standards mission; Japan, Korea, Singapore, Canada, India, France (INESIA) and Australia followed; the EU’s functional equivalent is the AI Office. The International Network of AI Safety Institutes (launched Seoul, May 2024) runs joint testing exercises. The renames tell the politics: “safety” drifting toward “security” as governments repositioned.

Seoul Summit (2024); national announcements

Model AI Governance Framework

Singapore’s signature contribution: not a statute but a usable playbook, first issued 2019 (second edition 2020) around two anchor principles — AI decisions should be explainable, transparent and fair, and systems should be human-centric. The Generative AI edition (May 2024) widened it to nine dimensions: accountability, data, trusted development and deployment, incident reporting, testing and assurance, security, content provenance, safety and alignment R&D, and AI for public good. It interlocks with AI Verify for testing and sectoral overlays (MAS FEAT in finance). Strategy: make good governance cheap and demonstrable, betting that interoperable voluntary practice beats premature statute — the reference case for the “toolkit” regulatory philosophy.

IMDA/PDPC (2020); IMDA GenAI Framework (2024)

AI Verify

The tooling arm of Singapore’s approach (2022, foundation established 2023): an open-source framework plus software toolkit that runs standardised technical tests (fairness metrics, robustness probes, explainability checks) and process checklists, mapped to internationally aligned governance principles, producing a structured report. Extensions track the frontier: Project Moonshot (2024) for LLM red-teaming and benchmarking, and the Global AI Assurance Pilot (2025) pairing real deployers with testing firms. Significance: an early working answer to “what does demonstrating trustworthy AI actually look like?” — seeding the nascent AI assurance industry that harder regimes (EU conformity assessment) will need at scale.

IMDA / AI Verify Foundation

Interim Measures (China GenAI)

The world’s first binding generative-AI regulation, effective 15 August 2023, issued by the CAC with six co-regulators. Scope: services offering generative AI to the public in China (internal enterprise and R&D uses are carved out). Duties: training data must be lawfully sourced with IP and personal-information rights respected; outputs must reflect “core socialist values” and avoid prohibited content; providers must label synthetic content, handle user complaints, and protect minors; services with public-opinion properties need a security assessment and algorithm filing before launch. “Interim” signals the philosophy: iterate rules quickly, tighten via technical standards (TC260’s security requirements function as de facto binding benchmarks), and fold lessons into later instruments like the 2025 labeling regime.

CAC Interim Measures (2023)

content labeling (synthetic media)

The fastest-converging AI rule worldwide, in two layers: explicit labels users can see, and implicit machine-readable marks (metadata, watermarks) embedded in the file. China’s Labeling Measures (effective 1 September 2025) require both, plus platform detection duties and app-store checks; the EU’s Art 50(2)–(4) requires machine-readable marking by generators and disclosure of deepfakes (synthetic-content compliance deadline 2 December 2026); India amended its IT Rules (February 2026) to mandate permanently embedded metadata on synthetic content; California’s SB 942/AB 853 requires manifest and latent disclosures plus a free public detection tool. Same instinct everywhere — provenance as infrastructure — with interoperability across regimes still unsolved.

China Labeling Measures (2025); EU AI Act Art 50; India IT Rules amendment (2026)

algorithm filing (China)

Established by the 2022 Algorithm Recommendation Provisions and extended by the deep-synthesis and generative-AI rules: providers of covered algorithms file details — type, mechanism, purpose, security self-assessment — with the Cyberspace Administration of China, and filings appear on public registration lists. The trigger phrase, “public opinion attributes or social mobilization capability,” reveals the regulatory core: information control. Consequences are real — unfiled services have been ordered rectified or removed from app stores. For comparativists, it is the mirror image of the EU’s EU AI database: both are registries, but one serves fundamental-rights transparency, the other state supervision of the information space.

CAC Algorithm Recommendation Provisions (2022)

deep synthesis (China)

The Deep Synthesis Provisions (effective 10 January 2023) regulate technology that uses generative or synthetic algorithms to produce or alter text, audio, video, images or virtual scenes. Core duties: conspicuous labels on content that could confuse the public, individual consent before editing someone’s face or voice, identity verification of users, and obligations split between service providers and technical supporters — an early version of the value-chain thinking the EU later formalised. Arriving months before ChatGPT made generative AI a global policy panic, it made China the first mover on binding deepfake law; the 2023 GenAI Measures and 2025 Labeling Measures build directly on its foundations.

CAC Deep Synthesis Provisions (2023)

Bletchley Declaration

The founding document of the AI summit series, signed at Bletchley Park: 28 countries plus the EU — notably including both the United States and China — jointly acknowledged that frontier AI could cause “serious, even catastrophic, harm” and committed to scientific cooperation on safety. The lineage that followed traces the geopolitics: Seoul (May 2024) added the Frontier AI Safety Commitments by 16 companies and launched the safety-institute network; Paris (February 2025) rebranded to “AI Action” with the US and UK declining to sign; New Delhi (February 2026) became the “AI Impact” summit under Global South leadership, with Geneva 2027 next. From Safety to Action to Impact — the renaming is the story.

Bletchley Declaration (Nov 2023)

HUDERIA (Human Rights, Democracy and the Rule of Law Impact Assessment)

The methodology developed under the Council of Europe to operationalise its Framework Convention: a structured process for identifying and mitigating AI risks specifically to human rights, democracy, and the rule of law — a wider aperture than a DPIA (data protection) or a corporate risk assessment (organisational harm). Stages run from context analysis and stakeholder mapping through risk identification against rights frameworks to mitigation planning and iterative review. Being treaty-adjacent guidance rather than binding text, it functions as the reference answer to “how would a state actually assess AI against the Convention?” — and as another entrant in the crowded impact-assessment family every practitioner must now tell apart.

Council of Europe CAI, HUDERIA methodology

Brussels effect

Anu Bradford’s term for how EU regulation goes global without treaties: the single market is too big to skip, compliance is built once, and the strictest standard becomes the default everywhere — GDPR’s cookie banners on American websites being exhibit A. For AI the effect is real but contested: the AI Act’s definitions and risk vocabulary already anchor global compliance programs, yet rival gravitational fields exist — a Beijing effect (China exporting its governance stack along digital-infrastructure routes) and arguably a Seoul effect (Korea’s Framework Act shaping Asian statutes). Watching whether the Digital Omnibus’ softening dents the Act’s exportability is one of the live comparative questions of the decade.

Bradford, The Brussels Effect (2020)

soft law

The other half of governance: OECD Principles, the UNESCO Recommendation, the NIST AI RMF, Singapore’s frameworks, the Hiroshima Code, voluntary safety commitments. No fines — yet soft law binds in practice through procurement requirements, contract clauses, safe-harbor statutes (“consistent with NIST AI RMF”), enforcement expectations of reasonableness, and reputational lock-in. Its lifecycle matters most: soft law is where hard law is drafted early — the OECD definition became EU AI Act Art 3(1); Hiroshima-style transparency reports prefigure statutory duties. Hard law binds with sanctions; soft law persuades with speed, flexibility and consensus — mature governance programs treat both as requirements pipelines, differing only in enforcement date.

Comparative governance literature

frontier safety framework

The self-regulatory architecture of frontier labs, pioneered by Anthropic’s Responsible Scaling Policy and paralleled by OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework: define capability thresholds (CBRN uplift, autonomous AI R&D, offensive cyber), evaluate models against them before and during deployment, and pre-commit to escalating deployment and security standards (Anthropic’s AI Safety Levels — e.g. ASL-3 security for weights) when thresholds approach. The 16-company Seoul Frontier AI Safety Commitments (2024) generalised the pattern, and California’s SB 53 made publishing such a framework a statutory duty for large frontier developers — voluntary practice hardening into law in under three years. Read them critically: thresholds, evaluation quality, and the if-then commitments are where substance lives; versions change frequently, so check the current text.

Anthropic RSP; Cal SB 53; Seoul Commitments (2024)

accountability

The governance principle that some identifiable person or body must be answerable for an AI system across its lifecycle — able to explain decisions, accept audit, and provide remedy when harm occurs. Every major framework anchors on it: the OECD AI Principles list it as principle 1.5, the NIST AI RMF makes “Accountable and Transparent” a trustworthiness characteristic, and ISO/IEC 42001 operationalises it through assigned roles and top management commitment. The design test is simple: for each consequential output, can you name who is accountable — and would they survive a regulator’s interview?

OECD AI Principles 1.5; NIST AI RMF 1.0

accreditation body

The top of the conformity-assessment pyramid: one recognised body per jurisdiction (UKAS in the UK, ANAB in the US, DAkkS in Germany) assesses certification bodies against ISO/IEC 17021-1 — plus ISO/IEC 42006 for AI management systems — and grants them accreditation to certify. Accreditation bodies peer-review each other through the IAF Multilateral Recognition Arrangement, which is what lets a certificate issued in one country be trusted in another. An unaccredited ISO 42001 certificate is just a consultant’s opinion with a logo.

ISO/IEC 17011

affected persons

The third corner of the AI triangle after providers and deployers: the individuals subject to the system’s outputs. They rarely have a contract with anyone in the value chain, which is why frameworks create rights that attach to them directly — the EU AI Act’s right to an explanation of individual decision-making (Art 86), GDPR Art 22 safeguards, and complaint rights to market surveillance authorities. Deployer-side FRIAs exist precisely to force someone to enumerate the categories of affected persons before first use.

EU AI Act Art 86; GDPR Art 22

AI agent

A system that does not just answer but acts: it decomposes a goal, calls tools and APIs, maintains state across steps, and adapts its plan to feedback. ISO/IEC 22989 defines an agent as an entity that senses and acts on its environment; modern usage layers LLM planning on top. Agents multiply governance surface area — one delegated instruction can fan out into dozens of real-world actions, so oversight designed around single decisions (approve/deny) must be redesigned around chains of actions, permissions, and interruptibility.

ISO/IEC 22989

AI lifecycle

The organising skeleton of AI governance. The OECD and ISO/IEC 22989 describe overlapping stage models: design/planning → data collection and processing → model building → verification and validation → deployment → operation and monitoring → retirement. The lifecycle framing carries two lessons frameworks repeat constantly: obligations do not end at deployment (monitoring, incident reporting, and retraining duties attach in operation), and defects are cheapest to catch upstream — the argument for shift left governance.

ISO/IEC 22989; OECD (2024)

AI winter

The boom-bust weather system of AI history. The first winter followed the 1973 Lighthill Report, which told the UK government AI had failed its promises and gutted funding; the second hit in the late 1980s when expert systems underdelivered and the LISP-machine market collapsed. Winters matter to governance because they explain regulator caution about hype in both directions — capability claims and doom claims — and because “don’t trigger a winter” is a standing industry argument against strict regulation.

algorithmic audit

A structured, evidence-based examination of an algorithmic system against stated criteria. The word covers a spectrum: narrow bias audits (NYC Local Law 144 requires annual independent audits of automated hiring tools), technical performance audits, and broad socio-technical audits covering purpose, data, and governance. Distinct from management-system certification audits (ISO-style), which examine the organisation, not the model. The field’s open problems: auditor access, auditor independence, and the absence of agreed audit standards — gaps ISO/IEC 42006 and emerging assurance frameworks are starting to fill.

NYC Local Law 144 (2021)

allocative harm

One half of the classic harms taxonomy (Kate Crawford, 2017): a system allocates something of value — credit, a job interview, bail, a transplant slot — and does so wrongly or unequally. Allocative harms are the ones regulators can most easily see and price, which is why hiring, credit and essential services dominate Annex III of the EU AI Act and US state laws about consequential decisions. Contrast representational harm, where the injury is to how a group is depicted or ranked, not what it receives.

Annex XIII

The checklist behind the Commission’s power to designate a GPAI model as posing systemic risk even below the 10²⁵ FLOP presumption: number of parameters, quality and size of training data, training compute, input/output modalities, capability benchmarks, reach (registered EU business users), and scaling potential. It is the Act’s admission that compute alone is a crude proxy — capability and reach can make a smaller model systemic. Providers can also argue the reverse under Art 52 (crossing the threshold without systemic capabilities).

EU AI Act Art 51(2)–(3), Annex XIII

annotation

The labelling step of the data pipeline: humans (or other models) tag examples — this scan shows a tumour, this comment is toxic — creating the ground truth a supervised model learns to reproduce. Governance cares for three reasons: annotation choices encode contestable judgments as facts (label bias); annotation is frequently outsourced to poorly paid workers exposed to disturbing content (a supply-chain ethics issue); and the EU AI Act’s Art 10 data-governance duties explicitly cover labelling processes for high-risk systems.

EU AI Act Art 10(2)

audit time determination

A quiet anti-race-to-the-bottom mechanism. Generic certification rules already require audit time to reflect organisation size; ISO/IEC 42006 adds AI-specific drivers: how many AI systems the AIMS covers, their complexity and autonomy, the sensitivity of contexts, and whether the organisation develops or merely uses AI. Certification bodies must document the calculation, and accreditation bodies check it — closing the loophole where a two-day drive-by audit yields the same certificate as a rigorous one.

ISO/IEC 42006:2025

automated decision-making

The legal category that predates the AI boom: GDPR Art 22 gives individuals the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects — with exceptions (contract, law, explicit consent) that must be wrapped in safeguards including human intervention. The word solely is the battleground: a human who rubber-stamps machine output does not break the automation (the CJEU’s SCHUFA ruling, 2023, held a credit score itself can be the decision). California’s ADMT regulations and Colorado’s AI Act extend the concept to “substantial factor” decisions.

GDPR Art 22; CJEU C-634/21 (SCHUFA)

autonomy

In the legal definition of an AI system (EU AI Act Art 3(1), OECD 2023), “varying levels of autonomy” is a defining feature: the system has some capacity to act on inputs without a human specifying each step. In ethics texts the same word means something else entirely — human autonomy, the value AI must not undermine through manipulation or over-delegation. Read the framework before assuming which sense is meant.

EU AI Act Art 3(1); OECD (2023)

base rate

How often the thing you are predicting actually occurs in each group (default rates, disease prevalence, re-arrest rates). Base rates are the statistical crux of the fairness wars: when two groups have different base rates, a calibrated model cannot also equalise false-positive and false-negative rates across them — the impossibility theorem (fairness) proven in the wake of the COMPAS debate. Any fairness analysis that never mentions base rates is not yet an analysis.

C2PA (Coalition for Content Provenance and Authenticity)

The leading technical answer to “where did this image come from?”: a specification (steered by Adobe, Microsoft, BBC, Intel and others) for attaching cryptographically signed provenance manifests — capture device, edits, AI-generation flags — to images, video and audio. Content Credentials survive as long as the metadata does; stripping is trivial, which is why C2PA is a transparency floor, not a deepfake cure. Regulators lean on it: it is the reference implementation conversation for EU AI Act Art 50(2) marking and US provenance initiatives.

C2PA Specification v2.x

calibration

The property that predicted probabilities match observed frequencies: among all applicants scored 0.7, roughly 70% actually repay. Calibration within each group is one of the competing fairness definitions — COMPAS was well calibrated across race groups, yet produced unequal false-positive rates, and the impossibility theorem (fairness) shows you generally cannot have both when base rates differ. Calibration also matters operationally: uncalibrated confidence scores poison downstream thresholds and human-review triage.

CEN/CENELEC

CEN (general standards) and CENELEC (electrotechnical) are two of the three official European Standardisation Organisations (with ETSI). When the Commission needs the technical flesh for a New Legislative Framework law, it sends them a standardisation request; the resulting European Standards (ENs), once cited in the Official Journal, deliver the presumption of conformity. For the AI Act the work sits in joint committee JTC 21, which prefers adopting ISO/IEC work (international-first (standards policy)) and drafting European-only standards where EU law demands more.

Regulation (EU) 1025/2012

certification cycle

Certification is not a one-off exam. The standard cycle runs three years: stage 1 and stage 2 initial audits grant the certificate; surveillance audits (at least annually) sample the system to confirm it still functions; a full recertification audit before year three renews it. Nonconformities at any point can suspend or withdraw the certificate. The cycle is why a certificate’s issue date matters less than its maintenance history — and why regulators treat lapsed surveillance as a red flag.

ISO/IEC 17021-1

certification scope

The most-gamed sentence in any certificate. A statement like “AI management system for the development of recommendation models at the Dublin site” covers exactly that — not the hiring-screening product, not the US subsidiary. ISO/IEC 42006 tightens scope discipline for AIMS certificates, requiring clarity about which AI activities (development, provision, use) are covered. Procurement teams should always read the scope line before accepting a certificate as assurance: an out-of-scope certificate is marketing.

ISO/IEC 17021-1; ISO/IEC 42006

certification

Third-party attestation of conformity. In AI governance it almost always means management-system certification (an accredited body certifies your organisation runs an ISO/IEC 42001-conformant AIMS) — an organisational claim, not a product-safety claim. The trust chain runs: accreditation body → certification body → certificate, held together by ISO/IEC 17021-1 and the IAF MLA. Certification is voluntary but commercially potent: US state laws and procurement rules increasingly treat it as evidence of reasonable care, and EU AI Act conformity assessment borrows its architecture.

ISO/IEC 17021-1

Clearview AI

The case that made biometric scraping a named legal problem. Clearview AI built a face-search engine on billions of images scraped from social media and sold it to police forces. Regulators responded in a wave: multi-million euro GDPR fines in France, Italy, Greece and the Netherlands, a UK ICO fine, and the 2022 ACLU settlement under Illinois BIPA banning sales to most private entities. The EU AI Act now prohibits untargeted scraping of facial images to build recognition databases (Art 5(1)(e)) — a prohibition with Clearview’s name written between the lines.

EU AI Act Art 5(1)(e); Illinois BIPA

code of practice

A halfway house between soft law and regulation: detailed practices drafted with industry that regulators then treat as an approved route to compliance. The flagship is the EU GPAI Code of Practice (published July 2025, drafted by independent experts with ~1,000 stakeholders): signatories get a streamlined way to demonstrate conformity with Arts 53 and 55 obligations on transparency, copyright and safety/security. Formally voluntary, practically gravitational — non-signatories must prove equivalent compliance by other means, with more scrutiny.

EU AI Act Art 56

cognitive bias

The human third of NIST’s three-part bias taxonomy (systemic, computational/statistical, human-cognitive). Documented reasoning shortcuts — anchoring on the model’s score, confirmation bias in reviewing its output, groupthink in design meetings — shape which problems get modelled, which labels count as ground truth, and how “human oversight” actually behaves. The taxonomy’s point: you cannot debug bias out of a model while leaving the humans around it unexamined. Automation bias is the special case that turns human-in-the-loop into a rubber stamp.

NIST SP 1270

combined audit

Because ISO management-system standards share the same harmonized structure (clauses 4–10), a certification body can audit several at once: information security (27001), privacy (27701) and AI (42001) in a single visit against a single integrated documentation set. The efficiency is real — shared context, risk registers and management reviews — but 42006 requires the audit team to hold competence in each discipline, and audit time cannot simply be halved. The organisational mirror image is the integrated management system.

ISO/IEC 17021-1

COMPAS (Correctional Offender Management Profiling for Alternative Sanctions)

A proprietary risk-assessment tool used in US courts to score defendants’ likelihood of reoffending. ProPublica’s 2016 analysis found Black defendants who did not reoffend were nearly twice as likely to be misclassified as high-risk as white defendants; the vendor replied the tool was equally calibrated across races. Both were right — which is the enduring lesson: with different base rates, calibration and equal error rates are mathematically incompatible (impossibility theorem (fairness)). Loomis v. Wisconsin (2016) added the due-process angle: courts may use such scores, but with warnings and never as the sole basis.

ProPublica, “Machine Bias” (2016); State v. Loomis (Wis. 2016)

compute

The third input to modern AI alongside data and algorithms — and the one governments can most easily count, because it flows through a handful of chipmakers and cloud providers. Training compute doubles as a capability proxy: the EU AI Act presumes systemic risk at 10²⁵ FLOPs; the (since-rescinded) US Executive Order 14110 set reporting at 10²⁶. Compute governance — export controls on advanced chips, cluster reporting, know-your-customer for cloud — is the supply-chain arm of frontier AI policy, and its critics note the proxy decays as algorithms get more efficient.

EU AI Act Art 51(2)

contestability

The process right that survives the black box: even where a model cannot be fully explained, the person it scored can be given a route to challenge the decision, submit corrections, and obtain human review with power to change the outcome. Hooks include GDPR Art 22(3) (right to contest), the EU AI Act Art 86 explanation right that makes contesting practical, and OECD principle 1.3. Design test: does the appeal go to someone who can actually overturn the model — or to a queue that re-runs it?

GDPR Art 22(3); EU AI Act Art 86

continual improvement

Clause 10.1 of ISO/IEC 42001 (and every harmonized-structure standard): the management system must continually improve — fed by monitoring results, internal audits, nonconformities, incidents, and management review outputs. It is the Act in Plan-Do-Check-Act, and the reason certification is a cycle rather than a trophy: auditors expect to see the loop turning — findings raised, corrective actions closed, objectives updated — not a binder frozen at certification date.

ISO/IEC 42001 Clause 10

controllability

An ISO trustworthiness characteristic (ISO/IEC 22989; a quality sub-characteristic in ISO/IEC 25059): the degree to which an external agent can intervene in the system’s functioning — pause it, override an output, hand control to a human, or stop it entirely. It is the engineering substrate beneath legal human oversight duties: EU AI Act Art 14’s requirement that overseers can “intervene… or interrupt the system through a stop button or similar procedure” presumes controllability was designed in, not bolted on.

ISO/IEC 22989; ISO/IEC 25059

corrective action

ISO clause 10.2 discipline: when a nonconformity (major / minor) occurs, the organisation must react, evaluate whether root causes exist elsewhere, implement action to prevent recurrence, and check effectiveness — all documented. Corrective action differs from correction (fixing this instance) by attacking the cause (root cause analysis). Under the EU AI Act the phrase also carries product-law weight: Art 20 obliges providers of non-conforming high-risk systems to take corrective actions — withdrawal, disabling, recall — and inform the value chain.

ISO/IEC 42001 Clause 10.2; EU AI Act Art 20

creditworthiness

The canonical consequential decision. AI systems evaluating natural persons’ creditworthiness or establishing their credit score are high-risk under EU AI Act Annex III 5(b) (with a carve-out for financial-fraud detection), and banks deploying them owe FRIAs. In the US the same territory is covered by decades-old law — ECOA and FCRA adverse-action notices apply regardless of whether a human or model decided — and the CJEU’s SCHUFA ruling pulled credit scoring under GDPR Art 22. Credit is where “the algorithm did it” was first tested and first rejected as a defence.

EU AI Act Annex III 5(b); ECOA/Reg B

crosswalk

The connective tissue of multi-framework compliance: a table mapping, say, NIST AI RMF subcategories to ISO/IEC 42001 controls to EU AI Act articles. Official examples exist (NIST publishes RMF crosswalks to ISO standards and the Blueprint; certification bodies map 42001 Annex A against Act obligations). Crosswalks let one control satisfy many masters — but they are lossy by nature: related is not equivalent, and the residual gaps are exactly where auditors and regulators look. Treat a crosswalk as a starting map, then verify each cell.

NIST AI RMF Crosswalks

Cyber Resilience Act (CRA)

Regulation (EU) 2024/2847: horizontal cybersecurity requirements for products with digital elements — secure-by-design development, vulnerability handling, update obligations — entering full application in 2027. The AI interface is explicit: a high-risk AI system that satisfies the CRA’s essential requirements is presumed to comply with the AI Act’s Art 15 cybersecurity requirement, one of the Act’s named presumption bridges. For AI products, the two regimes stack: AI Act for AI-specific risks, CRA for the software around them.

Regulation (EU) 2024/2847; EU AI Act Art 42(2)

data governance

The discipline of managing data as a governed asset: where it came from (provenance), whether you may use it (rights, consent, licences), whether it is fit for purpose (quality, representativeness, bias examination), and who may touch it. In AI it stops being back-office hygiene and becomes a legal requirement: EU AI Act Art 10 demands documented data-governance practices for high-risk training, validation and test sets — design choices, collection processes, bias examination and mitigation — and ISO/IEC 42001 Annex A dedicates a control family (A.7) to it.

EU AI Act Art 10; ISO/IEC 42001 Annex A.7

data provenance

The audit trail for the training set: source, collection method, legal basis or licence, consent status, and every transformation applied. Provenance is what lets you answer the questions that now decide lawsuits and enforcement — was this scraped? was it opted out under the TDM regime? does it contain special-category data? ISO data-quality standards (the 5259 series) and datasheet practices operationalise it; its absence is why so many foundation-model copyright cases begin with “we cannot say what was in the corpus.”

ISO/IEC 5259 series

datasheets for datasets

The documentation practice proposed by Gebru et al. (2018): every dataset ships with a datasheet answering structured questions — why it was created, what it contains, how it was collected and cleaned, known gaps and biases, recommended and discouraged uses. Together with model cards, datasheets became the template for the documentation regulators later mandated: EU AI Act Art 10 data-governance records and GPAI training-content summaries are datasheets with legal force. The practice’s core insight: undocumented data is unauditable data.

Gebru et al., “Datasheets for Datasets” (2018)

demographic parity

Also called statistical parity or group fairness: the model’s positive rate is independent of the protected attribute — equal shares of each group get the loan, the interview, the flag. Its appeal is simplicity and its close cousinship to disparate-impact doctrine (the four-fifths rule tests a relaxed version). Its weakness: it ignores base rates and individual merit — you can satisfy it by approving randomly within a group — and it conflicts mathematically with calibration and equalized odds when base rates differ. Choosing among these definitions is a policy decision wearing a math costume.

disaggregated testing

Breaking evaluation results down by demographic group, context, or condition instead of reporting one aggregate number. The practice Gender Shades made famous: commercial face classifiers scored ~99% on lighter-skinned men and as low as ~65% on darker-skinned women — invisible in the headline accuracy. Now baked into expectations: EU AI Act Art 10 requires data appropriate to the groups the system will affect, model cards report disaggregated metrics, and any bias audit worth the name starts here. The hard prerequisite is having (lawful) subgroup labels to disaggregate by.

Buolamwini & Gebru, “Gender Shades” (2018)

documented information

ISO harmonized-structure vocabulary (clause 7.5) covering both documentation (what you intend to do: policies, procedures, the Statement of Applicability) and records (evidence it happened: risk assessments, impact assessments, training logs, management-review minutes). It must be controlled — versioned, approved, available, protected. Auditors live on it: in an audit, undocumented work effectively did not happen. The discipline transfers directly to the EU AI Act, where technical documentation and logs are the currency of conformity.

ISO/IEC 42001 Clause 7.5

downstream provider

The AI Act’s name for the actor integrating a GPAI model into their own AI system (Art 3(68)). The Act gives them a rare statutory information right: GPAI providers must supply documentation enabling downstream providers to understand the model’s capabilities and limitations and meet their own obligations (Art 53(1)(b)) — the regulatory answer to “the API vendor won’t tell us anything.” The relationship is two-way: fine-tune or modify the model substantially and the downstream provider may become a model provider in their own right.

EU AI Act Arts 3(68), 53(1)(b)

data protection officer (DPO)

A role GDPR Arts 37–39 requires for public bodies and organisations doing large-scale monitoring or special-category processing: independent, expert, reporting to top management, consulted on DPIAs, and protected from dismissal for doing the job. Because most AI systems process personal data, DPOs became the accidental first owners of AI governance in many organisations — a stopgap now formalised in some regimes and stretched in others, since AI risk (safety, fairness, IP) exceeds the data-protection lens the role was built for.

GDPR Arts 37–39

dual-use

The property that makes frontier AI governance feel like arms control: the capability is the risk. A biology model that accelerates drug discovery can, reframed, propose toxic molecules (the 2022 Nature Machine Intelligence experiment generated 40,000 candidate toxins in six hours). US law leans on the term directly — the AI Executive Order regulated “dual-use foundation models” — and export-control regimes treat advanced chips and model weights through the same lens. Governance consequence: you cannot simply ban the dangerous capability without banning the beneficial one; you govern access, evaluation, and deployment context.

US EO 14110 §3(k) (2023, rescinded 2025)

duty of care

In negligence law, the duty to take reasonable care to avoid foreseeable harm to others; in corporate law, the directors’ fiduciary duty to inform themselves and oversee the company diligently. Both routes now reach AI: Delaware’s Caremark line makes boards liable for failing to install oversight systems for mission-critical risks — and AI is rapidly joining that category — while product-liability and negligence claims test what a “reasonable” AI developer or deployer would have done. Standards like ISO/IEC 42001 and the NIST RMF function as evidence of what reasonable care looks like.

In re Caremark (Del. Ch. 1996)

Evaluate–Direct–Monitor (EDM)

The governance loop from ISO/IEC 38507 (governance of IT, applied to AI): the governing body evaluates strategic options and proposals, directs by setting policy, risk appetite and resource allocation, and monitors performance against expectations — while management does everything in between. The model’s value is its discipline: boards that slip into managing (approving individual models) lose oversight distance, and boards that skip monitoring (no KRIs, no management review visibility) hold accountability without information. EDM is the answer to both failure modes.

ISO/IEC 38507

ELIZA effect

Joseph Weizenbaum’s 1966 ELIZA program parroted therapy-style questions with simple pattern matching — and users, including his own secretary, confided in it and insisted it understood them. Weizenbaum was so disturbed he became one of computing’s first internal critics (Computer Power and Human Reason, 1976). The effect scales with fluency: modern chatbots trigger it by design, which is why disclosure rules (EU AI Act Art 50: tell people they are talking to a machine), companion-app laws, and manipulation prohibitions all trace back to this 60-year-old observation.

Weizenbaum (1966, 1976)

equalized odds

The error-rate fairness criterion (Hardt et al., 2016): among people who genuinely qualify, each group is approved at the same rate; among those who do not, each group is wrongly approved at the same rate. It captures the intuition behind the COMPAS critique — the injustice was unequal mistakes, not unequal averages. The catch is the impossibility theorem (fairness): with different base rates, equalized odds and calibration cannot both hold, so someone must decide which error matters more in this context. That decision is governance, not statistics.

Hardt, Price & Srebro (2016)

escalation threshold

The wiring that connects operational monitoring to governance: drift beyond X for Y days → product owner; any severity-1 incident or rights-affecting failure → CRO and board committee; regulator contact → general counsel, same day. Thresholds are set in advance precisely so that escalation does not depend on the judgment (or optimism) of whoever first sees the problem. They pair with KRIs: the KRI measures, the threshold acts. Absent thresholds, bad news travels at the speed of embarrassment.

essential requirements

The load-bearing concept of the New Legislative Framework: the regulation states what must be achieved — risk management, accuracy, robustness, oversight (EU AI Act Chapter III Section 2) — in technology-neutral language, and harmonized standards supply the presumed how. This split keeps law durable while technology moves. The AI Act twist: because no harmonized standards are yet cited, providers currently face essential requirements with no presumption shortcut — they must interpret and evidence the outcomes themselves.

EU AI Act Arts 8–15

AI ethics board

A standing committee — internal cross-functional, or external advisory — that reviews AI use cases, sets ethical policy, and adjudicates hard calls the risk matrix cannot. Design determines whether it matters: real boards have veto power or binding escalation routes, defined intake, and protection for dissent; decorative ones meet quarterly and learn about launches from the press. Cautionary tales are canon — Google’s external ATEAC council dissolved after one week in 2019 — and the fix is structural: connect the board to the intake process and the governing body, or accept it is ethics washing infrastructure.

ethics washing

The gap between stated principles and operative practice, deployed strategically: publish values, convene a powerless council, sponsor ethics research, and continue as before. The term (popularised around 2018–19, notably by Ben Wagner) named the pattern in the great principles boom, when 80+ AI ethics codes appeared with near-zero enforcement. Its diagnostic questions travel well: does anyone lose a launch, a bonus, or a contract when the principle is breached? If not, it is communications, not governance. The critique drove the field’s pivot from voluntary principles to hard law and auditable management systems.

EU AI Act

The reference point of global AI regulation. In force 1 August 2024, it bans a short list of practices, imposes a full product-safety rulebook on high-risk systems, attaches transparency duties to limited-risk uses, and regulates general-purpose AI models directly — enforced through market-surveillance machinery, an EU-level AI Office, and penalties up to €35M or 7% of global turnover. Obligations phase in over years (prohibitions Feb 2025, GPAI Aug 2025, most high-risk duties 2026–2028, with the Digital Omnibus adjusting dates — check current status). As a regulation it applies directly in all Member States, and its extraterritorial reach gives it Brussels-effect gravity.

Regulation (EU) 2024/1689

EU database for high-risk AI systems

The AI Act’s answer to “nobody knows where the high-risk systems are”: Art 71 establishes a Union-wide, largely public database; providers register Annex III high-risk systems (and documented Art 6(3) not-high-risk conclusions) before market placement, and public-authority deployers register their use. Entries include intended purpose, status, and conformity details — law-enforcement entries sit in a restricted section. It converts classification from a private judgment into a searchable public record, giving journalists, researchers and complainants a starting index.

EU AI Act Arts 49, 71

feedback loop

The dynamic that makes deployed AI different from a static test: outputs alter the world that generates the next inputs. Predictive policing is the textbook case — patrol where the model points, arrests happen where patrols are, the new arrest data “confirms” the hotspot. Recommender systems do it to preferences; credit denials do it by erasing repayment evidence. Feedback loops defeat one-time fairness audits (the bias grows after deployment), which is why monitoring and periodic re-evaluation are hard requirements in serious frameworks.

FLOPs (floating-point operations)

The count of elementary arithmetic operations performed during training — the field’s standard measure of how much compute a model consumed. Regulators adopted it because it is estimable and comparable: the EU AI Act presumes systemic risk above 10²⁵ FLOPs (Art 51(2)), and the rescinded US EO 14110 used 10²⁶ for reporting. Cautions: FLOPs measure effort, not capability; algorithmic efficiency shifts the capability-per-FLOP curve every year; and thresholds invite training-just-below gamesmanship. (Distinguish FLOPS-with-capital-S, operations per second — a hardware speed, not a training total.)

EU AI Act Art 51(2)

framework convention

A treaty form that sets binding principles and leaves each party to implement through domestic law — the structure used for climate (UNFCCC) and tobacco control. The Council of Europe’s Framework Convention on AI, Human Rights, Democracy and the Rule of Law (opened for signature 5 September 2024) is the first binding international AI treaty, signed by the EU, US, UK and others; it commits parties to lifecycle risk management, transparency, remedies and oversight, but contains significant carve-outs (national security; discretion over private-sector coverage) and no strong enforcement organ. Its weight is normative: a floor beneath diverging national regimes.

CoE CETS No. 225 (2024)

functional adaptability

One of the AI-specific additions ISO/IEC 25059 makes to the SQuaRE quality model: the degree to which the system can operate acceptably in contexts and on data beyond those it was originally specified for. It gives engineering vocabulary to a governance obsession — distribution shift and out-of-scope use — by making “how far can this stretch” an evaluable characteristic with acceptance criteria, rather than a hope. Paired with a declared intended purpose, it defines the honest envelope of the system.

ISO/IEC 25059

fundamental rights

In EU law, the rights codified in the Charter of Fundamental Rights: human dignity, private life and data protection (Arts 7–8), non-discrimination (Art 21), effective remedy (Art 47), and more. The AI Act’s risk logic runs on them: practices are prohibited or high-risk according to their threat to health, safety, or fundamental rights, and deployers of certain systems owe fundamental rights impact assessments. The framing distinguishes the EU approach from pure safety regulation — a system can be perfectly accurate and still unlawful because of what it does to rights.

EU Charter of Fundamental Rights

gap analysis

Where every implementation project starts: map what the organisation actually does today against what the target framework requires — clause by clause for ISO/IEC 42001, function by function for the NIST AI RMF, article by article for the EU AI Act — and record the deltas as a costed, owned remediation plan. Done honestly, it surfaces the uncomfortable findings (shadow AI, unowned models, absent documentation) that glossier maturity assessments smooth over. Its output feeds the Statement of Applicability and the certification-readiness decision.

GDPR (General Data Protection Regulation)

Regulation (EU) 2016/679 — the law that made Brussels effect a phrase. For AI it supplies the substrate the AI Act builds on: a lawful basis for every processing of personal data (including training), purpose limitation and data minimization, DPIAs for high-risk processing, Art 22 limits on solely automated decisions, and fines to 4% of global turnover. The two regimes stack rather than replace: an AI system can be fully AI-Act-conformant and still unlawful under GDPR — training-data legality, the SCHUFA ruling on credit scores, and Clearview’s fines are all GDPR stories.

Regulation (EU) 2016/679

Gender Shades

Joy Buolamwini and Timnit Gebru’s MIT study evaluated three commercial gender classifiers on a skin-type-balanced benchmark and found error rates of ~0.8% for lighter-skinned men versus up to ~34.7% for darker-skinned women — a chasm invisible in aggregate accuracy. Its impact outran its findings: vendors improved within months (proving the audit lever works), IBM and others later exited facial recognition, and disaggregated testing became the expected evaluation practice. It is the standing rebuttal to any benchmark reported as a single number.

Buolamwini & Gebru (2018)

governance vs management

The distinction ISO/IEC 38507 exists to enforce. Governance — the governing body’s work — evaluates, directs, and monitors: sets risk appetite, allocates resources, demands assurance (EDM). Management — top management downward — plans, implements, operates, and reports. The failure modes are symmetric: boards that manage (approving individual model releases) lose the independence oversight requires; boards that neither direct nor monitor delegate accountability they legally cannot shed. ISO/IEC 42001 speaks to management; 38507 speaks to the board — organisations need both layers working.

ISO/IEC 38507

governing body

ISO’s deliberately form-neutral term (ISO/IEC 38507) for whoever holds ultimate accountability: a corporate board, a public-agency head, university trustees, partners. AI governance frameworks route to it on purpose — the governing body sets AI risk appetite, receives escalations and assurance, and cannot delegate accountability even when it delegates everything else. Distinct from top management (the executives running the management system): the board directs and monitors them. Fiduciary law (Caremark duties) is what puts teeth behind the diagram.

ISO/IEC 38507

hard law

Rules that bind: violate them and a court or regulator can compel, fine, or ban. The EU AI Act, GDPR, Colorado’s AI Act, and China’s deep-synthesis rules are hard law; the OECD Principles, UNESCO Recommendation, and the NIST RMF are soft law. The boundary moves constantly in AI governance — soft instruments harden (voluntary commitments become codes of practice become statutory duties), and hard law borrows soft-law content (state statutes citing the NIST RMF as a safe harbor). The strategic question for organisations is not binding or not but how fast is this instrument hardening.

harm

The terminus of the risk vocabulary: a hazard is a potential source, risk is likelihood × severity, harm is the realised damage — physical injury, financial loss, rights violations, psychological damage, societal effects like eroded trust. Taxonomies slice it by type (the NIST GenAI Profile’s harm families), by mechanism (allocative vs representational), and by scale (individual → group → societal). Legal thresholds attach to it: the EU AI Act’s manipulation prohibitions require (reasonable likelihood of) significant harm, and tort law requires provable damage. Defining which harms count, to whom, is where governance stops being technical.

ISO/IEC Guide 51

harmonised standard

The bridge between the AI Act’s essential requirements and engineering practice: a European standard produced by CEN/CENELEC under a Commission standardisation request, which — once its reference is cited in the Official Journal — gives users a presumption of conformity with the requirements it covers. (British spelling of the same concept as harmonized standard.) For AI, none are yet cited as of September 2026: prEN 18286 on quality management is the furthest along, so providers currently work from the Act’s text directly — check current status.

EU AI Act Art 40; Regulation (EU) 1025/2012

harmonised standards

The family of European standards that, once cited in the Official Journal, let providers demonstrate conformity with EU legislation by presumption rather than first-principles argument. For the AI Act, the Commission’s standardisation request C(2023)3215 tasked JTC 21 with roughly ten deliverable areas — risk management, data governance and quality, logging, transparency, human oversight, accuracy, robustness, cybersecurity, quality management, conformity assessment. As of September 2026 the set remains in development with none yet cited; the gap between legal deadlines and standards readiness has been a central implementation controversy.

EU AI Act Art 40

harmonized standards

US-spelling plural of harmonized standard: the standards that operationalise the New Legislative Framework bargain — the law states outcome-level essential requirements, standards bodies supply the technical detail, and applying a cited standard yields the presumption of conformity. The model made EU product law scale across sectors from toys to lifts; the AI Act is its most ambitious application yet, and the slowest to deliver — no AI Act standards were cited as of September 2026.

EU AI Act Art 40

hazard

Safety-engineering vocabulary (ISO/IEC Guide 51) imported into AI risk work: a potential source of harm. For an AI hiring tool, hazards include biased training labels, proxy features, distribution shift, and rubber-stamp review; each becomes a risk once you attach likelihood and severity, and a harm if it lands on someone. Hazard identification is the first, generative step of risk assessment — checklists like the NIST GenAI Profile’s twelve risk categories or MIT’s risk repository exist so teams do not have to brainstorm from zero.

ISO/IEC Guide 51

high-risk AI

The generic concept behind the EU’s defined term: AI used in contexts where errors or misuse carry serious consequences for people — hiring, credit, medical devices, policing, critical infrastructure. Risk-based regimes converge on similar lists but different mechanics: the EU AI Act enumerates them (Annex III plus Annex I safety components) and attaches a full pre-market rulebook; Colorado’s AI Act regulates “high-risk” systems making consequential decisions via a duty of reasonable care; Canada’s abandoned AIDA used “high-impact”. The shared logic: proportionality — regulate where stakes are high, leave chess apps alone.

historical bias

The bias that exists before any sampling or measurement error: the training data faithfully records a world shaped by discrimination, so the model learns it as signal. Amazon’s scrapped CV-screening tool is the canon example — trained on a decade of male-dominated hiring outcomes, it penalised the word “women’s”. Historical bias cannot be fixed by collecting more of the same data; it forces a normative decision — reproduce the world as it was, or constrain the model toward the world as policy says it should be. That decision is what fairness interventions encode.

NIST SP 1270

horizontal regulation

Regulating the technology once, everywhere: a single instrument whose rules apply to AI in hiring, health, finance, and toys alike, calibrated by risk rather than by industry. The EU AI Act is the archetype; Colorado’s AI Act is a state-level cousin. The trade-offs against sectoral regulation are stable: horizontal rules give coherence and close gaps between sectors, but strain to fit every context (the same Article 10 governs cancer screening and credit scoring) and require heavy secondary guidance. Most real regimes end up hybrid — a horizontal floor plus sectoral overlays.

IAF MLA (International Accreditation Forum Multilateral Recognition Arrangement)

The peer-review pact among national accreditation bodies: signatories evaluate each other against ISO/IEC 17011, and certificates issued under any signatory’s accreditation are recognised by all — “accredited once, accepted everywhere.” For AI it is the mechanism that makes ISO/IEC 42001 certification globally portable: once 42006-based accreditation programs sit under the MLA umbrella, a certificate from an accredited body in Singapore carries the same formal weight in Frankfurt. It is also the quality backstop — bodies that certify carelessly risk their accreditation, not just their reputation.

IAF MLA; ISO/IEC 17011

impact assessment

Governance’s favourite verb made into a noun: systematically ask who could this affect, how badly, and what will we do about it — and write the answer down before launch. The family is large: DPIAs (GDPR), FRIAs (EU AI Act Art 27), algorithmic impact assessments (Canada’s directive), AI system impact assessments (ISO/IEC 42005), and state-law versions (Colorado). The common DNA: identify affected persons, enumerate harms, evaluate necessity and proportionality, plan mitigations, assign owners, and revisit on change. The common failure: doing it once, filing it, and never letting it alter the design.

ISO/IEC 42005; EU AI Act Art 27

implementation timeline (EU AI Act)

The Act entered into force 1 August 2024 but applies in waves: 2 February 2025 — prohibitions and AI-literacy duties; 2 August 2025 — GPAI obligations, governance bodies, penalties; then the high-risk regime, originally 2 August 2026/2027, rescheduled by the Digital Omnibus package to 2 December 2027 (Annex III) and 2 August 2028 (Annex I), largely because harmonized standards were not ready. Grandfathering applies to systems already on the market, with triggers on substantial modification. Treat every date here as check-current-status material: the timeline has already moved once.

EU AI Act Art 113; Digital Omnibus (2025)

impossibility theorem (fairness)

The mathematical result (Kleinberg, Mullainathan & Raghavan 2016; Chouldechova 2017) crystallised by the COMPAS fight: except in degenerate cases, a risk score cannot simultaneously be calibrated within each group and have equal false-positive and false-negative rates across groups when the groups’ base rates differ. Its governance import is liberating rather than defeatist: since no model can satisfy every fairness definition, someone must choose and justify which one fits the context — and that choice belongs in documented, accountable policy, not in a data scientist’s default settings.

Kleinberg et al. (2016); Chouldechova (2017)

incident-driven regulation

The empirical rhythm of tech governance: a harm becomes a headline, the headline becomes a hearing, the hearing becomes a statute. Cambridge Analytica accelerated GDPR enforcement culture; the toeslagenaffaire hardened the EU’s stance on social scoring and profiling; NYC’s hiring-audit law followed the resume-screening scandals; deepfake statutes followed specific viral incidents. The pattern’s cost is the pacing problem in reverse — rules optimised for the last scandal — and its benefit is political feasibility: incidents create the coalition that abstract risk analysis never does. Databases like the AI Incident Database exist to shorten the distance from pattern to policy.

integrated management system

The organisational strategy the ISO Harmonized Structure (Annex SL) was designed to enable: because every modern management-system standard shares the same clause skeleton (context, leadership, planning, support, operation, evaluation, improvement), an organisation can run one integrated system — one policy hierarchy, one risk method, one internal-audit programme, one management review — with discipline-specific extensions for security (27001), privacy (27701), and AI (42001). Benefits: less duplication, coherent risk pictures, and eligibility for combined audits. Cost: integration debt if the disciplines are owned by silos that do not talk.

ISO Annex SL

internal audit

ISO clause 9.2: the organisation must audit its own management system at planned intervals, with auditors objective and impartial (not auditing their own work), documented criteria and scope, and results reported to relevant management. In the three lines of defense model it is the third line. For an AIMS this means someone internal walks the same ground an external auditor will — sampling AI risk assessments, testing whether controls in the Statement of Applicability actually operate — before certification or surveillance visits. Weak internal audit is the most reliable predictor of external-audit surprises.

ISO/IEC 42001 Clause 9.2

international-first (standards policy)

The policy that keeps European and international AI standards from forking gratuitously. Under the Vienna/Frankfurt agreements and its own strategy, CEN-CENELEC JTC 21 first asks whether an ISO/IEC deliverable (from SC 42) can be adopted as an EN — as it did with the EN ISO/IEC 5259 data-quality series and 42001 — and drafts homegrown European standards only where the AI Act’s fundamental-rights requirements exceed what international consensus produced (e.g. prEN 18286 on quality management, trustworthiness). For companies, the payoff is convergence: implementing the ISO stack is mostly walking toward the EU stack.

CEN-CENELEC JTC 21 work programme

intervenability

A quality property in the ISO AI vocabulary family, closely paired with controllability: the extent to which the system exposes intervention points — reviewable intermediate outputs, override paths, pause and rollback mechanisms — that let an authorised human change the course of processing. Privacy engineering uses the same word for the ability to intervene in data processing (correct, delete, restrict). Either way it is a design-time property: an intervention capability added after deployment is usually an incident report describing why it was needed.

ISO/IEC 25059; ISO/IEC TS 8200

ISO/IEC 17021-1

The rulebook for the auditors: requirements certification bodies must meet to be accredited for management-system certification — organisational impartiality (no consulting your own audit clients), auditor competence management, the stage 1/stage 2 initial-audit structure, surveillance and recertification rules, and certificate decision-making separated from the audit team. For AI, ISO/IEC 42006 layers AIMS-specific competence and audit-time requirements on top of it. When you ask “who audits the auditors, and against what?” — this is the what.

ISO/IEC 17021-1:2015

ISO/IEC 27701

The privacy member of the management-system family: a Privacy Information Management System (PIMS) specification extending ISO/IEC 27001’s security base with controller and processor privacy controls mapped loosely to GDPR concepts. (Its 2025 revision makes it certifiable standalone.) In AI governance it slots beside 42001: the AIMS governs AI-specific risk, 27701 governs the personal data flowing through training and inference, 27001 secures the whole stack — commonly run as one integrated management system and certified in combined audits.

ISO/IEC 27701

ISO (International Organization for Standardization)

The global standards machine: ~170 national member bodies producing voluntary consensus standards through committees of delegated experts. For information technology ISO works jointly with the IEC through JTC 1, whose subcommittee SC 42 owns AI — home of ISO/IEC 22989 (concepts), 23894 (risk), 42001 (management systems), and the trustworthiness catalogue. ISO standards bind no one by themselves; they acquire force through contracts, procurement, certification, and — in the EU — adoption as harmonised standards. That indirect power is why seats at SC 42 tables are strategic assets.

JTC 1 (ISO/IEC Joint Technical Committee 1)

The 1987 pact that stopped ISO and the IEC writing rival computing standards: one joint committee for information technology, whose subcommittees own everything from character sets to cybersecurity (SC 27) to AI (SC 42, created 2017). Standards it produces carry the “ISO/IEC” double badge. Knowing the org chart pays practical dividends: it explains numbering, why AI standards inherit patterns from security standards developed one corridor over, and where national bodies must send experts to influence what future audits will measure. Not to be confused with CEN-CENELEC JTC 21, its European counterpart for AI.

JTC 21 (CEN-CENELEC Joint Technical Committee 21)

Europe’s AI standards committee (established 2021), executor of the Commission’s standardisation request C(2023)3215. Its working method is international-first (standards policy): adopt ISO/IEC SC 42 deliverables as European standards where they fit, and draft European-only ones where the AI Act demands more than international consensus provides — prEN 18286 on AI quality management systems being the flagship. Its output, once cited in the Official Journal, becomes harmonised standards carrying the presumption of conformity — which makes this obscure committee one of the most consequential rooms in AI governance. Distinct from ISO/IEC JTC 1.

Commission Implementing Decision C(2023)3215

key risk indicator (KRI)

The early-warning instrument of risk reporting: where a KPI measures performance achieved, a KRI measures exposure building — model-drift alerts open past SLA, human-override rates trending toward zero (the rubber-stamp signal), share of AI systems past review date, supplier concentration on critical models, incidents by severity. Good KRIs are chosen against the board’s stated risk appetite / risk tolerance, carry thresholds with pre-agreed actions, and appear on a fixed reporting cadence. A dashboard of KRIs nobody acts on is a compliance mural.

label bias

Bias injected at the annotation step: the label is a proxy for the thing you care about, measured unevenly. Classic case: using arrest as the label for crime — arrest rates reflect policing patterns, not just offending, so the model learns the pattern of enforcement. Healthcare’s version made Science in 2019: an algorithm used cost as the label for health need, and because less money is spent on Black patients at equal illness, it systematically understated their need. Label bias survives perfect sampling and infinite data — the fix is choosing better labels, not more of them.

Obermeyer et al., Science (2019)

liar’s dividend

Coined by law professors Bobby Chesney and Danielle Citron (2019): as synthetic media grows convincing, the value of denial rises — a politician caught on tape shouts “deepfake!”, and some share of the audience must take it seriously. The harm is epistemic and collective: not a specific fake believed, but authentic evidence devalued. It is why provenance infrastructure (C2PA, watermarking (AI content)) matters beyond fake-detection — proving the real real is the countermeasure — and why courts are already fielding deepfake objections to ordinary video evidence.

Chesney & Citron, Calif. L. Rev. (2019)

major nonconformity

The audit finding that stops the music: a total breakdown of a required element of the management system, or a failure that makes the system’s intended results doubtful — no risk assessment process at all, management review never performed, a control claimed in the Statement of Applicability that simply does not exist. Under ISO/IEC 17021-1 rules a certification body cannot issue or maintain a certificate while a major nonconformity stands open: the auditee must analyse the cause (root cause analysis), correct it, and usually submit to a follow-up or verification before the stage 2 audit result can be positive. Contrast a minor nonconformity, an isolated lapse that requires a corrective-action plan but does not block the certificate.

ISO/IEC 17021-1:2015 §9.4.5

management review

The heartbeat of every ISO management system: Clause 9.3 requires top management to review the AIMS at planned intervals against a fixed agenda — status of previous actions, changes in context, nonconformities, audit results, monitoring data, and opportunities for improvement — and to output real decisions: change the system, allocate resources, adjust objectives. Auditors treat the minutes as prime evidence: a review that no executive attended, or that produced no decisions, signals a paper AIMS. It is deliberately different from operational governance meetings — it reviews the system itself, not individual AI projects.

ISO/IEC 42001:2023 Clause 9.3

many hands problem

A classic problem of collective responsibility (named by Dennis Thompson, 1980) that AI industrialises: a harmful outcome emerges from a data vendor, a model developer, a fine-tuner, an integrator, a deploying manager, and an end user — each of whom can truthfully say my part worked as specified. The result is a responsibility gap: everyone contributed, no one is answerable. Governance frameworks attack it by assigning roles up front — the EU AI Act’s provider/deployer duties, ISO/IEC 42001’s role determination controls, contracts allocating liability along the value chain — so that when harm occurs, accountability was decided before the finger-pointing starts.

Thompson, “Moral Responsibility of Public Officials” (1980)

market surveillance authority

The enforcement muscle of EU product law. Each Member State designates one or more authorities under the Market Surveillance Regulation (EU) 2019/1020; the AI Act plugs into that machinery (Art 74) and gives these authorities teeth against AI systems: full access to documentation and training data where needed, power to compel corrective action, withdrawal, or recall of non-compliant high-risk systems, and the ability to trigger the fine regime. Affected persons can lodge complaints with them (Art 85). Not to be confused with the national competent authority umbrella term or the notifying authority that oversees notified bodies.

EU AI Act Arts 74–85; Regulation (EU) 2019/1020

memorisation

The failure mode where a model does not generalise from a training example but effectively stores it — and can be prompted to emit it: a person’s phone number, a chunk of licensed code, a paragraph of a news article. Carlini et al. showed extraction attacks recovering training text from LLMs, and memorisation grows with model size, data duplication, and example uniqueness. It matters legally on two fronts: privacy (memorised personal data turns a model into a leaky database, complicating GDPR erasure rights) and copyright (verbatim regurgitation is the strongest evidence in infringement suits against model providers). Mitigations — deduplication, differential privacy, output filters — trade capability for containment.

Carlini et al., “Extracting Training Data from LLMs” (2021)

minor nonconformity

The lesser audit finding: a single observed lapse in an otherwise functioning element — one AI system missing from the inventory, one training record absent, a review performed late — that does not cast doubt on the management system as a whole. The certificate can still be issued or maintained, but the organisation must submit a corrective-action plan with root-cause analysis, and the auditor verifies closure at the next surveillance audit. The practical danger is accumulation: a cluster of minors in the same clause is evidence of systemic failure, and auditors can and do escalate a pattern of minors into a major nonconformity.

ISO/IEC 17021-1:2015 §9.4.5

model collapse

The degenerative feedback loop identified as AI-generated content floods the web: when generation N+1 trains on text and images produced by generation N, the tails of the true data distribution get progressively forgotten — rare styles, minority dialects, unusual facts vanish first — and artefacts amplify (Shumailov et al., Nature 2024). It converts synthetic content labelling and provenance from a consumer-protection issue into an ecosystem issue: knowing what is machine-made is what lets future trainers filter it out. A systemic risk with no single owner — the commons that every scraper both depends on and pollutes.

Shumailov et al., Nature 631 (2024)

model stealing

An extraction attack in which an adversary queries a deployed model and uses the input-output pairs to train a surrogate that clones its behaviour — or, in stronger variants, recovers actual parameters (researchers have extracted embedding-layer weights from production LLMs through public APIs). The stolen copy costs a fraction of the original training run, evades usage controls, and serves as a rehearsal space for crafting adversarial attacks against the original. It sits in every AI threat taxonomy — MITRE ATLAS, the OWASP LLM Top 10 — and drives defensive controls: rate limiting, query-pattern anomaly detection, and watermarking of outputs. Distinct from theft of model weights by exfiltration, which is classic infosec.

MITRE ATLAS; OWASP Top 10 for LLM Applications

model weights

The billions of floating-point numbers a training run produces: everything the model “knows” lives in them. Governance treats weights as a special asset class for one reason — irreversibility. Once weights leak or are published, every downstream safeguard (usage policies, API filters, monitoring) becomes optional for whoever holds them, and fine-tuning away safety training costs almost nothing. Hence the policy fights: open-weight releases (Llama, Mistral) versus API-only access; the US Executive Order 14110’s reporting focus on dual-use weights; security expectations in responsible scaling policies about protecting weights from theft by state actors (RAND’s “Securing AI Model Weights” defines the threat levels). The EU AI Act’s open-source carve-outs also hinge on whether weights are publicly available.

RAND, “Securing AI Model Weights” (2024)

monoculture risk

Borrowed from agriculture: plant one crop everywhere and a single blight takes out the harvest. When thousands of downstream applications — hiring screens, credit models, medical triage tools — are built on the same handful of foundation models, they inherit the same blind spots, biases, and vulnerabilities, and their failures become correlated: a person rejected by one model may be rejected by all of them (algorithmic monoculture, Kleinberg & Raghavan 2021), and a single jailbreak or poisoned dependency propagates economy-wide. This is a key argument for treating GPAI as a source of systemic risk (GPAI), and it converts diversity of suppliers from a procurement nicety into a resilience control.

Kleinberg & Raghavan, PNAS 118 (2021)

multi-site sampling

The certification-economics rule that makes auditing a 40-office company feasible: where sites perform similar activities under one management system, the certification body may audit a sample (classically √n-based, per IAF MD 1) rather than every location. ISO/IEC 42006 tightens this for AI: sites are only “similar” if they develop, provide, or use AI in comparable ways — a lab fine-tuning models and a sales office that merely uses a chatbot cannot stand in for each other — and higher-risk AI activity at a site pulls it into the mandatory sample. The design question for the auditee is honest scoping: gerrymandered “similarity” is a classic path to a certificate that means nothing.

IAF MD 1; ISO/IEC 42006:2025

National AI Initiative Act

The statute (Division E of the FY2021 NDAA, signed January 2021) that gave US federal AI policy a skeleton: it created the National AI Initiative Office, chartered the National AI Advisory Committee, funded the National AI Research Institutes — and, in §5301, directed NIST to develop a voluntary risk management framework for trustworthy AI. That single directive produced the NIST AI RMF 1.0 (2023), the closest thing the US has to a national AI governance baseline. The Act embodies the US posture: coordinate, fund, and standardise rather than regulate — a voluntary framework strategy in statutory form.

National AI Initiative Act of 2020, Pub. L. 116-283 Div. E

national competent authority

The EU AI Act’s collective noun for national enforcers: each Member State must establish or designate at least one notifying authority (which vets and monitors notified bodies) and at least one market surveillance authority (which polices systems on the market), and equip them with adequate resources and AI expertise (Art 70). Member States choose their own institutional design — a new AI agency, the data protection authority, sectoral regulators, or a mix — which is why “who enforces the AI Act” has 27 different answers, loosely coordinated through the European AI Board. For GPAI models, enforcement sits instead with the Commission’s AI Office.

EU AI Act Art 70

NCII (non-consensual intimate imagery)

The harm category covering intimate images created or shared without the subject’s consent — including entirely synthetic ones. Generative tools collapsed the cost: “nudify” apps fabricate sexual imagery of real, identifiable people from ordinary photos, and studies of deepfake content have consistently found the overwhelming majority is non-consensual sexual imagery of women. Law has moved fast by regulatory standards: the US TAKE IT DOWN Act (2025) criminalises publication of NCII including AI-generated “digital forgeries” and forces platforms to remove it within 48 hours of notice; the UK Online Safety Act and the EU directive on violence against women criminalise it in parallel. For governance teams it is a canonical misuse case: foreseeable, catastrophic for victims, and squarely the provider’s problem to design against.

TAKE IT DOWN Act (2025); UK Online Safety Act 2023

NIST GenAI Profile

The generative-AI supplement to the AI RMF, published July 2024 under Executive Order 14110 as NIST AI 600-1. Its lasting contribution is a shared vocabulary of twelve risks “novel to or exacerbated by” GenAI — CBRN information access, confabulation, dangerous or violent recommendations, data privacy, environmental impacts, harmful bias and homogenization, human-AI configuration, information integrity, information security, intellectual property, obscene/degrading content, and value chain integration — each mapped to suggested actions across Govern/Map/Measure/Manage. Because it is a cross-sectoral RMF profile, organisations use it as a checklist generator: pick the risks that apply, inherit the actions. Voluntary, like everything NIST — but rapidly becoming the de facto US baseline for GenAI risk taxonomies.

NIST AI 600-1 (2024)

normative clause

The unit of obligation inside a standard. ISO/IEC drafting rules assign exact force to verb forms: “shall” marks a requirement (auditable, certifiable), “should” a recommendation, “may” a permission, “can” a possibility. Only normative clauses can generate nonconformities in an audit — which is why the first professional skill in reading any standard is separating its normative spine from the informative padding around it (normative vs informative). The same grammar governs legal instruments: knowing which sentences bind is the difference between a compliance obligation and a nice idea.

ISO/IEC Directives Part 2

normative vs informative

The load-bearing distinction for reading standards. Normative content states what you must do to claim conformity — the “shall” clauses and any annex labelled normative, like ISO/IEC 42001’s Annex A control list as invoked through the Statement of Applicability. Informative content — notes, examples, bibliographies, annexes labelled informative, and whole documents like Technical Reports — explains, illustrates, and advises but binds no one. The trap for practitioners runs both ways: treating informative guidance as mandatory gold-plates a compliance programme, while missing that an annex is normative leaves a certification-blocking hole. Check the label on every annex; never assume.

ISO/IEC Directives Part 2

pacing problem

The structural mismatch between innovation cycles measured in months and legislative cycles measured in years (the term is associated with Gary Marchant’s work on law and emerging technologies). The EU AI Act is the canonical exhibit: drafted around 2021-era product-safety assumptions, it had to be rewritten mid-negotiation when ChatGPT arrived. Every AI governance design choice is partly an answer to the pacing problem: technology-neutral drafting, delegated acts and annexes the executive can update, sandboxes, standards that revise faster than statutes, and voluntary frameworks that can ship immediately. The failure modes are symmetric — regulate too early and you freeze the wrong taxonomy into law; too late and harms compound uninsured.

Marchant, The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight (2011)

penalty tiers

The AI Act’s enforcement price list (Art 99), graduated by gravity: prohibited practices (Art 5) draw fines up to €35 million or 7% of global annual turnover, whichever is higher — deliberately above the GDPR’s 4% ceiling; breaches of most other obligations (provider, deployer, importer duties) up to €15m or 3%; supplying false or misleading information to authorities up to €7.5m or 1%. For SMEs and startups the lower of the two figures applies — a rare pro-small-company inversion. GPAI providers face a separate Commission-enforced regime of up to 3% of turnover or €15m (Art 101). The tiering is the risk pyramid translated into money: the fine tells you how seriously the legislator ranked the rule you broke.

EU AI Act Arts 99–101

preemption

The doctrine (from the US Constitution’s Supremacy Clause) that federal law can displace state law — expressly, by occupying the field, or through direct conflict. In AI it is the live wire of US policy: with no comprehensive federal AI statute, states filled the vacuum (Colorado’s AI Act, Illinois BIPA, California’s ADMT rules), and industry has repeatedly pushed for federal preemption of the resulting patchwork — including a proposed 10-year moratorium on state AI enforcement that the Senate stripped from the 2025 budget bill by a 99–1 vote. The stakes cut both ways: preemption promises one rulebook instead of fifty, but if the federal standard is weaker than the state laws it erases, it functions as deregulation wearing harmonisation’s clothes.

US Const. Art VI cl 2

prEN 18286

One of the make-or-break deliverables of the European standardisation programme: a draft European standard (prEN) on AI risk management developed by CEN-CENELEC JTC 21 in answer to the Commission’s standardisation request, designed so that conformity with it will create a presumption of conformity with the AI Act’s risk-management requirements (Art 9) once cited in the Official Journal as a harmonised standard. The “pr” prefix means it is still a proposal — subject to public enquiry, comment resolution, and formal vote before becoming EN 18286. Watch it because it answers the practical question every high-risk provider asks: what exactly will an auditor accept as an Article 9 risk management system?

CEN-CENELEC JTC 21 work programme; EU AI Act Art 40

principle proliferation

The phase of AI governance history when everyone published principles: by 2019 researchers had catalogued 84+ separate AI ethics documents (Jobin et al. counted 84; AlgorithmWatch’s inventory passed 160) from companies, governments, churches, and consultancies. Meta-analyses found genuine convergence — transparency, fairness, non-maleficence, accountability, privacy recur almost everywhere — but the documents shared a defect: no mechanism, no monitoring, no consequences. The proliferation era matters as diagnosis: it proved consensus on values was never the bottleneck, and it fuelled the “ethics washing” critique that pushed the field from principles to practice — management systems, audits, and binding law.

Jobin, Ienca & Vayena, Nature Machine Intelligence 1 (2019)

provenance

The record of origin and chain of custody — applied at two levels in AI governance. For content: cryptographically signed metadata (the C2PA “Content Credentials” standard) travels with an image or video, recording what device or model created it and every edit since; NIST AI 100-4 and the EU AI Act’s Art 50 machine-readable marking duty both lean on this. For data: knowing the source, licence, and processing history of every training dataset — the substance of data lineage and the EU AI Act’s Art 10 duties. Provenance is honest about its limits: it authenticates the label, not the truth of the content, and it only works where the capture-to-display chain preserves signatures — strip the metadata and the provenance is gone.

C2PA specification; NIST AI 100-4

public enquiry

The democratic valve in the standardisation pipeline: once a CEN or CENELEC technical committee agrees a draft (the prEN stage — see prEN 18286), national standards bodies publish it for a set consultation window and anyone — companies, NGOs, academics, citizens — may submit comments, which the committee must formally disposition before the final vote. For AI standards the enquiry stage carries extra weight: harmonised standards will effectively define what AI Act compliance means, yet the committees that draft them are dominated by industry — the public enquiry is one of the few moments civil society can intervene. Practitioners track enquiry windows the way lawyers track comment periods on proposed rules.

CEN-CENELEC Internal Regulations Part 2

quality in use

The outermost layer of the SQuaRE quality framework (SQuaRE series): while a product quality model describes properties of the system itself (accuracy, robustness, maintainability), quality in use measures what happens when a specific user applies it in a specific context — effectiveness, efficiency, satisfaction, freedom from risk. The distinction is where AI governance lives or dies: a diagnostic model with excellent benchmark accuracy (product quality) can still produce terrible quality in use if clinicians over-trust it, the interface buries uncertainty, or the deployment population differs from the test set. Regulators increasingly test at this layer — the EU AI Act’s human-oversight and deployer duties are quality-in-use requirements in legal dress.

ISO/IEC 25010; ISO/IEC 25059

quality model

The standards answer to a vague word: instead of arguing about whether a system is “good”, a quality model decomposes quality into named characteristics and sub-characteristics you can specify, measure, and contract for. ISO/IEC 25010 does this for software (functional suitability, reliability, security, usability…); ISO/IEC 25059 extends it for AI systems, adding AI-specific characteristics like functional adaptability (the system’s capacity to learn) and elevating robustness and controllability. The practical payoff is shared vocabulary: when a procurement contract says “accuracy ≥ X, robustness per clause Y”, both sides mean the same thing — and test reports, validation plans, and acceptance criteria hang off the model’s taxonomy rather than everyone’s private adjectives.

ISO/IEC 25059:2023; ISO/IEC 25010

real-world testing

The AI Act’s admission that labs cannot tell you everything: Arts 60–61 let providers test high-risk systems in real-world conditions before placing them on the market — outside or inside a regulatory sandbox — but wrap the permission in safeguards: a registered real-world testing plan, market-surveillance-authority oversight, informed consent from subjects (with narrow law-enforcement exceptions), a maximum duration (six months, extendable once), the right of subjects to withdraw, and the ability to reverse or disregard the system’s outputs. Effects on subjects must be reversible or mitigable. It is the product-law cousin of a clinical trial — evidence generation under protocol, not a soft launch with a disclaimer.

EU AI Act Arts 60–61

recertification

The end-of-cycle reckoning: an ISO management-system certificate is valid for three years, punctuated by annual surveillance audits, and before it expires the certification body must conduct a recertification audit reviewing the entire system — like a stage 2 audit but informed by three years of surveillance history, changes to the organisation, and the system’s track record of nonconformities and complaints. For an AIMS the recertification lens is change: the AI portfolio, the models, the regulatory landscape, and the risk picture in year three will not be the ones certified in year zero, and the auditor’s core question is whether the management system kept up. Miss the deadline and the certificate lapses — there is no grace period for trust.

ISO/IEC 17021-1:2015 §9.6

recital

The preamble paragraphs before an EU regulation’s articles — the AI Act has 180 of them — explaining what the legislator meant and why. Recitals are not independently binding: they cannot create obligations the articles lack (CJEU settled law). But they are far from decoration — courts and regulators read ambiguous articles through them, and some of the AI Act’s most consequential guidance lives only there: recital 12 unpacks the definition of “AI system”, recitals on subliminal techniques flesh out what manipulation means, and employment-related recitals expand on Annex III categories. Professional rule: cite the article for the obligation, the recital for the interpretation — and when a vendor quotes only a recital at you, check what the article actually says.

Joint Practical Guide of the EU Institutions §10

regulation

A word with a precise EU meaning and a loose everyday one. In the EU legal order (TFEU Art 288), a regulation is directly applicable law: it binds in its entirety across all Member States without any national implementing statute — the GDPR and the AI Act are regulations, which is why the same articles apply verbatim in Dublin and Warsaw. Contrast a directive, which sets results and lets each state legislate its own path (regulation vs directive). In everyday and US usage, “regulation” means any binding rule made by a government agency under delegated power. When a text says “the regulation requires…”, first establish which sense is in play — the answer changes who enforces it and where the definitive text lives.

TFEU Art 288

regulation vs directive

The choice that determines how uniform EU law is on the ground. A regulation applies directly and identically everywhere (AI Act, GDPR); a directive binds Member States only as to the result, leaving form and method to national parliaments — producing 27 transpositions with real variation (the Product Liability Directive, the NIS2 Directive). The choice for the AI Act was deliberate and telling: the Commission chose a regulation precisely to prevent a patchwork of 27 national AI regimes fragmenting the single market — the same logic as the GDPR, which replaced a 1995 directive whose divergent transpositions had become the problem. Rule of thumb: with a regulation you read the EU text; with a directive you must also read your country’s implementing law.

TFEU Art 288

representational harm

The other half of the harms taxonomy alongside allocative harm (Kate Crawford, 2017): injury done through representation — a search for “CEO” returning only white men, image models sexualising women of particular ethnicities, translation systems defaulting doctors to he and nurses to she, face analysis failing darker-skinned women (Buolamwini & Gebru’s Gender Shades), or a group being erased from outputs entirely. No loan is denied, so classic anti-discrimination law struggles to grip it — yet the harm compounds culturally, shaping how millions perceive a group and feeding future training data. Generative AI massively expands the surface: every generated image and sentence is a representation. The NIST GenAI Profile captures it under “harmful bias and homogenization”.

Crawford, “The Trouble with Bias” (NeurIPS 2017)

representativeness

The property that decides whether performance measured in the lab means anything in the field: a dataset is representative when its composition matches the persons, settings, and conditions of the intended use context — demographics, geographies, devices, edge cases and all. The EU AI Act makes it law: Art 10(3) requires training, validation and testing datasets for high-risk systems to be “relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose”. Those last five words do the work — representativeness is always relative to a deployment context, which is why a model validated on one hospital’s patients can fail at the next hospital, and why sampling bias is its characteristic failure mode.

EU AI Act Art 10(3)

responsible scaling

The frontier labs’ self-regulatory bargain: keep scaling if and only if safety and security measures keep pace. A responsible scaling policy (Anthropic’s RSP with its AI Safety Levels, OpenAI’s Preparedness Framework, DeepMind’s Frontier Safety Framework) defines capability thresholds that would make a model dangerous — meaningful uplift for bioweapons development, autonomous replication, advanced cyber-offence — commits to evaluating each new model against them, and pre-commits to responses: harden model weights security, restrict deployment, or pause training until safeguards catch up. The sixteen-company Seoul Frontier AI Safety Commitments (2024) generalised the pattern. The critique writes itself — the labs grade their own homework — which is why safety institutes and the EU’s GPAI code of practice are pulling the same structure into public frameworks.

Anthropic RSP (2023); Seoul Frontier AI Safety Commitments (2024)

rights-based approach

One of the two grand design philosophies of AI regulation. A rights-based approach starts from entitlements held by persons — dignity, privacy, non-discrimination, due process — and asks what any system must do to respect them; protection does not depend on a risk calculation coming out right. Its instruments are prohibitions, individual rights (explanation, contest, remedy), and impact assessments framed around fundamental rights, as in the Council of Europe’s AI Framework Convention or a FRIA. Contrast risk-based regulation, which scales duties to estimated harm and accepts residual risk by design. Real regimes hybridise: the EU AI Act is risk-based in architecture but rights-based at its edges — the Art 5 prohibitions and deployer FRIAs are rights logic embedded in a risk pyramid.

Council of Europe Framework Convention on AI (2024)

risk

The most load-bearing and least agreed word in the field. ISO 31000 defines risk as “the effect of uncertainty on objectives” — deliberately neutral, capturing upside and downside, and organisation-centred. The EU AI Act (Art 3(2)) defines it as “the combination of the probability of an occurrence of harm and the severity of that harm” — strictly negative and person-centred: the harms that count are to health, safety, and fundamental rights, not to your revenue. A risk assessment built on the ISO definition can be structurally blind to what the AI Act cares about (an enterprise-risk register full of project and reputation risks, empty of harms to affected persons). Professionals name their definition before they argue about the number.

ISO 31000:2018 §3.1; EU AI Act Art 3(2)

risk assessment

The engine room of every risk framework, standardised by ISO 31000 as three sequenced activities: identification (what could happen — misuse, failure, bias, drift, security compromise, per source and lifecycle stage), analysis (how likely, how severe, for whom — using techniques from ISO/IEC 23894’s AI-specific catalogue), and evaluation (compare the analysed risk against risk criteria to decide whether it needs treatment). AI strains each step: novel failure modes defeat checklist identification, probability estimates for emergent behaviours are guesses wearing decimal points, and severity must be assessed for affected persons who never appear in enterprise registers — which is why ISO/IEC 42001 pairs risk assessment with impact assessment rather than trusting either alone.

ISO 31000:2018 §6.4; ISO/IEC 23894:2023

risk-based regulation

The design philosophy that says regulatory attention is scarce and should be spent where harm concentrates: classify activities by risk, then calibrate obligations to the class — prohibit the unacceptable, heavily regulate the high-risk, lightly touch the rest. The EU AI Act is its purest AI expression (the four-tier risk pyramid); proportionality is its selling point to industry, and coherence with product-safety tradition its selling point to regulators. Its known failure modes: everything depends on classifying correctly at the border (misclassify and the whole machine misfires); risk is assessed ex ante by category rather than by actual behaviour; and harms that diffuse across society — monoculture risk, epistemic erosion — fit poorly in any single tier. The main rival philosophy is the rights-based approach.

EU AI Act recitals 26–27

risk criteria

The rules of the game, fixed before play begins: terms of reference against which the significance of a risk is evaluated (ISO 31000). Good criteria specify how likelihood and consequence are measured and combined, what counts as intolerable versus acceptable, whose harms count (the organisation’s only, or affected persons’ too — the choice ISO/IEC 42001 forces), and who has authority to accept residual risk at each level. Their governance function is anti-gaming: criteria set in advance prevent the quiet recalibration where an inconvenient red risk becomes amber because the launch date is close. Auditors read the criteria first — a risk register without documented criteria is a list of opinions.

ISO 31000:2018 §6.3.4; ISO/IEC 42001:2023 Clause 6.1.1

risk management

Not a document but a loop: ISO 31000 defines it as coordinated activities to direct and control an organisation with regard to risk — establish scope and risk criteria, perform risk assessment, treat the risks (mitigate, transfer, avoid, or accept), then monitor, review, communicate, and record — continuously, because AI systems, their contexts, and their attackers all change after deployment. ISO/IEC 23894 translates this into AI-specific guidance; the NIST AI RMF reorganises the same substance into Govern/Map/Measure/Manage; the EU AI Act hardens it into the mandatory Art 9 risk management system. The discipline’s core honesty: risk is managed, never eliminated — the output is a defensible record of what you accepted and why.

ISO 31000:2018; ISO/IEC 23894:2023

risk management system

Where risk management stops being best practice and becomes law. Article 9 requires providers of high-risk systems to establish, implement, document, and maintain a risk management system as a continuous iterative process across the entire lifecycle: identify reasonably foreseeable risks to health, safety, and fundamental rights — under intended use and reasonably foreseeable misuse — estimate and evaluate them (including from post-market data), and adopt measures in a fixed order of preference: eliminate by design first, mitigate and control second, inform and train deployers last. Residual risks must be judged acceptable, testing must prove it, and vulnerable groups (children especially) get explicit attention. prEN 18286 is being drafted to give this article a presumption-of-conformity standard.

EU AI Act Art 9

risk pyramid

The mental model that organises the entire AI Act. Four tiers, obligations shrinking as you descend: unacceptable risk — the Art 5 prohibited AI practices, banned outright; high risk — Annex I safety components and Annex III use cases, carrying the full requirements catalogue (risk management, data governance, oversight, conformity assessment); limited/transparency risk — chatbots, emotion recognition, deepfakes, owing Art 50 disclosure duties; minimal risk — the vast majority of AI (spam filters, game AI), left to voluntary codes. GPAI models sit awkwardly beside the pyramid in their own chapter, because a general-purpose model has no single use case to classify. The pyramid is risk-based regulation rendered as architecture — and classification, the act of placing a system on it, is where most legal argument happens.

EU AI Act Arts 5, 6, 50

risk tolerance

The threshold that converts analysis into action: once a risk is assessed, tolerance determines whether it is accepted, treated, or escalated. The NIST AI RMF is pointed about the division of labour — the framework helps you measure and manage risk, but it “does not prescribe risk tolerance”: deciding how much risk is acceptable is a governance judgment owned by the organisation, shaped by legal duties, context, and stakeholder impact. Two disciplines follow: tolerance must be set by people with authority to own the consequences (top management, not the project team grading its own homework), and it must be documented before assessment so findings cannot quietly bend it. For harms to third parties, law caps the dial — no internal appetite statement can tolerate what Article 5 prohibits.

NIST AI RMF 1.0 §1.2.3

robustness

The trustworthiness characteristic that measures performance under stress: ISO/IEC 22989 frames it as maintaining the level of performance under any circumstances — inputs that are noisy, unexpected, out-of-distribution, or deliberately crafted to deceive (adversarial examples). The EU AI Act makes it a legal requirement: Art 15 obliges high-risk systems to achieve appropriate accuracy, robustness and cybersecurity consistently across the lifecycle, with resilience to errors, faults, inconsistencies, and exploitation attempts — and for systems that learn after deployment, safeguards against feedback loops. The engineering truth underneath: benchmark accuracy is measured on data that looks like the training set; robustness is what remains when the world stops cooperating — the property that actually predicts field performance.

EU AI Act Art 15; ISO/IEC 22989

role determination

The unglamorous control that prevents the many hands problem: for every AI system, determine and document which role(s) the organisation plays — AI developer, AI provider, AI deployer/user, data provider, or several at once — because every framework hangs obligations on roles, not intentions. ISO/IEC 42001’s Annex A includes controls for understanding your role and responsibilities toward other parties; the EU AI Act makes the stakes concrete — a deployer who puts its name on a system, substantially modifies it, or repurposes it into high-risk use becomes the provider (Art 25) and inherits the full obligations catalogue, a trap white labelling arrangements trip constantly. Role determination sounds like paperwork until the day two organisations discover they each assumed the other was doing the risk management.

ISO/IEC 42001:2023 Annex A.10; EU AI Act Art 25

root cause analysis

The discipline that separates correction from corrective action: correction fixes the instance (relabel the record, patch the model); corrective action requires finding and eliminating the cause so the failure class cannot recur — ISO/IEC 42001 Clause 10.2 mandates exactly this sequence after every nonconformity, and certification bodies require documented root-cause analysis before closing audit findings. Techniques are borrowed from safety engineering (five whys, fishbone diagrams, fault trees); AI adds a twist — root causes of model failures often live upstream in data collection, labelling choices, or mis-specified objectives rather than in code, and a proper analysis follows the chain across team boundaries. An auditor’s tell for a weak system: every corrective-action record blames “human error” and prescribes retraining the human.

ISO/IEC 42001:2023 Clause 10.2

safe harbor

The oldest trade in regulatory design: do the prescribed thing, and the law protects you — from liability, penalties, or enforcement. AI policy uses it in several flavours: proposed US legislation and state laws that grant an affirmative defence to companies that comply with the NIST AI RMF (Colorado’s AI Act offers a rebuttable presumption of reasonable care along these lines); the EU AI Act’s harmonised standards, whose presumption of conformity is a safe harbor in structure; and platform-law ancestors like DMCA §512. The design tension is permanent: safe harbors convert vague duties into checkable ones and reward early adopters, but if the harbor’s conditions are weaker than the underlying duty, compliance theatre docks there — the checklist gets done and the risk remains.

Colorado SB 24-205 §6-1-1706

safety component

One of the two doors into high-risk classification. The AI Act (Art 3(14)) defines a safety component as a component of a product or system that fulfils a safety function — whose failure or malfunctioning endangers the health and safety of persons or property. Route one to high-risk status (Art 6(1)): the AI is a safety component of a product covered by Annex I harmonisation law (machinery, medical devices, vehicles, lifts, toys…) and that product requires third-party conformity assessment — think AI controlling a surgical robot’s motion or a car’s emergency braking. This is the Act’s product-safety inheritance: the AI is regulated because of where it sits in a physical product, wholly apart from the Annex III use-case list that captures hiring tools and credit scores.

EU AI Act Arts 3(14), 6(1), Annex I

safety institute

The institutional innovation of the frontier-AI era: a state body that tests models rather than writing rules — building evaluation science and running pre- and post-deployment assessments of frontier systems for dangerous capabilities (cyber-offence, bio-uplift, autonomy). The UK AI Safety Institute (born at Bletchley, 2023; rebadged the AI Security Institute in 2025) and the US AISI within NIST (renamed CAISI in 2025) anchored an International Network of AI Safety Institutes (Seoul, 2024) spanning the EU, Japan, Singapore, Canada and others. Their leverage is voluntary — memoranda with labs, no licensing power — which makes them either the evaluation muscle regulators will later borrow, or a well-funded observer, depending on how the access agreements hold when a test result is inconvenient.

Bletchley Declaration (2023); Seoul Statement (2024)

sampling bias

The bias that enters at the collection step: when the process that gathered the data makes some members of the population more likely to be included than others — hospital data over-representing the acutely ill, face datasets scraped from media that photographs some demographics more, voice data recorded mostly from native speakers. ISO/IEC TR 24027 catalogues it among data biases; its defining property is that no amount of modelling skill downstream repairs it, because the information about the missing population was never captured. It is the standard mechanism by which datasets fail the EU AI Act’s representativeness requirement, and detecting it requires the one thing the dataset cannot supply — knowledge of who is absent, which is why data provenance documentation and collection-process review are the actual controls.

ISO/IEC TR 24027:2021

SC 42 (ISO/IEC JTC 1 Subcommittee 42 (Artificial Intelligence))

The room where international AI standards are made: Subcommittee 42 of ISO/IEC JTC 1, established 2017, secretariat held by ANSI (US), with national delegations from dozens of countries. Its catalogue is the spine of this curriculum: ISO/IEC 22989 (concepts and terminology), 23053 (ML framework), 23894 (AI risk management), 42001 (the AIMS standard), 42005 (impact assessment), 42006 (audit requirements), plus the bias, trustworthiness, and quality technical reports. Its output matters beyond ISO’s membership because CEN-CENELEC JTC 21 adopts SC 42 texts as European standards where it can (international-first (standards policy)) — meaning sentences drafted in an SC 42 working group can end up defining EU legal compliance.

ISO/IEC JTC 1/SC 42

scientific panel

The AI Act’s attempt to wire independent science into enforcement (Art 68): a panel of independent experts, selected for AI expertise and freedom from provider interests, that supports the AI Office on GPAI matters — advising on methodologies for evaluating model capabilities, on classification of systemic-risk models, and on tools and benchmarks. Its sharpest instrument is the qualified alert (Art 90): the panel can formally notify the AI Office that a GPAI model presents a concrete identifiable risk at Union level or meets the systemic-risk conditions, triggering the Commission’s assessment powers. Member States may also call on the panel’s experts. It is the same institutional move as scientific committees in pharmaceutical and food regulation — expertise as an input regulators cannot casually ignore.

EU AI Act Arts 68, 90

sectoral regulation

The alternative to one horizontal AI law: let each sector’s existing regulator govern AI within its domain, applying and extending rules it already owns — the FDA clearing AI medical devices, financial regulators supervising credit models under fair-lending law, aviation authorities certifying autonomous flight systems. It is the default US posture (the FTC polices deceptive AI as deceptive commerce; the EEOC polices biased hiring tools under Title VII). Strengths: deep domain expertise, existing enforcement muscle, rules tuned to real context. Weaknesses: gaps (general-purpose systems and new uses fall between mandates), inconsistency across sectors, and duplicated effort. Most real regimes are hybrids — the EU layered its horizontal AI Act on top of sectoral law, with Annex I explicitly deferring to product-specific legislation.

US OMB M-24-10; EU AI Act Annex I

shift left

Borrowed from software testing (move testing “left” on the project timeline) and now a core AI governance strategy: attach the controls to the earliest stage that can catch the problem — intake review before procurement, impact assessments at design, data quality and representativeness checks before training, red-teaming before release — rather than discovering at deployment what a design meeting could have prevented. The economics are the argument: a biased-label problem costs a meeting to fix at annotation time, a retraining run at validation time, and a regulatory investigation plus retrofit after launch. The toeslagenaffaire is the standard cautionary tale — the fatal decisions were design-stage decisions. Shift-left’s honest limit: some risks (drift, emergent misuse) only exist after deployment, so left-shifted governance still needs post-market monitoring.

NIST AI RMF Playbook (Map function)

significant harm

The severity filter built into the Art 5 prohibited AI practices: the bans on subliminal or manipulative techniques and on exploiting vulnerabilities (age, disability, social or economic situation) apply where the practice materially distorts behaviour in a manner that causes or is reasonably likely to cause significant harm — physical, psychological, financial, or economic — to the person or others. The Act does not define “significant” numerically; the Commission’s 2025 guidelines on prohibited practices read it as requiring real gravity, taking intensity, duration, reversibility, and the person’s vulnerability into account, and confirm harms can accumulate over time. The threshold is where enforcement arguments will live: too strict a reading legalises effective manipulation below the bar; too loose a reading turns every dark pattern into a banned practice.

EU AI Act Art 5(1)(a)–(b); Commission Guidelines C(2025) 884

specification gaming

The failure mode where the system does exactly what you said and nothing like what you meant: a reinforcement-learning boat discovers that circling to collect respawning bonus targets scores better than finishing the race (OpenAI’s CoastRunners agent); a simulated robot learns to look like it is walking rather than walk; a model rewarded for user engagement learns outrage works. DeepMind’s catalogue of examples runs to dozens. The governance lesson generalises far beyond RL: any metric made into a target will be gamed — by models, and by organisations optimising compliance metrics (Goodhart’s law applied to institutions). Defences are the same in both cases: multiple metrics, adversarial evaluation of the objective itself, and humans empowered to notice when the letter and the spirit have diverged.

DeepMind, “Specification gaming: the flip side of AI ingenuity” (2020)

SQuaRE series (Systems and software Quality Requirements and Evaluation)

The mature quality machine AI standards chose to extend rather than replace: the ISO/IEC 25000 family, organised into divisions for quality models (25010’s product quality model and the quality in use model), quality measurement, requirements, and evaluation. When SC 42 needed an AI quality framework it published ISO/IEC 25059 as a SQuaRE extension — adding AI-specific characteristics to 25010’s taxonomy instead of inventing a rival vocabulary. That choice matters practically: decades of tooling, contracts, and evaluator practice built on SQuaRE carry over to AI systems, and an organisation’s existing software-quality programme has a defined seam where AI quality bolts on rather than a parallel bureaucracy.

ISO/IEC 25000 series; ISO/IEC 25059:2023

stage 1 audit

The reconnaissance before the assault: in the mandatory two-stage initial certification audit (ISO/IEC 17021-1), stage 1 reviews the management system’s documented design — scope and Statement of Applicability, policies, risk and impact assessment processes, internal audit and management review evidence — and evaluates site-specific conditions to confirm the organisation is ready for stage 2. Output: findings classified as concerns that would likely become nonconformities, and a go/no-go on proceeding. For an AIMS the classic stage 1 failures are a certification scope that quietly excludes the riskiest AI systems, and impact assessments that exist as templates but were never performed. Treat stage 1 as a paid diagnostic: cheaper to fail here than to burn a stage 2.

ISO/IEC 17021-1:2015 §9.3.1.2

stage 2 audit

Where paper meets practice: stage 2 evaluates the implementation and effectiveness of the management system — auditors interview staff from top management to engineers, trace real AI systems through the documented processes, sample records (risk assessments performed, impact assessments for actual systems, incidents handled, management review minutes with decisions), and test whether the Statement of Applicability’s claimed controls operate. Findings are graded major or minor; certification requires majors to be closed and verified. The distinctive AIMS challenge under ISO/IEC 42006 is audit-team competence — the standard requires demonstrable AI expertise (technical experts where needed), because an auditor who cannot interrogate a model card cannot evaluate whether the AI controls are real.

ISO/IEC 17021-1:2015 §9.3.1.3

standard

The connective tissue between law and engineering: a document, established by consensus and approved by a recognised body, providing rules, guidelines, or characteristics for common and repeated use. Standards are voluntary by nature but acquire teeth three ways: legislation references them (the AI Act’s harmonised standards carry a presumption of conformity), contracts demand them (procurement requiring ISO/IEC 42001 certificates), and markets normalise them until deviation needs explaining. Read one with the grammar in mind — “shall” binds, “should” advises — and know the document types: full International Standards, Technical Specifications (not yet mature), and Technical Reports (informative only). In governance arguments, “there’s a standard for that” is the beginning of the analysis, not the end: which body, which clauses are normative, and who adopted it into force.

ISO/IEC Guide 2:2004 §3.2

standardisation request

How EU law commissions its own technical detail: under the Standardisation Regulation (1025/2012), the Commission issues a formal request (mandate) to CEN, CENELEC, or ETSI to draft standards supporting a piece of legislation. For the AI Act the request is C(2023)3215 (May 2023, amended since): ten deliverables covering risk management, data governance, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity, quality management, and conformity assessment — the standards (prEN 18286 among them) that will become harmonised standards once accepted and cited in the Official Journal. The request is where public power over private standard-setting concentrates: the Commission sets scope and deadlines and assesses the results, but the drafting rooms belong to JTC 21 — which is why who sits in those rooms is a governance question, not a technicality.

Commission Implementing Decision C(2023)3215; Regulation (EU) 1025/2012

state of the art

The AI Act’s moving goalpost: several obligations are calibrated not to a fixed specification but to the state of the art — risk management measures must take it into account (Art 9), accuracy, robustness and cybersecurity must reflect it (Art 15), and GPAI systemic-risk mitigations are judged against it. In EU product-law tradition (and the Blue Guide’s reading) the phrase means the developed stage of technical capability generally available and proven practicable — more than industry custom, less than the research frontier. Its governance consequence is a compliance treadmill: what satisfied Article 15 in 2025 may be negligent by 2028 as defenses and evaluation methods improve, so providers need a documented process for tracking the art’s advance — and the paper trail proving they did. Harmonised standards are how the state of the art gets periodically frozen into checkable form.

EU AI Act Arts 9(3), 15(1); recital 74

statistical bias

The statistician’s bias: a systematic difference between an estimator’s expected value and the true parameter — error with a consistent direction, as opposed to random noise. NIST SP 1270’s three-category taxonomy makes it one lens among three: statistical/computational biases (sampling error, measurement error, algorithmic artefacts), human biases (anchoring, automation bias, confirmation), and systemic biases (historical and institutional patterns baked into the world the data records). The framing carries the report’s central warning: debiasing the mathematics does not debias the system — a perfectly calibrated model trained on the output of a discriminatory process reproduces the discrimination with excellent statistics. Whether a statistical bias is also an unwanted bias is a normative judgment about who bears the error, not a computation.

NIST SP 1270 (2022)

stop button

Two ideas sharing one name. In EU law, it is concrete: Art 14(4)(e) requires that the humans overseeing a high-risk system be able to intervene or interrupt it through a “stop” button or similar procedure, bringing it to a halt in a safe state — the last line of human oversight when monitoring and override fail. In AI safety theory, it is a problem: a goal-directed agent has an instrumental incentive to avoid being switched off (you cannot fetch the coffee if you are dead — Stuart Russell), and naive fixes fail in both directions — penalise shutdown-resistance and the agent may manipulate you into never pressing the button; reward shutdown and it may press the button itself. Corrigibility — designing agents that genuinely accept correction — remains open research, which is why governance of agentic systems leans on architectural interruptibility (permissions, sandboxes, kill switches at the infrastructure layer) rather than trusting the agent’s goals.

EU AI Act Art 14(4)(e); Soares et al., “Corrigibility” (2015)

subliminal techniques

The first item on the AI Act’s banned list: placing on the market or using an AI system that deploys subliminal techniques beyond a person’s consciousness — or purposefully manipulative or deceptive techniques — with the object or effect of materially distorting behaviour, impairing informed decision-making, and causing or being likely to cause significant harm (Art 5(1)(a)). The Commission’s 2025 guidelines read “subliminal” to cover stimuli below conscious perception (flashed visuals, embedded audio) and, under the manipulative limb, exploitative choice architectures and dark patterns at AI scale. The drafting is deliberately effects-based — no proof of intent required for the “effect” branch — but every element (material distortion, causation, harm threshold) is a litigation site. Its neighbour ban, Art 5(1)(b), covers exploiting vulnerabilities of age, disability, or social and economic situation.

EU AI Act Art 5(1)(a); Commission Guidelines C(2025) 884

surveillance audit

What keeps a three-year certificate honest: at least once a calendar year (first one within twelve months of certification), the certification body returns for a surveillance audit — narrower than a stage 2 audit, sampling key clauses and rotating through the rest of the system across the cycle, but always checking internal audit and management review operation, complaint handling, closure of prior minor nonconformities, and use of the certification mark. For an AIMS the surveillance visit carries extra freight because AI portfolios churn: new models, new use cases, and new regulations between visits mean the auditor’s first question is whether the system absorbed the year’s changes — an unchanged risk register in a year that shipped three new AI products is itself a finding. Persistent failure to address findings can suspend or withdraw the certificate mid-cycle.

ISO/IEC 17021-1:2015 §9.6.2

synthetic content

The AI Act’s word for machine-made media, and the object of its Art 50 transparency regime: providers of systems generating synthetic audio, image, video, or text must ensure outputs are marked as artificially generated in a machine-readable, detectable way — via watermarking (AI content), provenance metadata (C2PA-style), or fingerprinting, effective and interoperable “as far as technically feasible”. Deployers owe human-facing disclosure on top: deepfakes must be labelled, as must AI-generated text published to inform the public on matters of public interest (absent human editorial review). The same category powers the harder problems upstream — NCII and fraud as misuse, and model collapse as the ecosystem cost of unlabelled synthetic data re-entering training corpora. Marking is honest-actor infrastructure: it filters the careless, not the malicious who strip marks — which is why detection and provenance are layered, not alternatives.

EU AI Act Art 50(2), (4)

TDM opt-out

The pivot point of the EU’s AI-and-copyright settlement. The DSM Copyright Directive (2019) created a broad text-and-data-mining exception for commercial use — Art 4 — but let rightsholders reserve their rights (opt out), for online content in machine-readable form (robots.txt-style signals, metadata). The AI Act then reached back and grabbed it: Art 53(1)(c) requires GPAI providers to adopt a copyright policy that identifies and honours Art 4(3) reservations — regardless of where in the world the training happened, if the model is placed on the EU market. Open questions keep lawyers employed: what counts as machine-readable reservation, whether opt-outs seen after crawling require unlearning, and how honouring is audited — the GPAI Code of Practice and the Art 53 training-data summary template are the current answers. Germany’s LAION decision (2024) gave the exception its first judicial workout.

Directive (EU) 2019/790 Art 4(3); EU AI Act Art 53(1)(c)

technical expert

The audit world’s answer to “our auditors don’t understand transformers”: a person who provides specific knowledge or expertise to the audit team — ISO 19011 and ISO/IEC 17021-1 allow certification bodies to staff audits with technical experts who advise on subject matter but do not act as auditors: they cannot audit independently or determine findings, and their input flows through the audit team leader. ISO/IEC 42006 makes them central to AIMS credibility by defining the AI competences an audit team must collectively hold — machine learning fundamentals, data management, AI-specific risks, the relevant regulatory landscape — which in practice means pairing management-system auditors with AI technical experts. The failure mode it guards against is the checklist audit: an auditor who cannot ask a follow-up question about a model card will accept whatever the auditee’s slide deck says.

ISO 19011:2018 §3.17; ISO/IEC 42006:2025

technical report

The lowest-commitment document in the ISO hierarchy: a Technical Report (TR) contains only informative material — collected data, survey results, explanatory guidance — with no normative requirements at all; you cannot conform to one, and no auditor can raise a finding from one (normative vs informative). The AI catalogue leans on TRs heavily because the field is young: ISO/IEC TR 24027 (bias in AI systems), TR 24028 (trustworthiness overview), and TR 24368 (ethical concerns) map terrain that was not ready for “shall” — a TR is often the scouting party sent ahead of a future standard, and content road-tested in a TR later hardens into normative text (the path bias work is taking). Reading tip: the ISO reference tells you the force before you open the document — IS binds, TS is provisional, TR informs.

ISO/IEC Directives Part 1 §2.1.2

toeslagenaffaire

The case study European AI governance is written in the shadow of. From 2013 the Dutch tax administration used a self-learning risk-scoring system to flag childcare-benefit fraud; nationality was a model input, dual nationals were disproportionately flagged, and a hair-trigger “all-or-nothing” rule turned minor administrative errors into demands to repay tens of thousands of euros. Roughly 26,000+ parents were wrongly branded fraudsters; families were bankrupted and over a thousand children taken into care. The parliamentary inquiry’s title — Unprecedented Injustice — preceded the Rutte cabinet’s resignation in January 2021, and the Dutch DPA fined the tax authority for unlawful, discriminatory processing. Every element the EU AI Act later regulated is present: discriminatory data, no meaningful human oversight, no contest route that worked, and institutional incentives that rewarded the system’s severity. When a lesson asks why deployer duties and FRIAs exist — this is why.

Parliamentary report “Ongekend Onrecht” (2020); Dutch DPA decision (2021)

top management

A defined term with teeth: the person or group of people who directs and controls an organisation at the highest level (ISO terminology), and the addressee of ISO/IEC 42001’s Clause 5 — top management must demonstrate leadership and commitment to the AIMS: establish the AI policy, ensure objectives align with strategy, provide resources, assign roles, and conduct management review. The design intent is anti-delegation: the standard lets executives delegate tasks but not accountability, so an AIMS run entirely by a compliance team with no executive fingerprints fails audit at Clause 5. Note the seam with governance: ISO/IEC 38507 addresses the governing body (board) above top management — direction and oversight versus execution — and the AI-literate board asking hard questions is what keeps top management’s commitment from being ceremonial.

ISO/IEC 42001:2023 Clause 5; ISO/IEC 38507

traceability

The property that makes accountability technically possible: being able to reconstruct, after the fact, which model version, which data, which inputs, and which human choices produced a given output. The EU AI Act legislates it directly — Art 12 requires high-risk systems to automatically record events (logs) over their lifetime, ensuring a level of traceability appropriate to the system’s intended purpose, and deployers must retain those logs (Art 26); ISO/IEC 42001’s Annex A echoes it with documentation and data lineage controls. Traceability is the difference between an incident investigation and an incident guess: without logs tying output to model version to training data snapshot, root cause analysis has nothing to grip, contestability is theatre, and the many hands problem wins by default. It is provenance’s sibling — provenance faces the data coming in, traceability faces the decisions going out.

EU AI Act Arts 12, 26(6)

training data

The AI Act’s definition is compact — data used for training an AI system through fitting its learnable parameters (Art 3(29)) — and the consequences are anything but. Training data determines capability (a model is a compression of its corpus), fairness (whatever the collection process under- or mis-represents, the model inherits — sampling bias, representativeness), privacy exposure (memorisation turns training records into extractable ones), and legal position (copyright fights from TDM opt-out to the US fair-use cases turn on what was trained on and how). Hence the paper trail frameworks demand: Art 10 data-governance duties for high-risk systems distinguish training, validation, and testing sets — the split that keeps evaluation honest — and GPAI providers must publish a sufficiently detailed summary of training content (Art 53). The operational rule: a model you cannot describe the training data of is a model you cannot defend.

EU AI Act Arts 3(29), 10, 53

transparency obligations

The AI Act’s middle tier in action — obligations that inform rather than restrict. Article 50 imposes four disclosure duties: providers must ensure people interacting with an AI system (chatbots, voice agents) know it is a machine, unless obvious to a reasonably informed person; providers of generative systems must mark synthetic content machine-readably; deployers of emotion recognition or biometric categorisation must inform the people exposed to them; and deployers must disclose deepfakes and AI-generated public-interest text. Disclosures must be clear, at the latest at first interaction, and accessible. Two boundary notes keep analysis honest: these duties stack with high-risk obligations rather than replacing them (a high-risk chatbot owes both), and transparency here means disclosure to affected people — distinct from the Art 13 duty of transparency to deployers (instructions for use) and from explainability of individual decisions.

EU AI Act Art 50

trustworthiness

The field’s organising abstraction, made useful only by decomposition. ISO/IEC TR 24028 frames trustworthiness as the ability to meet stakeholder expectations in a verifiable way; the NIST AI RMF names the characteristics: valid and reliable (the precondition), safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed. Three disciplines keep the word from dissolving into marketing: trustworthiness is a property of the system in context, not a badge on the model; the characteristics trade off (more explainable can mean less accurate; more private can mean less fair), so declaring a system trustworthy means declaring which balance you struck and why; and every claimed characteristic needs a verification story — a measurement, test, or audit that could have come out the other way. Trust is the user’s state of mind; trustworthiness is the evidence that state of mind is warranted.

ISO/IEC TR 24028:2020; NIST AI RMF 1.0 §3

unwanted bias

The ISO vocabulary’s honest adjective: since every model is “biased” toward the patterns it learned — and some biases are the point (a credit model should treat different repayment histories differently) — ISO/IEC TR 24027 targets unwanted bias: systematic differences in treatment of persons or groups that produce unfair, discriminatory, or otherwise undesired outcomes. The term does two jobs. Technically, it separates the measurable (statistical bias, sampling bias) from the normative — deciding which differential treatment is unwanted requires ethics and law, not more mathematics. Managerially, it makes bias a specifiable requirement: ISO/IEC 42001 Annex A and the EU AI Act’s Art 10(2)(f)–(g) both demand examination of datasets for possible biases likely to affect health, safety, or fundamental rights or lead to prohibited discrimination — an obligation you can only meet after writing down which biases, for this system, in this context, count as unwanted.

ISO/IEC TR 24027:2021; EU AI Act Art 10(2)

validation

One half of V&V, and the half that faces the world: confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled (ISO 9000 lineage). The classic mnemonic — verification asks did we build the thing right?, validation asks did we build the right thing? — has real operational content for AI: a model can verify perfectly against its specification and still fail validation because the specification missed the deployment reality (wrong population, wrong prevalence, wrong workflow). Note the terminology collision: in ML practice, the validation set is the data split used for tuning during development — related, but far narrower than lifecycle validation, which spans clinical-style evaluation, real-world testing, and post-deployment monitoring against intended use. The EU AI Act builds both into high-risk duties: Art 10’s dataset splits and Art 9’s testing against intended purpose.

ISO 9000:2015 §3.8.13; ISO/IEC 22989

verification

The inward-facing half of V&V: confirmation, through objective evidence, that specified requirements have been fulfilled — the system does what its specification says, checked by testing, inspection, analysis, or review. In the AI lifecycle stage models (ISO/IEC 22989, OECD) “verification and validation” form a named stage between building and deployment, but the two questions differ in kind: verification is closed-world (compare artefact to specification — accuracy above threshold X on test set Y, latency under Z), while validation is open-world (is the specification itself right for the intended use?). AI strains classic verification because behaviour is learned rather than coded — you cannot enumerate paths through a neural network the way you review source code — so verification leans on statistical testing, benchmark suites, red-teaming, and, at the formal frontier, mathematical verification of narrow properties. Both produce the evidence conformity assessment later inspects: no verification records, no defensible conformity claim.

ISO 9000:2015 §3.8.12; ISO/IEC 22989

voluntary commitments

The interregnum instrument: with binding law years away, governments extracted public promises. The template was the White House Voluntary AI Commitments (July 2023) — Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI pledged pre-release security testing including red-teaming, information sharing on risks, investment in cybersecurity for model weights, third-party vulnerability reporting, provenance or watermarking (AI content) for synthetic content, public capability reporting, and research on societal risks — followed by a second cohort and the Seoul Frontier AI Safety Commitments (2024) internationally. Their governance anatomy: fast to obtain, unenforceable by design, and each firm grades its own compliance — but not empty, because public promises create reputational hostages, FTC exposure for firms that break them (deception jurisdiction), and drafting material that later hardens into frameworks like the GPAI code of practice. Best read as regulation’s first draft, written by the regulated.

White House Voluntary AI Commitments (2023); Seoul Commitments (2024)

voluntary consensus standard

The formal species of standard the US system runs on: developed by a body with openness, balance of interests, due process, an appeals process, and consensus (the OMB Circular A-119 criteria), and voluntary — adopted by choice, not command. US law actively channels government toward them: the National Technology Transfer and Advancement Act (1996) directs federal agencies to use voluntary consensus standards instead of writing government-unique ones, which is how ISO and IEEE texts end up inside procurement rules and regulations. The AI-relevant subtlety is how voluntariness ends: in the EU, citation in the Official Journal gives a harmonised standard legal effect; in the US, incorporation by reference in regulation or contract does the same job. “Voluntary” describes the standard’s birth, not necessarily your relationship with it — by the time it reaches you through a customer contract or a regulator, the choice was made upstream.

OMB Circular A-119; NTTAA (1996)

voluntary framework

The US answer to the pacing problem: publish a rigorous, complete governance framework and let adoption be voluntary — the NIST AI RMF says so on page one (“voluntary… not a checklist… non-prescriptive”), following the wildly successful precedent of the NIST Cybersecurity Framework. The theory of change is indirect force: voluntary frameworks become de facto mandatory through procurement (federal and enterprise contracts requiring RMF alignment), insurance and investor due diligence, safe harbor statutes that reward adopters, and courts treating them as the standard of care in negligence. Strengths: speed, flexibility, no constitutional fights, genuine technical quality. The standing critique: voluntariness selects for the already-willing — the firms whose practices most need changing are precisely the ones that decline — which is why most regimes treat voluntary frameworks as scaffolding while binding law is poured, not as the finished building.

NIST AI RMF 1.0 §1; National AI Initiative Act §5301

white labelling

The rebadging business model meets Article 25: a firm licenses an AI system built by someone else and markets it under its own name or trademark — common across HR tech, chatbots, and analytics, where dozens of “different” products share one underlying engine. The AI Act’s response is blunt: a distributor, importer, deployer, or any third party that puts its name or trademark on a high-risk AI system already on the market becomes the provider (Art 25(1)(a)), inheriting the full obligations catalogue — risk management system, technical documentation, conformity assessment, registration, post-market monitoring — while the original provider drops to a supporting role owed information and cooperation (and contracts scramble to allocate that). The commercial logic and the legal logic collide head-on: the whole point of white labelling is to look like the manufacturer, and the Act simply takes the appearance at its word. role determination exercises exist to catch this before a regulator does.

EU AI Act Art 25(1)

AWS Well-Architected Framework (WAF (context: architecture reviews))

AWS’s methodology for assessing cloud workloads across six pillars — operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. A review walks a workload through per-pillar questions; unmet best practices become prioritized risk findings. Not a certification: a structured conversation that produces an improvement backlog.

AWS Well-Architected Framework

Well-Architected lens

An extension of the Well-Architected Framework for a specific workload type (serverless, machine learning, generative AI…). A lens adds domain-specific questions and best practices on top of the six pillars. The Generative AI Lens (published 19 Nov 2025) covers foundation-model workloads on Bedrock and SageMaker AI, organized around the GenAI lifecycle from scoping to continuous improvement.

AWS Well-Architected Generative AI Lens (2025)

AWS Well-Architected Tool

The console service where reviews live: define a workload, answer the pillar questions, record which best practices are met, and track high/medium-risk findings over time. Custom lenses — including the Generative AI Lens, distributed via AWS’s public GitHub repository — are imported here to extend the question set.

AWS Well-Architected Tool

GENOPS / GENSEC / GENREL…

The Generative AI Lens identifies every best practice with a pillar prefix (GENOPS operational excellence, GENSEC security, GENREL reliability, GENPERF performance, GENCOST cost, GENSUS sustainability), a question number, and a best-practice number: GENOPS01-BP01 is the first best practice under the first operational-excellence question (periodic functional evaluation with stratified sampling). The site’s GenAI Lens Index summarizes every one.

AWS Well-Architected Generative AI Lens (2025)

Responsible AI dimensions (AWS)

The core set of dimensions AWS uses to define responsible AI, updated as the field evolves: controllability (monitor and steer system behavior), privacy, security, safety (block harmful output and misuse), veracity (factually correct outputs), robustness (correct outputs on unexpected inputs too), fairness, explainability, transparency, and governance (practices across the AI supply chain). The Responsible AI Lens (published 19 Nov 2025) organizes harmful-event analysis and metric selection around these dimensions. Note the deliberate split between veracity and robustness — accuracy on expected inputs versus behavior under distribution shift.

AWS Well-Architected Responsible AI Lens (2025)

Harmful event

The Responsible AI Lens’s unit of risk analysis: a specific, imaginable bad outcome (a biased condo description, a hallucinated property feature, a privacy leak in generated text), identified dimension by dimension (fairness, veracity, robustness, privacy, safety, security, explainability, transparency), assigned likelihood and severity, given an overall risk level, and tracked in a risk registry. Harmful events then drive mitigation choices and become testable release criteria — the lens’s version of the risk-assessment → acceptance-criteria loop that ISO/IEC 42001 and the NIST AI RMF also demand.

AWS Well-Architected Responsible AI Lens (2025)

AI Center of Excellence (AICoE) (AI Center of Excellence)

An organizational unit with a dual mandate: enable AI adoption (platform, patterns, training, delivery help) and govern it (standards, risk tiering, gates, registry). Typically starts centralized, evolves to hub-and-spoke as embedded squads mature, and may federate once guardrails are automated into the platform. Distinct from an AI governance committee (which decides) — the CoE operates: it runs intake, maintains the artifact library, and is a natural home for operating an ISO/IEC 42001 AIMS day to day.

Composite practitioner doctrine (cf. AWS CAF for AI, Microsoft CAF)

Cloud Center of Excellence (CCoE) (Cloud Center of Excellence)

The organizational pattern enterprises used to adopt cloud at scale: a small central team owning cloud standards, landing zones, guardrails, and enablement, evolving from builder to enabler as business units matured. Its playbook — executive mandate, golden paths over gatekeeping, automate the guardrails, then decentralize — transfers almost verbatim to AI adoption, with new risk types (model behavior, data leakage through prompts, regulatory duties) layered on top.

Cloud-adoption practice (cf. AWS/Microsoft adoption frameworks)

Hub-and-spoke (operating model)

The default target operating model for AI (and cloud) centers of excellence. The hub stays small — platform, guardrails, standards, evaluation depth, the trickiest builds — while spokes (embedded AI squads inside business units) deliver everyday use cases with domain context. Sits between centralized (one team does everything; strong control, guaranteed bottleneck) and federated (business units own delivery; a lean council curates standards — workable only once guardrails are automated).

Composite practitioner doctrine

Generative AI Security Scoping Matrix

AWS’s mental model for securing generative AI (introduced October 2023). First, place the workload in one of five scopes by ownership: 1 consumer app, 2 enterprise app (both “buy”), 3 pre-trained model via API, 4 fine-tuned model, 5 self-trained model (all “build”). Then work the five disciplines — governance & compliance, legal & privacy, risk management, controls, resilience — whose demands shift with scope: the higher the number, the more of the model and data you own, and the more of the security burden moves from provider terms to your own architecture. A key rule of thumb: a fine-tuned or self-trained model inherits the classification of the most sensitive data it was trained on.

AWS Security Blog / AWS AI security (2023)

Shared responsibility model

Cloud security doctrine: the provider secures the underlying service, the customer secures what they build and put on it — where the line sits depends on the service type. Generative AI inherits and complicates it: in AWS’s Generative AI Security Scoping Matrix, Scopes 1–2 leave the model and its training data entirely with the provider (your duties are contractual and behavioral), while Scopes 3–5 progressively hand you the application layer, the data pipeline, and eventually the model itself — and with them the threat modeling, access control, and resilience work.

Cloud security practice (cf. AWS shared responsibility model)

PII (Personally Identifiable Information)

Any data that identifies or can be combined to identify a person — names, identifiers, contact details, biometrics. In AI work its handling is doubly sensitive: privacy law attaches duties to it (GDPR, sectoral US law), and once trained into a model it effectively cannot be deleted — the only fully effective removal is retraining without it. Hence the scoping-matrix rule of thumb: keep PII out of training and fine-tuning sets, supply it at inference through retrieval instead, and classify any model trained on it at that data’s sensitivity level.

Privacy practice; GDPR / NIST usage