Standards bodies, auditors, and the assurance market

Lesson 4 of 5 in Who’s Who in AI Governance: Actors, Roles, and Responsibilities.

Laws say what must be true — “the system shall be accurate and robust”. Standards say how to demonstrate it. The bodies that write them are therefore among the most quietly powerful actors on this map, and most professionals meet them late because standards live behind paywalls and acronyms.

The centre of gravity is ISO/IEC JTC 1/SC 42, the joint international committee on AI, home of ISO/IEC 42001 (the AI management system standard — the “ISO 9001 of AI”, published December 2023), ISO/IEC 23894 (AI risk management) and ISO/IEC 22989 (the vocabulary whose role taxonomy you met in lesson one). In Europe, CEN-CENELEC adapts and drafts the harmonised standards that operationalise the AI Act. IEEE contributes ethics-driven design standards, and NIST publishes the frameworks (AI RMF and its Generative AI Profile) that function as de facto standards in the US despite being voluntary.

Here is the trick that makes “voluntary” standards binding in practice, and it is one of the most examinable ideas in this module: the presumption of conformity. Under the EU AI Act (Art 40), a provider that complies with a harmonised standard cited in the Official Journal is presumed to comply with the corresponding legal requirement. The standard stays technically voluntary — but following it becomes the cheapest defensible route to market, so the market follows it. A committee of engineers in a CEN working group ends up deciding what “sufficient robustness” means for every high-risk system in Europe.

How a voluntary standard becomes the de facto law

  1. Legislature writes essential requirements

    E.g. EU AI Act Arts 8–15: risk management, data governance, accuracy, robustness — stated as outcomes, not methods.

  2. Commission requests standards

    A standardisation request tasks CEN-CENELEC (JTC 21) with drafting harmonised standards for each requirement.

  3. Committees draft & vote

    National delegations and industry experts negotiate the technical detail — this is where the real definitions get written.

  4. Citation in the Official Journal?

    The Commission assesses whether the standard adequately covers the legal requirement.

  5. Presumption of conformity

    Providers complying with the cited standard are presumed compliant with the law (Art 40). Voluntary in name, universal in practice.

  6. Standard revised

    If coverage is inadequate, the standard goes back to committee — or the Commission can adopt common specifications itself (Art 41).

Once standards define “good”, someone must check whether a given system meets them. That is the assurance ecosystem, and it comes in escalating grades of independence:

Internal audit (third line, previous lesson) checks the company’s own framework. Third-party algorithmic auditors — a young industry born partly from NYC Local Law 144’s mandatory hiring-tool bias audits — test specific systems against specific criteria. Conformity assessment bodies / notified bodies are formally designated organisations that certify high-risk systems against the AI Act’s requirements before market entry. Certification bodies audit management systems: an ISO/IEC 42001 certificate says your organisation’s governance process meets the standard — note carefully, it does not say any particular model is safe or fair.

The honest caveat: this market is nascent. Audit methodologies are not yet standardised, auditor accreditation is patchy, and an “AI audit” can mean anything from a week of document review to adversarial testing with model access. When you commission or read one, the first question is always: audited against what criteria, with what access, by whom, paid by whom?

Internal audit — how independent is it really?

Independent of management (it reports to the board’s audit committee), but not of the company. Good for assurance that the governance framework operates; weak as a public trust signal, because the public never sees the reports.

Third-party algorithmic audit — the NYC LL144 model

New York City’s Local Law 144 (enforced July 2023) made annual independent bias audits mandatory for automated employment decision tools, with published results. It created the first compliance-driven market for algorithmic auditing — and immediately exposed the field’s growing pains: no accreditation regime, varying methodologies, and audits scoped narrowly to the impact-ratio metrics the law names.

Notified body — the EU’s heavyweight

A conformity assessment body designated by a member state and notified to the Commission (AI Act Art 31). For certain high-risk systems (notably biometrics, and products under existing product-safety regimes), a notified body must review the system before it can carry the CE marking. This is the same machinery Europe uses for medical devices and lifts — deliberately so.

Management-system certification — what ISO/IEC 42001 does and does not prove

A 42001 certificate attests that your AI management system — policies, roles, risk processes, documentation — conforms to the standard. It is organisational assurance, not product assurance: a certified organisation can still ship a biased model. Treat “we are ISO 42001 certified” as the start of due diligence, never the end.

Key terms: harmonised standard, presumption of conformity, notified body, conformity assessment, algorithmic audit, certification

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.