A quality model for AI — ISO/IEC 25059
Lesson 5 of 5 in Trustworthiness, Bias, and AI Quality: The TR Series and 25059.
For twenty years, software teams have specified and evaluated products against the SQuaRE family (Systems and Software Quality Requirements and Evaluation, the ISO/IEC 25000 series), whose centrepiece — ISO/IEC 25010 — defines the quality characteristics every requirements document borrows: functional suitability, reliability, security, usability, maintainability and the rest. Procurement contracts cite it; acceptance tests are structured by it.
Then AI broke its quietest assumption. 25010 presumes deterministic correctness: given a specification, an output is right or wrong, and testing can prove which. An ML model is statistically correct — right 94% of the time, wrong in patterned ways, and possibly different next month. "Functional correctness" as classically defined is unachievable and unmeasurable for such a system. ISO/IEC 25059:2023 exists to repair exactly this: it extends the 25010 model with AI-specific characteristics so that quality requirements, acceptance criteria, and evaluations can be written for systems that are probabilistic by nature. It is a full International Standard — the taxonomy is standardisable even where the thresholds are not.
| Characteristic | New or adapted | What it captures for AI systems |
|---|---|---|
Functional correctness | Adapted | Redefined for probabilistic outputs: correctness with degrees and confidence bounds, specified statistically (e.g. per TS 4213 metrics), not as pass/fail determinism |
Functional adaptability | New | The system’s capacity to perform in contexts and on data beyond its original specification — the quality-model home of generalisation and distribution shift |
Robustness | New | Maintaining performance under perturbation and adversarial pressure — the 24028 attribute promoted into a specifiable product quality, measurable via the 24029 series |
User controllability | New | Whether users and operators can direct, correct, and override the system — including intervenability, the ability to intervene in an operation in progress |
Transparency | New | Whether appropriate information about the system — capabilities, limitations, data, logic — is available to the right stakeholders as a product property, not a policy afterthought |
Societal & ethical risk mitigation (quality in use) | New | Quality experienced in real use now includes whether the system mitigates harm to individuals and society — TR 24368’s themes arriving as evaluable quality-in-use criteria |
Closing the loop: from guidance to AIMS artifacts. You now hold all four documents plus the measurement ring. Here is how a competent AIMS lead wires them together — the pattern this whole module has been building toward:
- Annex C objectives (fairness, robustness, transparency, safety…) are selected and justified using TR 24028’s attribute definitions and TR 24368’s themes.
- Clause 6 risk assessment draws its AI-specific risk sources from TR 24028’s threat catalogue and TR 24027’s bias taxonomy — that is what "informed by the state of the art" looks like in evidence.
- A.6 lifecycle controls state requirements and acceptance criteria in 25059’s vocabulary; A.7 data controls lean on the 5259 series for data-quality process.
- Clause 9 monitoring measures what the objectives promised, with methods from TS 4213 and 24029 — numbers with pedigrees, reviewable at management review.
An auditor who sees this chain — objective, risk source, control, metric, each traceable to published state of the art — sees an organisation that knows why its management system looks the way it does. That traceability, not any individual document, is the professional skill this module certifies.
The guidance-to-artifact pipeline
- TR 24028 / 24027 / 24368
Informative state of the art: attributes, threats, bias taxonomy, ethics themes.
- Annex C objectives
The organisation selects and justifies AI objectives — fairness, robustness, transparency — for its context.
- Clause 6 risk & impact assessment
Threats and bias types become identified risk sources; 42005 adds the affected-party lens.
- A.6 / A.7 controls with 25059 criteria
Lifecycle and data controls state acceptance criteria in quality-model vocabulary.
- Clause 9 metrics (TS 4213, 24029, 5259)
Measurement standards supply valid methods; results feed management review.
- Management review → improvement
Clause 9.3 and 10: results drive corrective action and updated objectives — the loop closes.
Key terms: quality model, SQuaRE series, functional adaptability, intervenability, quality in use
Tool: Fairness Lab — Feel the TR 24027 trade-offs yourself: tune per-group thresholds on a toy classifier and watch demographic parity, equalized odds, and accuracy fight each other.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.