Naming the bias — ISO/IEC TR 24027

Lesson 3 of 5 in Trustworthiness, Bias, and AI Quality: The TR Series and 25059.

“The model is biased” is a diagnosis at the level of “the patient is unwell”. TR 24027 exists to give you the differential: which bias, entering where, treatable how. Its first move is to split three meanings that everyday language collapses into one word:

  • Statistical bias — a systematic deviation between an estimate and the true value. Morally neutral; every model has some.
  • Human cognitive bias — systematic error patterns in human judgment, which enter AI systems through the people who label data, choose features, and act on outputs.
  • Societal bias — historical and structural inequality embedded in the world the data describes. The data can be a perfectly accurate record of an unjust process.

The governance payoff of the distinction: "unwanted bias" — the report’s target — is bias that produces unfair outcomes for people, and it can exist with flawless statistics. Amazon’s recruiting model was statistically faithful to ten years of hiring data; the data was faithful to a decade of skewed decisions. Accuracy was never the problem.

Human cognitive biases

Enter through every human touchpoint in the lifecycle. Confirmation bias: annotators and evaluators see what they expect — a labeller told the model is usually right agrees with it. Automation bias: operators defer to machine output, hollowing out human oversight (you met the rubber-stamp trap in foundations). Group attribution bias: judging individuals by beliefs about their group — which becomes measurement error the moment a human encodes it into a label.

Where to look for it: labelling instructions, annotator agreement statistics, reviewer override rates.

Data biases

The largest family. Selection/sampling bias: the data collection process over- or under-samples parts of the population. Coverage bias: whole groups or situations are missing — the Gender Shades study traced commercial face-analysis failures on darker-skinned women to exactly this. Label/annotation bias: the target variable encodes human judgment errors. Measurement bias: the feature measures the construct unequally across groups (arrest records as a proxy for crime). Representation bias: minorities present but swamped. Historical/societal bias: the world the data records was itself unjust — the Dutch childcare-benefits data faithfully recorded years of discriminatory enforcement.

Where to look for it: dataset datasheets, per-group sample counts, provenance records — the artefacts ISO/IEC 5259 and control A.7 demand.

Engineering biases

Introduced by design decisions after the data exists. Feature selection: dropping or keeping variables changes who the model can see — deleting the protected attribute while keeping its proxies is the classic failure. Aggregation bias: one model for heterogeneous groups fits none of them well. Model expressiveness: a model too simple to capture a minority pattern defaults to the majority one; a model expressive enough will happily learn the societal bias too.

Where to look for it: design-choice documentation (control A.6.2.3 makes these decisions writable-down and reviewable), disaggregated evaluation results.

Measuring fairness. TR 24027 catalogues the same metric families you studied in the foundations path — demographic parity, equalized odds, equal opportunity, predictive parity — and states the same uncomfortable arithmetic: when base rates differ between groups, the metrics cannot all hold at once, so choosing one is a policy decision. We will not re-teach that material here; if the COMPAS fight and the impossibility result are not fresh, revisit Ethics and Trustworthy AI → Fairness and the impossibility result before continuing. What TR 24027 adds is the standards framing: the metric choice must be documented, justified against context, and monitored — an Annex C fairness objective operationalised through clause 9.

Treating bias. The report organises interventions by lifecycle position, and the placement matters because each stage treats a different family:

  1. Pre-processing — fix the data: rebalancing, reweighting, augmenting coverage gaps, cleaning labels. Reaches data biases; cannot reach a biased objective.
  2. In-processing — fix the training: fairness constraints in the loss function, adversarial debiasing. Reaches engineering and some societal bias, at a measurable accuracy trade-off someone must own.
  3. Post-processing — fix the outputs: per-group thresholds, calibration adjustments. Fast, model-agnostic — and legally delicate in some jurisdictions, since it treats groups explicitly differently at decision time.
  4. Monitoring — because bias is not a launch property: drift re-skews data, and feedback loops amplify small disparities. Clause 9.1 metrics reviewed at management review is where this lives in the AIMS.

Key terms: unwanted bias, sampling bias, label bias, historical bias, proxy variable, disaggregated testing

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.