Where bias enters: a taxonomy you can test against

Lesson 1 of 6 in Testing and Evaluation: Fairness Math, Red-Teaming, and Explainability.

You cannot test for “bias” in the abstract. You can only test for specific failure modes with specific entry points — and every serious evaluation program starts by naming them. The taxonomy below (aligned with NIST SP 1270’s systemic/statistical/human framing) is the working checklist practitioners use to decide what to measure.

Historical bias — the world the data describes was already unfair. Amazon’s recruiting model learned from ten years of male-dominated hiring and downgraded the word “women’s”. No data error occurred; the data was an accurate record of a biased process.

Representation bias — some groups are thin or missing in the training sample, so the model is simply worse for them. Gender Shades made this measurable: commercial face analysis erred on darker-skinned women at up to 34.7% versus 0.8% for lighter-skinned men.

Measurement and label bias — the target variable is a flawed proxy for what you actually care about. This is the subtlest and most dangerous entry point, because the model can be perfectly accurate on the proxy and still discriminate on the real objective.

Two more entry points complete the checklist:

Aggregation bias — one model fitted across populations that need different models. A single diabetes-risk model across ethnic groups with different biomarker baselines will be miscalibrated for some of them even with perfect data.

Deployment and feedback-loop bias — the model’s own outputs distort the future data it learns from. Predictive policing is the canonical loop: patrol where the model points, find crime where you patrol, retrain on the arrests you generated. The system manufactures its own confirmation.

The governance payoff of the taxonomy: each source implies a different test and a different fix. Historical bias calls for outcome audits and label redesign; representation bias for disaggregated performance testing and data collection; measurement bias for proxy-validity review before training; feedback loops for monitoring designs that hold out unpoliced ground truth. A bias-testing plan that names none of these is a plan to find nothing.

Key terms: historical bias, representation bias, measurement bias, proxy variable, feedback loop, disaggregated testing

Interactive sorting exercise: Diagnose the primary bias source in each scenario.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.