Testing and Evaluation: Fairness Math, Red-Teaming, and Explainability

The technical core of AI assurance: where bias enters, the fairness metrics and their impossibility results in full detail, disparate-impact math and bias audits, benchmarks and red-teaming, and the explainability toolkit — SHAP, LIME, counterfactuals — matched to the audiences governance must serve.

Content current as of 2026-09.

Lessons

1. Where bias enters: a taxonomy you can test against
2. Fairness metrics, precisely
3. Disparate-impact math and the bias-audit machine
4. Benchmarks, their limits, and evaluating generative systems
5. Red-teaming and the adversarial threat catalogue
6. Explainability at governance altitude: SHAP, LIME, counterfactuals