Harms analysis, scoring mechanics, and who signs

Lesson 4 of 5 in AI Risk and Impact Assessment Methodologies.

Strip away the legal wrappers and every assessment does the same three moves: find the harms, size them, and get someone accountable to accept what remains. Each move has a craft — and a characteristic way of being faked.

Finding harms is a stakeholder exercise, not a brainstorm. Teams asked to “list the risks” reliably produce the risks to themselves: outages, bad press, fines. Sociotechnical harms taxonomies force the outward look — allocative harms (who is denied money, jobs, services), quality-of-service harms (for whom does it work worse), representational harms (who is stereotyped or erased), dignitary and autonomy harms, and social-system harms (chilling effects, information degradation). NIST’s Generative AI Profile (AI 600-1) does the same job for genAI systems, handing assessors a ready-made list of twelve risks — confabulation, data privacy, harmful bias and homogenisation, information integrity, CBRN information uplift, value-chain and component integration among them — that works as a completeness checklist: for each, present or absent, and why?

Then map who is affected before asking how badly. The affected-stakeholder map should always contain more rows than “users”: the person scored, their dependents, the frontline worker executing the decision, the groups statistically over-represented in the data, and the people who never appear in the data at all. Canada’s GBA Plus lens and the consultation questions in its AIA exist precisely because affected people identify harms that system owners cannot see — the toeslagenaffaire families could have told Dutch authorities in year one what took investigative journalists years to surface.

Sizing harms: the severity × likelihood matrix, and why it lies. The workhorse of assessment is the heat map — severity on one axis, likelihood on the other, harms placed into cells, cell colour driving action. Use it, but know its four documented failure modes:

  1. Inter-rater unreliability. The same harm, scored by two competent assessors, routinely lands two cells apart. Fix: anchored scales with concrete exemplars per level (“severity 4 = irreversible exclusion from an essential service”), and calibration sessions where assessors score the same case and reconcile.
  2. Likelihood theatre. For novel AI harms nobody has frequency data; “unlikely” often means “we would prefer not to think about it”. Fix: for rights-impacting harms, weight severity over likelihood — the FRIA and most rights-based methodologies deliberately do not let low claimed likelihood wash out high severity.
  3. Aggregation hiding. One average score for “bias” conceals that harm concentrates on one subgroup. Fix: score per affected group where the harm plausibly differs.
  4. Gaming the boundary. When tier thresholds drive cost, scores cluster magically just below the boundary. Fix: second-line challenge rights, and audit the score distribution — a healthy portfolio does not have a cliff at tier-2/tier-3.

Interactive sorting exercise: Place each finding from a benefits-eligibility AI assessment where a rights-weighted methodology would put it — remembering that high severity cannot be washed out by claimed low likelihood.

The sign-off is the assessment. Everything upstream exists to put a named person in a position to say, on the record: I understand what remains, and I accept it on behalf of the organisation. Three rules make residual risk acceptance real rather than ceremonial:

The signer must have authority proportional to the risk. A product manager cannot accept a risk that could halt the business line. Your risk criteria should map residual-risk levels to acceptance authority — team lead, business owner, executive, board risk committee — exactly as ISO/IEC 42001’s risk-criteria clause envisions.

Acceptance is conditional and time-boxed by default. “Approved, provided confidence-based human review runs on all denials, disaggregated performance is reported monthly, and this approval lapses in 12 months or upon substantial modification” is a governance instrument. “Approved” is a signature.

Compensating controls must be verified, not promised. An approval conditioned on a control that never ships is worse than a rejection — it creates the paper trail of an organisation that knew. Close the loop: conditions get owners, deadlines, and a verification step before the condition is marked met.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.