Canada’s AIA: the scored assessment, dissected
Lesson 3 of 5 in AI Risk and Impact Assessment Methodologies.
Canada built the assessment the rest of the world keeps citing. Since 2019, the federal Directive on Automated Decision-Making has required every department using an automated system to make or recommend administrative decisions about clients to complete the Algorithmic Impact Assessment — a standardised, scored, published questionnaire. It remains mandatory in 2026, and it is worth dissecting in detail for two reasons: it is the most mature worked example of turning “assess your AI” into an operating procedure, and its design choices — scoring, mitigation credit, tiered obligations, publication — are the levers every internal assessment program has to set one way or another.
The mechanics: the questionnaire poses 65 risk questions (maximum 169 points) across the project, the system, the algorithm, the decision, the impact, and the data — and 41 mitigation questions (maximum 77 points) covering consultation, de-risking measures, data quality, procedural fairness, and privacy. Answer them and the tool computes a raw impact score; if your mitigation score reaches 80% or more of the mitigation maximum, 15% is deducted from the raw score. The net score maps to an Impact Level from I (little impact) to IV (very high impact) — and the level, not the score, is what drives obligations.
The AIA pipeline: from questionnaire to published obligations
- Design phase begins
The Directive requires the AIA at the design stage — early enough that its outcome can still change the system, not just document it.
- 65 risk questions
Project, system, algorithm, decision, impact, and data dimensions. Max 169 points. Questions include whether the system affects vulnerable groups, uses personal data, and how reversible the decision’s effects are.
- 41 mitigation questions
Consultation performed, data-quality measures, procedural-fairness safeguards, privacy protections. Max 77 points.
- Mitigation ≥ 80%?
If the mitigation score reaches 80% of its maximum, the tool deducts 15% from the raw impact score — a designed incentive to actually build the safeguards before assessment, not after.
- Impact Level I–IV
The net score maps to one of four levels. The level — not the number — drives obligations: peer review, notice, human involvement, explanation, training, contingency planning.
- Publish on Open Government Portal
Every completed AIA is public. Journalists, researchers, and affected people read them — Canada’s portal is one of the best open datasets on government AI anywhere.
- Re-assess before production
The AIA runs twice at minimum: at design and again before production, and must be updated when system functionality or scope changes.
| Obligation | Level I | Level II | Level III | Level IV |
|---|---|---|---|---|
Peer review | None required | One qualified expert review (or equivalent, e.g. published specifications) | Qualified expert review plus additional scrutiny appropriate to the risk | Two independent expert reviews and publication — the maximum external-challenge tier |
Human involvement in decisions | Decisions may be fully automated | Decisions may be fully automated | Specific human intervention points required — the final decision must involve a human | Same as III, with the most stringent design of intervention |
Explanation to affected persons | General explanation available (e.g. FAQ) | Meaningful explanation on request | Meaningful explanation provided with each decision | Meaningful explanation with each decision |
Notice | Plain-language notice of automation | Plain-language notice, prominently | Published, prominent notice plus documentation of the system | Maximum-transparency notice and documentation |
Training & contingency | Basic documentation | Role training on system design and function | Recurring training; contingency/fallback plans required | Recurring training and approved contingency plans before operation |
Why should a private-sector practitioner care about a Canadian public-sector directive? Because its four design choices answer the four questions your own internal assessment scheme must answer:
Standardised questions beat blank pages. A fixed questionnaire is gameable, but a blank “describe your risks” template produces nothing comparable across systems. Canada chose comparability — and its portal proves the payoff: you can line up every federal AI system by impact level.
Mitigation credit creates incentive — and a gaming surface. The 15% deduction rewards teams that build safeguards before assessing. It also invites optimistic self-scoring, which is why second-line challenge of mitigation answers matters more than challenge of risk answers.
Levels, not scores, drive obligations. Nobody argues about whether 87 versus 91 points matters; they argue about level boundaries. Categorical tiers make obligations legible and defensible — the same reason the EU chose risk tiers over risk scores.
Publication disciplines honesty. An assessment the public will read is written differently from one filed in a drawer. Few private organisations will publish their AIAs — but internal transparency (to the board, to internal audit, to works councils) buys a measurable slice of the same discipline.
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.