Focus tour II: datasets, system design, and the release gate

RAIDP — Dataset planning: know your data like an auditor will (18 BPs)

The largest focus area, and deliberately so — most responsible-AI failures are data failures wearing a model costume. Its practices centre on three artifacts. A dataset registry: an inventory of every dataset the system touches — training, tuning, evaluation — with ownership and permitted uses, the data-shaped sibling of your AI inventory. Representativeness analysis: does the data actually cover the population and conditions the use case serves, per stakeholder group — the RAIBR risk registry tells you which groups must be represented for the fairness criteria to be testable at all. And documentation: datasheets recording provenance, collection method, consent basis, known gaps, and permitted uses, so the next team inherits knowledge instead of folklore.

RAISP — System planning: bake protections in, do not bolt them on (18 BPs)

The focus area’s central design argument: protections belong in the core system design, with add-on filters as defence in depth — not as the load-bearing wall. A toxicity filter taped over an unconstrained generator fails differently than a system whose retrieval scope, action space, and output format were designed so whole harm categories are structurally impossible. Filters are bypassable and drift out of date; architecture is neither.

Two more RAISP themes reach beyond the system boundary. Provenance indicators: generative outputs should carry signals of their origin — content provenance credentials or watermarking (AI content) — so downstream audiences can make the informed choices the transparency dimension demands. And paired tests: design evaluations as matched input pairs that differ in exactly one attribute — a name, a dialect, a demographic marker — so that a difference in output is attributable to that attribute rather than to noise. It is the counterfactual logic of a bias audit, moved forward into test design.

RAIER — Evaluate and release: the gate, and the honest exits (7 BPs)

The smallest focus area holds the sharpest teeth. Evaluation here means demonstrating — not asserting — that the release criteria are met: results carry the statistical confidence RAIRC demanded, and they are corroborated — reproduced independently enough that a single lucky run, a leaked test set, or one enthusiastic evaluator cannot manufacture a green light.

When criteria cannot be met, the lens offers an exit most frameworks forget: narrow the use case. A model that cannot meet the veracity bar for open medical questions might meet it for questions about a curated formulary; a classifier that fails on one language might release for the two where the criteria hold. Narrowing shrinks the claim to fit the evidence — and because the use case determines risks and criteria, a narrowed use case loops back through RAIUC honestly, with smaller numbers. The dishonest alternative — keeping the broad claim and shipping anyway — is precisely what written release criteria exist to make visible.

Walk the release gate

Interactive decision tree — outcomes:

• Release — with the monitoring plan armed

  Criteria met, at confidence, corroborated. Release is defensible — and RAIMON now inherits the release metrics as its drift baselines.

• Corroborate before you celebrate

  RAIER treats a single uncorroborated result as insufficient: lucky runs, contaminated test sets, and motivated grading are all real. Reproduce independently before the gate opens.

• Iterate — the lifecycle is a loop

  Return to dataset planning and system planning with the failed criterion as the target. This is the nonlinearity the lens’s how to use guidance promised.

• Narrow the use case

  Shrink the claim to fit the evidence, then loop through RAIUC: the narrower use case gets its own risks and criteria, honestly re-derived. Smaller and true beats broad and hopeful.

• Do not release

  If no honest scope meets the criteria, the benefits no longer justify the residual risk. Written release criteria exist precisely so this outcome is visible — and survivable — before launch, not after an incident.