Interpretability and the safety case

Lesson 5 of 5 in Compute Governance, Export Controls, and the Science of AI Safety.

Behavioural testing has the limits you just catalogued. The research bet on getting underneath behaviour is mechanistic interpretability: reverse-engineering what a network’s internals actually compute. The field’s progress is real and easy to overstate in the same breath. Sparse autoencoders now decompose model activations into millions of human-recognisable features — Anthropic’s 2024 work famously isolated a Golden Gate Bridge feature and, by amplifying it, made the model obsessively steer conversations toward the bridge — and 2025 attribution-graph work began tracing multi-step circuits: watching a model plan a rhyme before writing the line, or fabricate a plausible-looking reasoning chain after already fixing its answer.

Hold both halves of the expert view. Interpretability offers things behavioural evals never can: detecting why a model produced an output, flagging deception-associated features firing while the model claims honesty, and auditing for capabilities never elicited in testing. And it covers, today, a small fraction of what frontier models compute. When Anthropic’s CEO published The Urgency of Interpretability in 2025, the argument was precisely that the field is in a race: capability is scaling faster than our ability to see inside it. A governance professional should treat interpretability findings as powerful, partial evidence — never as the full transparency mandate-writers sometimes imagine has already arrived.

Which brings us to the artifact designed to hold all this evidence together: the safety case. Borrowed from nuclear, aviation, and rail regulation, a safety case is a structured argument, supported by evidence, that a system is acceptably safe for a specific use in a specific context. Not a checklist, not a pile of eval scores — an argument, with explicit claims, the evidence for each, and the assumptions doing silent work.

Frontier safety cases in practice lean on three argument families, usually in combination:

• Inability — the model cannot do the dangerous thing (evidence: capability evals — with every limit from the last lesson attached).
• Control — even if the model tried, safeguards would catch or contain it (evidence: monitoring, sandboxing, red-team results against the safeguards).
• Trustworthiness/alignment — the model would not try (evidence: alignment training, interpretability probes — today the weakest and most contested family).

The UK safety institute and several labs have published safety-case sketches; the EU’s GPAI Code of Practice pushes systemic-risk providers toward exactly this structure in their model reports. The methodology matters to you for one reason above all: it gives the interrogator a shape to attack. A checklist can only be complete or incomplete; an argument can be wrong, and you can find where.

Key terms: mechanistic interpretability, safety case, sparse autoencoder, frontier safety framework

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.