Findings that survive challenge

Lesson 4 of 5 in Lead Auditor Track: Auditing an AIMS with ISO 19011 Discipline.

A finding is a small legal document. It will be read by people who did not attend the audit, disputed by people whose bonus it threatens, and reviewed by a decision-maker who must act on it. So write every nonconformity in four parts, and never let one part borrow from another:

  1. The requirement — the criterion, cited: ‘ISO/IEC 42001, A.6.2.8 requires event logging for AI systems, and the auditee’s procedure AIP-14 requires decision logs retained 24 months.’
  2. The evidence — what you objectively found: ‘For the credit model, logs for 14 May and 2 June could not be retrieved; the logging service showed no entries before 1 July.’
  3. The nature of the nonconformity — the gap, stated as a gap: ‘Decision logging was not operating for at least eight weeks of the retention period.’
  4. The statement of nonconformity — the formal conclusion, graded: ‘This is a minor nonconformity against A.6.2.8 and procedure AIP-14.’

Notice what is absent: adjectives, blame, and advice. ‘The team failed to…’ becomes ‘logs could not be retrieved’. Findings describe the system; they never prosecute people.

Grading is the auditor’s judgment call, and it must run on articulable logic — the major/minor tests you learned from the auditee’s side (systemic absence or breakdown versus isolated lapse; patterns of related minors; contamination of core outputs) are now your tests, and you must be able to recite why a finding landed where it did. Grade on the evidence of system failure, never on how awkward the consequence is for anyone — a major that blocks a certificate the sales team promised is still a major.

Below nonconformity live two labels worth keeping distinct, though scheme vocabularies vary. An opportunity for improvement (OFI) flags something conforming that could work better. An observation records a fact worth carrying forward that is neither breach nor improvement advice — an early-warning condition, or something the sample could not fully resolve, flagged for the next audit’s attention. Neither obliges the auditee to act.

And here runs the brightest line in the profession: the auditor never consults. You may say what is weak (‘the SoA justifications for the A.7 exclusions are generic’); you may never say how to fix it (‘you should adopt this template, hire a data steward, buy this tool’). The moment you prescribe the solution, you have designed part of the system — and neither you nor your CB can ever impartially audit it again. 17021-1 bans CBs from consulting for their certification clients for exactly this reason. The auditee will invite you across the line daily, flatteringly: ‘what would you do?’ The professional answer: ‘My opinion of solutions is the one thing I am not allowed to give you. The requirement is X; the evidence is Y; how you close the gap is yours.’

Interactive sorting exercise: You are the team leader consolidating day-three notes. Grade each finding: major NC, minor NC, OFI, or observation.

The closing meeting is where discipline pays out. Present the findings — each in its four parts, graded, with the evidence — and the audit conclusion against the objectives. Restate the sampling caveat. Explain what happens next: corrective-action timelines, verification, the CB’s independent decision, and the appeals channel. If you shared findings as they emerged during the week, this meeting confirms rather than ambushes — the single best predictor of a civil closing.

Disputes still happen, and they are resolved on one axis only: evidence. If the auditee produces the missing record in the meeting, thank them and amend or withdraw the finding — fair presentation cuts both ways, and an auditor who cannot retract has stopped being evidence-based. If they argue interpretation, restate requirement and evidence, and hold. If agreement cannot be reached, ISO 19011’s answer is candour: record the diverging opinion in the report and let the process (CB review, appeal) resolve it. What you never do: haggle a major into a minor to end the meeting, or accept a promise of future fixes as if it erased a present gap.

Key terms: nonconformity statement, major nonconformity, minor nonconformity, opportunity for improvement, closing meeting

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.