Report, follow-up, and the certification decision

Lesson 5 of 5 in Lead Auditor Track: Auditing an AIMS with ISO 19011 Discipline.

The report is the audit’s only durable output — the week of interviews evaporates; the report is what the decision-maker, the accreditation body’s witness, and next year’s audit team will read. ISO 19011 wants it complete, accurate, concise, and clear, carrying: the objectives, scope, and criteria; the audit client and team; dates and locations; the findings with their evidence; the conclusions against the objectives; unresolved diverging opinions; and the sampling caveat. Craft points that separate professionals: conclusions answer the objectives as stated in the plan, positive evidence gets recorded too (what was sampled and found conforming — the decision-maker needs the whole picture, and fair presentation demands it), and the report reaches the auditee promptly under the confidentiality terms agreed up front.

Then the part juniors underestimate: evaluating the corrective-action response. For each nonconformity the auditee returns a plan, and you accept or reject it against the four-element anatomy you already know from the auditee’s side — correction, root cause, systemic action, evidence of effectiveness. The evaluation discipline: a root cause that ends at a person is not a root cause. ‘The engineer forgot’ answers nothing; why did the system depend on one engineer’s memory? Reject restatements of the finding dressed as analysis, reject pure quick fixes (‘we uploaded the missing document’ — that is correction, where is the corrective action?), and reject effectiveness claims with no evidence plan. Accepting a weak plan is quietly signing next year’s repeat finding — which will then be your finding to escalate.

From closing meeting to certification decision — and around again

Closing meeting
Findings presented in four parts, graded; conclusion stated; disputes resolved on evidence or recorded as diverging opinions.
Audit report issued
Objectives, scope, criteria, findings + evidence, conclusions, unresolved disagreements, sampling caveat. Distributed per the agreed confidentiality terms.
Auditee submits corrective action plan
Per NC: correction, root-cause analysis, systemic corrective action, effectiveness evidence plan — on the CB’s clock (majors commonly ~90 days).
Plan credible?
Team leader evaluates: does the root cause reach the system? Does the action prevent recurrence? Quick fixes and blame-a-person analyses bounce.
Plan rejected — rework
State precisely why: which element is missing or hollow. Rejection is not hostility; accepting a bad plan is.
Verify implementation & effectiveness
Evidence review for most minors; follow-up audit for majors where the CB requires it. Verification asks: did it work, not was it promised.
Team leader recommends
Grant / maintain / suspend — a recommendation, never a decision. The file goes to the CB’s independent function.
Independent CB decision
17021-1 separation: personnel independent of the audit team review the file and decide. The auditor who built rapport all week does not sign the certificate.
Surveillance → recertification
The three-year rhythm from the previous module — every surveillance re-checks whether past corrective actions stayed effective.

A corrective-action plan, evaluated

Finding: bias re-testing was not performed after the hiring model’s retraining. Submitted plan: ‘Root cause: the ML engineer overlooked the re-test. Action: the engineer has been reminded and the test has now been run.’ Evaluation: reject. The ‘root cause’ blames a person; the ‘action’ is a correction of the instance plus a conversation. The system still contains no mechanism that triggers re-testing on retraining. A creditable plan would find the systemic cause (no release-gate step ties retraining to re-validation), change the system (re-test made a blocking step in the deployment pipeline, with a named owner), and offer effectiveness evidence (the next two retraining cycles’ gate records). The re-written plan is the auditee’s job to produce — yours is to know the difference on sight.

Note the constitutional point buried in that flow: the auditor recommends; the certification body decides. The 17021-1 separation you met from the auditee’s side is, from this chair, a protection — of the decision from your week of built rapport, and of you from a week of pressure. ‘I cannot grant or deny your certificate; I gather the evidence’ is both true and the best de-escalation line in the profession. Surveillance and recertification then keep the relationship honest across the three-year cycle: every return visit re-opens the corrective-action file before anything else.

What remains is the situations no procedure fully scripts — the ones that end careers when misjudged. The profession’s answer is always some combination of the seven principles plus one instinct: when in doubt, disclose to your CB and document.

The generous auditee

Lunch during the audit at the site canteen: fine and normal. A case of wine at the closing meeting, an invitation to the executive box, a ‘speaker fee’ for an internal talk next month: threats to integrity and independence, and to how the audit looks — which for impartiality is nearly the same as how it is. Decline, disclose to your CB, record it. Most CBs’ codes of conduct set explicit gift thresholds; the professional instinct is to stay far below them.

The soft-pressure client

‘We have been certified for six years, the renewal is priced, and your predecessor never raised this.’ Every clause of that sentence is true and none of it is evidence. Findings rest on this audit’s sample against this audit’s criteria. If the pressure escalates — hints about complaint letters, about switching CBs — document it and inform your CB: client pressure on auditors is itself information the impartiality committee needs.

The discovery outside scope

Auditing A.7 data controls, you notice what looks like processing of scraped biometric data with no legal basis — a possible legal violation, not just a clause gap. You are not a regulator and the audit is not an investigation, but confidentiality is not complicity: report the concern through your CB’s defined process. 17021-1-accredited bodies have procedures for legally significant discoveries, and some laws impose reporting duties that override contract. What you never do is trade silence for goodwill.

The revolving door

The auditee offers you a job — genuinely, flatteringly, mid-engagement. Disclose immediately and withdraw from the engagement; a person negotiating employment cannot audit their prospective employer. Post-employment the door swings the other way too: having worked for (or consulted for) an organisation triggers cooling-off periods before you may audit it. The register of who audited whom, when, exists because memories are conveniently short.

The competence cliff

Mid-audit, the auditee’s architecture turns out to be far beyond the team’s AI depth — the ‘simple deployer’ is fine-tuning foundation models. Due professional care includes knowing your limits out loud: inform the CB, get the technical area re-assessed, add competence or re-scope. Bluffing through evidence you cannot evaluate produces a worthless audit wearing a confident report.

Tool: ISO/IEC 42001 Exam Hall — Drill the Lead Auditor body of knowledge under exam conditions — audit method, findings grading, and ethics calls against the clock.

Tool: AIMS Builder & Audit — See the other side of the table: build the scope and SoA an auditee would defend — knowing how implementers think is how auditors sample.

Key terms: audit report, corrective action, root cause analysis, verification of effectiveness, certification decision

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.