Education: proctoring, grading, and the GenAI classroom
Lesson 4 of 5 in Sector by Sector: Health, Employment, Finance, Education, and Vehicles.
Education concentrates every AI governance problem into one setting: subjects who cannot opt out, decisions that shape life trajectories, and a user base that is largely minors. Three episodes define the field.
The Ofqual A-levels fiasco (UK, 2020). With exams cancelled, the regulator standardised teacher-predicted grades with an algorithm weighted by each school’s historical results — so bright students at historically weak (disproportionately poor) schools were downgraded en masse; nearly 40% of grades came down. Days of protests (“F*** the algorithm”) forced a full reversal. It remains the cleanest demonstration that an algorithm applied uniformly can still be structurally unfair, and that public legitimacy is a deployment requirement, not a nicety.
Remote proctoring. Pandemic-era exam surveillance normalised face detection, gaze tracking, and room scans in students’ bedrooms. The failures were predictable from this module’s earlier lessons: face-detection systems repeatedly failed to see darker-skinned students (documented complaints against ExamSoft during the 2020 California bar exam), flagging them for lighting “violations”; and in Ogletree v. Cleveland State (N.D. Ohio 2022), a federal court held a public university’s mandatory webcam room scan was an unreasonable search under the Fourth Amendment. The EU AI Act now lists proctoring-type systems — AI monitoring and detecting prohibited behaviour of students during tests — as high-risk (Annex III point 3(d)), and emotion inference in education institutions is flatly prohibited (Art 5(1)(f)).
AI-writing detectors. After ChatGPT, schools reached for detectors that claim to spot machine-written text. Stanford research showed them systematically misclassifying non-native English writers as AI (over half of TOEFL essays in the study flagged), and false accusations are effectively unappealable for the student. OpenAI withdrew its own classifier for low accuracy in 2023. Governance conclusion: a probabilistic accusation tool with documented subgroup bias, deployed in a high-stakes disciplinary context with no due process, fails every test this curriculum has taught you.
US
Instruments: FERPA (education records — vendor data flows), COPPA for under-13 ed-tech, state student-privacy statutes (California’s SOPIPA template: no targeted ads, no profiling, no sale of student data), plus constitutional limits at public institutions (Ogletree).
GenAI posture: guidance-first — the Department of Education’s 2023 AI report urged “humans in the loop” and named teacher displacement and surveillance risks; a 2025 executive order pushed AI literacy. No binding federal classroom-AI rule exists; districts set policy, and detector-driven discipline is largely ungoverned.
EU
Annex III point 3 makes the four gateway moments high-risk: admission/access decisions, learning-outcome evaluation (grading), assessment of the appropriate level of education, and test proctoring. Deployers that are public bodies owe fundamental-rights impact assessments (Art 27); emotion recognition in education is prohibited.
An Ofqual-style grading system deployed today would be a textbook high-risk system — dataset-representativeness duties would have surfaced the school-history weighting problem before results day.
UNESCO
The first global GenAI-in-education guidance (September 2023): a human-centred frame, a recommended minimum age of 13 for independent classroom GenAI use, mandatory teacher training, and government validation of GenAI systems before educational use — pointedly noting most tools were trained on data from and for adults in the Global North.
Non-binding, but it is the reference text education ministries cite, and it pairs with UNESCO’s 2021 AI Recommendation you met in the global domain.
Key terms: proctoring, ai detector, ferpa, student privacy, fundamental rights impact assessment
Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.