Drift: how models rot, and how monitoring catches it

Lesson 2 of 5 in Human Oversight, Post-Deployment Monitoring, and Incident Response.

A deployed model is a snapshot of the world at training time, and the world keeps moving. You met distribution shift conceptually in foundations; here is the operational taxonomy monitoring teams actually alert on:

• Data drift (covariate shift) — the inputs change distribution: a new customer segment arrives, a marketing campaign shifts the applicant mix. The input-output relationship may still hold; the model is now extrapolating.
• Concept drift — the relationship between inputs and outcome changes: fraudsters adapt tactics, a pandemic rewrites spending behaviour. The same input now means something different. This is the killer, because inputs can look perfectly normal while accuracy collapses.
• Upstream / schema drift — the pipeline changes: a vendor renames a field, a unit switches from dollars to cents, a feed silently starts sending nulls. Not statistics — engineering. It causes some of the fastest, dumbest production failures.
• Performance decay — the symptom the others produce: accuracy, precision, or fairness metrics sliding.

The workhorse statistic at governance level: the Population Stability Index (PSI) compares today’s distribution of a feature or score against the training baseline. Banking rule of thumb: below 0.1 stable, 0.1–0.25 investigate, above 0.25 significant shift — act. PSI on the model’s score distribution is the single most watched drift dial in credit risk.

What a working monitoring operation contains. Metrics chosen per risk, not per convenience: performance where ground truth allows, drift proxies where it lags, fairness metrics on a schedule (drift is not colour-blind — a model can decay faster for one group), and usage patterns (volume spikes, new geographies, off-label use). Thresholds with named owners and runbooks — an alert nobody owns is decoration. Dashboards split by audience: the first line watches operational dials; the second line watches trend lines, threshold breaches, and override rates.

Generative systems change the dial set, not the logic. With no single ground truth, monitoring shifts to output-quality sampling (rubric-scored by humans and calibrated LLM judges), safety-event rates (policy violations, jailbreak attempts caught), hallucination rates on answerable-from-source questions, and prompt-trend analysis — what users ask reveals emerging off-label use faster than any survey. The EU AI Act makes monitoring a legal duty for high-risk systems: Art 72 requires providers to run a documented post-market monitoring system that actively collects and analyses performance data throughout the lifetime — the pharma model imported into AI.

Interactive sorting exercise: Read the symptom, name the drift.

Interactive checkpoint quiz (2 questions) — open this page in a browser to take it.