Detecting Contamination
Lesson 2 of 3 in Data Contamination.
If contamination is the default, detection is the response — and the classic detector is n-gram overlap, the method the GPT-3 paper used on its own corpus. The concept: treat a test item as suspect if it shares sufficiently long runs of tokens with any document in the training data. Scan the corpus, flag the overlapping items, then report two numbers — the score on the full benchmark and the score on the clean subset. If the two diverge, contamination was doing real work in the headline number.
N-gram overlap earned its status honestly. It is cheap enough to run against a corpus of trillions of tokens, simple enough to explain in a sentence, and it produces an artifact anyone can audit: a concrete list of flagged items. That transparency is why it became the standard disclosure format for contamination analyses in model reports — and why its blind spots matter so much, because everything the standard method misses tends to stay missed.
| Method | What it catches | What it misses |
|---|---|---|
N-gram overlap scan | Verbatim and near-verbatim copies of test items, at full corpus scale, with an auditable list of flagged items | Paraphrases, translations, and reformatted items — a rewrite shares few or no long token runs with the original, so surface matching cannot see it |
Training-time deduplication | Repeated copies of leaked text — mirrors and re-quotes are where much benchmark text hides, and dedup measurably reduces train–test overlap (Lee et al., 2021) | An item that appears once in the corpus — dedup removes repeats, not first occurrences — and every semantically equivalent rewrite |
Canary strings | Proof of ingestion: a unique marker embedded in eval documents that pipelines can filter on, and that an auditor can later test by prompting the model to complete it | Everything published before the canary existed, and any pipeline that strips markers or is never checked against them — a prospective defense only |
Memorization probing | Strong verbatim memorization even without corpus access — prompt with the start of a test item and check whether the model completes it word for word | Contamination that lifted the score without leaving verbatim recall behind; results are suggestive rather than conclusive, and thresholds are hard to calibrate |
Two structural limits run through that whole table. The first is surface form: overlap scans and Deduplication both operate on strings, so any transformation that preserves the content while changing the words — a paraphrase, a translation, a table rendered as prose — slides past them. The deep dive below explains why this is not an engineering gap that better tooling will eventually close.
The second is corpus access. Overlap scanning and dedup require reading the training data, and for most commercial models that data is closed. Third parties are left with inferential methods — memorization probes, suspicious accuracy patterns on items known to circulate publicly — which produce evidence, not proof. When a lab reports its own contamination analysis, you are trusting its methodology; when nobody reports one, you are trusting nothing at all. Deduplication remains the strongest upstream mitigation an honest lab controls — Lee et al. (2021) showed that deduplicating training data reduces both memorized output and train–test overlap — and the pipeline where it runs is covered in Data Pipelines and Curation.
Why paraphrased contamination is fundamentally hard to detect
Surface-form detectors ask a checkable question: does this exact text appear in the corpus? Paraphrase detection has to ask does text with the same content appear? — and that question gets harder the more carefully you pose it.
Start with the search space. Content-preserving rewrites are effectively unbounded: reorder clauses, swap synonyms, rename variables, alter the numbers that do not matter, translate into another language and back. Each transformation destroys the long shared token runs that n-gram scans depend on while leaving intact exactly what helps a model answer the item — a translated copy of a test set shares almost no n-grams with the original. No finite list of transformations covers the space, so no scanner enumerates it.
The obvious escalation is semantic matching: embed every test item and every training document, flag near neighbors. Now you need a similarity threshold, and every setting fails in one direction or the other. Set it strict and paraphrases slip through; set it loose and ordinary topical overlap gets flagged — any corpus that teaches arithmetic is “semantically similar” to a math benchmark, and a corpus with no such text would make for a strange model. There is no threshold at which the flags reliably mean this is a disguised test item rather than this is on-topic text.
The deepest problem is conceptual, not technical. A model trained on many rewordings of a test problem may hold no verbatim memory at all, yet still be recalling a specific solved instance rather than exercising a general skill. Deciding which of those is happening is not something any corpus scan can settle — it is the question did the model generalize?, which is the very thing evaluation was trying to measure in the first place. At the limit, contamination detection and capability evaluation collapse into the same unsolved problem. This is why careful contamination reports state bounds (“no verbatim overlap above the scan threshold”) rather than verdicts (“clean”).
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.