Gaming, Goodhart, and Leaderboards
Lesson 3 of 4 in Benchmarks and Their Limits.
The proverb attributed to Goodhart’s work says it in one line: when a measure becomes a target, it ceases to be a good measure. Benchmarks were built as measures — thermometers held up to a model’s abilities. Leaderboards turned them into targets: rankings drive press coverage, enterprise shortlists, hiring, and funding. Once a number is worth money, it gets optimized, and optimization pressure finds every gap between the number and the thing the number was supposed to track.
Three families of gaming recur, in rising order of subtlety:
Benchmark-aware data selection. Choosing or generating training data because it resembles the test — from outright training on test items (Contamination, next module) to the greyer practice of synthesizing thousands of examples in a benchmark’s exact style and format. The model gets better at the benchmark’s dialect, not the ability.
Per-benchmark prompt and shot tuning. Reporting each benchmark under whichever shot count, example choice, and prompt format flatters your model most — while baselines run under defaults. Every setting is defensible in isolation; the asymmetry is the trick.
Cherry-picked harness settings. Running the eval several times and quoting the best; choosing the answer-extraction rule that happens to rescue your model’s formatting quirks; quoting your best decoding configuration against rivals’ defaults. Nothing was faked — the flattering subset was simply the only part published.
Interactive sorting exercise: Sort each practice: does it make the model genuinely better, or just make the number bigger?
None of this requires villains. Leaderboard dynamics produce gaming through ordinary selection effects: teams try many training runs, checkpoints, and eval configurations, and the one that tops the chart is the one that ships and gets the blog post. Even if every individual choice was made in good faith, publication itself filters for flattering noise. This is why serious model reports document their eval methodology and why the GPT-3 paper already shipped its own train–test overlap analysis (Brown et al., 2020) — at web scale, everyone knows the exam papers are lying around in the training data. The next module is about exactly that.
For you as a consumer of these numbers, the takeaway is calibrated distrust: a leaderboard rank is evidence, but weak evidence, and weakest precisely where the stakes are highest — at the top, between close rivals, right after a launch.
How harness choices move a score without touching the model
Mechanically, where does the movement come from? Walk the harness knobs one by one.
Shot count. Few-shot examples teach the format as much as the task: with zero shots, some models lose points to answer-formatting mismatches the extractor cannot parse; with several shots, those free losses vanish. So “5-shot” versus “0-shot” numbers for the same model on the same benchmark can differ meaningfully — neither is wrong, they measure different conditions. The examples’ identity and order matter too, which is why fixed-seed example selection belongs in an eval config.
Scoring mode. Multiple-choice can be scored by generating a letter and parsing it, or by comparing the model’s likelihood on each option’s text. The two modes reward different things (parseable formatting versus raw preference), and models rank differently across them.
Answer extraction. A regex hunting for the final number in a chain of reasoning either finds it or does not. A model whose arithmetic is right but whose layout confuses the extractor scores as wrong — so extraction rules are silent point-assignment policies.
Decoding settings. Greedy decoding versus Sampling, Temperature, number of samples — pass@k moves with all of them, as lesson one’s deep dive showed. And whether Chain-of-thought (CoT) prompting is permitted changes what reasoning benchmarks measure at all: “with CoT” footnotes are load-bearing.
Each knob is a legitimate experimental condition. The gaming pattern from this lesson is never the knob itself — it is asymmetric knob settings across the models being compared. These are the same prompt-side levers you tune deliberately when engineering an application; the adaptation domain’s prompting module covers them from that angle: see prompting as engineering.
Interactive checkpoint quiz (1 questions) — open this page in a browser to take it.