Benchmark Contamination — How to Doubt a High Score

How to tell whether a benchmark score reflects ability or memorization. Covers the three kinds of contamination, the three detection tools — n-gram overlap, embedding neighbors, membership inference — and where each stops working, plus canary strings and time-based splits, ending in a checklist for reading papers skeptically.

Rethinking Benchmark and Contamination for Language Models with Rephrased Samples

The Student Who Memorized the Answer Key

A student scores in the 99th percentile on a practice exam. It might be skill. But if they happened to get hold of the same question booklet the night before, that percentile is proof of memory, not of ability. The awkward part is that you cannot tell the two apart by looking at the answer sheet. Correct answers look identical either way.

This is exactly the situation in language model evaluation. Training data comes from large web crawls, and the benchmarks used for evaluation sit on GitHub, in explainer blog posts, on dataset-sharing sites, in notebooks. The more famous a benchmark is, the more copies of it exist, and the higher the chance it lands in the next model's training data. That is not anyone cheating. It is the inevitable result of measuring with publicly released questions.

This article is about how contamination is detected — and, more importantly, how far detection can possibly go.

Contamination Comes in Three Kinds

People lump it all together, but separating it by actual damage makes the argument much clearer.

Input-only contamination means the question text is in the corpus but the answer is not. The effect is mild, though not zero: a model can still pick up quirks unrelated to the content, like "questions written in this style tend to have C as the answer."

Input-plus-answer contamination is the real problem. When question and answer appear together, the model can recall instead of reason. Explainer articles and leaderboard submission repos usually publish questions and answers side by side, which makes this the most common kind.

Task-format contamination is the subtle third. The questions themselves are absent, but the benchmark's formatting, its instruction templates, and the shape of answers its grader rewards are all over the training data. Scores go up, but what went up is fluency in taking that particular test. It is not cheating; it just isn't the ability you meant to measure.

What all three share: a contaminated score is high only on the contaminated test. Move to different questions probing the same skill and it falls. That property is the foothold for every detection method below.

Memorization and Generalization Separate on the Graph

A contaminated model behaves much like the textbook case of overfitting. Performance on training data keeps climbing while performance on unseen data plateaus and then declines.

The difference from ordinary overfitting is that the gap is invisible. Overfitting reveals itself when you plot training and test error together. Contamination is the situation where the thing you believed was test error was training error all along. The ruler itself is broken, so staring at the graph shows nothing wrong. The basics are in Overfitting and Evaluation Design.

That said, memorization does leave symptoms. Reword the question slightly and the model suddenly fails. Shuffle the answer options and the score drops. The final answer is right while the working that led to it is incoherent. The last one is the clearest tell: if a model actually reasoned, its steps and its conclusion would agree. A correct conclusion propped up by nonsense steps is what recalling an answer and inventing a justification looks like.

Why This Is Structural, Not an Accident

Treating contamination as "avoidable if you're careful" leads to the wrong countermeasures. Three forces push at once. Scale of data — nobody can read trillions of tokens to check, so removal must be automated, and the moment it is automated, copies that don't match as strings slip through. Popularity of benchmarks — good ones get cited, redistributed, and translated, which means the more useful a benchmark is, the faster it breaks. Leaderboard pressure — even with perfectly clean training data, tuning your data mix and prompts dozens of times while watching the score routes test-set information into the model through a human.

So the question is never "can we prevent contamination" but "how far can we detect it". And detection comes in only two flavors: inspect the corpus and look for matches, or poke the model and watch how it breaks. The first requires public training data; the second has a hard ceiling on how strong its evidence can be. Knowing both is what lets you judge how much a paper's "we checked for contamination" is actually claiming.