#data-leakage
3 articles
01
·Evaluation & Judging·★ MEMBER·PAPER·10 min read
Benchmark Contamination — How to Doubt a High Score
How to tell whether a benchmark score reflects ability or memorization. Covers the three kinds of contamination, the three detection tools — n-gram overlap, embedding neighbors, membership inference — and where each stops working, plus canary strings and time-based splits, ending in a checklist for reading papers skeptically.
02
·Machine Learning Basics·★ MEMBER·10 min read
Data Leakage and Experiment Hygiene — When the Score Is Too Good, Suspect It
When a model scores better than you expected, that is not good news yet. This article splits data leakage into five concrete shapes — outcome-derived columns, preprocessing before the split, time running backwards, duplicates and groups, and a worn-out test set — then covers the detection moves that actually find them and the experiment records that let you dissect the day the number jumped.
03
·Machine Learning Basics·★ MEMBER·8 min read
Overfitting and Evaluation Design — Be Suspicious of 99% Accuracy
Why a model that fits its training data perfectly falls apart in production: the division of labour between train, validation and test, cross-validation, the data leakage that ruins more projects than any modelling mistake, and how accuracy lies on imbalanced data.