#cross-validation

3 articles

01 ·Machine Learning Basics·FREE·12 min read What Ratio Should the Validation Split Be — I Swept It, and What Mattered Was the Count, Not the Ratio When you split your data into training and validation, is it 80/20 or 70/30? I ran model selection over 30 candidates 2,000 times per condition and swept the validation share from 5% to 50%. There is a valley — but changing the ratio tenfold moves the loss by only 1.2x, while holding the ratio fixed and changing the count fourfold moves it 6.5x. Refitting on all the data with the chosen setting erases the valley entirely. When you cannot spare a few hundred rows, k-fold wins on average, but it is a coin flip on any single run. 02 ·Machine Learning Basics·★ MEMBER·10 min read Data Leakage and Experiment Hygiene — When the Score Is Too Good, Suspect It When a model scores better than you expected, that is not good news yet. This article splits data leakage into five concrete shapes — outcome-derived columns, preprocessing before the split, time running backwards, duplicates and groups, and a worn-out test set — then covers the detection moves that actually find them and the experiment records that let you dissect the day the number jumped. 03 ·Machine Learning Basics·★ MEMBER·8 min read Overfitting and Evaluation Design — Be Suspicious of 99% Accuracy Why a model that fits its training data perfectly falls apart in production: the division of labour between train, validation and test, cross-validation, the data leakage that ruins more projects than any modelling mistake, and how accuracy lies on imbalanced data.