#train-test-split

1 articles

01 ·Machine Learning Basics·FREE·12 min read What Ratio Should the Validation Split Be — I Swept It, and What Mattered Was the Count, Not the Ratio When you split your data into training and validation, is it 80/20 or 70/30? I ran model selection over 30 candidates 2,000 times per condition and swept the validation share from 5% to 50%. There is a valley — but changing the ratio tenfold moves the loss by only 1.2x, while holding the ratio fixed and changing the count fourfold moves it 6.5x. Refitting on all the data with the chosen setting erases the valley entirely. When you cannot spare a few hundred rows, k-fold wins on average, but it is a coin flip on any single run.