Memorizing vs. Understanding: Catching Overfitting with Train, Validation and Test Sets (Tech Explained from Zero #2)

A curve that scores perfectly on the training data, and a straight line that misses a little. You can't tell which is better until you measure on data the model never saw. We add five unseen students (validation) and five sealed students (test) to last part's five, and follow the overfitting curve, why we split three ways, and regularization — a fine on wiggly curves — with numbers you can check by hand. With a companion video.

About this series — Tech Explained from Zero is built so that each part stacks on the previous one: read (or watch) from #1 in order. Every part has a companion video with the same content. Part 2 continues Part 1 (What does AI actually learn?). If you know what a loss (mean squared error) is, you're ready.

Homework from last time: is a curve with zero loss a good model?

Last time we fitted two rules to five students' study time $x$ and quiz score $y$ (made-up data for the explanation).

• Straight line $\hat{y} = 2x$: loss (mean squared error) on the five students = 0.8
• Quartic curve (passes exactly through all five points): loss on the five students = 0

On these numbers alone, the curve wins. But it predicted −14 points for someone who studied zero hours, and 0 points for seven hours. Something's off. How do we put "something's off" into a number? That's today's topic.

The student who memorized past exams

Think of exam prep. Two students each get five past exams.

• Student A memorizes the answers. Give A a past exam, and A scores 100%.
• Student B learns how to solve the problems, and still slips on a calculation now and then.

On past exams, A comes out ahead. But the real exam has different questions. What we really want to know is the real exam score, and we can't know it until we give questions they haven't seen.

Machine learning works the same way. The data used for learning is called the training data. The loss on the training data is the "past exam score." What we want is the loss on data the model hasn't seen yet: the real exam.

Measure on data you didn't show: the validation set

So we collect five more students and keep them completely out of learning. This is the validation set (also made-up data).

We ask both rules, built from last part's five students, to predict these five scores.

The line $\hat{y}=2x$ predicts 1, 3, 5, 7, 9. The misses are 0, 0, 1, 0, 0. The mean of the squared misses:

$$L_{\text{val}} = \frac{0^2+0^2+1^2+0^2+0^2}{5} = \frac{1}{5} = 0.2$$

In words: on five students it never saw, the line is off by 1 point for one student and exactly right for the other four, so its average squared miss is 0.2.

The quartic curve predicts:

The worst miss is the half-hour student: the truth is 1 point, the curve says −3.55. That one student alone contributes a squared miss of about 20.7. Averaged over all five, the curve's validation loss is 5.13.

The curve wins on training; the line wins easily on validation. The curve had just memorized last part's five scores, without capturing how study time relates to the score. Fitting the training data so closely that you miss on data you haven't seen is called overfitting.

The line's validation loss (0.2) being lower than its training loss (0.8) is luck. With only five students on each side, which five you happened to get moves the numbers a lot. Validation is not guaranteed to be worse than training. What matters is measuring on data the model never saw.

The overfitting curve: what happens as the degree goes up

There are candidates between the line and the quartic. We vary the polynomial degree from 0 to 4, fit each one to last part's five students (the training data) to make its loss as small as possible, and compare training and validation losses.

• Degree 0: a flat line (the same score for everyone — here the mean, 6 points)
• Degree 1: a straight line (it comes out as the same $\hat{y}=2x$ as last time)
• Degree 2: a parabola
• Degree 3: a cubic curve
• Degree 4: the quartic through all five points

Drawn as two lines, the shape is clear.

• Training loss keeps falling as the degree goes up. More freedom to bend means getting closer to the five points in hand. At degree 4 it hits 0.
• Validation loss bottoms out at degree 1 and then rises: 0.80 at degree 2, 1.23 at 3, 5.13 at 4.

On the left (degree 0), the rule is too simple to fit even the data in hand. That's called underfitting. On the right (degrees 3 and 4), training loss is near zero while validation loss climbs: that's overfitting. A widening gap between training and validation is the clearest sign of overfitting.

In this example the lowest validation loss is at degree 1: the straight line. Looking at this table, we pick the line.

Why split three ways: the test set

There's a catch. We picked, out of five candidates, the one with the lowest validation loss. That means we've already looked at the validation data in order to choose.

It's like trying five study methods, checking each against the same practice exam, and keeping the one with the best practice score. That method's practice score may look good partly because it happened to suit that particular practice exam. The more candidates you try, the easier it is to pick one whose low number was partly luck.

So we split the data three ways.

The test set stays sealed until you're done choosing, and you open it once. For this series' example, we set aside five more students for testing (made-up data).

The chosen line $\hat{y}=2x$ predicts 2, 4, 6, 8, 10. The misses are 1, 0, 0, 0, −1, so the test loss is

$$L_{\text{test}} = \frac{1^2+0^2+0^2+0^2+(-1)^2}{5} = \frac{2}{5} = 0.4$$

The number you report is this 0.4 — not the validation 0.2.

What if you looked at the test result and picked a different degree? Then the test data becomes data you "looked at to choose," and drops to the same role as validation. You can't keep reopening the test — that's the whole point of splitting three ways.

Real splits are much larger than five each. scikit-learn's train_test_split puts 25% of the data on the test side if you don't specify a size. When you don't have enough data to set aside a fixed validation set, you use cross-validation: split the training data several ways and take turns using each part for validation (How big should the validation split be? measures whether the ratio or the count matters more).

Regularization: a fine on wiggles

Choosing a simpler model isn't the only way to fight overfitting. You can also keep the freedom to bend, but make the model reluctant to bend for no reason. This is called regularization.

Write the quartic with coefficients:

$$\hat{y} = c_0 + c_1 x + c_2 x^2 + c_3 x^3 + c_4 x^4$$

For the curve through all five points, the coefficients were $c_1 \approx 27.08$, $c_2 \approx -13.79$, $c_3 \approx 2.92$, $c_4 \approx -0.21$. Large coefficients cancel each other out to force the curve through every point. So we add a penalty on the size of the coefficients to the loss:

$$L_{\text{total}} = \underbrace{\frac{1}{N}\sum_{i=1}^{N}(y_i-\hat{y}_i)^2}_{\text{last part's loss}} \;+\; \lambda \sum_{j=1}^{4} c_j^2$$

In words: the score to minimize is now "how badly the curve misses the data" plus "a charge for large coefficients," with $\lambda$ setting how steep that charge is.

• The first part is the same mean squared error as before: smaller when the fit is closer
• The second part is the penalty: the squares of the coefficients $c_1$ to $c_4$, added up (the intercept $c_0$ isn't penalized)
• $\lambda$ (lambda) is the strength of the penalty. Zero means no penalty; the bigger it is, the harder the model tries to keep the coefficients small

Learning now looks for the coefficients that make this total smallest. Pushing the coefficients up to fit the data exactly raises the penalty. As a result, a smooth curve that misses the data a little tends to win. Regression with this kind of penalty is called ridge regression.

Here's the quartic refitted to last part's five students with different values of $\lambda$:

• As $\lambda$ goes up, the sum of squared coefficients shrinks from 932 to 0.04
• Training loss rises gradually from 0 (the curve no longer passes through all five points)
• Validation loss falls from 5.13 to 0.52, then climbs back to 1.39 at $\lambda=1$ (the penalty is now so strong that the coefficients are held too small and the curve no longer fits the data — training loss is up to 0.55 as well)

It's the same valley shape as with the degree. And $\lambda$, too, is chosen by validation loss. Here, $\lambda=0.01$ wins.

Note that in this example the plain line (validation 0.2) still beats the regularized quartic (validation 0.52), because the data comes from a nearly straight-line relationship. Regularization isn't a trick that wins every time; think of it as a knob for holding back overfitting when you use a complex model. The Deep Learning textbook (Chapter 7) notes that in practical deep learning, the best-fitting model is almost always "a large model that has been regularized appropriately."

Values like $\lambda$ and the degree, which a person sets before learning, are called hyperparameters. Last part's learning rate is one too. Parameters (coefficients) are set with the training data; hyperparameters are set with the validation data. Keep this division of labor in mind, and the reason for the three-way split becomes much clearer.

In code

Training and validation losses for degrees 0 to 4, using NumPy's polyfit:

import numpy as np

x_tr = np.array([1, 2, 3, 4, 5]);           y_tr = np.array([2, 5, 5, 7, 11])
x_va = np.array([0.5, 1.5, 2.5, 3.5, 4.5]); y_va = np.array([1, 3, 6, 7, 9])
mse = lambda c, x, y: np.mean((np.polyval(c, x) - y) ** 2)

for deg in range(5):
    c = np.polyfit(x_tr, y_tr, deg)
    print(deg, round(mse(c, x_tr, y_tr), 2), round(mse(c, x_va, y_va), 2))

Output:

0 8.8 8.8
1 0.8 0.2
2 0.57 0.8
3 0.07 1.23
4 0.0 5.13

polyfit sets the coefficients by looking only at the training data (x_tr, y_tr). The validation data (x_va, y_va) is used only to measure the loss — that's the key point.

How it's used in practice

When data scientists and machine learning engineers start a new prediction problem, they split the data first, before anything else. If you explore the whole dataset to decide on preprocessing, or build features over all of it, information from validation and test leaks into training and the numbers look better than they are.

Names you'll actually touch

• scikit-learn's train_test_split(X, y, test_size=..., random_state=...): splits into training and test. If you set neither test_size nor train_size, the test side gets 25%
• scikit-learn's cross_val_score(model, X, y, cv=...): cross-validation. If you don't set cv, it uses 5 folds
• scikit-learn's Ridge(alpha=...): the ridge regression in this article. Its documented objective is "sum of squared errors + alpha × sum of squared coefficients," so with this article's mean-based $\lambda$, $\alpha = N\lambda$
• PyTorch's optimizer argument weight_decay: pulls the weights slightly toward zero at every update, which works the same as a penalty on the sum of squared weights (for torch.optim.SGD)

Pitfalls that cause real accidents

• Preprocessing the whole dataset before splitting (data leakage). If you standardize with the mean and standard deviation of all the data and then split, validation and test information leaks into training. Compute the mean and standard deviation from the training data only, and apply those to validation and test
• Reopening the test set. If you keep adjusting settings after looking at the test number, the test set takes on the role of validation, and the number you report becomes optimistic
• Shuffling time-series data before splitting. If a model that predicts tomorrow is trained on data that includes the future, it's predicting with information it won't have in production. Split in time order: train on the past, validate on the future (TimeSeriesSplit in scikit-learn)
• The same person or object on both sides. If different photos of the same patient land in both training and validation, the model can score high just by recognizing that person. Split by person or object (GroupKFold)

Questions you'll get in interviews and at work

• "Training accuracy is 99%, validation is 70%. What's going on, and what do you do?" — Line of reasoning: the large gap between training and validation suggests overfitting. Try a simpler model, stronger regularization, or more data, watching the validation number
• "What's the difference between validation and test data? Isn't validation enough?" — Line of reasoning: validation is used to choose models and hyperparameters, so once you've chosen, it's optimistic. The final score is measured once on test data that played no part in choosing
• "How do you set the strength of regularization?" — Line of reasoning: prepare several candidates and pick the one with the lowest loss on validation data (or in cross-validation). The $\lambda$ table in this article is exactly that procedure

Summary

• Training loss is the "past exam score." What you want is the loss on data the model hasn't seen
• Split the data into training, validation and test. Set the parameters with training, pick candidates with validation, and open the test set once at the end
• In this example, as the degree went up, training loss kept falling while validation loss bottomed out at degree 1 and rose. When training loss falls but validation loss rises, suspect overfitting
• Regularization adds a penalty of "$\lambda$ × sum of squared coefficients" to the loss, holding back needless bending. $\lambda$ is chosen by validation too

Next (Part 3): How is a neural network different from a straight line? — "activation" that bends a line, and what stacking layers buys you.

Go deeper

• Overfitting and evaluation design — cross-validation, data leakage, choosing metrics on imbalanced data
• How big should the validation split be? — an experiment that varies the ratio
• Weight initialization and regularization — weight decay and dropout in neural networks