overfitting

Definition

When a model memorizes the noise and idiosyncrasies of its training set and therefore generalizes poorly to new data. The classic symptom is a training loss that keeps falling while validation loss turns upward, and it tends to appear when model capacity is large relative to the amount of data. The opposite failure is underfitting, where the model is too constrained to capture the real structure and error stays high even on the training set.