data-constrained

Definition

The regime where available training data, not compute, is the binding constraint. It forces repetition — multiple epochs over the same corpus — and the research question becomes how many repeats still buy improvement before returns diminish and overfitting sets in. This matters because the supply of high-quality human-written text is finite, which is much of the motivation behind synthetic data and aggressive data-quality filtering.