diverse

Definition

Describes spread rather than volume: how much variety a dataset covers, or how different a model's samples are from one another. It matters because decoding settings that maximise likelihood collapse toward safe, repetitive outputs, and because narrow training data yields narrow behaviour. Diversity trades off against per-sample quality, so papers should report both.

Explainers using this term

• Building a Pretraining Corpus — From Web Sludge to Textbook QualityThe Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Paper Explained: Co-RL — Reasoning Without Labels, Emerging From a Diverse CohortCo-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL