data

Definition

The raw material a model learns from and is measured against. Modern practice treats curation — filtering, deduplication, mixture weights, licensing — as at least as decisive as architecture, and many reported gains trace back to data rather than modeling. Keep training and evaluation data strictly separate; overlap between them, called contamination, silently inflates benchmark scores.

Explainers using this term

• Paper Explained: RayOrch — Batching Children Across Parents on GPUs Without Losing Track of Who Owns WhatRayOrch: Programming and Executing Lineage-Controlled Multi-Grain Dataflows for Foundation-Model Data Preparation
• Paper Explained: LimiX-2 — A Tabular Foundation Model That Learns the Whole Table, Not Just the Answer ColumnLimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence
• Paper Explained: DataFlex-RL — Did 11 RLVR Data Policies Beat Uniform Sampling? Re-measured Over 12 SeedsDataFlex-RL: An Evaluation Platform for RLVR Data Policies

In the textbook

• Moving Data Costs More Than Computing