mix

Definition

The recipe for a training corpus: how much code, math, web text, or each language goes into the blend. Papers report the mix because downstream ability shifts substantially with the proportions even at a fixed token budget, making it a first-class pre-training decision rather than a detail. Not the "mixture" of mixture-of-experts, which is about routing at inference time.

Explainers using this term

• Paper Walkthrough: From Production Traffic to Post-Training — Folding 200 Internal Apps Into One Self-Hosted LLMFrom Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix