#data-curation
2 articles
01
·Distillation & Compression·★ MEMBER·PAPER·8 min read
Designing Distillation Data — Deciding What to Ask the Teacher
What decides a distilled student's quality is less how smart the teacher is than what you asked the teacher to answer. Synthetic data generation, coverage design, why you should skew toward hard problems, correctness filtering, and why DeepSeek-R1's distillation worked — from first principles.
02
·Training & Alignment·★ MEMBER·PAPER·11 min read
Building a Dataset in Practice — Collect, Clean, Blend
Most of the work in building a model is really the work of building its dataset. From designing the population you sample from, to calibrating filter thresholds, measuring leakage into your test set, converting mixture ratios into epochs, using synthetic data where it belongs, and writing annotation guidelines that actually hold — explained from zero, with the formulas and the procedures.