#synthetic-data
5 articles
01
·Inference & Serving·★ MEMBER·PAPER·10 min read
Paper Explained: Compile by Training — Turning a Natural-Language Spec into a Function That Runs Locally
"Sort my email into urgent and later" — a spec that vague, turned into a small function you can run locally after about a minute of training. A ground-up walkthrough of Compile by Training: analogy, mechanism, equations, measured numbers.
02
·Distillation & Compression·★ MEMBER·PAPER·8 min read
Designing Distillation Data — Deciding What to Ask the Teacher
What decides a distilled student's quality is less how smart the teacher is than what you asked the teacher to answer. Synthetic data generation, coverage design, why you should skew toward hard problems, correctness filtering, and why DeepSeek-R1's distillation worked — from first principles.
03
·Training & Alignment·★ MEMBER·PAPER·11 min read
Building a Dataset in Practice — Collect, Clean, Blend
Most of the work in building a model is really the work of building its dataset. From designing the population you sample from, to calibrating filter thresholds, measuring leakage into your test set, converting mixture ratios into epochs, using synthetic data where it belongs, and writing annotation guidelines that actually hold — explained from zero, with the formulas and the procedures.
04
·Agents·★ MEMBER·PAPER·12 min read
Paper Walkthrough — FACET: Grounding Instruction, Environment, Solution and Verifier in One Executable State
A terminal task is a four-part bundle: instruction, environment, reference solution, verifier. FACET builds and runs the environment first, then uses that realized state as shared ground for every other artifact. From 71K agent skills it produces 6,078 validated tasks, and 1.2K successful trajectories lift Qwen3.5 at 4B, 9B and 27B alike.
05
·Agents·★ MEMBER·PAPER·13 min read
Paper Deep-Dive: Recursive Synthesis — Extending Verified Tasks Into 40,000 Long-Horizon Terminal Problems
Long-horizon terminal tasks cost hundreds to thousands of dollars each to author. RST mass-produces them for about $0.05 by recursively extending already-verified seeds. Fifteen rounds yielded 37,484 tasks, with oracle solutions growing from a median of 67 lines to 374, and the resulting trajectories genuinely improved Qwen3.5 under SFT and PPO — a walkthrough of the data-synthesis paper from Tencent and collaborators.