#reproducibility

3 articles

01 ·Training & Alignment·★ MEMBER·PAPER·14 min read Paper Explained: DataFlex-RL — Did 11 RLVR Data Policies Beat Uniform Sampling? Re-measured Over 12 Seeds "Train only on the good rollouts and the model gets stronger" is fast becoming RLVR folk wisdom. DataFlex-RL (arXiv:2609.06107) compares 13 selection, reweighting and mixture-adaptation configurations under one GRPO recipe and 12 seeds, and reports that every difference from uniform sampling has a 95% interval that crosses zero. We walk through the paper, including the finding that the choice of evaluation benchmarks reshuffles the winners. 02 ·Training & Alignment·★ MEMBER·11 min read Versioning Data and Models — An Experiment You Cannot Reproduce Never Happened If you cannot get "92% accuracy" back six months later, it was an anecdote, not an experiment. We build up content-addressing (naming things by their contents), lineage (the graph from inputs to outputs), and the three levels of "how much is worth pinning down" — from the hash math to a manifest you can write today to the mistakes that quietly ruin a year of runs. 03 ·Machine Learning Basics·★ MEMBER·10 min read Data Leakage and Experiment Hygiene — When the Score Is Too Good, Suspect It When a model scores better than you expected, that is not good news yet. This article splits data leakage into five concrete shapes — outcome-derived columns, preprocessing before the split, time running backwards, duplicates and groups, and a worn-out test set — then covers the detection moves that actually find them and the experiment records that let you dissect the day the number jumped.