#dataset
4 articles
01
·★ MEMBER·PAPER·14 min read
SolarWM: Training on 5 Seconds, Walking for an Hour — and Opening the Whole Stack
A data engine that folds 10 datasets into 1.43M clips under one contract, plus a three-stage recipe that adapts four different video backbones without erasing them. The causal models roll out for minutes to hours after training on 5-second sequences alone.
02
·Inference & Serving·★ MEMBER·PAPER·11 min read
Paper walkthrough: CyberFactory — turning wild CVEs into runnable training problems
An open-source pipeline that converts real CVEs into executable, verifiable tasks and uses a reusable vulnerability-analysis skill to synthesize teacher trajectories the student then internalizes. 58.1% Pass@1 on CyberGym.
03
·Large Language Models·★ MEMBER·PAPER·10 min read
Building a Pretraining Corpus — From Web Sludge to Textbook Quality
Behind the single line "pretrained on a large corpus of web text" sit four stages: text extraction, quality filtering, deduplication, and mixture weights. This is a from-scratch walkthrough of how a gravel heap called Common Crawl gets sifted into textbook-quality prose — from the MinHash equation to the parameter names you actually touch.
04
·Training & Alignment·★ MEMBER·PAPER·11 min read
Building a Dataset in Practice — Collect, Clean, Blend
Most of the work in building a model is really the work of building its dataset. From designing the population you sample from, to calibrating filter thresholds, measuring leakage into your test set, converting mixture ratios into epochs, using synthetic data where it belongs, and writing annotation guidelines that actually hold — explained from zero, with the formulas and the procedures.