#dpo

2 articles

01 ·Time Series·★ MEMBER·PAPER·7 min read Paper Explained: Video Generation Models: A Survey of Post-Training and Alignment — Four Toolboxes for Making Video Models Do What You Ask A from-scratch reading of the survey that organizes how pretrained video generators are steered toward instruction-following, temporal coherence, physical plausibility, and safety without retraining—via supervised fine-tuning, self-training and distillation, preference and reward optimization, and inference-time methods. 02 ·Training & Alignment·★ MEMBER·PAPER·13 min read DPO and What Came After — The Lineage That Simplified RLHF Derives DPO one line at a time, starting from the closed-form solution to KL-constrained reward maximization, to show why no separate reward model is needed. Then organizes IPO (which explains DPO's overfitting mathematically), KTO (which drops the pairing requirement), and GRPO (which drops the value model and goes back online) by what each one deleted — and gives a rule for choosing based on the shape of the data you actually have.