#rlvr

6 articles

01 ·Large Language Models·★ MEMBER·PAPER·8 min read Paper Explained: Bellman Policy Optimization — Policy Mirror Descent for LLMs, Without a Value Model In LLM reinforcement learning where only the final answer is scored, BPO (arXiv:2609.15987) targets the same optimum as Policy Mirror Descent without estimating a single intermediate state value. We walk from the Bellman 'telescoping sum' to the practical loss, using only the paper itself as the source. 02 ·Training & Alignment·★ MEMBER·PAPER·14 min read Paper Explained: DataFlex-RL — Did 11 RLVR Data Policies Beat Uniform Sampling? Re-measured Over 12 Seeds "Train only on the good rollouts and the model gets stronger" is fast becoming RLVR folk wisdom. DataFlex-RL (arXiv:2609.06107) compares 13 selection, reweighting and mixture-adaptation configurations under one GRPO recipe and 12 seeds, and reports that every difference from uniform sampling has a 95% interval that crosses zero. We walk through the paper, including the finding that the choice of evaluation benchmarks reshuffles the winners. 03 ·Inference & Serving·★ MEMBER·PAPER·11 min read Paper Walkthrough: On-Policy Reverse Distillation — Borrowing a Weak Teacher's Direction, Not Its Answers OPRD takes the policy change a weak teacher acquired during post-training and uses it only as a direction to amplify the student's own policy gradient — never as an optimization target. A ground-up walkthrough. 04 ·Large Language Models·★ MEMBER·PAPER·13 min read Paper Explained: FlowBalance — Verifier-Grounded Self-Improvement from a Model's Own Reasoning Learning from your own answers breaks in two ways: the grading is too sparse, or you believe your own confidence. FlowBalance fixes both with one move — let the verifier decide the sign of the dense self-score — and learns a normalized distribution over whole responses. Explained from first principles. 05 ·FREE·PAPER·15 min read Paper Explained: VBVR-Pro — Making "Thinking in Pictures" Trainable, Scorable, and Comparable Native visual reasoning treats image and video generation as the medium of thought. VBVR-Pro supplies 300 procedurally generated tasks and deterministic scorers that replace the VLM judge, then pits image, video, and interleaved generation against each other on identical ground. Written for readers starting from zero. 06 ·FREE·PAPER·9 min read Paper explained: DEFT-RLVR — show a driving VLM the future trajectory too early and it fabricates its reasoning Let a teacher model peek at the ground-truth trajectory while writing chain-of-thought annotations for driving, and severe hallucinations double from 29% to 50% — the paper calls this trajectory anchoring bias. This article walks through AD-MCQ, which turns planning into a verifiable multiple-choice task, and DEFT-RLVR (arXiv:2608.01755), which reveals trajectories only after the model commits to a decision, using nothing but the paper itself as the source.