#reward-model

6 articles

01 ·Inference & Serving·★ MEMBER·PAPER·8 min read Paper Explained: RewardVerse — A Video Reward Model That Writes the Rubric Before It Scores (RGPO) Video reward models tend to squeeze their scores into a narrow band, and those scores move when the wording changes. The paper calls this scalar drift. RewardVerse first generates an evaluation rubric, then reads a continuous score for each item from token probabilities. Its two-stage RGPO training uses only 30 preference pairs per dimension. 02 ·Large Language Models·★ MEMBER·PAPER·9 min read Paper Explained: Beyond Solver Verdicts — When the Solver Says Yes but the Translation Is Wrong An LLM translates a problem into logic and Z3 solves it. This paper names the failure where the solver's verdict is right but the translation is wrong (VPU), and distills Z3's equivalence check into a Yes/No next-token verifier called GenV. Explained from zero. 03 ·Training & Alignment·★ MEMBER·PAPER·12 min read Paper Walkthrough: StudentSim — Training a Simulator That Is Actually *That* Student How to build the practice students an AI tutor needs, by training on real learner records. From the two-axis definition of behavioral fidelity and guidance responsiveness, through the pooled-then-specialized pipeline, to using the simulator as a reward model. 04 ·Paper Deep-Dives·★ MEMBER·PAPER·12 min read Paper Walkthrough — J-Zero: Growing the Challenger, the Solver, and the Judge Together from Zero Data A model that writes its own problems, solves them, and grades them — with no external data and no human labels. J-Zero's twist is that the grader learns too, which is what lets it keep improving past the two-iteration wall where prior methods stall. 05 ·Inference & Serving·★ MEMBER·PAPER·13 min read Paper explained: Self-OPD — an image generator that distills itself, with no teacher No teacher model at all: at every denoising step the student spawns K copies of itself, scores them, and pulls toward the good branches while pushing away from the bad ones. A ground-up walkthrough of Self-OPD, an alignment method for flow matching image generators. 06 ·Agents·FREE·PAPER·10 min read Paper Explained: OSReward — Can You Trust the AI That Grades AI? Remeasuring Rewards for Computer-Use Agents The VLM judges that decide whether an agent driving a PC or a phone succeeded turn out to share a leniency bias that reads failure as success — a paper that exposes it with 1,019 human-verified trajectories and closes the gap with OS-Shepherd, an open reward model 30–60× cheaper, explained from zero background.