on-policy

Definition

A reinforcement learning regime where the data used for an update comes from the current policy itself. Because the training distribution matches the policy being improved, updates are better behaved, but every update invalidates the collected data, so sample efficiency is poor. Off-policy methods reuse old or externally generated data via replay buffers or importance weighting, trading stability for efficiency.

Explainers using this term

• Paper Explained: Beyond Teacher Assignment — Why Choosing the Right Teacher Isn't Enough When You Distill Many Specialists into One ModelBeyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
• Paper Explained: When EOS Tokens Disagree — Why On-Policy Distillation Students Forget How to StopWhen EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation
• Paper Walkthrough: On-Policy Reverse Distillation — Borrowing a Weak Teacher's Direction, Not Its AnswersEliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation