#policy-optimization

1 articles

01 ·Large Language Models·★ MEMBER·PAPER·8 min read Paper Explained: Bellman Policy Optimization — Policy Mirror Descent for LLMs, Without a Value Model In LLM reinforcement learning where only the final answer is scored, BPO (arXiv:2609.15987) targets the same optimum as Policy Mirror Descent without estimating a single intermediate state value. We walk from the Bellman 'telescoping sum' to the practical loss, using only the paper itself as the source.