#llm-as-judge

2 articles

01 ·Time Series·★ MEMBER·PAPER·13 min read Paper Review: MetroLLM-Bench — What Happens When an LLM Runs the Ticket Machine. Why a 2.6 GB 4B Model Tied GPT-5.6, and Why Fine-Tuning Hurt at 27B A 955-case benchmark that asks how well a language model can serve as the decision layer of a transit ticket kiosk. We read, from the paper alone, how a 4B QLoRA student beat GPT-5.6 on the deterministic tier and why PEFT gains shrink with model size until they turn negative at 27B. 02 ·Agents·FREE·PAPER·10 min read Paper Explained: OSReward — Can You Trust the AI That Grades AI? Remeasuring Rewards for Computer-Use Agents The VLM judges that decide whether an agent driving a PC or a phone succeeded turn out to share a leniency bias that reads failure as success — a paper that exposes it with 1,019 human-verified trajectories and closes the gap with OS-Shepherd, an open reward model 30–60× cheaper, explained from zero background.