#terminal-bench

3 articles

01 ·Agents·★ MEMBER·PAPER·12 min read HarnessDev explained (arXiv:2609.01437) — agent harness benchmark Identical weights, different harness: GPT-5 solves 35.2% of Terminal-Bench 2.1 inside Terminus 2 and 49.6% inside Codex CLI. HarnessDev asks whether a model can build its own harness from a useless seed — and what survives held-out tasks. 02 ·Agents·★ MEMBER·PAPER·15 min read Paper Walkthrough: PILOT in the Loop — Fixing the Run While It Is Still Running Agent self-improvement that only starts after execution ends arrives too late. A ground-up walkthrough of PILOT, a supervisor–worker harness that redirects the live run while distilling skills from it, from intuition to equations to measured numbers and stated limits. 03 ·Agents·★ MEMBER·PAPER·11 min read Paper Walkthrough: AutoSaddler — Growing a Harness That Doesn't Break, from Agent Failure Logs A ground-up walkthrough of AutoSaddler, which automatically optimizes the harness around an LLM agent — prompts, tools, and middleware — by repeatedly diagnosing failure traces and generating structured patches. It beat the base harnesses on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 by 9.0, 9.6, and 10.0 points.