#harness
9 articles
01
·Agents·★ MEMBER·PAPER·12 min read
Paper Walkthrough: HarnessDev — Can an LLM Build and Maintain the System It Runs Inside?
A ground-up walkthrough of HarnessDev, a benchmark that asks whether an LLM can build an agent harness from a deliberately useless seed and then improve it from execution feedback. Models can build one; the gains rarely survive contact with held-out tasks or a different runtime model.
02
·Agents·★ MEMBER·PAPER·15 min read
Paper Walkthrough: Training Agents to Evolve with Their Harness
Why compact models break when you rewrite prompts and tool definitions every week, and what the team behind Taobao Live's AI avatar streamers does about it. A ground-up walkthrough of Harness-Aware Training, from intuition to equations to measured numbers and stated limits.
03
·Agents·★ MEMBER·PAPER·13 min read
Paper Explained: StarHarness — Evolving the Scaffold Instead of the Weights
Freeze the model weights and search over the harness instead — prompts, tool schemas, skills, MCP providers, subagents, agent loop. Across three enterprise benchmarks this bought 20–35 points, held on tasks never used for evolution, and transferred to other models unchanged.
04
·Agents·★ MEMBER·PAPER·15 min read
Paper Walkthrough: PILOT in the Loop — Fixing the Run While It Is Still Running
Agent self-improvement that only starts after execution ends arrives too late. A ground-up walkthrough of PILOT, a supervisor–worker harness that redirects the live run while distilling skills from it, from intuition to equations to measured numbers and stated limits.
05
·Agents·★ MEMBER·PAPER·11 min read
Paper Walkthrough: AutoSaddler — Growing a Harness That Doesn't Break, from Agent Failure Logs
A ground-up walkthrough of AutoSaddler, which automatically optimizes the harness around an LLM agent — prompts, tools, and middleware — by repeatedly diagnosing failure traces and generating structured patches. It beat the base harnesses on GAIA2, SWE-Bench Pro, and Terminal-Bench 2.0 by 9.0, 9.6, and 10.0 points.
06
·Agents·★ MEMBER·PAPER·13 min read
Paper Explained: JIT-Agent — A Model That Writes the Agent Harness On Demand
An agent's capability is not the model's alone. This is a ground-up walkthrough of JIT-Agent, a model trained to synthesize the four-module harness — memory, planning, action, tools — freshly for each task.
07
·Agents·★ MEMBER·PAPER·11 min read
Evaluating Agents — How Benchmarks and Harnesses Are Built
An agent score is never a property of the model alone — it is a property of model plus harness plus environment plus grading rule. This piece opens up a single SWE-bench instance, shows why a tiny per-step gap becomes an order-of-magnitude gap over a long horizon, walks the four routes by which the answer leaks into the working environment, and sets out the conditions under which partial credit is safe.
08
·Agents·★ MEMBER·PAPER·11 min read
Paper walkthrough: StateM — 95.3% on Terminal-Bench 2.1 and a USD 15 run, without touching a single weight
Long-horizon agents fail even when the model underneath can solve every individual step. StateM leaves the weights alone and hardens only the execution system around the agent, reporting 95.3% on Terminal-Bench 2.1 and a final scoring run billed at roughly USD 15 instead of USD 574.68. A ground-up walkthrough of the harness-scaling bet.
09
·Agents·★ MEMBER·PAPER·8 min read
Paper Explained: LongHorizon-Harness — Long-Horizon Agent Tasks Are a State-Management Problem, Not an Execution Problem
Without touching the model at all, keeping task state outside execution and running a Manage-Execute-Audit loop dramatically improves long-horizon task performance — a walkthrough of Alibaba DreamX team's harness-design paper, from analogy to pseudocode.