longhorizon-harness

Definition

A named evaluation runner for tasks that unfold over many steps. In this field a harness is the scaffolding around a benchmark, not the benchmark itself: it launches the environment, mediates tool calls, enforces step and time limits, and collects transcripts. The long-horizon qualifier signals the thing being measured is whether an agent holds its plan and state together across a long run, not whether it answers one question correctly.

Explainers using this term

• Paper Explained: LongHorizon-Harness — Long-Horizon Agent Tasks Are a State-Management Problem, Not an Execution ProblemLongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks