starharness

Definition

A coined name for an execution or evaluation harness — the software that loads tasks, calls the model, sets up the environment, collects outputs, and scores them. Harnesses get named and published for reproducibility: the same benchmark run under two different harnesses can yield materially different numbers, because prompt formatting, retry policy, and answer extraction all live there. The name refers to infrastructure, not to a model or a dataset.

Explainers using this term

• Paper Explained: StarHarness — Evolving the Scaffold Instead of the WeightsStarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments