hacking

Definition

Achieving a high score while defeating the purpose the score was meant to track. Reward hacking is a policy exploiting a loophole in the reward function; benchmark hacking is a research culture optimizing a leaderboard rather than the underlying capability. Both are instances of Goodhart's law — a measure ceases to be a good measure once it becomes a target — and neither implies the ordinary security sense of breaking into a system.