gaia

Definition

A benchmark for general AI assistants, built around questions that are straightforward for a person but hard for current systems. Answering them requires chaining several capabilities — browsing the web, reading attached files, handling images, executing multi-step plans — so it probes tool use and orchestration rather than recall. Answers are short and unambiguous, which keeps grading automatic and cheap, and it is a standard reference for agent evaluations.