computer-use

Definition

An AI operating a computer the way a person does: reading a screenshot, deciding the next click or keystroke, driving existing applications without any purpose-built API. That generality is the appeal — it reaches software nobody will build an integration for. It is also what makes it hard: perceiving state from pixels, recovering from misclicks and holding a long plan together are all unsolved to varying degrees.

Explainers using this term

• Paper Explained: OSReward — Can You Trust the AI That Grades AI? Remeasuring Rewards for Computer-Use AgentsOSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models