What is a computer-use agent (CUA) in AI?
Also called computer use, CUA, browser agent, computer-using agent
Definition
A computer-use agent is an AI agent that operates a graphical interface, such as a desktop or web browser, by reading screenshots and issuing mouse and keyboard actions instead of calling APIs.
A computer-use agent is an AI agent that operates a graphical interface the way a person does: it looks at a screenshot, decides where to click or what to type, and checks the next screenshot to see what changed. It can work with applications that have no API, such as older desktop software or websites without integrations. Anthropic released computer use in public beta on October 22, 2024, describing a model that uses a computer by looking at a screen, moving a cursor, clicking buttons, and typing text. A browser agent is a computer-use agent limited to a web browser.
How a computer-use agent works#
A computer-use agent runs the standard agent loop with screenshots as observations and input events as actions. The model receives a screenshot and returns an action, such as a click at a coordinate or text to type. The application performs it in a virtual machine or browser and sends back a new screenshot, until the task is done.
The model never touches the machine directly. Anthropic's computer use tool documentation puts the application in the middle: it receives Claude's tool requests, translates them into actions such as screenshot, left_click, type, key, and scroll, and returns the results. OpenAI's computer tool in the Responses API works the same way: the model returns a computer_call with an ordered list of actions, and the application replies with a computer_call_output carrying the new screenshot.
Browser agents can be built from separate parts. Open source libraries such as Browser Use run the agent loop against a website, and hosted services such as Browserbase run the browsers in the cloud.
Computer-use agent risks and safeguards#
Computer-use agents read untrusted content on every screen, which exposes them to prompt injection. Anthropic's documentation warns that instructions on web pages or in images can override the user's instructions. Its recommended controls are a dedicated virtual machine or container with minimal privileges, no access to sensitive data, a domain allowlist, and human confirmation of consequential actions.
That confirmation step covers actions such as financial transactions, accepting cookies, and agreeing to terms of service. OpenAI's guide gives similar advice: use an isolated browser or VM, treat screen content as untrusted, and keep users in control of purchases. In practice that means an agent sandbox for the environment and a human-in-the-loop checkpoint for anything irreversible.
When an API or tool exists for a task, prefer it. A structured call completes in one step, while the same task through a graphical interface takes several actions, each with a screenshot for the model to process.
Computer-use benchmarks#
OSWorld (Xie et al., 2024) is a benchmark of 369 tasks on real Ubuntu, Windows, and macOS computers, covering desktop and web apps, file operations, and workflows that span several applications. Each task has a scripted initial state and an execution-based evaluation script. At publication, the authors reported that humans completed over 72% of tasks and the best model 12.24%.
The authors traced most model failures to GUI grounding, meaning matching an instruction to the right element on screen, and to missing operational knowledge of the applications. Scores on OSWorld and similar benchmarks describe the model together with the scaffold around it, so compare results only under the same settings.
What AI agents are places computer-use agents alongside the other common agent types, and the agent infrastructure directory lists cloud browser providers.
Related terms
Guides that cover computer-use agent
Cite this definition
The AI Engineering Editors. "What is a computer-use agent (CUA) in AI?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/computer-use-agent
- Title
- What is a computer-use agent (CUA) in AI?
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated