AI coding agents: how they work and how the major tools compare
AI coding agents are developer tools that take a task written in natural language and carry it through by reading a repository, editing files, running commands, and checking the results in a loop. AI coding agents work at the level of a task or a pull request, while earlier autocomplete assistants worked at the level of the next line of code.
This comparison explains what changed between assistants and agents, the four categories that matter when choosing a tool, how the major products compare as of November 2026, why public benchmark scores need care, and the workflows and security controls teams use when agents write production code. Pricing and plan limits are left out because they change too often to stay accurate.
What are AI coding agents?#
An AI coding agent is software that connects a language model to tools for reading code, editing files, and running commands, and repeats that loop until a task is done. The agent gathers its own context from the repository instead of relying on pasted snippets, and it checks its work against compilers, tests, and linters before handing it back.
A coding agent is a specialized AI agent. The model behind it is a large language model that emits structured tool calls, such as "read this file," "search for this symbol," or "run this test command." The agent's runtime executes each call, returns the output to the model, and continues until the model produces a final answer or a stop condition is reached.
The quality of that loop depends on two things besides the model. The first is how the agent fills its context window with the right files, instructions, and tool output, which is the subject of context engineering. The second is the environment the agent runs in: an agent that cannot install dependencies or run the test suite cannot verify its own changes.
AI coding assistant vs agent: what changed#
An AI coding assistant suggests code for a developer to accept, while an AI coding agent executes a multi-step task and returns a result for review. The practical difference is who runs the loop. With an assistant, the developer edits, runs, and verifies each change. With an agent, the software performs those steps and the developer reviews the outcome.
| Dimension | Inline completion | Chat assistant | Coding agent |
|---|---|---|---|
| Unit of work | Next line or block | One answer or snippet | Task, branch, or pull request |
| Who runs the code | Developer | Developer | Agent, in a sandbox or VM |
| Context source | Open file and nearby files | Pasted or selected code | Repository search through tools |
| Feedback loop | None | Developer reports errors back | Agent reads compiler and test output |
| Review point | Each suggestion | Each answer | Final diff or pull request |
| Typical duration | Milliseconds | Seconds | Minutes or longer |
Most products now cover more than one column. Visual Studio Code's Copilot documentation, for example, separates inline suggestions, which predict your next edit, from agents, which find relevant code, make changes, and run checks toward a goal you set. Choosing a tool therefore means choosing how much of the loop you want to delegate and where that loop should run.
Types of AI coding agents#
AI coding agents fall into four overlapping categories: terminal agents that run in a shell, IDE agents built into an editor, cloud or background agents that work in remote sandboxes and open pull requests, and open source agents that can be self-hosted and pointed at any model. Many products now ship in two or three of these forms at once.
Terminal coding agents#
Terminal coding agents run as command-line programs inside your project directory. Examples include Claude Code, OpenAI's Codex CLI, Cursor CLI, GitHub Copilot CLI, Aider, OpenCode, and Google's Gemini CLI and Antigravity CLI. A terminal agent can be scripted: Claude Code accepts a prompt with the -p flag for non-interactive runs, and Codex CLI provides codex exec for pipelines and CI jobs. Terminal agents suit developers who already work in a shell and teams that want to automate agent runs.
IDE agents#
IDE agents run inside the editor and show proposed changes as inline diffs. GitHub Copilot's agent mode in VS Code, Cursor's Agent, the Cline extension, and the Claude Code and Codex extensions for VS Code fall here. Devin Desktop, the editor formerly called Windsurf, now opens to an agent command center for managing several local and cloud agents. IDE agents make review easier for changes a developer wants to watch closely.
Cloud and background coding agents#
Cloud coding agents, also called background coding agents, run in a remote environment, so the work continues when your laptop is closed. GitHub Copilot cloud agent runs in a cloud sandbox, a GitHub Actions-based environment, or a remote custom environment. Codex Cloud gives each task its own workspace. Cursor's cloud agents run in isolated virtual machines and can be started from Slack, GitHub, Linear, or an API. Claude Code on the web and Cognition's Devin work the same way. The trade-off is setup: the remote environment needs your dependencies, secrets, and network access configured before the agent can test anything.
Open source AI coding agents#
Open source coding agents publish their source code and let you choose the model provider. Codex CLI, Gemini CLI, Cline, and Aider use the Apache 2.0 license; OpenHands and OpenCode use MIT. An open source agent still needs a model: Cline, Aider, and OpenCode can connect to hosted model providers or to local models, which is relevant if code must not leave your network (see vLLM vs Ollama for local serving). A related trend is the Agent Client Protocol (ACP): Devin Desktop and OpenHands Agent Canvas can both host third-party ACP-compatible agents, so the editor and the agent no longer have to come from the same vendor.
AI coding agents comparison#
The table below compares major AI coding agents by maintainer, the surfaces each one runs on, where the agent executes code, and source availability. Every entry reflects the vendor's or project's official documentation as of November 2026. Pricing is excluded because plans change frequently.
| Tool | Maintainer | Surfaces | Where the agent runs | Source |
|---|---|---|---|---|
| Claude Code | Anthropic | Terminal, VS Code and JetBrains extensions, desktop app, web, Slack | Local machine; cloud sessions from web and mobile | Proprietary |
| OpenAI Codex | OpenAI | Codex CLI, IDE extension, ChatGPT desktop, web, and mobile apps | CLI and IDE: local, OS-enforced sandbox; Codex Cloud: per-task cloud workspace | CLI is Apache 2.0 |
| GitHub Copilot | GitHub | Agent mode in IDEs including VS Code, Copilot CLI, Copilot cloud agent on GitHub.com | IDE and CLI: local; cloud agent: cloud sandbox, Actions-based, or remote environment | Proprietary |
| Cursor | Anysphere | Cursor editor, Cursor CLI, web, iOS, Slack, GitHub, Linear | Editor and CLI: local; cloud agents: isolated VMs | Proprietary |
| Devin and Devin Desktop | Cognition | Devin web app, Devin CLI, Devin Desktop (formerly Windsurf), Slack, Teams, API | Devin: cloud workspace with shell, editor, and browser; Desktop: local agent plus cloud handoff | Proprietary |
| Gemini CLI | Terminal | Local | Apache 2.0 | |
| Cline | Open source project | VS Code and JetBrains extensions, CLI, desktop app | Local, with any supported model provider | Apache 2.0 |
| Aider | Open source project | Terminal | Local; commits each change to git | Apache 2.0 |
| OpenHands | Open source project | Agent Canvas, a self-hosted control center | Local, Docker, VMs, or OpenHands Cloud | MIT |
| OpenCode | Open source project | Terminal UI, desktop app, IDE extension | Local, with many model providers | MIT |
Three changes in 2026 make older comparisons out of date:
- Windsurf is now Devin Desktop. Cognition, which announced its acquisition of Windsurf in July 2025, launched Devin Desktop as "the next generation of Windsurf" on June 2, 2026.
- Copilot coding agent is now Copilot cloud agent. GitHub's April 2026 changelog uses the new name and notes the agent is no longer limited to pull-request workflows; it can also research a codebase or produce a plan before changing code.
- Gemini CLI now targets enterprise users. Since June 18, 2026, Google serves free individual users and Google AI Pro and Ultra subscribers through Antigravity CLI. Gemini CLI remains for Gemini Code Assist Standard and Enterprise customers.
For a broader list that includes code review bots and app builders, see the AI coding tools directory.
Which AI coding agent is best? How to evaluate them#
No single AI coding agent is best for every team, because results depend on the model, the agent's tooling, the codebase, and the mix of tasks. Public benchmarks such as SWE-bench Verified give a rough signal at best. The reliable way to choose is a short trial on your own repositories, using tasks you already know how to grade.
What SWE-bench Verified measures#
The original SWE-bench paper describes 2,294 problems drawn from real GitHub issues across 12 popular Python repositories. Each task gives the agent a codebase and an issue, and the agent passes if its patch makes the project's tests pass. SWE-bench Verified, released by OpenAI in August 2024, is a 500-task subset that human annotators screened for solvable, well-specified problems.
Why SWE-bench Verified scores mislead#
On February 23, 2026, OpenAI published why it no longer evaluates on SWE-bench Verified. The post gives two reasons. First, contamination: the tasks come from public repositories that model developers train on, and OpenAI found frontier models could reproduce original fixes or problem wording for some tasks. Second, flawed tests: in an audit of 138 problems that its o3 model did not reliably solve, OpenAI found that at least 59.4% had material issues in test design or problem description. OpenAI recommended SWE-Bench Pro, a benchmark from Scale AI with public, held-out, and commercial task sets, as an interim alternative.
Two further caveats apply to any coding benchmark. A score describes a model and an agent configuration together, so numbers produced with different setups are not directly comparable. A benchmark built from Python libraries also says little about a TypeScript monorepo or a legacy Java service. The LLM benchmarks guide covers these limits in more depth.
Run your own evaluation#
A useful internal evaluation takes a day or two. Pick 10 to 30 recently closed issues with known fixes and tests, strip the fixes, and give each agent the same instructions file and task text. Then score the results with criteria that reflect how your team merges code:
| Criterion | What to measure | Why it matters |
|---|---|---|
| Task success | Tests pass and the fix matches intent | Passing tests alone can hide hard-coded or deleted tests |
| Merge readiness | Would a reviewer approve with minor edits | Captures style, scope, and correctness together |
| Human interventions | Prompts, corrections, or approvals per task | Measures how much attention the agent consumes |
| Diff size | Lines changed relative to the human fix | Large diffs raise review cost and risk |
| Token usage and time | Tokens and wall-clock time per task | Predicts cost and throughput at team scale |
| Failure modes | Loops, wrong files, unsafe commands | Shows which guardrails you need |
Treat this as a small LLM evaluation project. If you grade with a model instead of reviewers, calibrate the LLM-as-a-judge against human ratings on a sample first.
How teams use AI coding agents#
Teams that use AI coding agents at scale treat them like new contributors. They document project conventions in an instruction file, scope tasks into reviewable units, require the same CI checks and human review as any other pull request, and connect agents to issue trackers and internal systems through the Model Context Protocol.
Instruction files#
Most agents read a project instruction file at the start of each session. AGENTS.md is an open format for this purpose, stewarded by the Agentic AI Foundation under the Linux Foundation, and supported by tools including Codex, Cursor, Aider, and Gemini CLI. Claude Code reads CLAUDE.md and can read an existing AGENTS.md; Cline reads .clinerules. Keep these files short and specific: build and test commands, directory layout, and conventions the agent cannot infer from the code.
Task sizing and parallel work#
Give agents tasks with a clear definition of done, such as a failing test, a lint migration, a dependency upgrade, or a documented bug. Cloud agents make it easy to run many tasks in parallel, which moves the bottleneck to code review. A practical control is to cap the number of open agent pull requests per reviewer rather than the number of agent runs.
Review and CI gates#
Agent output should pass the same branch protection, CI, and review rules as human code. GitHub builds this into Copilot cloud agent: according to GitHub's risks and mitigations documentation, the agent cannot approve or merge its own pull requests, the person who requested the pull request cannot approve it, and by default GitHub Actions workflows do not run on the agent's changes until a user with write access approves them.
Connecting tools with MCP#
Claude Code, Cursor, Gemini CLI, and Cline can connect to MCP servers, which expose issue trackers, documentation, databases, and internal APIs as tools the agent can call. The Model Context Protocol explainer covers how those servers work. Each server added also adds tool definitions to the agent's context and new paths for untrusted data, so connect only the servers a workflow needs. An MCP gateway can enforce that centrally: Bifrost, for example, serves configured MCP tools to Claude Code and Cursor at one /mcp endpoint and limits each virtual key to the tools it allows.
Security risks of AI coding agents#
The main security risks of AI coding agents are prompt injection from untrusted content such as issues, dependencies, and web pages; leaked secrets; destructive commands; and unreviewed code reaching production. The standard controls are sandboxed execution, network allowlists, least-privilege credentials, approval policies for risky actions, and mandatory human review before merge.
Simon Willison's "lethal trifecta" is a useful test: an agent that has access to private data, exposure to untrusted content, and a way to communicate externally can be manipulated into exfiltrating that data. A coding agent that reads a malicious issue, can see your environment variables, and can run curl meets all three conditions. Prompt injection cannot be fully prevented by instructions to the model, so the controls have to sit in the execution environment.
Default protections differ by product and deserve a check before rollout. Codex CLI and the Codex IDE extension default to no network access and to write permissions limited to the active workspace. Claude Code's sandbox for shell commands is off by default; when enabled, the sandbox limits writes to the working directory and paths you add, and routes network traffic through a proxy that allows only listed domains. Reads from sandboxed commands still reach most of the machine unless you add deny rules for paths such as ~/.ssh.
| Risk | Example | Controls |
|---|---|---|
| Prompt injection | Hidden instructions in an issue, README, or fetched page | Treat repository and web content as untrusted; restrict network egress |
| Secret exposure | Agent reads .env or cloud credentials and echoes them into logs |
Deny-read rules, short-lived scoped tokens, secret scanning |
| Destructive commands | Deleting files, force-pushing, dropping a development database | Workspace-only writes, approval policies, protected branches |
| Dependency risk | Agent adds a misspelled or nonexistent package | Package allowlists, lockfile review in CI |
| Unreviewed changes | Agent code merged without a second reader | Required reviewers, CI checks, no self-approval |
Organizations that route agent model traffic through an AI gateway can add logging and guardrails on top of these controls. Bifrost documents setups for Claude Code, Codex CLI, Cursor, Gemini CLI, and other agents. Claude Code, for instance, points ANTHROPIC_BASE_URL at the gateway and sends a virtual key, so each request is logged against the developer or team that made it. The sandbox and review gates remain the primary defense.
Where to go next#
AI coding agents combine three skills covered elsewhere on this site: the tool calling mechanism that lets a model run commands, the context management that keeps long sessions accurate, and the agent design patterns described in what are AI agents. For news, comparisons, and updates on coding tools as they change, follow the AI coding topic hub.
Frequently asked questions
What is the difference between an AI coding assistant and an AI coding agent?
- An AI coding assistant suggests code that a developer accepts, edits, and runs. An AI coding agent receives a task, searches the repository, edits files, runs tests and other commands, reads the output, and repeats until the task passes or needs input. The developer's role shifts from writing each change to reviewing the agent's diff or pull request.
Is Claude Code open source?
- No. Anthropic hosts a public GitHub repository for Claude Code, but the license file in that repository states that all rights are reserved and use is subject to Anthropic's commercial terms. OpenAI's Codex CLI, Google's Gemini CLI, Cline, and Aider are released under the Apache 2.0 license, and OpenHands and OpenCode use the MIT license.
Which AI coding agent is best for VS Code?
- Several agents run inside VS Code: GitHub Copilot's agent mode, Cline, the Claude Code extension, and the OpenAI Codex IDE extension. Cursor and Devin Desktop are separate editors with built-in agents. The right choice depends on which models your team can use, whether you need cloud execution, and your security requirements, so a short trial on your own code is the most reliable test.
Are open source AI coding agents as capable as commercial ones?
- Capability depends mostly on the model behind the agent and on how well the agent manages tools and context. Open source agents such as Cline, OpenCode, Aider, and OpenHands can connect to the same frontier models that commercial tools use, and can also run local models. Commercial tools often add managed cloud execution, team administration, and tighter integrations.
How good are AI coding agents at real software engineering tasks?
- AI coding agents do best on well-specified tasks with clear tests, such as bug fixes, dependency upgrades, and added test coverage. Results drop on long, ambiguous, or cross-repository work. Scale AI's SWE-Bench Pro benchmark was built to measure that harder class of long-horizon tasks, and its paper reports much lower solve rates there than on SWE-bench Verified.
Terms in this guide
Cite this guide
The AI Engineering Editors. "AI coding agents: how they work and how the major tools compare." The AI Engineering, updated Nov 11, 2026. https://theaiengineering.com/articles/ai-coding-agents
- Title
- AI coding agents: how they work and how the major tools compare
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Published
- Updated