---
title: "Best LLM for coding: models compared on agentic coding benchmarks"
description: "The best LLM for coding depends on the job: 11 models compared on Terminal-Bench 4.0, DeepSWE, and SWE-bench Pro, plus context, open weights, and cost."
summary: "No single model leads every agentic coding benchmark in November 2026, and labs now report different tests run with different agent setups. This comparison groups 11 models by use, from frontier APIs to open-weights and local models, with each score's source, date, and setting."
type: "comparison"
canonical_url: "https://theaiengineering.com/articles/best-llm-for-coding"
markdown_url: "https://theaiengineering.com/articles/best-llm-for-coding.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-11-17"
updated: "2026-11-17"
keywords: ["best llm for coding","best ai model for coding","best llm for coding locally","best open source llm for coding","coding llm comparison"]
citation: "The AI Engineering Editors. \"Best LLM for coding: models compared on agentic coding benchmarks.\" The AI Engineering, updated Nov 17, 2026. https://theaiengineering.com/articles/best-llm-for-coding"
---

# Best LLM for coding: models compared on agentic coding benchmarks

## TL;DR

- No model leads every agentic coding benchmark as of November 2026: Claude Fable 5.2 tops the FrontierSWE v2 leaderboard, Gemini 4 Argon reports the highest DeepSWE v1.1 score here, and Claude Sonnet 5.5 reports the highest Terminal-Bench 4.0 score.
- Frontier labs no longer headline SWE-bench Verified; their 2026 announcements report Terminal-Bench 4.0, DeepSWE v1.1, FrontierSWE v2, and FrontierCode instead.
- GPT-6.1 Sol and Claude Sonnet 5.5 both list at $2 input and $10 output per million tokens, and each matches or beats a pricier sibling from its lab on an agentic coding benchmark its lab reports.
- Open-weights models Kimi K3.1 Pro and DeepSeek-V4.1-Flash-1017 report agentic coding scores within about five points of the best proprietary results here.
- Scores depend on the agent scaffold: Qwen reports a 10-point spread for Qwen3-Coder-Next on Terminal-Bench 2.0 across four scaffolds, so test candidates on your own repository with one fixed setup.

No single model leads every agentic coding benchmark as of November 2026, so the best LLM for coding depends on the work: multi-hour autonomous tasks, high-volume edits in an agent loop, self-hosted inference, or a model that runs on one workstation. Claude Fable 5.2 tops the FrontierSWE v2 leaderboard, Gemini 4 Argon reports the highest DeepSWE v1.1 score among the models below, and Claude Sonnet 5.5 reports the highest lab-reported Terminal-Bench 4.0 score.

This comparison groups 11 models by use. Each score names its source, date, and disclosed setting, and where a lab publishes no score, the entry says so. For how coding benchmarks work, start with [LLM benchmarks explained](https://theaiengineering.com/articles/llm-benchmarks), part of the [models topic hub](https://theaiengineering.com/topics/models).

## How the models were selected

Each model was judged on six criteria: published agentic coding scores, context window, tool-use behavior, availability in APIs and coding agents, whether weights are open, and cost structure. Every entry had to be generally available on November 16, 2026, so limited-access models such as GPT-6 Cyber and Gemini 4 Carbon appear only in the closing list.

- **Agentic coding benchmarks.** Scores from the lab's announcement or model card, or the benchmark's official leaderboard, on tests that run many turns of file edits and commands.
- **Context window.** A longer window holds more of a repository, its logs, and the agent's history.
- **Tool-use behavior.** Agentic benchmarks exercise hundreds of tool calls per task, and API limits matter: Claude Fable 5.2 rejects requests that force a specific tool.
- **Availability.** APIs, cloud platforms, and coding agents such as Claude Code, Codex, Google Antigravity, and Kimi Code.
- **Open weights.** An [open-weights model](https://theaiengineering.com/glossary/open-weights-model) publishes its parameters for download, often without training data, under a license that may add conditions.
- **Cost structure.** Dated list prices from each pricing page and, for open weights, the memory the full model needs.

### Which coding benchmarks labs report in 2026

In February 2026, OpenAI stopped reporting [SWE-bench](https://theaiengineering.com/glossary/swe-bench) Verified, citing flawed tests and evidence that models had seen the solutions, and recommended SWE-bench Pro. By November, the launch materials for Claude Fable 5.2, GPT-6 Astra, and Gemini 4 Argon reported no SWE-bench variant, and the top ten entries on Scale AI's [public SWE-Bench Pro leaderboard](https://labs.scale.com/leaderboard/swe_bench_pro_public), checked November 16, were all models released by July 2026.

| Benchmark | Maintainer | What it tests | How it is scored |
|---|---|---|---|
| SWE-bench Pro | Scale AI | 1,865 tasks from 41 repositories; 731 public | Percent resolved; 250-turn limit on the public leaderboard |
| Terminal-Bench 4.0 | Terminal-Bench project, on the Harbor framework | Multi-step terminal tasks; 4.0 removed 8 tasks and fixed 19 | Resolution rate, 8-hour agent timeout per task |
| DeepSWE v1.1 | Datacurve | 113 long-horizon tasks written from scratch in 91 repositories | Hand-written verifiers; leaderboard uses mini-swe-agent for all models |
| FrontierSWE v2 | Proximal | 34 open-ended tasks, such as a Lean 4 type checker in Pascal | Mean of 5 trials, 20-hour budget each |
| FrontierCode | Cognition | 150 tasks built with open-source maintainers; Main is the 100 hardest | Fails on any blocker criterion, else a weighted rubric score |

Sources: [Terminal-Bench 4.0](https://www.tbench.ai/news/terminal-bench-4-0), [DeepSWE](https://deepswe.datacurve.ai), [FrontierSWE](https://frontierswe.com), and [FrontierCode](https://cognition.com/blog/frontier-code). Scores on different benchmarks never compare directly, because each measures different tasks under different rules.

## Best LLMs for coding at a glance

The table summarizes all 11 entries: lab, license, context window, the most prominent coding score each lab or leaderboard reports, and the job each model fits. Read each headline score as evidence for that benchmark only, because each row uses whichever benchmark its lab chose to publish. Dates, prices, and settings are in the entries.

| Model | Lab | Open weights | Context | Headline coding score (source, setting) | Best for |
|---|---|---|---|---|---|
| Claude Fable 5.2 | Anthropic | No | 1M | FrontierSWE v2: 69.9%, mean of 5 trials (Proximal leaderboard) | Multi-hour autonomous engineering |
| GPT-6 Astra | OpenAI | No | 1.05M | Terminal-Bench 4.0: 57.9% (OpenAI) | Hard tasks in Codex and the OpenAI API |
| Gemini 4 Argon | Google | No | 1,048,576, input and output | DeepSWE v1.1: 77.9% (Google model card) | Long-horizon work on Google's stack |
| Claude Sonnet 5.5 | Anthropic | No | 1M | Terminal-Bench 4.0: 70.6%, effort not stated (Anthropic) | Daily agentic coding at a mid-tier price |
| GPT-6.1 Sol | OpenAI | No | 1.05M | DeepSWE v1.1: matches GPT-6 Astra (OpenAI) | Near-flagship results at a fifth of Astra's price |
| Gemini 3.8 Flash | Google | No | 1,048,576 | DeepSWE v1.1: 74%, high effort, mini-swe-agent (DeepSWE leaderboard) | High-volume agent loops |
| Kimi K3.1 Pro | Moonshot AI | Yes, custom license | 1,048,576 | Terminal-Bench 4.0: 65.7%, Claude Code scaffold, max effort (Moonshot AI) | Frontier-class open weights |
| DeepSeek-V4.1-Flash-1017 | DeepSeek | Yes, MIT | 1M | DeepSWE v1.1: 77.0%, Mini-SWE-V2 scaffold (DeepSeek) | Low-cost API or MIT-licensed hosting |
| GLM-5.3 | Z.ai | Yes, custom license | 1M | DeepSWE v1.1: 66.9%, mini-swe-agent (Z.ai) | Self-hosted coding agents |
| Qwen3.8-27B | Alibaba Qwen | Yes, Apache 2.0 | 262,144 | SWE-bench Pro: 61.7%, Qwen's corrected tasks, Claude Code scaffold (Qwen) | One-workstation local use |
| Qwen3-Coder-Next | Alibaba Qwen | Yes, Apache 2.0 | 262,144 | SWE-bench Verified: 70.6%, SWE-Agent (Qwen) | Fast local agent loops |

## Best overall for agentic coding

The three strongest generally available models for long, autonomous coding tasks are Claude Fable 5.2, GPT-6 Astra, and Gemini 4 Argon. Each is first on at least one current agentic coding benchmark, by leaderboard position or lab-reported score, and each sits at or near the top of its lab's price list.

### Claude Fable 5.2

**What it is.** [Claude Fable 5.2](https://theaiengineering.com/releases/2026/11#claude-fable-5-2) is Anthropic's most capable generally available model, released November 9, 2026, for long-horizon agentic work, per its [model documentation](https://platform.claude.com/docs/en/models/fable-5-2/overview). Claude Mythos 5.2, the same model with fewer safeguards, is limited to verified organizations.

**Deployment and license.** Proprietary: the Claude API (`claude-fable-5-2`), Amazon Bedrock, Google Cloud, Microsoft Foundry, and Claude Code, with a 1M-token context window. Anthropic listed it at $8 input and $40 output per million tokens on November 16, 2026.

**Strengths.** It ranks first on the [FrontierSWE v2 leaderboard](https://frontierswe.com) at 69.9%, ahead of GPT-6 Astra's 65.5%, at an average of $168.78 per trial against Astra's $1,029.65. Anthropic's [announcement](https://www.anthropic.com/claude-fable-and-mythos-5-2) also reports 49.1% on FrontierCode 2.0 at xhigh effort.

**Trade-offs.** Anthropic published no Terminal-Bench 4.0 or SWE-bench score for it. Forced tool use (`tool_choice` of `any` or `tool`) returns a 400 error, `temperature` must be omitted, and 30-day data retention is required by default. Safeguards can refuse some security work.

**Best for.** Multi-hour autonomous engineering where finishing matters more than token price.

### GPT-6 Astra

**What it is.** [GPT-6 Astra](https://theaiengineering.com/releases/2026/09#gpt-6-astra) is the first model of OpenAI's GPT-6 generation, released September 3, 2026, and positioned in OpenAI's [announcement](https://openai.com/index/gpt-6-astra/) for the hardest work across reasoning, coding, and computer use.

**Deployment and license.** Proprietary: the OpenAI API (`gpt-6-astra`), the Codex CLI and IDE extension, and ChatGPT. The [API model page](https://developers.openai.com/api/docs/models/gpt-6-astra) lists a 1,050,000-token context window and, on November 16, 2026, $10 input and $50 output per million tokens.

**Strengths.** OpenAI reported 57.9% on Terminal-Bench 4.0 (high effort, per Anthropic's comparison) and 74.1% on DeepSWE v1.1. The DeepSWE leaderboard (updated September 22, 2026) lists it first at 74% with xhigh effort on mini-swe-agent, tied with Gemini 3.8 Flash and Claude Opus 5, at $4.43 per task.

**Trade-offs.** It has the highest per-token price here and about six times Fable 5.2's cost per FrontierSWE v2 trial. Anthropic reports higher Terminal-Bench 4.0 scores for Claude Opus 5.5 (66.4%) and Sonnet 5.5 (70.6%).

**Best for.** Codex and OpenAI API teams that need OpenAI's most capable model for long, ambiguous tasks.

### Gemini 4 Argon

**What it is.** [Gemini 4 Argon](https://theaiengineering.com/releases/2026/09#gemini-4-argon) is Google's frontier reasoning model for complex coding and agentic work, announced September 30, 2026, and generally available from October 13.

**Deployment and license.** Proprietary: the Gemini API and Google AI Studio (`gemini-4-argon`), Google Antigravity, and the Gemini app. Input and output share a 1,048,576-token window. Google AI Studio listed $2 input and $10 output per million tokens through November 30, 2026, rising to $4 and $20 from December 1.

**Strengths.** The [Gemini 4 Argon model card](https://deepmind.google/models/model-cards/gemini-4-argon/) reports 77.9% on DeepSWE v1.1 as of October 2026, the highest DeepSWE score among the models here, at the lowest list price of the three flagships.

**Trade-offs.** The same card reports 57.4% on Terminal-Bench 4.0 and 55.0% on FrontierSWE v2, both below Anthropic's reported Claude results, and defers evaluation settings to a separate methodology document.

**Best for.** Long-horizon tasks on Google Cloud, Antigravity, or the Gemini API.

## Best for price-sensitive or high-volume work

Mid-priced and Flash-class models now report agentic coding scores close to, and sometimes above, their labs' flagships. Claude Sonnet 5.5 and GPT-6.1 Sol both list at $2 input and $10 output per million tokens, and Gemini 3.8 Flash costs less than either while matching GPT-6 Astra on the DeepSWE leaderboard.

### Claude Sonnet 5.5

**What it is.** [Claude Sonnet 5.5](https://theaiengineering.com/releases/2026/09#claude-sonnet-5-5) is the mid-tier model in Anthropic's Claude 5.5 family, released September 28, 2026, for well-scoped everyday tasks and bug fixing.

**Deployment and license.** Proprietary: the Claude API (`claude-sonnet-5-5`), Bedrock, Google Cloud, Foundry, and Claude Code, with a 1M-token window. Anthropic listed it at $2 input and $10 output per million tokens on November 16, 2026.

**Strengths.** Anthropic's [announcement](https://www.anthropic.com/claude-sonnet-5-5) reports 70.6% on Terminal-Bench 4.0, above Claude Opus 5.5's 66.4% at xhigh effort in the same table, and 52.1% on FrontierCode 1.1 Main at xhigh effort.

**Trade-offs.** Anthropic does not state the effort level behind the 70.6%. At max effort, its FrontierCode score fell to 46.2%, which Anthropic attributes to extra subagent review causing timeouts.

**Best for.** A default model for daily agentic coding in Claude Code.

### GPT-6.1 Sol

**What it is.** [GPT-6.1 Sol](https://theaiengineering.com/releases/2026/09#gpt-6-1-sol) is OpenAI's upgrade to GPT-6 Sol, released September 29, 2026, which its [announcement](https://openai.com/index/introducing-gpt-6-1-sol/) positions as close to GPT-6 Astra at substantially lower cost.

**Deployment and license.** Proprietary: the OpenAI API (`gpt-6.1-sol`), the Codex CLI and IDE extension, and ChatGPT Work. The [API model page](https://developers.openai.com/api/docs/models/gpt-6.1-sol) lists a 1,050,000-token window and, on November 16, 2026, $2 input and $10 output per million tokens, one-fifth of Astra's price.

**Strengths.** OpenAI's announcement says GPT-6.1 Sol matches GPT-6 Astra on DeepSWE v1.1 at roughly one-fifth of the cost, and beats GPT-6 Sol's best score by 6.4 percentage points at a lower reasoning effort. The page shows the scores in a chart without stating them in text, so no exact figure is quoted here.

**Trade-offs.** The score is OpenAI's own; the DeepSWE leaderboard's last update predates the release.

**Best for.** High-volume Codex use and API agents where Astra's price is hard to justify.

### Gemini 3.8 Flash

**What it is.** [Gemini 3.8 Flash](https://theaiengineering.com/releases/2026/09#gemini-3-8-flash) is Google's Flash-tier model, released September 2, 2026. Google's [announcement](https://blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flash-and-3-8-flash-cyber/) highlights gains in long-horizon software engineering.

**Deployment and license.** Proprietary: the Gemini API and Google AI Studio (`gemini-3.8-flash`), Android Studio, and Antigravity, with 1,048,576 input tokens. Google AI Studio listed $0.75 input and $3.75 output per million tokens through December 31, 2026, rising to $1.50 and $7.50 on January 1, 2027.

**Strengths.** The DeepSWE leaderboard (updated September 22, 2026) lists it at 74% with high effort, tied with GPT-6 Astra, at $2.36 per task against Astra's $4.43.

**Trade-offs.** Google shows DeepSWE results only as a chart and publishes no Terminal-Bench figure for the model.

**Best for.** High-volume loops such as subagents, CI fixes, and code review passes.

## Best open-weights models for coding

Open-weights coding models from Moonshot AI, DeepSeek, and Z.ai report agentic coding scores near the proprietary frontier. They can be self-hosted or bought from hosting providers, but licenses differ: DeepSeek uses MIT, while Kimi and GLM add conditions for very large hosting businesses. The [model providers directory](https://theaiengineering.com/platforms/model-providers) lists more labs.

### Kimi K3.1 Pro

**What it is.** [Kimi K3.1 Pro](https://theaiengineering.com/releases/2026/10#kimi-k3-1-pro) is Moonshot AI's October 2026 successor to Kimi K3, a multimodal mixture-of-experts model with 2.8 trillion total and 104 billion active parameters.

**Deployment and license.** Weights are on [Hugging Face](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) under the Kimi K3.1-Pro License, which permits commercial use but requires a separate agreement for model-as-a-service businesses above $20 million in 12-month revenue. It is also served as `kimi-k3.1-pro` on the Kimi API and in Kimi Code, with a 1,048,576-token window.

**Strengths.** Moonshot reports 65.7% on Terminal-Bench 4.0 and 75.5% on DeepSWE v1.1, both in the Claude Code scaffold at max effort. The Terminal-Bench figure is within a point of Opus 5.5's 66.4%.

**Trade-offs.** The full weights occupy about 2.8 TB at 8-bit precision, so most teams will use the API or a host.

**Best for.** Frontier-class agentic coding from open weights.

### DeepSeek-V4.1-Flash-1017

**What it is.** [DeepSeek-V4.1-Flash-1017](https://theaiengineering.com/releases/2026/10#deepseek-v4-1-flash-1017) is DeepSeek's October 17, 2026, update to V4.1-Flash, a multimodal mixture-of-experts model with a 552B backbone.

**Deployment and license.** MIT-licensed weights are on [Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017), and the `deepseek-flash` API has a 1M-token window. On November 16, 2026, DeepSeek's pricing page listed cache-miss input at $0.30 and output at $1.20 per million tokens at peak, with off-peak rates at half.

**Strengths.** The model card reports 77.0% on DeepSWE v1.1 (Mini-SWE-V2 scaffold), within a point of Gemini 4 Argon, and 42.3% on Terminal-Bench 4.0 (Claude Code scaffold).

**Trade-offs.** Its Terminal-Bench 4.0 score trails Kimi K3.1 Pro's by more than 20 points, and the card omits effort levels and trial counts.

**Best for.** Low-cost API agent runs, or self-hosting under MIT terms.

### GLM-5.3

**What it is.** [GLM-5.3](https://theaiengineering.com/releases/2026/08#glm-5-3) is Z.ai's flagship coding model, released in August 2026: a text-only mixture-of-experts model with 744 billion total and 40 billion active parameters, with reasoning always on.

**Deployment and license.** Weights are on [Hugging Face](https://huggingface.co/zai-org/GLM-5.3) under the GLM-5.3 License, which grants MIT-style rights but requires a Z.ai security review for model-as-a-service businesses above US$10 billion in 12-month revenue. It serves with SGLang and vLLM, and the Z.ai API offers a 1M-token window.

**Strengths.** Z.ai reports 66.9% on DeepSWE v1.1 (mini-swe-agent) and 28.3% on Terminal-Bench 3.0 (Claude Code scaffold, 3-run average). The DeepSWE leaderboard lists it at 69% with max effort.

**Trade-offs.** Z.ai publishes no Terminal-Bench 4.0 score, its DeepSWE result trails DeepSeek's and Kimi's, and input is text only.

**Best for.** Self-hosted coding agents under a near-MIT license.

## Best small and local coding models

On published scores, the two best fits for local use are Alibaba's Qwen3.8-27B, a dense model, and Qwen3-Coder-Next, a mixture-of-experts model with 3 billion active parameters. Both use the Apache 2.0 license, and each needs a quantized build to fit most single-GPU workstations, which can change results.

### Qwen3.8-27B

**What it is.** [Qwen3.8-27B](https://theaiengineering.com/releases/2026/08#qwen3-8-27b) is the dense 27-billion-parameter model in Qwen's Qwen3.8 series, with a vision encoder, released August 14, 2026.

**Deployment and license.** Apache 2.0 weights are on [Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B), with a 262,144-token native window extensible to 1M. The BF16 weights take about 54 GB (27 billion parameters at 2 bytes each), and community quantized builds exist for llama.cpp, Ollama, and LM Studio.

**Strengths.** Qwen reports 61.7% on SWE-bench Pro and 73.0% on Terminal-Bench 2.1.

**Trade-offs.** Qwen ran SWE-bench Pro on a version it corrected, so the score is not comparable with Scale's leaderboard. It reports 42.2% on DeepSWE v1.1, far below the frontier.

**Best for.** Local coding on one workstation, and fine-tuning under Apache 2.0.

### Qwen3-Coder-Next

**What it is.** [Qwen3-Coder-Next](https://theaiengineering.com/releases/2026/02#qwen3-coder-next) is Qwen's latest dedicated coding model, released February 3, 2026: a mixture-of-experts model with 80 billion total and 3 billion active parameters.

**Deployment and license.** Apache 2.0, with a 262,144-token window and FP8 and GGUF builds that work with agents such as Claude Code, Qwen Code, and Cline. All 80 billion parameters must fit in memory (about 160 GB in BF16), but only about 3 billion compute each token, so generation is fast.

**Strengths.** The [Qwen3-Coder-Next technical report](https://arxiv.org/abs/2603.00729) reports 70.6% on SWE-bench Verified with SWE-Agent and 42.7% on SWE-bench Pro, with agents capped at 300 turns.

**Trade-offs.** It is the oldest entry here, and Qwen's GitHub repository listed no newer coder model on November 16, 2026.

**Best for.** Fast local agent loops on a high-memory machine.

## Other models worth testing

Several models were left off the main list because a sibling covers the same job, their scores trail comparable entries, or access is restricted. Two of them, MiniMax M3.1 Flash and North Mini Code, publish open weights. Prices below are list prices on November 16, 2026.

- **Claude Opus 5.5.** Anthropic's suggested default reports 66.4% on Terminal-Bench 4.0 at xhigh effort, at $4 input and $20 output per million tokens.
- **Claude Haiku 5.5.** Reports 39.2% on Terminal-Bench 4.0, from $0.10 input and $0.50 output per million tokens.
- **MiniMax M3.1 Flash.** Open weights; reports 73.7% on DeepSWE v1.1 and 38.1% on Terminal-Bench 4.0 (Codex scaffold), under a license with attribution and revenue conditions.
- **North Mini Code.** Cohere's Apache 2.0 model reports 67.6% on SWE-bench Verified with SWE-agent.
- **Mistral.** Mistral retired Devstral 2 from its API on July 31, 2026, naming Mistral Medium 3.5 as the replacement.
- **Limited access.** Claude Mythos 5.2 requires Anthropic's verification programs, GPT-6 Cyber is limited to approved Daybreak Red customers, Gemini 4 Carbon went first to a subset of cyber defenders, and SWE-2 Flash launched only in Cognition's Devin Desktop and CLI.

## Why benchmark rank is not the whole answer

A benchmark rank measures one model inside one agent setup, on one task set, at one effort level, over a few trials. Change any of those and the score moves, sometimes by more than the gap between models. Four effects explain most disagreements between published coding scores, and each argues for testing on your own code.

### Contamination

[Benchmark contamination](https://theaiengineering.com/glossary/benchmark-contamination) means test tasks or their solutions leaked into training data, so a score reflects recall rather than skill. SWE-bench draws on public GitHub fixes, which is part of why OpenAI dropped Verified. Newer benchmarks defend against it: DeepSWE never publishes its solutions, SWE-Bench Pro keeps held-out and private sets, and Terminal-Bench 4.0 removed two tasks whose solutions had become public.

### Scaffold differences

An agentic score describes a model inside a scaffold: the agent program that sends prompts, exposes tools, runs commands, and decides when to stop. Kimi, Z.ai, and DeepSeek report Terminal-Bench in the Claude Code scaffold, while MiniMax uses Codex.

**Figure 1: An agentic coding score depends on the scaffold, sandbox, and verifier as well as the model.**

The agent scaffold sends the task to the model, executes each tool call in a sandbox, loops until done, and sends the final repository state to a verifier.

Steps:

1. Agent scaffold → Model: Task, tools, context
2. Model → Agent scaffold: Tool call
3. Agent scaffold → Sandbox: Run in container
4. Sandbox → Agent scaffold: Output, exit code
5. Agent scaffold → Model: Result, next turn
6. Agent scaffold → Verifier: Final repo state
7. Verifier → Agent scaffold: Pass or fail

Qwen's technical report shows the effect for one model:

| Benchmark | Scaffold | Qwen3-Coder-Next score |
|---|---|---|
| SWE-bench Verified | SWE-Agent | 70.6% |
| SWE-bench Verified | OpenHands | 71.3% |
| Terminal-Bench 2.0 | Terminus 2 (JSON) | 36.2% |
| Terminal-Bench 2.0 | Claude Code | 30.9% |
| Terminal-Bench 2.0 | Qwen Code | 25.8% |

The 10.4-point Terminal-Bench spread exceeds the gap between many neighboring models here.

### Trials and pass@k

Agent runs are random, so benchmarks repeat them. [pass@k](https://theaiengineering.com/glossary/pass-at-k) counts a task as solved if any of k attempts passes, so it rises with k; pass@1 and mean@5 describe a typical single attempt. The DeepSWE leaderboard shows confidence intervals of plus or minus 1 to 6 points, and Anthropic gives a standard error of 2.6 points for Opus 5.5 on Terminal-Bench 4.0, so smaller gaps are noise.

### Effort, safeguards, and self-reporting

Headline scores usually use the highest effort setting, while Claude Code defaults Sonnet 5.5 to medium. More effort is not always better, as Sonnet 5.5's lower FrontierCode score at max effort shows; the [reasoning models explainer](https://theaiengineering.com/articles/reasoning-models) covers that trade-off. Anthropic also says that when safeguards intervened in Opus 5.5's evaluations, other Claude models finished those tasks, likely lowering its scores. And self-reported numbers can differ from leaderboards: Z.ai reports 66.9% for GLM-5.3 on DeepSWE, the leaderboard 69%.

## How to choose and test on your own repository

Use public scores to build a shortlist of three to five models, then decide with a short trial on your own code. Hold the scaffold, effort setting, and task set fixed, run each task several times, and compare models on cost per merged fix rather than benchmark rank. The process takes a day or two.

1. **Filter by constraints.** Data residency points to open weights, your coding agent limits which models you can use, and budget sets the tier.
2. **Build tasks from your history.** Take 10 to 30 closed issues with known fixes and tests, and remove the fixes.
3. **Fix the setup.** Use one scaffold, one instructions file, and one effort level for every candidate, with at least three runs per task.
4. **Grade like a reviewer.** Run the tests, then judge whether each passing diff would merge with minor edits.
5. **Compare cost per merged fix.** Divide token spend by mergeable results, and record time per task.

**Figure 2: Public scores build the shortlist; a fixed trial on your own repository makes the decision.**

A shortlist from public scores feeds a task set from your repository, run under one fixed setup, graded by tests and review, and compared by cost per merged fix.

Components:

- **Shortlist**: public scores, 3 to 5 models
- **Your tasks**: closed issues, fixes removed
- **Fixed setup**: one scaffold, 3+ runs each
- **Grade**: tests, then human review
- **Decide**: cost per merged fix

Connections:

- Shortlist → Your tasks
- Your tasks → Fixed setup
- Fixed setup → Grade
- Grade → Decide

The [AI coding agents comparison](https://theaiengineering.com/articles/ai-coding-agents) includes a scoring rubric for this kind of trial. To check a published number, rerun the benchmark in the lab's scaffold. Terminal-Bench runs on Harbor, which uses Docker by default:

```bash
# Run Terminal-Bench 4.0 with Claude Code driving one candidate model.
export ANTHROPIC_API_KEY=<your-key>
harbor run --dataset terminal-bench/terminal-bench@4.0.0 \
  --agent claude-code \
  --model anthropic/claude-opus-5-5 \
  --n-concurrent 4
```

Scores and prices change with each release, so recheck each lab's page and rerun the trial when a new model ships.

## Frequently asked questions

### What is the best LLM for coding right now?

As of November 2026, no model leads every coding benchmark. Claude Fable 5.2 leads the FrontierSWE v2 leaderboard at 69.9%, Gemini 4 Argon reports 77.9% on DeepSWE v1.1, and Claude Sonnet 5.5 reports 70.6% on Terminal-Bench 4.0. Mid-priced models such as Sonnet 5.5 and GPT-6.1 Sol are practical daily defaults; confirm with a trial on your own repository.

### Is Claude better than GPT for coding?

It depends on the benchmark and price tier. Claude Fable 5.2 leads GPT-6 Astra on the FrontierSWE v2 leaderboard, 69.9% to 65.5%, and Anthropic reports higher Terminal-Bench 4.0 scores for Claude models than OpenAI reports for Astra. OpenAI says GPT-6.1 Sol matches Astra on DeepSWE v1.1 at about a fifth of the cost, and Sol costs the same per token as Sonnet 5.5. Test both in your agent.

### What is the best open-source LLM for coding?

Kimi K3.1 Pro reports 65.7% on Terminal-Bench 4.0 and 75.5% on DeepSWE v1.1, and DeepSeek-V4.1-Flash-1017 reports 77.0% on DeepSWE v1.1 under the MIT license. Both publish weights but not full training data, so they are open-weights rather than open source in the strict sense, and Kimi's license adds conditions for large hosts.

### What is the best LLM for coding locally?

Two Apache 2.0 Qwen models fit local use. Qwen3.8-27B, a dense 27B model, reports 61.7% on Qwen's corrected SWE-bench Pro. Qwen3-Coder-Next has 80B parameters but only 3B active per token, and reports 70.6% on SWE-bench Verified with SWE-Agent. Both need quantization for most single-GPU workstations, so test the build you will run.

### Why don't AI labs report SWE-bench Verified anymore?

In February 2026, OpenAI stopped reporting SWE-bench Verified, citing tests that reject correct solutions and evidence that models had seen the solutions in training, and recommended SWE-bench Pro. By November 2026, frontier announcements report Terminal-Bench 4.0 and DeepSWE v1.1 instead, and Verified appears mainly on smaller open-weights model cards.

## Terms in this guide

- [SWE-bench](https://theaiengineering.com/glossary/swe-bench): SWE-bench is a benchmark of 2,294 real GitHub issues from 12 Python repositories, in which a model or agent must write a patch that resolves the issue and passes the repository's tests.
- [Open-weights model](https://theaiengineering.com/glossary/open-weights-model): An open-weights model is an AI model whose trained parameters are published for download, so anyone can run, fine-tune, or quantize it on their own hardware under the terms of its license.
- [Benchmark contamination](https://theaiengineering.com/glossary/benchmark-contamination): Benchmark contamination is the presence of a benchmark's test questions, answers, or solutions in a model's training data, so its score can reflect memorization instead of the capability the benchmark measures.
- [pass@k](https://theaiengineering.com/glossary/pass-at-k): pass@k is an evaluation metric that estimates the probability that at least one of k sampled outputs for a task passes its tests, used mainly in code generation and agent benchmarks.

## Related guides

- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.
- [AI coding agents: how they work and how the major tools compare](https://theaiengineering.com/articles/ai-coding-agents): AI coding agents compared: how they differ from autocomplete, the four main categories, a comparison table of major tools, and how to evaluate and secure them.
- [Reasoning models: how they work and when to use them](https://theaiengineering.com/articles/reasoning-models): Reasoning models explained: how test-time compute and reinforcement learning produce chain-of-thought reasoning, when to use them, and their cost and latency.
- [Agentic workflows: five patterns and when to use each](https://theaiengineering.com/articles/agentic-workflows): Agentic workflows run LLM and tool calls through steps fixed in code. Learn five patterns, their cost and latency, failure modes, and when to use an agent.

Part of the [LLM models and benchmarks](https://theaiengineering.com/topics/models) topic hub.

## Cite this guide

The AI Engineering Editors. "Best LLM for coding: models compared on agentic coding benchmarks." The AI Engineering, updated Nov 17, 2026. https://theaiengineering.com/articles/best-llm-for-coding

- Title: Best LLM for coding: models compared on agentic coding benchmarks
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-11-17
- Updated: 2026-11-17
- URL: https://theaiengineering.com/articles/best-llm-for-coding
- Markdown: https://theaiengineering.com/articles/best-llm-for-coding.md

To cite one section, link to its anchor:

- [How the models were selected](https://theaiengineering.com/articles/best-llm-for-coding#how-the-models-were-selected)
- [Best LLMs for coding at a glance](https://theaiengineering.com/articles/best-llm-for-coding#best-llms-for-coding-at-a-glance)
- [Best overall for agentic coding](https://theaiengineering.com/articles/best-llm-for-coding#best-overall-for-agentic-coding)
- [Best for price-sensitive or high-volume work](https://theaiengineering.com/articles/best-llm-for-coding#best-for-price-sensitive-or-high-volume-work)
- [Best open-weights models for coding](https://theaiengineering.com/articles/best-llm-for-coding#best-open-weights-models-for-coding)
- [Best small and local coding models](https://theaiengineering.com/articles/best-llm-for-coding#best-small-and-local-coding-models)
- [Other models worth testing](https://theaiengineering.com/articles/best-llm-for-coding#other-models-worth-testing)
- [Why benchmark rank is not the whole answer](https://theaiengineering.com/articles/best-llm-for-coding#why-benchmark-rank-is-not-the-whole-answer)
- [How to choose and test on your own repository](https://theaiengineering.com/articles/best-llm-for-coding#how-to-choose-and-test-on-your-own-repository)
- [Frequently asked questions](https://theaiengineering.com/articles/best-llm-for-coding#faq)
