Best LLM for coding: models compared on agentic coding benchmarks
No single model leads every agentic coding benchmark as of November 2026, so the best LLM for coding depends on the work: multi-hour autonomous tasks, high-volume edits in an agent loop, self-hosted inference, or a model that runs on one workstation. Claude Fable 5.2 tops the FrontierSWE v2 leaderboard, Gemini 4 Argon reports the highest DeepSWE v1.1 score among the models below, and Claude Sonnet 5.5 reports the highest lab-reported Terminal-Bench 4.0 score.
This comparison groups 11 models by use. Each score names its source, date, and disclosed setting, and where a lab publishes no score, the entry says so. For how coding benchmarks work, start with LLM benchmarks explained, part of the models topic hub.
How the models were selected#
Each model was judged on six criteria: published agentic coding scores, context window, tool-use behavior, availability in APIs and coding agents, whether weights are open, and cost structure. Every entry had to be generally available on November 16, 2026, so limited-access models such as GPT-6 Cyber and Gemini 4 Carbon appear only in the closing list.
- Agentic coding benchmarks. Scores from the lab's announcement or model card, or the benchmark's official leaderboard, on tests that run many turns of file edits and commands.
- Context window. A longer window holds more of a repository, its logs, and the agent's history.
- Tool-use behavior. Agentic benchmarks exercise hundreds of tool calls per task, and API limits matter: Claude Fable 5.2 rejects requests that force a specific tool.
- Availability. APIs, cloud platforms, and coding agents such as Claude Code, Codex, Google Antigravity, and Kimi Code.
- Open weights. An open-weights model publishes its parameters for download, often without training data, under a license that may add conditions.
- Cost structure. Dated list prices from each pricing page and, for open weights, the memory the full model needs.
Which coding benchmarks labs report in 2026#
In February 2026, OpenAI stopped reporting SWE-bench Verified, citing flawed tests and evidence that models had seen the solutions, and recommended SWE-bench Pro. By November, the launch materials for Claude Fable 5.2, GPT-6 Astra, and Gemini 4 Argon reported no SWE-bench variant, and the top ten entries on Scale AI's public SWE-Bench Pro leaderboard, checked November 16, were all models released by July 2026.
| Benchmark | Maintainer | What it tests | How it is scored |
|---|---|---|---|
| SWE-bench Pro | Scale AI | 1,865 tasks from 41 repositories; 731 public | Percent resolved; 250-turn limit on the public leaderboard |
| Terminal-Bench 4.0 | Terminal-Bench project, on the Harbor framework | Multi-step terminal tasks; 4.0 removed 8 tasks and fixed 19 | Resolution rate, 8-hour agent timeout per task |
| DeepSWE v1.1 | Datacurve | 113 long-horizon tasks written from scratch in 91 repositories | Hand-written verifiers; leaderboard uses mini-swe-agent for all models |
| FrontierSWE v2 | Proximal | 34 open-ended tasks, such as a Lean 4 type checker in Pascal | Mean of 5 trials, 20-hour budget each |
| FrontierCode | Cognition | 150 tasks built with open-source maintainers; Main is the 100 hardest | Fails on any blocker criterion, else a weighted rubric score |
Sources: Terminal-Bench 4.0, DeepSWE, FrontierSWE, and FrontierCode. Scores on different benchmarks never compare directly, because each measures different tasks under different rules.
Best LLMs for coding at a glance#
The table summarizes all 11 entries: lab, license, context window, the most prominent coding score each lab or leaderboard reports, and the job each model fits. Read each headline score as evidence for that benchmark only, because each row uses whichever benchmark its lab chose to publish. Dates, prices, and settings are in the entries.
| Model | Lab | Open weights | Context | Headline coding score (source, setting) | Best for |
|---|---|---|---|---|---|
| Claude Fable 5.2 | Anthropic | No | 1M | FrontierSWE v2: 69.9%, mean of 5 trials (Proximal leaderboard) | Multi-hour autonomous engineering |
| GPT-6 Astra | OpenAI | No | 1.05M | Terminal-Bench 4.0: 57.9% (OpenAI) | Hard tasks in Codex and the OpenAI API |
| Gemini 4 Argon | No | 1,048,576, input and output | DeepSWE v1.1: 77.9% (Google model card) | Long-horizon work on Google's stack | |
| Claude Sonnet 5.5 | Anthropic | No | 1M | Terminal-Bench 4.0: 70.6%, effort not stated (Anthropic) | Daily agentic coding at a mid-tier price |
| GPT-6.1 Sol | OpenAI | No | 1.05M | DeepSWE v1.1: matches GPT-6 Astra (OpenAI) | Near-flagship results at a fifth of Astra's price |
| Gemini 3.8 Flash | No | 1,048,576 | DeepSWE v1.1: 74%, high effort, mini-swe-agent (DeepSWE leaderboard) | High-volume agent loops | |
| Kimi K3.1 Pro | Moonshot AI | Yes, custom license | 1,048,576 | Terminal-Bench 4.0: 65.7%, Claude Code scaffold, max effort (Moonshot AI) | Frontier-class open weights |
| DeepSeek-V4.1-Flash-1017 | DeepSeek | Yes, MIT | 1M | DeepSWE v1.1: 77.0%, Mini-SWE-V2 scaffold (DeepSeek) | Low-cost API or MIT-licensed hosting |
| GLM-5.3 | Z.ai | Yes, custom license | 1M | DeepSWE v1.1: 66.9%, mini-swe-agent (Z.ai) | Self-hosted coding agents |
| Qwen3.8-27B | Alibaba Qwen | Yes, Apache 2.0 | 262,144 | SWE-bench Pro: 61.7%, Qwen's corrected tasks, Claude Code scaffold (Qwen) | One-workstation local use |
| Qwen3-Coder-Next | Alibaba Qwen | Yes, Apache 2.0 | 262,144 | SWE-bench Verified: 70.6%, SWE-Agent (Qwen) | Fast local agent loops |
Best overall for agentic coding#
The three strongest generally available models for long, autonomous coding tasks are Claude Fable 5.2, GPT-6 Astra, and Gemini 4 Argon. Each is first on at least one current agentic coding benchmark, by leaderboard position or lab-reported score, and each sits at or near the top of its lab's price list.
Claude Fable 5.2#
What it is. Claude Fable 5.2 is Anthropic's most capable generally available model, released November 9, 2026, for long-horizon agentic work, per its model documentation. Claude Mythos 5.2, the same model with fewer safeguards, is limited to verified organizations.
Deployment and license. Proprietary: the Claude API (claude-fable-5-2), Amazon Bedrock, Google Cloud, Microsoft Foundry, and Claude Code, with a 1M-token context window. Anthropic listed it at $8 input and $40 output per million tokens on November 16, 2026.
Strengths. It ranks first on the FrontierSWE v2 leaderboard at 69.9%, ahead of GPT-6 Astra's 65.5%, at an average of $168.78 per trial against Astra's $1,029.65. Anthropic's announcement also reports 49.1% on FrontierCode 2.0 at xhigh effort.
Trade-offs. Anthropic published no Terminal-Bench 4.0 or SWE-bench score for it. Forced tool use (tool_choice of any or tool) returns a 400 error, temperature must be omitted, and 30-day data retention is required by default. Safeguards can refuse some security work.
Best for. Multi-hour autonomous engineering where finishing matters more than token price.
GPT-6 Astra#
What it is. GPT-6 Astra is the first model of OpenAI's GPT-6 generation, released September 3, 2026, and positioned in OpenAI's announcement for the hardest work across reasoning, coding, and computer use.
Deployment and license. Proprietary: the OpenAI API (gpt-6-astra), the Codex CLI and IDE extension, and ChatGPT. The API model page lists a 1,050,000-token context window and, on November 16, 2026, $10 input and $50 output per million tokens.
Strengths. OpenAI reported 57.9% on Terminal-Bench 4.0 (high effort, per Anthropic's comparison) and 74.1% on DeepSWE v1.1. The DeepSWE leaderboard (updated September 22, 2026) lists it first at 74% with xhigh effort on mini-swe-agent, tied with Gemini 3.8 Flash and Claude Opus 5, at $4.43 per task.
Trade-offs. It has the highest per-token price here and about six times Fable 5.2's cost per FrontierSWE v2 trial. Anthropic reports higher Terminal-Bench 4.0 scores for Claude Opus 5.5 (66.4%) and Sonnet 5.5 (70.6%).
Best for. Codex and OpenAI API teams that need OpenAI's most capable model for long, ambiguous tasks.
Gemini 4 Argon#
What it is. Gemini 4 Argon is Google's frontier reasoning model for complex coding and agentic work, announced September 30, 2026, and generally available from October 13.
Deployment and license. Proprietary: the Gemini API and Google AI Studio (gemini-4-argon), Google Antigravity, and the Gemini app. Input and output share a 1,048,576-token window. Google AI Studio listed $2 input and $10 output per million tokens through November 30, 2026, rising to $4 and $20 from December 1.
Strengths. The Gemini 4 Argon model card reports 77.9% on DeepSWE v1.1 as of October 2026, the highest DeepSWE score among the models here, at the lowest list price of the three flagships.
Trade-offs. The same card reports 57.4% on Terminal-Bench 4.0 and 55.0% on FrontierSWE v2, both below Anthropic's reported Claude results, and defers evaluation settings to a separate methodology document.
Best for. Long-horizon tasks on Google Cloud, Antigravity, or the Gemini API.
Best for price-sensitive or high-volume work#
Mid-priced and Flash-class models now report agentic coding scores close to, and sometimes above, their labs' flagships. Claude Sonnet 5.5 and GPT-6.1 Sol both list at $2 input and $10 output per million tokens, and Gemini 3.8 Flash costs less than either while matching GPT-6 Astra on the DeepSWE leaderboard.
Claude Sonnet 5.5#
What it is. Claude Sonnet 5.5 is the mid-tier model in Anthropic's Claude 5.5 family, released September 28, 2026, for well-scoped everyday tasks and bug fixing.
Deployment and license. Proprietary: the Claude API (claude-sonnet-5-5), Bedrock, Google Cloud, Foundry, and Claude Code, with a 1M-token window. Anthropic listed it at $2 input and $10 output per million tokens on November 16, 2026.
Strengths. Anthropic's announcement reports 70.6% on Terminal-Bench 4.0, above Claude Opus 5.5's 66.4% at xhigh effort in the same table, and 52.1% on FrontierCode 1.1 Main at xhigh effort.
Trade-offs. Anthropic does not state the effort level behind the 70.6%. At max effort, its FrontierCode score fell to 46.2%, which Anthropic attributes to extra subagent review causing timeouts.
Best for. A default model for daily agentic coding in Claude Code.
GPT-6.1 Sol#
What it is. GPT-6.1 Sol is OpenAI's upgrade to GPT-6 Sol, released September 29, 2026, which its announcement positions as close to GPT-6 Astra at substantially lower cost.
Deployment and license. Proprietary: the OpenAI API (gpt-6.1-sol), the Codex CLI and IDE extension, and ChatGPT Work. The API model page lists a 1,050,000-token window and, on November 16, 2026, $2 input and $10 output per million tokens, one-fifth of Astra's price.
Strengths. OpenAI's announcement says GPT-6.1 Sol matches GPT-6 Astra on DeepSWE v1.1 at roughly one-fifth of the cost, and beats GPT-6 Sol's best score by 6.4 percentage points at a lower reasoning effort. The page shows the scores in a chart without stating them in text, so no exact figure is quoted here.
Trade-offs. The score is OpenAI's own; the DeepSWE leaderboard's last update predates the release.
Best for. High-volume Codex use and API agents where Astra's price is hard to justify.
Gemini 3.8 Flash#
What it is. Gemini 3.8 Flash is Google's Flash-tier model, released September 2, 2026. Google's announcement highlights gains in long-horizon software engineering.
Deployment and license. Proprietary: the Gemini API and Google AI Studio (gemini-3.8-flash), Android Studio, and Antigravity, with 1,048,576 input tokens. Google AI Studio listed $0.75 input and $3.75 output per million tokens through December 31, 2026, rising to $1.50 and $7.50 on January 1, 2027.
Strengths. The DeepSWE leaderboard (updated September 22, 2026) lists it at 74% with high effort, tied with GPT-6 Astra, at $2.36 per task against Astra's $4.43.
Trade-offs. Google shows DeepSWE results only as a chart and publishes no Terminal-Bench figure for the model.
Best for. High-volume loops such as subagents, CI fixes, and code review passes.
Best open-weights models for coding#
Open-weights coding models from Moonshot AI, DeepSeek, and Z.ai report agentic coding scores near the proprietary frontier. They can be self-hosted or bought from hosting providers, but licenses differ: DeepSeek uses MIT, while Kimi and GLM add conditions for very large hosting businesses. The model providers directory lists more labs.
Kimi K3.1 Pro#
What it is. Kimi K3.1 Pro is Moonshot AI's October 2026 successor to Kimi K3, a multimodal mixture-of-experts model with 2.8 trillion total and 104 billion active parameters.
Deployment and license. Weights are on Hugging Face under the Kimi K3.1-Pro License, which permits commercial use but requires a separate agreement for model-as-a-service businesses above $20 million in 12-month revenue. It is also served as kimi-k3.1-pro on the Kimi API and in Kimi Code, with a 1,048,576-token window.
Strengths. Moonshot reports 65.7% on Terminal-Bench 4.0 and 75.5% on DeepSWE v1.1, both in the Claude Code scaffold at max effort. The Terminal-Bench figure is within a point of Opus 5.5's 66.4%.
Trade-offs. The full weights occupy about 2.8 TB at 8-bit precision, so most teams will use the API or a host.
Best for. Frontier-class agentic coding from open weights.
DeepSeek-V4.1-Flash-1017#
What it is. DeepSeek-V4.1-Flash-1017 is DeepSeek's October 17, 2026, update to V4.1-Flash, a multimodal mixture-of-experts model with a 552B backbone.
Deployment and license. MIT-licensed weights are on Hugging Face, and the deepseek-flash API has a 1M-token window. On November 16, 2026, DeepSeek's pricing page listed cache-miss input at $0.30 and output at $1.20 per million tokens at peak, with off-peak rates at half.
Strengths. The model card reports 77.0% on DeepSWE v1.1 (Mini-SWE-V2 scaffold), within a point of Gemini 4 Argon, and 42.3% on Terminal-Bench 4.0 (Claude Code scaffold).
Trade-offs. Its Terminal-Bench 4.0 score trails Kimi K3.1 Pro's by more than 20 points, and the card omits effort levels and trial counts.
Best for. Low-cost API agent runs, or self-hosting under MIT terms.
GLM-5.3#
What it is. GLM-5.3 is Z.ai's flagship coding model, released in August 2026: a text-only mixture-of-experts model with 744 billion total and 40 billion active parameters, with reasoning always on.
Deployment and license. Weights are on Hugging Face under the GLM-5.3 License, which grants MIT-style rights but requires a Z.ai security review for model-as-a-service businesses above US$10 billion in 12-month revenue. It serves with SGLang and vLLM, and the Z.ai API offers a 1M-token window.
Strengths. Z.ai reports 66.9% on DeepSWE v1.1 (mini-swe-agent) and 28.3% on Terminal-Bench 3.0 (Claude Code scaffold, 3-run average). The DeepSWE leaderboard lists it at 69% with max effort.
Trade-offs. Z.ai publishes no Terminal-Bench 4.0 score, its DeepSWE result trails DeepSeek's and Kimi's, and input is text only.
Best for. Self-hosted coding agents under a near-MIT license.
Best small and local coding models#
On published scores, the two best fits for local use are Alibaba's Qwen3.8-27B, a dense model, and Qwen3-Coder-Next, a mixture-of-experts model with 3 billion active parameters. Both use the Apache 2.0 license, and each needs a quantized build to fit most single-GPU workstations, which can change results.
Qwen3.8-27B#
What it is. Qwen3.8-27B is the dense 27-billion-parameter model in Qwen's Qwen3.8 series, with a vision encoder, released August 14, 2026.
Deployment and license. Apache 2.0 weights are on Hugging Face, with a 262,144-token native window extensible to 1M. The BF16 weights take about 54 GB (27 billion parameters at 2 bytes each), and community quantized builds exist for llama.cpp, Ollama, and LM Studio.
Strengths. Qwen reports 61.7% on SWE-bench Pro and 73.0% on Terminal-Bench 2.1.
Trade-offs. Qwen ran SWE-bench Pro on a version it corrected, so the score is not comparable with Scale's leaderboard. It reports 42.2% on DeepSWE v1.1, far below the frontier.
Best for. Local coding on one workstation, and fine-tuning under Apache 2.0.
Qwen3-Coder-Next#
What it is. Qwen3-Coder-Next is Qwen's latest dedicated coding model, released February 3, 2026: a mixture-of-experts model with 80 billion total and 3 billion active parameters.
Deployment and license. Apache 2.0, with a 262,144-token window and FP8 and GGUF builds that work with agents such as Claude Code, Qwen Code, and Cline. All 80 billion parameters must fit in memory (about 160 GB in BF16), but only about 3 billion compute each token, so generation is fast.
Strengths. The Qwen3-Coder-Next technical report reports 70.6% on SWE-bench Verified with SWE-Agent and 42.7% on SWE-bench Pro, with agents capped at 300 turns.
Trade-offs. It is the oldest entry here, and Qwen's GitHub repository listed no newer coder model on November 16, 2026.
Best for. Fast local agent loops on a high-memory machine.
Other models worth testing#
Several models were left off the main list because a sibling covers the same job, their scores trail comparable entries, or access is restricted. Two of them, MiniMax M3.1 Flash and North Mini Code, publish open weights. Prices below are list prices on November 16, 2026.
- Claude Opus 5.5. Anthropic's suggested default reports 66.4% on Terminal-Bench 4.0 at xhigh effort, at $4 input and $20 output per million tokens.
- Claude Haiku 5.5. Reports 39.2% on Terminal-Bench 4.0, from $0.10 input and $0.50 output per million tokens.
- MiniMax M3.1 Flash. Open weights; reports 73.7% on DeepSWE v1.1 and 38.1% on Terminal-Bench 4.0 (Codex scaffold), under a license with attribution and revenue conditions.
- North Mini Code. Cohere's Apache 2.0 model reports 67.6% on SWE-bench Verified with SWE-agent.
- Mistral. Mistral retired Devstral 2 from its API on July 31, 2026, naming Mistral Medium 3.5 as the replacement.
- Limited access. Claude Mythos 5.2 requires Anthropic's verification programs, GPT-6 Cyber is limited to approved Daybreak Red customers, Gemini 4 Carbon went first to a subset of cyber defenders, and SWE-2 Flash launched only in Cognition's Devin Desktop and CLI.
Why benchmark rank is not the whole answer#
A benchmark rank measures one model inside one agent setup, on one task set, at one effort level, over a few trials. Change any of those and the score moves, sometimes by more than the gap between models. Four effects explain most disagreements between published coding scores, and each argues for testing on your own code.
Contamination#
Benchmark contamination means test tasks or their solutions leaked into training data, so a score reflects recall rather than skill. SWE-bench draws on public GitHub fixes, which is part of why OpenAI dropped Verified. Newer benchmarks defend against it: DeepSWE never publishes its solutions, SWE-Bench Pro keeps held-out and private sets, and Terminal-Bench 4.0 removed two tasks whose solutions had become public.
Scaffold differences#
An agentic score describes a model inside a scaffold: the agent program that sends prompts, exposes tools, runs commands, and decides when to stop. Kimi, Z.ai, and DeepSeek report Terminal-Bench in the Claude Code scaffold, while MiniMax uses Codex.
Qwen's technical report shows the effect for one model:
| Benchmark | Scaffold | Qwen3-Coder-Next score |
|---|---|---|
| SWE-bench Verified | SWE-Agent | 70.6% |
| SWE-bench Verified | OpenHands | 71.3% |
| Terminal-Bench 2.0 | Terminus 2 (JSON) | 36.2% |
| Terminal-Bench 2.0 | Claude Code | 30.9% |
| Terminal-Bench 2.0 | Qwen Code | 25.8% |
The 10.4-point Terminal-Bench spread exceeds the gap between many neighboring models here.
Trials and pass@k#
Agent runs are random, so benchmarks repeat them. pass@k counts a task as solved if any of k attempts passes, so it rises with k; pass@1 and mean@5 describe a typical single attempt. The DeepSWE leaderboard shows confidence intervals of plus or minus 1 to 6 points, and Anthropic gives a standard error of 2.6 points for Opus 5.5 on Terminal-Bench 4.0, so smaller gaps are noise.
Effort, safeguards, and self-reporting#
Headline scores usually use the highest effort setting, while Claude Code defaults Sonnet 5.5 to medium. More effort is not always better, as Sonnet 5.5's lower FrontierCode score at max effort shows; the reasoning models explainer covers that trade-off. Anthropic also says that when safeguards intervened in Opus 5.5's evaluations, other Claude models finished those tasks, likely lowering its scores. And self-reported numbers can differ from leaderboards: Z.ai reports 66.9% for GLM-5.3 on DeepSWE, the leaderboard 69%.
How to choose and test on your own repository#
Use public scores to build a shortlist of three to five models, then decide with a short trial on your own code. Hold the scaffold, effort setting, and task set fixed, run each task several times, and compare models on cost per merged fix rather than benchmark rank. The process takes a day or two.
- Filter by constraints. Data residency points to open weights, your coding agent limits which models you can use, and budget sets the tier.
- Build tasks from your history. Take 10 to 30 closed issues with known fixes and tests, and remove the fixes.
- Fix the setup. Use one scaffold, one instructions file, and one effort level for every candidate, with at least three runs per task.
- Grade like a reviewer. Run the tests, then judge whether each passing diff would merge with minor edits.
- Compare cost per merged fix. Divide token spend by mergeable results, and record time per task.
The AI coding agents comparison includes a scoring rubric for this kind of trial. To check a published number, rerun the benchmark in the lab's scaffold. Terminal-Bench runs on Harbor, which uses Docker by default:
# Run Terminal-Bench 4.0 with Claude Code driving one candidate model.
export ANTHROPIC_API_KEY=<your-key>
harbor run --dataset terminal-bench/terminal-bench@4.0.0 \
--agent claude-code \
--model anthropic/claude-opus-5-5 \
--n-concurrent 4Scores and prices change with each release, so recheck each lab's page and rerun the trial when a new model ships.
Frequently asked questions
What is the best LLM for coding right now?
- As of November 2026, no model leads every coding benchmark. Claude Fable 5.2 leads the FrontierSWE v2 leaderboard at 69.9%, Gemini 4 Argon reports 77.9% on DeepSWE v1.1, and Claude Sonnet 5.5 reports 70.6% on Terminal-Bench 4.0. Mid-priced models such as Sonnet 5.5 and GPT-6.1 Sol are practical daily defaults; confirm with a trial on your own repository.
Is Claude better than GPT for coding?
- It depends on the benchmark and price tier. Claude Fable 5.2 leads GPT-6 Astra on the FrontierSWE v2 leaderboard, 69.9% to 65.5%, and Anthropic reports higher Terminal-Bench 4.0 scores for Claude models than OpenAI reports for Astra. OpenAI says GPT-6.1 Sol matches Astra on DeepSWE v1.1 at about a fifth of the cost, and Sol costs the same per token as Sonnet 5.5. Test both in your agent.
What is the best open-source LLM for coding?
- Kimi K3.1 Pro reports 65.7% on Terminal-Bench 4.0 and 75.5% on DeepSWE v1.1, and DeepSeek-V4.1-Flash-1017 reports 77.0% on DeepSWE v1.1 under the MIT license. Both publish weights but not full training data, so they are open-weights rather than open source in the strict sense, and Kimi's license adds conditions for large hosts.
What is the best LLM for coding locally?
- Two Apache 2.0 Qwen models fit local use. Qwen3.8-27B, a dense 27B model, reports 61.7% on Qwen's corrected SWE-bench Pro. Qwen3-Coder-Next has 80B parameters but only 3B active per token, and reports 70.6% on SWE-bench Verified with SWE-Agent. Both need quantization for most single-GPU workstations, so test the build you will run.
Why don't AI labs report SWE-bench Verified anymore?
- In February 2026, OpenAI stopped reporting SWE-bench Verified, citing tests that reject correct solutions and evidence that models had seen the solutions in training, and recommended SWE-bench Pro. By November 2026, frontier announcements report Terminal-Bench 4.0 and DeepSWE v1.1 instead, and Verified appears mainly on smaller open-weights model cards.
Terms in this guide
Cite this guide
The AI Engineering Editors. "Best LLM for coding: models compared on agentic coding benchmarks." The AI Engineering, updated Nov 17, 2026. https://theaiengineering.com/articles/best-llm-for-coding
- Title
- Best LLM for coding: models compared on agentic coding benchmarks
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Published
- Updated