---
title: "What is LMArena (Chatbot Arena)? How it ranks LLMs"
description: "LMArena, formerly Chatbot Arena and now branded Arena, ranks LLMs from pairwise human votes with a Bradley-Terry model. How it works and its known biases."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/lmarena"
markdown_url: "https://theaiengineering.com/glossary/lmarena.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["LMArena","Chatbot Arena","LMSYS Chatbot Arena"]
citation: "The AI Engineering Editors. \"What is LMArena (Chatbot Arena)? How it ranks LLMs.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/lmarena"
---

# What is LMArena (Chatbot Arena)? How it ranks LLMs

Also called Chatbot Arena, LMSYS Chatbot Arena.

> **Definition of LMArena:** LMArena, originally Chatbot Arena and branded Arena since January 2026, is a public leaderboard that ranks language models from crowdsourced human votes between pairs of anonymous model responses.

LMArena is a crowdsourced leaderboard that ranks [large language models](https://theaiengineering.com/glossary/large-language-model) by human preference. A user enters a prompt, two anonymous models answer, the user votes for the better response, and the model identities are revealed only after the vote. A statistical model turns the accumulated votes into a rating and a confidence interval for each model. The prompts come from real users rather than a fixed test set, so the leaderboard measures which answers people prefer on open-ended chat, including the effect of style and length, not correctness on a defined task.

The project began at UC Berkeley in 2023 as LMSYS Chatbot Arena. It moved to its own domain, lmarena.ai, in September 2024 and became known as LMArena, and in April 2025 the team started a company to support the platform. On January 28, 2026, LMArena [rebranded as Arena](https://arena.ai/blog/lmarena-is-now-arena/) at arena.ai, saying it had outgrown the "LM" in its name as it grew from a research project into a broader platform.

## How LMArena ranks models

LMArena fits a Bradley-Terry model to the pairwise votes. Each model gets a strength coefficient, and the gap between two coefficients predicts how often one model beats the other. A model's rank is one plus the number of models whose confidence interval sits entirely above its own, so close models can share a rank.

Chiang et al. describe the method in [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://arxiv.org/abs/2403.04132) (2024), written when the platform had collected over 240,000 votes. The authors report that crowdsourced votes agree well with expert raters, and they choose which model pairs to show with an active sampling rule that favors pairs whose comparison most shrinks the confidence intervals. The leaderboard first used online Elo ratings and switched to Bradley-Terry in December 2023. Bradley-Terry assumes each model's strength is fixed, so the order in which votes arrive no longer moves the ratings.

[Style control](https://lmsys.org/blog/2024-08-28-style-control/), added in August 2024, puts answer length and counts of markdown headers, bold text, and list items into the same regression as separate covariates. A model's style-controlled rating then reflects preference after those effects are accounted for; the authors found length to be the strongest style factor.

## LMArena vs static benchmarks

Static benchmarks such as [MMLU](https://theaiengineering.com/glossary/mmlu) or [SWE-bench](https://theaiengineering.com/glossary/swe-bench) score answers against fixed references, so they measure correctness on a defined task. LMArena measures preference on prompts users choose, which captures helpfulness, tone, and formatting but not whether an answer is right. The Chatbot Arena paper notes that static test sets can suffer [benchmark contamination](https://theaiengineering.com/glossary/benchmark-contamination), while crowdsourcing gathers fresh prompts.

Arena rankings have documented weaknesses. [The Leaderboard Illusion](https://arxiv.org/abs/2504.20879) (Singh et al., 2025) reported that undisclosed private testing let a few providers try many variants and publish only the best score, citing 27 private variants Meta tested before the Llama 4 release. The authors estimated that Google and OpenAI received 19.2% and 20.4% of arena data, while 83 open-weight models together received about 29.7%, and argued that these dynamics reward overfitting to the arena rather than general model quality.

Arena votes are also training and validation data. Zheng et al. used Chatbot Arena preferences to measure how often an [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge) agrees with humans, and RouteLLM trained its [LLM routers](https://theaiengineering.com/glossary/llm-routing) on 80,000 Chatbot Arena battles.

Treat an Arena rating as evidence of user preference on general chat, and confirm a shortlist with your own [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation) suite. The [LLM benchmarks explainer](https://theaiengineering.com/articles/llm-benchmarks) compares LMArena with correctness benchmarks.

## Related terms

- [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge): LLM-as-a-judge is an evaluation method in which a language model scores or compares the outputs of an LLM application against a rubric, a reference answer, or another response, replacing or supplementing human graders.
- [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation): LLM evaluation is the practice of measuring how well a language model or LLM-based application performs on defined tasks, using test datasets, automated graders, model-based judges, and human review.
- [Benchmark contamination](https://theaiengineering.com/glossary/benchmark-contamination): Benchmark contamination is the presence of a benchmark's test questions, answers, or solutions in a model's training data, so its score can reflect memorization instead of the capability the benchmark measures.
- [LLM routing](https://theaiengineering.com/glossary/llm-routing): LLM routing is the practice of directing each request to one of several language models or providers, based on rules or a learned classifier, to balance cost, quality, latency, and availability.
- [MMLU](https://theaiengineering.com/glossary/mmlu): MMLU (Massive Multitask Language Understanding) is a benchmark of 15,908 four-option multiple-choice questions across 57 subjects, from elementary mathematics to law, that measures a language model's broad academic knowledge.
- [SWE-bench](https://theaiengineering.com/glossary/swe-bench): SWE-bench is a benchmark of 2,294 real GitHub issues from 12 Python repositories, in which a model or agent must write a patch that resolves the issue and passes the repository's tests.

## Guides that cover lmarena

- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.

Part of the [LLM models and benchmarks](https://theaiengineering.com/topics/models) topic hub.

## Cite this definition

The AI Engineering Editors. "What is LMArena (Chatbot Arena)? How it ranks LLMs." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/lmarena

- Title: What is LMArena (Chatbot Arena)? How it ranks LLMs
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/lmarena
- Markdown: https://theaiengineering.com/glossary/lmarena.md
