---
title: "What is BM25? Keyword ranking for search and RAG"
description: "BM25 is the keyword ranking function behind most search engines and the lexical half of hybrid search. Learn how BM25 scores documents and how it compares."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/bm25"
markdown_url: "https://theaiengineering.com/glossary/bm25.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["BM25","Okapi BM25","keyword search","BM25 ranking"]
citation: "The AI Engineering Editors. \"What is BM25? Keyword ranking for search and RAG.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/bm25"
---

# What is BM25? Keyword ranking for search and RAG

Also called Okapi BM25, keyword search, BM25 ranking.

> **Definition of BM25:** BM25 is a keyword ranking function that scores a document against a query using how often each query term appears, how rare the term is across the corpus, and how long the document is.

BM25 is a ranking function that scores how well a document matches a keyword query. It combines three signals: how often each query term appears in the document, how rare that term is across the whole corpus, and how long the document is compared with the average. BM25 needs no training and no model, only an inverted index of terms, and it is the default relevance ranking in Elasticsearch. In RAG systems it is the keyword half of [hybrid search](https://theaiengineering.com/glossary/hybrid-search), the retriever that catches exact strings that [embeddings](https://theaiengineering.com/glossary/embeddings) blur.

## How BM25 scores a document

BM25 scores a document by adding up one weight per query term. Each weight multiplies the term's rarity in the corpus (its inverse document frequency) by a term frequency component that saturates as the term repeats and is normalized for document length. Two parameters, k1 and b, control the saturation and the length normalization.

Robertson and Zaragoza's review, [The Probabilistic Relevance Framework: BM25 and Beyond](https://doi.org/10.1561/1500000019) (2009), derives the formula. Per query term, the classic form is:

weight = IDF × tf / (k1 × ((1 − b) + b × dl / avgdl) + tf)

Here tf is the term's count in the document, dl is the document's length, and avgdl is the average length across the corpus.

- **Term frequency saturation.** The first mention of "refund" in a document is strong evidence of relevance; the tenth adds little. The tf / (k1 + tf) shape rises quickly and then flattens, so no single term's contribution can pass a ceiling however often it repeats. A larger k1 delays the flattening.
- **Length normalization.** A long document mentions more terms by chance, so BM25 scales term frequency by the document's length relative to the average. Setting b to 1 applies full length normalization, and setting b to 0 switches it off.

The [Elasticsearch similarity documentation](https://www.elastic.co/docs/reference/elasticsearch/index-settings/similarity) lists BM25 as the default similarity, with k1 = 1.2 and b = 0.75. Lucene's BM25Similarity uses the same defaults.

## BM25 vs vector search

BM25 matches the exact tokens in a query, while vector search matches meaning through embeddings. BM25 finds rare identifiers, product codes, and names that embeddings can miss, but it cannot connect a query to a document that uses different words for the same idea. Each finds documents the other misses, which is why production RAG pipelines often run both.

| | BM25 | Vector search |
|---|---|---|
| Matches | Exact terms after tokenization | Similar meaning |
| Synonyms and paraphrases | Missed unless the words overlap | Handled |
| Rare identifiers and codes | Strong | Often weak |
| Model required | None | An embedding model |
| Index | Inverted index of terms | [Approximate nearest neighbor](https://theaiengineering.com/glossary/approximate-nearest-neighbor) index |
| Score | Not normalized; depends on the corpus | Usually cosine similarity |

BM25 also holds up on unfamiliar domains. The [BEIR benchmark](https://arxiv.org/abs/2104.08663) (Thakur et al., 2021) evaluated retrievers zero-shot across 18 datasets and found BM25 a robust baseline, with reranking and late-interaction models scoring best on average at a much higher computational cost. In Anthropic's [contextual retrieval](https://theaiengineering.com/glossary/contextual-retrieval) experiments, [adding a BM25 index](https://www.anthropic.com/news/contextual-retrieval) to contextual embeddings raised the cut in top-20 retrieval failures from 35% to 49%, and the write-up notes that BM25 is especially effective on queries with unique identifiers or technical terms.

The two scores sit on incompatible scales, so hybrid systems merge the ranked lists by position and often pass the result to a [reranker](https://theaiengineering.com/glossary/reranking). [What is RAG](https://theaiengineering.com/articles/what-is-rag) shows where BM25 sits alongside vector retrieval in a full pipeline.

## Related terms

- [Hybrid search](https://theaiengineering.com/glossary/hybrid-search): Hybrid search is a retrieval method that runs keyword search, typically BM25, and vector similarity search on the same query and merges the two ranked lists into one result set.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Vector database](https://theaiengineering.com/glossary/vector-database): A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.
- [Contextual retrieval](https://theaiengineering.com/glossary/contextual-retrieval): Contextual retrieval is a RAG indexing technique in which a language model writes a short note situating each chunk within its source document, and the note is prepended before embedding and BM25 indexing.
- [Reranking](https://theaiengineering.com/glossary/reranking): Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.
- [Approximate nearest neighbor search](https://theaiengineering.com/glossary/approximate-nearest-neighbor): Approximate nearest neighbor (ANN) search is a family of indexing methods that find vectors close to a query vector without comparing it to every stored vector, accepting that some true nearest neighbors are missed.

## Guides that cover bm25

- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is BM25? Keyword ranking for search and RAG." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/bm25

- Title: What is BM25? Keyword ranking for search and RAG
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/bm25
- Markdown: https://theaiengineering.com/glossary/bm25.md
