---
title: "What is reranking in RAG and vector search?"
description: "Reranking reorders retrieved documents with a more precise model before they reach the LLM. Learn how rerankers work in RAG and how they differ from embeddings."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/reranking"
markdown_url: "https://theaiengineering.com/glossary/reranking.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Reranking","re-ranking","reranker","two-stage retrieval"]
citation: "The AI Engineering Editors. \"What is reranking in RAG and vector search?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/reranking"
---

# What is reranking in RAG and vector search?

Also called re-ranking, reranker, two-stage retrieval.

> **Definition of Reranking:** Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.

Reranking is a second stage in a retrieval pipeline that takes the candidates returned by a fast first-stage search and reorders them with a slower, more accurate model. The first stage, such as [vector search](https://theaiengineering.com/glossary/vector-database) or [hybrid search](https://theaiengineering.com/glossary/hybrid-search), returns a broad candidate list, for example the top 100 results. The reranker scores each candidate against the query and passes only the top few to the language model.

## How reranking works

Reranking works by scoring each query-document pair jointly instead of comparing two precomputed vectors. Most rerankers are cross-encoders: the query and the document go into one transformer together, so attention can compare every query token with every document token, and the model outputs a single relevance score. The [Sentence Transformers documentation](https://sbert.net/examples/cross_encoder/applications/README.html) recommends this two-stage pattern: retrieve candidates with a bi-encoder, then re-rank them with a cross-encoder.

Other approaches include late-interaction models such as ColBERT, which compare stored per-token vectors, and LLM-based rerankers that prompt a language model to judge relevance. In Anthropic's [contextual retrieval](https://www.anthropic.com/news/contextual-retrieval) tests, adding a reranking step that scored the top 150 chunks and kept 20 raised the reduction in top-20 retrieval failures from 49% to 67%.

## Reranker vs embedding model

A reranker and an [embedding model](https://theaiengineering.com/glossary/embeddings) trade speed for precision in opposite directions.

| | Embedding model (bi-encoder) | Reranker (cross-encoder) |
|---|---|---|
| Input | Query or document, encoded alone | Query and document, encoded together |
| Output | A vector, computed ahead of time | A relevance score for one pair |
| Work per query | One embedding plus an index lookup | One model pass per candidate |
| Practical scale | Millions of documents | Tens to hundreds of candidates |
| Strength | Recall | Precision |

A cross-encoder produces no reusable vector, so its scores cannot be indexed; it only works on a short list that another retriever has already produced. That is why reranking is a second stage rather than a replacement for embedding search.

Reranking adds latency to every query, so measure whether the precision gain is worth it on an evaluation set drawn from real [RAG](https://theaiengineering.com/glossary/retrieval-augmented-generation) traffic. [What is RAG](https://theaiengineering.com/articles/what-is-rag) shows where reranking fits in the pipeline.

## Related terms

- [Hybrid search](https://theaiengineering.com/glossary/hybrid-search): Hybrid search is a retrieval method that runs keyword search, typically BM25, and vector similarity search on the same query and merges the two ranked lists into one result set.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Retrieval-augmented generation (RAG)](https://theaiengineering.com/glossary/retrieval-augmented-generation): Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
- [Vector database](https://theaiengineering.com/glossary/vector-database): A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.
- [Chunking](https://theaiengineering.com/glossary/chunking): Chunking is the step in a retrieval pipeline that splits documents into smaller passages, called chunks, so each one can be embedded, indexed, and retrieved on its own.
- [Recall@k](https://theaiengineering.com/glossary/recall-at-k): Recall@k is a retrieval metric that measures the fraction of a query's relevant documents that appear in the top k results, averaged over a set of test queries.
- [BM25](https://theaiengineering.com/glossary/bm25): BM25 is a keyword ranking function that scores a document against a query using how often each query term appears, how rare the term is across the corpus, and how long the document is.

## Guides that cover reranking

- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.
- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is reranking in RAG and vector search?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/reranking

- Title: What is reranking in RAG and vector search?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/reranking
- Markdown: https://theaiengineering.com/glossary/reranking.md
