---
title: "What is contextual retrieval in RAG?"
description: "Contextual retrieval has an LLM write a short note placing each chunk in its document before indexing. How it works, its reported gains, and how it compares."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/contextual-retrieval"
markdown_url: "https://theaiengineering.com/glossary/contextual-retrieval.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Contextual retrieval","contextual embeddings","contextual chunk headers","contextual BM25"]
citation: "The AI Engineering Editors. \"What is contextual retrieval in RAG?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/contextual-retrieval"
---

# What is contextual retrieval in RAG?

Also called contextual embeddings, contextual chunk headers, contextual BM25.

> **Definition of Contextual retrieval:** Contextual retrieval is a RAG indexing technique in which a language model writes a short note situating each chunk within its source document, and the note is prepended before embedding and BM25 indexing.

Contextual retrieval is a technique for indexing RAG documents in which a language model writes a short, chunk-specific context for every chunk and prepends it to the chunk text before indexing. Anthropic introduced the method in a [September 2024 write-up](https://www.anthropic.com/news/contextual-retrieval). It targets a basic flaw of [chunking](https://theaiengineering.com/glossary/chunking), where a chunk cut out of its document loses the names, dates, and subjects that made it findable.

## How contextual retrieval works

Contextual retrieval works by sending each chunk, together with its full source document, to an LLM that writes a brief note explaining where the chunk fits. The note, usually 50 to 100 tokens, is prepended to the chunk, and the combined text is indexed twice: as an [embedding](https://theaiengineering.com/glossary/embeddings) and in a [BM25](https://theaiengineering.com/glossary/bm25) keyword index.

Anthropic's example is a chunk from a company filing that says only that revenue grew 3% over the previous quarter. On its own, the chunk names neither the company nor the period. The contextualized version first states that the chunk comes from an SEC filing on ACME Corp's Q2 2023 performance and gives the previous quarter's revenue, then repeats the original text. A query about ACME's second-quarter growth can now match it.

Anthropic calls the two halves contextual embeddings, which embed the contextualized chunk instead of the bare one, and contextual BM25, which builds the keyword index over the same contextualized text.

Anthropic measured the retrieval failure rate, defined as 1 minus [recall@20](https://theaiengineering.com/glossary/recall-at-k), across codebases, fiction, arXiv papers, and science papers. Against a 5.7% baseline, contextual embeddings cut failures to 3.7% (a 35% reduction), adding contextual BM25 cut them to 2.9% (49%), and adding a [reranking](https://theaiengineering.com/glossary/reranking) step that scored the top 150 chunks and kept 20 cut them to 1.9% (67%).

The cost is one LLM call per chunk at indexing time. Anthropic pairs the method with [prompt caching](https://theaiengineering.com/glossary/prompt-caching), which loads each document into the cache once and reuses it for every chunk drawn from it, so the model does not reprocess the full document per chunk. The same write-up notes that a knowledge base under about 200,000 tokens can go into the prompt whole, with no retrieval step at all.

## Contextual retrieval vs contextual chunk headers and late chunking

Contextual retrieval, contextual chunk headers, and late chunking all try to keep document context attached to each chunk. Contextual chunk headers prepend fixed metadata, such as the title and heading path, with no model call. Late chunking computes context inside the embedding model. Contextual retrieval asks an LLM to write context specific to each chunk.

| | Contextual chunk headers | Late chunking | Contextual retrieval |
|---|---|---|---|
| Where context comes from | Title and heading path | The embedding model reads the whole document | An LLM writes a note per chunk |
| Indexing cost | Negligible | One long embedding pass per document | One LLM call per chunk |
| Improves BM25 too | Yes, if the header is indexed | No | Yes |
| Model requirements | None | Token-level outputs from a long-context embedding model | Any LLM and any embedding model |

Anthropic reports that adding generic document summaries to chunks gave very limited gains, which is why the method asks for context specific to each chunk. [Late chunking](https://theaiengineering.com/glossary/late-chunking) reaches a similar goal without an LLM, and its authors ([Günther et al., 2024](https://arxiv.org/abs/2409.04701)) note that advantage when comparing against an LLM-based contextual approach. Headers cost almost nothing and stack with either method.

The [chunking strategies guide](https://theaiengineering.com/articles/chunking-strategies-for-rag) shows a heading-path header in code and places contextual retrieval among the other strategies.

## Related terms

- [Chunking](https://theaiengineering.com/glossary/chunking): Chunking is the step in a retrieval pipeline that splits documents into smaller passages, called chunks, so each one can be embedded, indexed, and retrieved on its own.
- [Late chunking](https://theaiengineering.com/glossary/late-chunking): Late chunking is an embedding technique that runs a long-context embedding model over a whole document first, then splits the token embeddings at chunk boundaries and averages each span into one chunk vector.
- [BM25](https://theaiengineering.com/glossary/bm25): BM25 is a keyword ranking function that scores a document against a query using how often each query term appears, how rare the term is across the corpus, and how long the document is.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.
- [Reranking](https://theaiengineering.com/glossary/reranking): Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.

## Guides that cover contextual retrieval

- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is contextual retrieval in RAG?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/contextual-retrieval

- Title: What is contextual retrieval in RAG?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/contextual-retrieval
- Markdown: https://theaiengineering.com/glossary/contextual-retrieval.md
