What is contextual retrieval in RAG?

Also called contextual embeddings, contextual chunk headers, contextual BM25

Definition

Contextual retrieval is a RAG indexing technique in which a language model writes a short note situating each chunk within its source document, and the note is prepended before embedding and BM25 indexing.

Contextual retrieval is a technique for indexing RAG documents in which a language model writes a short, chunk-specific context for every chunk and prepends it to the chunk text before indexing. Anthropic introduced the method in a September 2024 write-up. It targets a basic flaw of chunking, where a chunk cut out of its document loses the names, dates, and subjects that made it findable.

How contextual retrieval works#

Contextual retrieval works by sending each chunk, together with its full source document, to an LLM that writes a brief note explaining where the chunk fits. The note, usually 50 to 100 tokens, is prepended to the chunk, and the combined text is indexed twice: as an embedding and in a BM25 keyword index.

Anthropic's example is a chunk from a company filing that says only that revenue grew 3% over the previous quarter. On its own, the chunk names neither the company nor the period. The contextualized version first states that the chunk comes from an SEC filing on ACME Corp's Q2 2023 performance and gives the previous quarter's revenue, then repeats the original text. A query about ACME's second-quarter growth can now match it.

Anthropic calls the two halves contextual embeddings, which embed the contextualized chunk instead of the bare one, and contextual BM25, which builds the keyword index over the same contextualized text.

Anthropic measured the retrieval failure rate, defined as 1 minus recall@20, across codebases, fiction, arXiv papers, and science papers. Against a 5.7% baseline, contextual embeddings cut failures to 3.7% (a 35% reduction), adding contextual BM25 cut them to 2.9% (49%), and adding a reranking step that scored the top 150 chunks and kept 20 cut them to 1.9% (67%).

The cost is one LLM call per chunk at indexing time. Anthropic pairs the method with prompt caching, which loads each document into the cache once and reuses it for every chunk drawn from it, so the model does not reprocess the full document per chunk. The same write-up notes that a knowledge base under about 200,000 tokens can go into the prompt whole, with no retrieval step at all.

Contextual retrieval vs contextual chunk headers and late chunking#

Contextual retrieval, contextual chunk headers, and late chunking all try to keep document context attached to each chunk. Contextual chunk headers prepend fixed metadata, such as the title and heading path, with no model call. Late chunking computes context inside the embedding model. Contextual retrieval asks an LLM to write context specific to each chunk.

Contextual chunk headers Late chunking Contextual retrieval
Where context comes from Title and heading path The embedding model reads the whole document An LLM writes a note per chunk
Indexing cost Negligible One long embedding pass per document One LLM call per chunk
Improves BM25 too Yes, if the header is indexed No Yes
Model requirements None Token-level outputs from a long-context embedding model Any LLM and any embedding model

Anthropic reports that adding generic document summaries to chunks gave very limited gains, which is why the method asks for context specific to each chunk. Late chunking reaches a similar goal without an LLM, and its authors (Günther et al., 2024) note that advantage when comparing against an LLM-based contextual approach. Headers cost almost nothing and stack with either method.

The chunking strategies guide shows a heading-path header in code and places contextual retrieval among the other strategies.

Cite this definition

The AI Engineering Editors. "What is contextual retrieval in RAG?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/contextual-retrieval

Title
What is contextual retrieval in RAG?
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the RAG and vector search hub · Updated