---
title: "What is late chunking for embeddings in RAG?"
description: "Late chunking embeds a whole document with a long-context model, then pools token embeddings per chunk so each vector keeps document context. How it works."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/late-chunking"
markdown_url: "https://theaiengineering.com/glossary/late-chunking.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Late chunking","contextual chunk embeddings","late chunking embeddings"]
citation: "The AI Engineering Editors. \"What is late chunking for embeddings in RAG?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/late-chunking"
---

# What is late chunking for embeddings in RAG?

Also called contextual chunk embeddings, late chunking embeddings.

> **Definition of Late chunking:** Late chunking is an embedding technique that runs a long-context embedding model over a whole document first, then splits the token embeddings at chunk boundaries and averages each span into one chunk vector.

Late chunking is a way to embed document chunks so that each chunk vector reflects the whole document around it. Instead of splitting text first and embedding each piece alone, it runs a long-context embedding model over the full document, then applies the chunk boundaries to the resulting token embeddings and averages each span. Günther et al. at Jina AI introduced the method in [Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models](https://arxiv.org/abs/2409.04701) (2024). It matters for [chunking](https://theaiengineering.com/glossary/chunking) because a chunk that says "the new version fixes this bug" otherwise loses which product and which bug it means.

## How late chunking works

Late chunking works by moving the chunking step from before the embedding model to after it. The model reads the whole document, so every token embedding is computed with attention over the full text. Chunk boundaries are then mapped to token ranges, and each range is mean-pooled into one vector. The method needs no extra training.

**Figure 1: Naive chunking pools each chunk in isolation; late chunking pools spans of token embeddings computed over the whole document.**

Naive chunking splits text, then embeds each chunk alone; late chunking embeds the whole document, then splits token embeddings and pools each span.

Components:

- **Document**
- **Split into chunks**
- **Embed each chunk**: Sees one chunk
- **Chunk vectors**
- **Document**
- **Embed all tokens**: Full context
- **Split token vectors**: At chunk boundaries
- **Mean-pool spans**: One vector per chunk

Groups:

- Naive chunking: Document, Split into chunks, Embed each chunk, Chunk vectors
- Late chunking: Document, Embed all tokens, Split token vectors, Mean-pool spans

Connections:

- Document → Split into chunks
- Split into chunks → Embed each chunk
- Embed each chunk → Chunk vectors
- Document → Embed all tokens
- Embed all tokens → Split token vectors
- Split token vectors → Mean-pool spans

Any splitter can supply the boundaries: fixed token counts, sentences, or semantic breaks. For documents longer than the model's [context window](https://theaiengineering.com/glossary/context-window), the paper describes long late chunking, which splits the text into large macro chunks with some overlapping tokens for context and applies late chunking inside each. The authors also propose a fine-tuning method that trains the model with pooling restricted to spans, which improves results further.

## Late chunking vs naive chunking

Naive chunking embeds each chunk as a separate input, so the model never sees the surrounding text. Late chunking embeds the document once and derives every chunk vector from token embeddings that already carry document context. The chunks and the number of vectors stay the same; only the information inside each vector changes.

The paper tested three embedding models on four BEIR retrieval datasets and reports average nDCG@10 for both approaches:

| Chunk boundaries | Naive chunking | Late chunking |
|---|---|---|
| Fixed size, 256 tokens | 52.2 | 54.0 |
| Five sentences | 52.4 | 54.3 |
| Semantic sentence breaks | 52.4 | 53.8 |

The authors found the largest gains at small chunk sizes, where an isolated chunk has the least context of its own. Late chunking did not help on synthetic needle and passkey tasks, where the surrounding text is irrelevant by design, and some reading comprehension tasks favored naive chunking with large chunks. Adding overlap between chunks showed no clear advantage in their experiments.

Late chunking needs the model's token-level outputs, so an embedding API that returns only one pooled vector per input cannot support it on the client side. Some providers expose the idea directly: [Jina's embeddings API](https://api.jina.ai/redoc) has a late_chunking flag that concatenates the inputs and processes them as one sequence before splitting, and Perplexity's [pplx-embed-context-v1](https://theaiengineering.com/releases/2026/02#pplx-embed-v1) produces document-aware chunk embeddings. [Contextual retrieval](https://theaiengineering.com/glossary/contextual-retrieval) reaches a similar goal with an LLM call per chunk and works with any embedding model.

The [chunking strategies guide](https://theaiengineering.com/articles/chunking-strategies-for-rag) compares late chunking with the other ways to restore lost context.

## Related terms

- [Chunking](https://theaiengineering.com/glossary/chunking): Chunking is the step in a retrieval pipeline that splits documents into smaller passages, called chunks, so each one can be embedded, indexed, and retrieved on its own.
- [Contextual retrieval](https://theaiengineering.com/glossary/contextual-retrieval): Contextual retrieval is a RAG indexing technique in which a language model writes a short note situating each chunk within its source document, and the note is prepended before embedding and BM25 indexing.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Context window](https://theaiengineering.com/glossary/context-window): A context window is the maximum number of tokens a language model can take into account in a single request, including the system prompt, conversation history, tool results, and the generated output.
- [Recall@k](https://theaiengineering.com/glossary/recall-at-k): Recall@k is a retrieval metric that measures the fraction of a query's relevant documents that appear in the top k results, averaged over a set of test queries.

## Guides that cover late chunking

- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is late chunking for embeddings in RAG?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/late-chunking

- Title: What is late chunking for embeddings in RAG?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/late-chunking
- Markdown: https://theaiengineering.com/glossary/late-chunking.md
