---
title: "What is chunking in RAG? Strategies explained"
description: "Chunking splits documents into smaller passages before they are embedded for retrieval. Learn how chunking works in RAG and which chunking strategies to use."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/chunking"
markdown_url: "https://theaiengineering.com/glossary/chunking.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Chunking","document chunking","text chunking","chunking strategies"]
citation: "The AI Engineering Editors. \"What is chunking in RAG? Strategies explained.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/chunking"
---

# What is chunking in RAG? Strategies explained

Also called document chunking, text chunking, chunking strategies.

> **Definition of Chunking:** Chunking is the step in a retrieval pipeline that splits documents into smaller passages, called chunks, so each one can be embedded, indexed, and retrieved on its own.

Chunking is the step in a [retrieval-augmented generation](https://theaiengineering.com/glossary/retrieval-augmented-generation) pipeline that splits source documents into smaller passages before indexing. Each chunk becomes one unit of retrieval: it gets its own [embedding](https://theaiengineering.com/glossary/embeddings), its own entry in the index, and competes on its own for a place in the prompt. Chunk boundaries therefore determine which pieces of information can be retrieved together.

## How chunking works

Chunking works by applying a splitting rule to each document and, optionally, attaching metadata such as the document title, section heading, or page number to every chunk. The main strategies:

| Strategy | How it splits | Trade-off |
|---|---|---|
| Fixed-size | Every N tokens, usually with overlap | Simple, but can cut sentences and ideas in half |
| Recursive | On paragraphs, then sentences, then words, until under a size limit | Keeps natural boundaries; a common default |
| Structure-aware | On headings, sections, code blocks, or table rows | Follows the document's layout; needs a parser per format |
| Semantic | Where embedding similarity between adjacent sentences drops | Groups related sentences; costs an extra embedding pass |

Overlap, which repeats a few sentences between neighboring chunks, lowers the chance that an answer spans a boundary, at the cost of a larger index.

## Why chunking matters

Chunking matters because a retriever can only return what a chunk contains. Chunks that are too small lose context: a sentence such as "coverage begins after 90 days" is useless without the name of the policy it belongs to. Chunks that are too large dilute the embedding with unrelated text and consume more of the [context window](https://theaiengineering.com/glossary/context-window) per retrieved result.

Anthropic's [contextual retrieval](https://www.anthropic.com/news/contextual-retrieval) method targets the lost-context problem. A model writes a short, chunk-specific note (typically 50 to 100 tokens) that situates the chunk within the whole document, and the note is prepended before embedding. In Anthropic's tests, this cut top-20 retrieval failures by 35% with contextual embeddings alone, and by 49% when combined with contextual [BM25](https://theaiengineering.com/glossary/bm25).

A practical approach is to start with a recursive splitter, build a small set of test questions with known answer passages, and measure retrieval recall while changing chunk size and overlap. [Reranking](https://theaiengineering.com/glossary/reranking) can recover precision when chunks are small, and [What is RAG](https://theaiengineering.com/articles/what-is-rag) walks through the full pipeline.

## Related terms

- [Retrieval-augmented generation (RAG)](https://theaiengineering.com/glossary/retrieval-augmented-generation): Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Vector database](https://theaiengineering.com/glossary/vector-database): A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.
- [Reranking](https://theaiengineering.com/glossary/reranking): Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.
- [Context window](https://theaiengineering.com/glossary/context-window): A context window is the maximum number of tokens a language model can take into account in a single request, including the system prompt, conversation history, tool results, and the generated output.
- [Late chunking](https://theaiengineering.com/glossary/late-chunking): Late chunking is an embedding technique that runs a long-context embedding model over a whole document first, then splits the token embeddings at chunk boundaries and averages each span into one chunk vector.
- [Contextual retrieval](https://theaiengineering.com/glossary/contextual-retrieval): Contextual retrieval is a RAG indexing technique in which a language model writes a short note situating each chunk within its source document, and the note is prepended before embedding and BM25 indexing.

## Guides that cover chunking

- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is chunking in RAG? Strategies explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/chunking

- Title: What is chunking in RAG? Strategies explained
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/chunking
- Markdown: https://theaiengineering.com/glossary/chunking.md
