---
title: "What is retrieval-augmented generation (RAG)?"
description: "Retrieval-augmented generation (RAG) gives an LLM relevant documents at query time so answers use current or private data. Learn how RAG works, step by step."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/retrieval-augmented-generation"
markdown_url: "https://theaiengineering.com/glossary/retrieval-augmented-generation.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Retrieval-augmented generation (RAG)","RAG","retrieval augmented generation","retrieval-augmented LLM"]
citation: "The AI Engineering Editors. \"What is retrieval-augmented generation (RAG)?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/retrieval-augmented-generation"
---

# What is retrieval-augmented generation (RAG)?

Also called RAG, retrieval augmented generation, retrieval-augmented LLM.

> **Definition of Retrieval-augmented generation (RAG):** Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.

Retrieval-augmented generation (RAG) is a technique that retrieves relevant information from an external source and adds it to a language model's prompt before generation. The model then answers from the supplied passages as well as from what it learned in training. RAG lets an application use private, recent, or fast-changing data without retraining the model.

The term comes from [Lewis et al. (2020)](https://arxiv.org/abs/2005.11401), a NeurIPS paper that paired a pretrained sequence-to-sequence model (parametric memory) with a retriever over a dense vector index of Wikipedia (non-parametric memory).

## How retrieval-augmented generation works

Retrieval-augmented generation works in two phases: an offline indexing phase that prepares the knowledge source, and an online phase that retrieves passages and generates an answer for each query.

**Figure 1: A typical RAG pipeline.**

Offline, documents are chunked and embedded into a vector index. Online, each question is matched against that index, reranked, and sent with the top passages to the LLM.

Components:

- **Chunk and embed**: Docs into embedded chunks
- **Vector index**: Vector database
- **Question**: From the user
- **Retrieve**: Embed query, search index Rerank, keep the best few
- **LLM**: Prompt plus top passages Generates the answer

Groups:

- Offline: index the documents: Chunk and embed, Vector index
- Online: answer each query: Question, Retrieve, LLM

Connections:

- Chunk and embed → Vector index
- Vector index → Retrieve
- Question → Retrieve
- Retrieve → LLM

1. **Chunk**: documents are split into passages by [chunking](https://theaiengineering.com/glossary/chunking).
2. **Embed and index**: each chunk becomes an [embedding](https://theaiengineering.com/glossary/embeddings) stored in a [vector database](https://theaiengineering.com/glossary/vector-database).
3. **Retrieve**: the query is embedded and matched against the index, often combined with keyword results through [hybrid search](https://theaiengineering.com/glossary/hybrid-search).
4. **Rerank**: a [reranking](https://theaiengineering.com/glossary/reranking) model reorders the candidates and keeps the best few.
5. **Generate**: the top passages go into the prompt, usually with an instruction to cite them.

Retrieval quality sets the ceiling on answer quality. In Anthropic's [contextual retrieval](https://www.anthropic.com/news/contextual-retrieval) experiments, combining contextual embeddings, contextual [BM25](https://theaiengineering.com/glossary/bm25), and reranking cut the top-20 retrieval failure rate by 67% compared with a standard embedding baseline.

## RAG vs fine-tuning

RAG changes what the model sees at query time; [fine-tuning](https://theaiengineering.com/glossary/fine-tuning) changes the model's weights. RAG suits knowledge that changes often, must be cited, or must respect per-user access controls. Fine-tuning suits teaching a consistent format, tone, or narrow skill. Many production systems use both. [RAG vs fine-tuning](https://theaiengineering.com/articles/rag-vs-fine-tuning) compares the two in depth, and [What is RAG](https://theaiengineering.com/articles/what-is-rag) walks through a full implementation.

## Related terms

- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Vector database](https://theaiengineering.com/glossary/vector-database): A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.
- [Chunking](https://theaiengineering.com/glossary/chunking): Chunking is the step in a retrieval pipeline that splits documents into smaller passages, called chunks, so each one can be embedded, indexed, and retrieved on its own.
- [Reranking](https://theaiengineering.com/glossary/reranking): Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.
- [Fine-tuning](https://theaiengineering.com/glossary/fine-tuning): Fine-tuning is the process of further training a pretrained model on a smaller, task-specific dataset so its weights change to improve performance on a particular task, domain, style, or output format.
- [Agentic RAG](https://theaiengineering.com/glossary/agentic-rag): Agentic RAG is a retrieval-augmented generation design in which an AI agent treats retrieval as a tool, deciding when to search, what query to send, which source to use, and whether to search again.
- [Faithfulness](https://theaiengineering.com/glossary/faithfulness): Faithfulness is an evaluation metric that measures what share of the claims in a generated answer can be inferred from the context the model was given, usually scored from 0 to 1.

## Guides that cover retrieval-augmented generation (rag)

- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.
- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.
- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [What is AI engineering? The discipline, the stack, and the work](https://theaiengineering.com/articles/what-is-ai-engineering): What is AI engineering? A plain definition, how it differs from ML and software engineering, the layers of the stack, and what AI engineers do day to day.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is retrieval-augmented generation (RAG)?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/retrieval-augmented-generation

- Title: What is retrieval-augmented generation (RAG)?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/retrieval-augmented-generation
- Markdown: https://theaiengineering.com/glossary/retrieval-augmented-generation.md
