What is retrieval-augmented generation (RAG)?
Also called RAG, retrieval augmented generation, retrieval-augmented LLM
Definition
Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
Retrieval-augmented generation (RAG) is a technique that retrieves relevant information from an external source and adds it to a language model's prompt before generation. The model then answers from the supplied passages as well as from what it learned in training. RAG lets an application use private, recent, or fast-changing data without retraining the model.
The term comes from Lewis et al. (2020), a NeurIPS paper that paired a pretrained sequence-to-sequence model (parametric memory) with a retriever over a dense vector index of Wikipedia (non-parametric memory).
How retrieval-augmented generation works#
Retrieval-augmented generation works in two phases: an offline indexing phase that prepares the knowledge source, and an online phase that retrieves passages and generates an answer for each query.
- Chunk: documents are split into passages by chunking.
- Embed and index: each chunk becomes an embedding stored in a vector database.
- Retrieve: the query is embedded and matched against the index, often combined with keyword results through hybrid search.
- Rerank: a reranking model reorders the candidates and keeps the best few.
- Generate: the top passages go into the prompt, usually with an instruction to cite them.
Retrieval quality sets the ceiling on answer quality. In Anthropic's contextual retrieval experiments, combining contextual embeddings, contextual BM25, and reranking cut the top-20 retrieval failure rate by 67% compared with a standard embedding baseline.
RAG vs fine-tuning#
RAG changes what the model sees at query time; fine-tuning changes the model's weights. RAG suits knowledge that changes often, must be cited, or must respect per-user access controls. Fine-tuning suits teaching a consistent format, tone, or narrow skill. Many production systems use both. RAG vs fine-tuning compares the two in depth, and What is RAG walks through a full implementation.
Related terms
Guides that cover retrieval-augmented generation (rag)
- What is retrieval-augmented generation (RAG)? How RAG works
- RAG vs fine-tuning: how to choose for your LLM application
- Chunking strategies for RAG: how to choose, tune, and evaluate them
- What is context engineering? A practical guide for AI agents
- LLM evaluation: a practical guide to evals, metrics, and judges
- What is AI engineering? The discipline, the stack, and the work
Cite this definition
The AI Engineering Editors. "What is retrieval-augmented generation (RAG)?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/retrieval-augmented-generation
- Title
- What is retrieval-augmented generation (RAG)?
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated