---
title: "What is agentic RAG? How it differs from classic RAG"
description: "Agentic RAG lets an AI agent decide when and how to retrieve, rewriting queries and searching again across sources. How it works and how it differs from RAG."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/agentic-rag"
markdown_url: "https://theaiengineering.com/glossary/agentic-rag.md"
topic: "RAG and vector search"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Agentic RAG","agentic retrieval","agentic retrieval-augmented generation","RAG agent"]
citation: "The AI Engineering Editors. \"What is agentic RAG? How it differs from classic RAG.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agentic-rag"
---

# What is agentic RAG? How it differs from classic RAG

Also called agentic retrieval, agentic retrieval-augmented generation, RAG agent.

> **Definition of Agentic RAG:** Agentic RAG is a retrieval-augmented generation design in which an AI agent treats retrieval as a tool, deciding when to search, what query to send, which source to use, and whether to search again.

Agentic RAG is a form of [retrieval-augmented generation](https://theaiengineering.com/glossary/retrieval-augmented-generation) in which an [AI agent](https://theaiengineering.com/glossary/ai-agent) controls retrieval instead of a fixed pipeline. The agent decides whether a question needs retrieval at all, writes the search query, picks among sources such as a vector index, a SQL database, or web search, checks what came back, and searches again when the results fall short. The design helps on multi-step questions, at the cost of more model calls and less predictable latency. A [2025 survey by Singh et al.](https://arxiv.org/abs/2501.09136) describes it as embedding autonomous agents into the RAG pipeline, using patterns such as reflection, planning, tool use, and multi-agent collaboration.

## How agentic RAG works

Agentic RAG works by exposing each retriever to the model as a tool and running the model in a loop. On each turn the model either answers or calls a search tool with a query it wrote. After reading the results, it answers, rewrites the query and searches again, or tries another source.

**Figure 1: The agentic RAG loop. The agent decides when to search, and weak results send it back with a rewritten query.**

A question goes to an agent that either answers or calls a search tool; results are graded, and weak results trigger a rewritten query and another search.

Components:

- **User question**
- **Agent**: Search or answer
- **Search tool**: Index, SQL, or web
- **Grade results**: Relevant enough?
- **Rewrite query**
- **Answer**: With citations

Connections:

- User question → Agent
- Agent → Search tool
- Search tool → Grade results
- Grade results → Answer: yes
- Grade results → Rewrite query: no
- Rewrite query → Agent

The retrieval step uses ordinary [tool calling](https://theaiengineering.com/glossary/tool-calling): each search function has a name, a description, and a parameter schema, and the model chooses among them. LangGraph's [agentic RAG tutorial](https://docs.langchain.com/oss/python/langgraph/agentic-rag) builds this loop explicitly: a node lets the model respond directly or call a retriever tool, a grading step scores the retrieved documents for relevance, and irrelevant results route to a node that rewrites the question before the agent tries again. The loop follows the [ReAct](https://theaiengineering.com/glossary/react-agent) pattern of alternating reasoning steps with tool calls.

Some research moves the decision into the model itself. [Self-RAG](https://arxiv.org/abs/2310.11511) (Asai et al., 2023) trains a model to retrieve passages on demand and to critique both the passages and its own output with special reflection tokens, instead of retrieving a fixed number of passages for every input. The authors report that their 7B and 13B models outperformed ChatGPT and retrieval-augmented Llama 2 chat on open-domain question answering, reasoning, and fact verification tasks.

## Agentic RAG vs a fixed RAG pipeline

A fixed pipeline retrieves once, with the user's question or one rewrite of it, and always generates from whatever came back. Agentic RAG lets the model run several searches, choose sources, and stop when the evidence is sufficient. The fixed pipeline is cheaper and easier to test; the agentic version answers questions that one search cannot.

The deciding factor is whether the second search depends on the first. "What is our refund window?" needs one lookup, and a fixed pipeline answers it with one embedding call, one search, and one generation. "Which customers on the legacy plan reported the SSO bug fixed in the last release?" needs the release notes first, then a ticket search for that bug, then a filter by plan. No single query can express that chain, because each hop's query depends on what the previous hop returned.

Every extra search adds a model call and latency, loops can run away without a step limit, and the trace is harder to debug than a straight pipeline. Cap the number of tool calls, log every query the agent writes, and score answers for [faithfulness](https://theaiengineering.com/glossary/faithfulness) to the retrieved passages. [What is RAG](https://theaiengineering.com/articles/what-is-rag) compares classic and agentic RAG side by side.

## Related terms

- [Retrieval-augmented generation (RAG)](https://theaiengineering.com/glossary/retrieval-augmented-generation): Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
- [AI agent](https://theaiengineering.com/glossary/ai-agent): An AI agent is a software system in which a large language model runs in a loop, calling tools, reading their results, and repeating until the task is complete or a stop condition is reached.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [ReAct agent](https://theaiengineering.com/glossary/react-agent): A ReAct agent is an AI agent that alternates a written reasoning step, an action such as a tool call, and an observation of the result, repeating the cycle until it can answer.
- [Faithfulness](https://theaiengineering.com/glossary/faithfulness): Faithfulness is an evaluation metric that measures what share of the claims in a generated answer can be inferred from the context the model was given, usually scored from 0 to 1.
- [Agentic AI](https://theaiengineering.com/glossary/agentic-ai): Agentic AI is the class of AI systems that pursue a goal through multiple steps, using a language model to plan, call tools, and act on the results with limited human input.

## Guides that cover agentic rag

- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.

Part of the [RAG and vector search](https://theaiengineering.com/topics/rag) topic hub.

## Cite this definition

The AI Engineering Editors. "What is agentic RAG? How it differs from classic RAG." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agentic-rag

- Title: What is agentic RAG? How it differs from classic RAG
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/agentic-rag
- Markdown: https://theaiengineering.com/glossary/agentic-rag.md
