What is agentic RAG? How it differs from classic RAG

Also called agentic retrieval, agentic retrieval-augmented generation, RAG agent

Definition

Agentic RAG is a retrieval-augmented generation design in which an AI agent treats retrieval as a tool, deciding when to search, what query to send, which source to use, and whether to search again.

Agentic RAG is a form of retrieval-augmented generation in which an AI agent controls retrieval instead of a fixed pipeline. The agent decides whether a question needs retrieval at all, writes the search query, picks among sources such as a vector index, a SQL database, or web search, checks what came back, and searches again when the results fall short. The design helps on multi-step questions, at the cost of more model calls and less predictable latency. A 2025 survey by Singh et al. describes it as embedding autonomous agents into the RAG pipeline, using patterns such as reflection, planning, tool use, and multi-agent collaboration.

How agentic RAG works#

Agentic RAG works by exposing each retriever to the model as a tool and running the model in a loop. On each turn the model either answers or calls a search tool with a query it wrote. After reading the results, it answers, rewrites the query and searches again, or tries another source.

A question goes to an agent that either answers or calls a search tool; results are graded, and weak results trigger a rewritten query and another search. User questionAgentSearch or answerSearch toolIndex, SQL, or webGrade resultsRelevant enough?Rewrite queryAnswerWith citationsyesno
Figure 1: The agentic RAG loop. The agent decides when to search, and weak results send it back with a rewritten query.

The retrieval step uses ordinary tool calling: each search function has a name, a description, and a parameter schema, and the model chooses among them. LangGraph's agentic RAG tutorial builds this loop explicitly: a node lets the model respond directly or call a retriever tool, a grading step scores the retrieved documents for relevance, and irrelevant results route to a node that rewrites the question before the agent tries again. The loop follows the ReAct pattern of alternating reasoning steps with tool calls.

Some research moves the decision into the model itself. Self-RAG (Asai et al., 2023) trains a model to retrieve passages on demand and to critique both the passages and its own output with special reflection tokens, instead of retrieving a fixed number of passages for every input. The authors report that their 7B and 13B models outperformed ChatGPT and retrieval-augmented Llama 2 chat on open-domain question answering, reasoning, and fact verification tasks.

Agentic RAG vs a fixed RAG pipeline#

A fixed pipeline retrieves once, with the user's question or one rewrite of it, and always generates from whatever came back. Agentic RAG lets the model run several searches, choose sources, and stop when the evidence is sufficient. The fixed pipeline is cheaper and easier to test; the agentic version answers questions that one search cannot.

The deciding factor is whether the second search depends on the first. "What is our refund window?" needs one lookup, and a fixed pipeline answers it with one embedding call, one search, and one generation. "Which customers on the legacy plan reported the SSO bug fixed in the last release?" needs the release notes first, then a ticket search for that bug, then a filter by plan. No single query can express that chain, because each hop's query depends on what the previous hop returned.

Every extra search adds a model call and latency, loops can run away without a step limit, and the trace is harder to debug than a straight pipeline. Cap the number of tool calls, log every query the agent writes, and score answers for faithfulness to the retrieved passages. What is RAG compares classic and agentic RAG side by side.

Cite this definition

The AI Engineering Editors. "What is agentic RAG? How it differs from classic RAG." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agentic-rag

Title
What is agentic RAG? How it differs from classic RAG
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the RAG and vector search hub · Updated