What is faithfulness in RAG and LLM evaluation?
Also called groundedness, answer faithfulness, faithfulness metric
Definition
Faithfulness is an evaluation metric that measures what share of the claims in a generated answer can be inferred from the context the model was given, usually scored from 0 to 1.
Faithfulness is a measure of whether a generated answer sticks to the context the model was given. An answer is faithful when every claim in it can be inferred from the retrieved passages, documents, or tool results in the prompt. Evaluation frameworks for retrieval-augmented generation score it alongside retrieval metrics, because it separates a model that invents details from a retriever that fetched the wrong text. Some tools call the same idea groundedness.
How faithfulness is scored#
Faithfulness is scored by splitting an answer into individual claims and checking each claim against the retrieved context. The score is the number of supported claims divided by the total number of claims, from 0 to 1. A score of 1 means every claim is backed by the context; 0.5 means half the claims have no support there.
The Ragas paper (Es et al., 2023) defined the metric for RAG pipelines in two steps. First, an LLM breaks the answer into short statements. Second, an LLM decides for each statement whether the context supports it. Faithfulness is the supported share. The authors tie the metric to avoiding hallucinations and to making the retrieved context a justification for the answer.
The Ragas documentation gives a worked example. Asked where and when Einstein was born, with context that gives his birth date as March 14, 1879, an answer stating he was born in Germany on March 20, 1879, contains two claims. The birthplace is supported and the date is not, so faithfulness is 0.5. The same page offers HHEM-2.1-Open, a small open classifier from Vectara, as an alternative to an LLM for the claim check.
Because faithfulness needs only the question, the context, and the answer, it requires no reference answer. That makes it usable on production traffic, where an LLM-as-a-judge can score a sample of live traces as online evaluation. TruLens includes the same check, under the name groundedness, in its RAG triad with context relevance and answer relevance.
Faithfulness vs correctness and hallucination#
Faithfulness compares an answer with the retrieved context, correctness compares it with the truth or a reference answer, and hallucination names the failure. The two metrics can disagree. An answer can be faithful and wrong when retrieval returned an outdated document, and correct but unfaithful when the model relied on its own knowledge.
| Faithfulness | Correctness | |
|---|---|---|
| Compared with | Retrieved context | Reference answer or ground truth |
| Needs labeled data | No | Yes |
| Catches | Invented or unsupported claims | Wrong answers from any cause |
| Misses | Faithful answers built on bad sources | Claims the sources cannot back up |
A low faithfulness score points to the prompt or the model, which added claims the context does not contain. High faithfulness with low correctness points to retrieval: the model repeated what it was given, and what it was given was wrong or incomplete. Pairing faithfulness with a retrieval metric such as recall@k shows which stage failed.
An unsupported claim in a RAG answer is one kind of hallucination, but faithfulness does not detect every kind. It cannot flag a false statement that the retrieved documents also contain. The LLM evaluation guide covers how to calibrate faithfulness judges and gate releases on the score.
Related terms
Guides that cover faithfulness
Cite this definition
The AI Engineering Editors. "What is faithfulness in RAG and LLM evaluation?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/faithfulness
- Title
- What is faithfulness in RAG and LLM evaluation?
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated