---
title: "AI agent evaluation: metrics, graders, and how to test agents"
description: "AI agent evaluation guide: measure task success, tool use, safety, cost, and pass^k reliability with code graders, LLM judges, eval sets from traces, and CI."
summary: "AI agent evaluation measures whether an agent finishes multi-step tasks correctly, safely, and consistently, by grading the end state it leaves and the tool calls it made. This guide covers agent metrics, pass@k versus pass^k, graders, eval sets from traces, CI gates, and public benchmarks."
type: "guide"
canonical_url: "https://theaiengineering.com/articles/ai-agent-evaluation"
markdown_url: "https://theaiengineering.com/articles/ai-agent-evaluation.md"
topic: "LLM evaluation and observability"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-10-31"
updated: "2026-10-31"
keywords: ["ai agent evaluation","how to evaluate ai agents","ai agent evaluation framework","ai agent evaluation metrics","ai agent evaluation tools","how to test ai agents","how to measure ai agent performance","agent evals"]
citation: "The AI Engineering Editors. \"AI agent evaluation: metrics, graders, and how to test agents.\" The AI Engineering, updated Oct 31, 2026. https://theaiengineering.com/articles/ai-agent-evaluation"
---

# AI agent evaluation: metrics, graders, and how to test agents

## TL;DR

- AI agent evaluation grades a multi-step run rather than a single response: the end state the agent leaves in its environment, the trajectory of tool calls it took, and the cost and time it spent.
- Outcome checks against the final environment state are the most reliable agent graders, but they miss policy violations such as a refund issued without the user's confirmation, so pair them with trajectory checks.
- pass@k is the chance that at least one of k trials succeeds, while pass^k, introduced with τ-bench in 2024, is the chance that all k trials succeed, which is the measure that matters for customer-facing agents.
- The best agent eval tasks come from production traces and known failures, each with a seeded initial state, a clean sandbox per trial, and success criteria that two domain experts would agree on.
- Public agent benchmarks such as SWE-bench Verified, τ-bench, GAIA, WebArena, OSWorld, and Terminal-Bench score a model plus its scaffold in their own environments, not your agent on your tasks.

AI agent evaluation is the practice of measuring whether an [AI agent](https://theaiengineering.com/glossary/ai-agent) completes its tasks correctly, safely, and efficiently, by grading the state it leaves behind, the tool calls it made along the way, and how consistently it succeeds across repeated runs. It extends [LLM evaluation](https://theaiengineering.com/articles/llm-evaluation) from one model response to a multi-step run that reads data, writes data, and spends money. This guide covers metrics, reliability across runs, graders, eval sets, CI gates, and public agent benchmarks, and belongs to our [LLM evaluation and observability](https://theaiengineering.com/topics/llm-evaluation) hub.

## What is AI agent evaluation?

AI agent evaluation is the repeatable testing of an agent system on tasks with defined success criteria, scored by graders that inspect the final environment state and the transcript. Because agent runs vary, each task runs several times, and the scores describe the model, prompts, tools, and orchestration code together, never the model alone.

Anthropic's engineering guide, [Demystifying evals for AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents), defines the core vocabulary:

- **Task:** one test with defined inputs and success criteria.
- **Trial:** one attempt at a task; several run per task.
- **Transcript** (also trace or trajectory): the full record of a trial, including messages, tool calls, and results.
- **Outcome:** the final state of the environment, such as whether a reservation actually exists.
- **Grader:** logic that scores one aspect of a trial.
- **Scaffold:** the code that turns a model into an agent by orchestrating its tool calls.

Because the scaffold is part of the system under test, a prompt edit, a new tool, and a model swap each need a fresh eval run. The parts are those of any [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation), plus an environment. Our [AI agents explainer](https://theaiengineering.com/articles/what-are-ai-agents) covers the loop the scaffold drives.

**Figure 1: One trial of an agent eval. Graders read two artifacts, the transcript and the end state, and repeated trials give the reliability metrics.**

One agent eval trial: a task drives the agent, which calls tools in a sandbox; trajectory graders read the transcript, outcome graders read the end state, and k trials give pass@k and pass^k.

Components:

- **Task**: Instruction Success criteria
- **Agent under test**: Model, prompts, tools
- **Sandbox environment**: Database, files, APIs Seeded per trial
- **Transcript**: Messages, tool calls Arguments, results
- **End state**: Records, files, tests
- **Trajectory graders**: Tools, args, policy
- **Outcome graders**: State diff, unit tests
- **Trial score**: Pass or fail, cost, turns
- **Across k trials**: pass@k and pass^k

Groups:

- One trial, in a clean sandbox: Task, Agent under test, Sandbox environment, Transcript, End state
- Grading: Trajectory graders, Outcome graders, Trial score, Across k trials

Connections:

- Task → Agent under test
- Agent under test → Sandbox environment
- Agent under test → Transcript
- Sandbox environment → End state
- Transcript → Trajectory graders
- End state → Outcome graders
- Trajectory graders → Trial score
- Outcome graders → Trial score
- Trial score → Across k trials

## How agent evaluation differs from evaluating an LLM call

Evaluating a single LLM call scores one output against one input. Evaluating an agent scores a run of many model calls and tool calls that changes external state, can fail at any step, takes a different path on each trial, and varies in cost. The grader needs access to the environment, not only to the agent's final reply.

| | Single LLM call | Agent run |
|---|---|---|
| Unit graded | One response | A trajectory of model turns and tool calls |
| What correct means | Output matches a reference or rubric | Environment ends in the goal state, by any valid path |
| Side effects | None | Writes to databases, files, tickets, or payments |
| Where it fails | The response | Any step: plan, tool choice, arguments, error handling, stopping |
| Run-to-run variation | Sampling noise in one output | Noise compounds across steps and changes the path |

Errors compound: if each of 10 steps were independent and 95% reliable, the run would succeed about 60% of the time. The final message is not evidence: an agent can report success after a failed tool call, a form of [hallucination](https://theaiengineering.com/glossary/hallucination), so the grader checks the database. And many valid paths exist: the [WebArena paper](https://arxiv.org/abs/2307.13854) criticizes comparing an agent's actions with a reference action sequence, because that ignores functional correctness and alternative solutions.

## What to measure: AI agent evaluation metrics

AI agent evaluation metrics fall into five groups: outcome (did the task succeed), trajectory (right tools, right arguments, no wasted steps), safety and policy (did the agent stay within its rules), efficiency (cost, latency, and turns), and reliability (how often it succeeds across repeated trials). Track all five, and gate releases on outcome and safety first.

| Group | Metric | How to measure it | Typical grader |
|---|---|---|---|
| Outcome | Task success rate | End state matches the goal state, or tests pass | Code |
| Trajectory | Tool selection | Required tools called, no unexpected tools | Code |
| Trajectory | Argument correctness | Arguments fit the schema and match expected values | Code |
| Trajectory | Redundant steps | Repeated calls, loops, steps beyond a reference path | Code or LLM judge |
| Safety | Policy violations | Forbidden tools, writes without confirmation, data leaks | Code plus LLM judge |
| Safety | Injection resistance | Tasks whose tool results contain planted instructions | Code on end state |
| Efficiency | Cost, latency, turns | Token cost, wall-clock time, and model turns per task | Trace data |
| Reliability | pass@k and pass^k | Success across k trials of the same task | Computed from trials |

### Outcome and task success

Measure task success on the environment: query the database, run the tests, or inspect the files written. The [τ-bench paper](https://arxiv.org/abs/2406.12045) (tau-bench) compares the final database with an annotated goal state and checks that the agent's replies contained the information the user needed. For multi-part tasks, Anthropic's guide recommends partial credit.

### Trajectory and tool-use quality

Trajectory metrics explain why an outcome failed and catch runs that succeeded wastefully. LangChain's MIT-licensed agentevals package names four ways to compare a trajectory with a reference: strict (same tool calls, same order), unordered (same calls, any order), subset (no tools beyond the expected ones), and superset (expected calls appear, extras allowed). Strict matching is brittle because agents find valid paths nobody anticipated, so check writes and critical arguments exactly and let reads vary.

### Safety and policy compliance

An outcome check can pass while the agent broke a rule. The τ-bench authors note that a successful reward is necessary but not sufficient, because an agent might issue a return without the explicit user confirmation the policy requires. Check that forbidden tools were never called and that writes followed confirmation. Include tasks where the agent should refuse or ask, and tool results that carry prompt injection attempts. Runtime guardrails block some violations; evals measure how often the agent attempts them.

### Cost, latency, and turn count

Measure efficiency per completed task, next to success rate: succeeding slightly more often at three times the cost is a product decision. Runs that hit the step limit count as failures. The data comes from traces, covered in our [LLM observability guide](https://theaiengineering.com/articles/llm-observability).

## pass@k vs pass^k: measuring reliability across runs

[pass@k](https://theaiengineering.com/glossary/pass-at-k) is the probability that at least one of k independent trials of a task succeeds; pass^k is the probability that all k trials succeed. pass@k rises with k and fits tasks where a person or verifier can pick the one good attempt. pass^k falls with k and fits agents that must handle the same request correctly every time.

In code generation, Kulal et al. (2019) counted a problem as solved if any of k samples passed its unit tests, and Chen et al. (2021), in the paper that introduced HumanEval, gave the unbiased estimator still in use: generate n ≥ k samples, count the c that pass, and average `1 - C(n-c, k) / C(n, k)` over problems. The τ-bench paper proposed pass^k (read "pass hat k") for agent tasks that need consistency, defining it as the chance that all k i.i.d. trials succeed, averaged across tasks, with the estimator `C(c, k) / C(n, k)`. At k = 1 both equal the average success rate.

```python
from math import comb

def pass_at_k(n, c, k):
    """P(at least one of k trials passes), given c of n trials passed."""
    return 1.0 - comb(n - c, k) / comb(n, k)

def pass_hat_k(n, c, k):
    """P(all k trials pass): the pass^k estimator from tau-bench."""
    return comb(c, k) / comb(n, k)

# One task, 10 trials, 8 passed.
for k in (1, 3, 5):
    at, hat = pass_at_k(10, 8, k), pass_hat_k(10, 8, k)
    print(f"k={k}  pass@k={at:.3f}  pass^k={hat:.3f}")
# k=1  pass@k=0.800  pass^k=0.800
# k=3  pass@k=1.000  pass^k=0.467
# k=5  pass@k=1.000  pass^k=0.222
```

A task that passes 8 of 10 trials looks solved under pass@3 and fails more often than it succeeds under pass^3. Run at least as many trials as the largest k you report, reset the environment so trials stay independent, and keep per-task values so one flaky task does not vanish into an average.

## Graders for agents and how they fail

Agent evals use three kinds of graders: code that checks the end state and the transcript, LLM judges that apply a rubric to the transcript, and human reviewers. Code graders are the default because they are fast, cheap, and reproducible. Judges cover qualities code cannot check, and humans calibrate both.

| Grader | Best at | Failure mode | Mitigation |
|---|---|---|---|
| Code: end-state checks | Whether the task happened | Misses how it happened, such as a skipped confirmation | Add trajectory and policy checks |
| Code: trajectory match | Tool choice, arguments, forbidden calls | Rejects valid alternative paths when order is strict | Match writes and critical arguments only |
| LLM judge on transcript | Plan quality, error recovery, policy in conversation | Trusts the agent's claims; misses details in long transcripts | Give it tool results and the end state; ask one question per call |
| Human review | Ground truth and new failure modes | Slow, costly, reviewers disagree | Use for calibration and spot checks |

An [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge) grading a trajectory carries the position, verbosity, and self-preference biases covered in our LLM evaluation guide, plus an agent-specific weakness: the agent wrote much of the transcript, so a judge that reads only its messages will believe "the refund is done." Pass the judge the raw tool results and ask narrow binary questions, such as whether the agent confirmed before every write.

Graders also fail when the environment leaks. Anthropic's guide reports that in some internal evals, Claude examined git history left by previous trials to gain an unfair advantage. Reset state between trials, keep grading files outside the agent's writable workspace, and read sampled transcripts with their grades to tell agent mistakes from graders that rejected valid solutions.

## A code-based grader for an agent trajectory

A code-based grader for an agent takes the recorded transcript and the environment's end state and returns named pass or fail checks. The example grades a refund task: the refund must exist in the database, and the agent must write once with the right arguments, ask for confirmation first, stay within a tool budget, and tell the user the amount.

```python
WRITE_TOOLS = {"issue_refund", "cancel_order", "update_address"}
EXPECTED = [{"name": "issue_refund",
             "args": {"order_id": "W1001", "amount": 54.04}}]

def grade_refund_trial(trajectory, end_state, max_tool_calls=6):
    """Code-based grader for one trial of a refund task."""
    steps = list(enumerate(trajectory))
    tools = [(i, s) for i, s in steps if s["type"] == "tool"]
    writes = [(i, s) for i, s in tools if s["name"] in WRITE_TOOLS]

    def confirmed(i):
        # Policy: the agent asked, then the user replied, before the write.
        turns = [s["type"] for s in trajectory[:i] if s["type"] != "tool"]
        return turns[-2:] == ["assistant", "user"]

    final = [s["text"] for s in trajectory if s["type"] == "assistant"][-1]
    refunds = [{"order_id": "W1001", "amount": 54.04}]
    checks = {
        "end_state": end_state["refunds"] == refunds
        and end_state["orders"]["W1001"]["status"] == "refunded",
        "expected_write": [{"name": s["name"], "args": s["args"]}
                           for _, s in writes] == EXPECTED,
        "confirmed_before_write": all(confirmed(i) for i, _ in writes),
        "step_budget": len(tools) <= max_tool_calls,
        "told_user_amount": "54.04" in final,
    }
    return {"passed": all(checks.values()), **checks}


good = [
    {"type": "user", "text": "Refund order W1001, please."},
    {"type": "tool", "name": "get_order", "args": {"order_id": "W1001"}},
    {"type": "assistant", "text": "W1001 qualifies for $54.04. Proceed?"},
    {"type": "user", "text": "Yes."},
    {"type": "tool", "name": "issue_refund",
     "args": {"order_id": "W1001", "amount": 54.04}},
    {"type": "assistant", "text": "Refunded $54.04 to your card."},
]
eager = [good[0], good[1], good[4], good[5]]  # no confirmation step
state = {"orders": {"W1001": {"status": "refunded"}},
         "refunds": [{"order_id": "W1001", "amount": 54.04}]}

print(grade_refund_trial(good, state)["passed"])  # True
print(grade_refund_trial(eager, state))
# {'passed': False, 'end_state': True, 'expected_write': True,
#  'confirmed_before_write': False, 'step_budget': True,
#  'told_user_amount': True}
```

The eager trajectory leaves the same database as the good one, so an end-state check alone would pass it; only the trajectory check catches the missing confirmation. The check verifies the order of turns, not that the user agreed, so a production grader would read a structured confirmation flag from the tool layer or ask an LLM judge that one question. In a real run, the trajectory comes from the trace export and the end state from querying the sandbox. Per-check failure rates tell an engineer what to fix.

## How to build an agent eval set from real traces

Build [agent eval tasks](https://theaiengineering.com/glossary/golden-dataset) from production traces and known failures rather than invented scenarios. Each task needs an instruction, a reproducible initial state, the tools or mocks the agent may use, success criteria two domain experts would agree on, and ideally a reference solution that proves the task is solvable in that environment.

1. **Sample traces** through your [LLM observability](https://theaiengineering.com/glossary/llm-observability) pipeline, weighted toward failures, escalations, retries, and negative feedback.
2. **Read and label** each failure: wrong tool, wrong arguments, missed policy, gave up early, or looped.
3. **Convert it to a task.** Rebuild the initial state with personal data removed, write the instruction, and define the goal state. For conversational agents, give a simulated user a goal and persona, as τ-bench does with a language model.
4. **Add a reference solution** that proves the task can pass; Terminal-Bench ships a human-written one with every task, according to its [paper](https://arxiv.org/abs/2601.11868).
5. **Balance the set** with tasks where the agent should act and tasks where it should refuse, ask, or hand off.
6. **Version it** and start small: Anthropic recommends 20 to 50 simple tasks drawn from real failures.

**Figure 2: The agent eval loop. Production failures become seeded tasks, and the CI gate checks pass^k, policy, and cost before each release.**

Offline, versioned eval tasks run as sandboxed trials, go to graders, and pass a CI gate; online, production runs are scored, failures are reviewed, and new tasks return to the eval set.

Components:

- **Eval tasks**: Versioned set
- **Sandboxed trials**: k trials each
- **Graders**: Code and judges
- **CI gate**: pass^k, cost
- **Production**: Traced runs
- **Online scoring**: Sampled traces
- **Failure review**: Read, label
- **New tasks**: From failures

Groups:

- Offline: Eval tasks, Sandboxed trials, Graders, CI gate
- Online: New tasks, Failure review, Online scoring, Production

Connections:

- Eval tasks → Sandboxed trials
- Sandboxed trials → Graders
- Graders → CI gate
- CI gate → Production: deploy
- Production → Online scoring
- Online scoring → Failure review
- Failure review → New tasks
- New tasks → Eval tasks

Isolation matters as much as coverage: Anthropic's guide warns that leftover files, cached data, or exhausted resources from one trial can cause correlated failures that reflect infrastructure, not the agent.

## Offline evals in CI vs online evals in production

Offline agent evals run versioned tasks in sandboxed environments before a change ships, so they can check end states and gate merges. [Online evals](https://theaiengineering.com/glossary/online-evaluation) score sampled production runs after release, where no goal state exists, using reference-free judges, policy checks on traces, and user signals. Offline evals catch regressions; online evals find the failures that become next week's tasks.

| | Offline evals in CI | Online evals in production |
|---|---|---|
| Environment | Sandboxes, seeded databases, mocked or test-mode APIs | Real tools and real users |
| Ground truth | Goal state and success criteria per task | Usually none |
| Graders | State checks, trajectory checks, calibrated judges | Reference-free judges, policy checks on traces, escalation rates |
| Trials | Several per task, to estimate pass^k | One per user request |
| Main job | Block regressions and compare configurations | Detect drift and discover new failure modes |

Anthropic's guide separates capability evals, which start with a low pass rate, from regression evals, which should sit near 100%. Gate merges on the regression suite: pass^k on critical tasks, zero policy violations, and cost per task within budget. Move capability tasks into it once they pass reliably. Since 50 tasks at 5 trials each is 250 full agent runs, run a sample on pull requests and the full suite on release branches. The gate script in our [LLM evaluation guide](https://theaiengineering.com/articles/llm-evaluation) works unchanged for agent metrics.

Online, trajectory checks still apply, because a forbidden tool call or an unconfirmed write is visible in logged spans. Many [LLM observability and evaluation platforms](https://theaiengineering.com/platforms/llm-observability) run judges on sampled traces and turn flagged runs into dataset items.

## Public agent benchmarks and what they measure

Public agent benchmarks score a model and its scaffold in a fixed environment: real repositories, simulated customer service, web apps, desktops, or terminals. They help shortlist models and supply proven grading designs, but they do not measure your agent on your tasks, and a reported score depends on the scaffold, tools, and turn limits used.

| Benchmark | Environment | Task | Grading |
|---|---|---|---|
| [SWE-bench Verified](https://www.swebench.com/SWE-bench/) | Real Python repositories | Resolve a GitHub issue with a patch | Fail-to-pass and pass-to-pass tests; 500 human-validated tasks |
| τ-bench | Retail and airline support with API tools, a policy, and an LLM-simulated user | Complete the request within policy | Final database against a goal state, plus required facts in replies; reports pass^k |
| [τ²-bench](https://arxiv.org/abs/2506.07982) | Telecom support where agent and user both act on shared state | Resolve the issue while guiding the user | Verifiable tasks from a compositional generator |
| [GAIA](https://arxiv.org/abs/2311.12983) | The open web and the agent's tools | Real-world questions needing browsing, multimodality, and tool use | Quasi exact match on a short answer; 466 questions |
| WebArena | Four self-hosted web apps: shopping, forums, code collaboration, content management | 812 long-horizon tasks | Programmatic validators of functional correctness |
| [OSWorld](https://arxiv.org/abs/2404.07972) | Real Ubuntu, Windows, and macOS computers | 369 tasks across apps, files, and multi-app workflows | A per-task evaluation script from a defined initial state |
| Terminal-Bench | Containerized terminals | 89 hard tasks in version 2.0 | Tests on the final container state; each task has a human-written solution |

OpenAI stopped reporting SWE-bench Verified in February 2026, citing flawed tests and [contamination](https://theaiengineering.com/glossary/benchmark-contamination); our [LLM benchmarks guide](https://theaiengineering.com/articles/llm-benchmarks) covers that decision and SWE-bench Pro. Terminal-Bench's site now leads with Terminal-Bench 4.0, so check which version a result used. Coding-agent results depend heavily on the scaffold, as our [AI coding agents comparison](https://theaiengineering.com/articles/ai-coding-agents) shows.

Where to go next: our guide on [how to build an AI agent](https://theaiengineering.com/articles/how-to-build-an-ai-agent) writes the loop these evals test, the [AI agent frameworks comparison](https://theaiengineering.com/articles/ai-agent-frameworks) shows which frameworks emit traces, and the [tool calling guide](https://theaiengineering.com/articles/tool-calling) covers tool design behind many trajectory failures.

## Frequently asked questions

### How do you evaluate AI agents?

Give the agent realistic tasks in a sandboxed environment with a known initial state, run each task several times, and grade every trial on two artifacts: the end state, such as database records or passing tests, and the transcript of tool calls. Add checks for policy, cost, and step counts, calibrate any LLM judge against human labels, and gate releases on the results.

### What are the most important AI agent evaluation metrics?

Task success rate, measured against the final environment state, is the primary metric. Supporting metrics cover the trajectory (correct tool selection, correct arguments, redundant steps), safety (policy violations, forbidden tool calls, resistance to prompt injection), efficiency (cost, latency, and turns per task), and reliability across repeated trials, usually reported as pass^k for agents that must succeed every time.

### What is the difference between pass@k and pass^k?

pass@k is the probability that at least one of k independent trials of a task succeeds, so it rises as k grows. pass^k, introduced with the τ-bench benchmark in 2024, is the probability that all k trials succeed, so it falls as k grows. Both equal the average success rate when k is 1. pass^k measures the consistency that users actually experience.

### How do you test AI agents before production?

Build 20 to 50 tasks from real traces and known failures, each with a seeded environment, success criteria, and a reference solution. Run every task several times in a fresh sandbox, with mocked or test-mode tools for anything that has real side effects and a simulated user for conversational agents. Gate merges on a regression suite and track a harder capability suite separately.

### What tools are used for AI agent evaluation?

Agent evaluation tools fall into three groups. Open source frameworks such as Inspect, from the UK AI Security Institute and Meridian Labs, run agent tasks with tool calling and Docker sandboxes. Harbor, from the creators of Terminal-Bench, runs agents in container environments. Observability platforms add trace-based datasets, judges on sampled production traffic, and annotation queues. Most teams also write custom graders in plain code.

## Terms in this guide

- [AI agent](https://theaiengineering.com/glossary/ai-agent): An AI agent is a software system in which a large language model runs in a loop, calling tools, reading their results, and repeating until the task is complete or a stop condition is reached.
- [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation): LLM evaluation is the practice of measuring how well a language model or LLM-based application performs on defined tasks, using test datasets, automated graders, model-based judges, and human review.
- [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge): LLM-as-a-judge is an evaluation method in which a language model scores or compares the outputs of an LLM application against a rubric, a reference answer, or another response, replacing or supplementing human graders.
- [LLM observability](https://theaiengineering.com/glossary/llm-observability): LLM observability is the practice of instrumenting LLM applications to record each request's prompts, responses, tool calls, token usage, latency, cost, and quality scores as traces that teams can search and analyze.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [Guardrails](https://theaiengineering.com/glossary/guardrails): Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.
- [Prompt injection](https://theaiengineering.com/glossary/prompt-injection): Prompt injection is an attack in which text supplied to a large language model, typed by a user or hidden in content the model reads, alters the application's intended behavior or output.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.
- [Agent orchestration](https://theaiengineering.com/glossary/agent-orchestration): Agent orchestration is the control logic that decides which agent, model call, or tool runs next in an AI system and what each receives, whether written in code or chosen by a model.
- [Human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop): Human-in-the-loop (HITL) is an agent design in which the run pauses at defined checkpoints, such as before a sensitive tool call, so a person can approve, edit, or reject the action.
- [pass@k](https://theaiengineering.com/glossary/pass-at-k): pass@k is an evaluation metric that estimates the probability that at least one of k sampled outputs for a task passes its tests, used mainly in code generation and agent benchmarks.
- [Agent sandbox](https://theaiengineering.com/glossary/agent-sandbox): An agent sandbox is an isolated execution environment in which an AI agent runs code, shell commands, or a browser, with enforced limits on the files, network hosts, and credentials it can reach.
- [Golden dataset](https://theaiengineering.com/glossary/golden-dataset): A golden dataset is a curated, versioned set of representative inputs, each paired with an expected output or grading criteria, that a team treats as ground truth when evaluating an LLM application.
- [Online evaluation](https://theaiengineering.com/glossary/online-evaluation): Online evaluation is the practice of scoring an LLM application's live production traffic, usually a sample of traces, with reference-free graders, policy checks, and user feedback to detect quality problems after release.
- [SWE-bench](https://theaiengineering.com/glossary/swe-bench): SWE-bench is a benchmark of 2,294 real GitHub issues from 12 Python repositories, in which a model or agent must write a patch that resolves the issue and passes the repository's tests.
- [Multimodal model](https://theaiengineering.com/glossary/multimodal-model): A multimodal model is an AI model that takes in or produces more than one type of data, such as text, images, audio, or video, and combines them within a single model.
- [Benchmark contamination](https://theaiengineering.com/glossary/benchmark-contamination): Benchmark contamination is the presence of a benchmark's test questions, answers, or solutions in a model's training data, so its score can reflect memorization instead of the capability the benchmark measures.

## Related guides

- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [LLM observability: what to capture, which metrics, and tools](https://theaiengineering.com/articles/llm-observability): LLM observability guide: what to capture in traces and spans, metrics to track, OpenTelemetry GenAI conventions, tool categories, and keeping prompts private.
- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.
- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.

Part of the [LLM evaluation and observability](https://theaiengineering.com/topics/llm-evaluation) topic hub.

## Cite this guide

The AI Engineering Editors. "AI agent evaluation: metrics, graders, and how to test agents." The AI Engineering, updated Oct 31, 2026. https://theaiengineering.com/articles/ai-agent-evaluation

- Title: AI agent evaluation: metrics, graders, and how to test agents
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-10-31
- Updated: 2026-10-31
- URL: https://theaiengineering.com/articles/ai-agent-evaluation
- Markdown: https://theaiengineering.com/articles/ai-agent-evaluation.md

To cite one section, link to its anchor:

- [What is AI agent evaluation?](https://theaiengineering.com/articles/ai-agent-evaluation#what-is-ai-agent-evaluation)
- [How agent evaluation differs from evaluating an LLM call](https://theaiengineering.com/articles/ai-agent-evaluation#how-agent-evaluation-differs-from-evaluating-an-llm-call)
- [What to measure: AI agent evaluation metrics](https://theaiengineering.com/articles/ai-agent-evaluation#what-to-measure-ai-agent-evaluation-metrics)
- [pass@k vs pass^k: measuring reliability across runs](https://theaiengineering.com/articles/ai-agent-evaluation#passk-vs-passk-measuring-reliability-across-runs)
- [Graders for agents and how they fail](https://theaiengineering.com/articles/ai-agent-evaluation#graders-for-agents-and-how-they-fail)
- [A code-based grader for an agent trajectory](https://theaiengineering.com/articles/ai-agent-evaluation#a-code-based-grader-for-an-agent-trajectory)
- [How to build an agent eval set from real traces](https://theaiengineering.com/articles/ai-agent-evaluation#how-to-build-an-agent-eval-set-from-real-traces)
- [Offline evals in CI vs online evals in production](https://theaiengineering.com/articles/ai-agent-evaluation#offline-evals-in-ci-vs-online-evals-in-production)
- [Public agent benchmarks and what they measure](https://theaiengineering.com/articles/ai-agent-evaluation#public-agent-benchmarks-and-what-they-measure)
- [Frequently asked questions](https://theaiengineering.com/articles/ai-agent-evaluation#faq)
