---
title: "What is OpenTelemetry? Tracing LLM calls and agents"
description: "OpenTelemetry is the open source standard for traces, metrics, and logs. See how its GenAI conventions record LLM calls, token usage, tool calls, and agents."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/opentelemetry"
markdown_url: "https://theaiengineering.com/glossary/opentelemetry.md"
topic: "LLM evaluation and observability"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["OpenTelemetry","OTel","OpenTelemetry GenAI semantic conventions"]
citation: "The AI Engineering Editors. \"What is OpenTelemetry? Tracing LLM calls and agents.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/opentelemetry"
---

# What is OpenTelemetry? Tracing LLM calls and agents

Also called OTel, OpenTelemetry GenAI semantic conventions.

> **Definition of OpenTelemetry:** OpenTelemetry is an open source, vendor-neutral observability framework of APIs, SDKs, a wire protocol, and naming conventions for traces, metrics, and logs, including conventions for LLM calls and agent runs.

OpenTelemetry, usually shortened to OTel, is an open source observability framework for generating, collecting, and exporting traces, metrics, and logs. It is a Cloud Native Computing Foundation project made up of APIs, language SDKs, the OTLP wire protocol, a Collector, and semantic conventions that fix attribute names. Storage and visualization are left to other tools, according to the [OpenTelemetry overview](https://opentelemetry.io/docs/what-is-opentelemetry/), so the data goes to whichever backend a team chooses. Its generative AI conventions give model calls, tool executions, and agent runs one schema, so spans from any framework can be read by any compatible [LLM observability](https://theaiengineering.com/glossary/llm-observability) tool.

## How OpenTelemetry traces an LLM call

OpenTelemetry records an LLM call as a span: one timed operation with a name, a parent, a status, and key-value attributes. Spans that share a trace ID form a trace, the tree of work for one request. The GenAI conventions name an inference span `{gen_ai.operation.name} {gen_ai.request.model}`, such as `chat` followed by the model name.

The [client inference conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/client-inference.md) mark `gen_ai.operation.name` and `gen_ai.provider.name` as required and `gen_ai.request.model` as conditionally required. Token counts go in `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`, with breakouts such as `gen_ai.usage.cache_read.input_tokens` and `gen_ai.usage.reasoning.output_tokens` when the provider reports them. `gen_ai.response.finish_reasons` records why generation stopped. Prompt and response content, in `gen_ai.input.messages` and `gen_ai.output.messages`, is opt-in, and instrumentations should not capture it by default.

As of November 2026, every GenAI convention carries Development status, the earliest OpenTelemetry maturity level, so span and attribute names can still change between releases. The core tracing API, SDK, and OTLP protocol are stable. In 2026 the GenAI conventions moved out of the main semantic conventions repository into a dedicated [GenAI semantic conventions repository](https://github.com/open-telemetry/semantic-conventions-genai), so pin the convention version your instrumentation emits.

## How OpenTelemetry traces an agent run

An agent run becomes one trace rooted in an `invoke_agent` span. Each model call is a child inference span, and each tool call is an `execute_tool {gen_ai.tool.name}` span. Work a tool does, such as an HTTP request, nests under its tool span, so the trace shows which step of the run was slow, expensive, or wrong.

**Figure 1: One agent run as an OpenTelemetry trace, with span names from the GenAI conventions. Timings and token counts are illustrative.**

An agent run traced as an invoke_agent root span containing two chat model spans and an execute_tool span with a nested HTTP request.

Spans:

- invoke_agent helpdesk: 0 to 4300 ms (4.3 s)
  - chat {model}: 40 to 1640 ms (1,920 in · 84 out tokens)
  - execute_tool get_order: 1680 to 2280 ms (gen_ai.tool.call.id)
    - GET /orders/{id}: 1720 to 2240 ms (HTTP client span)
  - chat {model}: 2320 to 4260 ms (2,210 in · 190 out tokens)

The [tool execution conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/execute-tool.md) require `gen_ai.tool.name` and set `gen_ai.tool.call.id` to the identifier the model supplied, which ties each tool span to the [tool call](https://theaiengineering.com/glossary/tool-calling) in the model's response. `gen_ai.agent.name` labels the agent, and `gen_ai.conversation.id` groups the turns of one session. OpenTelemetry propagates trace context between processes, so a request that passes through an [AI gateway](https://theaiengineering.com/glossary/ai-gateway), an MCP server, or a remote agent can stay in one trace; the conventions include separate documents for MCP and A2A.

## OpenTelemetry vs LLM observability platforms

OpenTelemetry produces and transports telemetry, while an LLM observability platform stores it, displays it, and runs analysis such as evaluations on it. Instrumentation libraries such as OpenLLMetry and OpenInference emit OpenTelemetry-compatible spans for common model SDKs and agent frameworks, and OTLP sends those spans to any backend that accepts the protocol.

Instrumentation written once can therefore feed an open source tracing platform, a general APM product, or an [online evaluation](https://theaiengineering.com/glossary/online-evaluation) pipeline, and switching backends does not require re-instrumenting the application. Evaluators can attach their scores to the same trace as `gen_ai.evaluation.result` events. The [LLM observability guide](https://theaiengineering.com/articles/llm-observability) covers what to capture in each span and which metrics to track.

## Related terms

- [LLM observability](https://theaiengineering.com/glossary/llm-observability): LLM observability is the practice of instrumenting LLM applications to record each request's prompts, responses, tool calls, token usage, latency, cost, and quality scores as traces that teams can search and analyze.
- [Online evaluation](https://theaiengineering.com/glossary/online-evaluation): Online evaluation is the practice of scoring an LLM application's live production traffic, usually a sample of traces, with reference-free graders, policy checks, and user feedback to detect quality problems after release.
- [AI agent](https://theaiengineering.com/glossary/ai-agent): An AI agent is a software system in which a large language model runs in a loop, calling tools, reading their results, and repeating until the task is complete or a stop condition is reached.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [AI gateway](https://theaiengineering.com/glossary/ai-gateway): An AI gateway is a proxy layer between applications and model providers that exposes one API for many models and centralizes routing, failover, access control, cost tracking, caching, and logging.
- [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation): LLM evaluation is the practice of measuring how well a language model or LLM-based application performs on defined tasks, using test datasets, automated graders, model-based judges, and human review.

## Guides that cover opentelemetry

- [LLM observability: what to capture, which metrics, and tools](https://theaiengineering.com/articles/llm-observability): LLM observability guide: what to capture in traces and spans, metrics to track, OpenTelemetry GenAI conventions, tool categories, and keeping prompts private.
- [How to build an AI agent: a step-by-step guide in Python](https://theaiengineering.com/articles/how-to-build-an-ai-agent): Learn how to build an AI agent from scratch in Python: when you need one, tool schemas, the agent loop, stop conditions, guardrails, evals, and tracing.
- [What is AI engineering? The discipline, the stack, and the work](https://theaiengineering.com/articles/what-is-ai-engineering): What is AI engineering? A plain definition, how it differs from ML and software engineering, the layers of the stack, and what AI engineers do day to day.
- [AI agent frameworks compared: LangGraph, CrewAI, ADK, and more](https://theaiengineering.com/articles/ai-agent-frameworks): An AI agent frameworks comparison of LangGraph, LangChain, CrewAI, the OpenAI, Claude, and Google SDKs, Pydantic AI, Mastra, and more, by language and license.
- [A2A protocol vs MCP: differences and when to use each](https://theaiengineering.com/articles/a2a-vs-mcp): A2A protocol vs MCP: A2A connects AI agents to other agents, and MCP connects an agent to tools and data. Compare agent cards, tasks, transports, and security.
- [Bifrost vs LiteLLM: architecture, features, and performance compared](https://theaiengineering.com/articles/bifrost-vs-litellm): Bifrost vs LiteLLM compared: a Go single binary against a Python proxy, with deployment, routing, virtual keys, caching, MCP, benchmarks, and migration.
- [Claude Code vs Cursor: how the two coding agents differ](https://theaiengineering.com/articles/claude-code-vs-cursor): Claude Code vs Cursor compared: terminal agent vs AI editor, models, subagents, hooks, rules files, code review, CI use, pricing, and enterprise controls.

Part of the [LLM evaluation and observability](https://theaiengineering.com/topics/llm-evaluation) topic hub.

## Cite this definition

The AI Engineering Editors. "What is OpenTelemetry? Tracing LLM calls and agents." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/opentelemetry

- Title: What is OpenTelemetry? Tracing LLM calls and agents
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/opentelemetry
- Markdown: https://theaiengineering.com/glossary/opentelemetry.md
