What is OpenTelemetry? Tracing LLM calls and agents
Also called OTel, OpenTelemetry GenAI semantic conventions
Definition
OpenTelemetry is an open source, vendor-neutral observability framework of APIs, SDKs, a wire protocol, and naming conventions for traces, metrics, and logs, including conventions for LLM calls and agent runs.
OpenTelemetry, usually shortened to OTel, is an open source observability framework for generating, collecting, and exporting traces, metrics, and logs. It is a Cloud Native Computing Foundation project made up of APIs, language SDKs, the OTLP wire protocol, a Collector, and semantic conventions that fix attribute names. Storage and visualization are left to other tools, according to the OpenTelemetry overview, so the data goes to whichever backend a team chooses. Its generative AI conventions give model calls, tool executions, and agent runs one schema, so spans from any framework can be read by any compatible LLM observability tool.
How OpenTelemetry traces an LLM call#
OpenTelemetry records an LLM call as a span: one timed operation with a name, a parent, a status, and key-value attributes. Spans that share a trace ID form a trace, the tree of work for one request. The GenAI conventions name an inference span {gen_ai.operation.name} {gen_ai.request.model}, such as chat followed by the model name.
The client inference conventions mark gen_ai.operation.name and gen_ai.provider.name as required and gen_ai.request.model as conditionally required. Token counts go in gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, with breakouts such as gen_ai.usage.cache_read.input_tokens and gen_ai.usage.reasoning.output_tokens when the provider reports them. gen_ai.response.finish_reasons records why generation stopped. Prompt and response content, in gen_ai.input.messages and gen_ai.output.messages, is opt-in, and instrumentations should not capture it by default.
As of November 2026, every GenAI convention carries Development status, the earliest OpenTelemetry maturity level, so span and attribute names can still change between releases. The core tracing API, SDK, and OTLP protocol are stable. In 2026 the GenAI conventions moved out of the main semantic conventions repository into a dedicated GenAI semantic conventions repository, so pin the convention version your instrumentation emits.
How OpenTelemetry traces an agent run#
An agent run becomes one trace rooted in an invoke_agent span. Each model call is a child inference span, and each tool call is an execute_tool {gen_ai.tool.name} span. Work a tool does, such as an HTTP request, nests under its tool span, so the trace shows which step of the run was slow, expensive, or wrong.
The tool execution conventions require gen_ai.tool.name and set gen_ai.tool.call.id to the identifier the model supplied, which ties each tool span to the tool call in the model's response. gen_ai.agent.name labels the agent, and gen_ai.conversation.id groups the turns of one session. OpenTelemetry propagates trace context between processes, so a request that passes through an AI gateway, an MCP server, or a remote agent can stay in one trace; the conventions include separate documents for MCP and A2A.
OpenTelemetry vs LLM observability platforms#
OpenTelemetry produces and transports telemetry, while an LLM observability platform stores it, displays it, and runs analysis such as evaluations on it. Instrumentation libraries such as OpenLLMetry and OpenInference emit OpenTelemetry-compatible spans for common model SDKs and agent frameworks, and OTLP sends those spans to any backend that accepts the protocol.
Instrumentation written once can therefore feed an open source tracing platform, a general APM product, or an online evaluation pipeline, and switching backends does not require re-instrumenting the application. Evaluators can attach their scores to the same trace as gen_ai.evaluation.result events. The LLM observability guide covers what to capture in each span and which metrics to track.
Related terms
Guides that cover opentelemetry
- LLM observability: what to capture, which metrics, and tools
- How to build an AI agent: a step-by-step guide in Python
- What is AI engineering? The discipline, the stack, and the work
- AI agent frameworks compared: LangGraph, CrewAI, ADK, and more
- A2A protocol vs MCP: differences and when to use each
- Bifrost vs LiteLLM: architecture, features, and performance compared
- Claude Code vs Cursor: how the two coding agents differ
Cite this definition
The AI Engineering Editors. "What is OpenTelemetry? Tracing LLM calls and agents." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/opentelemetry
- Title
- What is OpenTelemetry? Tracing LLM calls and agents
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated