LLM observability and evaluation
LLM observability is the practice of recording each step of an LLM application, including prompts, model responses, tool calls, retrieval results, latency, token usage, and cost, so engineers can debug failures and measure quality. The tools in this category collect those traces, score them with evaluations, or both.
The category has four groups. Open-source platforms such as Langfuse, Opik, MLflow, Laminar, and Agenta can be self-hosted. Evaluation frameworks such as DeepEval, Ragas, promptfoo, and Inspect run tests in code and CI. Commercial platforms such as LangSmith, Braintrust, Maxim AI, Galileo, and HoneyHive combine tracing, datasets, experiments, and online scoring. Application monitoring vendors, including Datadog, New Relic, Dynatrace, Grafana, and Sentry, add LLM views to existing APM.
Most tools accept OpenTelemetry data, which matters because ownership is changing: ClickHouse acquired Langfuse, Cisco acquired Galileo, OpenAI agreed to buy promptfoo, Traceloop is joining ServiceNow, and Helicone moved to maintenance mode in 2026.
LLM observability and evaluation compared
| Platform | Open source | Deployment | Best for |
|---|---|---|---|
| Agenta | Yes (MIT) | Self-hosted, Managed cloud | Teams where non-engineers edit and evaluate prompts |
| Arize AX | No | Managed cloud | Enterprise agent monitoring beyond self-hosted Phoenix |
| Arize Phoenix | No | Local, Self-hosted, Managed cloud | Local and self-hosted tracing with OpenTelemetry |
| Braintrust | No | Managed cloud | Eval-driven development using production traces |
| Confident AI and DeepEval | Yes (Apache-2.0) | Library, Managed cloud | Pytest-style LLM tests with a hosted dashboard |
| Datadog LLM Observability | No | Managed cloud | Teams already monitoring infrastructure with Datadog |
| Dynatrace AI Observability | No | Managed cloud | Dynatrace customers adding LLM monitoring |
| Evidently | Yes (Apache-2.0) | Library, Self-hosted | Teams monitoring both classic ML and LLM systems |
| Fiddler AI | No | Managed cloud, Self-hosted | Regulated enterprises monitoring ML and agents together |
| Galileo | No | Managed cloud | Turning offline evals into production guardrails |
| Grafana Cloud AI Observability | No | Managed cloud | Teams standardized on Grafana and OpenTelemetry |
| Helicone | Yes (Apache-2.0) | Managed cloud, Self-hosted | Existing users needing proxy-based request logging |
| HoneyHive | No | Managed cloud, Self-hosted | Enterprises with strict data residency for traces |
| Inspect | Yes (MIT) | Library, CLI | Research-grade model and agent evaluations |
| Laminar | Yes (Apache-2.0) | Managed cloud, Self-hosted | Debugging long-running and browser agents |
| Langfuse | Yes (MIT) | Self-hosted, Managed cloud | Self-hosted tracing and prompt management |
| LangSmith | No | Managed cloud, Self-hosted | Teams building with LangChain or LangGraph |
| LangWatch | Yes (Apache-2.0) | Managed cloud, Self-hosted | Platform teams governing agents and coding assistants |
| Lunary | No | Managed cloud, Self-hosted | Small teams combining logs, prompts, and reviews |
| Maxim AI | No | Managed cloud, Self-hosted | Pre-release agent simulation plus production monitoring |
| MLflow | Yes (Apache-2.0) | Self-hosted, Managed cloud, Library | Teams already using MLflow for ML lifecycle work |
| New Relic AI observability | No | Managed cloud | Teams already using New Relic APM |
| OpenLIT | Yes (Apache-2.0) | Self-hosted, Library | Self-hosted OpenTelemetry stack with GPU monitoring |
| Opik | Yes (Apache-2.0) | Self-hosted, Managed cloud | Open-source evals with built-in prompt optimization |
| promptfoo | Yes (MIT) | CLI, Library, Managed cloud | Prompt and model regression tests in CI |
| PromptLayer | No | Managed cloud | Prompt iteration by domain experts |
| Pydantic Logfire | No | Managed cloud, Self-hosted | Python teams using Pydantic AI or FastAPI |
| Ragas | Yes (Apache-2.0) | Library | RAG metrics and synthetic test data |
| Raindrop | No | Managed cloud | Finding silent failures in production agent conversations |
| Respan (formerly Keywords AI) | Yes (Apache-2.0) | Managed cloud, Self-hosted | Gateway and tracing from one vendor |
| Sentry AI observability | No | Managed cloud | Developers already using Sentry for errors |
| Traceloop and OpenLLMetry | Yes (Apache-2.0) | Library, Managed cloud | OpenTelemetry-native instrumentation for LLM apps |
| TruLens | Yes (MIT) | Library | Scoring RAG pipelines step by step |
| W&B Weave | Yes (Apache-2.0) | Managed cloud, Library | Teams already using Weights & Biases |
Profiles
Agenta
agenta.ai
Agenta is an open-source workspace, MIT-licensed outside its enterprise directories, for building agents through chat and improving them with versioned prompts and tools, test-set evaluations with LLM-as-a-judge or custom evaluators, and tracing of every run.

Arize AX
Arize AI · arize.com
Arize AX is Arize AI's managed platform for observing, evaluating, and improving AI agents, with tracing built on OpenInference, the open convention Arize created, plus span, trace, and session evaluations, experiments, and production monitoring.

Arize Phoenix
Arize AI · arize.com
Arize Phoenix is a self-hostable, source-available tool under the Elastic License 2.0 for tracing LLM applications and agents with OpenTelemetry and OpenInference, running evals, and comparing experiments on datasets.
Braintrust
braintrust.dev
Braintrust is a commercial platform for evaluating and observing AI agents that traces production traffic, turns traces into versioned datasets, scores outputs with LLM, code, or human scorers, and catches regressions before release.

Confident AI and DeepEval
Confident AI · confident-ai.com
DeepEval is an Apache-2.0 Python framework for testing LLM applications with plug-and-play metrics for agents, RAG, multi-turn chat, and voice, and Confident AI is the commercial platform from the same team for running those evals and observing production traces.
Datadog LLM Observability
Datadog · datadoghq.com
Datadog LLM Observability, part of Datadog's Agent Observability offering, traces prompts, retrieval, and tool calls in agent workflows, runs offline experiments and built-in or custom evaluations, and flags hallucinations, prompt injection, and PII exposure.
Dynatrace AI Observability
Dynatrace · dynatrace.com
Dynatrace AI Observability monitors AI applications, LLMs, and agents from the user experience through orchestration to models and infrastructure, tracking token usage, latency, and cost and detecting hallucinations, prompt injection, PII leakage, and toxicity.

Evidently
Evidently AI · evidentlyai.com
Evidently is an Apache-2.0 Python framework for evaluating, testing, and monitoring ML and LLM systems, with a library of built-in metrics that combines rules, classifiers, and LLM judges for testing AI applications, RAG systems, and multi-agent workflows.

Fiddler AI
fiddler.ai
Fiddler AI is a commercial control plane for enterprise AI agents and models that combines evaluation, monitoring, guardrail policy enforcement, governance, and ML observability, with SaaS, VPC, and AWS GovCloud deployments.

Galileo
Cisco · galileo.ai · Burlingame, California, United States
Galileo is an evaluation and observability platform for AI agents with built-in and custom evaluators, an insights engine that finds failure modes, and Luna models distilled from evaluators to monitor production traffic and drive guardrails.
Cisco completed its acquisition of Galileo on May 22, 2026, and is bringing its capabilities into Splunk Observability Cloud.
Grafana Cloud AI Observability
Grafana Labs · grafana.com
Grafana Cloud AI Observability, including Agent Observability, traces production agents with OpenTelemetry, runs live evaluations for hallucinations and quality drift, and tracks latency, tokens, and cost across agents, runs, and models.

Helicone
helicone.ai
Helicone is an Apache-2.0 LLM observability platform and AI gateway that logs requests through a proxy or SDK and shows sessions, users, and costs, with prompt management, datasets, a playground, rate limits, and alerts.
Mintlify acquired Helicone in March 2026 and moved it to maintenance mode; Helicone's site now states that it is no longer part of Mintlify.

HoneyHive
honeyhive.ai
HoneyHive is a commercial platform for tracing, evaluating, monitoring, and improving production AI agents across teams, with OpenTelemetry ingestion and hybrid or fully self-hosted deployments that keep trace data inside the customer's network.
Inspect
UK AI Security Institute · inspect.aisi.org.uk
Inspect is an MIT-licensed Python framework from the UK AI Security Institute for building and running LLM evaluations, with components for datasets, solvers, scorers, tool use, agents, and sandboxed execution, plus a log viewer for results.
Laminar
laminar.sh
Laminar is an Apache-2.0 observability platform for AI agents that traces runs, extracts agent inputs automatically, evaluates results, and surfaces failures, with integrations for agent SDKs and browser agents and Helm charts for self-hosting.

Langfuse
ClickHouse · langfuse.com
Langfuse is an open-source platform for tracing, evaluating, and improving LLM applications and agents, with prompt management and datasets, MIT-licensed outside its enterprise directories and available self-hosted or as Langfuse Cloud.
ClickHouse acquired Langfuse in January 2026, and the project remains MIT-licensed outside its enterprise directories.

LangSmith
LangChain · langchain.com
LangSmith is LangChain's commercial platform for tracing, monitoring, and evaluating LLM and agent applications; it works with or without LangChain, accepts OpenTelemetry data, and runs as managed cloud, bring-your-own-cloud, or self-hosted.
LangWatch
langwatch.ai
LangWatch is an Apache-2.0 platform for AI platform teams that traces, tests, routes, and governs LLM calls across a company, from internal agents to the coding assistants engineers use, with evaluations and observability in one place.

Lunary
lunary.ai
Lunary is an observability and prompt management platform for LLM applications that logs traces and errors, versions prompt templates with A/B testing, supports human review of responses, and reports usage and cost, in the cloud or self-hosted.

Maxim AI
getmaxim.ai
Maxim AI is a commercial platform for simulating, evaluating, and observing AI agents, with AI-generated scenario simulations, LLM-as-a-judge, programmatic, statistical, and human evaluators, and real-time trace logging with alerts.
MLflow
mlflow.org
MLflow is an open-source platform for agents, LLMs, and machine learning models that adds OpenTelemetry-compatible tracing, LLM evaluation, prompt management, and an AI gateway to its experiment tracking and model registry, self-hosted or as managed services.
New Relic AI observability
New Relic · newrelic.com
New Relic's AI observability extends its full-stack monitoring to AI applications, showing response time, quality, and token metrics next to APM signals, mapping interactions between agents and tools, and integrating with OpenAI, vector databases, and LangChain.

OpenLIT
openlit.io
OpenLIT is an Apache-2.0 platform built on OpenTelemetry for agent observability, evaluations, guardrails, prompt management, and cost and GPU monitoring, covering both custom agents and coding agents, with auto-instrumentation SDKs and a self-hostable dashboard.

Opik
Comet · comet.com
Opik is an Apache-2.0 platform from Comet for tracing agents, scoring traces with LLM-as-a-judge metrics, running test suites, optimizing prompts, and applying guardrails, available self-hosted or on Comet's cloud.
promptfoo
promptfoo.dev
promptfoo is an MIT-licensed CLI and library for testing prompts, agents, and RAG pipelines against test cases and comparing models side by side, with red teaming and a commercial platform for teams.
OpenAI agreed to acquire Promptfoo in March 2026 and said it would keep developing the MIT-licensed open-source project.

PromptLayer
promptlayer.com
PromptLayer is a prompt management platform with a prompt registry and visual editor, LLM evaluations and regression sets, and production tracing and logs, designed so domain experts can iterate on prompts without changing the codebase.
Pydantic Logfire
Pydantic · pydantic.dev
Pydantic Logfire is an observability platform built on OpenTelemetry that traces LLM calls and agents together with the rest of an application, with an MIT-licensed Python SDK and support for JavaScript, Rust, Go, Java, and other OpenTelemetry languages.

Ragas
ragas.io
Ragas is an Apache-2.0 Python library for evaluating LLM applications with LLM-based and traditional metrics, generating synthetic test sets, and building feedback loops from production data; it began with metrics for retrieval-augmented generation.
Raindrop
raindrop.ai
Raindrop is a commercial monitoring platform for AI agents that traces every run, detects silent failures in production conversations, and lets teams simulate changes before shipping them.

Respan (formerly Keywords AI)
Respan · respan.ai
Respan, formerly Keywords AI, pairs an open-source AI gateway written in Rust and licensed Apache-2.0 with a hosted platform for tracing, monitoring, and evaluating every agent run.
Keywords AI rebranded as Respan in 2026 and released its Rust gateway as open source.
Sentry AI observability
Sentry · sentry.io
Sentry's AI observability traces agent runs, LLM calls, and tool calls in production applications and flags errors, latency, and token cost spikes, with SDK integrations for the OpenAI, Anthropic, Vercel AI, LangChain, and Pydantic AI libraries.

Traceloop and OpenLLMetry
Traceloop · traceloop.com
Traceloop is an LLM reliability platform that runs evaluators and monitors on production traces, built on OpenLLMetry, its Apache-2.0 set of OpenTelemetry instrumentations that can also export to other observability backends.
Traceloop announced in 2026 that it is joining ServiceNow; OpenLLMetry remains an Apache-2.0 open-source project.

TruLens
Snowflake · trulens.org
TruLens is an MIT-licensed Python library, created by TruEra and maintained by Snowflake since the acquisition, that instruments AI agents with OpenTelemetry and scores each step with LLM judges such as the RAG triad of context relevance, groundedness, and answer relevance.

W&B Weave
Weights & Biases (CoreWeave) · wandb.ai
W&B Weave is an Apache-2.0 toolkit from Weights & Biases, now part of CoreWeave, for tracing agent sessions, running evaluations with built-in and custom scorers, applying guardrails, and comparing models and prompts in a playground.
How to choose
- 01Tracing and standards
- Support for OpenTelemetry and OpenInference conventions, nested spans for agents, and multi-turn sessions.
- 02Evaluation
- Datasets, experiments, LLM-as-a-judge and code scorers, human annotation, and online evaluation of production traffic.
- 03Deployment
- Self-hosted open source, managed SaaS, or bring-your-own-cloud, which matters when traces contain sensitive data.
- 04Integrations
- SDKs and auto-instrumentation for the frameworks, gateways, and model providers you already use.
- 05Prompt management
- Versioned prompts, playgrounds, and links between prompt versions and evaluation results.
- 06Cost and data controls
- Token and cost tracking, sampling, retention settings, and PII redaction.
Frequently asked questions
What is LLM observability?
- LLM observability is the ability to see what an LLM application did for each request: the prompts sent, the responses returned, the tools and retrievers called, and the latency, tokens, and cost of every step. Observability tools record this as traces and add evaluations, so teams can find failures, detect regressions, and compare changes with evidence.
Langfuse vs LangSmith: what is the difference?
- Langfuse is an open-source platform, MIT-licensed outside its enterprise directories, that teams can self-host or use as a managed cloud, and ClickHouse acquired it in January 2026. LangSmith is LangChain's commercial platform, with managed, bring-your-own-cloud, and self-hosted options. Both trace any framework and support evaluations, and LangSmith integrates closely with LangChain and LangGraph.
Is Langfuse open source?
- Yes. Langfuse's repository is MIT-licensed except for code in its enterprise directories, which carries a separate commercial license. Teams can self-host the core platform, including tracing, evaluations, prompt management, and datasets, or use Langfuse Cloud. ClickHouse acquired Langfuse in January 2026, and the license notice now names ClickHouse as the copyright holder while keeping the MIT terms.
DeepEval vs Ragas: what is the difference?
- DeepEval and Ragas are both Apache-2.0 Python libraries for scoring LLM outputs. DeepEval is organized like a unit-testing framework, with pytest integration and metrics for agents, RAG, chatbots, and voice, and connects to the Confident AI platform. Ragas began with retrieval-augmented generation metrics and adds synthetic test set generation. Many teams use either one to score traces collected by a separate observability tool.
Are there open source LangSmith alternatives?
- Yes. Langfuse, Opik, MLflow, Laminar, Agenta, and OpenLIT can be self-hosted under open-source licenses, and Arize Phoenix is self-hostable under the source-available Elastic License 2.0. Most accept OpenTelemetry traces, so instrumentation such as OpenLLMetry or OpenInference can send the same data to a new backend. LangSmith also offers a self-hosted edition, but it is commercial software.
Related guides
LLM evaluation and observability hubOther categories
Product details change quickly; check each vendor’s site for current features, licensing and pricing.