---
title: "LLM observability and evaluation"
description: "Compare LLM observability and evaluation tools, including Langfuse, LangSmith, Arize Phoenix, Braintrust, DeepEval, and Datadog, by tracing, evals, and license."
type: "platform-category"
canonical_url: "https://theaiengineering.com/platforms/llm-observability"
markdown_url: "https://theaiengineering.com/platforms/llm-observability.md"
topic: "LLM evaluation and observability"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
citation: "The AI Engineering Editors. \"LLM observability and evaluation.\" The AI Engineering. https://theaiengineering.com/platforms/llm-observability"
---

# LLM observability and evaluation

LLM observability is the practice of recording each step of an LLM application, including prompts, model responses, tool calls, retrieval results, latency, token usage, and cost, so engineers can debug failures and measure quality. The tools in this category collect those traces, score them with evaluations, or both.

The category has four groups. Open-source platforms such as Langfuse, Opik, MLflow, Laminar, and Agenta can be self-hosted. Evaluation frameworks such as DeepEval, Ragas, promptfoo, and Inspect run tests in code and CI. Commercial platforms such as LangSmith, Braintrust, Maxim AI, Galileo, and HoneyHive combine tracing, datasets, experiments, and online scoring. Application monitoring vendors, including Datadog, New Relic, Dynatrace, Grafana, and Sentry, add LLM views to existing APM.

Most tools accept OpenTelemetry data, which matters because ownership is changing: ClickHouse acquired Langfuse, Cisco acquired Galileo, OpenAI agreed to buy promptfoo, Traceloop is joining ServiceNow, and Helicone moved to maintenance mode in 2026.

## LLM observability and evaluation compared

| Platform | Open source | Deployment | Best for |
| --- | --- | --- | --- |
| [Agenta](https://agenta.ai) | Yes (MIT) | Self-hosted, Managed cloud | Teams where non-engineers edit and evaluate prompts |
| [Arize AX](https://arize.com) | No | Managed cloud | Enterprise agent monitoring beyond self-hosted Phoenix |
| [Arize Phoenix](https://arize.com/phoenix/) | No | Local, Self-hosted, Managed cloud | Local and self-hosted tracing with OpenTelemetry |
| [Braintrust](https://www.braintrust.dev) | No | Managed cloud | Eval-driven development using production traces |
| [Confident AI and DeepEval](https://www.confident-ai.com) | Yes (Apache-2.0) | Library, Managed cloud | Pytest-style LLM tests with a hosted dashboard |
| [Datadog LLM Observability](https://www.datadoghq.com/products/ai/agent-observability/) | No | Managed cloud | Teams already monitoring infrastructure with Datadog |
| [Dynatrace AI Observability](https://www.dynatrace.com/solutions/ai-observability/) | No | Managed cloud | Dynatrace customers adding LLM monitoring |
| [Evidently](https://www.evidentlyai.com) | Yes (Apache-2.0) | Library, Self-hosted | Teams monitoring both classic ML and LLM systems |
| [Fiddler AI](https://www.fiddler.ai) | No | Managed cloud, Self-hosted | Regulated enterprises monitoring ML and agents together |
| [Galileo](https://galileo.ai) | No | Managed cloud | Turning offline evals into production guardrails |
| [Grafana Cloud AI Observability](https://grafana.com/products/cloud/ai-observability/) | No | Managed cloud | Teams standardized on Grafana and OpenTelemetry |
| [Helicone](https://www.helicone.ai) | Yes (Apache-2.0) | Managed cloud, Self-hosted | Existing users needing proxy-based request logging |
| [HoneyHive](https://www.honeyhive.ai) | No | Managed cloud, Self-hosted | Enterprises with strict data residency for traces |
| [Inspect](https://inspect.aisi.org.uk) | Yes (MIT) | Library, CLI | Research-grade model and agent evaluations |
| [Laminar](https://laminar.sh) | Yes (Apache-2.0) | Managed cloud, Self-hosted | Debugging long-running and browser agents |
| [Langfuse](https://langfuse.com) | Yes (MIT) | Self-hosted, Managed cloud | Self-hosted tracing and prompt management |
| [LangSmith](https://www.langchain.com/langsmith) | No | Managed cloud, Self-hosted | Teams building with LangChain or LangGraph |
| [LangWatch](https://langwatch.ai) | Yes (Apache-2.0) | Managed cloud, Self-hosted | Platform teams governing agents and coding assistants |
| [Lunary](https://lunary.ai) | No | Managed cloud, Self-hosted | Small teams combining logs, prompts, and reviews |
| [Maxim AI](https://www.getmaxim.ai/evals) | No | Managed cloud, Self-hosted | Pre-release agent simulation plus production monitoring |
| [MLflow](https://mlflow.org) | Yes (Apache-2.0) | Self-hosted, Managed cloud, Library | Teams already using MLflow for ML lifecycle work |
| [New Relic AI observability](https://newrelic.com/platform/ai-observability) | No | Managed cloud | Teams already using New Relic APM |
| [OpenLIT](https://openlit.io) | Yes (Apache-2.0) | Self-hosted, Library | Self-hosted OpenTelemetry stack with GPU monitoring |
| [Opik](https://www.comet.com/site/products/opik/) | Yes (Apache-2.0) | Self-hosted, Managed cloud | Open-source evals with built-in prompt optimization |
| [promptfoo](https://www.promptfoo.dev) | Yes (MIT) | CLI, Library, Managed cloud | Prompt and model regression tests in CI |
| [PromptLayer](https://www.promptlayer.com) | No | Managed cloud | Prompt iteration by domain experts |
| [Pydantic Logfire](https://pydantic.dev/logfire) | No | Managed cloud, Self-hosted | Python teams using Pydantic AI or FastAPI |
| [Ragas](https://www.ragas.io) | Yes (Apache-2.0) | Library | RAG metrics and synthetic test data |
| [Raindrop](https://www.raindrop.ai) | No | Managed cloud | Finding silent failures in production agent conversations |
| [Respan (formerly Keywords AI)](https://www.respan.ai) | Yes (Apache-2.0) | Managed cloud, Self-hosted | Gateway and tracing from one vendor |
| [Sentry AI observability](https://sentry.io/solutions/ai-observability/) | No | Managed cloud | Developers already using Sentry for errors |
| [Traceloop and OpenLLMetry](https://www.traceloop.com) | Yes (Apache-2.0) | Library, Managed cloud | OpenTelemetry-native instrumentation for LLM apps |
| [TruLens](https://www.trulens.org) | Yes (MIT) | Library | Scoring RAG pipelines step by step |
| [W&B Weave](https://wandb.ai/site/weave/) | Yes (Apache-2.0) | Managed cloud, Library | Teams already using Weights & Biases |

## Profiles

### Agenta

Agenta is an open-source workspace, MIT-licensed outside its enterprise directories, for building agents through chat and improving them with versioned prompts and tools, test-set evaluations with LLM-as-a-judge or custom evaluators, and tracing of every run.

- Website: https://agenta.ai
- GitHub: https://github.com/Agenta-AI/agenta
- Open source: Yes (MIT)
- Deployment: Self-hosted, Managed cloud
- Languages: Python, TypeScript
- Best for: Teams where non-engineers edit and evaluate prompts
- Permalink: https://theaiengineering.com/platforms/llm-observability#agenta

### Arize AX

Arize AX is Arize AI's managed platform for observing, evaluating, and improving AI agents, with tracing built on OpenInference, the open convention Arize created, plus span, trace, and session evaluations, experiments, and production monitoring.

- Company: Arize AI
- Website: https://arize.com
- Open source: No
- Deployment: Managed cloud
- Languages: Python, TypeScript
- Best for: Enterprise agent monitoring beyond self-hosted Phoenix
- Permalink: https://theaiengineering.com/platforms/llm-observability#arize-ax

### Arize Phoenix

Arize Phoenix is a self-hostable, source-available tool under the Elastic License 2.0 for tracing LLM applications and agents with OpenTelemetry and OpenInference, running evals, and comparing experiments on datasets.

- Company: Arize AI
- Website: https://arize.com/phoenix/
- GitHub: https://github.com/Arize-ai/phoenix
- Open source: No
- Deployment: Local, Self-hosted, Managed cloud
- Languages: Python, TypeScript
- Best for: Local and self-hosted tracing with OpenTelemetry
- Permalink: https://theaiengineering.com/platforms/llm-observability#arize-phoenix

### Braintrust

Braintrust is a commercial platform for evaluating and observing AI agents that traces production traffic, turns traces into versioned datasets, scores outputs with LLM, code, or human scorers, and catches regressions before release.

- Website: https://www.braintrust.dev
- Open source: No
- Deployment: Managed cloud
- Languages: Python, TypeScript, Go, Ruby, C#
- Best for: Eval-driven development using production traces
- Permalink: https://theaiengineering.com/platforms/llm-observability#braintrust

### Confident AI and DeepEval

DeepEval is an Apache-2.0 Python framework for testing LLM applications with plug-and-play metrics for agents, RAG, multi-turn chat, and voice, and Confident AI is the commercial platform from the same team for running those evals and observing production traces.

- Company: Confident AI
- Website: https://www.confident-ai.com
- GitHub: https://github.com/confident-ai/deepeval
- Open source: Yes (Apache-2.0)
- Deployment: Library, Managed cloud
- Languages: Python
- Best for: Pytest-style LLM tests with a hosted dashboard
- Permalink: https://theaiengineering.com/platforms/llm-observability#confident-ai

### Datadog LLM Observability

Datadog LLM Observability, part of Datadog's Agent Observability offering, traces prompts, retrieval, and tool calls in agent workflows, runs offline experiments and built-in or custom evaluations, and flags hallucinations, prompt injection, and PII exposure.

- Company: Datadog
- Website: https://www.datadoghq.com/products/ai/agent-observability/
- Open source: No
- Deployment: Managed cloud
- Best for: Teams already monitoring infrastructure with Datadog
- Permalink: https://theaiengineering.com/platforms/llm-observability#datadog-llm-observability

### Dynatrace AI Observability

Dynatrace AI Observability monitors AI applications, LLMs, and agents from the user experience through orchestration to models and infrastructure, tracking token usage, latency, and cost and detecting hallucinations, prompt injection, PII leakage, and toxicity.

- Company: Dynatrace
- Website: https://www.dynatrace.com/solutions/ai-observability/
- Open source: No
- Deployment: Managed cloud
- Best for: Dynatrace customers adding LLM monitoring
- Permalink: https://theaiengineering.com/platforms/llm-observability#dynatrace-ai-observability

### Evidently

Evidently is an Apache-2.0 Python framework for evaluating, testing, and monitoring ML and LLM systems, with a library of built-in metrics that combines rules, classifiers, and LLM judges for testing AI applications, RAG systems, and multi-agent workflows.

- Company: Evidently AI
- Website: https://www.evidentlyai.com
- GitHub: https://github.com/evidentlyai/evidently
- Open source: Yes (Apache-2.0)
- Deployment: Library, Self-hosted
- Languages: Python
- Best for: Teams monitoring both classic ML and LLM systems
- Permalink: https://theaiengineering.com/platforms/llm-observability#evidently

### Fiddler AI

Fiddler AI is a commercial control plane for enterprise AI agents and models that combines evaluation, monitoring, guardrail policy enforcement, governance, and ML observability, with SaaS, VPC, and AWS GovCloud deployments.

- Website: https://www.fiddler.ai
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Best for: Regulated enterprises monitoring ML and agents together
- Permalink: https://theaiengineering.com/platforms/llm-observability#fiddler-ai

### Galileo

Galileo is an evaluation and observability platform for AI agents with built-in and custom evaluators, an insights engine that finds failure modes, and Luna models distilled from evaluators to monitor production traffic and drive guardrails.

Cisco completed its acquisition of Galileo on May 22, 2026, and is bringing its capabilities into Splunk Observability Cloud.

- Company: Cisco
- Website: https://galileo.ai
- Headquarters: Burlingame, California, United States
- Open source: No
- Deployment: Managed cloud
- Best for: Turning offline evals into production guardrails
- Permalink: https://theaiengineering.com/platforms/llm-observability#galileo

### Grafana Cloud AI Observability

Grafana Cloud AI Observability, including Agent Observability, traces production agents with OpenTelemetry, runs live evaluations for hallucinations and quality drift, and tracks latency, tokens, and cost across agents, runs, and models.

- Company: Grafana Labs
- Website: https://grafana.com/products/cloud/ai-observability/
- Open source: No
- Deployment: Managed cloud
- Best for: Teams standardized on Grafana and OpenTelemetry
- Permalink: https://theaiengineering.com/platforms/llm-observability#grafana-ai-observability

### Helicone

Helicone is an Apache-2.0 LLM observability platform and AI gateway that logs requests through a proxy or SDK and shows sessions, users, and costs, with prompt management, datasets, a playground, rate limits, and alerts.

Mintlify acquired Helicone in March 2026 and moved it to maintenance mode; Helicone's site now states that it is no longer part of Mintlify.

- Website: https://www.helicone.ai
- GitHub: https://github.com/Helicone/helicone
- Open source: Yes (Apache-2.0)
- Deployment: Managed cloud, Self-hosted
- Languages: TypeScript
- Best for: Existing users needing proxy-based request logging
- Permalink: https://theaiengineering.com/platforms/llm-observability#helicone

### HoneyHive

HoneyHive is a commercial platform for tracing, evaluating, monitoring, and improving production AI agents across teams, with OpenTelemetry ingestion and hybrid or fully self-hosted deployments that keep trace data inside the customer's network.

- Website: https://www.honeyhive.ai
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Best for: Enterprises with strict data residency for traces
- Permalink: https://theaiengineering.com/platforms/llm-observability#honeyhive

### Inspect

Inspect is an MIT-licensed Python framework from the UK AI Security Institute for building and running LLM evaluations, with components for datasets, solvers, scorers, tool use, agents, and sandboxed execution, plus a log viewer for results.

- Company: UK AI Security Institute
- Website: https://inspect.aisi.org.uk
- GitHub: https://github.com/UKGovernmentBEIS/inspect_ai
- Open source: Yes (MIT)
- Deployment: Library, CLI
- Languages: Python
- Best for: Research-grade model and agent evaluations
- Permalink: https://theaiengineering.com/platforms/llm-observability#inspect-ai

### Laminar

Laminar is an Apache-2.0 observability platform for AI agents that traces runs, extracts agent inputs automatically, evaluates results, and surfaces failures, with integrations for agent SDKs and browser agents and Helm charts for self-hosting.

- Website: https://laminar.sh
- GitHub: https://github.com/lmnr-ai/lmnr
- Open source: Yes (Apache-2.0)
- Deployment: Managed cloud, Self-hosted
- Languages: TypeScript, Python
- Best for: Debugging long-running and browser agents
- Permalink: https://theaiengineering.com/platforms/llm-observability#laminar

### Langfuse

Langfuse is an open-source platform for tracing, evaluating, and improving LLM applications and agents, with prompt management and datasets, MIT-licensed outside its enterprise directories and available self-hosted or as Langfuse Cloud.

ClickHouse acquired Langfuse in January 2026, and the project remains MIT-licensed outside its enterprise directories.

- Company: ClickHouse
- Website: https://langfuse.com
- GitHub: https://github.com/langfuse/langfuse
- Open source: Yes (MIT)
- Deployment: Self-hosted, Managed cloud
- Languages: Python, TypeScript
- Best for: Self-hosted tracing and prompt management
- Permalink: https://theaiengineering.com/platforms/llm-observability#langfuse

### LangSmith

LangSmith is LangChain's commercial platform for tracing, monitoring, and evaluating LLM and agent applications; it works with or without LangChain, accepts OpenTelemetry data, and runs as managed cloud, bring-your-own-cloud, or self-hosted.

- Company: LangChain
- Website: https://www.langchain.com/langsmith
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Languages: Python, TypeScript, Go, Java
- Best for: Teams building with LangChain or LangGraph
- Permalink: https://theaiengineering.com/platforms/llm-observability#langsmith

### LangWatch

LangWatch is an Apache-2.0 platform for AI platform teams that traces, tests, routes, and governs LLM calls across a company, from internal agents to the coding assistants engineers use, with evaluations and observability in one place.

- Website: https://langwatch.ai
- GitHub: https://github.com/langwatch/langwatch
- Open source: Yes (Apache-2.0)
- Deployment: Managed cloud, Self-hosted
- Languages: TypeScript, Python
- Best for: Platform teams governing agents and coding assistants
- Permalink: https://theaiengineering.com/platforms/llm-observability#langwatch

### Lunary

Lunary is an observability and prompt management platform for LLM applications that logs traces and errors, versions prompt templates with A/B testing, supports human review of responses, and reports usage and cost, in the cloud or self-hosted.

- Website: https://lunary.ai
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Languages: Python, TypeScript
- Best for: Small teams combining logs, prompts, and reviews
- Permalink: https://theaiengineering.com/platforms/llm-observability#lunary

### Maxim AI

Maxim AI is a commercial platform for simulating, evaluating, and observing AI agents, with AI-generated scenario simulations, LLM-as-a-judge, programmatic, statistical, and human evaluators, and real-time trace logging with alerts.

- Website: https://www.getmaxim.ai/evals
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Languages: Python, TypeScript, Java, Go
- Best for: Pre-release agent simulation plus production monitoring
- Permalink: https://theaiengineering.com/platforms/llm-observability#maxim-ai

### MLflow

MLflow is an open-source platform for agents, LLMs, and machine learning models that adds OpenTelemetry-compatible tracing, LLM evaluation, prompt management, and an AI gateway to its experiment tracking and model registry, self-hosted or as managed services.

- Website: https://mlflow.org
- GitHub: https://github.com/mlflow/mlflow
- Open source: Yes (Apache-2.0)
- Deployment: Self-hosted, Managed cloud, Library
- Languages: Python, TypeScript
- Best for: Teams already using MLflow for ML lifecycle work
- Permalink: https://theaiengineering.com/platforms/llm-observability#mlflow

### New Relic AI observability

New Relic's AI observability extends its full-stack monitoring to AI applications, showing response time, quality, and token metrics next to APM signals, mapping interactions between agents and tools, and integrating with OpenAI, vector databases, and LangChain.

- Company: New Relic
- Website: https://newrelic.com/platform/ai-observability
- Open source: No
- Deployment: Managed cloud
- Best for: Teams already using New Relic APM
- Permalink: https://theaiengineering.com/platforms/llm-observability#new-relic-ai-observability

### OpenLIT

OpenLIT is an Apache-2.0 platform built on OpenTelemetry for agent observability, evaluations, guardrails, prompt management, and cost and GPU monitoring, covering both custom agents and coding agents, with auto-instrumentation SDKs and a self-hostable dashboard.

- Website: https://openlit.io
- GitHub: https://github.com/openlit/openlit
- Open source: Yes (Apache-2.0)
- Deployment: Self-hosted, Library
- Languages: Python, TypeScript
- Best for: Self-hosted OpenTelemetry stack with GPU monitoring
- Permalink: https://theaiengineering.com/platforms/llm-observability#openlit

### Opik

Opik is an Apache-2.0 platform from Comet for tracing agents, scoring traces with LLM-as-a-judge metrics, running test suites, optimizing prompts, and applying guardrails, available self-hosted or on Comet's cloud.

- Company: Comet
- Website: https://www.comet.com/site/products/opik/
- GitHub: https://github.com/comet-ml/opik
- Open source: Yes (Apache-2.0)
- Deployment: Self-hosted, Managed cloud
- Languages: Python, TypeScript
- Best for: Open-source evals with built-in prompt optimization
- Permalink: https://theaiengineering.com/platforms/llm-observability#opik

### promptfoo

promptfoo is an MIT-licensed CLI and library for testing prompts, agents, and RAG pipelines against test cases and comparing models side by side, with red teaming and a commercial platform for teams.

OpenAI agreed to acquire Promptfoo in March 2026 and said it would keep developing the MIT-licensed open-source project.

- Website: https://www.promptfoo.dev
- GitHub: https://github.com/promptfoo/promptfoo
- Open source: Yes (MIT)
- Deployment: CLI, Library, Managed cloud
- Languages: TypeScript
- Best for: Prompt and model regression tests in CI
- Permalink: https://theaiengineering.com/platforms/llm-observability#promptfoo

### PromptLayer

PromptLayer is a prompt management platform with a prompt registry and visual editor, LLM evaluations and regression sets, and production tracing and logs, designed so domain experts can iterate on prompts without changing the codebase.

- Website: https://www.promptlayer.com
- Open source: No
- Deployment: Managed cloud
- Best for: Prompt iteration by domain experts
- Permalink: https://theaiengineering.com/platforms/llm-observability#promptlayer

### Pydantic Logfire

Pydantic Logfire is an observability platform built on OpenTelemetry that traces LLM calls and agents together with the rest of an application, with an MIT-licensed Python SDK and support for JavaScript, Rust, Go, Java, and other OpenTelemetry languages.

- Company: Pydantic
- Website: https://pydantic.dev/logfire
- GitHub: https://github.com/pydantic/logfire
- Open source: No
- Deployment: Managed cloud, Self-hosted
- Languages: Python, TypeScript, Rust
- Best for: Python teams using Pydantic AI or FastAPI
- Permalink: https://theaiengineering.com/platforms/llm-observability#pydantic-logfire

### Ragas

Ragas is an Apache-2.0 Python library for evaluating LLM applications with LLM-based and traditional metrics, generating synthetic test sets, and building feedback loops from production data; it began with metrics for retrieval-augmented generation.

- Website: https://www.ragas.io
- GitHub: https://github.com/vibrantlabsai/ragas
- Open source: Yes (Apache-2.0)
- Deployment: Library
- Languages: Python
- Best for: RAG metrics and synthetic test data
- Permalink: https://theaiengineering.com/platforms/llm-observability#ragas

### Raindrop

Raindrop is a commercial monitoring platform for AI agents that traces every run, detects silent failures in production conversations, and lets teams simulate changes before shipping them.

- Website: https://www.raindrop.ai
- Open source: No
- Deployment: Managed cloud
- Best for: Finding silent failures in production agent conversations
- Permalink: https://theaiengineering.com/platforms/llm-observability#raindrop

### Respan (formerly Keywords AI)

Respan, formerly Keywords AI, pairs an open-source AI gateway written in Rust and licensed Apache-2.0 with a hosted platform for tracing, monitoring, and evaluating every agent run.

Keywords AI rebranded as Respan in 2026 and released its Rust gateway as open source.

- Company: Respan
- Website: https://www.respan.ai
- GitHub: https://github.com/respanai/respan-gateway
- Open source: Yes (Apache-2.0)
- Deployment: Managed cloud, Self-hosted
- Languages: Rust
- Best for: Gateway and tracing from one vendor
- Permalink: https://theaiengineering.com/platforms/llm-observability#respan

### Sentry AI observability

Sentry's AI observability traces agent runs, LLM calls, and tool calls in production applications and flags errors, latency, and token cost spikes, with SDK integrations for the OpenAI, Anthropic, Vercel AI, LangChain, and Pydantic AI libraries.

- Company: Sentry
- Website: https://sentry.io/solutions/ai-observability/
- Open source: No
- Deployment: Managed cloud
- Best for: Developers already using Sentry for errors
- Permalink: https://theaiengineering.com/platforms/llm-observability#sentry-ai-observability

### Traceloop and OpenLLMetry

Traceloop is an LLM reliability platform that runs evaluators and monitors on production traces, built on OpenLLMetry, its Apache-2.0 set of OpenTelemetry instrumentations that can also export to other observability backends.

Traceloop announced in 2026 that it is joining ServiceNow; OpenLLMetry remains an Apache-2.0 open-source project.

- Company: Traceloop
- Website: https://www.traceloop.com
- GitHub: https://github.com/traceloop/openllmetry
- Open source: Yes (Apache-2.0)
- Deployment: Library, Managed cloud
- Languages: Python, TypeScript
- Best for: OpenTelemetry-native instrumentation for LLM apps
- Permalink: https://theaiengineering.com/platforms/llm-observability#traceloop

### TruLens

TruLens is an MIT-licensed Python library, created by TruEra and maintained by Snowflake since the acquisition, that instruments AI agents with OpenTelemetry and scores each step with LLM judges such as the RAG triad of context relevance, groundedness, and answer relevance.

- Company: Snowflake
- Website: https://www.trulens.org
- GitHub: https://github.com/truera/trulens
- Open source: Yes (MIT)
- Deployment: Library
- Languages: Python
- Best for: Scoring RAG pipelines step by step
- Permalink: https://theaiengineering.com/platforms/llm-observability#trulens

### W&B Weave

W&B Weave is an Apache-2.0 toolkit from Weights & Biases, now part of CoreWeave, for tracing agent sessions, running evaluations with built-in and custom scorers, applying guardrails, and comparing models and prompts in a playground.

- Company: Weights & Biases (CoreWeave)
- Website: https://wandb.ai/site/weave/
- GitHub: https://github.com/wandb/weave
- Open source: Yes (Apache-2.0)
- Deployment: Managed cloud, Library
- Languages: Python, TypeScript
- Best for: Teams already using Weights & Biases
- Permalink: https://theaiengineering.com/platforms/llm-observability#wandb-weave

## How to choose

1. **Tracing and standards**: Support for OpenTelemetry and OpenInference conventions, nested spans for agents, and multi-turn sessions.
2. **Evaluation**: Datasets, experiments, LLM-as-a-judge and code scorers, human annotation, and online evaluation of production traffic.
3. **Deployment**: Self-hosted open source, managed SaaS, or bring-your-own-cloud, which matters when traces contain sensitive data.
4. **Integrations**: SDKs and auto-instrumentation for the frameworks, gateways, and model providers you already use.
5. **Prompt management**: Versioned prompts, playgrounds, and links between prompt versions and evaluation results.
6. **Cost and data controls**: Token and cost tracking, sampling, retention settings, and PII redaction.

## Frequently asked questions

### What is LLM observability?

LLM observability is the ability to see what an LLM application did for each request: the prompts sent, the responses returned, the tools and retrievers called, and the latency, tokens, and cost of every step. Observability tools record this as traces and add evaluations, so teams can find failures, detect regressions, and compare changes with evidence.

### Langfuse vs LangSmith: what is the difference?

Langfuse is an open-source platform, MIT-licensed outside its enterprise directories, that teams can self-host or use as a managed cloud, and ClickHouse acquired it in January 2026. LangSmith is LangChain's commercial platform, with managed, bring-your-own-cloud, and self-hosted options. Both trace any framework and support evaluations, and LangSmith integrates closely with LangChain and LangGraph.

### Is Langfuse open source?

Yes. Langfuse's repository is MIT-licensed except for code in its enterprise directories, which carries a separate commercial license. Teams can self-host the core platform, including tracing, evaluations, prompt management, and datasets, or use Langfuse Cloud. ClickHouse acquired Langfuse in January 2026, and the license notice now names ClickHouse as the copyright holder while keeping the MIT terms.

### DeepEval vs Ragas: what is the difference?

DeepEval and Ragas are both Apache-2.0 Python libraries for scoring LLM outputs. DeepEval is organized like a unit-testing framework, with pytest integration and metrics for agents, RAG, chatbots, and voice, and connects to the Confident AI platform. Ragas began with retrieval-augmented generation metrics and adds synthetic test set generation. Many teams use either one to score traces collected by a separate observability tool.

### Are there open source LangSmith alternatives?

Yes. Langfuse, Opik, MLflow, Laminar, Agenta, and OpenLIT can be self-hosted under open-source licenses, and Arize Phoenix is self-hostable under the source-available Elastic License 2.0. Most accept OpenTelemetry traces, so instrumentation such as OpenLLMetry or OpenInference can send the same data to a new backend. LangSmith also offers a self-hosted edition, but it is commercial software.

## Related guides

- [LLM observability: what to capture, which metrics, and tools](https://theaiengineering.com/articles/llm-observability): LLM observability guide: what to capture in traces and spans, metrics to track, OpenTelemetry GenAI conventions, tool categories, and keeping prompts private.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.

Product details change quickly; check each vendor's site for current features, licensing and pricing.

## Cite this page

The AI Engineering Editors. "LLM observability and evaluation." The AI Engineering. https://theaiengineering.com/platforms/llm-observability

- Title: LLM observability and evaluation
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- URL: https://theaiengineering.com/platforms/llm-observability
- Markdown: https://theaiengineering.com/platforms/llm-observability.md
