---
title: "LiteLLM alternatives: 9 open-source and managed AI gateways compared"
description: "LiteLLM alternatives compared: Bifrost, Agent Router, agentgateway, Kong, APISIX, Portkey, Cloudflare, Vercel, and OpenRouter, plus how to migrate off LiteLLM."
summary: "Teams replace LiteLLM to cut proxy overhead, drop the Postgres and Redis footprint, get controls its Enterprise license reserves, or stop running a proxy. This comparison covers nine self-hosted and managed gateways by license, deployment, MCP support, and governance, plus a migration plan."
type: "comparison"
canonical_url: "https://theaiengineering.com/articles/litellm-alternatives"
markdown_url: "https://theaiengineering.com/articles/litellm-alternatives.md"
topic: "AI gateways and LLM routing"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-11-17"
updated: "2026-11-17"
keywords: ["litellm alternatives","litellm alternative","llm proxy"]
citation: "The AI Engineering Editors. \"LiteLLM alternatives: 9 open-source and managed AI gateways compared.\" The AI Engineering, updated Nov 17, 2026. https://theaiengineering.com/articles/litellm-alternatives"
---

# LiteLLM alternatives: 9 open-source and managed AI gateways compared

## TL;DR

- LiteLLM is an MIT-licensed Python SDK and self-hosted LLM proxy from BerriAI; teams evaluate alternatives over proxy overhead, the Postgres and Redis footprint, features reserved for its Enterprise license, two 2026 supply chain incidents, and the lack of a hosted option.
- Bifrost (Go, Apache 2.0), Agent Router (formerly Envoy AI Gateway), and agentgateway (Rust, a Linux Foundation project) are self-hosted alternatives that govern MCP tool traffic as well as model calls.
- Teams that already run Kong or Apache APISIX can add AI plugins to that gateway, but Kong's MCP proxy, semantic cache, and multi-model load balancing are not in its open-source repository.
- Cloudflare AI Gateway, Vercel AI Gateway, and OpenRouter remove the proxy to operate, but none of the three lists MCP gateway features in its AI gateway docs; Prisma AIRS AI Gateway (formerly Portkey) pairs a hosted control plane with one.
- Every gateway here accepts OpenAI-format requests, so the cheapest migration test is to run the new gateway beside LiteLLM, split a small share of traffic to it, and load test both against the same mock upstream.

LiteLLM alternatives are other AI gateways that do LiteLLM's job, one API in front of many model providers with keys, budgets, routing, and logs, using a different runtime, license, or operating model. LiteLLM itself is an open-source Python SDK and LLM proxy from BerriAI. This comparison explains why teams evaluate replacements, states the selection criteria, compares nine self-hosted and managed options, and closes with a migration plan. It is part of our [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) hub.

## What is LiteLLM?

LiteLLM is an open-source Python SDK and self-hosted proxy server that puts more than 100 model providers behind one [OpenAI-compatible API](https://theaiengineering.com/glossary/openai-compatible-api), with cost tracking, budgets, and routing.

The project has two parts. The SDK is a library that application code calls directly. The proxy server is a shared [AI gateway](https://theaiengineering.com/glossary/ai-gateway) that platform teams run for many applications, and it is the part the alternatives in this comparison replace. The proxy issues [virtual keys](https://theaiengineering.com/glossary/virtual-key), tracks spend per key and team, enforces budgets and [rate limits](https://theaiengineering.com/glossary/rate-limiting), load balances across deployments, falls back between models, and includes an MCP gateway with permissions by key, team, or organization.

The repository is MIT-licensed except for the `enterprise` directory, which carries a separate license. In June 2026, BerriAI [announced a migration of the gateway to Rust](https://docs.litellm.ai/blog/litellm-rust-launch) and promised that config, database schema, client API, and the Python SDK interface would stay the same. Our explainer on [what an AI gateway is](https://theaiengineering.com/articles/what-is-an-ai-gateway) covers the general architecture these tools share.

## Why teams look for LiteLLM alternatives

Teams look for LiteLLM alternatives for six documented reasons: per-request overhead and memory on the Python proxy path, the Postgres and Redis footprint of a production deployment, features that require an Enterprise license, two supply chain incidents in 2026, the absence of a hosted LiteLLM service, and how model and agent traffic should share one gateway.

### Proxy overhead under concurrency

LiteLLM's own Rust announcement measured about 7.5 ms of added latency per request and a peak of about 359 MB of memory on the Python forwarding path, and described CPU and memory rising with concurrency under load. The Rust migration is LiteLLM's answer, and it is partly shipped: the v1.110.0 release candidates run the router on the Rust core, and the [migration status page](https://docs.litellm.ai/rust-migration) targets running the common `/v1/messages` path end to end in Rust by December 1, 2026. As of November 2026, that page still listed chat completions translation for most providers on the Python path.

Published benchmarks disagree, and each was run by a vendor. In Maxim AI's [published benchmark](https://www.getmaxim.ai/bifrost/resources/benchmarks) (t3.medium instances on AWS EC2 in us-east-1, 500 virtual users for 60 seconds), Bifrost served 424 requests per second against 44.84 for LiteLLM's Python proxy, with P99 latency of 1.68 seconds versus 90.72 seconds, peak memory of 120 MB versus 372 MB, and a success rate of 100% versus 88.78%. LiteLLM's July 2026 forwarding-only benchmark, with logging, spend tracking, and persistence disabled, reported 0.7 ms of added P99 latency for its Rust gateway, 2.3 ms for Portkey, 4.5 ms for Bifrost, and 257.7 ms for its Python v1 path. Neither setup is your workload, so the migration section below describes how to run your own test.

### Operational footprint

LiteLLM's virtual keys require a Postgres database, and its [production guide](https://docs.litellm.ai/docs/proxy/prod) says to run Redis 7.0 or newer as soon as there is more than one proxy instance. The same guide sets 1 vCPU and 4 GiB of memory per worker as a floor and a default of 10 database connections per worker, which multiply across instances. LiteLLM's own 3,000-requests-per-second benchmark ran 33 gateway pods with 4 workers each, plus Postgres and Redis.

### Features in the Enterprise tier

LiteLLM's [Enterprise page](https://docs.litellm.ai/docs/enterprise) lists the features its license key enables. They include SSO for the admin UI beyond 5 users, audit logs, role-based access control, secret manager integrations, IP allowlists, tag-based budgets, model-specific budgets per virtual key, team-based logging controls, and [guardrails](https://theaiengineering.com/glossary/guardrails) scoped per key or team. Several built-in guardrail integrations, including Lakera prompt injection and OpenAI moderation, also require the license. Enterprise has no published price.

### Two supply chain incidents in 2026

LiteLLM published advisories for two compromised releases in 2026. On March 24, versions 1.82.7 and 1.82.8 on PyPI contained a credential stealer; LiteLLM said the official Proxy Docker image was not affected and that it believed the compromise originated from the Trivy dependency in its CI/CD scanning workflow. On [October 21](https://docs.litellm.ai/blog/security-incident-oct-21-2026), version 1.107.0 reached PyPI and the `v1.107.0`, `latest`, `main-latest`, and `main-stable` tags on Docker Hub and GHCR, built from an attacker-controlled repository; LiteLLM said customers on its private enterprise release were not affected. Both advisories list remediation steps.

### No hosted option, and agent traffic

LiteLLM's pricing page, checked on November 16, 2026, lists only self-hosted tiers, so teams that want no proxy to operate must look elsewhere. Agent traffic raises a different question. LiteLLM already proxies MCP servers, so the comparison point is how each gateway handles [Model Context Protocol](https://theaiengineering.com/glossary/model-context-protocol) tool filtering, per-user OAuth, and [A2A](https://theaiengineering.com/glossary/a2a-protocol) agent traffic alongside model calls.

## How we chose and compared the alternatives

Each alternative was checked against six criteria from our [AI gateways directory](https://theaiengineering.com/platforms/ai-gateways), using its official documentation, repository, and announcements as of November 16, 2026. Projects that are archived or in maintenance mode were left out, and every entry's current name, license, and ownership was confirmed at the source.

1. **Deployment:** self-hosted in your VPC or Kubernetes cluster, managed SaaS, or hybrid with a hosted control plane.
2. **Provider coverage and API format:** the providers supported and whether the gateway accepts OpenAI, Anthropic, or other native request formats.
3. **Routing and reliability:** [fallbacks](https://theaiengineering.com/glossary/model-failover), retries, [load balancing](https://theaiengineering.com/glossary/llm-load-balancing) across keys and regions, and rule-based routing.
4. **Governance:** virtual keys, per-team budgets, rate limits, audit logs, and single sign-on.
5. **Guardrails and MCP:** PII redaction, prompt-injection checks, and support for proxying MCP servers and tool calls.
6. **Overhead and observability:** latency added per request, plus [OpenTelemetry](https://theaiengineering.com/glossary/opentelemetry) export and usage dashboards.

Each entry covers what it is, deployment and license, strengths, trade-offs, best fit, and migration from LiteLLM.

## LiteLLM alternatives at a glance

The table summarizes license, deployment model, implementation language, MCP support, and governance for each alternative, based on official documentation and repositories as of November 16, 2026. Open-source gateways run where you deploy them; managed services run on the vendor's network; Kong and Apache APISIX extend existing API gateways with AI plugins.

| Gateway | License | Deployment | Language | MCP support | Governance |
|---|---|---|---|---|---|
| Bifrost | Apache 2.0 | Self-hosted: npx, Docker, Kubernetes | Go | MCP gateway with agent mode and Code Mode; A2A agent gateway | Virtual keys, budgets, rate limits; SSO and RBAC in Enterprise |
| Agent Router | Apache 2.0 | Self-hosted on Kubernetes with Envoy Gateway, or the `aigw` CLI | Go control plane, Envoy data plane | One router for MCP tools, filtered by requester | Token limits per team, app, or model |
| agentgateway | Apache 2.0 | Self-hosted binary or Kubernetes | Rust | MCP gateway with tool federation and OAuth; A2A | Budgets, rate limiting, RBAC |
| Kong AI Gateway | Kong Gateway OSS is Apache 2.0; AI Gateway Enterprise is commercial | Self-hosted, Konnect, or Kong-hosted | Lua | AI MCP Proxy plugin (Enterprise, 3.12+) | Consumers, token rate limits (Enterprise) |
| Apache APISIX | Apache 2.0 | Self-hosted | Lua | `openapi-to-mcp` turns HTTP APIs into MCP tools | Consumers, token quotas |
| Prisma AIRS AI Gateway (formerly Portkey) | Core gateway MIT; platform commercial | SaaS, hybrid data plane in your VPC, or open-source gateway | TypeScript | MCP Gateway | RBAC, budgets, SSO |
| Cloudflare AI Gateway | Proprietary | Managed | Not applicable | Not listed among AI Gateway features | Spend limits, rate limits, gateway auth |
| Vercel AI Gateway | Proprietary | Managed | Not applicable | Not an MCP gateway | Budgets per team, project, key, or member |
| OpenRouter | Proprietary | Managed | Not applicable | Not an MCP gateway | Guardrails with budgets and model allowlists |

## Self-hosted open-source gateways

Self-hosted open-source gateways are the closest replacements for a LiteLLM proxy, because they run in your own network with your own provider keys. Bifrost, Agent Router, and agentgateway all accept OpenAI-format requests, govern MCP tool traffic as well as model calls, and differ mainly in runtime, configuration model, and how much ships outside an enterprise edition.

### 1. Bifrost

Bifrost is an open-source AI gateway from Maxim AI, written in Go, that acts as an LLM gateway, an MCP gateway, and an agent gateway behind one OpenAI-compatible API. Its latest HTTP release on November 16, 2026 was v2.5.1, published November 10.

**Deployment and license:** Apache 2.0. Start it with `npx -y @maximhq/bifrost` or `docker run -p 8080:8080 maximhq/bifrost`, or deploy it to Kubernetes with Helm; Maxim describes it as a single binary. Configuration lives in SQLite by default or in PostgreSQL. Clustering, SSO, guardrails, and adaptive load balancing are in the paid Enterprise tier.

**Strengths:**
- [Drop-in SDK endpoints](https://docs.getbifrost.ai/features/drop-in-replacement) for the OpenAI, Anthropic, and LiteLLM SDKs, and a support matrix of more than 30 providers, including vLLM, SGLang, and Ollama.
- [Retries and fallbacks](https://docs.getbifrost.ai/features/retries-and-fallbacks) with exponential backoff, plus weighted [load balancing across keys](https://docs.getbifrost.ai/features/keys-management).
- [Virtual keys](https://docs.getbifrost.ai/features/governance/virtual-keys) with budgets, rate limits, allowed models and providers, and MCP tool filtering, grouped under teams and customers.
- An [MCP gateway](https://docs.getbifrost.ai/mcp/overview) that connects to servers over STDIO, HTTP, or SSE and exposes them at one `/mcp` endpoint, with agent mode and [Code Mode](https://docs.getbifrost.ai/mcp/code-mode), for which Maxim reports up to 92.8% fewer input tokens.
- [Semantic caching](https://docs.getbifrost.ai/features/semantic-caching) with exact-match and similarity lookups on Redis or Valkey, Qdrant, Weaviate, or Pinecone.
- Maxim's [benchmarking docs](https://docs.getbifrost.ai/benchmarking/getting-started) report 11 µs of added overhead at 5,000 requests per second on a t3.xlarge with mocked provider calls; these and the LiteLLM comparison above are Maxim's own numbers.

**Trade-offs:** Guardrails, cluster mode, SSO, and secret manager integrations require Enterprise. The semantic cache needs a vector store, and caching activates only when a request carries a cache key or a default key is configured. Bifrost's native endpoints take `provider/model` strings, so check how LiteLLM `model_name` aliases map after import.

**Best for:** Teams that want to self-host one gateway for model, MCP, and agent traffic, with a direct import path from an existing LiteLLM proxy.

**Migrating from LiteLLM:** Bifrost's [LiteLLM migration guide](https://docs.getbifrost.ai/migration-guides/litellm) documents a CLI that reads a LiteLLM proxy and recreates its providers, keys, budgets, rate limits, organizations (as customers), teams, users, and virtual keys. New `sk-bf-*` key values are generated, partial wildcards such as `openai/gpt-4*` are skipped and reported, and the tool never writes to LiteLLM. Code that uses the LiteLLM SDK can point at Bifrost's `/litellm` endpoint instead.

### 2. Agent Router (formerly Envoy AI Gateway)

Agent Router is the open-source control plane for AI and agent traffic that was previously named Envoy AI Gateway, now an Agentic AI Foundation project. It gives applications one OpenAI-compatible API for hosted models, self-hosted inference, and MCP servers, enforced by Envoy and Envoy Gateway.

**Deployment and license:** Apache 2.0, written in Go. Run it locally with `aigw run`, which serves an OpenAI-compatible endpoint on port 1975, or deploy it on Kubernetes with Envoy Gateway. The [Agentic AI Foundation announcement](https://aaif.io/blog/agent-router-joins-aaif) from September 2026 says existing deployments do not need to migrate, and the [repository README](https://github.com/theagentrouter/agent-router) confirms the CRDs, `aigw` CLI, images, and Helm charts kept their names. The latest release was v1.2.1 on November 6, 2026.

**Strengths:** Routing to hosted providers and self-hosted vLLM, token limits per team, app, or model, provider fallback, model name virtualization, one MCP tool catalog filtered per requester, and telemetry that follows OpenTelemetry GenAI conventions. A two-tier pattern adds an endpoint picker for self-hosted model clusters.

**Trade-offs:** Production configuration is Kubernetes custom resources (`AIGatewayRoute`, `AIServiceBackend`, `BackendSecurityPolicy`) on top of Envoy Gateway, which suits platform teams but is more to operate than a single config file for teams not already on Kubernetes.

**Best for:** Kubernetes platform teams that already run Envoy or Envoy Gateway.

**Migrating from LiteLLM:** Each LiteLLM `model_list` entry becomes an `AIServiceBackend` and a route rule, provider credentials move into `BackendSecurityPolicy` resources, and token limits replace LiteLLM's `tpm_limit` values. Applications change only their base URL.

### 3. agentgateway

agentgateway is an open-source proxy hosted by the Linux Foundation that handles LLM, MCP, agent-to-agent, and ordinary HTTP and gRPC traffic in one data plane. Solo.io contributed the project and sells Solo Enterprise for agentgateway as a commercial edition.

**Deployment and license:** Apache 2.0, written in Rust. It runs as a standalone binary with flat YAML configuration or on Kubernetes with its controller and Gateway API support. The latest release was v1.7.0 on October 27, 2026.

**Strengths:** The [project README](https://github.com/agentgateway/agentgateway) lists an OpenAI-compatible LLM gateway with budget controls, load balancing, and failover; an [MCP gateway](https://theaiengineering.com/glossary/mcp-gateway) with tool federation, several transports, and OAuth; an A2A gateway; inference routing to self-hosted models by GPU utilization, KV cache, LoRA adapters, and queue depth; and guardrails, RBAC, rate limiting, and OpenTelemetry. Its site also lists semantic caching and per-key or per-team spend caps.

**Trade-offs:** agentgateway is a general-purpose proxy, so teams that only need model routing take on a broader configuration surface. Solo's commercial edition adds a hardened distribution, further MCP security and governance features, and 24x7 support.

**Best for:** Teams that want one proxy for LLM, MCP, and A2A traffic, especially on Kubernetes.

**Migrating from LiteLLM:** Providers and models move into agentgateway's YAML or Gateway API resources, LiteLLM virtual keys are replaced by agentgateway's authentication and RBAC policies, and clients point at its OpenAI-compatible route.

## API gateways with AI plugins

An API gateway with AI plugins is the shortest path when the organization already runs one for REST traffic. Kong and Apache APISIX add model proxying, token-based rate limits, prompt guards, and caching as plugins, so AI traffic shares the authentication, consumers, and logging already in place, at the cost of fewer model-specific features than a dedicated gateway.

### 4. Kong AI Gateway

Kong AI Gateway is Kong's set of AI policies on Kong Gateway that secures, governs, and observes LLM, MCP, and A2A traffic. Kong now also offers AI Gateway 2.x, a dedicated control plane in Konnect that declares AI models, providers, policies, and MCP servers as resources.

**Deployment and license:** Kong Gateway's open-source repository is Apache 2.0, and its latest tag on November 16, 2026 was 3.9.4. That repository includes six AI plugins: AI Proxy, AI Prompt Guard, AI Prompt Decorator, AI Prompt Template, and AI request and response transformers. Kong runs self-hosted, with a Konnect control plane, or fully hosted on Dedicated Cloud Gateways.

**Strengths:** The [AI Gateway documentation](https://developer.konghq.com/ai-gateway/) lists semantic caching, prompt compression, AI rate limiting, semantic prompt and response guards, a PII sanitizer, and AWS Bedrock Guardrails, Azure Content Safety, and Google Model Armor integrations. The [AI MCP Proxy plugin](https://developer.konghq.com/plugins/ai-mcp-proxy/) fronts existing MCP servers or converts REST APIs into MCP tools.

**Trade-offs:** The AI MCP Proxy plugin is available only in Kong's AI Gateway Enterprise offering, from version 3.12, and AI Proxy Advanced, AI Semantic Cache, and AI Rate Limiting Advanced are not in the open-source repository, so open-source Kong gets basic proxying and prompt controls.

**Best for:** Organizations that already use Kong for API management and want AI traffic under the same policies.

**Migrating from LiteLLM:** Each LiteLLM model becomes an AI Proxy route or, in AI Gateway 2.x, an AI Model entity; LiteLLM keys become Kong consumers with key authentication; and token budgets move to AI Rate Limiting Advanced policies (Enterprise).

### 5. Apache APISIX

Apache APISIX is an open-source API gateway from the Apache Software Foundation whose AI plugins proxy, load balance, rate limit, and guard requests to model providers.

**Deployment and license:** Apache 2.0, written in Lua, self-hosted. The latest release was 3.20.0 on November 11, 2026.

**Strengths:** [`ai-proxy-multi`](https://apisix.apache.org/ai-gateway/) adds load balancing, retries, fallbacks, and health checks across OpenAI, Anthropic, Azure, Gemini, Vertex AI, Amazon Bedrock, DeepSeek, OpenRouter, and other OpenAI-compatible APIs. `ai-rate-limiting` enforces token quotas with local or Redis-backed counters, `ai-cache` provides exact and optional semantic caching, and `ai-prompt-guard` and `ai-lakera-guard` inspect content. The `openapi-to-mcp` plugin turns an HTTP API's OpenAPI operations into MCP tools served from the gateway.

**Trade-offs:** APISIX turns APIs into MCP tools, but the 3.20.0 plugin list has no plugin for proxying third-party MCP servers. Token quotas rely on the usage each provider reports, and the AI plugins document token quotas rather than dollar budgets per team.

**Best for:** Teams that already run APISIX for API traffic.

**Migrating from LiteLLM:** Providers become `ai-proxy-multi` instances with weights on a route, LiteLLM keys become APISIX consumers, and `tpm_limit` values move to `ai-rate-limiting` quotas.

## Managed and hybrid gateways

Managed gateways replace the LiteLLM proxy with a service the vendor operates, so there is no proxy, database, or cache to run. In exchange, prompts pass through a third party, governance follows the vendor's account model, and only Prisma AIRS AI Gateway, of the four here, documents an MCP gateway and a self-hosted data plane.

### 6. Prisma AIRS AI Gateway (formerly Portkey)

Prisma AIRS AI Gateway is the product name Palo Alto Networks now uses for Portkey's AI gateway, which routes requests to many model providers and adds guardrails, observability, governance, and an MCP gateway. Palo Alto Networks [completed its acquisition of Portkey](https://www.paloaltonetworks.com/company/press/2026/palo-alto-networks-completes-acquisition-of-portkey-to-secure-ai-agents) on May 29, 2026.

**Deployment and license:** The core [open-source gateway](https://github.com/Portkey-AI/gateway) is MIT-licensed TypeScript and runs with `npx @portkey-ai/gateway`. The commercial platform runs as SaaS or as a hybrid deployment, with the data plane in your VPC on Kubernetes and the control plane hosted.

**Strengths:** Fallbacks, retries with exponential backoff, weighted load balancing, simple caching, and dozens of built-in guardrails in the open-source gateway; RBAC, budgets, SSO, semantic caching, PII redaction, and an MCP Gateway with OAuth 2.1 in the commercial platform; and integration with Palo Alto's Prisma AIRS security products.

**Trade-offs:** The open-source repository's latest release was v1.15.2 in January 2026, and on November 16, 2026 its latest commit dated from May 2026. The hybrid data plane depends on a hosted control plane.

**Best for:** Enterprises that want a hosted control plane, an MCP gateway, and Palo Alto security integration.

**Migrating from LiteLLM:** Point the OpenAI SDK at the gateway's `/v1` endpoint, recreate fallback and load-balancing rules in the gateway's own configuration format, and recreate team keys and budgets in the platform.

### 7. Cloudflare AI Gateway

Cloudflare AI Gateway is a managed gateway on Cloudflare's network that proxies requests to model providers and adds analytics, logging, caching, rate limiting, spend limits, routing, guardrails, and data loss prevention.

**Deployment and license:** Proprietary and managed. Requests go to provider-specific endpoints (`https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}`) that keep each provider's native schema, or to Cloudflare's unified REST API. According to Cloudflare's pricing page on November 16, 2026, core features were free on all plans, and Unified Billing credits carried a 5% fee with provider inference passed through at no markup.

**Strengths:** The [feature list](https://developers.cloudflare.com/ai-gateway/features/) covers caching, spend limits by model, provider, or custom metadata, sliding or fixed-window rate limiting, model aliases with dynamic routes and fallbacks, guardrails, DLP scanning, stored provider keys (BYOK), custom metadata, custom costs, and Logpush export.

**Trade-offs:** Spend limits apply by model, provider, or custom metadata rather than to per-team virtual keys. Cloudflare's feature list does not include MCP proxying. Log storage limits differ by Workers plan.

**Best for:** Teams that already build on Cloudflare Workers or want a free managed layer for analytics, caching, and spend limits.

**Migrating from LiteLLM:** Replace the LiteLLM base URL with a provider-specific gateway URL, store provider keys with BYOK, tag requests with custom metadata for team attribution, and recreate budgets as spend limits.

### 8. Vercel AI Gateway

Vercel AI Gateway is a managed endpoint for models across providers that accepts OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages requests and works from any cloud, not only from Vercel deployments.

**Deployment and license:** Proprietary and managed, at `https://ai-gateway.vercel.sh`. A Vercel account is needed for keys and billing, but no Vercel project. Vercel's documentation states that the gateway adds zero markup to provider token prices, including with BYOK.

**Strengths:** The [AI Gateway docs](https://vercel.com/docs/ai-gateway) describe ordered provider and model fallbacks, budgets for a team, project, API key, or team member, BYOK, request logs with every routing attempt, cross-provider prompt caching, and modalities from text to video, embeddings, and realtime. Its Python integrations list includes LiteLLM.

**Trade-offs:** Budgets cover spend billed through Vercel's keys; BYOK spend is metered separately. Vercel documents AI Gateway as the model layer and MCP as a separate tool layer, so it is not an MCP gateway. Response caching stays in your application.

**Best for:** TypeScript teams using the AI SDK, and teams that want managed failover without operating a proxy.

**Migrating from LiteLLM:** Set the base URL to `https://ai-gateway.vercel.sh/v1`, switch model names to `provider/model` strings, and issue an AI Gateway API key per team or project with its own budget.

### 9. OpenRouter

OpenRouter is a hosted service that provides hundreds of models from many providers through one OpenAI-compatible endpoint and one account, with automatic provider fallbacks.

**Deployment and license:** Proprietary and managed, at `https://openrouter.ai/api/v1`. According to OpenRouter's FAQ on November 16, 2026, it charged a 5.5% fee ($0.80 minimum) on credit purchases, passed provider inference rates through without markup, and on pay-as-you-go included $25,000 a month of BYOK usage before a 5% BYOK fee.

**Strengths:** One account instead of many provider contracts, automatic fallback to the next provider on errors, [model fallbacks](https://openrouter.ai/docs/guides/routing/model-fallbacks) through a `models` array, guardrails with budgets and model or provider allowlists, and zero data retention routing options. Prompts and completions are not logged by default.

**Trade-offs:** Every request passes through OpenRouter's service. Guardrails cover budgets and allowlists, not the per-team virtual keys, custom guardrails, and self-hosted logs of a gateway you run. OpenRouter documents an MCP client in its Agent SDK, not an MCP gateway.

**Best for:** Trying many models through one account, and as one upstream behind a self-hosted gateway.

**Migrating from LiteLLM:** Change the base URL and key, and map model names to OpenRouter slugs. Alternatively, keep a self-hosted gateway and add OpenRouter as one of its providers; LiteLLM and Bifrost both support it.

## Other options and why they were left out

Several other gateways fit narrower cases. Higress is an Apache 2.0 AI gateway and CNCF Sandbox project with an MCP gateway and a managed edition, Alibaba Cloud AI Gateway. TrueFoundry AI Gateway is a commercial gateway for VPC, on-premises, and air-gapped deployment. Azure API Management, Apigee, and Amazon Bedrock AgentCore Gateway suit teams standardized on those clouds.

Helicone AI Gateway was left out because Mintlify, which acquired Helicone, said on March 3, 2026 that [Helicone would run in maintenance mode](https://mintlify.com/blog/mintlify-acquires-helicone), its gateway repository had no commits after November 2025, and Helicone's site said on November 16, 2026 that it is now part of Experiential Labs. The [AI gateways directory](https://theaiengineering.com/platforms/ai-gateways) lists these and other options with their current status.

## Which LiteLLM alternative should you choose?

Choose by where traffic must run, then by what you already operate. If prompts and keys can leave your network, a managed gateway removes the proxy entirely. If they cannot, extend the API gateway you already run, or adopt a standalone AI gateway when you need MCP and agent governance next to model routing.

**Figure 1: Two questions narrow the field: whether traffic must stay in your network, and whether you already run an API gateway or need MCP governance.**

Decision flow: managed teams choose by MCP needs between hosted gateways and Prisma AIRS; self-hosted teams extend Kong, APISIX, or Envoy, or adopt Bifrost or agentgateway.

Components:

- **Replacing a LiteLLM proxy**
- **Must traffic stay in your network?**
- **Managed: need MCP governance?**
- **Self-hosted: existing API gateway?**
- **Managed**: Cloudflare, Vercel, OpenRouter
- **Hybrid + MCP**: Prisma AIRS AI Gateway
- **Extend it**: Kong, APISIX, Agent Router
- **Standalone**: Bifrost, agentgateway

Connections:

- Replacing a LiteLLM proxy → Must traffic stay in your network?
- Must traffic stay in your network? → Managed: need MCP governance?: no
- Must traffic stay in your network? → Self-hosted: existing API gateway?: yes
- Managed: need MCP governance? → Managed: no
- Managed: need MCP governance? → Hybrid + MCP: yes
- Self-hosted: existing API gateway? → Extend it: yes
- Self-hosted: existing API gateway? → Standalone: no

The branches overlap in practice. Bifrost and agentgateway also suit teams that run an API gateway but want a separate layer for AI traffic, and Prisma AIRS AI Gateway's open-source core can be self-hosted.

| If your main reason for leaving LiteLLM is | Start by evaluating | Why |
|---|---|---|
| Proxy overhead or memory | Bifrost, agentgateway | Compiled Go and Rust gateways; test against LiteLLM's Rust path too |
| Postgres and Redis to operate | Bifrost, a managed gateway | Bifrost stores config in SQLite by default; managed services run their own state |
| Enterprise-only controls | Each vendor's open-source and paid tiers | Bifrost, Kong, agentgateway, and Prisma AIRS also keep some controls in paid editions |
| Supply chain exposure | Any option, with pinned versions and verified images | Version pinning and artifact verification apply to every gateway |
| No proxy to run | Cloudflare, Vercel, OpenRouter | The vendor operates the gateway |
| MCP and agent traffic | Bifrost, agentgateway, Agent Router, Prisma AIRS | Each documents MCP gateway features alongside model routing |
| An existing API gateway | Kong, APISIX, Agent Router | AI traffic shares existing consumers, policies, and logs |

## How to migrate off LiteLLM

Migrating off LiteLLM takes five steps: inventory what the proxy does today, stand up the new gateway with the same providers, change only the base URL and key in applications, run both gateways side by side with a small traffic share, and load test both before cutting over. Application code rarely changes, because every option here accepts OpenAI-format requests.

**1. Inventory the LiteLLM configuration.** List each `model_list` entry and its `model_name` alias, fallbacks, virtual keys with their budgets and `tpm_limit` and `rpm_limit` values, teams, logging callbacks, and guardrails. Aliases matter most: clients send LiteLLM's `model_name` values, and most alternatives expect `provider/model` strings or their own aliases.

**2. Recreate the configuration in the new gateway.** For Bifrost, the migration CLI does this from a running proxy; run it in dry-run mode first to review what will be skipped:

```bash
export LITELLM_URL="http://localhost:4000"
export LITELLM_MASTER_KEY="sk-1234"
export LITELLM_CONFIG="/path/to/litellm/config.yaml"
export BIFROST_URL="http://localhost:8080"
export BIFROST_API_KEY="your-bifrost-api-key"

DRY_RUN=1 npx @maximhq/bifrost-migration-cli   # report only, writes nothing
npx @maximhq/bifrost-migration-cli             # import into Bifrost
```

For other gateways, use the mappings in each entry above. Budgets and rate limits rarely translate one to one, so check reset windows and whether limits apply per key, team, or model.

**3. Make the gateway a configuration value in applications.** Read the base URL and key from the environment, so moving a service between gateways needs only a deploy:

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_GATEWAY_URL"],  # e.g. http://localhost:4000 or http://localhost:8080/openai
    api_key=os.environ["LLM_GATEWAY_KEY"],   # the gateway's virtual key, not a provider key
)
```

**4. Run both gateways side by side.** Put a weighted split in the load balancer or ingress in front of both, or move one team's keys at a time. Both gateways call the same provider accounts, so watch shared provider rate limits during the overlap.

**Figure 2: Running the new gateway beside LiteLLM behind a weighted split lets you compare latency, errors, and cost on real traffic before cutting over.**

An application sends traffic through a load balancer that splits requests between the existing LiteLLM proxy and a new gateway, both calling the same model providers.

Components:

- **Application**: OpenAI SDK, base URL from env
- **Load balancer or ingress**: weighted split, such as 95 / 5
- **LiteLLM proxy**: existing keys and budgets Postgres and Redis
- **New gateway**: imported config new virtual keys
- **Model providers**: same accounts and quotas

Connections:

- Application → Load balancer or ingress
- Load balancer or ingress → LiteLLM proxy: 95%
- Load balancer or ingress → New gateway: 5%
- LiteLLM proxy → Model providers
- New gateway → Model providers

**5. Load test both gateways the same way.** Point both at the same mock upstream with a fixed response time, so the test measures gateway overhead rather than provider variance. Ramp concurrency from 1 to above your expected peak, and record added P50 and P99 latency, error rate, memory and CPU per instance, and [time to first token](https://theaiengineering.com/glossary/time-to-first-token) for streamed responses. Enable the features you will run in production, such as logging, budgets, and caching, because forwarding-only numbers understate their cost. LiteLLM reports its own overhead in the `x-litellm-overhead-duration-ms` response header, which helps separate gateway time from provider time.

Cut over when the new gateway meets your latency target and its spend records reconcile with provider invoices, and keep LiteLLM running, unrouted, for a full billing cycle. Our guide to [LLM observability](https://theaiengineering.com/articles/llm-observability) covers the tracing that should run across both gateways during the overlap.

Where to go next: our guide to [LLM routing](https://theaiengineering.com/articles/llm-routing) covers fallback and load-balancing design in any gateway, [semantic caching](https://theaiengineering.com/glossary/semantic-caching) explains the cache trade-offs several of these gateways share, and the [AI gateways directory](https://theaiengineering.com/platforms/ai-gateways) lists more than 30 gateways by deployment model.

## Frequently asked questions

### What is the best alternative to LiteLLM?

The right LiteLLM alternative depends on what pushed you to look. For a self-hosted gateway with a direct import path from a LiteLLM proxy, Bifrost has a migration CLI. For Kubernetes teams on Envoy, Agent Router fits. For one proxy across LLM, MCP, and A2A traffic, agentgateway is an option. To stop operating a proxy, Cloudflare AI Gateway, Vercel AI Gateway, and OpenRouter are managed.

### Is LiteLLM safe to use after the 2026 supply chain attacks?

LiteLLM disclosed two compromised releases in 2026: PyPI versions 1.82.7 and 1.82.8 on March 24, and version 1.107.0 on PyPI and its Docker registries on October 21. Its advisories list the affected versions, the indicators to check for, and the credentials to rotate. Teams that keep LiteLLM should pin exact versions, avoid floating image tags such as latest, and verify artifacts before deploying them.

### What is an LLM proxy?

An LLM proxy is a server that sits between applications and model providers, accepts requests in one format, usually the OpenAI chat format, and forwards them to the selected provider. Basic proxies translate requests and retry failures. Fuller proxies, often called AI gateways, add virtual keys, budgets, rate limits, caching, guardrails, and logging. LiteLLM, Bifrost, and the other tools in this comparison all work as LLM proxies.

### Is Bifrost compatible with LiteLLM?

Yes, in two ways. Bifrost exposes a /litellm endpoint, so code that uses the LiteLLM Python SDK can point its base URL at Bifrost and keep its existing model strings. For LiteLLM proxy users, Bifrost publishes a migration CLI that reads providers, budgets, rate limits, organizations, teams, users, and virtual keys from LiteLLM and recreates them in Bifrost. Virtual key values are regenerated, so callers need new keys.

### Is LiteLLM written in Python or Rust?

LiteLLM began as a Python SDK and Python proxy server. In June 2026, BerriAI announced it was moving the gateway's request path to a Rust core while keeping its config, database schema, and APIs unchanged. As of November 2026, its v1.110.0 release candidates run the router on Rust, and the migration page targets an end-to-end Rust path for /v1/messages by December 1, 2026.

### Can OpenRouter replace LiteLLM?

OpenRouter can replace LiteLLM when the main need is one OpenAI-compatible endpoint for many models without separate provider accounts. It is a hosted service, so prompts pass through OpenRouter, and it offers guardrails with budgets and model allowlists rather than LiteLLM-style virtual keys per team. Teams that need self-hosted logs or internal budgets can run their own gateway and add OpenRouter as one upstream provider.

## Terms in this guide

- [AI gateway](https://theaiengineering.com/glossary/ai-gateway): An AI gateway is a proxy layer between applications and model providers that exposes one API for many models and centralizes routing, failover, access control, cost tracking, caching, and logging.
- [OpenAI-compatible API](https://theaiengineering.com/glossary/openai-compatible-api): An OpenAI-compatible API is an HTTP API that accepts OpenAI's request and response formats, so code written for OpenAI's SDKs can call another server by changing the base URL, API key, and model name.
- [Virtual key](https://theaiengineering.com/glossary/virtual-key): A virtual key is an API key issued by an AI gateway that maps a caller to its allowed models, budget, and rate limits, while the gateway keeps the real provider credentials.
- [Rate limiting](https://theaiengineering.com/glossary/rate-limiting): Rate limiting is the enforcement of a maximum number of requests or tokens a client may send to an API within a time window, with excess requests rejected, usually with HTTP 429.
- [Model failover](https://theaiengineering.com/glossary/model-failover): Model failover is the automatic rerouting of an LLM request to a backup model, deployment, or provider when the primary target returns an error, times out, or hits a rate limit.
- [LLM load balancing](https://theaiengineering.com/glossary/llm-load-balancing): LLM load balancing is the distribution of requests for one model across several interchangeable deployments, such as API keys, regions, providers, or replicas, by weight, latency, load, or remaining quota.
- [Semantic caching](https://theaiengineering.com/glossary/semantic-caching): Semantic caching is a technique that returns a stored LLM response when a new query is close in meaning to an earlier one, measured by embedding similarity instead of exact text match.
- [MCP gateway](https://theaiengineering.com/glossary/mcp-gateway): An MCP gateway is a proxy that sits between MCP hosts and many MCP servers, exposing them through one endpoint and centralizing authentication, tool access policy, and audit logging.
- [Model Context Protocol (MCP)](https://theaiengineering.com/glossary/model-context-protocol): Model Context Protocol (MCP) is an open protocol, introduced by Anthropic in November 2024, that standardizes how AI applications connect to external tools, data sources, and prompts through MCP servers.
- [A2A protocol](https://theaiengineering.com/glossary/a2a-protocol): The A2A (Agent2Agent) protocol is an open standard, introduced by Google in April 2025, that lets AI agents built on different frameworks discover each other, exchange messages, and delegate tasks.
- [Guardrails](https://theaiengineering.com/glossary/guardrails): Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.
- [OpenTelemetry](https://theaiengineering.com/glossary/opentelemetry): OpenTelemetry is an open source, vendor-neutral observability framework of APIs, SDKs, a wire protocol, and naming conventions for traces, metrics, and logs, including conventions for LLM calls and agent runs.
- [Time to first token](https://theaiengineering.com/glossary/time-to-first-token): Time to first token (TTFT) is the elapsed time from submitting a request to an LLM until the first token of the response arrives, covering request queuing, network transit, and prompt prefill.

## Related guides

- [What is an AI gateway? How it works and when you need one](https://theaiengineering.com/articles/what-is-an-ai-gateway): An AI gateway is a proxy between your apps and model providers. Learn how it works, its core features, AI gateway vs API gateway, and open source options.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.
- [LLM observability: what to capture, which metrics, and tools](https://theaiengineering.com/articles/llm-observability): LLM observability guide: what to capture in traces and spans, metrics to track, OpenTelemetry GenAI conventions, tool categories, and keeping prompts private.
- [OpenRouter alternatives: self-hosted gateways and managed model APIs compared](https://theaiengineering.com/articles/openrouter-alternatives): OpenRouter alternatives compared: Bifrost, LiteLLM, Vercel, Cloudflare, Requesty, Eden AI, and Bedrock by provider keys, deployment, fees, and governance.

Part of the [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) topic hub.

## Cite this guide

The AI Engineering Editors. "LiteLLM alternatives: 9 open-source and managed AI gateways compared." The AI Engineering, updated Nov 17, 2026. https://theaiengineering.com/articles/litellm-alternatives

- Title: LiteLLM alternatives: 9 open-source and managed AI gateways compared
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-11-17
- Updated: 2026-11-17
- URL: https://theaiengineering.com/articles/litellm-alternatives
- Markdown: https://theaiengineering.com/articles/litellm-alternatives.md

To cite one section, link to its anchor:

- [What is LiteLLM?](https://theaiengineering.com/articles/litellm-alternatives#what-is-litellm)
- [Why teams look for LiteLLM alternatives](https://theaiengineering.com/articles/litellm-alternatives#why-teams-look-for-litellm-alternatives)
- [How we chose and compared the alternatives](https://theaiengineering.com/articles/litellm-alternatives#how-we-chose-and-compared-the-alternatives)
- [LiteLLM alternatives at a glance](https://theaiengineering.com/articles/litellm-alternatives#litellm-alternatives-at-a-glance)
- [Self-hosted open-source gateways](https://theaiengineering.com/articles/litellm-alternatives#self-hosted-open-source-gateways)
- [API gateways with AI plugins](https://theaiengineering.com/articles/litellm-alternatives#api-gateways-with-ai-plugins)
- [Managed and hybrid gateways](https://theaiengineering.com/articles/litellm-alternatives#managed-and-hybrid-gateways)
- [Other options and why they were left out](https://theaiengineering.com/articles/litellm-alternatives#other-options-and-why-they-were-left-out)
- [Which LiteLLM alternative should you choose?](https://theaiengineering.com/articles/litellm-alternatives#which-litellm-alternative-should-you-choose)
- [How to migrate off LiteLLM](https://theaiengineering.com/articles/litellm-alternatives#how-to-migrate-off-litellm)
- [Frequently asked questions](https://theaiengineering.com/articles/litellm-alternatives#faq)
