Bifrost vs LiteLLM: architecture, features, and performance compared
Bifrost and LiteLLM are open-source AI gateways that put many model providers behind one API: Bifrost is a Go gateway from Maxim AI that ships as a single binary under the Apache 2.0 license, and LiteLLM is a Python proxy server plus a Python SDK from BerriAI, MIT licensed outside its enterprise directory. The Bifrost vs LiteLLM decision turns more on architecture than on feature lists, because the two now overlap on routing, keys, caching, MCP, and agent traffic. This comparison checks both projects' documentation as of November 16, 2026, and covers deployment, features, published benchmarks, and migration. It belongs to our AI gateways and LLM routing hub.
Bifrost vs LiteLLM at a glance#
Bifrost is a single Go process that keeps its state in local SQLite files by default, while the LiteLLM proxy is a Python service that needs Postgres for virtual keys and spend tracking, plus Redis once it runs more than one worker. Both expose an OpenAI-compatible API and sell an Enterprise tier on top of the open-source core.
| Bifrost | LiteLLM | |
|---|---|---|
| Maintainer | Maxim AI | BerriAI |
| Language | Go | Python for the SDK and proxy; a Rust gateway is in release candidates |
| License | Apache 2.0 | MIT, except the litellm/enterprise/ directory |
| Latest release (November 16, 2026) | Gateway v2.5.1, released November 10, 2026 | v1.108.9 stable, released November 14, 2026; v1.110.0 release candidates |
| What you run | Gateway binary with a web UI, or the Go SDK inside a Go app | Proxy server with an admin UI, or the Python SDK inside a Python app |
| Install options | npx, Docker, Helm, Go module | Docker, pip or uv, Helm, Terraform |
| Default port | 8080 | 4000 |
| State stores | SQLite config and log stores by default; Postgres optional | Postgres for keys, budgets, and spend; Redis for more than one worker |
| Providers | 35 in the docs support matrix | 100+ |
| Paid tier | Bifrost Enterprise | LiteLLM Enterprise |
LiteLLM is partway through a move from Python to Rust, covered in the architecture section.
What are Bifrost and LiteLLM?#
Bifrost and LiteLLM are self-hosted AI gateways: proxies that give applications one OpenAI-compatible API to many model providers and apply shared routing, keys, budgets, and logging.
Our explainer on what an AI gateway is covers the category.
What is the Bifrost AI gateway?#
The Bifrost AI gateway is an open-source gateway from Maxim AI, written in Go, that handles model, MCP, and agent traffic through one process. Its supported providers page lists 35 providers, including OpenAI, Anthropic, Bedrock, Vertex AI, Azure, Gemini, Groq, Mistral, and xAI, plus self-hosted vLLM, SGLang, and Ollama.
As a Bifrost LLM gateway, it accepts OpenAI-format requests and also serves provider-shaped routes, so existing OpenAI, Anthropic, Google GenAI, Bedrock, Cohere, and LiteLLM SDK code can point at it by changing a base URL. The same process runs an MCP gateway and an agent gateway for A2A agents. Teams that want the gateway inside a Go program can import the Go SDK (github.com/maximhq/bifrost/core) instead of running a server. Bifrost Enterprise adds guardrails, SSO, RBAC, audit logs, clustering, and adaptive routing.
What is the LiteLLM proxy?#
The LiteLLM proxy is a self-hosted, OpenAI-compatible gateway server that BerriAI's documentation describes as a gateway for platform teams managing model access across an organization. It sits in the same repository as the LiteLLM Python SDK, a library that calls more than 100 providers from Python code with one interface.
The SDK and the proxy share a router for retries, fallbacks, and load balancing. The proxy adds virtual keys, budgets, rate limits, spend tracking, an admin UI, guardrails, an MCP gateway, and an A2A agent gateway. Its supported endpoints include /chat/completions, /responses, /messages, /embeddings, /images, /audio, /batches, /rerank, and /a2a. LiteLLM Enterprise is a license key on the same image that adds SSO beyond 5 users, SCIM, audit logs, RBAC, secret managers, and per-key guardrails.
Architecture and deployment#
Bifrost runs as one Go process that holds configuration, keys, budgets, and usage in memory and persists them to SQLite or Postgres. The LiteLLM proxy runs as one or more Python worker processes that share keys, budgets, rate limits, and cooldowns through Postgres and Redis.
Deploying Bifrost#
Bifrost's gateway setup guide starts the gateway with one command. The API and the web UI share port 8080:
# Run locally with npx
npx -y @maximhq/bifrost
# Or run the Docker image, persisting data to ./data
docker run -p 8080:8080 -v $(pwd)/data:/app/data maximhq/bifrostThe app directory holds an optional config.json, a config.db SQLite database for configuration made in the UI, and a logs.db SQLite database for request logs. Postgres 16 or later can replace either store, and a vector store is needed only for semantic caching. For Kubernetes, Bifrost publishes a Helm chart and guides for EKS, GKE, and AKS. For production, the setup guide advises pinning an image version, enabling authentication, using TLS, and choosing durable storage.
Deploying the LiteLLM proxy#
The LiteLLM proxy runs from a Docker image or a pip or uv install and listens on port 4000, with the admin UI at /ui. The Docker quickstart runs it without a database from a config file:
docker run \
-v "$PWD/litellm_config.yaml:/app/config.yaml:ro" \
-e OPENAI_API_KEY \
-e LITELLM_MASTER_KEY \
-p 127.0.0.1:4000:4000 \
docker.litellm.ai/berriai/litellm:latest \
--config /app/config.yamlThat mode has no virtual keys, database-backed model management, or spend tracking: the quickstart states that virtual keys require a database, and LiteLLM's production guide calls PostgreSQL its primary supported database for production. LITELLM_MASTER_KEY authorizes management calls and is the default admin UI password, and LITELLM_SALT_KEY encrypts provider keys added in the UI. The production guide asks for Redis 7.0 or later once there is more than one proxy instance, sizes each Kubernetes pod at 1 vCPU and 4 GiB of memory as a floor with one Uvicorn worker per pod, and sets NUM_WORKERS to the vCPU count on a single VM. LiteLLM publishes Helm and Terraform deployments for Kubernetes, AWS, and GCP.
The LiteLLM Python SDK needs none of this, because it runs inside the application process.
LiteLLM's move to Rust#
LiteLLM is rewriting its gateway in Rust, and the work was in release candidates on November 16, 2026. The project's migration tracker reported 39 percent of its major APIs built in v1.110.0-rc.1, including a gateway binary in the Docker image that forwards requests it cannot serve to Python. In the planned design, an Axum server receives every request, serves supported ones in Rust, and forwards the rest to a Python worker on the same host, with both sides using the same Redis and Postgres protocols.
The tracker names December 1, 2026 as the target for running the common /v1/messages path end to end in Rust, initially with simple-shuffle routing, Anthropic models on Anthropic, Bedrock, or Vertex AI, and supported callbacks such as S3 logs and spend logging. Custom Python callbacks and other Python-only features stay on Python for now. Until that path ships and covers your configuration, plan capacity around the Python proxy.
Running more than one instance#
Open-source Bifrost keeps gateway state in each node's memory, while LiteLLM's open-source proxy shares state across workers and replicas through Redis. Bifrost's multinode guide states that running multiple OSS Bifrost nodes with a Postgres backend is not supported, because each node loads provider configs, keys, budgets, and usage into memory and does not read them back. Its recommended OSS pattern shares one config.json across nodes with no config store. The same page gives Maxim's view that one OSS instance can handle around 3,000 to 5,000 requests per second, and real-time state sync across nodes comes from Enterprise cluster mode, which uses gossip-based peer-to-peer clustering.
LiteLLM's Redis requirements page says to use Redis whenever more than one worker process runs, whether from --num_workers or replicas. Without Redis, a limit of 100 requests per minute across 4 workers lets through 400, budgets can overshoot, a cooled-down deployment stays in rotation on other workers, and cache hit rates drop roughly by the worker count.
Feature comparison#
Bifrost and LiteLLM cover the same feature categories, so the useful differences are in depth and in where the open-source line sits. Bifrost keeps virtual keys, caching, MCP, and observability in open source and puts guardrails, SSO, and clustering in Enterprise. LiteLLM keeps guardrails and multi-instance scaling in open source and gates SSO beyond 5 users, audit logs, and RBAC.
The table reflects each project's documentation on November 16, 2026.
| Capability | Bifrost | LiteLLM |
|---|---|---|
| Provider coverage | 35 providers in the support matrix, hosted and self-hosted | 100+ providers through the SDK and proxy |
| API formats | OpenAI-compatible endpoints plus Anthropic, Google GenAI, Bedrock, Cohere, and LiteLLM SDK routes | OpenAI format plus /messages, /responses, /rerank, /batches, and /a2a |
| Routing and fallbacks | Weighted key load balancing, retries with backoff and jitter, per-request fallback chains, CEL routing rules, model aliases, complexity router; adaptive routing in Enterprise | Router strategies (simple-shuffle default, usage-based, latency-based, least-busy, cost-based), fallbacks, context window fallbacks, cooldowns |
| Virtual keys and budgets | Budgets and token or request limits per customer, team, virtual key, and provider config | Postgres-backed keys with budgets and RPM or TPM limits per key, user, team, and end user; model-specific and tag budgets in Enterprise |
| Caching | Exact hash and semantic matching on Redis or Valkey, Qdrant, Weaviate, or Pinecone | Exact match on Redis, S3, GCS, memory, or disk; semantic on Redis, Valkey, or Qdrant |
| Guardrails | Enterprise: request and response rules, secrets detection, redaction modes, third-party integrations | Open-source framework with custom guardrails and Presidio; per-key guardrails and several built-in integrations need Enterprise |
| MCP gateway | STDIO, HTTP, and SSE servers behind one /mcp endpoint; tool filtering per virtual key; agent mode; Code Mode |
Streamable HTTP, SSE, and stdio (off by default); access by key, team, and organization; tool permissions per entity; OAuth 2.0 |
| A2A agent gateway | Agent Gateway over JSON-RPC, REST, or gRPC; virtual keys grant agent access | /a2a/{agent_id} endpoint; key and team checks; per-agent spend |
| Observability | Built-in log store and dashboard, Prometheus /metrics, OpenTelemetry traces with GenAI conventions |
Prometheus /metrics, OpenTelemetry callback with opt-in GenAI conventions, many logging callbacks |
| Plugins | Go plugins compiled to .so files, with LLM and HTTP transport hooks |
Python CustomLogger callbacks and CustomGuardrail classes |
| Admin UI | Built-in web UI on port 8080 | Admin UI at /ui on port 4000 |
| SSO, RBAC, audit logs | Enterprise: OIDC SSO, SCIM, RBAC, signed audit logs | SSO free for up to 5 users; SCIM, RBAC, and audit logs in Enterprise |
| Multi-node state sync | Enterprise cluster mode | Open source, through Redis and Postgres |
| License | Apache 2.0 | MIT outside litellm/enterprise/ |
Routing, keys, and budgets#
Both gateways implement model failover and load balancing across keys and deployments. Bifrost's retry and fallback rules retry network errors and 5xx responses on the same key with exponential backoff and jitter, rotate to another key on a 429 or a rejected credential, and give each entry in a request's fallbacks list its own retry budget. LiteLLM's router recommends simple-shuffle for production and puts a failing deployment into a cooldown, 5 seconds by default. Our LLM routing guide compares these strategies in depth.
Bifrost attaches budgets and rate limits to customers, teams, virtual keys, and individual providers within a key, and a request must pass every applicable budget; keys owned by users and budgets from access profiles are Enterprise features. LiteLLM attaches budgets and RPM or TPM limits to keys, users, teams, and customers, and adds a per-key max_parallel_requests cap. Open-source Bifrost marks a failing key dead for the rest of that request, while holding refused keys out of rotation across requests is part of Enterprise adaptive routing.
Caching and guardrails#
Both gateways offer exact-match and semantic caching. Bifrost's semantic cache runs a direct hash lookup first, falls back to similarity matching at a default threshold of 0.8, and scopes entries to the virtual key that wrote them. LiteLLM's caching documentation warns that semantic caching suits single-shot prompts and works badly on agentic traffic.
Guardrails mark the clearest licensing split. Bifrost's guardrails, which check prompts, responses, and MCP tool arguments and results, ship in Enterprise, with integrations for AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, Presidio, and others. LiteLLM's open-source guardrail framework runs checks before, during, or after a call and includes custom guardrails and Presidio; per-key guardrail control and integrations such as Lakera prompt injection need an Enterprise license.
MCP, agents, and observability#
Both projects run an MCP gateway. Bifrost's MCP gateway scopes every /mcp request to the tools its virtual key allows, runs an agent mode that can auto-execute approved tools, and offers Code Mode, where the model writes a short script in a sandboxed Starlark interpreter instead of receiving every tool definition. In Maxim AI's own Code Mode benchmark with 508 tools across 16 servers, input tokens fell 92.8 percent at the same 100 percent pass rate. LiteLLM's MCP gateway controls access per key, team, and organization, filters tools with mcp_tool_permissions, and supports OAuth 2.0 for upstream servers.
For A2A agents, Bifrost's agent gateway lets a client and an agent use different transports (JSON-RPC, REST, or gRPC). LiteLLM's agent gateway connects to agents hosted on LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, and Pydantic AI. Our A2A vs MCP comparison explains how the two protocols divide the work.
Both export Prometheus metrics and OpenTelemetry traces using the GenAI semantic conventions; LiteLLM turns those conventions on with an opt-in environment variable. Bifrost also keeps a request log with a per-request overhead breakdown in its UI. Our guide to LLM observability covers what gateway telemetry misses.
Performance and published benchmarks#
The Bifrost vs LiteLLM benchmarks that Maxim AI and LiteLLM publish are vendor-run. Maxim AI's tests and LiteLLM's own November 2026 tests both measured much lower overhead and tail latency for Bifrost than for LiteLLM's Python proxy, and LiteLLM's tests measured its unreleased Rust branch below Bifrost. Setups differ widely, so treat every figure as a starting hypothesis.
| Benchmark | Setup | Bifrost | LiteLLM Python proxy |
|---|---|---|---|
| Maxim AI, 500 RPS load test | One t3.medium (2 vCPUs, 4 GB) in AWS us-east-1, 500 virtual users, 60 seconds; versions not stated | 424 req/s, P50 804 ms, P99 1.68 s, 120 MB peak memory, 100% success | 44.84 req/s, P50 38.65 s, P99 90.72 s, 372 MB peak memory, 88.78% success |
| Maxim AI, overhead test | 60 ms mock OpenAI upstream | 0.99 ms overhead | 40 ms overhead |
| LiteLLM, short-prompt chat | 4 instances with 4 vCPUs, 8 GB, and 4 workers each; mock OpenAI endpoint; Postgres on, Redis off; 1,000 users | Not tested | 1,170 req/s, p95 overhead 8 ms, p99 overhead 13 ms |
| LiteLLM, full-features run (November 13, 2026) | One replica each with 4 cores and 16 GiB, 50 req/s for 5 minutes, 10k to 50k token requests, 1,000 virtual keys, single run | v2.3.0: median overhead 1.2 ms, p99 4.2 ms, 2,809 MiB peak | 4 workers: median overhead 79.3 ms, p99 1,587.8 ms, 9,553 MiB peak |
Sources: Maxim AI's benchmark page, with code in maximhq/bifrost-benchmarking; LiteLLM's benchmarks page; and LiteLLM's full-features benchmark post, with code and data in BerriAI/ai-gateway-bench.
The Python proxy scales by adding processes. LiteLLM's high-throughput profile reached 3,000 requests per second with a p99 gateway latency of 91.6 ms on 33 pods of 4 workers each, with Postgres, Redis, budgets, and spend tracking on. In LiteLLM's full-features run, the Python proxy served only 78.9 percent of requests at 75 requests per second and 29.3 percent at 100, while its Rust branch measured a median overhead of 0.9 ms, a p99 of 1.8 ms, and 357 MiB of peak memory. That branch is not a stable release, and LiteLLM's post expects comparable fast-path performance in its December 1 release. Bifrost's peak memory was 120 MB in Maxim's test and 2,809 MiB in LiteLLM's full-features run, which used 10k to 50k token requests with S3 request logs and Prometheus metrics on. LiteLLM's post also notes that Bifrost's Bedrock calls reached the mock over TLS in that run while the other gateways' did not.
Why a Go process and a Python proxy behave differently under concurrency#
A Go gateway can spread one process across every CPU core, while a CPython gateway runs Python code on one core at a time per process and must add processes to use more cores. Under high concurrency, that difference shows up as queueing inside each Python worker and as shared state moving out to Redis.
Bifrost uses goroutines inside one process. The Go runtime schedules lightweight goroutines across all available cores. Bifrost's concurrency architecture adds an isolated worker pool per provider, channel-based handoffs between stages, and object pools that reuse request and response structures to limit allocations. Keys, budgets, and usage counters sit in the same memory, so checking a budget needs no network call.
LiteLLM uses an event loop per worker process. In CPython's default build, the global interpreter lock lets only one thread execute Python bytecode at a time; Python 3.13 added an optional build that disables it. Each Uvicorn worker runs an asyncio event loop that interleaves many requests while they wait on the network. The CPU work around each request (parsing, auth lookup, routing, cost calculation, and callbacks) still runs on that one loop, so once that work exceeds what one core can do, requests queue and tail latency climbs. LiteLLM's November 2026 profiling, with all callbacks on and one pinned core per gateway, measured 33.7 ms of gateway time for one non-streamed request with a cached key on the Python proxy, against 1.7 ms on its Rust branch.
More workers add cores but also separate memory spaces. LiteLLM's benchmark post notes that in-process auth caches are harder to share in Python because the workers are forked processes, and its Redis page shows what happens to limits, budgets, and caches without shared state. Bifrost's single-process design avoids that coordination, but open-source Bifrost has no built-in way to share its in-memory state across nodes.
How to load-test both on your own traffic#
Both vendors publish their test code, so a comparison on your own traffic can reuse it with your own settings:
- Match production features. Turn on virtual keys, budgets, request logging, and the callbacks or plugins you plan to use. Overhead with logging on differs from forwarding-only overhead.
- Use realistic payloads. Replay your prompt sizes, streaming ratio, and tool-call patterns. LiteLLM's profiling found gateways holding several copies of a large request body in memory while it streams.
- Subtract a baseline. Run the same load against a mock upstream or the provider directly, then subtract it, as both benchmark kits do.
- Sweep concurrency. Step from low load to past your expected peak and record p50, p99, error rate, CPU, and memory at each step, not only peak throughput.
- Size each gateway the way you would run it. Compare one Bifrost node against the LiteLLM pod count, workers, and Redis you would deploy, and test failover by injecting 429 and 5xx errors.
- Read the gateway's own timing. LiteLLM returns its overhead in the
x-litellm-overhead-duration-msresponse header, and Bifrost can export a per-component overhead histogram namedbifrost_overhead_component_microseconds.
Migrating from LiteLLM to Bifrost#
Bifrost documents two migration paths from LiteLLM: keep the LiteLLM Python SDK and point it at Bifrost's /litellm endpoint, or copy a LiteLLM proxy's providers, teams, users, and virtual keys into Bifrost with a migration CLI. Either way, callers need Bifrost virtual keys, and code that calls the proxy changes its base URL and model names.
Keep the LiteLLM SDK#
Bifrost's LiteLLM SDK integration accepts LiteLLM SDK calls at /litellm, so an application changes one argument. A virtual key goes in the x-bf-vk header:
from litellm import completion
response = completion(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello!"}],
base_url="http://localhost:8080/litellm", # point the SDK at Bifrost
extra_headers={"x-bf-vk": "<BIFROST_VIRTUAL_KEY>"}, # Bifrost governance
)The integration only works for providers that both LiteLLM and Bifrost support; the docs warn that requests to a provider only one of them supports will fail. Bifrost's compatibility plugin, described in its docs as built for LiteLLM proxy and SDK compatibility, can also convert text completions to chat, drop parameters a model does not support, and route chat requests to the Responses API for models that need it.
Move a LiteLLM proxy's data#
Bifrost's LiteLLM migration guide describes a one-time tool that reads a running LiteLLM proxy and recreates its entities through Bifrost's management API. It only reads from LiteLLM. A dry run reports what would change:
export LITELLM_URL="http://localhost:4000"
export LITELLM_MASTER_KEY="<LITELLM_MASTER_KEY>"
export LITELLM_CONFIG="/path/to/litellm/config.yaml"
export BIFROST_URL="http://localhost:8080"
export BIFROST_API_KEY="<BIFROST_API_KEY>"
# Needed only when models live in the LiteLLM database:
# export LITELLM_DB_URL="postgresql://..." LITELLM_SALT_KEY="..."
DRY_RUN=1 npx @maximhq/bifrost-migration-cli # preview
npx @maximhq/bifrost-migration-cli # migrateThe tool maps model deployments to Bifrost providers, keys, and model configs; organizations to customers; teams to teams; internal users to users; and virtual keys to virtual keys, carrying over budgets and RPM and TPM limits. Environment references such as os.environ/OPENAI_API_KEY become env.OPENAI_API_KEY, so those variables must exist where Bifrost runs. The guide lists the gaps: partial wildcards such as openai/gpt-4* are skipped, users without an email are skipped, organization-level model restrictions fold into virtual keys, and virtual key token values are not migrated.
What changes in application code#
Applications that call the LiteLLM proxy with the OpenAI SDK change three things: the base URL, the key, and possibly the model name. Bifrost generates new virtual keys in the sk-bf- format, which must be reissued to every caller, and its OpenAI-compatible route takes provider/model names:
from openai import OpenAI
# Before: LiteLLM proxy on its default port, with a LiteLLM virtual key
# client = OpenAI(base_url="http://localhost:4000", api_key="<LITELLM_VIRTUAL_KEY>")
# After: Bifrost's OpenAI-compatible route, with a Bifrost virtual key
client = OpenAI(base_url="http://localhost:8080/openai", api_key="<BIFROST_VIRTUAL_KEY>")
response = client.chat.completions.create(
model="openai/gpt-4o-mini", # provider/model; unprefixed names default to OpenAI
messages=[{"role": "user", "content": "Hello!"}],
)The LiteLLM proxy serves the model_name values defined in its config.yaml, so confirm how each one resolves after migration and recreate any you still need with Bifrost's model aliasing. Some provider names also differ: the migration guide maps vertex_ai to vertex, hosted_vllm to vllm, ollama_chat to ollama, and azure_ai to azure. Run both gateways side by side, shift a share of traffic, and compare logs, costs, and error rates before cutting over.
When to choose Bifrost or LiteLLM#
Choose Bifrost for a single-binary gateway that handles high request rates on one node, MCP and agent traffic, and drop-in routes for several provider SDKs. Choose LiteLLM for a Python library inside your application, the widest provider list of the two, or open-source horizontal scaling with shared budgets. Then confirm the choice with a load test on your own traffic.
| Scenario | Start with | Why |
|---|---|---|
| One shared gateway with minimal infrastructure | Bifrost | Single binary with a web UI and SQLite stores; no Postgres or Redis required |
| High request rates on one node | Bifrost | One process uses every core; lower overhead than the Python proxy in both vendors' benchmarks |
| Agents that use many MCP tools | Bifrost | Tool filtering per virtual key, agent mode, and Code Mode |
| A Go service that wants an embedded gateway | Bifrost Go SDK | Provider routing in-process, no separate server |
| A Python application that wants a library, not a service | LiteLLM SDK | In-process calls to 100+ providers with router fallbacks |
| A provider only one gateway supports | Whichever lists it | Check both provider lists; LiteLLM's is longer |
| Several open-source replicas with shared limits and budgets | LiteLLM | Redis and Postgres share state across workers without an Enterprise license |
| Guardrails or a few SSO users without an Enterprise license | LiteLLM | Open-source guardrail framework; SSO free for up to 5 users |
| Custom request logic written in Python | LiteLLM | CustomLogger hooks and custom guardrails |
The two can also be combined, because the LiteLLM SDK can call Bifrost's /litellm endpoint. For other open-source and managed gateways, see our AI gateways directory. For how a gateway fits into a production stack, start with our explainer on what an AI gateway is, and for routing strategies behind both gateways, read the LLM routing guide.
Frequently asked questions
Is Bifrost faster than LiteLLM?
- In Maxim AI's published benchmark on one t3.medium instance at a 500 requests-per-second target, Bifrost served 424 requests per second against 44.84 for LiteLLM, with a P99 latency of 1.68 seconds against 90.72 seconds. LiteLLM's own November 2026 benchmark also measured lower overhead for Bifrost than for its Python proxy, and lower still for its unreleased Rust branch. Both tests are vendor-run, so test on your own traffic.
Is Bifrost open source?
- Yes. The Bifrost gateway, Go SDK, and plugins are published in the maximhq/bifrost repository under the Apache 2.0 license, and self-hosting the open-source gateway is free. Bifrost Enterprise adds guardrails, OIDC single sign-on, SCIM, role-based access control, signed audit logs, cluster mode, adaptive routing, and secret manager integrations. LiteLLM is MIT licensed outside its enterprise directory and also sells an Enterprise license.
Does LiteLLM proxy require a database?
- The LiteLLM proxy can run from a config file without a database, but virtual keys, database-backed model management, spend tracking, and global budgets require Postgres, which LiteLLM calls its primary supported production database. LiteLLM's documentation also says to add Redis once you run more than one worker process, because rate limits, budgets, cooldowns, and caches are otherwise counted separately in each worker.
How do I migrate from LiteLLM to Bifrost?
- Bifrost documents two paths. Applications built on the LiteLLM Python SDK can set base_url to Bifrost's /litellm endpoint and keep their code. Teams running a LiteLLM proxy can run npx @maximhq/bifrost-migration-cli, which reads the proxy's config and database and recreates providers, keys, customers, teams, users, and virtual keys in Bifrost. Virtual key tokens are not copied, so callers need new sk-bf- keys.
Is LiteLLM written in Rust now?
- Partly, and the work is in progress. On November 16, 2026, LiteLLM's migration tracker reported 39 percent of its major APIs built in v1.110.0-rc.1, with a Rust gateway binary in the Docker image that forwards requests it cannot serve to a Python worker. Its next milestone, targeted for December 1, 2026, is running the common /v1/messages path end to end in Rust.
Do Bifrost and LiteLLM support MCP?
- Yes, both act as MCP gateways. Bifrost connects to MCP servers over STDIO, HTTP, or SSE, exposes them at one /mcp endpoint, filters tools per virtual key, and adds agent mode and Code Mode. LiteLLM supports Streamable HTTP, SSE, and stdio servers, with stdio off by default, controls access by key, team, and organization, sets tool permissions per entity, and supports OAuth 2.0 for MCP servers.
Terms in this guide
Cite this guide
The AI Engineering Editors. "Bifrost vs LiteLLM: architecture, features, and performance compared." The AI Engineering, updated Nov 16, 2026. https://theaiengineering.com/articles/bifrost-vs-litellm
- Title
- Bifrost vs LiteLLM: architecture, features, and performance compared
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Published
- Updated