---
title: "What is an OpenAI-compatible API? Endpoints and limits"
description: "An OpenAI-compatible API accepts OpenAI's request format, so OpenAI SDKs work with a new base URL. Which endpoints it covers and where compatibility breaks."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/openai-compatible-api"
markdown_url: "https://theaiengineering.com/glossary/openai-compatible-api.md"
topic: "AI gateways and LLM routing"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["OpenAI-compatible API","OpenAI-compatible endpoint","OpenAI-compatible server","OpenAI API compatibility"]
citation: "The AI Engineering Editors. \"What is an OpenAI-compatible API? Endpoints and limits.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/openai-compatible-api"
---

# What is an OpenAI-compatible API? Endpoints and limits

Also called OpenAI-compatible endpoint, OpenAI-compatible server, OpenAI API compatibility.

> **Definition of OpenAI-compatible API:** An OpenAI-compatible API is an HTTP API that accepts OpenAI's request and response formats, so code written for OpenAI's SDKs can call another server by changing the base URL, API key, and model name.

An OpenAI-compatible API is an endpoint that implements the paths, request fields, and response shapes of OpenAI's API, most often `/v1/chat/completions`. Client code built on an OpenAI SDK can then call a self-hosted [inference engine](https://theaiengineering.com/glossary/inference-engine), another model provider, or an [AI gateway](https://theaiengineering.com/glossary/ai-gateway) by changing the base URL, API key, and model name. Serving engines such as vLLM and Ollama, many AI gateways, and model providers such as Anthropic and Google ([Gemini's compatibility layer](https://aistudio.google.com/docs/openai) is in beta) expose the format. In practice, each server supports a documented subset of OpenAI's fields.

## Which endpoints are usually covered

OpenAI-compatible servers cover Chat Completions first, usually with a models list and often with Completions and Embeddings. Support for the newer Responses API is less complete, especially for its server-side state. Treat "OpenAI-compatible" as a claim about specific endpoints and fields, and check the server's documentation for which ones.

OpenAI's [migration guide](https://developers.openai.com/api/docs/guides/migrate-to-responses) says Chat Completions remains supported but recommends the Responses API for all new projects. A compatibility layer has more to reproduce for Responses: typed input and output items instead of a `messages` array, optional server-side state through `previous_response_id`, hosted tools, and typed streaming events.

Two self-hosted engines show the usual coverage:

- [vLLM](https://docs.vllm.ai/en/latest/serving/online_serving/openai_compatible_server/) serves Completions, Chat Completions, and Responses for generation models, plus Embeddings and audio transcription for the matching model types. Chat requests need a chat template in the model's tokenizer configuration, or they fail.
- [Ollama](https://docs.ollama.com/api/openai-compatibility) describes its support as a subset of the OpenAI API: Chat Completions, Completions, Models, Embeddings, audio endpoints, and a `/v1/responses` endpoint that is non-stateful only, without `previous_response_id`.

## Where compatibility breaks

Compatibility breaks at the edges of the format: parameters a server ignores, provider features that have no OpenAI field, and [tool calling](https://theaiengineering.com/glossary/tool-calling) behavior that differs underneath identical JSON. Many servers ignore unsupported fields instead of rejecting them, so a request can succeed while dropping a setting the application depends on.

Each project documents its own gaps:

- Anthropic's [OpenAI SDK compatibility layer](https://platform.claude.com/docs/en/cli-sdks-libraries/libraries/openai-sdk) is intended mainly for testing and comparing models, and Anthropic does not consider it a long-term or production-ready path for most uses. It ignores the `strict` flag on tool definitions, so tool arguments are not guaranteed to match the schema, does not support [prompt caching](https://theaiengineering.com/glossary/prompt-caching), hoists every system message into one system prompt, and silently ignores most unsupported fields.
- Ollama's Chat Completions endpoint does not support `tool_choice`, `logprobs`, `logit_bias`, or `n`.
- vLLM accepts parameters that OpenAI's API lacks, such as `top_k`, through the SDK's `extra_body` argument. Code that depends on such extensions stops being portable.

The same request can also produce different [structured outputs](https://theaiengineering.com/glossary/structured-outputs), stop reasons, and usage fields on different backends. Test the exact features you rely on, including [streaming](https://theaiengineering.com/glossary/llm-streaming), tool calls, and JSON output, against each backend before switching, and keep a native-API path for provider features the compatible endpoint cannot express. The [vLLM vs Ollama comparison](https://theaiengineering.com/articles/vllm-vs-ollama) covers the OpenAI-compatible servers of the main self-hosted engines.

## Related terms

- [AI gateway](https://theaiengineering.com/glossary/ai-gateway): An AI gateway is a proxy layer between applications and model providers that exposes one API for many models and centralizes routing, failover, access control, cost tracking, caching, and logging.
- [Inference engine](https://theaiengineering.com/glossary/inference-engine): An inference engine is the software that serves a trained language model: it loads the weights, batches incoming requests, manages KV cache memory, runs the forward passes, and streams generated tokens over an API.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [LLM streaming](https://theaiengineering.com/glossary/llm-streaming): LLM streaming is the delivery of a language model's output in small chunks as it is generated, usually as server-sent events over HTTP, so clients can show text before the response is complete.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.

## Guides that cover openai-compatible api

- [What is an AI gateway? How it works and when you need one](https://theaiengineering.com/articles/what-is-an-ai-gateway): An AI gateway is a proxy between your apps and model providers. Learn how it works, its core features, AI gateway vs API gateway, and open source options.
- [vLLM vs Ollama vs llama.cpp vs SGLang: which engine to use](https://theaiengineering.com/articles/vllm-vs-ollama): vLLM vs Ollama compared with llama.cpp, SGLang, LM Studio, and TensorRT-LLM: scope, hardware, APIs, concurrency, and which engine fits local or production use.
- [Bifrost vs LiteLLM: architecture, features, and performance compared](https://theaiengineering.com/articles/bifrost-vs-litellm): Bifrost vs LiteLLM compared: a Go single binary against a Python proxy, with deployment, routing, virtual keys, caching, MCP, benchmarks, and migration.

Part of the [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) topic hub.

## Cite this definition

The AI Engineering Editors. "What is an OpenAI-compatible API? Endpoints and limits." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/openai-compatible-api

- Title: What is an OpenAI-compatible API? Endpoints and limits
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/openai-compatible-api
- Markdown: https://theaiengineering.com/glossary/openai-compatible-api.md
