What is an OpenAI-compatible API? Endpoints and limits
Also called OpenAI-compatible endpoint, OpenAI-compatible server, OpenAI API compatibility
Definition
An OpenAI-compatible API is an HTTP API that accepts OpenAI's request and response formats, so code written for OpenAI's SDKs can call another server by changing the base URL, API key, and model name.
An OpenAI-compatible API is an endpoint that implements the paths, request fields, and response shapes of OpenAI's API, most often /v1/chat/completions. Client code built on an OpenAI SDK can then call a self-hosted inference engine, another model provider, or an AI gateway by changing the base URL, API key, and model name. Serving engines such as vLLM and Ollama, many AI gateways, and model providers such as Anthropic and Google (Gemini's compatibility layer is in beta) expose the format. In practice, each server supports a documented subset of OpenAI's fields.
Which endpoints are usually covered#
OpenAI-compatible servers cover Chat Completions first, usually with a models list and often with Completions and Embeddings. Support for the newer Responses API is less complete, especially for its server-side state. Treat "OpenAI-compatible" as a claim about specific endpoints and fields, and check the server's documentation for which ones.
OpenAI's migration guide says Chat Completions remains supported but recommends the Responses API for all new projects. A compatibility layer has more to reproduce for Responses: typed input and output items instead of a messages array, optional server-side state through previous_response_id, hosted tools, and typed streaming events.
Two self-hosted engines show the usual coverage:
- vLLM serves Completions, Chat Completions, and Responses for generation models, plus Embeddings and audio transcription for the matching model types. Chat requests need a chat template in the model's tokenizer configuration, or they fail.
- Ollama describes its support as a subset of the OpenAI API: Chat Completions, Completions, Models, Embeddings, audio endpoints, and a
/v1/responsesendpoint that is non-stateful only, withoutprevious_response_id.
Where compatibility breaks#
Compatibility breaks at the edges of the format: parameters a server ignores, provider features that have no OpenAI field, and tool calling behavior that differs underneath identical JSON. Many servers ignore unsupported fields instead of rejecting them, so a request can succeed while dropping a setting the application depends on.
Each project documents its own gaps:
- Anthropic's OpenAI SDK compatibility layer is intended mainly for testing and comparing models, and Anthropic does not consider it a long-term or production-ready path for most uses. It ignores the
strictflag on tool definitions, so tool arguments are not guaranteed to match the schema, does not support prompt caching, hoists every system message into one system prompt, and silently ignores most unsupported fields. - Ollama's Chat Completions endpoint does not support
tool_choice,logprobs,logit_bias, orn. - vLLM accepts parameters that OpenAI's API lacks, such as
top_k, through the SDK'sextra_bodyargument. Code that depends on such extensions stops being portable.
The same request can also produce different structured outputs, stop reasons, and usage fields on different backends. Test the exact features you rely on, including streaming, tool calls, and JSON output, against each backend before switching, and keep a native-API path for provider features the compatible endpoint cannot express. The vLLM vs Ollama comparison covers the OpenAI-compatible servers of the main self-hosted engines.
Related terms
Guides that cover openai-compatible api
Cite this definition
The AI Engineering Editors. "What is an OpenAI-compatible API? Endpoints and limits." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/openai-compatible-api
- Title
- What is an OpenAI-compatible API? Endpoints and limits
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated