---
title: "What are structured outputs in LLMs?"
description: "Structured outputs force an LLM response to match a JSON Schema through constrained decoding. How it works and how it differs from JSON mode and tool calling."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/structured-outputs"
markdown_url: "https://theaiengineering.com/glossary/structured-outputs.md"
topic: "Prompt and context engineering"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Structured outputs","structured output","JSON schema mode","constrained decoding"]
citation: "The AI Engineering Editors. \"What are structured outputs in LLMs?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/structured-outputs"
---

# What are structured outputs in LLMs?

Also called structured output, JSON schema mode, constrained decoding.

> **Definition of Structured outputs:** Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.

Structured outputs is a feature of model APIs and [inference engines](https://theaiengineering.com/glossary/inference-engine) that guarantees a response matches a schema, usually [JSON Schema](https://theaiengineering.com/glossary/json-schema), instead of relying on the model to follow a formatting instruction. OpenAI launched Structured Outputs on August 6, 2024, and reported that on its own complex-schema eval, gpt-4o-2024-08-06 with the feature scored 100%, while gpt-4-0613 scored under 40%. Anthropic and open-source servers such as vLLM offer equivalent features.

## How structured outputs work

The underlying technique is constrained decoding. The provider compiles the schema into a grammar. At each generation step the model assigns a probability to every [token](https://theaiengineering.com/glossary/tokens) in its vocabulary, and the decoder masks out any token that would make the output violate the grammar. If the schema says the next value is a boolean, only tokens that can begin `true` or `false` remain eligible.

Anthropic's documentation says its structured outputs "guarantee schema-compliant responses through constrained decoding" and lists two exceptions: a refusal may not match the schema, and a response cut off at the token limit may be incomplete. Providers also support only a subset of JSON Schema. Anthropic, for example, rejects recursive schemas and numeric constraints such as `minimum` and `maximum`. vLLM supports JSON Schema, regular expressions, and context-free grammars through its xgrammar and guidance backends.

## Structured outputs vs JSON mode

| Approach | Valid JSON guaranteed | Schema match guaranteed | Typical use |
|---|---|---|---|
| Prompting for JSON | No | No | Prototypes |
| JSON mode | Yes | No | Free-form JSON objects |
| Structured outputs | Yes | Yes | Extraction, classification, API responses |
| Strict [tool calling](https://theaiengineering.com/glossary/tool-calling) | Yes, for arguments | Yes, for arguments | Actions the application executes |

OpenAI's [Structured Outputs guide](https://developers.openai.com/api/docs/guides/structured-outputs) states the difference directly: both features produce valid JSON, but only Structured Outputs ensures schema adherence. The same guide recommends tool calling when the model needs to connect to functions or data, and a response schema when the reply itself must have a fixed shape.

## Why structured outputs matter

Without schema enforcement, a pipeline needs parsing code, validation, and retry loops for malformed responses. Structured outputs remove that class of failure, which matters most in multi-step agents, where one malformed response can break every later step. Schema-valid output can still be wrong: a correctly typed field can contain a [hallucination](https://theaiengineering.com/glossary/hallucination). Value-level checks therefore still belong in [guardrails](https://theaiengineering.com/glossary/guardrails) and evals, and schema design is part of [prompt engineering](https://theaiengineering.com/glossary/prompt-engineering), since clear field names and descriptions guide what the model writes into each field.

## Related terms

- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [Prompt engineering](https://theaiengineering.com/glossary/prompt-engineering): Prompt engineering is the practice of designing the instructions, examples, and formatting given to a large language model so it produces accurate, consistent output for a task, without changing model weights.
- [Guardrails](https://theaiengineering.com/glossary/guardrails): Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.
- [JSON Schema](https://theaiengineering.com/glossary/json-schema): JSON Schema is a declarative vocabulary, itself written in JSON, for describing the structure of JSON data; LLM APIs and MCP use it to define tool arguments and constrain model output.

## Guides that cover structured outputs

- [What is tool calling? How LLM function calling works](https://theaiengineering.com/articles/tool-calling): Tool calling lets an LLM request actions from your code through structured calls. Learn the request loop, parallel calls, structured outputs, security, and MCP.
- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.
- [Agentic workflows: five patterns and when to use each](https://theaiengineering.com/articles/agentic-workflows): Agentic workflows run LLM and tool calls through steps fixed in code. Learn five patterns, their cost and latency, failure modes, and when to use an agent.
- [How to build an MCP server: a Python and TypeScript tutorial](https://theaiengineering.com/articles/how-to-build-an-mcp-server): Learn how to build an MCP server with the official Python and TypeScript SDKs: tools, resources, transports, testing, client config, and production security.
- [vLLM vs Ollama vs llama.cpp vs SGLang: which engine to use](https://theaiengineering.com/articles/vllm-vs-ollama): vLLM vs Ollama compared with llama.cpp, SGLang, LM Studio, and TensorRT-LLM: scope, hardware, APIs, concurrency, and which engine fits local or production use.
- [AI agent frameworks compared: LangGraph, CrewAI, ADK, and more](https://theaiengineering.com/articles/ai-agent-frameworks): An AI agent frameworks comparison of LangGraph, LangChain, CrewAI, the OpenAI, Claude, and Google SDKs, Pydantic AI, Mastra, and more, by language and license.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.
- [Reasoning models: how they work and when to use them](https://theaiengineering.com/articles/reasoning-models): Reasoning models explained: how test-time compute and reinforcement learning produce chain-of-thought reasoning, when to use them, and their cost and latency.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [What is the Model Context Protocol (MCP)? How it works](https://theaiengineering.com/articles/what-is-model-context-protocol): The Model Context Protocol (MCP) is an open standard for connecting AI apps to tools and data. Learn its architecture, transports, tool calls, and security.
- [AI engineer roadmap: skills, projects, and resources by stage](https://theaiengineering.com/articles/ai-engineer-roadmap): A practical AI engineer roadmap in seven stages, each with the skills to learn, a project to ship, and resources worth the time, from Python to production.

Part of the [Prompt and context engineering](https://theaiengineering.com/topics/prompt-engineering) topic hub.

## Cite this definition

The AI Engineering Editors. "What are structured outputs in LLMs?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/structured-outputs

- Title: What are structured outputs in LLMs?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/structured-outputs
- Markdown: https://theaiengineering.com/glossary/structured-outputs.md
