---
title: "Reasoning models: how they work and when to use them"
description: "Reasoning models explained: how test-time compute and reinforcement learning produce chain-of-thought reasoning, when to use them, and their cost and latency."
summary: "Reasoning models are LLMs trained to generate intermediate reasoning tokens before a final answer, trading extra inference compute for accuracy on multistep problems. This explainer covers test-time compute, the DeepSeek-R1 training recipe, when a standard model fits better, and prompting."
type: "explainer"
canonical_url: "https://theaiengineering.com/articles/reasoning-models"
markdown_url: "https://theaiengineering.com/articles/reasoning-models.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-09-23"
updated: "2026-09-23"
keywords: ["reasoning models","reasoning models vs llm","test-time compute","chain of thought","reasoning tokens","deepseek r1"]
citation: "The AI Engineering Editors. \"Reasoning models: how they work and when to use them.\" The AI Engineering, updated Sep 23, 2026. https://theaiengineering.com/articles/reasoning-models"
---

# Reasoning models: how they work and when to use them

## TL;DR

- A reasoning model is a large language model trained, usually with reinforcement learning on tasks with verifiable answers, to generate intermediate reasoning tokens before its final answer.
- Test-time compute is the extra inference work spent on each query, through longer reasoning, multiple samples, or search, and research shows that allocating it by problem difficulty can beat a much larger model on some problems.
- DeepSeek-R1 showed that large-scale reinforcement learning with rule-based rewards for answer accuracy and format can produce long chain-of-thought reasoning, and that the behavior can be distilled into smaller dense models.
- Reasoning tokens are billed as output tokens and count toward output limits on the OpenAI, Anthropic, and Google APIs, so cost and latency scale with how much reasoning each request triggers.
- Reasoning models need shorter prompts than standard models: state the goal, constraints, and success criteria, and skip instructions to think step by step.

Reasoning models are [large language models](https://theaiengineering.com/glossary/large-language-model) trained to generate a sequence of intermediate reasoning tokens, often called a chain of thought, before they produce a final answer. Reasoning models trade extra inference compute for accuracy on multistep problems such as competition math, complex coding, and planning, and OpenAI, Anthropic, Google, and DeepSeek all ship them. This explainer covers how reasoning models work, how they are trained, when they are worth their cost, and how prompting them differs. It is part of the [models topic hub](https://theaiengineering.com/topics/models).

## What are reasoning models?

A [reasoning model](https://theaiengineering.com/glossary/reasoning-model) is an LLM post-trained to produce an extended chain of intermediate reasoning before answering, spending more tokens on harder problems. Reasoning tokens give the model room to break a problem into steps, check intermediate results, and revise an approach before the visible answer begins.

OpenAI introduced o1 in September 2024, and DeepSeek released the [open-weight](https://theaiengineering.com/glossary/open-weights-model) DeepSeek-R1 in January 2025, with a detailed training paper later published in Nature. Since then, reasoning has become a configurable mode rather than a separate product line: OpenAI's API exposes a `reasoning.effort` parameter, Anthropic's API offers adaptive thinking controlled by an effort setting, and Google's Gemini API exposes a `thinking_level` setting. On many current models, one set of weights serves both quick answers and long reasoning.

### Reasoning models vs LLMs

A reasoning model is still a transformer LLM that predicts one token at a time; the difference is in post-training and in how much output each request generates.

| Dimension | Standard LLM response | Reasoning model response |
|---|---|---|
| Output before the answer | None or a short preamble | Often hundreds to thousands of reasoning tokens |
| Post-training emphasis | Instruction following, preference tuning | Reinforcement learning on verifiable tasks, plus preference tuning |
| Time to first visible token | Prefill time plus queueing | Prefill plus all reasoning tokens, unless summaries stream |
| Cost per request | Prompt plus answer tokens | Prompt plus reasoning plus answer tokens |
| Strongest on | Extraction, classification, chat, summarization | Math, complex code, multistep planning, analysis |
| Main risk | Shallow answers on hard problems | Latency, token cost, and overlong reasoning on easy inputs |

## What is test-time compute?

[Test-time compute](https://theaiengineering.com/glossary/test-time-compute) is the computation a model spends while answering a query, as opposed to the computation spent training it. Test-time compute rises in three ways: generating a longer chain of reasoning, sampling several answers and combining them, or searching over partial solutions with a verifier. More test-time compute generally raises accuracy on hard problems, with diminishing returns.

**Figure 1: A reasoning model generates billed reasoning tokens before the visible answer.**

Two rows compare a standard LLM, which goes from prompt straight to answer tokens, with a reasoning model, which generates billed reasoning tokens before the answer.

Components:

- **Prompt**
- **Answer tokens**: Visible answer
- **Prompt**
- **Reasoning tokens**: Test-time compute Hidden or summarized Billed as output tokens
- **Answer tokens**: Visible answer

Groups:

- Standard LLM: Prompt, Answer tokens
- Reasoning model: Prompt, Reasoning tokens, Answer tokens

Connections:

- Prompt → Answer tokens
- Prompt → Reasoning tokens
- Reasoning tokens → Answer tokens

The three approaches have separate research lineages:

- **Sequential reasoning.** The model writes a longer chain of thought and revises it. OpenAI reported in September 2024 that o1's performance improved with more reinforcement learning (train-time compute) and with more time spent reasoning (test-time compute).
- **Parallel sampling.** Self-consistency (Wang et al., 2022) samples several reasoning paths and takes a majority vote over final answers; best-of-N selection uses a verifier or reward model instead of a vote.
- **Search with verifiers.** A process reward model scores intermediate steps, and the system expands the most promising partial solutions.

[Scaling LLM Test-Time Compute Optimally](https://arxiv.org/abs/2408.03314) (Snell et al., 2024) found that the best method depends on problem difficulty. A compute-optimal strategy that allocates test-time compute per prompt improved efficiency by more than 4 times over a best-of-N baseline. In a FLOPs-matched comparison, a smaller model with extra test-time compute outperformed a model 14 times larger on problems where the smaller model already had a non-trivial success rate.

Test-time compute can also be controlled with simple tricks. The s1 paper (Muennighoff et al., 2025) fine-tuned Qwen2.5-32B-Instruct on 1,000 reasoning examples and used budget forcing: appending "Wait" when the model tried to stop, which extended reasoning and often led it to fix errors, or cutting reasoning off to save tokens.

## How are reasoning models trained?

Published recipes train reasoning models by taking a [pretrained](https://theaiengineering.com/glossary/pretraining) base model and applying reinforcement learning on problems with checkable answers, such as math with known solutions and code with unit tests. The reward depends on whether the final answer is correct and well formatted, not on matching human-written reasoning, so the model learns whatever intermediate steps raise its success rate.

This differs from [RLHF](https://theaiengineering.com/glossary/rlhf), which trains against a reward model learned from human preference ratings. Verifiable rewards are cheap to compute at scale and less exposed to reward hacking than learned reward models; DeepSeek-R1's final training stage combined rule-based rewards for reasoning with reward models for helpfulness and harmlessness.

### The DeepSeek-R1 recipe

The [DeepSeek-R1 paper](https://arxiv.org/abs/2501.12948) (DeepSeek-AI, 2025; published in Nature) gives a detailed public account of training a reasoning model. Its first model, DeepSeek-R1-Zero, applied reinforcement learning directly to the DeepSeek-V3-Base model with no supervised fine-tuning first. Key design choices:

- **GRPO.** Group Relative Policy Optimization estimates the baseline for each answer from the scores of a group of sampled answers, which removes the separate critic model that PPO requires.
- **Rule-based rewards.** An accuracy reward checks the final answer, and a format reward checks that reasoning appears between designated tags. The authors avoided neural reward models because those can suffer from reward hacking during large-scale reinforcement learning.
- **Emergent behavior.** Response length grew steadily during training, and the authors describe an "aha moment" in which an intermediate checkpoint learned to re-evaluate its initial approach.

R1-Zero's outputs had poor readability and mixed languages. The final DeepSeek-R1 used a four-stage pipeline: supervised fine-tuning on thousands of long chain-of-thought cold-start examples, reasoning-focused reinforcement learning, rejection sampling plus supervised fine-tuning on a broader dataset, and a final reinforcement learning stage covering helpfulness and harmlessness alongside reasoning.

### Distilling reasoning into smaller models

DeepSeek also released six dense models from 1.5B to 70B parameters, built on Qwen2.5 and Llama 3, by fine-tuning them on reasoning samples generated by DeepSeek-R1. The paper reports that this [knowledge distillation](https://theaiengineering.com/glossary/knowledge-distillation) outperformed applying reinforcement learning directly to the smaller models: a distilled Qwen 32B model beat a 32B model trained with the R1-Zero reinforcement learning recipe across the reported benchmarks. Distilled models of this kind fit on a single GPU, and the smallest run on laptops.

## When to use reasoning models

Use a reasoning model when a task requires several dependent steps and has a checkable answer, such as math, debugging, multistep planning, or analysis of long, ambiguous inputs. Use a standard model, or the lowest reasoning effort, for classification, extraction, short chat, or answers drawn from facts already in the context, where extra reasoning mostly adds latency.

OpenAI's [reasoning best practices guide](https://developers.openai.com/api/docs/guides/reasoning-best-practices) describes reasoning models as planners and faster general models as workhorses, and recommends combining them: a reasoning model plans a multistep task, and a cheaper model executes the individual steps. The same split appears in [AI agents](https://theaiengineering.com/articles/what-are-ai-agents), where a reasoning-capable model plans tool calls across a long loop and smaller models handle narrow subtasks.

Good fits:

- Competition-style math and quantitative analysis
- Code changes that span several files, and debugging from stack traces
- Planning and tool calling across long agent loops
- Contract, policy, or research review where conclusions depend on many details

Poor fits:

- High-volume classification and entity extraction
- Customer-facing chat with tight latency targets
- Retrieval-grounded answers where the answer sits in the provided documents

When traffic mixes easy and hard requests, route by difficulty instead of choosing one model for everything. The [LLM routing guide](https://theaiengineering.com/articles/llm-routing) covers classifiers and [cascades](https://theaiengineering.com/glossary/model-cascade) that send only hard prompts to a reasoning model or a higher effort setting.

## Reasoning tokens, cost, and latency

Reasoning tokens are billed as output tokens and count toward the output limit on the OpenAI, Anthropic, and Google APIs, even when the API returns only a summary or nothing at all. Because reasoning precedes the answer, reasoning length adds to the wait before the first visible answer token. Cost and latency therefore scale with the reasoning each request triggers.

Practical consequences for engineers:

- **Budget output limits.** OpenAI's reasoning guide warns that if a response hits `max_output_tokens` during reasoning, the request ends incomplete and you pay for input and reasoning tokens without receiving an answer. OpenAI recommends reserving at least 25,000 tokens for reasoning and output when starting out.
- **Measure reasoning separately.** OpenAI reports `output_tokens_details.reasoning_tokens`, Anthropic reports thinking tokens inside its usage object, and Gemini reports thought tokens. Track those fields per route in your observability stack.
- **Expect higher first-token latency.** [Time to first token](https://theaiengineering.com/glossary/time-to-first-token) for the visible answer includes all reasoning tokens. Streaming reasoning summaries, where the API offers them, gives users a progress signal.
- **Pass reasoning back in tool loops.** In multi-turn tool use, OpenAI asks callers to return reasoning items (encrypted in stateless mode), Anthropic asks callers to pass thinking blocks back unchanged, and Gemini requires thought signatures to be resent unchanged in stateless mode.
- **Keep thinking settings stable.** Anthropic's documentation notes that changing the thinking configuration between requests invalidates prompt caching breakpoints.

On self-hosted models, reasoning traffic is [decode-heavy](https://theaiengineering.com/glossary/prefill-and-decode): short prompts, very long outputs. Speculative decoding and KV cache capacity matter more than prefill speed, as the [LLM inference optimization guide](https://theaiengineering.com/articles/llm-inference-optimization) explains.

## How to prompt reasoning models

Prompt a reasoning model with a short, specific statement of the goal, the constraints, and what a correct result looks like, and leave the steps to its trained reasoning. OpenAI and Anthropic guidance treats step-by-step instructions as unnecessary, and prescriptive step lists can produce worse reasoning than a general instruction.

Provider guidance converges on a few rules:

- **State the goal and success criteria.** OpenAI recommends brief, direct instructions, explicit constraints, and precise success criteria.
- **Skip chain-of-thought instructions.** OpenAI advises against asking reasoning models to reason step by step. Anthropic finds that a general instruction such as "think thoroughly" often produces better reasoning than a prescriptive step-by-step plan.
- **Start zero-shot.** Add few-shot examples only if results fall short, and make them match your instructions. Anthropic notes that worked examples shape how its models approach similar problems.
- **Separate inputs clearly.** Use markdown headings or XML tags to delimit documents, data, and instructions.
- **Ask for a self-check.** Anthropic recommends asking the model to verify its answer against test criteria before finishing, while noting that some newer models already verify without being asked.
- **Read answers from structured fields.** Select content blocks by type rather than position, and use structured outputs instead of parsing reasoning text.

More on writing prompts and assembling inputs is in prompt engineering and the [context engineering explainer](https://theaiengineering.com/articles/context-engineering).

### Chain of thought prompting vs reasoning models

[Chain-of-thought prompting](https://theaiengineering.com/glossary/chain-of-thought) (Wei et al., 2022) added a few worked examples with intermediate steps to a prompt and showed that sufficiently large models then reasoned better: a 540B-parameter model with eight exemplars reached state-of-the-art accuracy on the GSM8K math benchmark at the time. Reasoning models internalize that behavior through training. Manual chain-of-thought prompting remains a fallback for standard models, or when thinking is turned off.

## Limitations of reasoning models

Reasoning models still produce wrong answers, sometimes with long and confident reasoning attached. Three limitations matter in production: the reasoning trace may not reflect how the answer was produced, hallucination rates do not automatically fall, and reasoning length can overshoot on easy inputs.

- **Unfaithful reasoning.** In [Reasoning Models Don't Always Say What They Think](https://arxiv.org/abs/2505.05410) (Chen et al., 2025), Anthropic researchers planted hints in prompts and found that models often used a hint without mentioning it, with reveal rates frequently below 20%. A reasoning trace is useful for debugging, but it is not an audit log.
- **Hallucination.** OpenAI's o3 and o4-mini system card reported that o3 made more claims than o1 on OpenAI's internal PersonQA evaluation, producing more accurate claims and more hallucinated ones. Reasoning does not replace grounding or verification.
- **Overthinking.** Anthropic's prompting documentation warns that some models can reason more extensively than a task needs, inflating tokens and latency, and recommends lowering the effort setting or adding instructions that constrain reasoning.
- **Benchmark comparability.** Scores depend on the effort setting used, so compare reasoning models at stated, matched settings; the [LLM benchmarks explainer](https://theaiengineering.com/articles/llm-benchmarks) covers what else to check.

Reasoning models are most valuable on the hard tail of a workload, so measure where that tail is before paying for reasoning on every request. Next, read the [LLM benchmarks explainer](https://theaiengineering.com/articles/llm-benchmarks) to interpret reasoning-model scores, the [LLM routing guide](https://theaiengineering.com/articles/llm-routing) to send only hard prompts to them, and the [release tracker](https://theaiengineering.com/releases) for new reasoning models as they ship.

## Frequently asked questions

### What is the difference between a reasoning model and an LLM?

A reasoning model is a type of LLM, not a separate architecture. Both are transformer language models that generate tokens one at a time. A reasoning model is additionally trained, usually with reinforcement learning on verifiable tasks, to produce a long chain of intermediate reasoning before its final answer. That extra output improves accuracy on multistep problems such as math, coding, and planning, at the cost of more tokens and higher latency per request.

### What is test-time compute?

Test-time compute is the computation a model spends while answering a query, as opposed to the computation spent training it. Reasoning models raise test-time compute by generating longer chains of reasoning, sampling several candidate answers and voting, or searching over partial solutions with a verifier. OpenAI reported that o1's accuracy improved with more test-time compute, and later research showed that allocating it by problem difficulty improves efficiency.

### How are reasoning models trained?

Reasoning models start from a pretrained base model and are post-trained with reinforcement learning on problems that have checkable answers, such as math with known solutions or code with unit tests. The reward depends on whether the final answer is correct, not on human-written reasoning steps. DeepSeek-R1 added a small cold-start fine-tuning stage for readability, more reinforcement learning rounds, and supervised fine-tuning on filtered outputs.

### What is the difference between reasoning tokens and output tokens?

Reasoning tokens are the intermediate tokens a reasoning model generates before its visible answer; output tokens are everything the model generates. On the OpenAI, Anthropic, and Google APIs, reasoning tokens are billed as output tokens and count toward the output limit, even when the API returns only a summary or nothing. Usage fields in each API response report how many billed output tokens were reasoning.

### When should you use reasoning vs non-reasoning models?

Use a reasoning model when a task needs several dependent steps and has an answer you can check: math, complex coding and debugging, multistep planning in agents, and analysis of long, ambiguous material. Use a non-reasoning model, or the lowest reasoning effort, for classification, extraction, short chat replies, and retrieval-grounded answers where the facts are already in the context. Route by difficulty when traffic mixes both.

### Are reasoning models more prone to hallucination?

Reasoning models are not uniformly more or less prone to hallucination. OpenAI's o3 and o4-mini system card reported that o3 made more claims overall than o1 on OpenAI's internal PersonQA evaluation, which produced more accurate claims and also more hallucinated ones. Measure hallucination on your own task with a grounded eval set, and do not treat a long reasoning trace as evidence that an answer is correct.

## Terms in this guide

- [Reasoning model](https://theaiengineering.com/glossary/reasoning-model): A reasoning model is a large language model trained, usually with reinforcement learning, to generate intermediate reasoning tokens before its final answer, spending more inference compute to solve multi-step problems.
- [Large language model (LLM)](https://theaiengineering.com/glossary/large-language-model): A large language model (LLM) is a neural network, usually a transformer with billions of parameters, trained on web-scale text to predict the next token and then adapted to follow instructions.
- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [RLHF](https://theaiengineering.com/glossary/rlhf): RLHF (reinforcement learning from human feedback) is a fine-tuning method that trains a reward model on human rankings of model outputs, then optimizes the language model with reinforcement learning to score highly.
- [Knowledge distillation](https://theaiengineering.com/glossary/knowledge-distillation): Knowledge distillation is a model compression technique in which a smaller student model is trained to reproduce the outputs or output probabilities of a larger, more capable teacher model.
- [Time to first token](https://theaiengineering.com/glossary/time-to-first-token): Time to first token (TTFT) is the elapsed time from submitting a request to an LLM until the first token of the response arrives, covering request queuing, network transit, and prompt prefill.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.
- [Chain-of-thought prompting](https://theaiengineering.com/glossary/chain-of-thought): Chain-of-thought prompting is a technique that has a language model write intermediate reasoning steps before its final answer, which improves large models' accuracy on arithmetic, commonsense, and symbolic reasoning tasks.
- [Open-weights model](https://theaiengineering.com/glossary/open-weights-model): An open-weights model is an AI model whose trained parameters are published for download, so anyone can run, fine-tune, or quantize it on their own hardware under the terms of its license.
- [Test-time compute](https://theaiengineering.com/glossary/test-time-compute): Test-time compute is the computation a language model spends while answering a query, through longer reasoning, multiple samples, or search, as opposed to the train-time compute spent building the model.
- [Pretraining](https://theaiengineering.com/glossary/pretraining): Pretraining is the first stage of training a language model, in which it learns to predict the next token across a large unlabeled corpus of text and code, producing a general-purpose base model.
- [Model cascade](https://theaiengineering.com/glossary/model-cascade): A model cascade is an LLM serving pattern that calls models in order from cheapest to most capable and returns the first answer that passes a quality check, escalating only when it fails.
- [LLM streaming](https://theaiengineering.com/glossary/llm-streaming): LLM streaming is the delivery of a language model's output in small chunks as it is generated, usually as server-sent events over HTTP, so clients can show text before the response is complete.
- [Prefill and decode](https://theaiengineering.com/glossary/prefill-and-decode): Prefill and decode are the two phases of LLM inference: prefill processes the whole prompt in one parallel pass and fills the KV cache, and decode generates one output token per forward pass.
- [Speculative decoding](https://theaiengineering.com/glossary/speculative-decoding): Speculative decoding is an inference technique where a cheap draft proposes a few tokens and the target model verifies them in one forward pass, emitting several tokens per step without changing the output distribution.
- [Few-shot prompting](https://theaiengineering.com/glossary/few-shot-prompting): Few-shot prompting is a technique that puts a few example inputs paired with desired outputs in a prompt, so a language model infers the task, format, and labels without changing its weights.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [Prompt engineering](https://theaiengineering.com/glossary/prompt-engineering): Prompt engineering is the practice of designing the instructions, examples, and formatting given to a large language model so it produces accurate, consistent output for a task, without changing model weights.

## Related guides

- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.
- [LLM inference optimization: a practical guide for engineers](https://theaiengineering.com/articles/llm-inference-optimization): LLM inference optimization guide: prefill vs decode, TTFT, KV cache and PagedAttention, continuous batching, quantization, speculative decoding, and GPUs.
- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.

Part of the [LLM models and benchmarks](https://theaiengineering.com/topics/models) topic hub.

## Cite this guide

The AI Engineering Editors. "Reasoning models: how they work and when to use them." The AI Engineering, updated Sep 23, 2026. https://theaiengineering.com/articles/reasoning-models

- Title: Reasoning models: how they work and when to use them
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-09-23
- Updated: 2026-09-23
- URL: https://theaiengineering.com/articles/reasoning-models
- Markdown: https://theaiengineering.com/articles/reasoning-models.md

To cite one section, link to its anchor:

- [What are reasoning models?](https://theaiengineering.com/articles/reasoning-models#what-are-reasoning-models)
- [What is test-time compute?](https://theaiengineering.com/articles/reasoning-models#what-is-test-time-compute)
- [How are reasoning models trained?](https://theaiengineering.com/articles/reasoning-models#how-are-reasoning-models-trained)
- [When to use reasoning models](https://theaiengineering.com/articles/reasoning-models#when-to-use-reasoning-models)
- [Reasoning tokens, cost, and latency](https://theaiengineering.com/articles/reasoning-models#reasoning-tokens-cost-and-latency)
- [How to prompt reasoning models](https://theaiengineering.com/articles/reasoning-models#how-to-prompt-reasoning-models)
- [Limitations of reasoning models](https://theaiengineering.com/articles/reasoning-models#limitations-of-reasoning-models)
- [Frequently asked questions](https://theaiengineering.com/articles/reasoning-models#faq)
