Reasoning models: how they work and when to use them

Explainer · Models

Reasoning models are large language models trained to generate a sequence of intermediate reasoning tokens, often called a chain of thought, before they produce a final answer. Reasoning models trade extra inference compute for accuracy on multistep problems such as competition math, complex coding, and planning, and OpenAI, Anthropic, Google, and DeepSeek all ship them. This explainer covers how reasoning models work, how they are trained, when they are worth their cost, and how prompting them differs. It is part of the models topic hub.

What are reasoning models?#

A reasoning model is an LLM post-trained to produce an extended chain of intermediate reasoning before answering, spending more tokens on harder problems. Reasoning tokens give the model room to break a problem into steps, check intermediate results, and revise an approach before the visible answer begins.

OpenAI introduced o1 in September 2024, and DeepSeek released the open-weight DeepSeek-R1 in January 2025, with a detailed training paper later published in Nature. Since then, reasoning has become a configurable mode rather than a separate product line: OpenAI's API exposes a reasoning.effort parameter, Anthropic's API offers adaptive thinking controlled by an effort setting, and Google's Gemini API exposes a thinking_level setting. On many current models, one set of weights serves both quick answers and long reasoning.

Reasoning models vs LLMs#

A reasoning model is still a transformer LLM that predicts one token at a time; the difference is in post-training and in how much output each request generates.

Dimension Standard LLM response Reasoning model response
Output before the answer None or a short preamble Often hundreds to thousands of reasoning tokens
Post-training emphasis Instruction following, preference tuning Reinforcement learning on verifiable tasks, plus preference tuning
Time to first visible token Prefill time plus queueing Prefill plus all reasoning tokens, unless summaries stream
Cost per request Prompt plus answer tokens Prompt plus reasoning plus answer tokens
Strongest on Extraction, classification, chat, summarization Math, complex code, multistep planning, analysis
Main risk Shallow answers on hard problems Latency, token cost, and overlong reasoning on easy inputs

What is test-time compute?#

Test-time compute is the computation a model spends while answering a query, as opposed to the computation spent training it. Test-time compute rises in three ways: generating a longer chain of reasoning, sampling several answers and combining them, or searching over partial solutions with a verifier. More test-time compute generally raises accuracy on hard problems, with diminishing returns.

Two rows compare a standard LLM, which goes from prompt straight to answer tokens, with a reasoning model, which generates billed reasoning tokens before the answer. Standard LLMReasoning modelPromptAnswer tokensVisible answerPromptReasoning tokensTest-time computeHidden or summarizedBilled as output tokensAnswer tokensVisible answer
Figure 1: A reasoning model generates billed reasoning tokens before the visible answer.

The three approaches have separate research lineages:

  • Sequential reasoning. The model writes a longer chain of thought and revises it. OpenAI reported in September 2024 that o1's performance improved with more reinforcement learning (train-time compute) and with more time spent reasoning (test-time compute).
  • Parallel sampling. Self-consistency (Wang et al., 2022) samples several reasoning paths and takes a majority vote over final answers; best-of-N selection uses a verifier or reward model instead of a vote.
  • Search with verifiers. A process reward model scores intermediate steps, and the system expands the most promising partial solutions.

Scaling LLM Test-Time Compute Optimally (Snell et al., 2024) found that the best method depends on problem difficulty. A compute-optimal strategy that allocates test-time compute per prompt improved efficiency by more than 4 times over a best-of-N baseline. In a FLOPs-matched comparison, a smaller model with extra test-time compute outperformed a model 14 times larger on problems where the smaller model already had a non-trivial success rate.

Test-time compute can also be controlled with simple tricks. The s1 paper (Muennighoff et al., 2025) fine-tuned Qwen2.5-32B-Instruct on 1,000 reasoning examples and used budget forcing: appending "Wait" when the model tried to stop, which extended reasoning and often led it to fix errors, or cutting reasoning off to save tokens.

How are reasoning models trained?#

Published recipes train reasoning models by taking a pretrained base model and applying reinforcement learning on problems with checkable answers, such as math with known solutions and code with unit tests. The reward depends on whether the final answer is correct and well formatted, not on matching human-written reasoning, so the model learns whatever intermediate steps raise its success rate.

This differs from RLHF, which trains against a reward model learned from human preference ratings. Verifiable rewards are cheap to compute at scale and less exposed to reward hacking than learned reward models; DeepSeek-R1's final training stage combined rule-based rewards for reasoning with reward models for helpfulness and harmlessness.

The DeepSeek-R1 recipe#

The DeepSeek-R1 paper (DeepSeek-AI, 2025; published in Nature) gives a detailed public account of training a reasoning model. Its first model, DeepSeek-R1-Zero, applied reinforcement learning directly to the DeepSeek-V3-Base model with no supervised fine-tuning first. Key design choices:

  • GRPO. Group Relative Policy Optimization estimates the baseline for each answer from the scores of a group of sampled answers, which removes the separate critic model that PPO requires.
  • Rule-based rewards. An accuracy reward checks the final answer, and a format reward checks that reasoning appears between designated tags. The authors avoided neural reward models because those can suffer from reward hacking during large-scale reinforcement learning.
  • Emergent behavior. Response length grew steadily during training, and the authors describe an "aha moment" in which an intermediate checkpoint learned to re-evaluate its initial approach.

R1-Zero's outputs had poor readability and mixed languages. The final DeepSeek-R1 used a four-stage pipeline: supervised fine-tuning on thousands of long chain-of-thought cold-start examples, reasoning-focused reinforcement learning, rejection sampling plus supervised fine-tuning on a broader dataset, and a final reinforcement learning stage covering helpfulness and harmlessness alongside reasoning.

Distilling reasoning into smaller models#

DeepSeek also released six dense models from 1.5B to 70B parameters, built on Qwen2.5 and Llama 3, by fine-tuning them on reasoning samples generated by DeepSeek-R1. The paper reports that this knowledge distillation outperformed applying reinforcement learning directly to the smaller models: a distilled Qwen 32B model beat a 32B model trained with the R1-Zero reinforcement learning recipe across the reported benchmarks. Distilled models of this kind fit on a single GPU, and the smallest run on laptops.

When to use reasoning models#

Use a reasoning model when a task requires several dependent steps and has a checkable answer, such as math, debugging, multistep planning, or analysis of long, ambiguous inputs. Use a standard model, or the lowest reasoning effort, for classification, extraction, short chat, or answers drawn from facts already in the context, where extra reasoning mostly adds latency.

OpenAI's reasoning best practices guide describes reasoning models as planners and faster general models as workhorses, and recommends combining them: a reasoning model plans a multistep task, and a cheaper model executes the individual steps. The same split appears in AI agents, where a reasoning-capable model plans tool calls across a long loop and smaller models handle narrow subtasks.

Good fits:

  • Competition-style math and quantitative analysis
  • Code changes that span several files, and debugging from stack traces
  • Planning and tool calling across long agent loops
  • Contract, policy, or research review where conclusions depend on many details

Poor fits:

  • High-volume classification and entity extraction
  • Customer-facing chat with tight latency targets
  • Retrieval-grounded answers where the answer sits in the provided documents

When traffic mixes easy and hard requests, route by difficulty instead of choosing one model for everything. The LLM routing guide covers classifiers and cascades that send only hard prompts to a reasoning model or a higher effort setting.

Reasoning tokens, cost, and latency#

Reasoning tokens are billed as output tokens and count toward the output limit on the OpenAI, Anthropic, and Google APIs, even when the API returns only a summary or nothing at all. Because reasoning precedes the answer, reasoning length adds to the wait before the first visible answer token. Cost and latency therefore scale with the reasoning each request triggers.

Practical consequences for engineers:

  • Budget output limits. OpenAI's reasoning guide warns that if a response hits max_output_tokens during reasoning, the request ends incomplete and you pay for input and reasoning tokens without receiving an answer. OpenAI recommends reserving at least 25,000 tokens for reasoning and output when starting out.
  • Measure reasoning separately. OpenAI reports output_tokens_details.reasoning_tokens, Anthropic reports thinking tokens inside its usage object, and Gemini reports thought tokens. Track those fields per route in your observability stack.
  • Expect higher first-token latency. Time to first token for the visible answer includes all reasoning tokens. Streaming reasoning summaries, where the API offers them, gives users a progress signal.
  • Pass reasoning back in tool loops. In multi-turn tool use, OpenAI asks callers to return reasoning items (encrypted in stateless mode), Anthropic asks callers to pass thinking blocks back unchanged, and Gemini requires thought signatures to be resent unchanged in stateless mode.
  • Keep thinking settings stable. Anthropic's documentation notes that changing the thinking configuration between requests invalidates prompt caching breakpoints.

On self-hosted models, reasoning traffic is decode-heavy: short prompts, very long outputs. Speculative decoding and KV cache capacity matter more than prefill speed, as the LLM inference optimization guide explains.

How to prompt reasoning models#

Prompt a reasoning model with a short, specific statement of the goal, the constraints, and what a correct result looks like, and leave the steps to its trained reasoning. OpenAI and Anthropic guidance treats step-by-step instructions as unnecessary, and prescriptive step lists can produce worse reasoning than a general instruction.

Provider guidance converges on a few rules:

  • State the goal and success criteria. OpenAI recommends brief, direct instructions, explicit constraints, and precise success criteria.
  • Skip chain-of-thought instructions. OpenAI advises against asking reasoning models to reason step by step. Anthropic finds that a general instruction such as "think thoroughly" often produces better reasoning than a prescriptive step-by-step plan.
  • Start zero-shot. Add few-shot examples only if results fall short, and make them match your instructions. Anthropic notes that worked examples shape how its models approach similar problems.
  • Separate inputs clearly. Use markdown headings or XML tags to delimit documents, data, and instructions.
  • Ask for a self-check. Anthropic recommends asking the model to verify its answer against test criteria before finishing, while noting that some newer models already verify without being asked.
  • Read answers from structured fields. Select content blocks by type rather than position, and use structured outputs instead of parsing reasoning text.

More on writing prompts and assembling inputs is in prompt engineering and the context engineering explainer.

Chain of thought prompting vs reasoning models#

Chain-of-thought prompting (Wei et al., 2022) added a few worked examples with intermediate steps to a prompt and showed that sufficiently large models then reasoned better: a 540B-parameter model with eight exemplars reached state-of-the-art accuracy on the GSM8K math benchmark at the time. Reasoning models internalize that behavior through training. Manual chain-of-thought prompting remains a fallback for standard models, or when thinking is turned off.

Limitations of reasoning models#

Reasoning models still produce wrong answers, sometimes with long and confident reasoning attached. Three limitations matter in production: the reasoning trace may not reflect how the answer was produced, hallucination rates do not automatically fall, and reasoning length can overshoot on easy inputs.

  • Unfaithful reasoning. In Reasoning Models Don't Always Say What They Think (Chen et al., 2025), Anthropic researchers planted hints in prompts and found that models often used a hint without mentioning it, with reveal rates frequently below 20%. A reasoning trace is useful for debugging, but it is not an audit log.
  • Hallucination. OpenAI's o3 and o4-mini system card reported that o3 made more claims than o1 on OpenAI's internal PersonQA evaluation, producing more accurate claims and more hallucinated ones. Reasoning does not replace grounding or verification.
  • Overthinking. Anthropic's prompting documentation warns that some models can reason more extensively than a task needs, inflating tokens and latency, and recommends lowering the effort setting or adding instructions that constrain reasoning.
  • Benchmark comparability. Scores depend on the effort setting used, so compare reasoning models at stated, matched settings; the LLM benchmarks explainer covers what else to check.

Reasoning models are most valuable on the hard tail of a workload, so measure where that tail is before paying for reasoning on every request. Next, read the LLM benchmarks explainer to interpret reasoning-model scores, the LLM routing guide to send only hard prompts to them, and the release tracker for new reasoning models as they ship.

Frequently asked questions

What is the difference between a reasoning model and an LLM?

A reasoning model is a type of LLM, not a separate architecture. Both are transformer language models that generate tokens one at a time. A reasoning model is additionally trained, usually with reinforcement learning on verifiable tasks, to produce a long chain of intermediate reasoning before its final answer. That extra output improves accuracy on multistep problems such as math, coding, and planning, at the cost of more tokens and higher latency per request.

What is test-time compute?

Test-time compute is the computation a model spends while answering a query, as opposed to the computation spent training it. Reasoning models raise test-time compute by generating longer chains of reasoning, sampling several candidate answers and voting, or searching over partial solutions with a verifier. OpenAI reported that o1's accuracy improved with more test-time compute, and later research showed that allocating it by problem difficulty improves efficiency.

How are reasoning models trained?

Reasoning models start from a pretrained base model and are post-trained with reinforcement learning on problems that have checkable answers, such as math with known solutions or code with unit tests. The reward depends on whether the final answer is correct, not on human-written reasoning steps. DeepSeek-R1 added a small cold-start fine-tuning stage for readability, more reinforcement learning rounds, and supervised fine-tuning on filtered outputs.

What is the difference between reasoning tokens and output tokens?

Reasoning tokens are the intermediate tokens a reasoning model generates before its visible answer; output tokens are everything the model generates. On the OpenAI, Anthropic, and Google APIs, reasoning tokens are billed as output tokens and count toward the output limit, even when the API returns only a summary or nothing. Usage fields in each API response report how many billed output tokens were reasoning.

When should you use reasoning vs non-reasoning models?

Use a reasoning model when a task needs several dependent steps and has an answer you can check: math, complex coding and debugging, multistep planning in agents, and analysis of long, ambiguous material. Use a non-reasoning model, or the lowest reasoning effort, for classification, extraction, short chat replies, and retrieval-grounded answers where the facts are already in the context. Route by difficulty when traffic mixes both.

Are reasoning models more prone to hallucination?

Reasoning models are not uniformly more or less prone to hallucination. OpenAI's o3 and o4-mini system card reported that o3 made more claims overall than o1 on OpenAI's internal PersonQA evaluation, which produced more accurate claims and also more hallucinated ones. Measure hallucination on your own task with a grounded eval set, and do not treat a long reasoning trace as evidence that an answer is correct.

Terms in this guide

Cite this guide

The AI Engineering Editors. "Reasoning models: how they work and when to use them." The AI Engineering, updated Sep 23, 2026. https://theaiengineering.com/articles/reasoning-models

Title
Reasoning models: how they work and when to use them
Author
The AI Engineering Editors
Publisher
The AI Engineering
Published
Updated
View as Markdown

The AI Engineering Editors ·