---
title: "What is chain-of-thought prompting in LLMs?"
description: "Chain-of-thought prompting has an LLM write reasoning steps before answering. How few-shot and zero-shot CoT work, and how CoT differs from reasoning models."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/chain-of-thought"
markdown_url: "https://theaiengineering.com/glossary/chain-of-thought.md"
topic: "Prompt and context engineering"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Chain-of-thought prompting","CoT","chain of thought","CoT prompting"]
citation: "The AI Engineering Editors. \"What is chain-of-thought prompting in LLMs?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/chain-of-thought"
---

# What is chain-of-thought prompting in LLMs?

Also called CoT, chain of thought, CoT prompting.

> **Definition of Chain-of-thought prompting:** Chain-of-thought prompting is a technique that has a language model write intermediate reasoning steps before its final answer, which improves large models' accuracy on arithmetic, commonsense, and symbolic reasoning tasks.

Chain-of-thought (CoT) prompting is a [prompt engineering](https://theaiengineering.com/glossary/prompt-engineering) technique that has a language model write out intermediate reasoning steps before it gives a final answer. Wei et al. introduced it in 2022 by adding worked examples with step-by-step solutions to the prompt. Writing the steps gives the model more generated tokens to work through a problem and makes the path to the answer readable, which helps on arithmetic, commonsense, and symbolic reasoning problems. [Reasoning models](https://theaiengineering.com/glossary/reasoning-model) generate a chain of thought without being asked, which changes when the technique is worth using.

## How chain-of-thought prompting works

Chain-of-thought prompting works by showing or telling the model to reason before it answers. Few-shot CoT places worked examples in the prompt, each with a question, the reasoning steps, and the answer, so the model imitates the format. Zero-shot CoT adds a single instruction, such as "Let's think step by step," with no examples.

A few-shot exemplar looks like this:

```text
Q: A warehouse has 3 shelves with 12 boxes each. 7 boxes ship. How many boxes remain?
A: 3 shelves times 12 boxes is 36 boxes. 36 minus 7 is 29. The answer is 29.
```

The research behind the technique:

- **Few-shot CoT.** [Wei et al. (2022)](https://arxiv.org/abs/2201.11903) reported that eight chain-of-thought exemplars let a 540B-parameter model reach the highest reported accuracy at the time on the GSM8K math word problem benchmark, beating a fine-tuned GPT-3 with a verifier. The authors describe the ability as emerging naturally in sufficiently large models.
- **Zero-shot CoT.** [Kojima et al. (2022)](https://arxiv.org/abs/2205.11916) found that appending "Let's think step by step" raised InstructGPT (text-davinci-002) accuracy from 17.7% to 78.7% on MultiArith and from 10.4% to 40.7% on GSM8K.
- **Self-consistency.** [Wang et al. (2022)](https://arxiv.org/abs/2203.11171) replaced a single greedy answer with several sampled reasoning paths and kept the most consistent final answer, reporting a 17.9% gain on GSM8K over standard chain-of-thought prompting. Sampling several paths is one form of [test-time compute](https://theaiengineering.com/glossary/test-time-compute).

CoT also affects how benchmarks are run. Chain-of-thought instructions are one reason two published scores for the same model differ, and the MMLU-Pro authors report a benefit from CoT that the original MMLU did not show. Fine-tuning recipes use it too: RAFT trains models to answer from retrieved documents in a chain-of-thought style that quotes the relevant passage verbatim.

## Chain-of-thought prompting vs reasoning models

Chain-of-thought prompting asks a standard model to reason in its visible output. A reasoning model is trained, usually with reinforcement learning, to generate its own chain of thought before answering. With a reasoning model, the reasoning happens whether or not the prompt asks for it, so step-by-step instructions add little and can sometimes hurt performance.

| | Chain-of-thought prompting | Reasoning model |
|---|---|---|
| Source of the reasoning | Examples or an instruction in the prompt | Training, usually reinforcement learning |
| Where the reasoning appears | In the visible response | In reasoning tokens, often hidden or summarized |
| How to control it | Prompt wording and exemplars | Effort or thinking settings |
| Works with | Standard [LLMs](https://theaiengineering.com/glossary/large-language-model) | Models trained to reason |

OpenAI's [reasoning best practices guide](https://developers.openai.com/api/docs/guides/reasoning-best-practices) says that prompting its reasoning models to think step by step or explain their reasoning is unnecessary. Manual chain-of-thought prompting remains a fallback for standard models and for requests where thinking is turned off. For how reasoning models generate their chain of thought, see [reasoning model](https://theaiengineering.com/glossary/reasoning-model) and the [reasoning models explainer](https://theaiengineering.com/articles/reasoning-models).

## Related terms

- [Reasoning model](https://theaiengineering.com/glossary/reasoning-model): A reasoning model is a large language model trained, usually with reinforcement learning, to generate intermediate reasoning tokens before its final answer, spending more inference compute to solve multi-step problems.
- [Prompt engineering](https://theaiengineering.com/glossary/prompt-engineering): Prompt engineering is the practice of designing the instructions, examples, and formatting given to a large language model so it produces accurate, consistent output for a task, without changing model weights.
- [Few-shot prompting](https://theaiengineering.com/glossary/few-shot-prompting): Few-shot prompting is a technique that puts a few example inputs paired with desired outputs in a prompt, so a language model infers the task, format, and labels without changing its weights.
- [Test-time compute](https://theaiengineering.com/glossary/test-time-compute): Test-time compute is the computation a language model spends while answering a query, through longer reasoning, multiple samples, or search, as opposed to the train-time compute spent building the model.
- [Large language model (LLM)](https://theaiengineering.com/glossary/large-language-model): A large language model (LLM) is a neural network, usually a transformer with billions of parameters, trained on web-scale text to predict the next token and then adapted to follow instructions.

## Guides that cover chain-of-thought prompting

- [Reasoning models: how they work and when to use them](https://theaiengineering.com/articles/reasoning-models): Reasoning models explained: how test-time compute and reinforcement learning produce chain-of-thought reasoning, when to use them, and their cost and latency.
- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.
- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.
- [AI agent frameworks compared: LangGraph, CrewAI, ADK, and more](https://theaiengineering.com/articles/ai-agent-frameworks): An AI agent frameworks comparison of LangGraph, LangChain, CrewAI, the OpenAI, Claude, and Google SDKs, Pydantic AI, Mastra, and more, by language and license.

Part of the [Prompt and context engineering](https://theaiengineering.com/topics/prompt-engineering) topic hub.

## Cite this definition

The AI Engineering Editors. "What is chain-of-thought prompting in LLMs?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/chain-of-thought

- Title: What is chain-of-thought prompting in LLMs?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/chain-of-thought
- Markdown: https://theaiengineering.com/glossary/chain-of-thought.md
