---
title: "What is temperature in LLMs? Settings explained"
description: "LLM temperature is a sampling setting that controls how predictable or varied model output is. Learn how temperature works and how it differs from top-p."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/temperature"
markdown_url: "https://theaiengineering.com/glossary/temperature.md"
topic: "AI engineering"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Temperature","LLM temperature","sampling temperature"]
citation: "The AI Engineering Editors. \"What is temperature in LLMs? Settings explained.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/temperature"
---

# What is temperature in LLMs? Settings explained

Also called LLM temperature, sampling temperature.

> **Definition of Temperature:** Temperature is a sampling parameter that rescales a language model's next-token probabilities: low values concentrate probability on the likeliest tokens, and high values flatten the distribution for more varied output.

Temperature is a sampling parameter that controls how random a language model's output is. At each step, a [large language model](https://theaiengineering.com/glossary/large-language-model) produces a score, called a logit, for every [token](https://theaiengineering.com/glossary/tokens) in its vocabulary, and temperature rescales those scores before they become probabilities. Lower temperature makes output more consistent across runs; higher temperature makes it more varied.

## How temperature works

Temperature works by dividing each logit by the temperature value T before the softmax function converts the logits into probabilities.

```text
p(token_i) = exp(logit_i / T) / Σ_j exp(logit_j / T)
```
*Figure 1: Softmax with temperature T.*

- **T below 1** sharpens the distribution, so the top tokens receive even more probability. As T approaches 0, sampling approaches greedy decoding, which always picks the single likeliest token.
- **T equal to 1** leaves the model's distribution unchanged.
- **T above 1** flattens the distribution, so unlikely tokens are sampled more often; very high values produce incoherent text.

Temperature 0 does not guarantee identical outputs. Anthropic's [migration guide for Claude Opus 5.5](https://platform.claude.com/docs/en/models/opus-5-5/migration-guide) states that `temperature = 0` never guaranteed identical outputs on earlier models, and it tells developers to omit `temperature`, `top_p`, and `top_k` on that model and guide behavior through prompting instead. Check each model's documentation before relying on the parameter.

## Temperature vs top-p

Temperature and top-p both shape sampling, but they act on different parts of the distribution. Temperature reshapes the whole distribution. Top-p, or nucleus sampling, introduced by [Holtzman et al. (2019)](https://arxiv.org/abs/1904.09751), keeps the smallest set of tokens whose cumulative probability reaches p and samples only from that set, cutting off the unreliable tail. Top-k is a simpler variant that keeps a fixed number of top tokens. The settings interact, so changing one at a time makes the effect easier to measure.

## Example

For extraction, classification, and code generation, a low temperature is the usual choice because repeated runs give similar output, and the output can then be validated with [structured outputs](https://theaiengineering.com/glossary/structured-outputs) or tests. For brainstorming or generating varied synthetic data, a higher temperature adds diversity. Because sampling adds variance at any setting above zero, [LLM evaluation](https://theaiengineering.com/articles/llm-evaluation) should run each test case several times rather than once.

## Related terms

- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [Large language model (LLM)](https://theaiengineering.com/glossary/large-language-model): A large language model (LLM) is a neural network, usually a transformer with billions of parameters, trained on web-scale text to predict the next token and then adapted to follow instructions.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [LLM evaluation](https://theaiengineering.com/glossary/llm-evaluation): LLM evaluation is the practice of measuring how well a language model or LLM-based application performs on defined tasks, using test datasets, automated graders, model-based judges, and human review.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.

## Guides that cover temperature

- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.

Part of the [AI engineering](https://theaiengineering.com/topics/ai-engineering) topic hub.

## Cite this definition

The AI Engineering Editors. "What is temperature in LLMs? Settings explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/temperature

- Title: What is temperature in LLMs? Settings explained
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/temperature
- Markdown: https://theaiengineering.com/glossary/temperature.md
