---
title: "What is a context window in LLMs?"
description: "A context window is the maximum number of tokens an LLM can process in one request, input and output combined. Learn how it works and why bigger is not better."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/context-window"
markdown_url: "https://theaiengineering.com/glossary/context-window.md"
topic: "AI engineering"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Context window","context length","context size","context limit"]
citation: "The AI Engineering Editors. \"What is a context window in LLMs?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/context-window"
---

# What is a context window in LLMs?

Also called context length, context size, context limit.

> **Definition of Context window:** A context window is the maximum number of tokens a language model can take into account in a single request, including the system prompt, conversation history, tool results, and the generated output.

A context window is the maximum number of [tokens](https://theaiengineering.com/glossary/tokens) a large language model can process in one request. Everything the model sees counts against it: the [system prompt](https://theaiengineering.com/glossary/system-prompt), messages, tool definitions, tool results, attached documents, and the response the model generates. Anthropic's documentation describes the context window as the model's working memory, distinct from the data it was trained on.

## How a context window works

A context window works as a hard token budget per request. The application assembles a prompt, the model processes it, and each generated token is added to the same budget until the response ends or the limit is reached. In a multi-turn chat, every turn resends the earlier history, so the window fills as the conversation grows.

When the input alone exceeds the window, the API rejects the request; Anthropic's API, for example, returns a prompt-too-long error. Applications handle long inputs by dropping old turns, summarizing history (often called [compaction](https://theaiengineering.com/glossary/context-compaction)), or retrieving only relevant passages with [retrieval-augmented generation](https://theaiengineering.com/glossary/retrieval-augmented-generation). [Prompt caching](https://theaiengineering.com/glossary/prompt-caching) makes a repeated prefix cheaper to process, but cached tokens still occupy the window.

## Context window vs max tokens

The context window and max tokens are two different limits. The context window is the model's total capacity for input plus output, fixed by the model. Max tokens, set with a request parameter such as `max_tokens`, is a cap on output length chosen by the caller, and the output must fit in whatever room the input leaves.

| | Context window | Max tokens |
|---|---|---|
| Set by | The model | The API caller |
| Covers | Input and output | Output only |
| When exceeded | Request rejected, or generation stopped at the limit | Output stops early |

## Why the context window matters

The context window matters because it bounds how much information a model can use at once, yet filling it has costs. [Lost in the Middle](https://arxiv.org/abs/2307.03172) (Liu et al., 2023) found that models use information at the start and end of a long input more reliably than information in the middle. Anthropic's [context window documentation](https://platform.claude.com/docs/en/build-with-claude/context-windows) notes that accuracy and recall degrade as token count grows, a pattern it calls [context rot](https://theaiengineering.com/glossary/context-rot). Long inputs also raise cost and latency, because attention compute and the [KV cache](https://theaiengineering.com/glossary/kv-cache) grow with sequence length.

A larger window is therefore a ceiling, not a target. [Context engineering](https://theaiengineering.com/articles/context-engineering) covers how to choose what goes into the window and what stays out.

## Related terms

- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [KV cache](https://theaiengineering.com/glossary/kv-cache): The KV cache is GPU memory that stores the attention key and value vectors already computed for previous tokens, so a transformer generating text computes them once instead of at every step.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.
- [Context engineering](https://theaiengineering.com/glossary/context-engineering): Context engineering is the practice of selecting, structuring, and pruning all the information placed in an LLM's context window at each step, including instructions, retrieved data, tool results, and memory.
- [Retrieval-augmented generation (RAG)](https://theaiengineering.com/glossary/retrieval-augmented-generation): Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
- [Context rot](https://theaiengineering.com/glossary/context-rot): Context rot is the decline in a language model's accuracy and recall as the number of tokens in its input grows, even when the information a task needs is present in the context.

## Guides that cover context window

- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.
- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [Claude Code vs Cursor: how the two coding agents differ](https://theaiengineering.com/articles/claude-code-vs-cursor): Claude Code vs Cursor compared: terminal agent vs AI editor, models, subagents, hooks, rules files, code review, CI use, pricing, and enterprise controls.
- [How to build an AI agent: a step-by-step guide in Python](https://theaiengineering.com/articles/how-to-build-an-ai-agent): Learn how to build an AI agent from scratch in Python: when you need one, tool schemas, the agent loop, stop conditions, guardrails, evals, and tracing.
- [AI coding agents: how they work and how the major tools compare](https://theaiengineering.com/articles/ai-coding-agents): AI coding agents compared: how they differ from autocomplete, the four main categories, a comparison table of major tools, and how to evaluate and secure them.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [Chunking strategies for RAG: how to choose, tune, and evaluate them](https://theaiengineering.com/articles/chunking-strategies-for-rag): Chunking strategies for RAG compared: fixed-size, recursive, document-aware, semantic, parent-child, and late chunking, with tested Python and an eval method.
- [RAG vs fine-tuning: how to choose for your LLM application](https://theaiengineering.com/articles/rag-vs-fine-tuning): RAG vs fine-tuning: RAG adds knowledge at query time, while fine-tuning changes model behavior. Compare freshness, cost, latency, data needs, LoRA, and hybrids.
- [What is tool calling? How LLM function calling works](https://theaiengineering.com/articles/tool-calling): Tool calling lets an LLM request actions from your code through structured calls. Learn the request loop, parallel calls, structured outputs, security, and MCP.
- [What is the Model Context Protocol (MCP)? How it works](https://theaiengineering.com/articles/what-is-model-context-protocol): The Model Context Protocol (MCP) is an open standard for connecting AI apps to tools and data. Learn its architecture, transports, tool calls, and security.
- [AI engineer roadmap: skills, projects, and resources by stage](https://theaiengineering.com/articles/ai-engineer-roadmap): A practical AI engineer roadmap in seven stages, each with the skills to learn, a project to ship, and resources worth the time, from Python to production.
- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.
- [What is AI engineering? The discipline, the stack, and the work](https://theaiengineering.com/articles/what-is-ai-engineering): What is AI engineering? A plain definition, how it differs from ML and software engineering, the layers of the stack, and what AI engineers do day to day.

Part of the [AI engineering](https://theaiengineering.com/topics/ai-engineering) topic hub.

## Cite this definition

The AI Engineering Editors. "What is a context window in LLMs?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/context-window

- Title: What is a context window in LLMs?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/context-window
- Markdown: https://theaiengineering.com/glossary/context-window.md
