What is context compaction for AI agents?

Also called compaction, conversation summarization

Definition

Context compaction is a context management technique that replaces older conversation turns with a model-written summary as the context window nears its limit, so an agent can keep working.

Context compaction is the practice of summarizing a conversation as it approaches the context window limit and continuing in a fresh window that starts from the summary. Anthropic's post Effective context engineering for AI agents (September 29, 2025) lists compaction among the main techniques for long-horizon agents. An AI agent that runs for hours adds tool calls and results on every step, so without compaction it either hits the window limit or carries so much stale history that context rot degrades its work.

How context compaction works#

Compaction works by sending the history to a model with instructions to summarize it, then replacing the summarized turns with that summary. A good summary keeps decisions, open problems, and the current plan, and drops redundant tool output. Compaction can run automatically at a token threshold, on demand when the application requests it, or in the background.

Anthropic's post describes how Claude Code does it: the message history goes to the model, which preserves architectural decisions, unresolved bugs, and implementation details while discarding redundant tool outputs, and the agent continues with that summary plus the five most recently accessed files. The post advises tuning the summarization prompt for recall first, then for precision. Claude Code's cost documentation shows the user-facing side: auto-compaction near the limit, and a /compact command that accepts instructions such as which details to preserve.

Model providers now run compaction on the server:

  • Anthropic: the Messages API's compaction feature, in beta, replaces older turns with a summary Claude writes on the server, either on demand or when input tokens reach a trigger the developer sets.
  • OpenAI: the Responses API compacts server-side once the token count crosses a compact_threshold, or through a standalone /responses/compact endpoint. Its compaction item is encrypted and not meant to be human-readable.

Compaction vs truncation#

Truncation drops the oldest messages outright, which is cheap and predictable but loses whatever they contained. Compaction replaces those messages with a summary, which costs a model call and can omit details, but carries decisions and state forward. Tool result clearing sits between the two: it removes old raw tool outputs while keeping the record of each call.

Truncation Tool result clearing Compaction
What it removes The oldest turns Old raw tool outputs Older turns, replaced by a summary
Extra model call No No Yes
What survives Recent turns only Every turn, minus bulky outputs A summary plus recent turns
Main risk Losing early instructions or facts Needing a cleared output again The summary dropping a needed detail

Anthropic's post calls tool result clearing one of the safest, lightest-touch forms of compaction. Compaction and tool result clearing both rewrite the history, so the next request misses the prompt cache for the rewritten part; Claude Code reports that miss after compaction or tool-result clearing as an expected rebuild.

Compaction vs agent memory#

Compaction keeps a running session inside the window. Agent memory stores information outside the window, in files or databases, so it survives across sessions and can be reloaded on demand. They work together when an agent writes durable notes before compaction and reloads them afterward, so the plan survives even if the summary loses a detail.

Anthropic's post calls this structured note-taking, with examples such as a to-do list or a NOTES.md file that the agent maintains. Compaction is lossy, so keep it restorable: when a summary drops a document's content, keep its file path or URL so the agent can fetch it again. Track how often tasks fail after a compaction to tell whether the summary keeps what later steps need.

The context engineering guide shows compaction alongside retrieval, note-taking, and sub-agents in one agent run.

Cite this definition

The AI Engineering Editors. "What is context compaction for AI agents?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/context-compaction

Title
What is context compaction for AI agents?
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the Prompt and context engineering hub · Updated