---
title: "What is context engineering? A practical guide for AI agents"
description: "Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results."
summary: "Context engineering is the discipline of deciding which instructions, tools, retrieved data, memory, and history fill a model's context window at each agent step. This explainer compares it with prompt engineering, reviews the evidence on context rot, and covers techniques and metrics."
type: "explainer"
canonical_url: "https://theaiengineering.com/articles/context-engineering"
markdown_url: "https://theaiengineering.com/articles/context-engineering.md"
topic: "Prompt and context engineering"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-10-16"
updated: "2026-11-11"
keywords: ["context engineering","context engineering vs prompt engineering","context window","agent memory","context rot","context engineering for ai agents"]
citation: "The AI Engineering Editors. \"What is context engineering? A practical guide for AI agents.\" The AI Engineering, updated Nov 11, 2026. https://theaiengineering.com/articles/context-engineering"
---

# What is context engineering? A practical guide for AI agents

## TL;DR

- Context engineering is the practice of curating every token a model receives at inference time, including the system prompt, tool definitions, retrieved documents, tool results, memory, and conversation history.
- Prompt engineering covers how instructions are written; context engineering manages everything that competes for the same context window, and it repeats on every step of an agent loop.
- Research including Lost in the Middle, NoLiMa, and Chroma's Context Rot report shows model accuracy falls as input length grows, so a larger context window does not remove the need to curate.
- The main techniques are just-in-time retrieval, compaction, tool result clearing, external memory, sub-agents with isolated context, and stable prompt prefixes that keep caches valid.
- Context engineering should be measured with task success on a fixed evaluation set alongside input tokens per task, cache hit rate, and failures caused by dropped information.

Context engineering is the practice of deciding which information fills a language model's context window at each step of a task: the instructions, tool definitions, retrieved documents, tool results, memory, and conversation history. Context engineering matters most for AI agents, because an agent rebuilds that context on every model call during a run and every token adds cost, latency, and noise.

This explainer defines the term, compares it with prompt engineering, breaks the context window into its components, reviews the research on long-context degradation, and covers the techniques and metrics teams use in production.

## What is context engineering?

Context engineering is the design of systems that select, format, and maintain the tokens a language model receives at inference time, so each call has what the task needs. Anthropic's engineering team frames the goal as finding the smallest set of high-signal tokens that produces the desired outcome, and treats context as a finite resource.

The term spread in June 2025, when Shopify CEO Tobi Lütke and former OpenAI researcher Andrej Karpathy both posted that they preferred "context engineering" to "prompt engineering." Karpathy's argument was that people associate prompts with short task descriptions, while production LLM applications must fill the [context window](https://theaiengineering.com/glossary/context-window) with much more than that. Anthropic's September 2025 guide, [Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents), then described the practice in detail for agent builders.

## Context engineering vs prompt engineering

Prompt engineering is the craft of writing instructions for a model; context engineering is the broader job of managing everything in the context window, including those instructions. A prompt is usually written once and tested. An agent's context is assembled again on every model call from changing sources, so context engineering resembles systems design more than writing.

| Dimension | Prompt engineering | Context engineering |
|---|---|---|
| Main object | Instructions and examples | Every token in the window |
| When it happens | Design time | Every model call at runtime |
| Typical owner | Whoever writes the prompt | Engineers building retrieval, memory, and tool layers |
| Common failure | Ambiguous or conflicting instructions | Missing facts, stale or distracting content, overflow |
| Main methods | Wording, structure, few-shot examples | Retrieval, compaction, memory stores, sub-agents, caching |
| How to test | Prompt variants on an eval set | Ablations of context sources, plus token and cache metrics |

[Prompt engineering](https://theaiengineering.com/glossary/prompt-engineering) does not go away. The [system prompt](https://theaiengineering.com/glossary/system-prompt) and tool descriptions are still written by hand, and their wording still changes results. Context engineering adds the runtime decisions around them: which documents to retrieve, which tool output to keep, and what to summarize or drop as the window fills.

## Why context engineering is important for AI agents

Context engineering became important with AI agents because agents accumulate context as they work. Each tool call adds its arguments and results to the conversation, so a long task can fill the window with stale file contents, logs, and search results. That growth raises cost and latency, and it lowers accuracy unless something removes or condenses the excess.

A chat application sends one question and receives one answer. An agent runs a loop of model calls, and each call re-sends the accumulated context as input [tokens](https://theaiengineering.com/glossary/tokens). The Manus team, writing about its production agent in July 2025, reported an average input-to-output token ratio of around 100 to 1. In that profile, input tokens account for most of the processing, which is why the reuse of a cached prefix through [prompt caching](https://theaiengineering.com/glossary/prompt-caching) and the KV cache matters so much for agents.

Quality is the second reason. Anthropic's guide describes an "attention budget": in a transformer, every token attends to every other token, which creates n² pairwise relationships for n tokens, and each added token draws down that budget. The [AI agents explainer](https://theaiengineering.com/articles/what-are-ai-agents) covers the loop itself; context engineering is the discipline of keeping that loop's input small and relevant.

## What goes into the context window?

An agent's context window holds six kinds of content: the system prompt, tool definitions, memory loaded for the task, retrieved context, conversation history, and tool calls with their results. Each has a different source and growth pattern, so each needs its own policy for what to include, when to include it, and in what form.

**Figure 1: A typical agent context window, ordered from most stable to most volatile, which keeps the cached prefix valid.**

Seven stacked layers of an agent context window, from the stable system prompt and tool definitions at the top down to fast-growing tool results and the latest message.

Layers, top to bottom:

1. **System prompt and project instructions**: Stable
2. **Tool definitions**: Stable · names and JSON schemas
3. **Memory loaded for this task**: Per task
4. **Retrieved documents and file contents**: Per step
5. **Conversation history or compacted summary**: Grows
6. **Tool calls and tool results**: Grows fast
7. **Latest user message or next instruction**: Per step

| Component | Source | Growth pattern | Main control |
|---|---|---|---|
| System prompt | Written by developers | Fixed | Specific rules, no contradictions |
| Tool definitions | Application code or MCP servers | Grows with each integration | Fewer, distinct tools; defer rare ones |
| Memory | Notes files, databases, memory tools | Loaded selectively | Write summaries; load by relevance |
| Retrieved context | Search, RAG, file reads | Per step | Retrieve just in time; rerank; cap size |
| Conversation history | User and model turns | Steady | Compact near the limit |
| Tool results | Tool execution | Fast, every step | Truncate, paginate, clear old results |

### Tool definitions

Tool definitions are easy to overlook because they do not appear in the conversation. OpenAI's function calling guide notes that function definitions are injected into the system message, count against the context limit, and are billed as input tokens. The same guide suggests keeping fewer than 20 functions available at the start of a turn as a soft target, and deferring rarely used tools with tool search. Each MCP server an agent connects to adds more definitions, so the [tool calling guide](https://theaiengineering.com/articles/tool-calling) treats tool count as a design decision.

A gateway can trim the list before it reaches the model. [Bifrost](https://docs.getbifrost.ai/mcp/code-mode) filters MCP tools per virtual key, and its Code Mode replaces the full tool catalog with four meta-tools that the model uses to look up tool signatures and run a short Python-like script against them. In Bifrost's own benchmark with 508 tools across 16 servers, Code Mode cut input tokens by 92.8% at the same pass rate.

### Memory

[Agent memory](https://theaiengineering.com/glossary/agent-memory) is information stored outside the window and loaded back when relevant. Short-term memory is the working state of the current task. Long-term memory persists across sessions in notes files, databases, or provider features such as Anthropic's memory tool. Coding agents use project files for the same purpose: Claude Code reads `CLAUDE.md` at the start of every session and can save its own notes across sessions.

### Conversation history

Conversation history is the record of user turns, model turns, and tool exchanges. History is the component that eventually overflows the window, so most agents need a [compaction policy](https://theaiengineering.com/glossary/context-compaction) that decides when to summarize and what the summary must preserve.

## What is context rot?

[Context rot](https://theaiengineering.com/glossary/context-rot) is the decline in a model's accuracy as the amount of input grows, even when the information the task needs is present. Chroma used the term as the title of a July 2025 research report, and Anthropic's context window documentation now uses it to explain why curating context matters as much as the size of the window.

Three lines of research support the finding:

| Study | What it tested | Key finding |
|---|---|---|
| [Lost in the Middle](https://arxiv.org/abs/2307.03172) (Liu et al., 2023) | Multi-document QA and key-value retrieval | Accuracy is highest when relevant information is at the start or end of the input and drops when it sits in the middle |
| [NoLiMa](https://arxiv.org/abs/2502.05167) (Modarressi et al., 2025) | Retrieval with minimal word overlap between question and answer | At 32K tokens, 11 of 13 models fell below 50% of their short-context scores |
| [Context Rot](https://www.trychroma.com/research/context-rot) (Hong, Troynikov, and Huber, Chroma, 2025) | 18 models on needle retrieval, conversational QA, and word replication | Performance changes with input length even on simple tasks, and a single distractor lowers accuracy |

The Chroma report adds two details that matter for agents. Accuracy falls faster as input grows when the question and the relevant passage are less similar, a common situation in agent work. Distractors, meaning passages that are topically related to the answer but do not contain it, lower accuracy, and more distractors lower it further; agent contexts are full of such near-miss content, including earlier versions of the same file and logs from previous test runs.

The practical conclusion is that a context window size is a capacity limit, not a quality guarantee. Models that accept a million tokens still benefit from receiving only the tokens a step needs. The Lost in the Middle results also argue for placing critical information at the start or end of the input, which the recitation technique described below applies to an agent's goals.

## How to manage the context window: core techniques

Context engineering techniques fall into five groups: select only what the current step needs, compress what has to stay, persist information outside the window, isolate subtasks in separate contexts, and order the window so caches stay valid. The groups are complementary, because each one addresses a different failure.

### Select: retrieve just in time

Instead of loading a codebase or document set up front, the agent keeps lightweight references such as file paths, search queries, or URLs and fetches content with tools when a step needs it. [Retrieval-augmented generation](https://theaiengineering.com/articles/what-is-rag) applies the same idea to knowledge bases, and reranking keeps only the strongest passages. Tools should return the smallest useful result: paginated lists, filtered fields, and truncated logs with a pointer to the full output.

### Compress: compaction and tool result clearing

Compaction summarizes the conversation as it nears the window limit and continues from the summary. Tool result clearing is a lighter form that removes old raw tool outputs while keeping the record that the call happened. Both are now provider features: Anthropic's context editing clears the oldest tool results on the server once a threshold is passed, and OpenAI's Responses API offers server-side compaction and a standalone compaction endpoint. Compaction is lossy, so keep it restorable: drop a page's content but keep its URL, and drop a file's content but keep its path.

### Persist: write notes outside the window

Structured note-taking lets an agent record decisions, progress, and open questions in a file or memory store, then reload them after compaction or in a new session. The Manus team describes having its agent rewrite a `todo.md` file as it works, which keeps the current objectives near the end of the context.

### Isolate: use sub-agents with clean context

A [sub-agent](https://theaiengineering.com/glossary/subagent) receives a narrow task in a fresh context window, does the exploration, and returns a condensed result to the lead agent. The lead agent's context then holds a short answer instead of every file the sub-agent read. The cost is coordination overhead and the risk that the summary drops a detail the lead agent needed.

### Order: keep the prefix stable

Prompt caches and KV caches reuse computation only for an identical prefix. Keep the system prompt and tool definitions first and unchanged, append new content instead of editing earlier turns, serialize data deterministically, and avoid placing changing values at the top; the Manus team names a timestamp at the start of the system prompt as a common mistake. The Manus team also recommends masking unavailable tools rather than removing their definitions mid-task, and leaving failed actions and error messages in the context so the model's next step accounts for them.

## Context engineering example: a coding agent fixing a bug

A coding agent fixing a failing test shows the techniques working together. The agent starts from a short, stable prefix, searches for what it needs instead of loading the repository, trims and clears test output, writes progress notes before compaction, and delegates broad searches to a sub-agent. The sequence below follows one run.

1. **Stable prefix.** The run starts with the system prompt, the project's [AGENTS.md](https://theaiengineering.com/glossary/agents-md) file, and tool definitions, in the same order as every other run.
2. **Just-in-time retrieval.** The agent runs the failing test, searches for the failing function name, and reads the two files that define and call that function.
3. **Trimmed results.** The test tool returns the last 50 lines of output and writes the full log to a file the agent can open if needed.
4. **Clearing.** After the second test run, earlier test output is replaced by a one-line record of the result.
5. **Notes.** The agent updates a progress file with the hypothesis it tested and what failed.
6. **Isolation.** A sub-agent searches for every caller of the changed function and returns a list of file paths.
7. **Compaction.** Near the window limit, the history is summarized; the progress file and file paths are reloaded so the plan survives.

The [AI coding agents comparison](https://theaiengineering.com/articles/ai-coding-agents) covers the tools that run loops like this one, and where each one executes.

## How to measure context engineering

Measure context engineering the same way as any other model change: run a [fixed evaluation set](https://theaiengineering.com/glossary/golden-dataset) before and after, and compare task success alongside cost and latency. Many context changes trade quality against tokens, so a change that halves input tokens but lowers success on long tasks is a regression, not an optimization.

Useful metrics include:

- **Task success rate**, split by task length, because context problems show up on long runs first.
- **Input tokens per task and per step**, including tool definitions and re-sent history.
- **Cache hit rate**, which the Manus team calls the most important metric for a production agent because it drives both latency and cost.
- **Compaction outcomes**: how often compaction runs and how often a task fails afterward because the summary dropped something.
- **Retrieval recall**: whether the retrieved context contained the fact the step needed.
- **Ablations**: remove one context source at a time and measure the change in success.

Traces make these metrics possible. The [LLM observability guide](https://theaiengineering.com/articles/llm-observability) covers how to capture each model call with its full input, and the [LLM evaluation guide](https://theaiengineering.com/articles/llm-evaluation) covers building the fixed task set.

## Where to go next

Context engineering connects to two adjacent skills: [tool calling](https://theaiengineering.com/articles/tool-calling), which determines how much each tool adds to the window, and [retrieval-augmented generation](https://theaiengineering.com/glossary/retrieval-augmented-generation), which determines what external knowledge enters it. For more on prompts, context, and tool use, start from the [prompt engineering topic hub](https://theaiengineering.com/topics/prompt-engineering).

## Frequently asked questions

### What is the difference between context engineering and prompt engineering?

Prompt engineering is the practice of writing clear instructions and examples for a model. Context engineering is the broader practice of deciding everything the model receives on each call, including those instructions, tool definitions, retrieved documents, tool results, memory, and history. Prompt engineering is usually done at design time, while context engineering runs at every step of an agent loop.

### Is context engineering the same as RAG?

No. Retrieval-augmented generation is one context engineering technique: it selects external documents and inserts them into the prompt. Context engineering also covers tool definitions, tool results, memory, conversation history, compaction, and ordering. An agent can use RAG for knowledge lookups and still need separate policies for trimming logs, summarizing history, and isolating subtasks.

### Does more context use more tokens?

Yes. Every token in the context window is processed on each model call and billed as input, including tool definitions and earlier turns that are re-sent. Prompt caching lowers the cost of a repeated prefix, but it does not reduce the window size. Larger inputs also add latency and, according to long-context research, often reduce accuracy.

### What is agent memory?

Agent memory is information an AI agent stores outside its context window and loads back when needed. Short-term memory is the current conversation and working state. Long-term memory includes notes files, databases, or provider memory tools that persist across sessions. Context engineering decides what gets written to memory, when it is retrieved, and how much of it enters the window.

### Is long context better than RAG?

Neither is better in general. Long context avoids retrieval errors when the full source fits and the task needs broad reading, but costs more per call and suffers from context rot. Retrieval keeps inputs short and focused but can miss relevant passages. Many production systems retrieve first, then use a long window for the retrieved set and the working history.

## Terms in this guide

- [Context engineering](https://theaiengineering.com/glossary/context-engineering): Context engineering is the practice of selecting, structuring, and pruning all the information placed in an LLM's context window at each step, including instructions, retrieved data, tool results, and memory.
- [Context window](https://theaiengineering.com/glossary/context-window): A context window is the maximum number of tokens a language model can take into account in a single request, including the system prompt, conversation history, tool results, and the generated output.
- [Prompt engineering](https://theaiengineering.com/glossary/prompt-engineering): Prompt engineering is the practice of designing the instructions, examples, and formatting given to a large language model so it produces accurate, consistent output for a task, without changing model weights.
- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [Agent memory](https://theaiengineering.com/glossary/agent-memory): Agent memory is the set of mechanisms that let an AI agent keep and retrieve information beyond a single model call, from the current context window to external stores that persist across sessions.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.
- [KV cache](https://theaiengineering.com/glossary/kv-cache): The KV cache is GPU memory that stores the attention key and value vectors already computed for previous tokens, so a transformer generating text computes them once instead of at every step.
- [Retrieval-augmented generation (RAG)](https://theaiengineering.com/glossary/retrieval-augmented-generation): Retrieval-augmented generation (RAG) is a technique that retrieves relevant passages from an external knowledge source at query time and adds them to a language model's prompt so the answer is grounded in that data.
- [Reranking](https://theaiengineering.com/glossary/reranking): Reranking is a second retrieval stage that rescores an initial list of candidate documents with a more accurate but slower model, usually a cross-encoder, and keeps only the most relevant ones.
- [MCP server](https://theaiengineering.com/glossary/mcp-server): An MCP server is a program that exposes tools, resources, and prompts to AI applications through the Model Context Protocol, running locally over stdio or remotely over Streamable HTTP.
- [Context rot](https://theaiengineering.com/glossary/context-rot): Context rot is the decline in a language model's accuracy and recall as the number of tokens in its input grows, even when the information a task needs is present in the context.
- [Context compaction](https://theaiengineering.com/glossary/context-compaction): Context compaction is a context management technique that replaces older conversation turns with a model-written summary as the context window nears its limit, so an agent can keep working.
- [Subagent](https://theaiengineering.com/glossary/subagent): A subagent is an AI agent that another agent starts for one subtask, running in a separate context window with its own instructions and tools, and returning only a condensed result.
- [System prompt](https://theaiengineering.com/glossary/system-prompt): A system prompt is the set of developer-written instructions sent to a language model ahead of the conversation, defining its role, rules, output format, and tool use for every turn that follows.
- [Few-shot prompting](https://theaiengineering.com/glossary/few-shot-prompting): Few-shot prompting is a technique that puts a few example inputs paired with desired outputs in a prompt, so a language model infers the task, format, and labels without changing its weights.
- [AGENTS.md](https://theaiengineering.com/glossary/agents-md): AGENTS.md is an open format for a plain Markdown file, placed at a repository's root and optionally in subdirectories, that gives AI coding agents project instructions such as build commands, test commands, and conventions.
- [Golden dataset](https://theaiengineering.com/glossary/golden-dataset): A golden dataset is a curated, versioned set of representative inputs, each paired with an expected output or grading criteria, that a team treats as ground truth when evaluating an LLM application.
- [Recall@k](https://theaiengineering.com/glossary/recall-at-k): Recall@k is a retrieval metric that measures the fraction of a query's relevant documents that appear in the top k results, averaged over a set of test queries.

## Related guides

- [What is tool calling? How LLM function calling works](https://theaiengineering.com/articles/tool-calling): Tool calling lets an LLM request actions from your code through structured calls. Learn the request loop, parallel calls, structured outputs, security, and MCP.
- [What is retrieval-augmented generation (RAG)? How RAG works](https://theaiengineering.com/articles/what-is-rag): Retrieval-augmented generation (RAG) grounds LLM answers in your documents. Learn the RAG pipeline, hybrid search, agentic RAG, evaluation, and failure modes.
- [AI coding agents: how they work and how the major tools compare](https://theaiengineering.com/articles/ai-coding-agents): AI coding agents compared: how they differ from autocomplete, the four main categories, a comparison table of major tools, and how to evaluate and secure them.
- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.

Part of the [Prompt and context engineering](https://theaiengineering.com/topics/prompt-engineering) topic hub.

## Cite this guide

The AI Engineering Editors. "What is context engineering? A practical guide for AI agents." The AI Engineering, updated Nov 11, 2026. https://theaiengineering.com/articles/context-engineering

- Title: What is context engineering? A practical guide for AI agents
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-10-16
- Updated: 2026-11-11
- URL: https://theaiengineering.com/articles/context-engineering
- Markdown: https://theaiengineering.com/articles/context-engineering.md

To cite one section, link to its anchor:

- [What is context engineering?](https://theaiengineering.com/articles/context-engineering#what-is-context-engineering)
- [Context engineering vs prompt engineering](https://theaiengineering.com/articles/context-engineering#context-engineering-vs-prompt-engineering)
- [Why context engineering is important for AI agents](https://theaiengineering.com/articles/context-engineering#why-context-engineering-is-important-for-ai-agents)
- [What goes into the context window?](https://theaiengineering.com/articles/context-engineering#what-goes-into-the-context-window)
- [What is context rot?](https://theaiengineering.com/articles/context-engineering#what-is-context-rot)
- [How to manage the context window: core techniques](https://theaiengineering.com/articles/context-engineering#how-to-manage-the-context-window-core-techniques)
- [Context engineering example: a coding agent fixing a bug](https://theaiengineering.com/articles/context-engineering#context-engineering-example-a-coding-agent-fixing-a-bug)
- [How to measure context engineering](https://theaiengineering.com/articles/context-engineering#how-to-measure-context-engineering)
- [Where to go next](https://theaiengineering.com/articles/context-engineering#where-to-go-next)
- [Frequently asked questions](https://theaiengineering.com/articles/context-engineering#faq)
