---
title: "What is agent memory in AI? Types and architecture"
description: "Agent memory is how an AI agent stores and recalls information across steps and sessions. Learn the main memory types and how they relate to the context window."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/agent-memory"
markdown_url: "https://theaiengineering.com/glossary/agent-memory.md"
topic: "AI agents"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Agent memory","AI agent memory","LLM memory","long-term memory for agents"]
citation: "The AI Engineering Editors. \"What is agent memory in AI? Types and architecture.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agent-memory"
---

# What is agent memory in AI? Types and architecture

Also called AI agent memory, LLM memory, long-term memory for agents.

> **Definition of Agent memory:** Agent memory is the set of mechanisms that let an AI agent keep and retrieve information beyond a single model call, from the current context window to external stores that persist across sessions.

Agent memory is the set of mechanisms that let an [AI agent](https://theaiengineering.com/glossary/ai-agent) keep information beyond a single model call. A language model holds no state between API requests, so anything an agent "remembers" must be stored by the application and placed back into the prompt. Designing agent memory means choosing what to store, where to store it, and when to retrieve it.

## How agent memory works

Agent memory works in two tiers. Short-term memory is whatever sits in the model's [context window](https://theaiengineering.com/glossary/context-window) for the current task. Long-term memory lives in external storage that the agent reads and writes through tools or retrieval. The [CoALA framework](https://arxiv.org/abs/2309.02427) (Sumers et al., 2023) pairs working memory with three long-term types.

| Memory type | What it holds | Typical storage |
|---|---|---|
| Working (short-term) | Current goal, recent messages, tool results | The context window |
| Episodic | Records of past runs and interactions | Logs, databases, vector indexes |
| Semantic | Facts about the user, the domain, or the world | Documents, key-value stores, [vector databases](https://theaiengineering.com/glossary/vector-database) |
| Procedural | How to perform tasks | Model weights, prompts, and agent code |

Retrieval usually relies on [embeddings](https://theaiengineering.com/glossary/embeddings) for similarity search, combined with filters on time, user, or topic. Writing is harder than reading. The application needs rules for which facts to keep, how to resolve a new fact that contradicts a stored one, and when to expire stale entries. Without those rules, memory stores fill with duplicates and outdated claims that degrade later answers.

## Agent memory vs context window

Agent memory and the context window are different layers. The context window is the fixed-size input a model reads on each call; agent memory is the storage and retrieval logic that determines what enters that window. A longer context window reduces the need for external memory within one session, but it does not carry state across sessions.

Filling the window is not free either. Anthropic's [context window documentation](https://platform.claude.com/docs/en/build-with-claude/context-windows) notes that accuracy and recall degrade as token count grows, which it calls [context rot](https://theaiengineering.com/glossary/context-rot). [MemGPT](https://arxiv.org/abs/2310.08560) (Packer et al., 2023) framed the problem like an operating system, paging data between a small fast tier (the context) and larger slow tiers (external storage).

Memory is one part of [context engineering](https://theaiengineering.com/glossary/context-engineering). The [context engineering guide](https://theaiengineering.com/articles/context-engineering) covers how to budget tokens between memory, instructions, and tool results, and the [AI agents hub](https://theaiengineering.com/topics/ai-agents) links related material.

## Related terms

- [AI agent](https://theaiengineering.com/glossary/ai-agent): An AI agent is a software system in which a large language model runs in a loop, calling tools, reading their results, and repeating until the task is complete or a stop condition is reached.
- [Context window](https://theaiengineering.com/glossary/context-window): A context window is the maximum number of tokens a language model can take into account in a single request, including the system prompt, conversation history, tool results, and the generated output.
- [Vector database](https://theaiengineering.com/glossary/vector-database): A vector database is a database built to store embedding vectors and quickly return the vectors most similar to a query vector, usually with approximate nearest neighbor indexes such as HNSW.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [Context engineering](https://theaiengineering.com/glossary/context-engineering): Context engineering is the practice of selecting, structuring, and pruning all the information placed in an LLM's context window at each step, including instructions, retrieved data, tool results, and memory.
- [Context compaction](https://theaiengineering.com/glossary/context-compaction): Context compaction is a context management technique that replaces older conversation turns with a model-written summary as the context window nears its limit, so an agent can keep working.

## Guides that cover agent memory

- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.
- [What is context engineering? A practical guide for AI agents](https://theaiengineering.com/articles/context-engineering): Context engineering is the practice of choosing what fills an LLM context window. Learn its components, why context rot happens, and how to measure results.
- [How to build an AI agent: a step-by-step guide in Python](https://theaiengineering.com/articles/how-to-build-an-ai-agent): Learn how to build an AI agent from scratch in Python: when you need one, tool schemas, the agent loop, stop conditions, guardrails, evals, and tracing.
- [AI agent frameworks compared: LangGraph, CrewAI, ADK, and more](https://theaiengineering.com/articles/ai-agent-frameworks): An AI agent frameworks comparison of LangGraph, LangChain, CrewAI, the OpenAI, Claude, and Google SDKs, Pydantic AI, Mastra, and more, by language and license.

Part of the [AI agents](https://theaiengineering.com/topics/ai-agents) topic hub.

## Cite this definition

The AI Engineering Editors. "What is agent memory in AI? Types and architecture." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agent-memory

- Title: What is agent memory in AI? Types and architecture
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/agent-memory
- Markdown: https://theaiengineering.com/glossary/agent-memory.md
