---
title: "What is PagedAttention? How vLLM manages KV cache"
description: "PagedAttention stores the LLM KV cache in fixed-size blocks mapped by a block table, cutting memory waste and letting requests share prefixes. How it works."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/paged-attention"
markdown_url: "https://theaiengineering.com/glossary/paged-attention.md"
topic: "LLM inference and serving"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["PagedAttention","paged attention","paged KV cache"]
citation: "The AI Engineering Editors. \"What is PagedAttention? How vLLM manages KV cache.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/paged-attention"
---

# What is PagedAttention? How vLLM manages KV cache

Also called paged attention, paged KV cache.

> **Definition of PagedAttention:** PagedAttention is an attention algorithm, introduced with vLLM, that stores each sequence's KV cache in fixed-size blocks placed anywhere in GPU memory and tracked by a block table, an idea taken from operating-system paging.

PagedAttention is the KV cache memory scheme that [Kwon et al.](https://arxiv.org/abs/2309.06180) introduced with the vLLM serving engine at SOSP 2023, describing it as an attention algorithm inspired by virtual memory and paging in operating systems. It splits each request's [KV cache](https://theaiengineering.com/glossary/kv-cache) into fixed-size blocks, allocates a new block only when the previous one is full, and lets the attention kernel read keys and values from blocks scattered across GPU memory. Less wasted memory means more sequences fit in each batch. The authors report 2 to 4 times higher throughput than FasterTransformer and Orca at the same latency.

## How PagedAttention works

PagedAttention treats the KV cache like virtual memory. Each sequence sees logical blocks filled left to right, and a block table maps every logical block to a physical block somewhere in GPU memory and records how many slots are filled. The attention kernel follows the table to fetch each block, so blocks need not be contiguous.

The paper maps the analogy directly, treating blocks as pages, tokens as bytes, and requests as processes. Each block holds the keys and values for a fixed number of tokens. vLLM's default block size in the paper is 16 tokens, chosen to be large enough for efficient GPU kernels and small enough to keep internal fragmentation low. A request's only unused memory is the unfilled tail of its last block. A 1,000-token sequence, for example, occupies 63 blocks of 16 slots and leaves 8 slots empty.

## PagedAttention vs contiguous allocation

Serving systems before PagedAttention reserved one contiguous memory region per request, sized for the maximum possible output. That design wastes memory three ways: slots reserved for tokens the request has not generated yet, internal fragmentation when the output ends early, and external fragmentation in the gaps between regions of different sizes.

The vLLM paper measured that only 20.4% to 38.2% of KV cache memory held actual token states in the existing systems it profiled. Paging removes most of that waste:

| | Contiguous allocation | PagedAttention |
|---|---|---|
| Allocation unit | One region per request, sized for the maximum length | One fixed-size block, allocated on demand |
| Reserved but unused memory | Up to the full maximum output length | At most the unfilled part of the last block |
| External fragmentation | Gaps between regions of different sizes | None, because all blocks are the same size |
| Sharing between requests | Each request holds its own copy | Several block tables can point at one physical block |

When a request finishes, its blocks return to a shared pool, so [continuous batching](https://theaiengineering.com/glossary/continuous-batching) can admit a queued request as soon as enough blocks are free.

## Prefix sharing and copy-on-write

Because a block table is only a mapping, two sequences can point at the same physical block. PagedAttention uses this to share one copy of a prompt's KV cache across parallel samples, beam search candidates, and requests with the same system prompt, and copies a shared block only when one sequence must write to it.

Reference counts track how many sequences use each block, and the paper implements copy-on-write at block granularity, so diverging samples copy one block instead of the whole cache. It also proposes reserving blocks for predefined shared prefixes so that a request's prefill computes only the user's own input. Current vLLM generalizes this as automatic prefix caching: its [prefix caching design](https://docs.vllm.ai/en/latest/design/prefix_caching/) hashes each full block together with its parent block's hash, reuses matching blocks across requests, and evicts unused blocks in least-recently-used order. Hosted APIs sell the same effect as [prompt caching](https://theaiengineering.com/glossary/prompt-caching). vLLM's kernels have been rewritten since the paper, but block-based KV cache management remains, and other engines such as TensorRT-LLM also document a paged KV cache.

The [LLM inference optimization guide](https://theaiengineering.com/articles/llm-inference-optimization) shows how PagedAttention fits with batching, prefix caching, and quantization.

## Related terms

- [KV cache](https://theaiengineering.com/glossary/kv-cache): The KV cache is GPU memory that stores the attention key and value vectors already computed for previous tokens, so a transformer generating text computes them once instead of at every step.
- [Continuous batching](https://theaiengineering.com/glossary/continuous-batching): Continuous batching is an LLM serving technique that rebuilds the batch at every decode iteration, so finished requests leave at once and queued requests take their place at the next step.
- [Prompt caching](https://theaiengineering.com/glossary/prompt-caching): Prompt caching is an LLM API and inference-server feature that stores the computed attention state for a repeated prompt prefix, so later requests sharing that prefix skip reprocessing it.
- [Inference engine](https://theaiengineering.com/glossary/inference-engine): An inference engine is the software that serves a trained language model: it loads the weights, batches incoming requests, manages KV cache memory, runs the forward passes, and streams generated tokens over an API.
- [Prefill and decode](https://theaiengineering.com/glossary/prefill-and-decode): Prefill and decode are the two phases of LLM inference: prefill processes the whole prompt in one parallel pass and fills the KV cache, and decode generates one output token per forward pass.

## Guides that cover pagedattention

- [LLM inference optimization: a practical guide for engineers](https://theaiengineering.com/articles/llm-inference-optimization): LLM inference optimization guide: prefill vs decode, TTFT, KV cache and PagedAttention, continuous batching, quantization, speculative decoding, and GPUs.
- [vLLM vs Ollama vs llama.cpp vs SGLang: which engine to use](https://theaiengineering.com/articles/vllm-vs-ollama): vLLM vs Ollama compared with llama.cpp, SGLang, LM Studio, and TensorRT-LLM: scope, hardware, APIs, concurrency, and which engine fits local or production use.

Part of the [LLM inference and serving](https://theaiengineering.com/topics/llm-inference) topic hub.

## Cite this definition

The AI Engineering Editors. "What is PagedAttention? How vLLM manages KV cache." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/paged-attention

- Title: What is PagedAttention? How vLLM manages KV cache
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/paged-attention
- Markdown: https://theaiengineering.com/glossary/paged-attention.md
