---
title: "What is a transformer in AI? Architecture explained"
description: "A transformer is the neural network architecture behind most LLMs, built on self-attention instead of recurrence. Learn how it works and why it scales so well."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/transformer"
markdown_url: "https://theaiengineering.com/glossary/transformer.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Transformer","transformer architecture","transformer model","transformers"]
citation: "The AI Engineering Editors. \"What is a transformer in AI? Architecture explained.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/transformer"
---

# What is a transformer in AI? Architecture explained

Also called transformer architecture, transformer model, transformers.

> **Definition of Transformer:** A transformer is a neural network architecture that processes sequences with self-attention, letting each token weigh information from other tokens in parallel; it is the basis of most modern large language models.

A transformer is a neural network architecture that processes a sequence of [tokens](https://theaiengineering.com/glossary/tokens) with self-attention, a mechanism that lets each position compute a weighted mix of information from other positions in the sequence. Vaswani et al. introduced the transformer in the 2017 paper [Attention Is All You Need](https://arxiv.org/abs/1706.03762), which built a translation model from attention alone and dropped recurrence and convolutions entirely. The transformer is the architecture behind most [large language models](https://theaiengineering.com/glossary/large-language-model) in use today.

## How a transformer works

A transformer works by turning each token into an [embedding](https://theaiengineering.com/glossary/embeddings) vector, adding position information, and passing the vectors through a stack of identical layers. Each layer has two parts:

- **Self-attention**: each token produces a query, a key, and a value vector. The query is compared with every key to produce attention weights, and the output is the weighted sum of the values. Several attention heads run in parallel to capture different relationships.
- **Feed-forward network**: a small neural network applied to each position separately, transforming the attention output.

**Figure 1: A decoder-only transformer, simplified.**

Tokens become embeddings with position information, pass through N stacked layers of self-attention and feed-forward networks, and come out as next-token probabilities.

Components:

- **Tokens**: Input sequence
- **Embeddings**: Plus position information
- **Self-attention**: Weighted mix across positions
- **Feed-forward**: Applied to each position
- **Next-token probabilities**

Groups:

- × N layers: Self-attention, Feed-forward

Connections:

- Tokens → Embeddings
- Embeddings → Self-attention
- Self-attention → Feed-forward
- Feed-forward → Next-token probabilities

Residual connections and normalization wrap both parts so deep stacks train stably. The original model paired an encoder with a decoder for translation. Most LLMs today use only the decoder with causal masking, so each token attends only to earlier tokens, which is what makes left-to-right generation possible.

## Why the transformer matters

The transformer matters because self-attention processes every position of a sequence at once, which fits GPU hardware far better than recurrent networks that step through tokens one at a time. The original paper reported better translation quality than earlier models with much less training time, and that parallelism made training on web-scale text practical.

The same design creates the main inference costs. Attention compute grows quadratically with sequence length, and generation stores keys and values for every previous token in a [KV cache](https://theaiengineering.com/glossary/kv-cache). Techniques such as [mixture of experts](https://theaiengineering.com/glossary/mixture-of-experts) layers and optimized attention kernels target those costs; [LLM inference optimization](https://theaiengineering.com/articles/llm-inference-optimization) covers them in detail.

## Related terms

- [Large language model (LLM)](https://theaiengineering.com/glossary/large-language-model): A large language model (LLM) is a neural network, usually a transformer with billions of parameters, trained on web-scale text to predict the next token and then adapted to follow instructions.
- [Tokens](https://theaiengineering.com/glossary/tokens): Tokens are the units of text a language model processes, typically whole words, word fragments, punctuation, or bytes, each mapped to an integer ID from the model's fixed vocabulary.
- [Embeddings](https://theaiengineering.com/glossary/embeddings): Embeddings are fixed-length vectors of numbers that represent data such as text or images so that items with similar meaning sit close together, which lets software compare meaning with arithmetic.
- [KV cache](https://theaiengineering.com/glossary/kv-cache): The KV cache is GPU memory that stores the attention key and value vectors already computed for previous tokens, so a transformer generating text computes them once instead of at every step.
- [Mixture of experts](https://theaiengineering.com/glossary/mixture-of-experts): Mixture of experts (MoE) is a neural network architecture in which a learned router sends each token to a few specialized subnetworks, called experts, so only part of the model runs per token.
- [Pretraining](https://theaiengineering.com/glossary/pretraining): Pretraining is the first stage of training a language model, in which it learns to predict the next token across a large unlabeled corpus of text and code, producing a general-purpose base model.
- [Foundation model](https://theaiengineering.com/glossary/foundation-model): A foundation model is a large AI model trained on broad data, usually with self-supervised learning at scale, that can be adapted through prompting or fine-tuning to a wide range of downstream tasks.

## Guides that cover transformer

- [LLM inference optimization: a practical guide for engineers](https://theaiengineering.com/articles/llm-inference-optimization): LLM inference optimization guide: prefill vs decode, TTFT, KV cache and PagedAttention, continuous batching, quantization, speculative decoding, and GPUs.
- [Reasoning models: how they work and when to use them](https://theaiengineering.com/articles/reasoning-models): Reasoning models explained: how test-time compute and reinforcement learning produce chain-of-thought reasoning, when to use them, and their cost and latency.

Part of the [LLM models and benchmarks](https://theaiengineering.com/topics/models) topic hub.

## Cite this definition

The AI Engineering Editors. "What is a transformer in AI? Architecture explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/transformer

- Title: What is a transformer in AI? Architecture explained
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/transformer
- Markdown: https://theaiengineering.com/glossary/transformer.md
