What is a transformer in AI? Architecture explained
Also called transformer architecture, transformer model, transformers
Definition
A transformer is a neural network architecture that processes sequences with self-attention, letting each token weigh information from other tokens in parallel; it is the basis of most modern large language models.
A transformer is a neural network architecture that processes a sequence of tokens with self-attention, a mechanism that lets each position compute a weighted mix of information from other positions in the sequence. Vaswani et al. introduced the transformer in the 2017 paper Attention Is All You Need, which built a translation model from attention alone and dropped recurrence and convolutions entirely. The transformer is the architecture behind most large language models in use today.
How a transformer works#
A transformer works by turning each token into an embedding vector, adding position information, and passing the vectors through a stack of identical layers. Each layer has two parts:
- Self-attention: each token produces a query, a key, and a value vector. The query is compared with every key to produce attention weights, and the output is the weighted sum of the values. Several attention heads run in parallel to capture different relationships.
- Feed-forward network: a small neural network applied to each position separately, transforming the attention output.
Residual connections and normalization wrap both parts so deep stacks train stably. The original model paired an encoder with a decoder for translation. Most LLMs today use only the decoder with causal masking, so each token attends only to earlier tokens, which is what makes left-to-right generation possible.
Why the transformer matters#
The transformer matters because self-attention processes every position of a sequence at once, which fits GPU hardware far better than recurrent networks that step through tokens one at a time. The original paper reported better translation quality than earlier models with much less training time, and that parallelism made training on web-scale text practical.
The same design creates the main inference costs. Attention compute grows quadratically with sequence length, and generation stores keys and values for every previous token in a KV cache. Techniques such as mixture of experts layers and optimized attention kernels target those costs; LLM inference optimization covers them in detail.
Related terms
Guides that cover transformer
Cite this definition
The AI Engineering Editors. "What is a transformer in AI? Architecture explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/transformer
- Title
- What is a transformer in AI? Architecture explained
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated