---
title: "What is mixture of experts (MoE) in LLMs?"
description: "Mixture of experts (MoE) is a model architecture that routes each token to a few expert subnetworks, so only part of the model runs per token. MoE vs dense."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/mixture-of-experts"
markdown_url: "https://theaiengineering.com/glossary/mixture-of-experts.md"
topic: "LLM models and benchmarks"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Mixture of experts","MoE","sparse mixture of experts","sparse MoE"]
citation: "The AI Engineering Editors. \"What is mixture of experts (MoE) in LLMs?\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/mixture-of-experts"
---

# What is mixture of experts (MoE) in LLMs?

Also called MoE, sparse mixture of experts, sparse MoE.

> **Definition of Mixture of experts:** Mixture of experts (MoE) is a neural network architecture in which a learned router sends each token to a few specialized subnetworks, called experts, so only part of the model runs per token.

A mixture of experts model replaces some dense layers with a set of parallel expert networks plus a router, also called a gating network, that selects which experts process each input. The idea dates to work by Jacobs, Jordan, Nowlan, and Hinton in 1991. Noam Shazeer and colleagues at Google brought it to modern deep learning with the [sparsely-gated mixture-of-experts layer](https://arxiv.org/abs/1701.06538) in 2017, reporting capacity gains of more than 1,000 times with only minor losses in computational efficiency. [Open-weight](https://theaiengineering.com/glossary/open-weights-model) [large language models](https://theaiengineering.com/glossary/large-language-model) including Mixtral 8x7B and DeepSeek-V3 use the architecture.

## How mixture of experts works

In an MoE [transformer](https://theaiengineering.com/glossary/transformer), the feed-forward block in each layer is replaced by several expert feed-forward networks. For every token, the router scores the experts, sends the token to the top-scoring few, and combines their outputs using the router's weights.

[Mixtral 8x7B](https://mistral.ai/news/mixtral-of-experts), released by Mistral AI in December 2023, is a well-documented example: 8 experts per layer, 2 selected per token, 46.7B total parameters, and 12.9B parameters used per token. [DeepSeek-V3](https://arxiv.org/abs/2412.19437) has 671B total parameters with 37B activated per token. MoE training also needs a load-balancing mechanism, often an auxiliary loss, so the router does not overload a few experts while others sit idle.

## Mixture of experts vs dense models

A dense model runs every parameter for every token. An MoE model's compute per token scales with its active parameters, while its capacity scales with its total parameters.

| | Dense model | MoE model |
|---|---|---|
| Parameters used per token | All | Active subset (Mixtral: 12.9B of 46.7B) |
| Compute per token | Scales with total size | Scales with active size |
| Memory to serve | All weights | All weights, since every expert must be loaded |
| Serving complexity | Lower | Higher: expert parallelism, uneven routing |

The memory row is the catch. An MoE model needs enough GPU memory for every expert even though each token uses only a few, which is why [quantization](https://theaiengineering.com/glossary/quantization) and multi-GPU serving matter for MoE deployment (see the [LLM inference optimization guide](https://theaiengineering.com/articles/llm-inference-optimization)). When comparing models on [LLM benchmarks](https://theaiengineering.com/articles/llm-benchmarks), check both total and active parameter counts.

## Related terms

- [Transformer](https://theaiengineering.com/glossary/transformer): A transformer is a neural network architecture that processes sequences with self-attention, letting each token weigh information from other tokens in parallel; it is the basis of most modern large language models.
- [Large language model (LLM)](https://theaiengineering.com/glossary/large-language-model): A large language model (LLM) is a neural network, usually a transformer with billions of parameters, trained on web-scale text to predict the next token and then adapted to follow instructions.
- [Quantization](https://theaiengineering.com/glossary/quantization): Quantization is a model compression technique that represents a neural network's weights, and sometimes its activations or KV cache, with fewer bits, such as 8-bit or 4-bit integers instead of 16-bit floats.
- [Small language model](https://theaiengineering.com/glossary/small-language-model): A small language model (SLM) is a language model compact enough to run on a consumer device such as a laptop or phone, which in 2025 generally meant fewer than about 10 billion parameters.
- [Tensor parallelism](https://theaiengineering.com/glossary/tensor-parallelism): Tensor parallelism is a model parallelism technique that splits the weight matrices inside each transformer layer across several GPUs, which compute every token together and combine partial results through all-reduce operations.

## Guides that cover mixture of experts

- [LLM inference optimization: a practical guide for engineers](https://theaiengineering.com/articles/llm-inference-optimization): LLM inference optimization guide: prefill vs decode, TTFT, KV cache and PagedAttention, continuous batching, quantization, speculative decoding, and GPUs.
- [LLM benchmarks explained: what they measure and how to read them](https://theaiengineering.com/articles/llm-benchmarks): LLM benchmarks explained: what MMLU-Pro, GPQA, SWE-bench, LiveCodeBench, ARC-AGI, and LMArena measure, why scores saturate, and how to read a leaderboard.
- [vLLM vs Ollama vs llama.cpp vs SGLang: which engine to use](https://theaiengineering.com/articles/vllm-vs-ollama): vLLM vs Ollama compared with llama.cpp, SGLang, LM Studio, and TensorRT-LLM: scope, hardware, APIs, concurrency, and which engine fits local or production use.

Part of the [LLM models and benchmarks](https://theaiengineering.com/topics/models) topic hub.

## Cite this definition

The AI Engineering Editors. "What is mixture of experts (MoE) in LLMs?" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/mixture-of-experts

- Title: What is mixture of experts (MoE) in LLMs?
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/mixture-of-experts
- Markdown: https://theaiengineering.com/glossary/mixture-of-experts.md
