What is agent orchestration? Patterns for AI agents
Also called multi-agent orchestration, orchestrator-worker pattern, orchestrator-workers
Definition
Agent orchestration is the control logic that decides which agent, model call, or tool runs next in an AI system and what each receives, whether written in code or chosen by a model.
Agent orchestration is the control logic that decides which agent, model call, or tool runs next in an AI system, and what each one receives as input. The logic can live in code, as a fixed pipeline or graph, or in a model, as an orchestrator agent that plans and delegates at run time. The orchestration model is the main practical difference between agent frameworks, and it determines how predictable, testable, and expensive a system is.
How agent orchestration works#
Agent orchestration works in one of two ways. In code-driven orchestration, the developer writes the sequence, branches, and parallel steps, and model output fills in the content. In model-driven orchestration, an orchestrator model reads the task and decides which worker to call next. The OpenAI Agents SDK documentation describes both approaches and notes that they can be mixed.
Anthropic's Building effective agents names five workflow patterns, and Microsoft Agent Framework adds multi-agent patterns of its own:
| Pattern | Who decides the next step | Example |
|---|---|---|
| Prompt chaining (sequential) | Code, in a fixed order | Outline, then draft, then edit |
| Routing | A classifier step, then code | Send a ticket to the billing or technical prompt |
| Parallelization (concurrent) | Code fans out and merges | Several reviewers score one answer |
| Orchestrator-workers | An orchestrator model | A lead agent splits research across subagents |
| Evaluator-optimizer | Code loops on a critic's verdict | Draft, critique, and revise until a check passes |
| Handoff | The active agent | Triage passes the user to a refund agent |
| Group chat and Magentic | A shared conversation or a manager agent | Specialists coordinated by a manager |
In the orchestrator-workers pattern, Anthropic describes a central LLM that "dynamically breaks down tasks, delegates them to worker LLMs, and synthesizes their results." It differs from parallelization because the orchestrator chooses the subtasks from the input instead of following a list fixed in code.
Orchestration vs choreography#
Orchestration routes work through a central coordinator, while choreography lets each agent decide where work goes next. A coordinator sees the whole task, which makes tracing, budgets, and retries easier to manage in one place. A choreographed design removes the coordinator, but it spreads the routing logic across every agent's instructions, so a single run is harder to follow.
Agent handoffs are a choreographed design. Microsoft Agent Framework implements handoff orchestration as a mesh in which agents connect directly without an orchestrator, while its Magentic pattern uses a manager agent to coordinate specialists. Across organizations, the A2A protocol lets an orchestrator agent delegate tasks to remote agents whose memory and tools stay hidden from it.
Orchestration vs a single agent loop#
A single agent loop has no orchestration layer: one model calls tools until it finishes. Add orchestration when a measured limit appears, such as a context window filled by exploration, independent subtasks that could run in parallel, or tools that need separate permissions. Anthropic recommends finding the simplest solution possible and adding complexity only when needed.
Each layer of orchestration adds model calls. Anthropic's multi-agent research system used about 15 times the tokens of a chat interaction, so a multi-agent design has to buy enough quality to cover that cost. Multi-agent systems also need evaluation of the whole trajectory, since a correct final answer can hide a wasteful path.
The AI agent frameworks comparison sorts current frameworks by orchestration model, from plain tool loops to graphs, handoffs, and role-based crews.
Related terms
Guides that cover agent orchestration
- AI agent frameworks compared: LangGraph, CrewAI, ADK, and more
- What are AI agents? How agents work and when to use one
- A2A protocol vs MCP: differences and when to use each
- How to build an AI agent: a step-by-step guide in Python
- AI agent evaluation: metrics, graders, and how to test agents
- AI engineer roadmap: skills, projects, and resources by stage
- Agentic workflows: five patterns and when to use each
Cite this definition
The AI Engineering Editors. "What is agent orchestration? Patterns for AI agents." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/agent-orchestration
- Title
- What is agent orchestration? Patterns for AI agents
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated