---
title: "Agentic workflows: five patterns and when to use each"
description: "Agentic workflows run LLM and tool calls through steps fixed in code. Learn five patterns, their cost and latency, failure modes, and when to use an agent."
summary: "An agentic workflow is an LLM system whose steps are fixed in code, unlike an autonomous agent that picks its own next step. This guide covers prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer, with code sketches, a decision table, and human checkpoints."
type: "guide"
canonical_url: "https://theaiengineering.com/articles/agentic-workflows"
markdown_url: "https://theaiengineering.com/articles/agentic-workflows.md"
topic: "AI agents"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-11-16"
updated: "2026-11-16"
keywords: ["agentic workflows","agentic workflow","agentic workflow patterns","what is an agentic workflow","agentic workflow vs agent","agentic workflows examples"]
citation: "The AI Engineering Editors. \"Agentic workflows: five patterns and when to use each.\" The AI Engineering, updated Nov 16, 2026. https://theaiengineering.com/articles/agentic-workflows"
---

# Agentic workflows: five patterns and when to use each

## TL;DR

- An agentic workflow is an LLM system in which code fixes the order of model and tool calls, while an AI agent lets model output choose each next step at runtime.
- The five standard agentic workflow patterns are prompt chaining, routing, parallelization (sectioning and voting), orchestrator-workers, and evaluator-optimizer.
- Chains and evaluator loops add latency with every call, parallelization cuts latency but multiplies calls, and orchestrator-workers makes cost and latency depend on a plan the model writes.
- Move a step from a workflow to an agent only when the number and order of actions depend on what the system finds, and keep human checkpoints before irreversible actions.
- Build workflows in plain code first, and adopt a framework or durable execution engine when runs must survive crashes, pause for approval, or retry steps without repeating side effects.

Agentic workflows are LLM systems in which models and tools run through steps defined in code: the program fixes the order of calls, and each model call does one bounded job, such as classifying, drafting, or grading. That places them between a single model call and an autonomous [AI agent](https://theaiengineering.com/glossary/ai-agent), which lets model output choose its own next step. This guide covers the five standard patterns (prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer) with a code sketch, cost and latency profile, and failure modes for each, then when to graduate to an agent, where human checkpoints belong, and how to build, run, and evaluate workflows. It builds on our explainer on [what AI agents are](https://theaiengineering.com/articles/what-are-ai-agents) and is part of the [AI agents topic hub](https://theaiengineering.com/topics/ai-agents).

## What is an agentic workflow?

An agentic workflow is an LLM system whose control flow is fixed in code, with model and tool calls running at steps the developer defined in advance.

Anthropic's engineering post [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents) (December 2024) draws the line this guide follows. It defines workflows as "systems where LLMs and tools are orchestrated through predefined code paths," and agents as systems in which the model directs its own process and [tool calls](https://theaiengineering.com/glossary/tool-calling). Anthropic calls both kinds agentic systems, so a workflow counts as [agentic AI](https://theaiengineering.com/glossary/agentic-ai) even though its steps are fixed.

A workflow can still contain model decisions. A router's classification picks a branch, and an evaluator's grade decides whether to loop again. What stays fixed is the set of possible branches, the loop bounds, and the tools each step can reach. In an agent, model output picks the next action from everything it has been given, and the number of steps is unknown until the run ends.

Observability standards now encode the same split. The OpenTelemetry GenAI [agent span conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md), still in Development status, tell instrumentation to report `invoke_workflow` when the application defines the control flow, with LangGraph's `StateGraph` as an example, and `invoke_agent` when the model drives control flow through tool calls.

Every pattern below is built from one unit, which Anthropic calls the augmented LLM: a model call with access to retrieval, tools, and memory. The patterns differ only in how code wires those calls together.

## The five agentic workflow patterns at a glance

The five agentic workflow patterns are prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer. They differ in shape: a line of steps, a branch, a fan-out, a fan-out planned by a model, and a loop. Anthropic's post names all five, and LangGraph's documentation and the OpenAI Agents SDK's examples implement each one.

**Figure 1: The five agentic workflow patterns. Parallelization and orchestrator-workers share a shape; what differs is whether code or a model decides the subtasks.**

Five panels: a chain of steps with a gate, a router choosing between two routes, a code-defined fan-out, a model-planned fan-out to workers, and a generator-evaluator loop.

Components:

- **Step 1**: model call
- **Gate**: check in code
- **Step 2**: model call
- **Step 3**: model call
- **Router**: classify input
- **Route A**: own prompt
- **Route B**: other model
- **Generator**: drafts
- **Evaluator**: grades draft
- **Split in code**: sections or copies
- **Call A**: model call
- **Call B**: model call
- **Aggregate**: merge or vote
- **Orchestrator**: model plans subtasks
- **Worker**: subtask 1
- **Worker**: subtask n
- **Synthesizer**: model combines

Groups:

- Prompt chaining: Step 1, Gate, Step 2, Step 3
- Routing: Router, Route A, Route B
- Evaluator-optimizer: Generator, Evaluator
- Parallelization: Split in code, Call A, Call B, Aggregate
- Orchestrator-workers: Orchestrator, Worker, Worker, Synthesizer

Connections:

- Step 1 → Gate
- Gate → Step 2: pass
- Step 2 → Step 3
- Router → Route A
- Router → Route B
- Generator → Evaluator: draft
- Evaluator → Generator: feedback
- Split in code → Call A
- Split in code → Call B
- Call A → Aggregate
- Call B → Aggregate
- Orchestrator → Worker
- Orchestrator → Worker
- Worker → Synthesizer
- Worker → Synthesizer

The sketches in this guide share two helpers. `llm()` wraps whichever provider SDK you use and returns the reply text; `allm()` runs it in a thread so calls can overlap. Model names are aliases you map to real model IDs.

```python
import asyncio
import json

def llm(prompt: str, *, system: str = "", model: str = "small",
        temperature: float | None = None) -> str:
    """Send one prompt through your provider's SDK; return the reply text."""
    raise NotImplementedError("wrap your provider SDK here")

async def allm(prompt: str, **kwargs) -> str:
    """Run llm() in a worker thread so several calls can run at once."""
    return await asyncio.to_thread(llm, prompt, **kwargs)
```

## Prompt chaining with gates

Prompt chaining splits a task into a fixed sequence of model calls, each working on the previous call's output, with programmatic checks called gates between steps. It fits tasks that decompose cleanly into known subtasks, and it trades latency for accuracy: each call does an easier job, and total latency is the sum of the steps.

**When it fits.** Use a chain when you could write the task as a checklist that never changes order. Anthropic's examples are writing marketing copy and then translating it, and writing an outline, checking it against criteria, and then drafting the document. Each step gets its own prompt, its own model size, and its own test cases.

**Gates.** A gate is a check between steps, written in code wherever possible: parse the output, validate it against a schema with [structured outputs](https://theaiengineering.com/glossary/structured-outputs), count items, confirm that cited IDs exist, or run a linter. On failure, the workflow stops, retries the step, or escalates. Gates make chains debuggable, because each failure is pinned to one step.

**Cost and latency.** Cost is the sum of the calls, and latency is the sum of their durations, since each call waits for the last. Short, focused steps can often run on smaller models than a single large prompt would need.

**Failure modes.** An error in an early step flows downstream unless a gate catches it. Steps that see only the previous output lose details from the original input, so pass the source along when later steps need it. Splitting too finely adds latency without adding accuracy, and gates that check only format let wrong content through.

**Example: release notes from a diff.**

```python
def release_notes(diff: str) -> str:
    changes = llm("List each user-visible change in this diff, one per "
                  f"line. Write NONE if there are none.\n\n{diff}")
    items = [line.strip() for line in changes.splitlines() if line.strip()]

    # Gate: checks in code between steps. Stop early instead of drafting.
    if items == ["NONE"]:
        return ""
    if len(items) > 40:
        raise ValueError(f"{len(items)} changes; split this release first")

    draft = llm("Write release notes from these changes. Do not add "
                "changes that are not listed.\n\n" + "\n".join(items))
    return llm(f"Edit for clarity. Keep every fact.\n\n{draft}",
               system="You edit release notes for developers.")
```

## Routing

Routing classifies an input and sends it to one of several specialized paths, each with its own prompt, tools, or model. It fits inputs that fall into distinct categories that are better handled separately, and it works only as well as its classifier: a misrouted input gets a confident answer from the wrong specialist.

**When it fits.** Anthropic's examples are sending customer service queries (general questions, refund requests, technical support) to different downstream processes, and sending easy questions to a small model and hard ones to a larger one. The classifier can be a model call or a traditional classifier trained on your own labels.

**Cost and latency.** Routing adds one classification call, usually on a small, fast model, before one path runs. It can lower total cost when most traffic lands on cheaper paths. Our guide to [LLM routing](https://theaiengineering.com/articles/llm-routing) covers cost-aware and learned routers at the gateway level.

**Failure modes.** The main failure is misclassification, especially between overlapping categories. New kinds of input appear over time and get forced into existing labels. Free-text labels sometimes fail to parse. Constrain the label to an enum, log every routing decision, and send unknown or low-confidence inputs to a default path, such as a person or a general handler, rather than a guess.

**Example: support ticket triage.** `escalate_to_human` stands in for your ticket queue.

```python
ROUTES = {
    "billing": lambda t: llm(t, system="You answer billing questions."),
    "technical": lambda t: llm(t, system="You debug customer setups.",
                               model="large"),
    "account": lambda t: llm(t, system="You handle account changes."),
}

def handle_ticket(ticket: str) -> str:
    label = llm("Classify this support ticket as billing, technical, or "
                "account. Reply with the label only.\n\n" + ticket,
                model="small", temperature=0).strip().lower()
    handler = ROUTES.get(label)
    if handler is None:  # unknown label: a person decides
        return escalate_to_human(ticket, reason=f"unrouted: {label!r}")
    return handler(ticket)
```

## Parallelization: sectioning and voting

Parallelization runs several model calls at the same time and combines their outputs in code. Sectioning splits a task into independent subtasks, such as checking a change for security, test coverage, and documentation at once. Voting runs the same task several times and aggregates the answers. Both cut latency compared with a chain, but both multiply the number of calls.

**When it fits.** Sectioning fits subtasks that do not depend on each other. Anthropic's examples include one model screening queries for inappropriate content while another answers them, and evals in which each call grades a different aspect. Voting fits tasks where several attempts raise confidence, such as several prompts reviewing code for vulnerabilities, with a vote threshold that balances false positives against false negatives.

Research supports voting on reasoning tasks. The self-consistency method of [Wang et al. (2022)](https://arxiv.org/abs/2203.11171) samples several reasoning paths and picks the most consistent answer, and the paper reports a 17.9% gain on the GSM8K math benchmark over a single greedy answer.

**Cost and latency.** Latency is the slowest branch plus the aggregation step. Cost is the sum of every branch, so five votes cost five times one call. Concurrent calls also hit provider rate limits sooner, which can turn a fan-out back into a queue.

**Failure modes.** Sections that secretly depend on each other produce contradictions the aggregator cannot resolve. Votes from the same model and prompt share the same blind spots, so vary prompts or models when errors are correlated. Decide in advance how to break ties and what to do when one branch fails: fail the run or return partial results.

**Example: pull request review by section, and a policy check by vote.**

```python
async def review_pull_request(diff: str) -> dict[str, str]:
    # Sectioning: independent checks, one narrow prompt each.
    aspects = {
        "security": "List security problems in this diff.",
        "tests": "List changed behavior that no test covers.",
        "docs": "List public API changes missing documentation.",
    }
    results = await asyncio.gather(
        *(allm(f"{task}\n\n{diff}") for task in aspects.values()))
    return dict(zip(aspects, results))

async def violates_policy(text: str, votes: int = 5,
                          threshold: int = 3) -> bool:
    # Voting: the same question several times, sampled for variety.
    answers = await asyncio.gather(*(
        allm("Does this text violate the content policy? Answer yes or "
             f"no.\n\n{text}", temperature=1.0)
        for _ in range(votes)))
    yes = sum(a.strip().lower().startswith("yes") for a in answers)
    return yes >= threshold
```

## Orchestrator-workers

In the orchestrator-workers pattern, a model reads the input, decides which subtasks are needed, hands each to a worker call, and a final call synthesizes the results. It resembles parallelization, but the model chooses the subtasks at runtime instead of code listing them, which makes it the most flexible and least predictable of the five patterns.

**When it fits.** Use it when the subtasks cannot be predicted from the input type alone. Anthropic's examples are coding products that change several files per request, and search tasks that gather and analyze information from several sources. Anthropic's Research feature uses this pattern, with a lead agent that plans and parallel [subagents](https://theaiengineering.com/glossary/subagent) that search. When the workers are agents with their own tools and context, the result is a [multi-agent system](https://theaiengineering.com/glossary/multi-agent-system).

**Cost and latency.** Cost and latency depend on the plan: one planning call, then the slowest worker, then synthesis, with the number of workers chosen by the model. In its [multi-agent research system post](https://www.anthropic.com/engineering/multi-agent-research-system) (June 2025), Anthropic reported that its agents used about 4 times as many tokens as chat interactions and its multi-agent systems about 15 times as many. The same post notes that most coding tasks contain fewer truly parallelizable subtasks than research does.

**Failure modes.** Plans can overlap, leave gaps, or split work too finely, and without a cap, fan-out grows with the input. Workers see only their subtask, so each needs an explicit objective, output format, and boundaries. Synthesis can drop or invent details, and the run waits on its slowest worker, a bottleneck Anthropic describes in its own system.

What keeps this pattern a workflow is what stays in code around the plan: its schema, the worker cap, the worker prompt and tools, and the synthesis step. That code is the workflow's [orchestration](https://theaiengineering.com/glossary/agent-orchestration) logic, even though a model writes the plan.

**Example: research with a capped fan-out.** Real workers would also get search tools.

```python
async def research(question: str, max_workers: int = 5) -> str:
    plan = json.loads(llm(
        "Break this question into independent research subtasks. Return "
        'JSON only: {"subtasks": ["..."]}\n\n' + question, model="large"))
    subtasks = plan["subtasks"][:max_workers]  # the cap lives in code

    findings = await asyncio.gather(*(
        allm("Research this subtask. Cite a source for each claim.\n\n"
             f"Overall question: {question}\nSubtask: {s}")
        for s in subtasks))

    notes = "\n\n".join(f"## {s}\n{f}" for s, f in zip(subtasks, findings))
    return llm("Answer the question using only these findings, and keep "
               f"their citations.\n\nQuestion: {question}\n\n{notes}",
               model="large")
```

## Evaluator-optimizer

The evaluator-optimizer pattern pairs a generator call with an evaluator call in a loop: the generator drafts, the evaluator grades the draft against criteria and returns feedback, and the loop repeats until the draft passes or a round limit ends it. It fits tasks with clear evaluation criteria, where revision measurably improves the result.

**When it fits.** Anthropic's examples are literary translation, where an evaluator catches nuances the first draft missed, and complex search, where an evaluator decides whether another round of searching is needed. Its test for fit has two parts: human feedback clearly improves the output, and a model can give that kind of feedback. The loop is strongest when the evaluator has an external signal, such as a test suite, a compiler, a schema validator, or a fact check against sources.

**Cost and latency.** Each round costs two calls and adds two calls' worth of latency, and the number of rounds varies by input, so cap it in code. The OpenAI Agents SDK's pattern examples suggest using a smaller model to generate and a larger model to give feedback, which lowers the cost per round.

**Failure modes.** A model judging its own output without outside information is the weakest version of this loop. [Huang et al. (2023)](https://arxiv.org/abs/2310.01798) found that large language models struggle to correct their own reasoning without external feedback, and that performance sometimes drops after self-correction. [Self-Refine (Madaan et al., 2023)](https://arxiv.org/abs/2303.17651) reported about 20% absolute average improvement from iterative self-feedback on its tasks, so results depend on the task and on whether the critic has something to check against. Loops can also oscillate, fixing one issue while breaking another, and a strict evaluator never passes while a lenient one passes everything. Calibrate the evaluator against human grades as you would any [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge).

**Example: translation with a round limit.**

```python
def translate(text: str, max_rounds: int = 3) -> str:
    draft = llm(f"Translate into French. Keep the tone.\n\n{text}")
    for _ in range(max_rounds):
        verdict = json.loads(llm(
            "Grade this translation for accuracy, tone, and fluency. Return "
            'JSON only: {"pass": true or false, "feedback": "..."}\n\n'
            f"Source:\n{text}\n\nTranslation:\n{draft}", model="large"))
        if verdict["pass"]:
            return draft
        draft = llm("Revise the translation using this feedback.\n\n"
                    f"Feedback: {verdict['feedback']}\n\n"
                    f"Source:\n{text}\n\nCurrent translation:\n{draft}")
    raise RuntimeError("no passing draft; send to a human reviewer")
```

## How to choose an agentic workflow pattern

Choose a pattern by the shape of the task: a known sequence calls for chaining, distinct input types for routing, independent subtasks for parallelization, subtasks you cannot list in advance for orchestrator-workers, and measurable quality criteria for an evaluator loop. Then check the pattern's latency and cost profile against your budget.

| Pattern | Task shape | Latency | Cost | Reliability profile | Example task |
|---|---|---|---|---|---|
| Prompt chaining | Fixed sequence of known subtasks | Sum of all steps | Sum of all calls; steps can use smaller models | High when gates check content between steps | Release notes from a diff |
| Routing | Inputs fall into distinct categories | One classification call plus one path | Can drop when most traffic takes cheap paths | Capped by classifier accuracy | Support ticket triage |
| Parallelization: sectioning | Independent subtasks | Slowest branch plus merge | Sum of all branches | High when sections are truly independent | Pull request review by aspect |
| Parallelization: voting | One task where confidence matters | Slowest sample plus tally | Number of votes times one call | Better on majority-right tasks; correlated errors remain | Content policy checks |
| Orchestrator-workers | Subtasks unknown until the input is read | Plan, slowest worker, synthesis; varies per run | Varies with the plan; usually the highest | Depends on plan quality and worker context | Multi-source research |
| Evaluator-optimizer | Clear criteria; revision helps | Two calls per round, times rounds | Two calls per round, times rounds | Strong with external checks; weak with self-judgment alone | Translation against a rubric |

The latency column follows from how each pattern schedules its calls. Figure 2 shows the difference with three model calls of similar length in each pattern.

**Figure 2: Schematic timing in relative units, not a measurement. A chain adds up its steps, sectioning waits for its slowest branch, and an evaluator loop adds two calls per round.**

Schematic timeline: a three-step chain runs calls end to end, sectioning runs three calls at once then merges, and an evaluator loop alternates drafts and grades.

Spans:

- Prompt chain: 0 to 9.5 (steps add up)
  - Step 1: 0 to 3
  - Gate in code: 3 to 3.5
  - Step 2: 3.5 to 6.5
  - Step 3: 6.5 to 9.5
- Sectioning: 0 to 4 (slowest branch wins)
  - Section A: 0 to 3
  - Section B: 0 to 3.5
  - Section C: 0 to 2.5
  - Merge in code: 3.5 to 4
- Evaluator loop: 0 to 9 (two calls per round)
  - Draft: 0 to 3
  - Grade: fail: 3 to 4.5
  - Revise: 4.5 to 7.5
  - Grade: pass: 7.5 to 9

Patterns compose. A support workflow might route each ticket, run a chain on the refund path (extract the order ID, gate on a successful lookup, draft a reply), check the draft with an evaluator call, and run a safety screen in parallel with drafting. Write each pattern as a function over plain data, and the functions nest.

## When to graduate from a workflow to an agent

Graduate a step from workflow to agent when the number and order of actions depend on what the system discovers, so the flowchart cannot be completed in advance. Keep the workflow when the steps are known, when latency or cost must be bounded per request, or when every possible path must be reviewable before it runs.

Signals that a step has outgrown its workflow:

- **Branches keep multiplying.** Each new edge case adds a route or a gate, and the fallback route keeps growing.
- **Plans need revision mid-run.** The orchestrator's plan is often wrong once workers report back, and fixing it requires another planning round after each result.
- **The tool count per task is unknown.** Debugging a failing test or investigating an alert takes three calls one time and thirty the next.
- **People keep finishing the job.** Reviewers routinely do steps by hand that the workflow cannot express.

Strict latency targets, audits that require every path to be listed, irreversible actions, and the lack of an eval set all favor keeping fixed steps. Anthropic's guidance reserves agents for open-ended problems where the steps cannot be predicted and the model's decisions can be trusted, and it warns that their autonomy brings higher costs and compounding errors.

The usual middle ground is a bounded agent step inside a workflow: a [ReAct-style](https://theaiengineering.com/glossary/react-agent) loop with a small tool list, a turn limit, and a token budget, followed by a gate in code that checks what the agent produced. The workflow keeps control of everything before and after it. Our guide on [how to build an AI agent](https://theaiengineering.com/articles/how-to-build-an-ai-agent) writes that loop in Python, with stop conditions and approval gates.

## Human-in-the-loop checkpoints

A human-in-the-loop checkpoint pauses a workflow at a defined step so a person can approve, edit, or reject what happens next. Put checkpoints before irreversible side effects, after low-confidence gates, and before expensive fan-outs, and build each pause as saved state rather than a blocking call, because a person may answer hours later.

Places where a [human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop) checkpoint earns its delay:

- Before irreversible side effects, such as payments, refunds, sent emails, deleted records, and deploys.
- After a failed gate or a low-confidence route, instead of guessing.
- When an evaluator loop hits its round limit without a passing draft.
- Before a large fan-out, so a person approves the orchestrator's plan before dozens of worker calls run.
- On a sample of passing outputs, to measure whether the gates themselves still work.

The mechanics are the same in every framework: save the workflow's state, notify a reviewer, and resume from the saved state with the decision. In LangGraph, a node calls `interrupt()`, the graph's checkpointer saves state under a thread ID, and a later call with `Command(resume=...)` and the same thread ID continues the run. The [LangGraph interrupts documentation](https://docs.langchain.com/oss/python/langgraph/interrupts) notes that the interrupted node restarts from its beginning on resume, so side effects placed before the interrupt must be idempotent or moved after it. The OpenAI Agents SDK's examples pause runs for approval before sensitive tools, and Microsoft Agent Framework's orchestrations support approval-required tools that pause the workflow for review.

**Figure 3: A human checkpoint in a durable workflow. State is saved before the pause and reloaded on approval, so a restart in between loses nothing.**

Sequence: the workflow saves state and asks a reviewer for approval; after approval it reloads state, calls the payments API once with an idempotency key, and records the result.

Steps:

1. Workflow → Durable store: save state, pause
2. Workflow → Reviewer: approval request
3. Reviewer → Workflow: approve (minutes or days later)
4. Durable store → Workflow: load state
5. Workflow → Payments API: refund, with key (idempotency key prevents a double refund)
6. Workflow → Durable store: record result

## How to build agentic workflows: plain code first

Build agentic workflows in plain code first: each pattern is a few functions around a model call, and ordinary control flow, `asyncio`, and JSON parsing express all five. Move to a framework when you need persisted state, pauses for approval, streaming of intermediate steps, or built-in tracing that you would otherwise write and maintain yourself.

Anthropic's post makes the same recommendation: start with LLM APIs directly, and understand the code underneath any framework you adopt, because abstractions can hide the prompts and responses you need to debug. Plain functions also keep each step testable with a stubbed `llm()`.

In a framework, each pattern maps to a primitive. The table covers two frameworks whose documentation and examples implement all five.

| Pattern | LangGraph | OpenAI Agents SDK |
|---|---|---|
| Prompt chaining | Nodes joined by `add_edge`; a gate is a conditional edge | Sequential `Runner.run` calls with checks in code, as in the SDK's deterministic-flow example |
| Routing | `add_conditional_edges` with a router function, often over structured output | Structured output, then code picks the next agent; or `handoffs=[...]`, which the model sees as `transfer_to_<agent>` tools |
| Parallelization | Fan-out edges from one node and fan-in edges to an aggregator node | `asyncio.gather` over several `Runner.run` calls |
| Orchestrator-workers | A conditional edge that returns one `Send` per planned subtask | A manager agent that calls specialists exposed through `Agent.as_tool()` |
| Evaluator-optimizer | A conditional edge from the evaluator back to the generator | A `while` loop with an evaluator agent that returns a typed verdict, as in the SDK's LLM-as-a-judge example |

The OpenAI SDK's [orchestration guide](https://openai.github.io/openai-agents-python/multi_agent/) separates the two ways of delegating: with an [agent handoff](https://theaiengineering.com/glossary/agent-handoff), the specialist takes over the conversation, while with an agent as a tool, the manager keeps control and owns the final answer. Other frameworks name the same shapes differently. Microsoft Agent Framework ships [sequential, concurrent, handoff, group chat, and Magentic orchestrations](https://learn.microsoft.com/en-us/agent-framework/workflows/orchestrations/), where Magentic means a manager agent coordinating specialists. Google ADK 2.0 defines a `Workflow` as a graph of nodes (agents, tools, or functions) joined by edges, and its documentation marks the older `SequentialAgent`, `ParallelAgent`, and `LoopAgent` templates as deprecated.

Here is the orchestrator-workers sketch from earlier as a LangGraph graph, following the pattern in LangGraph's [workflows and agents guide](https://docs.langchain.com/oss/python/langgraph/workflows-agents). `Send` creates one worker invocation per planned section, and the `operator.add` reducer collects their outputs:

```python
import operator
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.types import Send

class State(TypedDict):
    topic: str
    sections: list[str]
    drafts: Annotated[list[str], operator.add]  # workers append here
    report: str

class WorkerState(TypedDict):
    section: str
    drafts: Annotated[list[str], operator.add]

def plan(state: State):
    reply = llm('Return JSON only: {"sections": ["..."]} for a report on '
                + state["topic"], model="large")
    return {"sections": json.loads(reply)["sections"][:6]}

def write_section(state: WorkerState):
    return {"drafts": [llm(f"Write the report section: {state['section']}")]}

def assign_workers(state: State):
    return [Send("write_section", {"section": s}) for s in state["sections"]]

def synthesize(state: State):
    return {"report": "\n\n".join(state["drafts"])}

builder = StateGraph(State)
builder.add_node("plan", plan)
builder.add_node("write_section", write_section)
builder.add_node("synthesize", synthesize)
builder.add_edge(START, "plan")
builder.add_conditional_edges("plan", assign_workers, ["write_section"])
builder.add_edge("write_section", "synthesize")
builder.add_edge("synthesize", END)
graph = builder.compile()  # graph.invoke({"topic": "..."})
```

The graph is longer than the plain function; what it buys is a runtime with checkpointing, interrupts, and streaming. Our [AI agent frameworks comparison](https://theaiengineering.com/articles/ai-agent-frameworks) covers how 13 frameworks handle orchestration, and the [agent frameworks directory](https://theaiengineering.com/platforms/agent-frameworks) tracks more.

## Durable execution and retries for long workflows

Durable execution records each completed step of a workflow so that a crash, deploy, or long pause resumes from the last finished step instead of starting over. Long agentic workflows need it because model calls are slow, cost money, and are non-deterministic: rerunning a finished step wastes tokens and can return a different answer than later steps used.

Whichever engine you choose, the same rules apply:

- **Make every model call and side effect a recorded step.** On replay, the engine returns the recorded result instead of calling the model again.
- **Keep orchestration code deterministic.** The code that decides which step runs next must make the same decisions given the same recorded history, so time, randomness, and I/O belong inside steps.
- **Retry transient errors per step, with backoff and a limit.** Rate limits, timeouts, and server errors are worth retrying; invalid requests and content refusals are not.
- **Give side effects idempotency keys**, so a retried refund or email happens once.

A gate failure is not a transient error. Retrying the same call with the same input rarely fixes a wrong answer, so handle gate failures in workflow logic: revise the input, route to a fallback, or ask a person.

| Engine | How steps are recorded | Retries and waits |
|---|---|---|
| Temporal | Activity results, including LLM invocations, are stored in the event history and reused on replay | Activities retry by default (1 second initial interval, backoff coefficient 2.0, unlimited attempts); workflows do not |
| DBOS | Functions decorated with `@DBOS.workflow()` and `@DBOS.step()`; recovery resumes from the last completed step | Steps take configurable retries; workflows must be deterministic |
| Inngest | Each `step.run` result is saved; completed steps return it instead of rerunning | A failing step retries alone; `step.waitForEvent` resumes when an event arrives |
| LangGraph | A checkpointer saves graph state per thread | Durability modes `exit`, `async`, and `sync` trade write overhead against crash safety |

The Temporal defaults come from its [retry policy documentation](https://docs.temporal.io/encyclopedia/retry-policies), as checked in November 2026; for model calls, set a maximum attempt count and non-retryable error types instead of retrying forever. Anthropic's research system post reports the same need: its long-running agents resume from where they were when an error occurred instead of restarting. Our [agent infrastructure directory](https://theaiengineering.com/platforms/agent-infrastructure) lists durable execution platforms alongside sandboxes and memory layers.

## Evaluating and observing agentic workflows step by step

Workflows are easier to evaluate than agents because every step has a fixed job, so each step can be tested in isolation with its own dataset and metric. Evaluate the router as a classifier, the gates as detectors, the evaluator as a judge calibrated against people, and the whole run on outcome, cost, and latency.

Per-step evaluation catches regressions that an end-to-end score hides:

| Step | What to measure |
|---|---|
| Router | Accuracy and a confusion matrix on labeled inputs, showing which routes get mixed up |
| Gates | False-pass and false-fail rates; a gate that never fails is either unnecessary or broken |
| Generation steps | Task-specific checks on inputs captured from real traces, rerun when a prompt or model changes |
| Evaluator | Agreement with human grades on a labeled set |
| Orchestrator plans | Subtask coverage, overlap, and fan-out width |
| Whole run | Task success, cost per run, and tail latency over repeated runs |

Our [AI agent evaluation guide](https://theaiengineering.com/articles/ai-agent-evaluation) covers graders and reliability across repeated runs; the methods apply to workflows, with less variance in the path.

For observability, record one trace per run with one span per step, and attach what the step decided (the route label, gate outcome, evaluator rounds, or fan-out width) along with tokens and latency. Under the OpenTelemetry GenAI conventions, the run's root span reports the `invoke_workflow` operation, with model-call and `execute_tool` spans beneath it. Dashboards on those spans show whether the route mix is drifting, which gate fails most, how many rounds the evaluator needs, and which step costs the most. Our [LLM observability guide](https://theaiengineering.com/articles/llm-observability) covers tracing tools and what to capture. Turn each failed run into an eval case for the step that caused it.

## Where to go next

Start with the simplest pattern that fits the task, usually a chain or a router, and add parallel branches, evaluator loops, or a planning model only when an eval shows they help. When one step needs to choose its own actions, put a bounded agent loop inside the workflow instead of replacing the workflow.

## Frequently asked questions

### What is an example of an agentic workflow?

A support-ticket workflow is a common example. A small model classifies each ticket, code routes it to a billing, technical, or account path, a model drafts a reply from the customer's records, a second call checks the draft against policy, and a person approves any refund before it is issued. Every step and branch is fixed in code; models handle only classifying, drafting, and checking.

### What is the difference between an agentic workflow and an AI agent?

In an agentic workflow, code defines the steps and branches, and each model call does one bounded job. In an AI agent, model output chooses the next action at runtime, usually by calling tools in a loop until it decides the task is done. Workflows are more predictable, easier to budget, and easier to test; agents handle open-ended tasks whose steps cannot be listed in advance.

### Is RAG an agentic workflow?

A basic retrieval-augmented generation pipeline is a two-step chain: retrieve documents, then generate an answer from them. It is a workflow, but few teams call it agentic, because no model output changes the path. RAG becomes agentic when a model rewrites queries, decides whether to search again, grades retrieved passages, or chooses between sources, which adds routing, evaluator loops, or an agent loop around retrieval.

### How are agentic workflows different from traditional workflow automation?

Traditional workflow automation runs deterministic rules over structured data: when a field has a value, take an action. An agentic workflow keeps that deterministic skeleton but adds model calls for steps that need language understanding or judgment, such as classifying a free-text request, extracting fields from an email, drafting a reply, or grading output against a rubric. In both, the order of steps stays in code.

### Do you need a framework to build agentic workflows?

No. All five patterns fit in plain Python with functions, conditionals, loops, and asyncio for parallel calls. Frameworks such as LangGraph, the OpenAI Agents SDK, and Microsoft Agent Framework add persisted state, pauses for human approval, streaming, and tracing, and durable execution engines such as Temporal add recorded steps and retries. Adopt one when a workflow runs long enough to need those features.

### What is the difference between an agentic workflow and a multi-agent system?

An agentic workflow is defined by where control flow lives: in code. A multi-agent system is defined by its parts: several agents, each with its own prompt, tools, and context. The two overlap. An orchestrator-workers workflow whose workers are agents is a multi-agent system with a code-defined structure, while agents that pass control among themselves freely form a multi-agent system without one.

## Terms in this guide

- [AI agent](https://theaiengineering.com/glossary/ai-agent): An AI agent is a software system in which a large language model runs in a loop, calling tools, reading their results, and repeating until the task is complete or a stop condition is reached.
- [Agentic AI](https://theaiengineering.com/glossary/agentic-ai): Agentic AI is the class of AI systems that pursue a goal through multiple steps, using a language model to plan, call tools, and act on the results with limited human input.
- [Tool calling](https://theaiengineering.com/glossary/tool-calling): Tool calling is an LLM capability in which the model, given a list of tool definitions, outputs a structured request naming a tool and its arguments, which the application executes and returns.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [Multi-agent system](https://theaiengineering.com/glossary/multi-agent-system): A multi-agent system is an AI architecture in which several language-model agents, each with its own prompt, tools, and context, work on parts of a task and pass results to one another.
- [Subagent](https://theaiengineering.com/glossary/subagent): A subagent is an AI agent that another agent starts for one subtask, running in a separate context window with its own instructions and tools, and returning only a condensed result.
- [Agent orchestration](https://theaiengineering.com/glossary/agent-orchestration): Agent orchestration is the control logic that decides which agent, model call, or tool runs next in an AI system and what each receives, whether written in code or chosen by a model.
- [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge): LLM-as-a-judge is an evaluation method in which a language model scores or compares the outputs of an LLM application against a rubric, a reference answer, or another response, replacing or supplementing human graders.
- [ReAct agent](https://theaiengineering.com/glossary/react-agent): A ReAct agent is an AI agent that alternates a written reasoning step, an action such as a tool call, and an observation of the result, repeating the cycle until it can answer.
- [Human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop): Human-in-the-loop (HITL) is an agent design in which the run pauses at defined checkpoints, such as before a sensitive tool call, so a person can approve, edit, or reject the action.
- [Agent handoff](https://theaiengineering.com/glossary/agent-handoff): An agent handoff is a transfer of control in which one AI agent passes the conversation, and responsibility for the task, to another agent better suited to the next part of the work.

## Related guides

- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.
- [How to build an AI agent: a step-by-step guide in Python](https://theaiengineering.com/articles/how-to-build-an-ai-agent): Learn how to build an AI agent from scratch in Python: when you need one, tool schemas, the agent loop, stop conditions, guardrails, evals, and tracing.
- [AI agent frameworks compared: LangGraph, CrewAI, ADK, and more](https://theaiengineering.com/articles/ai-agent-frameworks): An AI agent frameworks comparison of LangGraph, LangChain, CrewAI, the OpenAI, Claude, and Google SDKs, Pydantic AI, Mastra, and more, by language and license.
- [AI agent evaluation: metrics, graders, and how to test agents](https://theaiengineering.com/articles/ai-agent-evaluation): AI agent evaluation guide: measure task success, tool use, safety, cost, and pass^k reliability with code graders, LLM judges, eval sets from traces, and CI.

Part of the [AI agents](https://theaiengineering.com/topics/ai-agents) topic hub.

## Cite this guide

The AI Engineering Editors. "Agentic workflows: five patterns and when to use each." The AI Engineering, updated Nov 16, 2026. https://theaiengineering.com/articles/agentic-workflows

- Title: Agentic workflows: five patterns and when to use each
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-11-16
- Updated: 2026-11-16
- URL: https://theaiengineering.com/articles/agentic-workflows
- Markdown: https://theaiengineering.com/articles/agentic-workflows.md

To cite one section, link to its anchor:

- [What is an agentic workflow?](https://theaiengineering.com/articles/agentic-workflows#what-is-an-agentic-workflow)
- [The five agentic workflow patterns at a glance](https://theaiengineering.com/articles/agentic-workflows#the-five-agentic-workflow-patterns-at-a-glance)
- [Prompt chaining with gates](https://theaiengineering.com/articles/agentic-workflows#prompt-chaining-with-gates)
- [Routing](https://theaiengineering.com/articles/agentic-workflows#routing)
- [Parallelization: sectioning and voting](https://theaiengineering.com/articles/agentic-workflows#parallelization-sectioning-and-voting)
- [Orchestrator-workers](https://theaiengineering.com/articles/agentic-workflows#orchestrator-workers)
- [Evaluator-optimizer](https://theaiengineering.com/articles/agentic-workflows#evaluator-optimizer)
- [How to choose an agentic workflow pattern](https://theaiengineering.com/articles/agentic-workflows#how-to-choose-an-agentic-workflow-pattern)
- [When to graduate from a workflow to an agent](https://theaiengineering.com/articles/agentic-workflows#when-to-graduate-from-a-workflow-to-an-agent)
- [Human-in-the-loop checkpoints](https://theaiengineering.com/articles/agentic-workflows#human-in-the-loop-checkpoints)
- [How to build agentic workflows: plain code first](https://theaiengineering.com/articles/agentic-workflows#how-to-build-agentic-workflows-plain-code-first)
- [Durable execution and retries for long workflows](https://theaiengineering.com/articles/agentic-workflows#durable-execution-and-retries-for-long-workflows)
- [Evaluating and observing agentic workflows step by step](https://theaiengineering.com/articles/agentic-workflows#evaluating-and-observing-agentic-workflows-step-by-step)
- [Where to go next](https://theaiengineering.com/articles/agentic-workflows#where-to-go-next)
- [Frequently asked questions](https://theaiengineering.com/articles/agentic-workflows#faq)
