How to build an AI agent: a step-by-step guide in Python
An AI agent is a program that calls a language model in a loop, runs the tools the model requests, and feeds the results back until the task is done or a limit stops it. This guide shows how to build an AI agent from scratch in Python: deciding whether you need one, defining tools, writing and testing the loop, and adding the state, guardrails, evals, and tracing that production requires. For the concepts behind the loop, read what AI agents are; this guide is part of our AI agents topic hub.
The steps to build an AI agent#
Building an AI agent takes seven steps: confirm that the task needs an agent, choose a model and define tools, write the loop with stop conditions, add state and context management, add guardrails, evaluate on real tasks, and trace production runs. Each step below includes the code or checks it needs, and the steps build on one example.
| Step | Done when |
|---|---|
| 1. Decide whether you need an agent | Fixed steps and judgment calls are separated |
| 2. Choose a model and define tools | A person could use each tool from its description alone |
| 3. Write the loop | A scripted run passes without an API key |
| 4. Add state and context management | Long runs stay inside the context window |
| 5. Add guardrails | No write runs without a check in code |
| 6. Evaluate | You can compare two versions by pass rate |
| 7. Observe | Any production run can be explained |
The running example is a support agent with two tools: get_order reads an order, and refund_order moves money, so it needs a person's approval.
Step 1: decide whether you need an agent#
Build an agent only when the next step depends on what the system finds at runtime. If you can draw the whole process as a flowchart, write it as a workflow: fixed code that calls a model at chosen points. Workflows are cheaper, faster, and easier to test, and many tasks labeled agentic need no loop at all.
Anthropic's post Building effective agents draws the same line: workflows run models and tools through predefined code paths, while agents let the model direct its own process. It recommends adding complexity only when needed. A workflow with one open-ended step is often the right design, even for systems described as agentic AI.
| If the task | Build |
|---|---|
| Runs the same steps every time | One model call or a fixed chain of calls |
| Follows one of a few known paths | A router: classify, then run fixed code |
| Needs a variable number of actions chosen from earlier results | An agent loop |
| Is mostly fixed, with one open-ended step | A workflow with one agent step inside it |
Then state the task in one sentence with an end state you can check, for example: resolve a refund request for one order, with at most one refund and a person's approval before money moves. That sentence shapes the system prompt, the tool list, and the eval criteria.
Step 2: choose a model and define tools#
Pick a model that handles tool calling reliably, then define each tool with a name, a description that says when to use it, and a JSON Schema for its arguments. Start with a capable model to prove the task is solvable, and test smaller, cheaper models against your eval set once it works.
Judge models on whole tasks, since latency and cost add up across an agent's calls. The example uses Anthropic's Python SDK (pip install anthropic, version 2.6.0 at the time of writing) with claude-opus-5-5, which Anthropic's documentation suggests as a starting point; claude-sonnet-5-5 and claude-haiku-5-5 are faster, cheaper candidates to test later.
Each tool is a Python function plus a schema. A registry holds the function, the schema the model sees, and a policy flag the model never sees:
import json
ORDERS = {"A-1001": {"status": "delivered", "total_usd": 42.50,
"refunded": False}}
def get_order(order_id: str) -> dict:
if order_id not in ORDERS:
raise LookupError(f"No order {order_id}. IDs look like A-1001.")
return {"order_id": order_id, **ORDERS[order_id]}
def refund_order(order_id: str, reason: str) -> dict:
order = get_order(order_id)
if order["refunded"]:
raise ValueError(f"Order {order_id} is already refunded.")
ORDERS[order_id]["refunded"] = True
return {"order_id": order_id, "reason": reason,
"refunded_usd": order["total_usd"]}
GET_ORDER = {
"name": "get_order",
"description": ("Look up one order by ID. Returns its status, "
"total in USD, and whether it was refunded."),
"input_schema": {
"type": "object",
"properties": {"order_id": {"type": "string",
"description": "e.g. A-1001"}},
"required": ["order_id"],
},
}
REFUND_ORDER = {
"name": "refund_order",
"description": ("Refund an order in full. Moves money, so a person "
"approves each call. Call get_order first and "
"never refund twice."),
"input_schema": {
"type": "object",
"properties": {
"order_id": {"type": "string"},
"reason": {"type": "string", "enum": [
"damaged", "not_delivered", "wrong_item"]},
},
"required": ["order_id", "reason"],
},
}
TOOLS = {
"get_order": {"run": get_order, "schema": GET_ORDER,
"needs_approval": False},
"refund_order": {"run": refund_order, "schema": REFUND_ORDER,
"needs_approval": True},
}Four choices here carry over to any agent:
- Descriptions are instructions. The refund description tells the model to look up the order first and never refund twice.
- Enums close off invalid input.
reasonaccepts only values the refund logic handles, and adding"strict": trueto a tool definition makes the model's arguments always match the schema. - Policy stays outside the schema. The model never sees
needs_approval, so no prompt can argue past it. - Errors teach. "IDs look like A-1001" tells the model how to fix its next call.
If your tools already run as Model Context Protocol servers, an MCP client can list them and pass their schemas into the same loop.
Step 3: write the agent loop in Python#
The agent loop sends the conversation and tool schemas to the model, checks why the model stopped, runs any requested tools, appends their results, and repeats. It ends on a final answer, or raises an error when a turn limit, a token budget, or an unexpected stop reason ends the run first.
import anthropic
MODEL = "claude-opus-5-5"
SYSTEM = ("You are a support agent for an online store. Check facts "
"with the tools before you answer. Never claim an action "
"you did not take.")
MAX_RESULT_CHARS = 4_000
def run_tool(block, approve) -> dict:
"""Run one tool_use block and return its tool_result block."""
try:
tool, args = TOOLS.get(block.name), block.input
if tool is None:
raise LookupError(f"Unknown tool {block.name}.")
if tool["needs_approval"] and not approve(block.name, args):
raise PermissionError("Reviewer declined; do not retry.")
output = json.dumps(tool["run"](**args))[:MAX_RESULT_CHARS]
result = {"content": output}
except Exception as exc: # the model reads the error and adjusts
result = {"content": f"{type(exc).__name__}: {exc}",
"is_error": True}
return {"type": "tool_result", "tool_use_id": block.id, **result}
def run_agent(client, task, approve, max_turns=10,
token_budget=100_000):
messages = [{"role": "user", "content": task}]
schemas = [t["schema"] for t in TOOLS.values()]
tokens_used = 0
for _ in range(max_turns):
response = client.messages.create(
model=MODEL, max_tokens=4096, system=SYSTEM,
tools=schemas, messages=messages,
)
usage = response.usage
tokens_used += usage.input_tokens + usage.output_tokens
content = response.content
messages.append({"role": "assistant", "content": content})
stop = response.stop_reason
if stop == "end_turn": # final answer
return "".join(b.text for b in content if b.type == "text")
if stop != "tool_use": # max_tokens, refusal, and so on
raise RuntimeError(f"Model stopped with {stop}.")
if tokens_used > token_budget:
raise RuntimeError(f"Over budget: {tokens_used} tokens.")
calls = [b for b in content if b.type == "tool_use"]
results = [run_tool(c, approve) for c in calls]
messages.append({"role": "user", "content": results})
raise RuntimeError(f"No final answer after {max_turns} turns.")
if __name__ == "__main__":
def ask(name, args):
return input(f"Allow {name}({args})? [y/N] ").strip() == "y"
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
print(run_agent(client, "Order A-1001 arrived broken.", ask))Four details make this loop correct rather than merely short:
- The full assistant turn goes back into history, including its
tool_useblocks. Anthropic's documentation on handling tool calls requires eachtool_resultto carry its call'stool_use_idand to arrive in the user message right after the call, before any text. - Tool failures become results.
run_toolreturns exceptions withis_error: true, so a bad argument or a denied approval becomes information the model can act on. - The budget counts every call. Each request resends the whole history, so input tokens grow every turn; the running sum tracks the run's cost.
- Unexpected stop reasons fail loudly, so a truncated or refused response is never mistaken for an answer.
The stop reasons come from Anthropic's stop reason reference; the last three exits are limits you add yourself.
| Exit | Trigger | What the loop should do |
|---|---|---|
| Final answer | stop_reason is end_turn |
Return the text |
| Tool call | stop_reason is tool_use |
Run the tools, append results, continue |
| Truncated output | max_tokens or model_context_window_exceeded |
Raise; retry with a higher max_tokens if a tool call was cut off |
| Refusal | refusal |
Raise and route to a person or a fallback |
| Server tool pause | pause_turn, only with provider-run tools |
Resend the assistant content to continue |
| Turn limit | max_turns reached |
Raise with the turn count |
| Token budget | Summed usage passes token_budget |
Raise before running more tools |
| Deadline | Wall-clock time passes your limit | Check elapsed time each turn |
Run the loop without an API key#
Because run_agent takes the client as a parameter, a test can pass in any object with a messages.create method. This stub returns scripted replies built from the SDK's own Message type, so the loop parses them as it would real API responses:
from anthropic.types import Message
def reply(stop_reason, *content):
return Message.model_validate({
"id": "msg_test", "type": "message", "role": "assistant",
"model": MODEL, "stop_reason": stop_reason,
"content": list(content),
"usage": {"input_tokens": 500, "output_tokens": 50},
})
def call(call_id, name, **args):
return {"type": "tool_use", "id": call_id, "name": name,
"input": args}
class ScriptedClient:
"""Stands in for anthropic.Anthropic(); replays canned replies."""
def __init__(self, *replies):
self.replies, self.requests = list(replies), []
self.messages = self # makes client.messages.create(...) work
def create(self, **request):
self.requests.append(request)
return self.replies.pop(0)
client = ScriptedClient(
reply("tool_use", call("t1", "get_order", order_id="A-1001")),
reply("tool_use", call("t2", "refund_order", order_id="A-1001",
reason="damaged")),
reply("end_turn", {"type": "text", "text": "Refunded $42.50."}),
)
approved = []
def approve(name, args):
approved.append(name)
return True
answer = run_agent(client, "Order A-1001 arrived broken.", approve)
assert answer == "Refunded $42.50."
assert approved == ["refund_order"] # only the write needed approval
assert ORDERS["A-1001"]["refunded"] # and the refund went through
assert len(client.requests) == 3
print("control flow OK")The script prints control flow OK. The same pattern covers the other branches: a denied approval should leave the order untouched, endless tool calls should hit the turn limit, and a max_tokens reply should raise. We ran these checks against SDK version 2.6.0, along with a run through the real client over a mock HTTP transport. Set ANTHROPIC_API_KEY and run the loop file to use the live API.
How the agent loop maps to other providers#
The loop's structure is the same on each API below: send history and tool definitions, detect tool calls, run them, and return results matched by call ID. What changes is field names, how arguments are encoded, and whether the provider stores the conversation for you or expects you to resend it every turn.
| Loop step | Anthropic Messages API | OpenAI Responses API | Gemini Interactions API |
|---|---|---|---|
| Tool definition | name, description, input_schema |
type: "function", name, description, parameters |
type: "function", name, description, parameters |
| Detect a call | tool_use blocks; stop_reason is tool_use |
function_call items in response.output |
function_call steps in interaction.steps |
| Arguments | input, already parsed |
arguments, a JSON string to parse |
arguments, already parsed |
| Return a result | tool_result block with tool_use_id |
function_call_output item with call_id |
function_result step with call_id |
| History | Resend all messages each turn | Append response.output and results to the input list |
Send new results with previous_interaction_id |
OpenAI's function calling guide adds one rule for reasoning models: reasoning items returned with tool calls must be passed back with the tool outputs. To port run_agent, change only the create call, call detection, and the result format; the tools, registry, approval gate, and limits stay the same.
Step 4: add memory, state, and context management#
An agent's short-term memory is the message list the loop builds; its long-term memory is anything stored outside the model and read back through tools. Context management keeps that message list inside the context window and focused, by trimming tool results, compacting old turns, and moving durable facts into storage.
- Persist the session. Save
messagesunder a session ID after every turn, so a crash or a pending approval can resume.model_dump(exclude_none=True)turns the SDK's content blocks into plain dictionaries. - Shape tool results.
MAX_RESULT_CHARSis a backstop. Better tools filter, paginate, and summarize at the source, because every result stays in the context window for the rest of the run. - Compact whole turns. Summarize older turns past a size threshold, but never separate a
tool_useblock from itstool_result, because the API rejects that history. - Add long-term memory as tools. A
save_noteandsearch_notespair backed by a database gives the agent memory across sessions; Anthropic's memory tool is one provider-defined version.
Keep the system prompt and tool list identical between calls so prompt caching can reuse the repeated prefix. Our context engineering guide covers the remaining techniques, including sub-agents.
Step 5: add guardrails#
The guardrails that matter most for an agent are checks in your code, not instructions in the prompt. Limit which tools each agent can call, run tools with the requesting user's permissions, require a person to approve actions with side effects, validate arguments against business rules, and check the final answer against what the tools actually did.
| Risk | Control | In the example |
|---|---|---|
| A harmful or mistaken action | Approval gate for writes | needs_approval checked in run_tool |
| Too much access | A small tool set per agent; user-scoped credentials | TOOLS exposes two tools |
| Out-of-policy arguments | Schema checks plus business rules in the function | enum on reason; no second refund |
| Repeated writes | Idempotency keys; denials that say not to retry | "do not retry" in the denial |
| Instructions hidden in tool results | Keep results in tool_result blocks; limit what writes can do |
Results never enter the system prompt |
| Runaway cost | Turn and token limits | max_turns, token_budget |
Prompt injection is the risk that prompt wording cannot fix. Anthropic's tool documentation warns that results often carry content from outside your control and advises keeping it inside tool_result blocks, not system prompts or user text. The stronger defense is scope: an agent whose only write needs approval cannot be talked into moving money on its own.
The example's approval callback blocks while it waits. In a web application, save the session, return a pending status, and resume after a person decides. Classifier-based guardrails complement these controls but do not replace them.
Step 6: test and evaluate the agent#
Test an agent at two levels. Deterministic tests with a scripted model check the control flow: limits, approvals, and error handling. Evaluations run the real model on a fixed set of real tasks and score both the outcome and the path taken, several times per task, because the same input can produce different runs.
The scripted tests from Step 3 belong in your unit test suite. For evaluations, collect tasks from real requests, including awkward ones: a missing order ID, an order already refunded, someone else's order, and a message with instructions aimed at the agent. Score each run on:
- Outcome: the right end state, such as exactly one refund or none.
- Trajectory: an order lookup before any refund, and no unapproved writes.
- Answer: a reply that matches what happened, checked in code or by an LLM as a judge.
- Cost: turns and tokens within your limits.
Rerun the set whenever the prompt, a tool, or the model changes, and compare pass rates rather than single runs. Our AI agent evaluation guide covers trajectory metrics, repeated-run reliability, and graders, and the LLM evaluation guide covers datasets and CI gates.
Step 7: observe the agent in production#
In production, record a trace for every agent run, with one span per model call and one per tool call, carrying tokens, stop reasons, tool arguments, errors, and approval decisions. Traces let you explain any single run; metrics built from them, such as turns and tokens per task, show drift across runs.
The OpenTelemetry GenAI semantic conventions name this structure: an invoke_agent span for the run, model-call spans whose gen_ai.operation.name is an operation such as chat, and an execute_tool {gen_ai.tool.name} span per tool call with attributes such as gen_ai.tool.call.id. The conventions still carry Development status, so pin the version you emit. In the example, spans belong around client.messages.create and inside run_tool.
Track turns and tokens per completed task, stop reasons and limit hits, error rate per tool, approval and denial rates, and model versus tool latency. Store captured prompts and results with access controls, since they often hold personal data, and turn each failed run into an eval case. Our LLM observability guide compares tracing tools.
When to move from a hand-written loop to a framework#
Keep the hand-written loop while one agent, one provider, and synchronous approvals cover the task. Move to a framework when you need durable execution that survives restarts and long approval waits, several agents coordinating, many model providers behind one interface, or built-in tracing and evals you would otherwise build and maintain yourself.
| Need | Signal that you have outgrown the loop |
|---|---|
| Durable state | Approvals take hours; runs must survive a deploy or crash |
| Coordination | Subtasks need their own agents and context, as in a multi-agent system |
| Provider portability | You maintain your own translation layer between model APIs |
| Tooling | You are building your own trace viewer or eval runner |
A middle step exists. Anthropic's Python SDK has a beta tool runner, client.beta.messages.tool_runner, that executes tools and returns results automatically, with max_iterations as a bound; Anthropic's documentation recommends the manual loop when you need human approval, custom logging, or conditional execution. Frameworks run a version of this loop underneath. Our AI agent frameworks comparison sorts current options by language, orchestration model, and license; the agent frameworks directory tracks them.
Where to go next#
Start small: two or three tools, the loop above, a scripted test, and a handful of real tasks to evaluate against. Add state, guardrails, and tracing as failures demand, and read our tool calling explainer next to improve the tools the loop depends on.
Frequently asked questions
Can you build an AI agent without a framework?
- Yes. An agent is a loop around a model API: call the model with tool definitions, run the tools it requests, append the results, and repeat until it returns an answer. With a provider's official Python SDK, that loop fits in a few dozen lines. Frameworks add durable state, multi-agent orchestration, and built-in tracing, which become worth their cost once a project needs them.
What is the best programming language for building AI agents?
- Python and TypeScript have the widest support across provider SDKs and agent frameworks, so most tutorials and examples use one of them. Other languages work too, because an agent only needs HTTP calls and JSON. Anthropic, for example, publishes official SDKs for Python, TypeScript, Java, Go, C#, Ruby, and PHP. Choose the language your team already runs in production.
Do you need machine learning experience to build an AI agent?
- No. Building an agent on a hosted model is software engineering: calling an API, writing JSON Schemas for tools, handling errors, storing state, and testing. You do not train or fine-tune a model. The skills that matter most are clear tool design, careful permission and approval rules, and an evaluation set that measures whether the agent completes real tasks.
How do you stop an AI agent from looping forever?
- Enforce limits in code, not in the prompt. Cap the number of model calls per task, add up the tokens reported on each response against a budget, set a wall-clock deadline, and stop when the model repeats the same failing tool call. Treat any stop reason other than a final answer or a tool call as an error, and log which limit ended each run.
How do you test an AI agent without an API key?
- Pass the model client into the agent loop as a parameter, then substitute a stub that returns scripted responses built from the SDK's own response types. The stub tests every branch of the control flow, including tool errors, denied approvals, turn limits, and token budgets, in milliseconds and at no cost. Answer quality still needs evaluations that run the real model on real tasks.
How much does it cost to run an AI agent?
- An agent's cost is the sum of its model calls, and each call is billed by input and output tokens. Every turn resends the growing history, so input tokens rise with each step, and a ten-turn task costs more than ten times its first call. Measure tokens per task in your traces, cap them with a budget, and use prompt caching to cut the cost of the repeated prefix.
Terms in this guide
- AI agent
- Agentic AI
- Tool calling
- Model Context Protocol (MCP)
- Tokens
- Context window
- Agent memory
- Prompt caching
- Prompt injection
- Guardrails
- LLM-as-a-judge
- LLM evaluation
- LLM observability
- Multi-agent system
- Human-in-the-loop
- System prompt
- JSON Schema
- Golden dataset
- Context compaction
- Subagent
- OpenTelemetry
- Agent orchestration
Cite this guide
The AI Engineering Editors. "How to build an AI agent: a step-by-step guide in Python." The AI Engineering, updated Nov 05, 2026. https://theaiengineering.com/articles/how-to-build-an-ai-agent
- Title
- How to build an AI agent: a step-by-step guide in Python
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Published
- Updated