AI engineer roadmap: skills, projects, and resources by stage
An AI engineer roadmap is a sequenced plan for learning to build production software on top of foundation models. This roadmap has seven stages, from programming foundations to production systems, and each stage lists the skills to learn, a project to ship, and the resources we consider worth the time. The roadmap assumes you want to build AI products, not train models; if the distinction is new, start with what AI engineering is and the AI engineering topic hub.
How to become an AI engineer: the roadmap at a glance#
To become an AI engineer, build a software engineering base first, then learn to call models, control them through context and tools, ground them with retrieval, measure them with evals, extend them into agents, and run them in production. The order matters: each stage depends on the one before it, and one project should grow through all of them.
| Stage | Focus | Project to ship | Ready to move on when |
|---|---|---|---|
| 0 | Programming foundations | A tested, containerized REST service | You can build, test, and deploy a small service alone |
| 1 | LLM fundamentals and model APIs | A document extraction CLI | You can predict token cost and get valid JSON back |
| 2 | Prompts, context, and tool calling | A tool-using assistant | Tool calls succeed on varied, messy inputs |
| 3 | Retrieval and RAG | Q&A over a docs set, with citations | You can report retrieval recall, not just answer quality |
| 4 | Evals and observability | An eval suite and tracing for Stage 3 | A CI job fails when quality drops |
| 5 | Agents and MCP | A bounded agent using your own MCP server | The agent stops reliably and asks before writes |
| 6 | Production systems | The project deployed behind a gateway | You can show cost, latency, and security data |
Stage 0: Python and programming foundations#
Stage 0 is the software base that every later stage assumes: one language used well, HTTP APIs, Git, SQL, testing, and deploying a small web service. Much of AI engineering work is integration code around model calls, so weak programming skill shows up in every later stage as slow progress and fragile projects.
Learn:
- Python: functions, classes, type hints, virtual environments, packaging, and
async/await. - HTTP and JSON: calling REST APIs, handling errors, retries, and timeouts.
- A web framework such as FastAPI, and data validation with Pydantic.
- Git, pull requests, and a basic CI pipeline.
- SQL and one relational database.
- Automated testing with pytest.
- Building and running a Docker container.
Build: a small REST service that wraps a public API, with typed request and response models, tests, and a Dockerfile.
Resources: the official Python tutorial, and the FastAPI and Pydantic documentation. Pydantic in particular reappears across AI tooling, including model SDKs and agent frameworks.
Stage 1: LLM fundamentals and model APIs#
Stage 1 builds an accurate mental model of how language models behave and how to call them from code. The goal is not training a model; the goal is predicting model behavior: why outputs vary between runs, how tokens drive cost and limits, why long inputs degrade answers, and how to get structured data back reliably.
Learn:
- Tokens, tokenization, and the context window as a hard limit and a cost driver.
- Sampling parameters: temperature, top-p, and maximum output tokens.
- Transformer basics at a conceptual level: attention, next-token prediction, pretraining, and post-training.
- Calling at least two providers' APIs, with streaming, error handling, and rate-limit backoff.
- Structured outputs with JSON schemas, plus validation of what comes back.
- Reasoning models and when their extra latency and tokens are worth paying for.
Build: a command-line tool that extracts fields from invoices or emails into validated JSON. Run it with two different models on 50 documents and record accuracy, latency, and token cost for each.
Resources: Andrej Karpathy's "Neural Networks: Zero to Hero" video series for intuition about how models work; the Hugging Face LLM Course for transformers and the open-model ecosystem; and each model provider's API reference.
Stage 2: Prompt engineering, context, and tool calling#
Stage 2 teaches you to control model behavior through what goes into context and which functions the model can request. Prompt and context design is the main lever an AI engineer has without retraining, and tool calling is the bridge from generating text to triggering actions in software.
Learn:
- System prompts: role, scope, constraints, output format, and few-shot examples.
- Context engineering: what to include on each call, how to trim history, and when to summarize.
- Tool calling: function schemas, tool descriptions, malformed arguments, and parallel calls.
- Prompt versioning: storing prompts in the repository and reviewing changes like code.
- Prompt caching for long, repeated prefixes.
Build: an assistant that answers questions by calling two or three real APIs through tool calls, such as a weather service, a calendar, and a small database. Keep the prompt in version control and log every tool call.
Resources: the prompt engineering and tool-use guides published by the major model providers, read side by side to see which advice is provider-specific.
Stage 3: Retrieval-augmented generation (RAG)#
Stage 3 covers retrieval-augmented generation: finding the right passages in your own data and placing them in context, so the model answers from facts it was never trained on. Answer quality is capped by retrieval quality, because a model cannot cite a passage the search step never returned, so this stage measures search before tuning prompts.
Learn:
- Embeddings and similarity search.
- Chunking strategies, document metadata, and filtering.
- Vector databases and their indexing tradeoffs.
- Hybrid search, combining keyword and vector results, followed by reranking.
- Retrieval metrics such as recall at k and mean reciprocal rank.
- Citations, and when to answer "not found" instead of guessing.
Build: a Q&A service over a documentation set, such as an open-source project's docs, that cites its sources. Label 50 questions with the passage that answers each, and report recall at 5 before and after adding hybrid search and reranking.
Resources: chapter 6, "RAG and Agents," of Chip Huyen's AI Engineering (O'Reilly, 2025), and our explainer on retrieval-augmented generation.
Stage 4: LLM evaluation and observability#
Stage 4 turns experiments into engineering: building evaluation sets, scoring outputs automatically, running evals on every change, and tracing production requests. Evals come before agents on this roadmap because agents multiply the number of ways a system can fail, and measurement has to exist before that complexity arrives.
Learn:
- Designing eval sets from real usage, including edge cases and adversarial inputs.
- Scoring methods: exact checks, code assertions, LLM-as-a-judge, and human review.
- Calibrating a judge model against your own labels before trusting its scores.
- Regression testing: running evals in CI on every prompt, retrieval, or model change.
- Tracing: spans for model calls, retrieval, and tools, with tokens, latency, and cost on each.
- Online signals: user feedback, escalation rates, and task completion.
Build: add an eval suite to the Stage 3 project with at least 100 cases, a judge prompt checked against a hand-labeled sample, and a CI job that fails when scores drop. Add tracing so any bad answer can be followed back to the retrieval step or the prompt.
Resources: chapters 3 and 4 of Huyen's AI Engineering, which cover evaluation methodology and evaluating AI systems, plus our guides to LLM evaluation and LLM observability.
Stage 5: AI agents and MCP#
Stage 5 covers agents: systems in which model output selects tool calls in a loop until a task is done. The skills are designing that loop, bounding it with step budgets and permissions, connecting tools through the Model Context Protocol, and recognizing when a fixed workflow would be simpler and more reliable than an agent.
Learn:
- The agent loop: model, tools, memory, planning, and stopping conditions, as covered in our explainer on AI agents.
- Workflow patterns: prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer.
- The Model Context Protocol: writing an MCP server and connecting a client to one.
- One agent framework in depth, chosen after writing a basic loop by hand; our agent framework comparison covers the options.
- Agent evals: task success rate, trajectory checks, and cost per task.
- Human approval steps for any action with side effects.
Build: an agent that completes a multi-step task, such as triaging new GitHub issues by reading, labeling, and drafting replies, through an MCP server you wrote. Give the agent a step budget and a cost cap, and require human approval before any write.
Resources: Anthropic's engineering post Building effective agents, which defines the workflow patterns above; the free Hugging Face Agents Course; and the MCP specification.
Stage 6: LLMs in production#
Stage 6 covers running AI systems for real users: gateways for routing and fallbacks, caching, cost and latency budgets, security against prompt injection, and, for teams that self-host, inference optimization. At this stage a working demo becomes a service with monitoring, budgets, and an owner who gets paged when the service fails.
Learn:
- AI gateways: routing, provider fallbacks, rate limits, key management, and budgets.
- Caching: prompt caching at the provider and semantic caching at the gateway.
- Security: prompt injection, sensitive data exposure, and least-privilege tool access.
- Latency: streaming, time to first token, and smaller models for easy requests.
- Self-hosting basics: serving engines, quantization, and KV cache behavior, covered in our guide to LLM inference optimization.
- Optional depth: fine-tuning with LoRA, and when it beats retrieval.
Build: deploy the Stage 4 or Stage 5 project behind an open source gateway such as Bifrost or LiteLLM, with a fallback provider, per-user rate limits, and a cost dashboard. Add a prompt-injection test set to the evals and fix what it finds.
Resources: the OWASP Top 10 for LLM Applications, whose 2025 list includes prompt injection, excessive agency, and unbounded consumption; and chapters 7 through 9 of Huyen's AI Engineering, on finetuning, dataset engineering, and inference optimization.
How long does it take to become an AI engineer?#
The time depends mostly on Stage 0. A working software engineer already has that base and can treat Stages 1 through 6 as a focused part-time project measured in months. Someone new to programming should plan for Stage 0 alone to take longer than the AI-specific stages combined, because general software skill is the larger body of knowledge.
Pacing by starting point#
The table below is a planning assumption for part-time study alongside a job, not a guarantee. Adjust it after the first stage, once you know your own pace.
| Starting point | Where to start | Planning assumption |
|---|---|---|
| Professional software engineer | Stage 1 | Two to four weeks per stage, one growing project |
| Data scientist or ML practitioner | Stage 0 gaps (services, testing, deployment), then Stage 1 | Faster through Stages 1 and 4, slower on Stages 0 and 6 |
| New to programming | Stage 0 | Ship a Stage 0 project before touching model APIs |
AI engineer projects for your resume#
One project that grows across all seven stages is stronger evidence than seven unrelated demos. Reviewers can see the same system gain retrieval, evals, an agent, and production controls, and they can read the tradeoffs you made at each step.
Make each project easy to assess:
- A README that states the problem, the architecture, and the measured results.
- Eval numbers before and after each major change, with the eval set published.
- A trace or two showing a failure and how you fixed it.
- Cost and latency figures per request.
- A short section on what still fails and why.
Where to go next#
When you reach Stage 4, the LLM evaluation guide goes deeper on eval design. For Stage 5, compare the options in our guide to AI agent frameworks before committing to one. To see how the seven stages map onto the real job, read what AI engineering is.
Frequently asked questions
What programming language should I learn to become an AI engineer?
- Learn Python first. Python dominates AI tooling, including model provider SDKs, retrieval libraries, evaluation tools, and most agent frameworks. TypeScript is the strong second choice, especially for engineers building web products, since the major model providers and several agent frameworks ship official TypeScript libraries. Pick one language, learn it well enough to build and test a web service, then add the other when a project needs it.
Do AI engineers need math?
- Application-focused AI engineering needs less math than model research. Basic probability and statistics matter for evaluation, because you compare pass rates and need to know when a difference is noise. Linear algebra helps you understand embeddings and similarity search. Calculus and optimization theory matter mainly if you move into training, fine-tuning, or inference optimization, which this roadmap treats as optional depth.
How to become an AI engineer with no experience?
- Start with Stage 0 of the roadmap: learn one programming language, HTTP APIs, Git, SQL, and testing, and deploy a small web service. Then work through the AI-specific stages with one project that grows over time, and publish the code, eval results, and a short write-up of what failed. Measured, public projects are the most direct way to show ability without prior job experience.
Can you become an AI engineer in 6 months?
- An experienced software engineer can cover the AI-specific stages of this roadmap in a few months of focused part-time work, because the programming base already exists. Someone starting without programming experience will usually need longer, since Stage 0 is a full discipline on its own. Hiring outcomes depend on the job market and employers, which no learning plan controls.
Do AI engineers need DSA or LeetCode?
- Data structures and algorithms are general programming skills, and they help with the integration code that makes up most AI engineering work. Whether you need LeetCode-style practice depends on the interview process at the companies you target, which varies widely. Projects that show retrieval, evaluation, and production judgment demonstrate the AI-specific skills that generic coding rounds do not test.
Terms in this guide
- Tokens
- Context window
- Structured outputs
- Embeddings
- Hybrid search
- LLM-as-a-judge
- Prompt injection
- Fine-tuning
- Foundation model
- Context rot
- Pretraining
- LLM streaming
- Rate limiting
- JSON Schema
- System prompt
- Few-shot prompting
- Approximate nearest neighbor search
- Recall@k
- Golden dataset
- Agent orchestration
- Human-in-the-loop
- Model failover
- Inference engine
Cite this guide
The AI Engineering Editors. "AI engineer roadmap: skills, projects, and resources by stage." The AI Engineering, updated Nov 11, 2026. https://theaiengineering.com/articles/ai-engineer-roadmap
- Title
- AI engineer roadmap: skills, projects, and resources by stage
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Published
- Updated