What are guardrails in AI? LLM guardrails explained
Also called AI guardrails, LLM guardrails, safety rails
Definition
Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.
Guardrails are controls that sit between users, a large language model, and the systems the model can reach, and that enforce rules the model itself cannot guarantee. A guardrail can be deterministic code, such as a regular expression for card numbers, a JSON Schema validator, or an allowlist of tools, or it can be a separate model trained to classify content. Guardrails exist because LLM behavior is probabilistic: a system prompt shapes behavior but does not enforce it.
How LLM guardrails work#
Guardrails run at fixed checkpoints in the request path. The open-source NVIDIA NeMo Guardrails toolkit defines five kinds of rails: input, retrieval, dialog, execution, and output. Most production systems use a subset of these checkpoints.
| Checkpoint | What it checks | Typical action |
|---|---|---|
| Input | User messages for prompt injection, off-topic requests, personal data | Block, redact, or reroute |
| Retrieval | Retrieved chunks before they enter the prompt | Drop or label untrusted content |
| Execution | Tool call names and arguments | Deny, or require human approval |
| Output | Responses for toxicity, leaked secrets, schema validity | Block, rewrite, or retry |
Classifier guardrails are common for content safety. Meta's Llama Guard, for example, is a Llama 2 7B model fine-tuned to label both prompts and responses against a safety risk taxonomy. Many teams run guardrails centrally in an AI gateway, so every application behind the gateway gets the same policy without code changes.
Why guardrails matter#
Small failure rates become large failure counts at production volume: a model that behaves correctly on 99% of requests still produces 10,000 failures per million requests. Some of those failures, such as leaking customer data or executing a destructive tool call, cost far more than a slow response. Guardrails turn those failure modes into explicit, testable rules, and every blocked input or rewritten output becomes a signal for LLM observability and a candidate test case for evaluation.
Guardrails have limits. Classifiers produce false positives that block legitimate users, and each extra check adds latency to the request. No filter reliably stops every prompt injection, which is why the OWASP Top 10 for LLM Applications pairs input and output filtering with least-privilege access and human approval for high-risk actions. Measuring guardrail precision and recall belongs in the same evaluation suite as model quality.
Related terms
Guides that cover guardrails
- What is an AI gateway? How it works and when you need one
- LLM evaluation: a practical guide to evals, metrics, and judges
- Bifrost vs LiteLLM: architecture, features, and performance compared
- How to build an MCP server: a Python and TypeScript tutorial
- How to build an AI agent: a step-by-step guide in Python
- AI coding agents: how they work and how the major tools compare
- AI agent evaluation: metrics, graders, and how to test agents
- What is tool calling? How LLM function calling works
- What are AI agents? How agents work and when to use one
Cite this definition
The AI Engineering Editors. "What are guardrails in AI? LLM guardrails explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/guardrails
- Title
- What are guardrails in AI? LLM guardrails explained
- Author
- The AI Engineering Editors
- Publisher
- The AI Engineering
- Updated