What are guardrails in AI? LLM guardrails explained

Also called AI guardrails, LLM guardrails, safety rails

Definition

Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.

Guardrails are controls that sit between users, a large language model, and the systems the model can reach, and that enforce rules the model itself cannot guarantee. A guardrail can be deterministic code, such as a regular expression for card numbers, a JSON Schema validator, or an allowlist of tools, or it can be a separate model trained to classify content. Guardrails exist because LLM behavior is probabilistic: a system prompt shapes behavior but does not enforce it.

How LLM guardrails work#

Guardrails run at fixed checkpoints in the request path. The open-source NVIDIA NeMo Guardrails toolkit defines five kinds of rails: input, retrieval, dialog, execution, and output. Most production systems use a subset of these checkpoints.

Checkpoint What it checks Typical action
Input User messages for prompt injection, off-topic requests, personal data Block, redact, or reroute
Retrieval Retrieved chunks before they enter the prompt Drop or label untrusted content
Execution Tool call names and arguments Deny, or require human approval
Output Responses for toxicity, leaked secrets, schema validity Block, rewrite, or retry

Classifier guardrails are common for content safety. Meta's Llama Guard, for example, is a Llama 2 7B model fine-tuned to label both prompts and responses against a safety risk taxonomy. Many teams run guardrails centrally in an AI gateway, so every application behind the gateway gets the same policy without code changes.

Why guardrails matter#

Small failure rates become large failure counts at production volume: a model that behaves correctly on 99% of requests still produces 10,000 failures per million requests. Some of those failures, such as leaking customer data or executing a destructive tool call, cost far more than a slow response. Guardrails turn those failure modes into explicit, testable rules, and every blocked input or rewritten output becomes a signal for LLM observability and a candidate test case for evaluation.

Guardrails have limits. Classifiers produce false positives that block legitimate users, and each extra check adds latency to the request. No filter reliably stops every prompt injection, which is why the OWASP Top 10 for LLM Applications pairs input and output filtering with least-privilege access and human approval for high-risk actions. Measuring guardrail precision and recall belongs in the same evaluation suite as model quality.

Cite this definition

The AI Engineering Editors. "What are guardrails in AI? LLM guardrails explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/guardrails

Title
What are guardrails in AI? LLM guardrails explained
Author
The AI Engineering Editors
Publisher
The AI Engineering
Updated
View as Markdown

Part of the AI gateways and LLM routing hub · Updated