---
title: "What are guardrails in AI? LLM guardrails explained"
description: "AI guardrails are checks that validate, filter, or block LLM inputs, outputs, and tool calls. How guardrails work, where they run, and what they cannot stop."
type: "glossary-term"
canonical_url: "https://theaiengineering.com/glossary/guardrails"
markdown_url: "https://theaiengineering.com/glossary/guardrails.md"
topic: "AI gateways and LLM routing"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
updated: "2026-11-06"
keywords: ["Guardrails","AI guardrails","LLM guardrails","safety rails"]
citation: "The AI Engineering Editors. \"What are guardrails in AI? LLM guardrails explained.\" The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/guardrails"
---

# What are guardrails in AI? LLM guardrails explained

Also called AI guardrails, LLM guardrails, safety rails.

> **Definition of Guardrails:** Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.

Guardrails are controls that sit between users, a [large language model](https://theaiengineering.com/glossary/large-language-model), and the systems the model can reach, and that enforce rules the model itself cannot guarantee. A guardrail can be deterministic code, such as a regular expression for card numbers, a [JSON Schema](https://theaiengineering.com/glossary/json-schema) validator, or an allowlist of tools, or it can be a separate model trained to classify content. Guardrails exist because LLM behavior is probabilistic: a [system prompt](https://theaiengineering.com/glossary/system-prompt) shapes behavior but does not enforce it.

## How LLM guardrails work

Guardrails run at fixed checkpoints in the request path. The open-source [NVIDIA NeMo Guardrails](https://docs.nvidia.com/nemo/guardrails/latest/index.html) toolkit defines five kinds of rails: input, retrieval, dialog, execution, and output. Most production systems use a subset of these checkpoints.

| Checkpoint | What it checks | Typical action |
|---|---|---|
| Input | User messages for [prompt injection](https://theaiengineering.com/glossary/prompt-injection), off-topic requests, personal data | Block, redact, or reroute |
| Retrieval | Retrieved chunks before they enter the prompt | Drop or label untrusted content |
| Execution | [Tool call](https://theaiengineering.com/glossary/tool-calling) names and arguments | Deny, or require human approval |
| Output | Responses for toxicity, leaked secrets, schema validity | Block, rewrite, or retry |

Classifier guardrails are common for content safety. Meta's [Llama Guard](https://arxiv.org/abs/2312.06674), for example, is a Llama 2 7B model fine-tuned to label both prompts and responses against a safety risk taxonomy. Many teams run guardrails centrally in an [AI gateway](https://theaiengineering.com/glossary/ai-gateway), so every application behind the gateway gets the same policy without code changes.

## Why guardrails matter

Small failure rates become large failure counts at production volume: a model that behaves correctly on 99% of requests still produces 10,000 failures per million requests. Some of those failures, such as leaking customer data or executing a destructive tool call, cost far more than a slow response. Guardrails turn those failure modes into explicit, testable rules, and every blocked input or rewritten output becomes a signal for [LLM observability](https://theaiengineering.com/glossary/llm-observability) and a candidate test case for evaluation.

Guardrails have limits. Classifiers produce false positives that block legitimate users, and each extra check adds latency to the request. No filter reliably stops every prompt injection, which is why the [OWASP Top 10 for LLM Applications](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) pairs input and output filtering with least-privilege access and [human approval](https://theaiengineering.com/glossary/human-in-the-loop) for high-risk actions. Measuring guardrail precision and recall belongs in the same [evaluation](https://theaiengineering.com/articles/llm-evaluation) suite as model quality.

## Related terms

- [Prompt injection](https://theaiengineering.com/glossary/prompt-injection): Prompt injection is an attack in which text supplied to a large language model, typed by a user or hidden in content the model reads, alters the application's intended behavior or output.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [AI gateway](https://theaiengineering.com/glossary/ai-gateway): An AI gateway is a proxy layer between applications and model providers that exposes one API for many models and centralizes routing, failover, access control, cost tracking, caching, and logging.
- [LLM observability](https://theaiengineering.com/glossary/llm-observability): LLM observability is the practice of instrumenting LLM applications to record each request's prompts, responses, tool calls, token usage, latency, cost, and quality scores as traces that teams can search and analyze.
- [Human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop): Human-in-the-loop (HITL) is an agent design in which the run pauses at defined checkpoints, such as before a sensitive tool call, so a person can approve, edit, or reject the action.

## Guides that cover guardrails

- [What is an AI gateway? How it works and when you need one](https://theaiengineering.com/articles/what-is-an-ai-gateway): An AI gateway is a proxy between your apps and model providers. Learn how it works, its core features, AI gateway vs API gateway, and open source options.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [Bifrost vs LiteLLM: architecture, features, and performance compared](https://theaiengineering.com/articles/bifrost-vs-litellm): Bifrost vs LiteLLM compared: a Go single binary against a Python proxy, with deployment, routing, virtual keys, caching, MCP, benchmarks, and migration.
- [How to build an MCP server: a Python and TypeScript tutorial](https://theaiengineering.com/articles/how-to-build-an-mcp-server): Learn how to build an MCP server with the official Python and TypeScript SDKs: tools, resources, transports, testing, client config, and production security.
- [How to build an AI agent: a step-by-step guide in Python](https://theaiengineering.com/articles/how-to-build-an-ai-agent): Learn how to build an AI agent from scratch in Python: when you need one, tool schemas, the agent loop, stop conditions, guardrails, evals, and tracing.
- [AI coding agents: how they work and how the major tools compare](https://theaiengineering.com/articles/ai-coding-agents): AI coding agents compared: how they differ from autocomplete, the four main categories, a comparison table of major tools, and how to evaluate and secure them.
- [AI agent evaluation: metrics, graders, and how to test agents](https://theaiengineering.com/articles/ai-agent-evaluation): AI agent evaluation guide: measure task success, tool use, safety, cost, and pass^k reliability with code graders, LLM judges, eval sets from traces, and CI.
- [What is tool calling? How LLM function calling works](https://theaiengineering.com/articles/tool-calling): Tool calling lets an LLM request actions from your code through structured calls. Learn the request loop, parallel calls, structured outputs, security, and MCP.
- [What are AI agents? How agents work and when to use one](https://theaiengineering.com/articles/what-are-ai-agents): AI agents are LLM systems that call tools in a loop until a task is done. Learn how the loop works, agents vs agentic AI, examples, failure modes, and limits.

Part of the [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) topic hub.

## Cite this definition

The AI Engineering Editors. "What are guardrails in AI? LLM guardrails explained." The AI Engineering, updated Nov 06, 2026. https://theaiengineering.com/glossary/guardrails

- Title: What are guardrails in AI? LLM guardrails explained
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Updated: 2026-11-06
- URL: https://theaiengineering.com/glossary/guardrails
- Markdown: https://theaiengineering.com/glossary/guardrails.md
