---
title: "AI guardrails: types, where they run, and how to test them"
description: "AI guardrails are checks that block, redact, or flag LLM inputs, outputs, and tool calls. Learn the types, where they run, trade-offs, and how to test them."
summary: "Guardrails are the checks that run before and after model and tool calls, from regex PII filters to classifier models and LLM judges. This explainer maps guardrail types to implementations, compares libraries, cloud services, security APIs, and AI gateways, and covers testing."
type: "explainer"
canonical_url: "https://theaiengineering.com/articles/ai-guardrails"
markdown_url: "https://theaiengineering.com/articles/ai-guardrails.md"
topic: "AI gateways and LLM routing"
author: "The AI Engineering Editors"
publisher: "The AI Engineering"
published: "2026-11-18"
updated: "2026-11-18"
keywords: ["ai guardrails","llm guardrails","nemo guardrails","guardrails ai"]
citation: "The AI Engineering Editors. \"AI guardrails: types, where they run, and how to test them.\" The AI Engineering, updated Nov 18, 2026. https://theaiengineering.com/articles/ai-guardrails"
---

# AI guardrails: types, where they run, and how to test them

## TL;DR

- An AI guardrail is a check placed before or after a model or tool call that blocks, redacts, or flags content and actions that break a policy.
- Guardrails fall into three families: input checks (injection, personal data, topics), output checks (harmful content, leaks, grounding, schema validity), and action checks on tool calls (allowlists, argument validation, approval).
- Guardrails run in application code, in cloud safety services, through dedicated security APIs, or centrally in an AI gateway, and production systems usually combine several layers.
- Every guardrail trades latency, cost, and false positives against the risk it removes, and streaming forces a choice between buffering output and checking it after users see it.
- Test guardrails like any classifier: measure attack recall on red-team sets and false-positive rates on real traffic, then rerun both whenever a model, threshold, or rule changes.

AI guardrails are checks that run before or after model and tool calls and block, modify, or flag content and actions that break a policy. A guardrail can be a regex that redacts card numbers, a classifier that flags [prompt injection](https://theaiengineering.com/glossary/prompt-injection), or a rule that stops an unapproved tool call. This explainer covers guardrail types, where they run (in code, a cloud safety service, a security API, or an [AI gateway](https://theaiengineering.com/glossary/ai-gateway)), their trade-offs, and how to test them. It belongs to our [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) hub.

## What are AI guardrails?

AI [guardrails](https://theaiengineering.com/glossary/guardrails) are checks placed around a language model that validate, change, or block its inputs, outputs, and actions so an application stays within its safety, security, and policy limits.

"LLM guardrails" means the same thing. Guardrails exist because a system prompt asks a model to behave but enforces nothing; a guardrail moves the rule into code, or into a separate model whose decision the application acts on. Every guardrail has three parts:

- **A checkpoint:** where it runs, such as on the user's message, a retrieved document, a tool call's arguments, or the response.
- **A detector:** how it decides, from a regular expression or a [JSON Schema](https://theaiengineering.com/glossary/json-schema) validator to a classifier model or an [LLM judge](https://theaiengineering.com/glossary/llm-as-a-judge).
- **An action:** block, redact or rewrite, ask the model to retry, route to a person, or only log. The same PII detector that breaks a support chatbot when it blocks can keep it working when it redacts.

## Types of AI guardrails

AI guardrails fall into three families by checkpoint. Input guardrails screen what reaches the model, output guardrails screen what the model returns, and action guardrails screen what an agent tries to do with tools. Each family mixes deterministic rules, classifier models, LLM judges, and schema validation, chosen by how precise, fast, and flexible the check must be.

| Type | Example checks | Typical implementation | Typical action |
|---|---|---|---|
| Input | Prompt injection, jailbreaks | Classifier (Prompt Guard 2, Prompt Shields) | Block or flag |
| Input | Personal data, secrets | Regex, entity recognition, secret scanners | Redact or block |
| Input | Topic restrictions | Topic classifier, dialog rules, LLM judge | Fixed reply |
| Output | Harmful content | Safety classifier (Llama Guard 4) | Block or rewrite |
| Output | PII or secret leakage | Regex, entity recognition, PII filters | Redact |
| Output | Grounding, [hallucination](https://theaiengineering.com/glossary/hallucination) | Grounding classifier or LLM judge | Block, flag, regenerate |
| Output | Schema validity | JSON Schema or Pydantic validation | Reask or fail |
| Action | Tool allowlists | Rules on tool name and caller | Deny |
| Action | Argument validation | Schema checks, business rules | Deny or fix |
| Action | High-impact actions | [Human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop) approval | Pause for a person |

### Input guardrails

Input guardrails inspect text before the model sees it, including retrieved documents and tool results, which can carry instructions too. Azure's [Prompt Shields](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection) analyzes a `userPrompt` for direct attacks and up to five `documents` for hidden instructions, and can scan agent tool responses in Microsoft Foundry, in preview. Meta's [Llama Prompt Guard 2](https://theaiengineering.com/releases/2025/04#llama-prompt-guard-2) reads 512 tokens per call, so its model card advises splitting longer inputs and scanning the segments in parallel. Topic restrictions, which Amazon Bedrock Guardrails calls denied topics, keep a narrow assistant narrow.

### Output guardrails

Output guardrails inspect the response before a user or downstream system receives it. Grounding checks flag claims the retrieved passages do not support, the property the [faithfulness](https://theaiengineering.com/glossary/faithfulness) metric scores offline. Bedrock's contextual grounding check scores grounding and relevance against thresholds you set, and Azure AI Content Safety's groundedness detection is a preview that Microsoft lists for retirement on April 30, 2027. Format checks validate output against a schema and reask on a mismatch; provider [structured outputs](https://theaiengineering.com/glossary/structured-outputs) prevent most format failures, but not wrong values inside a valid structure.

### Action and tool guardrails

Action guardrails inspect the tool an agent called, its arguments, and sometimes the result. Allowlists limit each agent to the tools its task needs, argument validation enforces rules such as a refund limit, and approval pauses payments or deletions until a person agrees; our [tool calling explainer](https://theaiengineering.com/articles/tool-calling) covers the surrounding loop. The OWASP entry on [excessive agency](https://genai.owasp.org/llmrisk/llm062025-excessive-agency/) (LLM06:2025) recommends enforcing authorization in downstream systems instead of relying on the model to decide whether an action is allowed.

## Where AI guardrails run

AI guardrails run in four places: inside the application as libraries and self-hosted classifier models, in a cloud provider's safety service, through a dedicated security API, or centrally in an AI gateway that every application calls. The locations are not exclusive, and most production systems layer two or more so that one missed check is not the only defense.

**Figure 1: Guardrails can run at four layers. Checks in application code and at the gateway run on your infrastructure; guardrail services and provider filters run on vendor infrastructure.**

Four layers where guardrails run: application code, an AI gateway, external guardrail services, and the model provider's built-in filters, with the first two under your control.

Layers, top to bottom:

1. **Application**: Code you write or import (Schema checks, Tool allowlists, NeMo Guardrails, Guardrails AI)
2. **AI gateway**: Shared rules for every app (Input rules, Output rules, MCP tool rules, Redaction)
3. **Guardrail services**: Called by app or gateway (Bedrock Guardrails, Content Safety, Model Armor, Lakera Guard)
4. **Model provider**: Filters built into the API (Content filters, Safety training)

Your infrastructure spans Application through AI gateway. Vendor APIs spans Guardrail services through Model provider.

| Location | Examples (November 2026) | Strengths | Limits |
|---|---|---|---|
| Application code | NeMo Guardrails, Guardrails AI, Llama Guard 4 | Full context: user, documents, tool permissions | Repeated per service, no shared logs |
| Cloud service | Bedrock Guardrails, Azure AI Content Safety, Model Armor | Managed detectors, nothing to host | Per-check charges, language limits |
| Security API | Lakera Guard (Check Point AI Guardrails) | Attack-focused detection | Another vendor in the data path |
| AI gateway | Bifrost, LiteLLM, Kong AI Gateway | One policy for every app, redacted logs | No view of application state |

### In the application: libraries and open guard models

In-application guardrails run in your process or a sidecar you host, so they suit checks that depend on what the application knows: the user, the retrieved documents, and each tool's permissions.

NVIDIA's [NeMo Guardrails](https://docs.nvidia.com/nemo/guardrails/latest/index.html) is an Apache 2.0 Python library with five rail types: input, dialog, retrieval, execution (for tools), and output. Rails live in `config.yml` and in Colang, a dialog flow language. It runs in-process through `LLMRails` or as a server with a `/v1/chat/completions` endpoint, and its built-in rails include LLM self-checks, jailbreak detection, and NVIDIA's content safety models, with integrations for Presidio and Llama Guard. Version 0.24.1 was published on September 16, 2026. An abbreviated `config.yml`, adapted from the README and the streaming docs:

```yaml
models:
  - type: main
    engine: openai
    model: gpt-4o-mini

rails:
  input:
    flows:
      - check jailbreak
  output:
    flows:
      - self check facts
    streaming:
      enabled: True
      chunk_size: 200
      context_size: 50
      stream_first: False
```

Open guard models run beside the application. Meta's [Llama Guard 4](https://theaiengineering.com/releases/2025/04#llama-guard-4) is a 12B multimodal classifier that labels prompts and responses against 14 hazard categories based on the MLCommons taxonomy, and Prompt Guard 2 labels input as benign or malicious. IBM's [Granite Guardian](https://www.ibm.com/granite/docs/models/guardian) models (Apache 2.0) check harm, jailbreaks, and groundedness, and [Qwen3Guard](https://theaiengineering.com/releases/2025/09#qwen3guard) has a Stream variant that checks output token by token.

### Guardrails AI

[Guardrails AI](https://github.com/guardrails-ai/guardrails) is an Apache 2.0 Python framework that runs input and output guards around LLM calls and generates structured data from Pydantic models. A guard combines validators from Guardrails Hub, such as `ToxicLanguage` or `RegexMatch`, and each validator takes an on-fail action: `reask`, `fix`, `filter`, `refrain`, `noop`, `exception`, `fix_reask`, or `custom`. `guardrails start` runs it as a REST server that clients can call through the OpenAI SDK. From the README:

```python
from guardrails import Guard, OnFailAction
from guardrails_ai.competitor_check import CompetitorCheck
from guardrails_ai.toxic_language import ToxicLanguage

guard = Guard().use(
    CompetitorCheck(["Apple", "Microsoft", "Google"], on_fail=OnFailAction.EXCEPTION),
    ToxicLanguage(threshold=0.5, validation_method="sentence", on_fail=OnFailAction.EXCEPTION)
)
```

On July 6, 2026, the project announced that validators were moving to standard PyPI packages and that hosted remote inference for validators would end, with a planned cutoff of August 25. On September 9, [Harvey announced](https://www.harvey.ai/blog/guardrails-ai-joins-harvey) that it had acquired Guardrails AI, without describing plans for the open-source framework. Version 0.11.0 was published on August 14, 2026.

### Cloud provider safety services

Cloud safety services run managed detectors, so you configure policies instead of hosting models, and each can check text bound for any model.

- **Amazon Bedrock Guardrails** offers content filters (including a prompt attack filter), denied topics, word filters, PII filters, contextual grounding checks, and Automated Reasoning checks. Its [ApplyGuardrail API](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-independent-api.html) checks any text, with `source` set to `INPUT` or `OUTPUT`, without invoking a model.
- **Azure AI Content Safety** scores text and images for hate, sexual content, violence, and self-harm by severity and detects prompt and document attacks with Prompt Shields, inside Microsoft Foundry or through standalone APIs.
- **Google Cloud Model Armor** screens for injection and jailbreaks, sensitive data, malicious URLs, and harmful content, for AI deployed on Google Cloud or elsewhere. Its [overview](https://docs.cloud.google.com/security-command-center/docs/model-armor-overview) notes that it inspects each prompt and response as a single-turn request, without conversation history.

### Dedicated security APIs

Dedicated security APIs come from vendors that specialize in AI attacks. Lakera Guard is one example. Check Point acquired Lakera and now documents the product as Check Point AI Guardrails, served from `api.lakera.ai`. One [`POST /v2/guard` call](https://docs.lakera.ai/docs/api/guard) takes the conversation in OpenAI chat format and screens the latest user message as input, the latest assistant message as output, and tool messages and tool calls as untrusted content and agent actions. Our [guardrails and AI security directory](https://theaiengineering.com/platforms/guardrails-security) lists more vendors.

### At an AI gateway

A gateway runs guardrails once for every application and agent behind it, so a new service inherits the organization's PII, secret, and content policies without code changes, including on tool calls it proxies to MCP servers.

[Bifrost](https://docs.getbifrost.ai/enterprise/guardrails), the open-source AI gateway from Maxim AI, includes guardrails in Bifrost Enterprise, built from two objects:

- **Rules** decide when a check runs: a CEL expression over metadata such as model, virtual key, team, or user; a phase (`input`, `output`, or `both`); a sampling rate; and a target, LLM calls or MCP tool executions. The expression `mcp_client == "github" && mcp_tool == "create_issue"` guards one tool.
- **Profiles** decide how content is checked. Bifrost runs [Secrets Detection](https://docs.getbifrost.ai/enterprise/guardrails/secrets-detection) (Gitleaks rules), Custom Regex with a PII template, and [Prompt Guardrails](https://docs.getbifrost.ai/enterprise/guardrails/prompt-guardrails), an LLM judge. External profiles call AWS Bedrock Guardrails, Azure Content Safety, Google Model Armor, Microsoft Presidio, Azure AI Language PII, CrowdStrike AIDR, Gray Swan Cygnal, Patronus AI, Check Point's AI Agent Security (the Lakera Guard API), Repello Argus, Singulr AI, Noma Security, and A10 Guardrails.

A rule's profiles run in order, and the documentation recommends a cheap local profile, such as Custom Regex, before external ones. A block stops the remaining profiles and returns HTTP 446 with a `guardrail_violation` error; a provider error or timeout fails open.

**Figure 2: A gateway checks the prompt before the provider receives it and the response before the application does; a block at either step ends the request.**

Sequence: an application sends a request to an AI gateway, which checks the prompt with a guardrail API, calls the model provider, and checks the response before returning it.

Steps:

1. Application → AI gateway: user request
2. AI gateway → Guardrail API: input check
3. Guardrail API → AI gateway: allow or redact
4. AI gateway → Model provider: checked prompt
5. Model provider → AI gateway: response
6. AI gateway → Guardrail API: output check
7. Guardrail API → AI gateway: allow, redact, block
8. AI gateway → Application: response or error

Bifrost's [redaction modes](https://docs.getbifrost.ai/enterprise/guardrails/redaction) set where a rewrite applies: `runtime` redacts live traffic and logs, `logs_only` forwards the original but stores placeholders in Bifrost logs and trace exports such as OpenTelemetry, and `runtime_reversible` uses placeholders that a permitted user can reveal in Bifrost logs; the reveal mapping is not exported. Logs need this care because they outlive requests. AWS notes that content blocked by Bedrock Guardrails appears as plain text in model invocation logs when those are enabled. Our guide to [LLM observability](https://theaiengineering.com/articles/llm-observability) covers keeping prompts private in traces.

Other gateways in our [AI gateways directory](https://theaiengineering.com/platforms/ai-gateways), including LiteLLM, Kong AI Gateway, agentgateway, and Prisma AIRS AI Gateway (formerly Portkey), also run guardrail checks.

## Trade-offs of AI guardrails

Every guardrail spends latency, money, and some legitimate traffic to reduce a risk. The costs compound because checks run on both input and output, often several per phase, and streaming forces a choice between delaying text and checking it after users see it. Weigh each check against the failure it prevents before enabling it everywhere.

### Latency and cost

An input check delays the first token by its full duration, and sequential checks add up. Classifiers are fast; Meta's model card lists 19.3 ms per 512-token call for Prompt Guard 2 22M and 92.4 ms for 86M on an A100 GPU. LLM judges cost the most because each is a full model call; Bifrost's documentation flags the added latency and token cost of its judge calls and recommends sampling rules on high-traffic endpoints. Run cheap checks first, run independent ones in parallel, and set a timeout on each external call, then decide whether a timeout fails open (traffic passes unchecked) or closed (users get errors). For payment or deletion tools, closed is usually safer.

Services charge per check. On November 16, 2026, the [Amazon Bedrock pricing page](https://aws.amazon.com/bedrock/pricing/) listed content filters and denied topics at $0.15 per 1,000 text units each and sensitive information filters at $0.10, where a text unit holds up to 1,000 characters. Each policy is charged separately, so three policies on a prompt and its response are each charged twice.

### False positives and false negatives

Every detector has a threshold, and moving it trades a blocked legitimate user (a false positive) for a missed attack (a false negative). Google recommends Model Armor's `High` confidence level for production apps that prioritize uninterrupted interactions, and suggests its lowest level, with caution, for high-stakes categories such as prompt injection. Placement also shifts errors. Meta's Llama Guard 4 card reports that in internal tests, input filtering raised the refusal rate more than output filtering did. The Prompt Guard 2 card reports 97.5% recall at a 1% false-positive rate for the 86M model on Meta's English evaluation set, and warns that adversaries may build attacks to bypass detection, so measure on your own traffic.

### Streaming responses

Streaming changes only output guardrails. A guardrail can buffer and check each chunk before release, which delays text, or stream immediately and check in the background, which lets some violating text reach the user before the stream stops.

| Product | Check before release | Stream first, check after | Note |
|---|---|---|---|
| Amazon Bedrock Guardrails | Synchronous mode (default) | Asynchronous `streamProcessingMode` | Async mode cannot mask PII |
| Azure OpenAI in Foundry | Default, in buffered chunks | Asynchronous Filter | Async signal within about 1,000 characters |
| NeMo Guardrails | `stream_first: False` | `stream_first: True` (default) | Checks `chunk_size` tokens at a time |
| Bifrost | Holds the stream if a rule can block | Detect-only and monitor rules | Redaction releases text by segment |

Microsoft's [content streaming documentation](https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/content-streaming) adds that even in default mode, the last segment can be returned before the policy is applied. In its test, a blocklisted term in the final 15 to 30 words reached the client.

### Multilingual coverage

Accuracy usually drops outside the languages a detector was tested on. As of November 2026, Bedrock Guardrails' Classic tier supports English, French, and Spanish; its Standard tier supports more. Azure AI Content Safety's models were trained and tested on eight languages, but groundedness and protected material detection are English only. Model Armor's safety and injection filters were tested on nine languages and Prompt Guard 2 on eight, with a multilingual AUC of 0.942 for the 22M model against 0.995 for 86M. Model Armor also does not decode Base64, hexadecimal, or URL encoding, so encoded payloads need their own check.

## How to test AI guardrails

Test guardrails the way you test any classifier: measure how often they catch what they should and how often they block what they should allow. That takes a red-team set of attacks, a labeled sample of real traffic, a shadow period before enforcement, and a regression suite that reruns whenever a model, threshold, rule, or guardrail version changes.

Our guide to [LLM evaluation](https://theaiengineering.com/articles/llm-evaluation) covers the datasets, graders, and CI gates these tests share with quality evals.

### Build the test sets

1. **Attack set:** prompt injections, jailbreaks, exfiltration attempts, and harmful requests for your application, including injections planted in documents and tool results. Open-source scanners such as NVIDIA's garak, Microsoft's PyRIT, and promptfoo generate attacks automatically.
2. **Benign set:** labeled production traffic with personal data masked, plus hard negatives such as security questions from real users, medical language, and code with secret-like strings. A [golden dataset](https://theaiengineering.com/glossary/golden-dataset) of these cases catches over-blocking.
3. **Expected actions:** label each case allow, redact, block, or approve, so the test checks the action as well as the detection.

### Measure the right rates

| Metric | Definition | What it tells you |
|---|---|---|
| Attack recall | Share of attack cases blocked or flagged | How much the check catches |
| False-positive rate | Share of benign cases blocked or flagged | How much legitimate traffic it costs |
| Live block rate | Share of production requests blocked, by rule | Drift and incidents after release |
| Added latency | p50 and p95 time spent in checks | Fit with the latency budget |
| Redaction accuracy | Planted PII removed, other text intact | Over- or under-matching |

Report each rate by category and language, because a low overall false-positive rate can hide a high one for a single language or customer segment.

### Shadow mode and regression suites

Run a new guardrail in a mode that records decisions without acting, then compare findings with labeled traffic: Bifrost's monitor mode logs what output guardrails would have done, Model Armor offers inspect-only enforcement, and Check Point recommends starting in detect mode and raising sensitivity gradually. After launch, rerun both sets on every model, threshold, or policy change, fail the build when recall or the false-positive rate moves past a limit, and turn each incident into a test case.

### Map guardrails to the OWASP Top 10 for LLM Applications

The [OWASP Top 10 for LLM Applications](https://genai.owasp.org/llm-top-10/) 2025 is the current edition, and it works as a coverage checklist for runtime guardrails and their tests.

| OWASP 2025 risk | Guardrails that address it | What to test |
|---|---|---|
| LLM01 Prompt Injection | Injection classifiers on input, documents, tool results | Direct and indirect attacks |
| LLM02 Sensitive Information Disclosure | PII and secret redaction in traffic and logs | Planted PII and keys |
| LLM05 Improper Output Handling | Schema validation, output encoding, URL checks | Malformed JSON, malicious links |
| LLM06 Excessive Agency | Tool allowlists, argument checks, approval | Unauthorized tool calls |
| LLM07 System Prompt Leakage | Output checks; no secrets in prompts | Extraction attempts |
| LLM09 Misinformation | Grounding checks against sources | Unsupported claims |
| LLM10 Unbounded Consumption | Gateway rate limits and token budgets | Oversized inputs, request floods |

LLM03, LLM04, and LLM08 (supply chain, data and model poisoning, and vector and embedding weaknesses) depend mostly on controls outside the request path. OWASP's [prompt injection entry](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) states that it is unclear whether fool-proof prevention exists, so it pairs filtering with least privilege, human approval, and adversarial testing. For agents, OWASP also published a [Top 10 for Agentic Applications for 2026](https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/), dated December 9, 2025.

Where to go next: our explainer on [what an AI gateway is](https://theaiengineering.com/articles/what-is-an-ai-gateway) covers the layer where shared guardrails usually live.

## Frequently asked questions

### What is NeMo Guardrails?

NeMo Guardrails is NVIDIA's Apache 2.0 Python library for adding programmable guardrails to LLM applications. It defines input, dialog, retrieval, execution, and output rails in YAML and the Colang language, calls safety models and detection APIs, and runs in-process or as a server with a chat completions endpoint. Version 0.24.1 was published on September 16, 2026.

### What is Guardrails AI?

Guardrails AI is an Apache 2.0 Python framework that wraps LLM calls in input and output guards built from validators, and that generates structured data from Pydantic models. In July 2026, the project announced that its validators were moving to standard PyPI packages. Harvey announced its acquisition of Guardrails AI on September 9, 2026, without describing plans for the open-source framework.

### What is the difference between NeMo Guardrails and Guardrails AI?

NeMo Guardrails controls a whole conversation, with rails at five checkpoints, dialog flows written in Colang, and calls to safety models such as Llama Guard. Guardrails AI validates individual inputs and outputs with validators and on-fail actions such as reask, fix, or exception. Both are Apache 2.0 Python libraries, and NeMo Guardrails lists Guardrails AI validators among its integrations.

### Can guardrails stop prompt injection?

No guardrail stops every prompt injection. Classifiers such as Llama Prompt Guard 2, Azure Prompt Shields, and Google Model Armor catch many known attack patterns, but OWASP says it is unclear whether fool-proof prevention exists. Pair detection with least-privilege tools, output validation, and human approval for high-impact actions, so a missed injection does limited damage.

### Do guardrails work with streaming responses?

Yes, with a trade-off. Input checks finish before streaming starts. For output, a guardrail either buffers and checks chunks before release, which delays text, or streams first and checks in the background, which can let some violating text through before the stream stops. Amazon Bedrock Guardrails, Azure OpenAI, and NeMo Guardrails document both modes.

### Should guardrails run in the application or in an AI gateway?

Run checks that need application context in the application, such as validating a tool's arguments against the current user's permissions. Run organization-wide policies, such as PII redaction, secret detection, and content safety, in an AI gateway, so every app and agent gets the same rules and logs without code changes. Most production systems use both.

## Terms in this guide

- [Guardrails](https://theaiengineering.com/glossary/guardrails): Guardrails are programmatic checks placed around a large language model that validate, filter, or block inputs, outputs, and tool calls so an AI application stays within its safety, security, and policy limits.
- [Prompt injection](https://theaiengineering.com/glossary/prompt-injection): Prompt injection is an attack in which text supplied to a large language model, typed by a user or hidden in content the model reads, alters the application's intended behavior or output.
- [AI gateway](https://theaiengineering.com/glossary/ai-gateway): An AI gateway is a proxy layer between applications and model providers that exposes one API for many models and centralizes routing, failover, access control, cost tracking, caching, and logging.
- [JSON Schema](https://theaiengineering.com/glossary/json-schema): JSON Schema is a declarative vocabulary, itself written in JSON, for describing the structure of JSON data; LLM APIs and MCP use it to define tool arguments and constrain model output.
- [LLM-as-a-judge](https://theaiengineering.com/glossary/llm-as-a-judge): LLM-as-a-judge is an evaluation method in which a language model scores or compares the outputs of an LLM application against a rubric, a reference answer, or another response, replacing or supplementing human graders.
- [Hallucination](https://theaiengineering.com/glossary/hallucination): A hallucination is output from a language model that is fluent and plausible but factually false, unsupported by the provided sources, or invented outright, such as a fabricated citation or package name.
- [Human-in-the-loop](https://theaiengineering.com/glossary/human-in-the-loop): Human-in-the-loop (HITL) is an agent design in which the run pauses at defined checkpoints, such as before a sensitive tool call, so a person can approve, edit, or reject the action.
- [Faithfulness](https://theaiengineering.com/glossary/faithfulness): Faithfulness is an evaluation metric that measures what share of the claims in a generated answer can be inferred from the context the model was given, usually scored from 0 to 1.
- [Structured outputs](https://theaiengineering.com/glossary/structured-outputs): Structured outputs is an LLM API feature that constrains generation so the model's response always conforms to a developer-supplied JSON Schema, producing output that code can parse without retries.
- [Golden dataset](https://theaiengineering.com/glossary/golden-dataset): A golden dataset is a curated, versioned set of representative inputs, each paired with an expected output or grading criteria, that a team treats as ground truth when evaluating an LLM application.

## Related guides

- [What is an AI gateway? How it works and when you need one](https://theaiengineering.com/articles/what-is-an-ai-gateway): An AI gateway is a proxy between your apps and model providers. Learn how it works, its core features, AI gateway vs API gateway, and open source options.
- [LLM evaluation: a practical guide to evals, metrics, and judges](https://theaiengineering.com/articles/llm-evaluation): LLM evaluation for engineers: build eval datasets, pick metrics, calibrate LLM-as-a-judge, evaluate RAG and agents, and gate every release in CI.
- [LLM observability: what to capture, which metrics, and tools](https://theaiengineering.com/articles/llm-observability): LLM observability guide: what to capture in traces and spans, metrics to track, OpenTelemetry GenAI conventions, tool categories, and keeping prompts private.
- [LLM routing: strategies, fallbacks, and learned routers](https://theaiengineering.com/articles/llm-routing): LLM routing guide: rule-based and cost-aware strategies, fallbacks, load balancing, semantic and learned routers like RouteLLM, router evals, and pitfalls.

Part of the [AI gateways and LLM routing](https://theaiengineering.com/topics/ai-gateways) topic hub.

## Cite this guide

The AI Engineering Editors. "AI guardrails: types, where they run, and how to test them." The AI Engineering, updated Nov 18, 2026. https://theaiengineering.com/articles/ai-guardrails

- Title: AI guardrails: types, where they run, and how to test them
- Author: The AI Engineering Editors
- Publisher: The AI Engineering
- Published: 2026-11-18
- Updated: 2026-11-18
- URL: https://theaiengineering.com/articles/ai-guardrails
- Markdown: https://theaiengineering.com/articles/ai-guardrails.md

To cite one section, link to its anchor:

- [What are AI guardrails?](https://theaiengineering.com/articles/ai-guardrails#what-are-ai-guardrails)
- [Types of AI guardrails](https://theaiengineering.com/articles/ai-guardrails#types-of-ai-guardrails)
- [Where AI guardrails run](https://theaiengineering.com/articles/ai-guardrails#where-ai-guardrails-run)
- [Trade-offs of AI guardrails](https://theaiengineering.com/articles/ai-guardrails#trade-offs-of-ai-guardrails)
- [How to test AI guardrails](https://theaiengineering.com/articles/ai-guardrails#how-to-test-ai-guardrails)
- [Frequently asked questions](https://theaiengineering.com/articles/ai-guardrails#faq)
