Guardrails and AI security

LLM guardrails are checks that inspect the inputs and outputs of language model applications and block, rewrite, or flag content that breaks a policy, such as prompt injection, leaked personal data, harmful text, or unsafe tool calls. AI security tools extend the same idea with red teaming, model scanning, and controls over how agents and employees use AI.

The category has four groups. Open-source libraries and safety models, including NVIDIA NeMo Guardrails, Guardrails AI, Llama Guard, and IBM Granite Guardian, run inside your application. Red teaming tools such as garak, PyRIT, promptfoo, and DeepTeam attack an application before release. Cloud services such as Amazon Bedrock Guardrails, Azure AI Content Safety, and Google Cloud Model Armor apply checks through an API. Commercial platforms such as Lakera, HiddenLayer, and Prisma AIRS combine runtime protection, testing, and governance.

Several early AI security startups now belong to larger vendors: Cisco bought Robust Intelligence, Palo Alto Networks bought Protect AI, Check Point bought Lakera, F5 bought CalypsoAI, and SentinelOne bought Prompt Security.

Guardrails and AI security compared

PlatformOpen sourceDeploymentBest for
Akamai Firewall for AINoManaged cloud, APIAkamai customers adding edge protection for AI apps
Amazon Bedrock GuardrailsNoManaged cloud, APIAWS teams applying one policy across models
Azure AI Content Safety and Prompt ShieldsNoManaged cloud, APIAzure and Foundry applications
Cisco AI DefenseNoManaged cloudCisco security customers governing enterprise AI
Cloudflare AI Security for Apps (formerly Firewall for AI)NoManaged cloudAI apps already behind Cloudflare
DeepTeamYes (Apache-2.0)LibraryRed teaming alongside DeepEval tests
F5 AI Guardrails and AI Red Team (formerly CalypsoAI)NoManaged cloudF5 customers adding AI runtime security
garakYes (Apache-2.0)CLI, LibraryScanning models for known failure modes
GiskardYes (Apache-2.0)Library, Managed cloudContinuous red teaming of agents
Google Cloud Model ArmorNoManaged cloud, APIGoogle Cloud teams securing models and agents
gpt-oss-safeguardYes (Apache-2.0)Local, Self-hostedCustom, policy-driven content classification
Guardrails AIYes (Apache-2.0)Library, Self-hostedValidating LLM outputs in Python code
HiddenLayerNoManaged cloudModel supply-chain scanning plus runtime defense
Highflame (formerly Javelin)NoManaged cloud, Self-hostedIdentity and authorization for agent actions
IBM Granite GuardianYes (Apache-2.0)Local, Self-hostedPermissively licensed risk classifiers for RAG and agents
LakeraNoManaged cloud, Self-hosted, APIRuntime prompt-injection defense for production apps
LassoNoManaged cloudVisibility into enterprise generative AI use
Llama Guard and Prompt GuardNoLocal, Self-hostedSelf-hosted content and injection classifiers
LlamaFirewallYes (MIT)LibraryLayered checks for agents that call tools
MindgardNoManaged cloudAutomated red teaming of deployed AI systems
Noma SecurityNoManaged cloudSecurity teams governing agent sprawl
NVIDIA NeMo GuardrailsYes (Apache-2.0)Library, Self-hostedProgrammable dialog and safety rails for chat apps
Pillar SecurityNoManaged cloudInventory and governance of internal AI agents
Prisma AIRSNoManaged cloud, APIPalo Alto Networks customers securing AI end to end
Prompt SecurityNoManaged cloudGoverning employee and developer AI tool use
promptfoo red teamingYes (MIT)CLI, Managed cloudApplication-specific attack generation in CI
PyRITYes (MIT)Library, CLIScripted multi-turn red teaming
Snyk Agent Scan (formerly MCP-Scan)Yes (Apache-2.0)CLIAuditing MCP servers and agent skills
StraikerNoManaged cloudRuntime detection for custom AI agents
WitnessAINoManaged cloudNetwork-level control of employee AI use
ZenityNoManaged cloudPolicies for agents built by business users

Profiles

  • Akamai Firewall for AI

    Akamai · akamai.com

    Akamai Firewall for AI inspects prompts and responses at the Akamai edge or through a REST API, blocking prompt injection, jailbreaks, sensitive data exfiltration, and toxic output for AI applications hosted on premises, in the cloud, or in hybrid environments.

    • Managed cloud
    • API
  • Amazon Bedrock Guardrails logo

    Amazon Bedrock Guardrails

    Amazon Web Services · aws.amazon.com

    Amazon Bedrock Guardrails applies configurable safeguards to model inputs and outputs, including content and word filters, prompt attack detection, denied topics, PII redaction, and contextual grounding and Automated Reasoning checks, and works with any model through the ApplyGuardrail API.

    • Managed cloud
    • API
  • Azure AI Content Safety and Prompt Shields logo

    Azure AI Content Safety and Prompt Shields

    Microsoft · learn.microsoft.com

    Prompt Shields is a feature of Azure AI Content Safety, which Microsoft now presents as Content Safety in Foundry Control Plane, that detects direct prompt attacks in user input and indirect attacks hidden in documents, alongside the service's harmful-content filters for text and images.

    • Managed cloud
    • API
  • Cisco AI Defense

    Cisco · cisco.com

    Cisco AI Defense discovers AI workloads, models, data, and users across cloud environments, validates models with algorithmic red teaming, and enforces runtime guardrails at the network level without agents or libraries; it builds on Cisco's acquisition of Robust Intelligence.

    • Managed cloud
  • Cloudflare AI Security for Apps (formerly Firewall for AI) logo

    Cloudflare AI Security for Apps (formerly Firewall for AI)

    Cloudflare · developers.cloudflare.com

    Cloudflare AI Security for Apps is a detection in Cloudflare's web application firewall that flags prompt injection, PII, and unsafe topics in traffic to AI applications proxied by Cloudflare, so teams can log or block those requests with WAF rules.

    Cloudflare renamed Firewall for AI to AI Security for Apps; the old documentation path now redirects to the new name.

    • Managed cloud
  • DeepTeam logo

    DeepTeam

    Confident AI · trydeepteam.com

    DeepTeam is an Apache-2.0 Python framework from Confident AI for red teaming LLM applications and agents with adversarial attacks, agentic vulnerability tests, guardrails, and checks mapped to compliance frameworks.

    • Open source · Apache-2.0
    • Library
    • Python
  • F5 AI Guardrails and AI Red Team (formerly CalypsoAI) logo

    F5 AI Guardrails and AI Red Team (formerly CalypsoAI)

    F5 · f5.com

    F5 AI Guardrails applies runtime security controls to AI interactions against model- and agent-specific risks, and F5 AI Red Team uses autonomous agents to simulate attack patterns against AI systems; both come from CalypsoAI.

    F5 closed its acquisition of CalypsoAI on September 26, 2025, and sells its products as F5 AI Guardrails and F5 AI Red Team.

    • Managed cloud
  • garak logo

    garak

    NVIDIA · garak.ai

    garak is NVIDIA's Apache-2.0 LLM vulnerability scanner that probes models and LLM applications for hallucination, data leakage, prompt injection, misinformation, toxic output, and jailbreaks, using static, dynamic, and adaptive probes.

    • Open source · Apache-2.0
    • CLI
    • Library
    • Python
  • Giskard logo

    Giskard

    giskard.ai

    Giskard offers continuous red teaming for AI agents through a commercial platform, alongside an Apache-2.0 open-source library for evaluations, red teaming, and test generation for agentic systems.

    • Open source · Apache-2.0
    • Library
    • Managed cloud
    • Python
  • Google Cloud Model Armor logo

    Google Cloud Model Armor

    Google Cloud · cloud.google.com

    Model Armor is a Google Cloud service that screens prompts and responses for prompt injection and jailbreaks, sensitive data leaks, malicious URLs, and harmful content, with inline integrations for Gemini, Google MCP servers, Apigee, Firebase, and LangChain.

    • Managed cloud
    • API
  • gpt-oss-safeguard logo

    gpt-oss-safeguard

    OpenAI · openai.com

    gpt-oss-safeguard is a pair of Apache-2.0 open-weight reasoning models from OpenAI that classify content against a policy the developer supplies at inference time, so teams can change safety rules without retraining a classifier.

    • Open source · Apache-2.0
    • Local
    • Self-hosted
  • Guardrails AI logo

    Guardrails AI

    Harvey · guardrailsai.com

    Guardrails AI is an Apache-2.0 Python framework that wraps LLM calls with validators for structured output, PII, toxicity, and other risks and can run as a standalone server; in July 2026 its validators began moving to standard PyPI packages.

    Harvey announced its acquisition of Guardrails AI on September 9, 2026, and Guardrails discontinued its hosted remote inference for validators.

    • Open source · Apache-2.0
    • Library
    • Self-hosted
    • Python
  • HiddenLayer logo

    HiddenLayer

    hiddenlayer.com

    HiddenLayer is an AI security platform that scans models for malicious code, backdoored weights, and vulnerable dependencies, runs continuous red teaming, detects attacks on AI systems at runtime, and protects AI coding agents.

    • Managed cloud
  • Highflame (formerly Javelin) logo

    Highflame (formerly Javelin)

    highflame.com

    Highflame, the company formerly known as Javelin, is an AI agent security platform that gives each agent a verifiable identity and makes an authorization decision for every action, enforced inline across LLM and MCP traffic through its gateway.

    Javelin, which built an AI gateway, now operates as Highflame and publishes its SDKs under the highflame-ai GitHub organization.

    • Managed cloud
    • Self-hosted
  • IBM Granite Guardian

    IBM · ibm.com

    IBM Granite Guardian is a family of Apache-2.0 open-weight models and LoRA adapters that detect risks in prompts and responses, including harmful content, jailbreaks, RAG groundedness problems, and function-calling hallucinations, with support for custom criteria.

    • Open source · Apache-2.0
    • Local
    • Self-hosted
  • Lakera logo

    Lakera

    Check Point Software Technologies · lakera.ai

    Lakera is an AI security platform with runtime protection for AI agents and applications against prompt injection, data leakage, and harmful content, controls for employee AI use, and Lakera Red for AI red teaming.

    Check Point Software Technologies completed its acquisition of Lakera after announcing the deal in September 2025.

    • Managed cloud
    • Self-hosted
    • API
  • Lasso logo

    Lasso

    Lasso Security · lasso.security

    Lasso is an AI security platform that gives enterprises visibility, control, and real-time protection across AI models, agents, and applications, including employee use of generative AI tools.

    • Managed cloud
  • Llama Guard and Prompt Guard logo

    Llama Guard and Prompt Guard

    Meta · dev.meta.ai

    Llama Guard and Prompt Guard are Meta's open-weight safety models released under Llama community licenses: Llama Guard classifies prompts and responses against harmful-content categories, and Prompt Guard detects prompt injection and jailbreak attempts in inputs.

    • Local
    • Self-hosted
  • LlamaFirewall logo

    LlamaFirewall

    Meta · meta-llama.github.io

    LlamaFirewall is Meta's MIT-licensed guardrail framework for AI agents that orchestrates guard models such as Prompt Guard together with other scanners, including CodeShield for insecure generated code, to secure AI agents and the tools they call.

    • Open source · MIT
    • Library
    • Python
  • Mindgard logo

    Mindgard

    mindgard.ai

    Mindgard is an AI security platform that discovers AI models, agents, and applications in use and runs attacker-aligned automated red teaming against them to find vulnerabilities.

    • Managed cloud
  • Noma Security logo

    Noma Security

    noma.security

    Noma Security is an agent security and governance platform for enterprise security teams that want to adopt AI agents and AI applications without losing visibility or control over what they do.

    • Managed cloud
  • NVIDIA NeMo Guardrails logo

    NVIDIA NeMo Guardrails

    NVIDIA · docs.nvidia.com

    NeMo Guardrails is NVIDIA's Apache-2.0 Python toolkit for adding programmable rails to LLM applications, defining input, output, dialog, retrieval, and execution checks in configuration and the Colang language, with protections against jailbreaks and prompt injection.

    • Open source · Apache-2.0
    • Library
    • Self-hosted
    • Python
  • Pillar Security logo

    Pillar Security

    pillar.security

    Pillar Security is a commercial platform for discovering, governing, and securing the AI agents and AI-driven workflows that teams build or adopt across an organization, managed from a single place.

    • Managed cloud
  • Prisma AIRS logo

    Prisma AIRS

    Palo Alto Networks · paloaltonetworks.com

    Prisma AIRS is Palo Alto Networks' AI security platform for discovering, assessing, protecting, and governing AI apps, agents, and models, including the AI security products of Protect AI, which Palo Alto acquired, and the Portkey-based AI gateway.

    • Managed cloud
    • API
  • Prompt Security logo

    Prompt Security

    SentinelOne · sentinelone.com

    Prompt Security, now part of the SentinelOne Singularity Platform, protects how employees, developers, and applications use AI, covering shadow AI discovery, AI code assistants, a runtime firewall for homegrown apps, and permissions for agents and MCP servers.

    SentinelOne acquired Prompt Security, and prompt.security now hosts research content while the product is sold through SentinelOne.

    • Managed cloud
  • promptfoo red teaming logo

    promptfoo red teaming

    promptfoo · promptfoo.dev

    promptfoo's red teaming mode generates adaptive attacks tailored to a specific application, detecting prompt injections, jailbreaks, and PII leaks, and runs them from the MIT-licensed CLI or the commercial platform.

    OpenAI agreed to acquire Promptfoo in March 2026 and said it would keep developing the MIT-licensed open-source project.

    • Open source · MIT
    • CLI
    • Managed cloud
    • TypeScript
  • PyRIT logo

    PyRIT

    Microsoft · microsoft.github.io

    PyRIT is an MIT-licensed toolkit from the Microsoft AI Red Team for finding security and safety weaknesses in generative AI systems through automated scans, adaptive multi-turn attacks, and human-led testing, with a CLI, a graphical interface, and a Python framework.

    • Open source · MIT
    • Library
    • CLI
    • Python
  • Snyk Agent Scan (formerly MCP-Scan) logo

    Snyk Agent Scan (formerly MCP-Scan)

    Snyk · github.com

    Snyk Agent Scan is an Apache-2.0 security scanner for AI agents, MCP servers, and agent skills that detects risks such as prompt injection, tool poisoning, tool shadowing, and malware payloads; Invariant Labs created it as MCP-Scan.

    Snyk acquired Invariant Labs, and MCP-Scan now lives at snyk/agent-scan under the Agent Scan name.

    • Open source · Apache-2.0
    • CLI
    • Python
  • Straiker logo

    Straiker

    straiker.ai

    Straiker is an agentic AI security company whose products detect prompt injection, tool misuse, and runtime attacks across AI agents that organizations build or deploy.

    • Managed cloud
  • WitnessAI logo

    WitnessAI

    witness.ai

    WitnessAI is an AI security and governance platform that combines network-level visibility into AI use, intent-based policy controls, and runtime defense for employees, models, applications, and agents.

    • Managed cloud
  • Zenity logo

    Zenity

    zenity.io

    Zenity is an AI agent security and governance platform that evaluates each agent action in real time and steers or blocks it according to policy.

    • Managed cloud

How to choose

01Risk coverage
Which risks the tool detects, such as prompt injection, jailbreaks, PII leakage, harmful content, hallucination, and unsafe tool calls.
02Enforcement point
Whether checks run in application code, in a gateway or proxy, at the network edge, or through a cloud API.
03Latency and cost
Added latency per request and whether checks use rules, small classifiers, or a full LLM judge.
04Customization
Support for custom policies, topics, and validators instead of a fixed set of categories.
05Testing
Built-in red teaming or scanning that measures exposure before release, with reports mapped to frameworks such as the OWASP Top 10 for LLM applications.
06Agent and MCP coverage
Controls over tool calls, MCP servers, and agent actions, not only chat text.

Frequently asked questions

What are LLM guardrails?

LLM guardrails are programmable checks placed around a language model that validate what goes in and what comes out. Input guardrails screen prompts for injection attempts, jailbreaks, and sensitive data, and output guardrails check responses for harmful content, leaked secrets, off-topic answers, or malformed structure. Guardrails can run as a library in application code, in an AI gateway, or as a cloud API, and can block, rewrite, or log the request.

NeMo Guardrails vs Guardrails AI: what is the difference?

NVIDIA NeMo Guardrails defines input, output, dialog, retrieval, and execution rails in configuration files and the Colang language, and can call safety models such as Llama Guard. Guardrails AI wraps individual LLM calls with validators that check structure, PII, toxicity, and other properties, and can run as a standalone server. Both are Apache-2.0 Python libraries, and Harvey acquired the company behind Guardrails AI in September 2026.

Llama Guard vs Prompt Guard: what is the difference?

Llama Guard is a language model that classifies prompts and responses against a taxonomy of harmful content categories, so it checks both what users ask and what the model says. Prompt Guard is a smaller classifier trained to detect prompt injection and jailbreak attempts in inputs. Meta designed them to work together, and LlamaFirewall combines both with other checks for agent applications.

Which AI red teaming tools are open source?

Open-source options include NVIDIA garak (Apache-2.0), Microsoft PyRIT (MIT), promptfoo (MIT), DeepTeam from Confident AI (Apache-2.0), and Giskard's open-source library (Apache-2.0). garak scans models with libraries of probes, PyRIT automates multi-turn attacks, and promptfoo and DeepTeam generate attacks tailored to an application. Commercial platforms such as Lakera Red, Mindgard, and F5 AI Red Team add managed attack libraries and reporting.

How do you detect prompt injection?

Prompt injection is usually detected with a classifier that scores incoming text, and ideally retrieved documents and tool results, for instructions that try to override the system prompt. Examples include Prompt Guard, Azure Prompt Shields, Lakera, and Google Cloud Model Armor. Detection lowers risk without removing it, so teams also limit which tools an agent can call, separate trusted and untrusted content, and require approval for sensitive actions.

Other categories

Product details change quickly; check each vendor’s site for current features, licensing and pricing.