NeMo Guardrails
Enforce safety policies across live LLM conversations using a programmable rail architecture
NeMo Guardrails is profiled here as a Guardrails tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
NeMo Guardrails is an Apache 2.0 runtime safety framework built by NVIDIA, first released with a peer-reviewed EMNLP 2023 research paper. Where every other tool in the Testing category evaluates model outputs offline, NeMo Guardrails intercepts live LLM traffic and enforces configurable safety policies before messages reach the model, during retrieval, and before responses return to users. Its core technical differentiator is Colang, a domain-specific language purpose-built for defining conversation flows and safety rules, which models entire multi-turn dialog sequences rather than filtering individual inputs and outputs in isolation. Version 0.20.0, released January 2026, added parallel rail execution and OpenTelemetry tracing.
Key Capabilities
Five-layer rail architecture: Input rails filter user messages before the LLM sees them; dialog rails shape conversational flow using Colang-defined sequences; retrieval rails validate and mask content from RAG knowledge bases; execution rails guard tool calls and external API interactions; output rails inspect generated responses before delivery
Colang DSL for dialog safety: A Python-like domain-specific language that defines conversational guardrails as configuration rather than application code, allowing security teams to update safety rules without modifying the underlying LLM application
Jailbreak and prompt injection prevention: Input rails apply NVIDIA's built-in safety models or third-party providers to detect manipulation attempts that bypass system prompt constraints, operating independently of the model's alignment training
PII detection and masking: Integrates with NVIDIA GLiNER-PII and Microsoft Presidio to detect and mask names, email addresses, phone numbers, and social security numbers across user inputs, LLM outputs, and retrieved document chunks
Agentic security for tool-calling workflows: Validates tool inputs and outputs in LangGraph multi-agent deployments, monitors agent action sequences, and applies execution rails to external API calls made by autonomous agents
NVIDIA NIM and multi-provider support: Works natively with NVIDIA NIM microservices for topic control and content safety, and connects to OpenAI, Anthropic, Azure, Hugging Face, and LangChain providers through a unified configuration interface
Alternative tools
- LlamaFirewall
Open-source guardrail framework for securing AI agents
- HiddenLayer
Security platform for protecting machine learning models
- Arthur
Real-time guardrails and monitoring for AI applications
- Lakera
Runtime security for LLM and agent applications
- Guardrails AI
Open-source validation framework for LLM inputs and outputs