NeMo Guardrails

NeMo Guardrails is an Apache 2.0 runtime safety framework built by NVIDIA, first released with a peer-reviewed EMNLP 2023 research paper. Where every other tool in the Testing category evaluates model outputs offline, NeMo Guardrails intercepts live LLM traffic and enforces configurable safety policies before messages reach the model, during retrieval, and before responses return to users. Its core technical differentiator is Colang, a domain-specific language purpose-built for defining conversation flows and safety rules, which models entire multi-turn dialog sequences rather than filtering individual inputs and outputs in isolation. Version 0.20.0, released January 2026, added parallel rail execution and OpenTelemetry tracing.

Key Capabilities

• Five-layer rail architecture: Input rails filter user messages before the LLM sees them; dialog rails shape conversational flow using Colang-defined sequences; retrieval rails validate and mask content from RAG knowledge bases; execution rails guard tool calls and external API interactions; output rails inspect generated responses before delivery

• Colang DSL for dialog safety: A Python-like domain-specific language that defines conversational guardrails as configuration rather than application code, allowing security teams to update safety rules without modifying the underlying LLM application

• Jailbreak and prompt injection prevention: Input rails apply NVIDIA's built-in safety models or third-party providers to detect manipulation attempts that bypass system prompt constraints, operating independently of the model's alignment training

• PII detection and masking: Integrates with NVIDIA GLiNER-PII and Microsoft Presidio to detect and mask names, email addresses, phone numbers, and social security numbers across user inputs, LLM outputs, and retrieved document chunks

• Agentic security for tool-calling workflows: Validates tool inputs and outputs in LangGraph multi-agent deployments, monitors agent action sequences, and applies execution rails to external API calls made by autonomous agents

• NVIDIA NIM and multi-provider support: Works natively with NVIDIA NIM microservices for topic control and content safety, and connects to OpenAI, Anthropic, Azure, Hugging Face, and LangChain providers through a unified configuration interface