NeMo Guardrails
Enforce safety policies across live LLM conversations using a programmable rail architecture
Description
NeMo Guardrails is an Apache 2.0 runtime safety framework built by NVIDIA, first released with a peer-reviewed EMNLP 2023 research paper. Where every other tool in the Testing category evaluates model outputs offline, NeMo Guardrails intercepts live LLM traffic and enforces configurable safety policies before messages reach the model, during retrieval, and before responses return to users. Its core technical differentiator is Colang, a domain-specific language purpose-built for defining conversation flows and safety rules, which models entire multi-turn dialog sequences rather than filtering individual inputs and outputs in isolation. Version 0.20.0, released January 2026, added parallel rail execution and OpenTelemetry tracing.
Key Capabilities
Five-layer rail architecture: Input rails filter user messages before the LLM sees them; dialog rails shape conversational flow using Colang-defined sequences; retrieval rails validate and mask content from RAG knowledge bases; execution rails guard tool calls and external API interactions; output rails inspect generated responses before delivery
Colang DSL for dialog safety: A Python-like domain-specific language that defines conversational guardrails as configuration rather than application code, allowing security teams to update safety rules without modifying the underlying LLM application
Jailbreak and prompt injection prevention: Input rails apply NVIDIA's built-in safety models or third-party providers to detect manipulation attempts that bypass system prompt constraints, operating independently of the model's alignment training
PII detection and masking: Integrates with NVIDIA GLiNER-PII and Microsoft Presidio to detect and mask names, email addresses, phone numbers, and social security numbers across user inputs, LLM outputs, and retrieved document chunks
Agentic security for tool-calling workflows: Validates tool inputs and outputs in LangGraph multi-agent deployments, monitors agent action sequences, and applies execution rails to external API calls made by autonomous agents
NVIDIA NIM and multi-provider support: Works natively with NVIDIA NIM microservices for topic control and content safety, and connects to OpenAI, Anthropic, Azure, Hugging Face, and LangChain providers through a unified configuration interface
Alternative tools
- Claude Code
Agentic coding tool that runs in your terminal
- Pythagora
Full-stack AI app builder with 14 specialized agents
- Refact.ai
Local-first AI coding agent with enterprise fine-tuning support
- Blackbox AI
Multi-model AI coding assistant with Chairman LLM orchestration
- Junie
JetBrains' AI coding agent with deep static analysis integration
- Evidently AI
Evaluate, test, and monitor traditional ML models and LLM applications from one framework
