garak
Vulnerability scanner for large language models
garak is profiled here as a Evaluation tool for engineering teams. Read about features, pricing, and how it compares to related options in the tools directory.
Description
Garak is an open-source LLM vulnerability scanner created by Leon Derczynski and now maintained under NVIDIA's GitHub organization. It fires libraries of adversarial probes at a model or endpoint, covering prompt injection, jailbreaks, data leakage, toxicity, and hallucination, then reports which attacks landed. Security teams run it before release to find weaknesses while they are still cheap to fix. Each run produces a report that maps findings to specific probe categories, so fixes can be verified by running the same scan again.
Key Capabilities:
Probe library covering prompt injection, jailbreak, encoding, and exfiltration attacks
Detectors that score outputs for leakage, harmful content, and hallucination
Support for OpenAI-compatible APIs, Hugging Face models, REST endpoints, and Ollama
Report generation with per-probe hit rates for security review
Plugin architecture for custom probes and detectors
Apache 2.0 license with NVIDIA-maintained releases
Alternative tools
- Gentrace
Testing and evaluation for generative AI applications
- HELM
Reproducible, multi-scenario benchmarking of foundation models
- lm-evaluation-harness
Standard framework for benchmarking language models
- DeepChecks
Validate ML models, LLM applications, and AI agent decisions across every development stage
- Evidently AI
Evaluate, test, and monitor traditional ML models and LLM applications from one framework
- Vectara HHEM
Detect hallucinations in RAG outputs using a dedicated classification model