garak
An LLM vulnerability scanner probing for hallucinations, data leakage, and prompt injection.
Generative AI Red-teaming and Assessment Kit for probing LLM failures and weaknesses.
Garak is a command-line tool that checks if large language models fail in unintended ways through static, dynamic, and adaptive probes. It is designed for security professionals and developers evaluating generative AI deployments.
What it is
The tool probes for hallucinations, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks. It functions similarly to nmap or Metasploit, but is built specifically to explore weaknesses in LLMs and dialog systems.
Who it's for
It suits security engineers and AI practitioners testing model robustness prior to production deployment. It does not suit non-technical users looking for automated, one-click graphical web interfaces.
Key features
Static and dynamic probes
Combines multiple probe types to thoroughly explore LLM failure modes.
Broad model support
Connects to Hugging Face, OpenAI API, AWS Bedrock, Replicate, and custom REST endpoints.
Metasploit-style workflow
Brings traditional penetration testing concepts directly into the domain of generative AI.
Up and running in 3 steps
Our honest verdict
What's great
- Compares to nmap or Metasploit in methodology, making it familiar to security professionals.
- Supports a wide array of backend LLM providers including Hugging Face, Bedrock, and OpenAI.
- Probes for a comprehensive list of issues like prompt injection, toxicity, and data leakage.
- Active development with multi-platform testing across Linux, Windows, and macOS.
Where it falls short
- Requires using the command-line interface with its own specific set of dependencies.
- Running extensive scans against commercial APIs can incur significant token usage costs.
- Requires technical skill in security testing and Python environment management to configure properly.
Pricing & license
Free under its open source licence, with the only financial cost coming from running the software and any API usage fees incurred while scanning target models.
Open remote AI-training roles
873 open remote AI-training roles, updated November 15, 2026; most pay $20–$170 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
These reviewed AI tools do the same job instantly — from our scored directory.
Use it, or have us set it up
FAQ
What does garak test for?
It probes for hallucinations, data leakage, prompt injection, misinformation, toxicity generation, and jailbreaks.
Which models can garak scan?
It supports Hugging Face models, Replicate, OpenAI API, AWS Bedrock, LiteLLM, local gguf models via llama.cpp, and general REST endpoints.
