ai-red-teaming-tools

This repository is a curated reference list, not something you install and run directly. It catalogs tools used for AI red teaming, which means deliberately attacking an AI system such as a language model, a retrieval-based chatbot, or an autonomous agent to find its weaknesses before real attackers do. It covers 105 current tools plus 12 older historical ones across 13 categories, both open source and commercial, and the list says it is reviewed monthly.

The categories span the whole lifecycle of testing an AI system's security. There are end-to-end platforms that automatically generate attacks, run them against a target, and score whether they succeeded. There are narrower scanners that check a specific model or endpoint for known weaknesses. There are libraries that reproduce specific jailbreak techniques, meaning tricks that get a model to ignore its safety rules, as well as tools focused specifically on prompt injection, where hidden instructions are smuggled into a model's input. Other sections cover defenses that block attacks while a system is running, tools for testing whether an AI agent's use of outside tools can be abused, benchmarks for measuring how robust or safe a model is, datasets of attack prompts researchers can reuse, scanners that check model files for hidden malicious code, and reference standards that organize how vulnerabilities are classified.

Every entry links back to its original source, such as a GitHub repository or research paper, so readers can check the claims themselves. The list is also published as a machine-readable JSON file and CSV file for anyone building their own comparison tools. It documents its own selection and review methodology, and keeps a separate section for tools that have been discontinued rather than quietly deleting them. The full README is longer than what was shown.