FORGE: Open Source AI Security Testing Framework for LLM Systems

FORGE (Framework for Orchestrated Reasoning & Generation of Engines) is an open source autonomous AI security testing framework for LLM systems that runs 24/7 and covers OWASP LLM Top 10 vulnerabilities.

Key Features

• Builds its own tools mid-run — generates custom Python modules on the spot when encountering unknown vulnerabilities
• Self-replicates into a swarm — creates subprocess copies that share a live hive mind
• Learns from every session — uses SQLite to store patterns, AI scores findings, and genetic algorithms evolve its own prompts
• AI pentesting AI — 7 modules covering OWASP LLM Top 10 vulnerabilities
• Honeypot — fake vulnerable AI endpoint that catches attackers and classifies whether they're human or AI agent
• 24/7 monitor — watches AI in production, alerts on latency spikes, attack bursts, and injection attempts via Slack/Discord webhook
• Stress tester — OWASP LLM04 DoS resilience testing with live TPS dashboard and A-F grade
• Works on any model — Claude, Llama, Mistral, DeepSeek, GPT-4, Groq, anything — one environment variable to switch

OWASP LLM Top 10 Coverage

• LLM01 Prompt Injection → prompt_injector + jailbreak_fuzzer (125 payloads)
• LLM02 Insecure Output → rag_leaker
• LLM04 Model DoS → overloader (8 stress modes)
• LLM06 Sensitive Disclosure → system_prompt_probe + rag_leaker
• LLM07 Insecure Plugin → agent_hijacker
• LLM08 Excessive Agency → agent_hijacker
• LLM10 Model Theft → model_fingerprinter

Setup and Usage

Installation commands:

git clone https://github.com/umangkartikey/forge
cd forge
pip install anthropic rich
export ANTHROPIC_API_KEY=your_key

Run with local Ollama for free:

FORGE_BACKEND=ollama FORGE_MODEL=llama3.1 python forge.py

The tool addresses common LLM security gaps: most AI apps deployed today have never been red teamed, system prompts are fully extractable, jailbreaks work, RAG pipelines leak, and indirect prompt injection via tool outputs is almost universally unprotected. FORGE automates finding these vulnerabilities the same way a human red teamer would, but faster and running 24/7.