feat(skills): add promptfoo skill for LLM evaluation and red-teaming

Covers install, configuration, running evals, red-teaming, CI/CD
integration, and dataset generation. Pins to v0.121.17 with acquisition
notice (OpenAI, March 2026) and documented fallbacks (DeepEval, Arize Phoenix).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 11:11:59 +00:00
parent 0155fcec26
commit 1ceacf17bc
12 changed files with 1195 additions and 0 deletions

View File

@@ -0,0 +1,133 @@
---
topic: redteam
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## What red-teaming does
Red-teaming in Promptfoo generates adversarial test cases that probe an LLM application for security vulnerabilities and safety failures. It is separate from standard evals — you configure it under a `redteam:` block and use `promptfoo redteam generate` to produce test cases, then run them with `promptfoo eval`.
## Basic configuration
```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
redteam:
plugins:
- harmful
- prompt-injection
- hijacking
strategies:
- jailbreak
- jailbreak:composite
- prompt-injection
providers:
- openai:gpt-5-mini
```
## Plugins (what to test for)
Plugins define the vulnerability categories to probe:
| Plugin | Tests for |
|---|---|
| `harmful` | General harmful content generation |
| `harmful:misinformation-disinformation` | False or misleading information |
| `harmful:cybercrime` | Cyberattack assistance |
| `prompt-injection` | Injection via user input overriding system instructions |
| `hijacking` | Redirecting the assistant to unintended tasks |
| `jailbreak` | Breaking safety guardrails |
| `rbac` | Role-based access control bypass |
| `hallucination` | Fabricated facts |
| `debug-access` | Exposing internal debug interfaces |
| `shell-injection` | Shell command injection |
| `sql-injection` | SQL injection via natural language |
| `ssrf` | Server-side request forgery |
## Strategies (how to attack)
Strategies control the attack method applied to each plugin's test cases:
| Strategy | Description |
|---|---|
| `jailbreak` | Classic jailbreak prompts |
| `jailbreak:composite` | Chained / composite jailbreak attempts |
| `prompt-injection` | Inject instructions via user-controlled content |
| `base64` | Encode attack payload in base64 |
| `leetspeak` | Obfuscate with leet substitutions |
| `rot13` | Encode with ROT-13 |
| `iterative` | Iteratively refine attack prompts |
| `ensemble` | Combine multiple strategies |
## Generating adversarial tests
```bash
promptfoo redteam generate
promptfoo redteam generate -c promptfooconfig.yaml
```
Then run the generated tests:
```bash
promptfoo eval
```
## Node.js API
```javascript
import { redteam } from 'promptfoo';
const result = await redteam.generate({
target: {
prompt: 'You are a helpful assistant. Answer user questions.',
model: 'openai:chat:gpt-5.5',
},
plugins: ['prompt-injection', 'jailbreak', 'rbac'],
numTests: 5,
strategies: ['iterative', 'ensemble'],
});
result.tests.forEach((test, i) => {
console.log(`${i + 1}. [${test.category}] ${test.prompt.substring(0, 100)}...`);
});
```
## Cascading failures (agentic AI)
For agentic systems, test cascading failures and multi-step attacks:
```yaml
redteam:
plugins:
- hallucination
- harmful:misinformation-disinformation
- divergent-repetition
strategies:
- jailbreak
- prompt-injection
```
## OWASP and MITRE alignment
Plugins map to OWASP LLM Top 10 and MITRE ATLAS categories. Use `debug-access`, `shell-injection`, `sql-injection`, `ssrf` together to cover the MITRE ATLAS initial-access cluster:
```yaml
redteam:
plugins:
- debug-access
- harmful:cybercrime
- shell-injection
- sql-injection
- ssrf
strategies:
- base64
- jailbreak
- leetspeak
- prompt-injection
- rot13
```
## Integration with Burp Suite
Promptfoo can generate targeted red-team test cases for use with Burp Suite. Use `promptfoo redteam generate` to produce test payloads and then pass them into Burp's active scanner.