feat(skills): add promptfoo skill for LLM evaluation and red-teaming
Covers install, configuration, running evals, red-teaming, CI/CD integration, and dataset generation. Pins to v0.121.17 with acquisition notice (OpenAI, March 2026) and documented fallbacks (DeepEval, Arize Phoenix). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
133
plugins/kyberforge/skills/promptfoo/references/redteam.md
Normal file
133
plugins/kyberforge/skills/promptfoo/references/redteam.md
Normal file
@@ -0,0 +1,133 @@
|
||||
---
|
||||
topic: redteam
|
||||
source_keys:
|
||||
- context7-promptfoo-dev
|
||||
- context7-promptfoo-github
|
||||
---
|
||||
|
||||
## What red-teaming does
|
||||
|
||||
Red-teaming in Promptfoo generates adversarial test cases that probe an LLM application for security vulnerabilities and safety failures. It is separate from standard evals — you configure it under a `redteam:` block and use `promptfoo redteam generate` to produce test cases, then run them with `promptfoo eval`.
|
||||
|
||||
## Basic configuration
|
||||
|
||||
```yaml
|
||||
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
|
||||
redteam:
|
||||
plugins:
|
||||
- harmful
|
||||
- prompt-injection
|
||||
- hijacking
|
||||
strategies:
|
||||
- jailbreak
|
||||
- jailbreak:composite
|
||||
- prompt-injection
|
||||
|
||||
providers:
|
||||
- openai:gpt-5-mini
|
||||
```
|
||||
|
||||
## Plugins (what to test for)
|
||||
|
||||
Plugins define the vulnerability categories to probe:
|
||||
|
||||
| Plugin | Tests for |
|
||||
|---|---|
|
||||
| `harmful` | General harmful content generation |
|
||||
| `harmful:misinformation-disinformation` | False or misleading information |
|
||||
| `harmful:cybercrime` | Cyberattack assistance |
|
||||
| `prompt-injection` | Injection via user input overriding system instructions |
|
||||
| `hijacking` | Redirecting the assistant to unintended tasks |
|
||||
| `jailbreak` | Breaking safety guardrails |
|
||||
| `rbac` | Role-based access control bypass |
|
||||
| `hallucination` | Fabricated facts |
|
||||
| `debug-access` | Exposing internal debug interfaces |
|
||||
| `shell-injection` | Shell command injection |
|
||||
| `sql-injection` | SQL injection via natural language |
|
||||
| `ssrf` | Server-side request forgery |
|
||||
|
||||
## Strategies (how to attack)
|
||||
|
||||
Strategies control the attack method applied to each plugin's test cases:
|
||||
|
||||
| Strategy | Description |
|
||||
|---|---|
|
||||
| `jailbreak` | Classic jailbreak prompts |
|
||||
| `jailbreak:composite` | Chained / composite jailbreak attempts |
|
||||
| `prompt-injection` | Inject instructions via user-controlled content |
|
||||
| `base64` | Encode attack payload in base64 |
|
||||
| `leetspeak` | Obfuscate with leet substitutions |
|
||||
| `rot13` | Encode with ROT-13 |
|
||||
| `iterative` | Iteratively refine attack prompts |
|
||||
| `ensemble` | Combine multiple strategies |
|
||||
|
||||
## Generating adversarial tests
|
||||
|
||||
```bash
|
||||
promptfoo redteam generate
|
||||
promptfoo redteam generate -c promptfooconfig.yaml
|
||||
```
|
||||
|
||||
Then run the generated tests:
|
||||
```bash
|
||||
promptfoo eval
|
||||
```
|
||||
|
||||
## Node.js API
|
||||
|
||||
```javascript
|
||||
import { redteam } from 'promptfoo';
|
||||
|
||||
const result = await redteam.generate({
|
||||
target: {
|
||||
prompt: 'You are a helpful assistant. Answer user questions.',
|
||||
model: 'openai:chat:gpt-5.5',
|
||||
},
|
||||
plugins: ['prompt-injection', 'jailbreak', 'rbac'],
|
||||
numTests: 5,
|
||||
strategies: ['iterative', 'ensemble'],
|
||||
});
|
||||
|
||||
result.tests.forEach((test, i) => {
|
||||
console.log(`${i + 1}. [${test.category}] ${test.prompt.substring(0, 100)}...`);
|
||||
});
|
||||
```
|
||||
|
||||
## Cascading failures (agentic AI)
|
||||
|
||||
For agentic systems, test cascading failures and multi-step attacks:
|
||||
|
||||
```yaml
|
||||
redteam:
|
||||
plugins:
|
||||
- hallucination
|
||||
- harmful:misinformation-disinformation
|
||||
- divergent-repetition
|
||||
strategies:
|
||||
- jailbreak
|
||||
- prompt-injection
|
||||
```
|
||||
|
||||
## OWASP and MITRE alignment
|
||||
|
||||
Plugins map to OWASP LLM Top 10 and MITRE ATLAS categories. Use `debug-access`, `shell-injection`, `sql-injection`, `ssrf` together to cover the MITRE ATLAS initial-access cluster:
|
||||
|
||||
```yaml
|
||||
redteam:
|
||||
plugins:
|
||||
- debug-access
|
||||
- harmful:cybercrime
|
||||
- shell-injection
|
||||
- sql-injection
|
||||
- ssrf
|
||||
strategies:
|
||||
- base64
|
||||
- jailbreak
|
||||
- leetspeak
|
||||
- prompt-injection
|
||||
- rot13
|
||||
```
|
||||
|
||||
## Integration with Burp Suite
|
||||
|
||||
Promptfoo can generate targeted red-team test cases for use with Burp Suite. Use `promptfoo redteam generate` to produce test payloads and then pass them into Burp's active scanner.
|
||||
Reference in New Issue
Block a user