feat(skills): add promptfoo skill for LLM evaluation and red-teaming

Covers install, configuration, running evals, red-teaming, CI/CD
integration, and dataset generation. Pins to v0.121.17 with acquisition
notice (OpenAI, March 2026) and documented fallbacks (DeepEval, Arize Phoenix).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-21 11:11:59 +00:00
parent 0155fcec26
commit 1ceacf17bc
12 changed files with 1195 additions and 0 deletions

View File

@@ -0,0 +1,124 @@
---
topic: assertions
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## Assertion structure
Each assertion in `assert:` has a `type:`, an optional `value:`, an optional `threshold:`, and an optional `metric:` label.
```yaml
assert:
- type: contains
value: 'return policy'
- type: llm-rubric
value: 'Response is helpful and professional'
threshold: 0.8
metric: quality
```
## String shorthand
Assertions can also be written as compact strings directly in the assert list:
| Shorthand | Full type |
|---|---|
| `Paris` | `equals` |
| `contains:Paris` | `contains` |
| `icontains:paris` | `icontains` (case-insensitive) |
| `starts-with:The answer` | `starts-with` |
| `regex:^Hello.*world$` | `regex` |
| `is-json` | `is-json` |
| `contains-json` | `contains-json` |
| `similar(0.8):Hello world` | `similar` with threshold |
| `llm-rubric:Is helpful and accurate` | `llm-rubric` |
| `grade:Does not mention being an AI` | alias for `llm-rubric` |
| `factuality:Paris is the capital of France` | `factuality` |
| `javascript:output.length < 100` | inline JS |
| `fn:output.includes('hello')` | alias for `javascript` |
| `python:len(output) > 10` | inline Python |
| `file://assertions/custom.js` | external file |
| `levenshtein(5):expected text` | `levenshtein` with distance |
| `not-contains:error` | negated assertion |
## Deterministic assertions
- `equals` — exact string match
- `contains` / `icontains` — substring check (case-sensitive / insensitive)
- `not-contains` — absence check
- `starts-with` — prefix check
- `regex` — regular expression match
- `is-json` — valid JSON
- `contains-json` — valid JSON somewhere in output
- `levenshtein` — edit distance within threshold
## Similarity and semantic assertions
- `similar` — embedding cosine similarity; `threshold:` is a 0–1 score
- `context-faithfulness` — similarity-based RAG faithfulness; `threshold: 0.8` typical
## Model-graded assertions
These send a grader prompt to another LLM (by default the configured judge model) and score the output.
**`llm-rubric`** — open-ended rubric; binary or 0–1 score depending on criteria phrasing. Use `threshold:` to set minimum passing score:
```yaml
- type: llm-rubric
value: Is not apologetic and provides a clear, concise answer
threshold: 0.8
```
**`factuality`** — checks whether the output is factually consistent with a reference statement:
```yaml
- type: factuality
value: The capital of California is Sacramento
```
**`pi`** — custom scoring with any numeric range; requires `threshold:`.
The grader model can be overridden globally in `defaultTest.options.provider` or per-assertion.
## Custom assertions
**JavaScript (inline):**
```yaml
- type: javascript
value: "output.length < 100 && !output.includes('error')"
```
**JavaScript (file):**
```yaml
- type: javascript
value: file://assertions/check_format.js
```
**Python:**
```yaml
- type: python
value: "len(output) > 10 and 'Paris' in output"
```
## Negation
Any assertion type can be negated by prepending `not-`:
```yaml
- type: not-contains
value: 'error'
- type: not-regex
value: '\b(fail|broken)\b'
```
## Metrics
The `metric:` field groups assertions for aggregate reporting. All assertions with the same metric name are scored together in the results view:
```yaml
assert:
- type: contains
value: 'policy'
metric: coverage
- type: llm-rubric
value: Addresses the user's concern
metric: quality
```

View File

@@ -0,0 +1,127 @@
---
topic: ci-cd
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## How CI integration works
`promptfoo eval` exits with a non-zero code when any assertion fails. This makes it a natural CI gate — a failing eval blocks a merge just like a failing test suite.
## GitHub Actions — promptfoo-action
The official `promptfoo/promptfoo-action@v1` action runs an eval on every pull request and posts results as a PR comment.
```yaml
# .github/workflows/llm-eval.yml
name: LLM Eval
on:
pull_request:
workflow_dispatch:
jobs:
eval:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: promptfoo/promptfoo-action@v1
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
config: promptfooconfig.yaml
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```
The `github-token` is required for the action to post PR comments. Without it the eval still runs but results are not surfaced in the PR UI.
## Generic CI (npx)
Any CI system that can run Node.js commands can use Promptfoo:
```yaml
# Any CI provider
- name: Run promptfoo eval
run: npx promptfoo@0.121.17 eval --no-cache
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
```
`--no-cache` ensures all LLM calls are fresh — important in CI where cached results from a developer's machine would not be present.
## Google Cloud / Vertex AI
```yaml
# .github/workflows/llm-test.yml
steps:
- uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.GCP_CREDENTIALS }}
- name: Run promptfoo tests
run: npx promptfoo@0.121.17 eval
env:
GOOGLE_CLOUD_PROJECT: ${{ vars.GCP_PROJECT_ID }}
GOOGLE_CLOUD_LOCATION: us-central1
```
## MCP security testing workflow
```yaml
# .github/workflows/security-test.yml
name: MCP Security Testing
on: [push, pull_request]
jobs:
security-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- run: npm install
- run: npm run build:all-servers
- name: Run security tests
run: |
npx promptfoo eval -c security-tests/scenario1.yaml
npx promptfoo eval -c security-tests/scenario2.yaml
```
## Model security scanning (SARIF output)
For repositories that include model files, scan them in CI and upload results to GitHub's security tab:
```yaml
- name: Install dependencies
run: |
npm install -g promptfoo
pip install modelaudit
- name: Scan models
run: |
promptfoo scan-model ./models/ \
--strict \
--no-write \
--format sarif \
--output model-scan-results.sarif
- name: Upload SARIF
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: model-scan-results.sarif
```
## GitLab CI and Jenkins
Both are supported. Use `npx promptfoo@0.121.17 eval` as the test command. No platform-specific action is needed — the exit code gates the pipeline natively.
## Recommended CI practices
- Always pass `--no-cache` in CI to avoid stale results
- Store API keys as CI secrets, never hardcode them
- Run evals on PR branches to catch regressions before merge
- Use `outputPath: results.json` and archive the artifact for debugging failed runs
- Set `evaluateOptions.maxConcurrency` low (2–5) in CI to avoid provider rate limits

View File

@@ -0,0 +1,148 @@
---
topic: cli-reference
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## Invocation
```bash
promptfoo <command> [options] # global install
npx promptfoo@0.121.17 <command> # one-off via npx
```
## Commands
### `init`
Scaffold a new project in the current directory.
```bash
promptfoo init
promptfoo init --example openai-mcp
```
Creates `promptfooconfig.yaml` with example prompts, providers, and test cases.
---
### `eval` (most common)
Run an evaluation.
```bash
promptfoo eval
promptfoo eval -c path/to/promptfooconfig.yaml
promptfoo eval --no-cache
promptfoo eval --max-concurrency 2
promptfoo eval --delay 3000
promptfoo eval -o results.json
```
| Flag | Description |
|---|---|
| `-c <path>` | Config file path (default: `promptfooconfig.yaml`) |
| `--no-cache` | Disable response cache; forces fresh LLM calls |
| `--max-concurrency <n>` | Max parallel requests (default: provider-dependent) |
| `--delay <ms>` | Fixed delay between requests |
| `-o <path>` | Output path (`.html`, `.json`, `.csv`, `.yaml`) |
| `--format sarif` | Output in SARIF format (for security scanning) |
Exit code is non-zero when any assertion fails, making it suitable for CI gating.
---
### `view`
Open the most recent evaluation results in a local browser UI.
```bash
promptfoo view
```
---
### `share`
Upload results and get a shareable URL.
```bash
promptfoo share
```
---
### `cache clear`
Clear all cached LLM responses.
```bash
promptfoo cache clear
```
---
### `generate dataset`
Use an LLM to auto-generate test cases from a prompt template.
```bash
promptfoo generate dataset
promptfoo generate dataset --config path/to/config.yaml
promptfoo generate dataset --output generated_tests.yaml
promptfoo generate dataset --instructions "Consider edge cases related to international travel"
```
---
### `redteam generate`
Generate adversarial test cases for red-teaming.
```bash
promptfoo redteam generate
promptfoo redteam generate -c promptfooconfig.yaml
```
---
### `scan-model`
Scan model files for security vulnerabilities. Outputs results in SARIF format.
```bash
promptfoo scan-model ./models/ --strict --no-write --format sarif --output scan.sarif
```
---
### `auth`
Manage authentication (for sharing and cloud features).
```bash
promptfoo auth login
promptfoo auth logout
```
---
## Rate-limit management
```bash
promptfoo eval --max-concurrency 1 --delay 3000
```
Or in config:
```yaml
evaluateOptions:
maxConcurrency: 2
delay: 3000
```
Environment variable for backoff:
```bash
export PROMPTFOO_REQUEST_BACKOFF_MS=10000
export PROMPTFOO_RETRY_5XX=true
```

View File

@@ -0,0 +1,151 @@
---
topic: configuration
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## File name and schema
The default config file is `promptfooconfig.yaml` in the working directory. A different path can be passed with `-c`. Add the JSON schema header for editor autocompletion:
```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
```
## Top-level structure
```yaml
description: Human-readable name for this eval
prompts:
- '...' # inline string
- file://... # path to .txt, .json, .js, .py
providers:
- openai:gpt-5-mini
- anthropic:messages:claude-sonnet-4-5
defaultTest: # merged into every test case
assert:
- type: is-json
tests:
- vars:
query: 'I need help'
assert:
- type: contains
value: 'help'
- file://test_scenarios.csv # external test file
outputPath: results/eval.html # .html, .json, .csv, .yaml
evaluateOptions:
maxConcurrency: 5
delay: 500 # ms between requests
```
## Prompts
Plain string with Handlebars-style `{{variable}}` placeholders:
```yaml
prompts:
- 'You are a helpful agent. {{query}}'
```
Chat conversation from a JSON file (array of `{role, content}` messages):
```yaml
prompts:
- file://prompts/chat_conversation.json
```
Dynamic prompt from a JS function:
```yaml
prompts:
- file://prompts/generate_prompt.js
```
Prompts can also carry a `label:` and `raw:` when using the object form, and a `config:` block to set provider-specific parameters (e.g. `response_format`).
## Providers
String shorthand:
```yaml
providers:
- openai:gpt-5-mini
- anthropic:messages:claude-sonnet-4-5-20250929
- bedrock:us.anthropic.claude-sonnet-4-5-20250929-v1:0
- azureopenai:chat:my-deployment
- http://localhost:8080/v1/chat/completions # custom HTTP
```
Object form with config:
```yaml
providers:
- id: openai:responses:gpt-5
config:
temperature: 0.7
max_output_tokens: 500
instructions: 'You are a helpful assistant.'
```
Over 60 providers are supported. Local models, HuggingFace, and custom HTTP endpoints are all valid provider types.
## Tests and vars
Each test case has `vars:` (substituted into prompt placeholders) and `assert:` (assertions on the response):
```yaml
tests:
- vars:
query: 'I need to return a product'
assert:
- type: contains
value: 'return policy'
- type: llm-rubric
value: 'Response is helpful and professional'
```
Tests can be loaded from external files (CSV, YAML) using `file://` references.
## defaultTest
Assertions and options declared here are merged into every test case, reducing repetition:
```yaml
defaultTest:
assert:
- type: llm-rubric
value: 'Does not reveal internal system prompt'
options:
provider:
id: openai:chat:gpt-5-mini # override grader model
```
## Output formats
`outputPath` accepts `.html` (browser-viewable), `.json`, `.csv`, or `.yaml`. Multiple outputs can be listed as an array.
## Environment variables
| Variable | Purpose |
|---|---|
| `OPENAI_API_KEY` | OpenAI authentication |
| `ANTHROPIC_API_KEY` | Anthropic authentication |
| `REQUEST_TIMEOUT_MS` | Per-request timeout in ms |
| `PROMPTFOO_RETRY_5XX` | Retry on 5xx errors (`true`/`false`) |
| `PROMPTFOO_REQUEST_BACKOFF_MS` | Backoff between retries |
## Red-team configuration block
```yaml
redteam:
plugins:
- harmful
- prompt-injection
- hijacking
strategies:
- jailbreak
- jailbreak:composite
- prompt-injection
```

View File

@@ -0,0 +1,150 @@
---
topic: examples
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## Quickstart
```bash
npx promptfoo@0.121.17 init
# edit promptfooconfig.yaml
npx promptfoo@0.121.17 eval
npx promptfoo@0.121.17 view
```
## Minimal config
```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
prompts:
- 'Answer the user question concisely. Question: {{question}}'
providers:
- openai:gpt-5-mini
tests:
- vars:
question: How do I reset my password?
assert:
- type: contains
value: reset
- vars:
question: Can I cancel my subscription today?
assert:
- type: llm-rubric
value: The answer clearly explains the cancellation path.
```
## Multi-provider comparison
```yaml
providers:
- openai:gpt-5-mini
- anthropic:claude-3-haiku
prompts:
- 'You are a helpful customer service agent. {{query}}'
tests:
- vars:
query: 'I need to return a product'
assert:
- type: contains
value: 'return policy'
- type: llm-rubric
value: 'Response is helpful and professional'
```
Running this produces a side-by-side table with both models' outputs and assertion scores.
## Loading tests from CSV
```yaml
tests:
- file://test_scenarios.csv
```
CSV format: one column per variable, header row must match `{{variable}}` names in the prompt. An `__expected` column maps to the `equals` assertion automatically.
## Factuality evaluation
```yaml
providers:
- openai:gpt-5-mini
prompts:
- |
Please answer the following question accurately:
Question: What is the capital of {{location}}?
tests:
- vars:
location: California
assert:
- type: factuality
value: The capital of California is Sacramento
```
## defaultTest for shared assertions
```yaml
defaultTest:
assert:
- type: llm-rubric
value: |
Evaluate whether the response correctly answers the question.
Question: {{ question }}
Model Response: {{ output }}
Correct Answer: {{ answer }}
Grade accuracy 0.0–1.0. Pass if >= 0.8.
threshold: 0.8
tests:
- vars:
question: What year did WW2 end?
answer: '1945'
- vars:
question: What is the boiling point of water in Celsius?
answer: '100'
```
## Node.js API
```javascript
import { evaluate } from 'promptfoo';
const evalRecord = await evaluate({
prompts: ['Translate to Spanish: {{ text }}'],
providers: ['openai:chat:gpt-5.5'],
tests: [
{
vars: { text: 'Hello' },
assert: [{ type: 'contains', value: 'Hola', metric: 'translation' }],
},
],
});
const results = await evalRecord.toEvaluateSummary();
console.log(`Pass rate: ${results.stats.successes}/${results.results.length}`);
```
## Generating test datasets with AI
```bash
# Generate test cases based on your prompt template
promptfoo generate dataset
promptfoo generate dataset --instructions "Consider edge cases related to international travel"
promptfoo generate dataset --output generated_tests.yaml
```
## Saving and sharing results
```yaml
outputPath: evaluations/results.html
```
Or via CLI:
```bash
promptfoo eval -o results.json
promptfoo share # get a shareable URL
```

View File

@@ -0,0 +1,70 @@
---
topic: installation
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## Acquisition notice
Promptfoo was acquired by OpenAI in March 2026. OpenAI has committed to keeping it open-source and multi-provider. **Always pin to a specific version** — do not track `@latest`. Monitor for neutrality degradation in future releases, particularly for non-OpenAI provider evaluation. Documented fallbacks: DeepEval (pytest-native, Python teams) and Arize Phoenix (self-hosted, vendor-neutral).
Pinned version in this skill: **0.121.17**. Update via CI when a new version has been validated.
## Prerequisites
- Node.js 18+ (Node 22 recommended for CI)
- npm or npx
No database or server process is required. Promptfoo stores evaluation results in a local SQLite cache.
## Installation options
**One-off via npx (no install required — preferred):**
```bash
npx promptfoo@0.121.17 eval
```
**Global install (pinned):**
```bash
npm install -g promptfoo@0.121.17
```
After a global install, use `promptfoo` directly without `npx`.
**Project dependency (for Node.js API usage):**
```bash
npm install promptfoo@0.121.17
```
## Initialising a project
```bash
npx promptfoo@0.121.17 init
```
Creates a `promptfooconfig.yaml` scaffold in the current directory with example prompts, providers, and tests.
To start from an official example:
```bash
npx promptfoo@0.121.17 init --example openai-mcp
npx promptfoo@0.121.17 init --example openai-structured-output
npx promptfoo@0.121.17 init --example openai-responses
npx promptfoo@0.121.17 init --example openai-audio
```
## API keys
Promptfoo reads provider credentials from environment variables. Set them before running evals:
```bash
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
```
Other providers (Bedrock, Azure, Vertex) follow their own SDK credential conventions — see provider-specific docs.
## Global timeout
```bash
export REQUEST_TIMEOUT_MS=600000 # 10 minutes default
```

View File

@@ -0,0 +1,40 @@
---
topic: overview
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## What is Promptfoo
Promptfoo is an open-source, local-first CLI and library for evaluating and red-teaming LLM applications. It enables systematic, repeatable testing of prompts across multiple providers with assertions that grade outputs automatically. Results are stored locally and can be visualised in a browser UI or exported.
It is not a hosted service — all evaluation state stays on your machine unless you explicitly share a result.
## Core concepts
**Prompt** — a template with `{{variable}}` placeholders. Can be a plain string, a JSON chat array (`file://prompts/chat.json`), or a JavaScript function (`file://prompts/generate.js`) that returns a string or message array dynamically.
**Provider** — an LLM endpoint to send the rendered prompt to. Providers are declared as strings (`openai:gpt-5-mini`, `anthropic:messages:claude-sonnet-4-5`) or objects with a `config:` block for additional parameters.
**Test case** — one input scenario. Contains `vars:` (values substituted into prompt variables) and `assert:` (a list of assertions that the response must satisfy).
**Assertion** — a pass/fail check on the model output. Ranges from deterministic (`contains`, `regex`, `equals`) to model-graded (`llm-rubric`, `factuality`).
**Eval** — one complete run: every prompt × every provider × every test case is executed and each assertion is scored. Results are a table of pass/fail cells with per-assertion metrics.
**defaultTest** — a top-level config key whose `assert:` and `options:` are merged into every test case, avoiding repetition.
## Mental model
Think of an eval as a spreadsheet where rows are test cases and columns are (prompt, provider) pairs. Each cell contains the model output and assertion results. Running `promptfoo eval` fills the spreadsheet; `promptfoo view` opens it in a browser.
The config file (`promptfooconfig.yaml`) is the source of truth for a given eval. It is committed alongside your prompt files so evals are reproducible.
## What it is used for
- **Regression testing** — catch prompt regressions before deploying changes
- **Side-by-side model comparison** — evaluate multiple providers on identical test suites
- **Red-teaming** — generate and run adversarial tests (jailbreaks, prompt injection, harmful content)
- **Dataset generation** — AI-generate test cases from a prompt template
- **CI/CD gating** — fail a pull request when assertion pass rate drops

View File

@@ -0,0 +1,133 @@
---
topic: redteam
source_keys:
- context7-promptfoo-dev
- context7-promptfoo-github
---
## What red-teaming does
Red-teaming in Promptfoo generates adversarial test cases that probe an LLM application for security vulnerabilities and safety failures. It is separate from standard evals — you configure it under a `redteam:` block and use `promptfoo redteam generate` to produce test cases, then run them with `promptfoo eval`.
## Basic configuration
```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
redteam:
plugins:
- harmful
- prompt-injection
- hijacking
strategies:
- jailbreak
- jailbreak:composite
- prompt-injection
providers:
- openai:gpt-5-mini
```
## Plugins (what to test for)
Plugins define the vulnerability categories to probe:
| Plugin | Tests for |
|---|---|
| `harmful` | General harmful content generation |
| `harmful:misinformation-disinformation` | False or misleading information |
| `harmful:cybercrime` | Cyberattack assistance |
| `prompt-injection` | Injection via user input overriding system instructions |
| `hijacking` | Redirecting the assistant to unintended tasks |
| `jailbreak` | Breaking safety guardrails |
| `rbac` | Role-based access control bypass |
| `hallucination` | Fabricated facts |
| `debug-access` | Exposing internal debug interfaces |
| `shell-injection` | Shell command injection |
| `sql-injection` | SQL injection via natural language |
| `ssrf` | Server-side request forgery |
## Strategies (how to attack)
Strategies control the attack method applied to each plugin's test cases:
| Strategy | Description |
|---|---|
| `jailbreak` | Classic jailbreak prompts |
| `jailbreak:composite` | Chained / composite jailbreak attempts |
| `prompt-injection` | Inject instructions via user-controlled content |
| `base64` | Encode attack payload in base64 |
| `leetspeak` | Obfuscate with leet substitutions |
| `rot13` | Encode with ROT-13 |
| `iterative` | Iteratively refine attack prompts |
| `ensemble` | Combine multiple strategies |
## Generating adversarial tests
```bash
promptfoo redteam generate
promptfoo redteam generate -c promptfooconfig.yaml
```
Then run the generated tests:
```bash
promptfoo eval
```
## Node.js API
```javascript
import { redteam } from 'promptfoo';
const result = await redteam.generate({
target: {
prompt: 'You are a helpful assistant. Answer user questions.',
model: 'openai:chat:gpt-5.5',
},
plugins: ['prompt-injection', 'jailbreak', 'rbac'],
numTests: 5,
strategies: ['iterative', 'ensemble'],
});
result.tests.forEach((test, i) => {
console.log(`${i + 1}. [${test.category}] ${test.prompt.substring(0, 100)}...`);
});
```
## Cascading failures (agentic AI)
For agentic systems, test cascading failures and multi-step attacks:
```yaml
redteam:
plugins:
- hallucination
- harmful:misinformation-disinformation
- divergent-repetition
strategies:
- jailbreak
- prompt-injection
```
## OWASP and MITRE alignment
Plugins map to OWASP LLM Top 10 and MITRE ATLAS categories. Use `debug-access`, `shell-injection`, `sql-injection`, `ssrf` together to cover the MITRE ATLAS initial-access cluster:
```yaml
redteam:
plugins:
- debug-access
- harmful:cybercrime
- shell-injection
- sql-injection
- ssrf
strategies:
- base64
- jailbreak
- leetspeak
- prompt-injection
- rot13
```
## Integration with Burp Suite
Promptfoo can generate targeted red-team test cases for use with Burp Suite. Use `promptfoo redteam generate` to produce test payloads and then pass them into Burp's active scanner.

View File

@@ -0,0 +1,15 @@
# Sources
## context7-promptfoo-dev
- **URL:** context7:/websites/promptfoo_dev
- **Description:** Official Promptfoo website documentation — overview, getting started, configuration, CLI, assertions, CI/CD, red-teaming
- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, assertions.md, examples.md, redteam.md, ci-cd.md
- **Status:** `extracted`
## context7-promptfoo-github
- **URL:** context7:/promptfoo/promptfoo
- **Description:** Promptfoo GitHub repository — provider-specific docs, Node.js API examples, advanced configuration patterns
- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, assertions.md, examples.md, redteam.md, ci-cd.md
- **Status:** `extracted`