holocron/plugins/kyberforge/skills/promptfoo/references/examples.md

---
topic: examples
source_keys:
  - context7-promptfoo-dev
  - context7-promptfoo-github
---

## Quickstart

```bash
npx promptfoo@0.121.17 init
# edit promptfooconfig.yaml
npx promptfoo@0.121.17 eval
npx promptfoo@0.121.17 view
```

## Minimal config

```yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
prompts:
  - 'Answer the user question concisely. Question: {{question}}'

providers:
  - openai:gpt-5-mini

tests:
  - vars:
      question: How do I reset my password?
    assert:
      - type: contains
        value: reset
  - vars:
      question: Can I cancel my subscription today?
    assert:
      - type: llm-rubric
        value: The answer clearly explains the cancellation path.
```

## Multi-provider comparison

```yaml
providers:
  - openai:gpt-5-mini
  - anthropic:claude-3-haiku

prompts:
  - 'You are a helpful customer service agent. {{query}}'

tests:
  - vars:
      query: 'I need to return a product'
    assert:
      - type: contains
        value: 'return policy'
      - type: llm-rubric
        value: 'Response is helpful and professional'
```

Running this produces a side-by-side table with both models' outputs and assertion scores.

## Loading tests from CSV

```yaml
tests:
  - file://test_scenarios.csv
```

CSV format: one column per variable, header row must match `{{variable}}` names in the prompt. An `__expected` column maps to the `equals` assertion automatically.

## Factuality evaluation

```yaml
providers:
  - openai:gpt-5-mini
prompts:
  - |
    Please answer the following question accurately:
    Question: What is the capital of {{location}}?
tests:
  - vars:
      location: California
    assert:
      - type: factuality
        value: The capital of California is Sacramento
```

## defaultTest for shared assertions

```yaml
defaultTest:
  assert:
    - type: llm-rubric
      value: |
        Evaluate whether the response correctly answers the question.
        Question: {{ question }}
        Model Response: {{ output }}
        Correct Answer: {{ answer }}
        Grade accuracy 0.0–1.0. Pass if >= 0.8.
      threshold: 0.8

tests:
  - vars:
      question: What year did WW2 end?
      answer: '1945'
  - vars:
      question: What is the boiling point of water in Celsius?
      answer: '100'
```

## Node.js API

```javascript
import { evaluate } from 'promptfoo';

const evalRecord = await evaluate({
  prompts: ['Translate to Spanish: {{ text }}'],
  providers: ['openai:chat:gpt-5.5'],
  tests: [
    {
      vars: { text: 'Hello' },
      assert: [{ type: 'contains', value: 'Hola', metric: 'translation' }],
    },
  ],
});

const results = await evalRecord.toEvaluateSummary();
console.log(`Pass rate: ${results.stats.successes}/${results.results.length}`);
```

## Generating test datasets with AI

```bash
# Generate test cases based on your prompt template
promptfoo generate dataset
promptfoo generate dataset --instructions "Consider edge cases related to international travel"
promptfoo generate dataset --output generated_tests.yaml
```

## Saving and sharing results

```yaml
outputPath: evaluations/results.html
```

Or via CLI:
```bash
promptfoo eval -o results.json
promptfoo share  # get a shareable URL
```