---
topic: assertions
source_keys:
  - context7-promptfoo-dev
  - context7-promptfoo-github
---

## Assertion structure

Each assertion in `assert:` has a `type:`, an optional `value:`, an optional `threshold:`, and an optional `metric:` label.

```yaml
assert:
  - type: contains
    value: 'return policy'
  - type: llm-rubric
    value: 'Response is helpful and professional'
    threshold: 0.8
    metric: quality
```

## String shorthand

Assertions can also be written as compact strings directly in the assert list:

| Shorthand | Full type |
|---|---|
| `Paris` | `equals` |
| `contains:Paris` | `contains` |
| `icontains:paris` | `icontains` (case-insensitive) |
| `starts-with:The answer` | `starts-with` |
| `regex:^Hello.*world$` | `regex` |
| `is-json` | `is-json` |
| `contains-json` | `contains-json` |
| `similar(0.8):Hello world` | `similar` with threshold |
| `llm-rubric:Is helpful and accurate` | `llm-rubric` |
| `grade:Does not mention being an AI` | alias for `llm-rubric` |
| `factuality:Paris is the capital of France` | `factuality` |
| `javascript:output.length < 100` | inline JS |
| `fn:output.includes('hello')` | alias for `javascript` |
| `python:len(output) > 10` | inline Python |
| `file://assertions/custom.js` | external file |
| `levenshtein(5):expected text` | `levenshtein` with distance |
| `not-contains:error` | negated assertion |

## Deterministic assertions

- `equals` — exact string match
- `contains` / `icontains` — substring check (case-sensitive / insensitive)
- `not-contains` — absence check
- `starts-with` — prefix check
- `regex` — regular expression match
- `is-json` — valid JSON
- `contains-json` — valid JSON somewhere in output
- `levenshtein` — edit distance within threshold

## Similarity and semantic assertions

- `similar` — embedding cosine similarity; `threshold:` is a 0–1 score
- `context-faithfulness` — similarity-based RAG faithfulness; `threshold: 0.8` typical

## Model-graded assertions

These send a grader prompt to another LLM (by default the configured judge model) and score the output.

**`llm-rubric`** — open-ended rubric; binary or 0–1 score depending on criteria phrasing. Use `threshold:` to set minimum passing score:
```yaml
- type: llm-rubric
  value: Is not apologetic and provides a clear, concise answer
  threshold: 0.8
```

**`factuality`** — checks whether the output is factually consistent with a reference statement:
```yaml
- type: factuality
  value: The capital of California is Sacramento
```

**`pi`** — custom scoring with any numeric range; requires `threshold:`.

The grader model can be overridden globally in `defaultTest.options.provider` or per-assertion.

## Custom assertions

**JavaScript (inline):**
```yaml
- type: javascript
  value: "output.length < 100 && !output.includes('error')"
```

**JavaScript (file):**
```yaml
- type: javascript
  value: file://assertions/check_format.js
```

**Python:**
```yaml
- type: python
  value: "len(output) > 10 and 'Paris' in output"
```

## Negation

Any assertion type can be negated by prepending `not-`:
```yaml
- type: not-contains
  value: 'error'
- type: not-regex
  value: '\b(fail|broken)\b'
```

## Metrics

The `metric:` field groups assertions for aggregate reporting. All assertions with the same metric name are scored together in the results view:
```yaml
assert:
  - type: contains
    value: 'policy'
    metric: coverage
  - type: llm-rubric
    value: Addresses the user's concern
    metric: quality
```