---
topic: ci-cd
source_keys:
  - context7-promptfoo-dev
  - context7-promptfoo-github
---

## How CI integration works

`promptfoo eval` exits with a non-zero code when any assertion fails. This makes it a natural CI gate — a failing eval blocks a merge just like a failing test suite.

## GitHub Actions — promptfoo-action

The official `promptfoo/promptfoo-action@v1` action runs an eval on every pull request and posts results as a PR comment.

```yaml
# .github/workflows/llm-eval.yml
name: LLM Eval

on:
  pull_request:
  workflow_dispatch:

jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: promptfoo/promptfoo-action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          config: promptfooconfig.yaml
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
```

The `github-token` is required for the action to post PR comments. Without it the eval still runs but results are not surfaced in the PR UI.

## Generic CI (npx)

Any CI system that can run Node.js commands can use Promptfoo:

```yaml
# Any CI provider
- name: Run promptfoo eval
  run: npx promptfoo@0.121.17 eval --no-cache
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
```

`--no-cache` ensures all LLM calls are fresh — important in CI where cached results from a developer's machine would not be present.

## Google Cloud / Vertex AI

```yaml
# .github/workflows/llm-test.yml
steps:
  - uses: google-github-actions/auth@v2
    with:
      credentials_json: ${{ secrets.GCP_CREDENTIALS }}
  - name: Run promptfoo tests
    run: npx promptfoo@0.121.17 eval
    env:
      GOOGLE_CLOUD_PROJECT: ${{ vars.GCP_PROJECT_ID }}
      GOOGLE_CLOUD_LOCATION: us-central1
```

## MCP security testing workflow

```yaml
# .github/workflows/security-test.yml
name: MCP Security Testing

on: [push, pull_request]

jobs:
  security-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'
      - run: npm install
      - run: npm run build:all-servers
      - name: Run security tests
        run: |
          npx promptfoo eval -c security-tests/scenario1.yaml
          npx promptfoo eval -c security-tests/scenario2.yaml
```

## Model security scanning (SARIF output)

For repositories that include model files, scan them in CI and upload results to GitHub's security tab:

```yaml
- name: Install dependencies
  run: |
    npm install -g promptfoo
    pip install modelaudit

- name: Scan models
  run: |
    promptfoo scan-model ./models/ \
      --strict \
      --no-write \
      --format sarif \
      --output model-scan-results.sarif

- name: Upload SARIF
  uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: model-scan-results.sarif
```

## GitLab CI and Jenkins

Both are supported. Use `npx promptfoo@0.121.17 eval` as the test command. No platform-specific action is needed — the exit code gates the pipeline natively.

## Recommended CI practices

- Always pass `--no-cache` in CI to avoid stale results
- Store API keys as CI secrets, never hardcode them
- Run evals on PR branches to catch regressions before merge
- Use `outputPath: results.json` and archive the artifact for debugging failed runs
- Set `evaluateOptions.maxConcurrency` low (2–5) in CI to avoid provider rate limits