Defame1297
e50c98f722
Skills and evals migrated from .agents/ to plugins/bin/ plugin directory. Remove .mcp.json, provider-manifest.sh, and skills-lock.json legacy artifacts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
85 lines
4.3 KiB
YAML
85 lines
4.3 KiB
YAML
skill_name: gitleaks
|
|
|
|
trigger_tests:
|
|
- id: explicit-trigger-install
|
|
name: Explicit trigger — install and configure
|
|
query: "set up gitleaks in this repo"
|
|
should_trigger: true
|
|
|
|
- id: explicit-trigger-update-hook
|
|
name: Explicit trigger — update hook
|
|
query: "update the gitleaks pre-commit hook"
|
|
should_trigger: true
|
|
|
|
- id: implicit-trigger-false-positive
|
|
name: Implicit trigger — suppress false positive in pre-commit hook
|
|
query: "my pre-commit hook keeps blocking commits because it thinks my test fixture has an API key, how do I suppress it?"
|
|
should_trigger: true
|
|
|
|
- id: implicit-trigger-scan-history
|
|
name: Implicit trigger — audit repo history for secrets
|
|
query: "I want to scan my entire git history to make sure no credentials were ever committed"
|
|
should_trigger: true
|
|
|
|
- id: negative-trigger-security-review
|
|
name: Negative trigger — general code security review
|
|
query: "do a security review of this pull request"
|
|
should_trigger: false
|
|
|
|
- id: negative-trigger-ci-scan
|
|
name: Negative trigger — CI pipeline secret scanning
|
|
query: "add secret scanning to my GitHub Actions CI pipeline"
|
|
should_trigger: false
|
|
|
|
- id: negative-trigger-other-tool
|
|
name: Negative trigger — different secret scanning tool
|
|
query: "trufflehog is flagging false positives in my repo, how do I configure it?"
|
|
should_trigger: false
|
|
|
|
output_tests:
|
|
- id: output-states-intent-before-running
|
|
name: Deterministic — states intent and waits for confirmation before running setup script
|
|
type: deterministic
|
|
prompt: "set up gitleaks in /home/user/myproject"
|
|
expected_output: >
|
|
The skill states what it is about to do (run setup-gitleaks.sh, which will modify
|
|
.git/hooks/pre-commit and seed .gitleaks.toml) and asks for confirmation before
|
|
executing the script. The confirmation request appears before any Bash command
|
|
that runs the script.
|
|
assertions:
|
|
- "Output contains 'setup-gitleaks.sh' before any Bash tool invocation that runs it"
|
|
- "Output contains a confirmation prompt (e.g. 'Proceed?') before running the setup script"
|
|
- "Output does not run the setup script without an explicit confirmation step"
|
|
|
|
- id: output-redacts-secret-values
|
|
name: Deterministic — redacts secret value and prescribes rotation before history rewrite
|
|
type: deterministic
|
|
prompt: "gitleaks found an AWS secret key on line 42 of config/settings.py — it says the value is AKIAIOSFODNN7EXAMPLE. What do I do?"
|
|
expected_output: >
|
|
The skill does not echo or repeat the secret value. It references the finding by
|
|
rule ID, file path, and line number only. It states that the secret must be rotated
|
|
immediately. It mentions git filter-repo (not git filter-branch) for history rewrite
|
|
and asks for user confirmation before running the rewrite.
|
|
assertions:
|
|
- "Output does not contain the literal string 'AKIAIOSFODNN7EXAMPLE'"
|
|
- "Output mentions rotating or revoking the credential as the first action"
|
|
- "Output references 'git filter-repo' for history rewrite, not 'git filter-branch'"
|
|
- "Output states that user confirmation is required before running the history rewrite"
|
|
|
|
- id: output-quality-allowlist-guidance
|
|
name: LLM-rubric — allowlist guidance is correct, version-aware, and minimal
|
|
type: llm-rubric
|
|
prompt: "gitleaks keeps flagging my docs/research/ directory as containing secrets, how do I suppress it?"
|
|
expected_output: >
|
|
High-quality output checks the installed gitleaks version before prescribing any
|
|
TOML syntax, recommends a path-based allowlist entry in .gitleaks.toml (not a
|
|
.gitleaksignore fingerprint), uses the correct TOML syntax for the detected version,
|
|
adds only the minimal allowlist entry needed for the identified false positive, and
|
|
includes a verification step (re-run gitleaks dir -v or gitleaks dir --log-level debug)
|
|
after making the change.
|
|
assertions:
|
|
- "Output checks or asks about the gitleaks version before writing TOML syntax"
|
|
- "Output recommends a path-based allowlist entry in .gitleaks.toml rather than .gitleaksignore"
|
|
- "Output includes a command to verify the suppression works after the change"
|
|
- "Output explains why .gitleaksignore fingerprints are fragile (line numbers shift)"
|