Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

chore: remove stale setup scripts, skills, and evals

Browse Source 
Remove artifacts from pre-commit migration and cleanup:
- scripts/setup-gitleaks.sh, scripts/gitleaks.toml: legacy setup scripts
- tests/test-setup-gitleaks.sh, tests/test-setup-hooks.sh: phantom test files
- plugins/bin/skills/gitleaks/: skill for deprecated shell-based setup
- plugins/bin/evals/cross-cutting/gitleaks/: eval for deleted skill
- plugins/bin/evals/cross-cutting/neuledge-context/: out-of-scope eval
- plugins/bin/evals/implement/write-docs/: incomplete eval
- package.json, package-lock.json: markdownlint dependencies (unused)

All validation now managed by .pre-commit-config.yaml.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Defame1297
2026-06-27 19:04:53 +00:00
parent 4cbc993af4
commit 5b8b6f529c
12 changed files with 0 additions and 1818 deletions
Download Patch File Download Diff File Expand all files Collapse all files

84
plugins/bin/evals/cross-cutting/gitleaks/eval.yaml
View File

@@ -1,84 +0,0 @@
skill_name: gitleaks
trigger_tests:
- id: explicit-trigger-install
name: Explicit trigger — install and configure
query: "set up gitleaks in this repo"
should_trigger: true
- id: explicit-trigger-update-hook
name: Explicit trigger — update hook
query: "update the gitleaks pre-commit hook"
should_trigger: true
- id: implicit-trigger-false-positive
name: Implicit trigger — suppress false positive in pre-commit hook
query: "my pre-commit hook keeps blocking commits because it thinks my test fixture has an API key, how do I suppress it?"
should_trigger: true
- id: implicit-trigger-scan-history
name: Implicit trigger — audit repo history for secrets
query: "I want to scan my entire git history to make sure no credentials were ever committed"
should_trigger: true
- id: negative-trigger-security-review
name: Negative trigger — general code security review
query: "do a security review of this pull request"
should_trigger: false
- id: negative-trigger-ci-scan
name: Negative trigger — CI pipeline secret scanning
query: "add secret scanning to my GitHub Actions CI pipeline"
should_trigger: false
- id: negative-trigger-other-tool
name: Negative trigger — different secret scanning tool
query: "trufflehog is flagging false positives in my repo, how do I configure it?"
should_trigger: false
output_tests:
- id: output-states-intent-before-running
name: Deterministic — states intent and waits for confirmation before running setup script
type: deterministic
prompt: "set up gitleaks in /home/user/myproject"
expected_output: >
The skill states what it is about to do (run setup-gitleaks.sh, which will modify
.git/hooks/pre-commit and seed .gitleaks.toml) and asks for confirmation before
executing the script. The confirmation request appears before any Bash command
that runs the script.
assertions:
- "Output contains 'setup-gitleaks.sh' before any Bash tool invocation that runs it"
- "Output contains a confirmation prompt (e.g. 'Proceed?') before running the setup script"
- "Output does not run the setup script without an explicit confirmation step"
- id: output-redacts-secret-values
name: Deterministic — redacts secret value and prescribes rotation before history rewrite
type: deterministic
prompt: "gitleaks found an AWS secret key on line 42 of config/settings.py — it says the value is AKIAIOSFODNN7EXAMPLE. What do I do?"
expected_output: >
The skill does not echo or repeat the secret value. It references the finding by
rule ID, file path, and line number only. It states that the secret must be rotated
immediately. It mentions git filter-repo (not git filter-branch) for history rewrite
and asks for user confirmation before running the rewrite.
assertions:
- "Output does not contain the literal string 'AKIAIOSFODNN7EXAMPLE'"
- "Output mentions rotating or revoking the credential as the first action"
- "Output references 'git filter-repo' for history rewrite, not 'git filter-branch'"
- "Output states that user confirmation is required before running the history rewrite"
- id: output-quality-allowlist-guidance
name: LLM-rubric — allowlist guidance is correct, version-aware, and minimal
type: llm-rubric
prompt: "gitleaks keeps flagging my docs/research/ directory as containing secrets, how do I suppress it?"
expected_output: >
High-quality output checks the installed gitleaks version before prescribing any
TOML syntax, recommends a path-based allowlist entry in .gitleaks.toml (not a
.gitleaksignore fingerprint), uses the correct TOML syntax for the detected version,
adds only the minimal allowlist entry needed for the identified false positive, and
includes a verification step (re-run gitleaks dir -v or gitleaks dir --log-level debug)
after making the change.
assertions:
- "Output checks or asks about the gitleaks version before writing TOML syntax"
- "Output recommends a path-based allowlist entry in .gitleaks.toml rather than .gitleaksignore"
- "Output includes a command to verify the suppression works after the change"
- "Output explains why .gitleaksignore fingerprints are fragile (line numbers shift)"

97
plugins/bin/evals/cross-cutting/neuledge-context/eval.yaml
View File

@@ -1,97 +0,0 @@
skill_name: neuledge-context
trigger_tests:
- id: explicit-install-register
name: "Explicit trigger — install and register"
query: "install @neuledge/context and register it as an MCP server in Claude Code"
should_trigger: true
- id: explicit-package-management
name: "Explicit trigger — package management"
query: "install the react documentation package using neuledge context"
should_trigger: true
- id: implicit-offline-docs
name: "Implicit trigger — offline docs for AI agent"
query: "I need React and Next.js docs available to my AI agent without web searches"
should_trigger: true
- id: negative-different-mcp
name: "Negative — different MCP server"
query: "Add the Gitea MCP server to Claude Code"
should_trigger: false
- id: negative-query-existing
name: "Negative — querying an already-running server"
query: "How do I query React docs using the context server that's already running?"
should_trigger: false
- id: negative-cursor-setup
name: "Negative — different provider"
query: "Set up context serve for Cursor"
should_trigger: false
output_tests:
- id: install-script-announced
name: "Install — script announced before running, version verified after"
type: deterministic
prompt: "install @neuledge/context"
expected_output: >
The skill announces that it will run scripts/setup-neuledge-context.sh before executing it,
then verifies the installation by running `context --version`.
assertions:
- "Output mentions 'scripts/setup-neuledge-context.sh' before any install command is run"
- "Output includes a `context --version` call after the install step"
- "Output does not contain `npm install -g @neuledge/context@latest` (no floating @latest)"
- id: mcp-list-before-add
name: "MCP registration — list checked before add, skipped if present"
type: deterministic
prompt: "register @neuledge/context as a Claude Code MCP server"
expected_output: >
The skill runs `claude mcp list` and checks for an existing 'context' entry before
running `claude mcp add`. If already registered, the add step is skipped.
assertions:
- "Output includes `claude mcp list` before `claude mcp add context`"
- "Output states that registration is skipped when the server is already present"
- "The `claude mcp add` command uses stdio form: `claude mcp add context -- context serve`"
- id: auth-chmod-paired
name: "Auth — secure-context-config.sh run immediately after auth add"
type: deterministic
prompt: "add auth credentials for docs.example.com to neuledge context"
expected_output: >
The skill runs `context auth add docs.example.com` with an environment variable reference
for the credential, then immediately runs scripts/secure-context-config.sh.
No credential value appears in the output.
assertions:
- "Output references an environment variable (e.g. $TOKEN) rather than a literal credential value"
- "Output runs `scripts/secure-context-config.sh` in the same step as or immediately after `context auth add`"
- "No bearer token, cookie value, or other credential string appears in the output"
- id: git-check-url-source
name: "context add — git prerequisite checked for URL sources only"
type: deterministic
prompt: "add documentation from https://github.com/prisma/prisma using context add"
expected_output: >
Before running `context add`, the skill checks `git --version` because the source is a
GitHub URL. The check is present for URL/repo sources and absent for local .db file paths.
assertions:
- "Output includes `git --version` before the `context add https://github.com/...` command"
- "If given a local .db file path instead, the git check is absent"
- id: install-flow-quality
name: "Full install + register flow quality"
type: llm-rubric
prompt: "install @neuledge/context and set it up as my Claude Code MCP server"
expected_output: >
A complete, ordered install-then-register flow: (1) announce the install script,
(2) run setup-neuledge-context.sh, (3) verify with context --version,
(4) check claude mcp list, (5) run claude mcp add if not already registered,
(6) confirm with claude mcp list. Steps are in the correct order with verification
between install and registration.
assertions:
- "Install step comes before MCP registration step"
- "A verification command (context --version) appears between install and registration"
- "The output would leave a user with a working @neuledge/context MCP server in Claude Code"
- "No step is skipped without an explanation of why it was skipped"

61
plugins/bin/evals/implement/write-docs/eval.yaml
View File

@@ -1,61 +0,0 @@
skill_name: write-docs
trigger_tests:
- id: explicit-trigger-document-module
name: "Explicit trigger — document a script"
query: "Write documentation for the install.sh script"
should_trigger: true
- id: explicit-trigger-create-docs
name: "Explicit trigger — create docs for a feature"
query: "Create docs for this feature"
should_trigger: true
- id: implicit-trigger-readme-update
name: "Implicit trigger — outdated README section, no trigger phrase"
query: "We need to update the README section for the auth module, the current one is outdated"
should_trigger: true
- id: negative-trigger-prd
name: "Negative — PRD request should route to to-prd"
query: "Write a PRD for the new logging feature"
should_trigger: false
- id: negative-trigger-write-skill
name: "Negative — skill authoring request should route to write-skill"
query: "Write a skill for generating documentation automatically"
should_trigger: false
- id: negative-trigger-skill-file
name: "Negative — SKILL.md update (skill files are self-describing)"
query: "Document how the write-docs skill works by updating its SKILL.md"
should_trigger: false
output_tests:
- id: output-proposes-files-before-reading
name: "Deterministic — candidates proposed or approval sought before reading files"
type: deterministic
prompt: "Write documentation for the config module"
expected_output: "Skill proposes candidate files or asks the user to name specific files before reading any file content"
assertions:
- "Response proposes candidate file paths or asks the user to confirm which files to read before showing any extracted content"
- "Response does not display extracted code content or API surface without first receiving file approval"
- id: output-gap-check-present
name: "Deterministic — gap check step present before drafting"
type: deterministic
prompt: "Write documentation for the install.sh script, audience: developer"
expected_output: "Skill presents extracted behaviour to the user and asks them to fill gaps before drafting any section"
assertions:
- "Response includes a gap check step that presents extracted behaviour and asks what the code does not explain"
- "Response does not skip directly to a drafted documentation section without presenting extracted content first"
- id: output-never-invents-behaviour
name: "LLM rubric — no invented behaviour, all claims sourced"
type: llm-rubric
prompt: "Document the src/config.py file for internal developers"
expected_output: "Documentation where every claim is attributed to code content or explicit user input, with no invented explanations, assumptions about intent, or unverifiable behaviour claims."
assertions:
- "The skill explicitly derives each documented claim from a named source — a code line, spec section, or user statement — and does not add claims without attribution"
- "The skill does not include descriptions of caller intent, design rationale, or future behaviour that are not present in the source material"
- "If a behaviour is undocumentable (internal detail with no public spec), the skill notes it as out-of-scope rather than inventing an explanation"

15
plugins/bin/skills/gitleaks/META.md
View File

@@ -1,15 +0,0 @@
```yaml
version: "1.0"
updated: 2026-06-20
when: >
Invoked when the user wants to install gitleaks and wire it as a git pre-commit secret
scanner, update the hook in an existing repo, tune allowlist rules to suppress false
positives, debug a scan finding, or rotate a real secret that was found. Covers the full
lifecycle: install → configure → maintain → remediate. Not invoked for general code
security review (security-review skill) or CI pipeline secret scanning (write-ci-pipeline skill).
references:
- https://github.com/gitleaks/gitleaks/releases/tag/v8.24.2
- https://github.com/gitleaks/gitleaks/blob/main/README.md
```

111
plugins/bin/skills/gitleaks/SKILL.md
View File

@@ -1,111 +0,0 @@
---
name: gitleaks
description: Use when the user wants to install gitleaks, wire it as a git pre-commit secret scanner, update the hook in an existing repo, tune allowlist rules, resolve false positives, or debug a gitleaks scan finding. Do NOT use when the user wants a general security review of code (use security-review), wants to add secret scanning to a CI pipeline (use write-ci-pipeline), or is asking about a different secret scanning tool such as trufflehog or git-secrets.
metadata:
category: cross-cutting
allowed-tools:
- Bash
- Read
- Edit
---
<requirements>
## Required inputs
- **Target repo path** — absolute path to the git repository to configure; inferred from current working directory if not stated, ask if ambiguous
- **Task type** — install/configure, update hook, tune allowlist, debug finding; inferred from the user's request
## Constraints
- Always state what you are about to do before running `setup-gitleaks.sh` — the script modifies `.git/hooks/pre-commit` and seeds `.gitleaks.toml`
- Never modify `.gitleaks.toml` if the user has not asked for allowlist changes — it is project-owned once seeded; treat it as user-controlled config
- Never run `gitleaks git` or `gitleaks dir` across the full history without warning the user it may be slow on large repos
- Redact any secret values that appear in gitleaks output before showing them to the user — show the rule ID, file, and line number only
- When the installed gitleaks version is unknown, check it with `gitleaks version` before suggesting config syntax — v8.24.2 uses `[allowlist]`; v8.25.0+ uses `[[allowlists]]`
- False positive suppression: prefer path-based allowlists in `.gitleaks.toml` over fingerprint-based entries in `.gitleaksignore` — fingerprints are line-number-sensitive and break on file edits
</requirements>
<steps>
## Process
### Install and configure
1. **Confirm target.** State: "I will run `scripts/setup-gitleaks.sh <path>` which will install gitleaks (if absent), seed `.gitleaks.toml` (first run only), and write the pre-commit hook. Proceed?" Wait for confirmation — this modifies the repo's git hook.
2. **Run setup script.** Execute from the ai-development repo root:
```
bash scripts/setup-gitleaks.sh <TARGET_REPO>
```
The script is idempotent — it replaces the gitleaks block in the hook on every run without disturbing other hook content.
3. **Verify installation.** Run `gitleaks version` to confirm the binary is available. Run `gitleaks git --staged --redact -v` in the target repo to confirm the hook would work on a staged commit (add a dummy change if needed to test).
4. **Commit `.gitleaks.toml`.** Remind the user that `.gitleaks.toml` belongs in version control so all contributors share the same allowlist rules.
### Update hook
Re-run `bash scripts/setup-gitleaks.sh <TARGET_REPO>` from the ai-development repo root. The managed block (delimited by `# managed by setup-gitleaks.sh` / `# end gitleaks` markers) is always replaced with the current version. Non-gitleaks hook content is preserved.
### Tune allowlist / resolve false positives
1. **Identify the false positive.** Run `gitleaks dir --log-level debug <path>` to see which rule fired and which allowlist entries (if any) are already active.
2. **Check the gitleaks version.** Run `gitleaks version`. Use `[allowlist]` syntax for v8.24.2; use `[[allowlists]]` syntax for v8.25.0+. Using the wrong syntax silently produces no errors but the allowlist does nothing — this is the most common configuration trap.
3. **Choose suppression strategy.** Read `.gitleaks.toml` first. See `references/allowlist-patterns.md` for syntax examples and when to use each approach:
- Path regex in `[allowlist]` — for files that can never contain real secrets (research notes, terminal captures, test fixtures). Preferred.
- Stopwords in `[allowlist]` — for placeholder patterns like "example", "changeme".
- `disabledRules` in `[extend]` — to disable a noisy default rule entirely. Use only when the rule has no value for this repo.
- `.gitleaksignore` fingerprint — last resort; breaks when the file is edited because line numbers shift.
4. **Edit `.gitleaks.toml`.** Add the minimal allowlist entry needed. Do not suppress more than the identified false positive.
5. **Verify.** Re-run `gitleaks dir -v <path>` or `gitleaks git -v` to confirm the false positive is suppressed and no real findings are hidden.
### Scan modes
| Mode | Command | When to use |
|---|---|---|
| Staged changes (pre-commit) | `gitleaks git --staged --redact -v` | What the hook runs |
| Full commit history | `gitleaks git -v` | Audit existing repo history |
| Working directory files | `gitleaks dir -v <path>` | Scan uncommitted files |
| Debug allowlists | `gitleaks dir --log-level debug <path>` | See which files are skipped and which allowlists fire |
### Resolve a real finding
1. Do not redact or show the secret value. Reference the rule ID, file, and line number only.
2. The secret is compromised the moment it was committed — rotate it immediately, regardless of whether the commit is reachable from the public remote.
3. Remove the secret from history using `git filter-repo` (not `git filter-branch`). This is a history-rewrite — confirm with the user before running. Force-push to all remotes after rewriting.
4. Add the file path to the `.gitleaks.toml` allowlist only if the file is known to be a false-positive source going forward (e.g. a test fixture). Do not add an allowlist entry to suppress a real finding that has been removed.
## Output format
No structured output file. The skill produces:
- Modified `.git/hooks/pre-commit` in the target repo (via the setup script)
- Modified `.gitleaks.toml` in the target repo (allowlist changes only, when requested)
- Terminal confirmation of what was changed and what to do next
</steps>
<checks>
## Failure handling
- `setup-gitleaks.sh` not found — stop; instruct the user to run from the ai-development repo root at `/root/ai-development/`
- Target path is not a git repository — report the error from the script and ask the user to confirm the correct path
- `gitleaks` binary not installed and download fails — report the curl/network error; direct the user to manual install at `https://github.com/gitleaks/gitleaks/releases`
- Wrong TOML syntax for installed version — detect via `gitleaks version`, show the correct syntax for that version, do not guess
## Self-check
- [ ] Target repo confirmed before running the setup script
- [ ] `gitleaks version` checked before writing any `.gitleaks.toml` allowlist syntax
- [ ] Secret values in scan output redacted before displaying to the user
- [ ] `.gitleaks.toml` edits are minimal — only the identified false positive suppressed
- [ ] After any allowlist change: re-ran scan to verify suppression works and no real findings are hidden
- [ ] For real findings: rotation step stated before history rewrite, user confirmed history rewrite before running `git filter-repo`
</checks>

83
plugins/bin/skills/gitleaks/references/allowlist-patterns.md
View File

@@ -1,83 +0,0 @@
# Gitleaks allowlist patterns
## Version syntax
| Version | Allowlist syntax |
|---|---|
| v8.24.2 and earlier | `[allowlist]` (singular table) |
| v8.25.0 and later | `[[allowlists]]` (array of tables) |
**Critical**: using the wrong syntax produces no error but the allowlist silently does nothing. Always check `gitleaks version` first.
## v8.24.2 syntax (this repo uses 8.24.2)
### Suppress by path regex
Use for files that can never contain real secrets (research notes, terminal captures, test fixtures, generated docs).
```toml
[allowlist]
description = "research notes and terminal captures"
paths = [
'''docs/research/.*''',
'''tests/fixtures/.*''',
]
```
### Suppress by stopword
Use for placeholder values that match secret patterns but are clearly not real.
```toml
[allowlist]
description = "placeholder values"
stopwords = ["example", "placeholder", "changeme", "your-api-key-here"]
```
### Disable a default rule entirely
Use only when a rule has no value for this repo and produces pervasive false positives.
```toml
[extend]
useDefault = true
disabledRules = ["generic-api-key"]
```
## v8.25.0+ syntax (for reference)
```toml
[[allowlists]]
description = "research notes"
paths = ['''docs/research/.*''']
[[allowlists]]
description = "placeholder values"
stopwords = ["example", "placeholder"]
```
## .gitleaksignore (fingerprint-based — last resort)
```
# Format: <fingerprint>:<line-number>
# Generated by: gitleaks git -v --report-format json | jq -r '.[] | "\(.Fingerprint):\(.StartLine)"'
abc123def456:42
```
Avoid this approach: fingerprints embed line numbers. Any edit to the file shifts line numbers and invalidates the entry, re-surfacing the false positive.
## Verification after any change
```bash
# Scan current files
gitleaks dir -v .
# Scan with debug output to see which allowlists fired
gitleaks dir --log-level debug .
# Scan commit history
gitleaks git -v
# Scan only staged changes (what the pre-commit hook runs)
gitleaks git --staged --redact -v
```