Files
holocron/plugins/kyberforge/skills/skill-audit/SKILL.md
Defame1297 55dc065644 fix(lint): resolve round-1 and round-2 review findings on the Vale prefilter
Addresses PR #85's outstanding review items after grilling the open
questions against ADR-0013/CONTEXT.md/ADR-0010:

Blocking fixes:
- vale-wrap.sh: replace json.dumps() escaping (which silently defeated
  Vale's frontmatter scope on any description containing a quote,
  backslash, or non-ASCII char — ~58% of the corpus) with a single-quoted
  YAML scalar, substituting a Unicode right single quote for embedded
  apostrophes rather than '' doubling (Vale's frontmatter scanner isn't a
  full YAML parser and silently truncates on '' too).
- vale-wrap.sh: fix a blank-line-inside-a-folded-description truncation
  bug via indentation-based, blank-line-tolerant body capture; narrow
  flattening to `>`-style scalars only (`|` already works unflattened).
- skill-audit/agent-audit Step 1: make the vale-wrap.sh invocation
  cwd-independent via git rev-parse --show-toplevel, fixing a bug where
  no single cwd satisfied all three Step 1 commands.
- styles/Kyberforge/VagueQualifier.yml: prune 17 tokens verified
  false-positive-dominated on this repo's own voice via a real corpus
  sweep (obvious, clearly, usually, several, simple, easy, completely,
  simply, tiny, etc.), keep 13 with real or unattested noise. Revert the
  28 prose "fixes" those tokens drove across 14 skill files back to their
  original, correct wording, including a functional regression to
  caveman/SKILL.md's own filler-word list (a mention, not a use) — now
  guarded with vale-off comments against recurrence.

Gaps:
- --minAlertLevel=warning on the pre-commit hook and Step 1 invocation
  so warning-level rules actually surface, without collapsing the
  FAIL/SUGGESTION severity mapping skill-audit/agent-audit rely on.
- vale-wrap.sh: fix --config=<path> equals-form, absolute-path silent
  no-op, and a zero-file-argument stdin hang.
- Route vale-run and lint-runner through a documented wrapper script
  when a target repo has one, instead of unconditionally recommending
  bare `vale`.
- Wire Kyberforge.VagueQualifier/SentenceOpenerThereIs into skill-audit/
  agent-audit's dimension-mapping prose (Body discipline).
- Add plugins/lint/sources.md provenance for lint-runner (ADR-0010).
- Sync both marketplace.json lint-entry descriptions with plugin.json.
- Retune skill-size-check.sh's MAX_WORDS 5000->2900 (measured ~1.6-1.7
  tokens/word on this repo's corpus, the old value gated at ~8,500
  tokens against a stated 5,000 ceiling); fix the >/>= line-count
  boundary and wc -l undercount on files with no trailing newline.
- Document the vale binary as a Setup prerequisite in AGENTS.md.
- Fix SentenceOpenerThereIs's dead regex alternative and add a real
  sentence-start anchor/scope.
- Fix a stale docs/research/docs/vale/ index pointer in kyberforge's
  docs README (moved to plugins/lint/ in e1a5403).
- Rewrite ADR-0013's Consequences section past-tense to describe what
  actually landed, and record the styles-portability limitation
  (repo-root placement stays intentional; deferred to a separate
  session per this PR's review).

Test coverage: 9 new vale-wrap.sh fixtures (quotes, backslash/unicode,
blank-line paragraphs, --config= form, zero-arg/absolute-path handling,
literal-block no-regression) and boundary-pair tests for
skill-size-check.sh's line/word ceilings.

bash tests/run-tests.sh: 9 scripts + 125 bats assertions, all passing.
scripts/check-manifests.sh and claude plugin validate --strict: clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
2026-08-08 19:24:40 +00:00

153 lines
9.4 KiB
Markdown

---
name: skill-audit
description: >
Use when the user wants to review a skill they wrote, says "audit this skill",
"check if my skill follows best practices", "review my SKILL.md", or wants to
know if a skill is ready to ship — even if they don't use the word "audit".
Also invoke proactively after directly hand-editing a skill's files outside
skill-author — an unaudited hand-edit is the same risk as unreviewed code.
Audits a skill directory against the agentskills.io specification — structural
checks plus qualitative review of description quality, body discipline, patterns,
formatting, file structure, scripts, and internal consistency, plus a provenance
chain check. Produces a compact findings report
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
handoff to /skill-improve or human auditability. Do not use to fix application
code bugs or perform general code review unrelated to skill quality.
Do not use when the user wants improvements applied — use /skill-improve instead.
allowed-tools: Bash Read
metadata:
category: factory
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
---
## Gotchas
- Do not output PASS/FAIL per check while auditing — gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
## Step 1 — Structural validation
```bash
bash scripts/validate.sh <skill-dir>
bash scripts/validate-provenance.sh <skill-dir>
"$(git rev-parse --show-toplevel)/scripts/vale-wrap.sh" --config "$(git rev-parse --show-toplevel)/.vale.ini" <skill-dir>/SKILL.md
```
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
`vale-wrap.sh` resolves its own path and `.vale.ini` via `git rev-parse --show-toplevel`, so it runs correctly regardless of the caller's cwd, using `.vale.ini`'s `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns dimensions below, not a replacement for Step 3. Map `error` → `FAIL` and `warning`/`suggestion` → `SUGGESTION` in those dimensions, citing the rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if vale or `.vale.ini` is unavailable.
## Step 2 — Read all skill files
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture.
## Step 3 — Qualitative audit
Work through each dimension internally. Collect findings only; report them in Step 4. Cite file and line number for every finding.
### Description
Vale's `Kyberforge.DescriptionOpener` (FAIL — "This skill..." openers) and `Kyberforge.VagueWording` (SUGGESTION — filler like "helps with", "utilize") alerts from Step 1 cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
- **Length**: under 1024 characters?
If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`.
### Body discipline
For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding.
- **Defaults not menus**: every decision point gives one default + one escape hatch, not a list of options
- **Why rationale**: include/exclude rules explain why, not just what
- **Control calibration**: prescriptive for fragile or critical sequences (e.g. a script invocation where flag order or exact arguments must not change); flexible where multiple approaches are valid
Vale's `Kyberforge.VagueQualifier` (SUGGESTION — vague filler like "clearly", "obviously") and `Kyberforge.SentenceOpenerThereIs` (SUGGESTION — sentences starting with "There is"/"There are") alerts from Step 1 cover pattern-matchable body-wide filler directly; report them as findings without re-deriving by judgment.
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
### Patterns
Check each pattern is appropriate and correctly formed:
- **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip
- **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block
- **Checklists**: used for multi-step workflows, not single steps
- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment.
- **Output templates**: present when the agent must produce a specific format; absent otherwise
### File structure
- Permitted directories: `scripts/`, `references/`, `assets/`, `tests/`; flag any other unlisted directory as FAIL — the spec allows additional dirs but this skill permits only these four to keep skills focused
- `scripts/` contains only executable code agents can run; test files (`.bats`, `*_test.*`, `test_*.sh`) in `scripts/` are a FAIL — they belong in `tests/`
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
- Optional directories contain real content — not just unfilled placeholder READMEs
- `README.md` present and accurately describes the skill and its files
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
- `references/sources.md` is exempt from the cross-plugin path check — `Research doc:` fields are development-only provenance pointers, not runtime references; they intentionally reference paths outside the skill directory and are expected to be non-resolvable after plugin install; `validate-provenance.sh` handles this gracefully by silently skipping upstream checks when those paths don't resolve
- `tests/` is exempt from the cross-plugin path check — test files are dev-only and may reference repo-level test infrastructure (e.g. a shared `tests/test_helper/`). This dependency must be declared in `tests/README.md`; flag if tests exist but `tests/README.md` is absent or does not document the dependency
### Formatting
- Heading levels consistent: H2 for main sections, H3 for subsections
- Code blocks fenced with a language tag where applicable (`bash`, `markdown`, `python`)
- Consistent whitespace: blank line between sections, consistent list indentation
- No broken relative paths in file references
### Scripts
- No interactive TTY prompts (`read`, `input()`, `readline`)
- `--help` exposed with concise usage
- Data to stdout, diagnostics to stderr
- Idempotent ("create if not exists")
- Meaningful exit codes documented in `--help`
- `--dry-run` present for destructive operations
### Internal consistency
- SKILL.md steps match what scripts actually do
- `README.md` file table lists every file that exists — no missing entries, no stale entries
- Placeholder READMEs in `scripts/`, `references/`, `assets/` consistent with what SKILL.md says about each directory
## Step 4 — Report
Open with a coverage line listing every dimension checked:
```text
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
```
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely — their absence confirms they passed.
For each finding:
```text
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
```
Close with a result block:
```text
## Result
PASS
PASS (N suggestions)
PASS · P info
PASS (N suggestions) · P info
FAIL (N fails · M suggestions)
FAIL (N fails · M suggestions) · P info
Run /skill-improve to address findings.
```
INFO findings are observational — do not affect PASS/FAIL. Omit `· P info` when there are no INFO findings. Omit the `/skill-improve` line when there are no findings at all. Do not apply fixes — report and propose only.