fix(kyberforge): redesign skill-audit output format
Replace verbose three-pass output (punch list + priority table + fix proposals) with a compact findings-only report: coverage line, findings grouped by dimension with Why+Fix per entry, and a result block with /skill-improve handoff. Suppress PASS lines — absence confirms pass. Fix validate.bats executable bit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -7,7 +7,7 @@ Audit a skill directory against the agentskills.io specification. Runs structura
|
||||
1. Runs `scripts/validate.sh` for structural checks (name format, description length, line count, placeholder detection, script rules)
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across seven dimensions
|
||||
4. Outputs a PASS/FAIL/SUGGESTION punch list with a specific fix proposal for every FAIL
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to /skill-improve
|
||||
|
||||
## Usage
|
||||
|
||||
|
||||
@@ -6,8 +6,9 @@ description: >
|
||||
know if a skill is ready to ship — even if they don't use the word "audit".
|
||||
Audits a skill directory against the agentskills.io specification — structural
|
||||
checks plus qualitative review of description quality, body discipline, formatting,
|
||||
file structure, and internal consistency. Produces a PASS/FAIL/SUGGESTION punch
|
||||
list with a specific fix proposal for every FAIL. Do not use to run evals, fix
|
||||
file structure, and internal consistency. Produces a compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
|
||||
handoff to /skill-improve or human auditability. Do not use to run evals, fix
|
||||
application code bugs, or perform general code review unrelated to skill quality.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
@@ -20,7 +21,7 @@ metadata:
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
```
|
||||
|
||||
List every FAIL from the structural check in the punch list before continuing. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
|
||||
@@ -28,7 +29,7 @@ Read every file in the skill directory: `SKILL.md`, `README.md` (if present), al
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Work through each dimension. Cite file and line number for every finding.
|
||||
Work through each dimension internally. Do not output PASS/FAIL per check — this is internal working. Collect findings only; report them in Step 4. Cite file and line number for every finding.
|
||||
|
||||
### Description
|
||||
|
||||
@@ -92,22 +93,29 @@ Check each pattern is appropriate and correctly formed:
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Output a punch list grouped by dimension:
|
||||
Open with a coverage line listing every dimension checked:
|
||||
|
||||
```
|
||||
PASS/FAIL/SUGGESTION <finding> — <file>:<line>
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency
|
||||
```
|
||||
|
||||
Follow with a priority table:
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely — their absence confirms they passed.
|
||||
|
||||
| Priority | Severity | Finding | File:Line |
|
||||
|----------|----------|---------|-----------|
|
||||
|
||||
Then for each FAIL, a fix proposal:
|
||||
For each finding:
|
||||
|
||||
```
|
||||
FAIL: <finding>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Do not apply fixes. Report and propose only.
|
||||
Close with a result block:
|
||||
|
||||
```
|
||||
## Result
|
||||
|
||||
PASS / PASS (N suggestions) / FAIL (N fails · M suggestions)
|
||||
Run /skill-improve to address findings.
|
||||
```
|
||||
|
||||
Omit the `/skill-improve` line when there are no findings. Do not apply fixes — report and propose only.
|
||||
|
||||
0
plugins/kyberforge/skills/skill-audit/scripts/validate.bats
Normal file → Executable file
0
plugins/kyberforge/skills/skill-audit/scripts/validate.bats
Normal file → Executable file
Reference in New Issue
Block a user