Defame1297
ede3f06689
Diffing each retrofitted SKILL.md against its replacement references/ files found rules that existed on main and now existed nowhere — relocated in intent, deleted in fact. A trim that loses a rule is not progressive disclosure, it is data loss with a smaller word count. Three had no survivor. The least-privilege guidance for `tools` kept its mechanics and lost the "restrict to what the agent needs" half, so the remaining text read as encouragement to omit the field. The improve flow lost its regression check, so nothing compared the closing audit against the pre-edit state and a PASS quietly becoming a SUGGESTION went unnoticed — restored on both halves of the author pair, since agent-author had dropped its equivalent too. And agent bodies lost "would the agent get this wrong without it?", which mattered more than it looks: ADR-0020 deliberately sets no body word gate for agents, three of the four already sit between 933 and 1,199 words, and the delegation check only fires on procedure a skill already owns. That heuristic was the only brake left. Two more were reachable only from the wrong scope. agent-author tells the reader to load only the file for the resolved scope, but the mcp__ glob syntax for disallowedTools and the five tools no subagent ever receives had both landed in project-user-scope.md. disallowedTools is the ONLY permitted fence at plugin/APM scope, so the scope that needs the syntax most could not reach it, and a plugin-scope run could write a body telling the agent to ask the user a question. Two documents were actively wrong rather than merely thin. agent-audit told auditors that validate.sh resolves boundary targets for skills only; it runs at both scopes, so the auditor was hand-resolving what the script had already decided and could contradict it. And skill-audit routed to its script-troubleshooting reference whenever validate.sh "fails" — but it exits 1 on ordinary content FAILs, the normal outcome for the whole #99 population, so 1,302 words loaded on nearly every audit. A context-budget regression inside the skill that enforces the context budget. Finally, two illustrations taught the shape the gate ERRORs on, unfenced, while an adjacent rubric called it a hard ERROR. LESSONS.md records the reference-chain depth rule flipping from "one level deep" to "two hops, never three". ADR-0020 is silent on it and the reversal rode entirely on the diff; the looser rule is what mandatory dispatch requires. Refs: #99 ADR: 0020 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015W3iwF9ncfRZddGBxsMCYi
4.5 KiB
name, description, allowed-tools, metadata
| name | description | allowed-tools | metadata | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| skill-audit | Use when the user wants a skill directory audited against the agentskills.io spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or after hand-editing a skill outside skill-author. Not applying fixes -> skill-author. | Bash Read |
|
Gotchas
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
- A skill carrying
disable-model-invocation: trueis hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead. validate.shreports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the body only. A skill can sit inside one and fail the other — report them separately.- Vale reporting
0 filesscanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
Step 1 — Deterministic checks
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
bash scripts/validate.sh <skill-dir>
bash scripts/validate-provenance.sh <skill-dir>
scripts/vale-wrap.sh <skill-dir>/SKILL.md
validate.sh findings become the ### Structure dimension — its FAILs and its SUGGESTIONs both.
If any of the three cannot run, or exits non-zero for a reason other than findings, read references/validation-scripts.md — it carries the manual fallback and the misleading exit codes. Ordinary content FAILs are the expected outcome here and need no fallback.
validate-provenance.sh prints nothing on success. Its FAIL and INFO findings become a separate ### Provenance dimension, and it emits Why and Fix itself — surface those verbatim.
vale-wrap.sh applies the bundled Kyberforge style as a prefilter. Pass no --config; the wrapper locates its own. Every rule is graded error, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
| Rule | Dimension |
|---|---|
Kyberforge.DescriptionOpener, Kyberforge.CompositionNote, Kyberforge.VagueWording |
description |
Kyberforge.SentenceOpenerThereIs |
body-discipline |
Kyberforge.PaddingPhrase |
patterns |
Step 2 — Read the whole skill
Read SKILL.md, README.md, and every text file under scripts/, references/, assets/ and tests/. Skip binaries only — internal-consistency findings need the full picture.
Step 3 — Qualitative audit
Load a dimension's rubric before judging that dimension. Each is self-contained, and each is grounded in the agentskills.io specification plus the house context-budget contract (ADR-0020).
| Dimension | Read |
|---|---|
| description | references/description-quality.md |
| body-discipline | references/body-discipline.md |
| patterns | references/patterns.md |
| file-structure, internal-consistency | references/file-structure.md |
| formatting, scripts | references/formatting-and-scripts.md |
Cite file and line number for every finding.
Step 4 — Report
Open with a coverage line naming every dimension checked:
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
Each finding:
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
Close with a ## Result block holding one line: PASS, PASS (N suggestions), or FAIL (N fails · M suggestions), each optionally followed by · P info. INFO findings are observational and never change PASS/FAIL; omit · P info when there are none. Add a second line, Run skill-author to address findings., whenever there is at least one finding. Do not apply fixes — report and propose only.