Defame1297
175ea89c0a
Check 9 shipped in #130 to close #118, but three defects meant it could not do the job it was added for. Why: - It is INFO-only, so it always exits 0 — and SKILL.md graded exit 0 "a genuine pass" and said the script "prints nothing on success". Every check-9 INFO was discarded before it reached a report, behind three further doors that only opened on a non-zero exit. - `parse_field_raw()` matched `(.+)`, which does not span newlines, so only the first physical line of a wrapped value was compared. Rewriting only the continuation line of a wrapped Description from a hedge to a confident claim produced no finding at all — verbatim the regression #118 was filed about. The bullet branch had the same shape: a wrapped bullet broke the loop and dropped every later entry. - A `git show` failure at the base ref was treated as "creation, nothing to flag" and skipped the whole skill with no output, collapsing "absent at that ref" with "not tracked under that name". A gitignored `.claude/skills/` copy reported clean while the authoring path reported four changed claims. The script's own usage text promises this is "never a silent skip". Implementation notes: - Exit-code guidance re-keyed on output as well as code: 0-and-silent passes, 0-with-output is INFO-only findings, 1 is FAILs, 2 never ran. - `parse_field_raw()` is line-based and joins continuation lines; `normalize_field_text()`'s docstring is now true rather than aspirational. A reorder deliberately fires: the two fields share one parser, and order-insensitivity would mean splitting a prose Description on commas. - The discarded `show_err` is now surfaced as one whole-check INFO naming both readings. - `--base-ref=` given empty now beats the env var, as the usage text always claimed. `validate.sh` gains an ADR-0022 `metadata.version` check at FAIL tier, because any lower tier lets skill-author Step 4 report done on a file the commit gate then refuses. Its `read` heuristic now skips here-doc bodies — reflowing the one offending line would have cleared the finding and left the cause, since every usage() heredoc is one wrap from putting the English verb in column 0. Impact: provenance tests 73 -> 82, validate tests 64 -> 72. Test 72 previously deleted origin/main before asserting the override, so it proved the flag works with no default rather than that it beats one; it now moves origin/main forward first. Refs: #118 ADR: 0022 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EeH8SCbcrCAQrtymkNuhKP
6.8 KiB
skill-audit
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
What it does
- Runs
scripts/validate.shandscripts/validate-provenance.shfor structural and provenance checks, plusscripts/vale-wrap.sh— a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers - Reads all files in the skill directory
- Applies qualitative checks across five dimension groups — always loading
references/finding-criteria.md, then one rubric fromreferences/per group the criteria put in play - Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to
skill-author
validate.sh enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words).
Alongside those it runs shape checks that are not length measurements at all. Three are FAILs: every routing target named in the description — in the compressed Not <thing> -> <name> arrow and in the prose form — must resolve to a real skill or agent; every references/<file>.md the body names must exist on disk; and metadata.version must be present and three-part semver (ADR-0022). That last one is FAIL rather than SUGGESTION because the skill-frontmatter pre-commit hook rejects the file without it — an audit grading it lower would report ready-to-ship on a file the commit gate refuses. Three are SUGGESTIONs: a missing boundary clause, a Gotchas section over five entries, and a Gotchas section over 25% of the body. The resolution universe for boundary targets is derived by walking up from the audited SKILL.md — the authoring root above it, its own apm package, and that package's declared apm.yml dependencies — so a fresh clone and a machine that has run apm install return the same verdict. When no universe can be determined the check prints INFO ... DID NOT RUN and does not silently pass.
Usage
/skill-audit
Provide the path to the skill directory to audit when invoking.
Files
| File | Purpose |
|---|---|
SKILL.md |
Skill instructions for agents |
scripts/validate.sh |
Structural validator — checks name format, name matches directory, description presence and length, metadata.version presence and semver shape (ADR-0022), body-only word count, line and whole-file word ceilings, boundary-clause presence, boundary-target resolution, references/ pointer existence, Gotchas entry count and body share, placeholder detection, script executable bit, and interactive-prompt detection |
scripts/validate-provenance.sh |
Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, upstream research doc alignment, and (check 9, INFO only) whether a slug's Description or Contributing files text has changed since a base ref — --base-ref=<ref> or VALIDATE_PROVENANCE_BASE_REF, defaulting to the merge base with origin/main |
scripts/vale-wrap.sh |
Vale prefilter wrapper — runs the bundled Kyberforge Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
assets/vale/.vale.ini |
Vale configuration — points Vale at the bundled Kyberforge style path, self-located relative to vale-wrap.sh |
assets/vale/styles/Kyberforge/CompositionNote.yml |
Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
assets/vale/styles/Kyberforge/DescriptionOpener.yml |
Vale rule — flags non-imperative "This..." description openers |
assets/vale/styles/Kyberforge/PaddingPhrase.yml |
Vale rule — flags generic "see references/" padding phrasing in conditional references |
assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml |
Vale rule — flags body sentences starting with "There is"/"There are" |
assets/vale/styles/Kyberforge/VagueWording.yml |
Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
references/finding-criteria.md |
Every dimension's FAIL and SUGGESTION criteria — the one Step 3 file loaded on every run; it decides which rubrics below are worth loading |
references/description-quality.md |
Rubric for the description dimension — why the description is the expensive part, the hand-invoked (disable-model-invocation) contract, the three-part shape, when an indirect trigger is warranted, near-miss exclusions, and a before/after pair |
references/body-discipline.md |
Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
references/patterns.md |
Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
references/file-structure.md |
Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
references/formatting-and-scripts.md |
Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
references/validation-scripts.md |
Step 1 troubleshooting — the manual structural fallback when validate.sh cannot run, and the script exit codes that are easy to misread (loaded on a script failure, and on any exit-0 run that printed something — validate-provenance.sh's check 9 is INFO-only, so its findings arrive that way) |
references/sources.md |
Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
tests/validate.bats |
(source-only) Bats test suite for validate.sh |
tests/validate-provenance.bats |
(source-only) Bats test suite for validate-provenance.sh |
tests/README.md |
(source-only) Setup instructions for bats-support and bats-assert test dependencies |
Rows marked (source-only) exist in the authoring source (.apm/skills/skill-audit/) but are
not present in an installed plugin: scripts/sync-plugin-content.sh strips
<category>/<name>/tests when it generates the flat mirror, because these are dev-time fixtures no
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.