598a7c326a
Retrofits all 39 skills to ADR-0020's description/body context contract, then fixes what six rounds of independent review found in that retrofit — including four ways the hot gate itself failed open. Closes #99, #107, #108, #110, #111, #114, #115, #120. ## The retrofit (waves 1-5) | | Start | Now | |---|---|---| | Description FAILs (>400 chars) | 26 | **0** | | Body FAILs (>900 words, body-only) | 9 | **0** | | Dangling routing targets | 2 | **0** | | `Kyberforge.CompositionNote` | 10 | **0** | | Preload tax | 21,005 chars | **~10,500** | Under the 12,000-char success criterion. Per-wave detail is on #99. ## The review fixes **The gate failed open four ways, three of them found after the retrofit shipped.** An unrecognised follower token made a dangling target vanish. A skill directory with no `SKILL.md` resolved as a valid target, so a commit could be green locally and red in a fresh clone — three existing fixtures were relying on that, one of which made the install-leak A/B pass vacuously. Then the free-standing `/name` sweep turned out to be gated on the sentence carrying a boundary marker, so route notation in any other sentence was invisible — not an ERROR, not a SUGGESTION, not an INFO — which left the documented "`/name` always blocks" promise false from a second direction. All four fixed and pinned. **Two checks were silently not running.** `validate-provenance.sh` checks 7-8 were dead across nine skills. Waking them exposed a deeper problem: they assume `Research doc:` names a source index, but 30 of 121 entries point at topic content documents, so every new check-7 INFO was a false positive and check 8 was saved from a false-FAIL flood only by an *unannounced* skip. Checks 7/8 are now scoped to source indexes and every skip announces itself (#121). **The retrofit's own anti-goal, four times.** ADR-0020 warns that a blunt gate gets satisfied by deleting content rather than relocating it. `diagnose` and `skill-audit` relocated prose and then read it unconditionally; `prototype` and `vale-config` deleted rules outright that survived nowhere. All four addressed. ## Verification - `bash tests/run-tests.sh --strict` — 24 suites, 0 skipped, 0 failed - `bash tests/run-bats.sh` — 325 tests, 0 failures - `pre-commit run --all-files` — 17/17 - `pre-commit run --hook-stage pre-push --all-files` — 16/16, with `apm marketplace check` and `apm pack --check-clean` run against the remote, not skipped - `scripts/skill-size-check.sh` over all 39 skills — rc 0, 0 ERROR/FAIL, SUGGESTION-only - Preload tax measured at **10,498 chars**, max description 390 — both inside budget - Every new test proven non-vacuous by a deliberate mutation of the behaviour it covers **Per-commit sync, stated accurately:** the ten commits from the latest review round each pass `check-plugin-content-sync` in isolation, verified by checking each out in a detached worktree with a clean between. The earlier gitea window (`dfacf05..bedbd1d`, nine commits) does **not** — its mirror was regenerated in one batch at `bbc7300`. An earlier revision of this description claimed the property held for every commit; it does not, and a bisect through that window lands on a red commit. **Squash-merge** to collapse it, or accept that this range is not bisectable. ## Version bump Six plugins and the catalog take a **patch**, not a minor. The branch is **89 commits — 40 `fix` / 30 `refactor` / 12 `docs` / 5 `chore` / 2 `test` — zero `feat`, zero `!`, zero `BREAKING CHANGE`** — and adds no skill, agent, command or hook. (Two earlier revisions of this section cited a stale histogram, most recently 78 commits; the figures above are measured at HEAD.) Both rules this repo ships (`forge/references/version-bump.md`, landing in this PR, and `git-commits/references/conventional-commits-spec.md`) make that a patch, and the catalog set is unchanged at 7 entries. Not settled by that: four published files were removed from the installed tree, three moved, and `caveman` gained `disable-model-invocation`, retiring its old triggers. Under a strict reading those are major-class and currently ship under `refactor:` with no marker. Whether the deployed skill surface is a public contract is written down nowhere — worth deciding, but it outlives this PR. ## Deliberately not in scope #112 (cherry-pick ownership, now resolved in favour of `git-commits`), #113 (`rtk git` normalisation), #116 (research fan-out), #101 (audit-skill merge), #122 (non-spec skill-root files), #123 (no PRD producer) stay open. #117 is the one worth reading: the contract's remedy is to move prose into `references/`, which is exactly where neither the size gate nor Vale looks — and the blind spot is wider than #117 currently records, since there is no root `.vale.ini` at all, so every ADR, `CONTEXT.md` and `README.md` is unlinted too. That blind spot let this branch carry two `level: error` `Kyberforge.SentenceOpenerThereIs` violations into `references/` files it created — `provider-adapter-author/references/provider-matrix.md:31` and `agent-audit/references/finding-criteria.md:95`. Both are reworded in `afadaae`, confirmed by routing each file through the audit's own `vale-wrap.sh` (1 error each before, 0 after). Five further occurrences sit in `references/` files already on `main`; those are the pre-existing corpus and stay with #117, which is the real fix. Also unfixed and not this PR's: `apm install` appends a duplicate `SessionStart` entry to `.claude/settings.json`, so a fresh clone cannot get pre-push green without an edit AGENTS.md warns against. Reproduces identically on `main`. Co-authored-by: Defame1297 <gitea@rkdr.net> Reviewed-on: https://git.dev.rkdr.net/Defame1297/holocron/pulls/129 Co-authored-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net> Co-committed-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net>
6.1 KiB
6.1 KiB
skill-audit
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
What it does
- Runs
scripts/validate.shandscripts/validate-provenance.shfor structural and provenance checks, plusscripts/vale-wrap.sh— a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers - Reads all files in the skill directory
- Applies qualitative checks across five dimension groups — always loading
references/finding-criteria.md, then one rubric fromreferences/per group the criteria put in play - Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to
skill-author
validate.sh enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words).
Alongside those it runs four shape checks that are not length measurements at all. Two are FAILs: every routing target named in the description — in the compressed Not <thing> -> <name> arrow and in the prose form — must resolve to a real skill or agent, and every references/<file>.md the body names must exist on disk. Three are SUGGESTIONs: a missing boundary clause, a Gotchas section over five entries, and a Gotchas section over 25% of the body. The resolution universe for boundary targets is derived by walking up from the audited SKILL.md — the authoring root above it, its own apm package, and that package's declared apm.yml dependencies — so a fresh clone and a machine that has run apm install return the same verdict. When no universe can be determined the check prints INFO ... DID NOT RUN and does not silently pass.
Usage
/skill-audit
Provide the path to the skill directory to audit when invoking.
Files
| File | Purpose |
|---|---|
SKILL.md |
Skill instructions for agents |
scripts/validate.sh |
Structural validator — checks name format, name matches directory, description presence and length, body-only word count, line and whole-file word ceilings, boundary-clause presence, boundary-target resolution, references/ pointer existence, Gotchas entry count and body share, placeholder detection, script executable bit, and interactive-prompt detection |
scripts/validate-provenance.sh |
Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, and upstream research doc alignment |
scripts/vale-wrap.sh |
Vale prefilter wrapper — runs the bundled Kyberforge Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
assets/vale/.vale.ini |
Vale configuration — points Vale at the bundled Kyberforge style path, self-located relative to vale-wrap.sh |
assets/vale/styles/Kyberforge/CompositionNote.yml |
Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
assets/vale/styles/Kyberforge/DescriptionOpener.yml |
Vale rule — flags non-imperative "This..." description openers |
assets/vale/styles/Kyberforge/PaddingPhrase.yml |
Vale rule — flags generic "see references/" padding phrasing in conditional references |
assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml |
Vale rule — flags body sentences starting with "There is"/"There are" |
assets/vale/styles/Kyberforge/VagueWording.yml |
Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
references/finding-criteria.md |
Every dimension's FAIL and SUGGESTION criteria — the one Step 3 file loaded on every run; it decides which rubrics below are worth loading |
references/description-quality.md |
Rubric for the description dimension — why the description is the expensive part, the hand-invoked (disable-model-invocation) contract, the three-part shape, when an indirect trigger is warranted, near-miss exclusions, and a before/after pair |
references/body-discipline.md |
Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
references/patterns.md |
Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
references/file-structure.md |
Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
references/formatting-and-scripts.md |
Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
references/validation-scripts.md |
Step 1 troubleshooting — the manual structural fallback when validate.sh cannot run, and the script exit codes that are easy to misread (loaded only on a script failure) |
references/sources.md |
Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
tests/validate.bats |
(source-only) Bats test suite for validate.sh |
tests/validate-provenance.bats |
(source-only) Bats test suite for validate-provenance.sh |
tests/README.md |
(source-only) Setup instructions for bats-support and bats-assert test dependencies |
Rows marked (source-only) exist in the authoring source (.apm/skills/skill-audit/) but are
not present in an installed plugin: scripts/sync-plugin-content.sh strips
<category>/<name>/tests when it generates the flat mirror, because these are dev-time fixtures no
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.