docs: trim skill READMEs and ADR/changelog narration
Two related simplification-audit findings, bundled because they edit some of the same skill-audit files and splitting would fragment single-file diffs. Finding 10: delete 48 per-skill/reference README.md files (they restated SKILL.md in narrative form and no agent ever loads them) plus 2 scaffold templates. Drop the README criterion from skill-audit's file-structure.md and finding-criteria.md, and the README-generation step from skill-author's new-skill.sh; update new-skill.bats to match. Plugin-root READMEs are kept intentionally, out of scope. Finding 12: strip historical ADR-0020/ADR-0023 citations and changelog-style narration from model-facing skill content across kyberforge and git plugin skills. Delete skill-author's one-time retrofit.md migration guide and its references. Some ADR-0023 tags were not narration but check-rtk-prefix's required opt-out marker for intentionally-bare git commands -- those were restored, not stripped. Mirror re-synced and full pre-commit/pre-push suite verified green. Refs: SIMPLIFICATION-AUDIT.md findings 10, 12 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
@@ -1,53 +0,0 @@
|
||||
# skill-audit
|
||||
|
||||
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across five dimension groups — always loading `references/finding-criteria.md`, then one rubric from `references/` per group the criteria put in play
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to `skill-author`
|
||||
|
||||
`validate.sh` enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words).
|
||||
|
||||
Alongside those it runs shape checks that are not length measurements at all. Three are FAILs: every routing target named in the description — in the compressed `Not <thing> -> <name>` arrow **and** in the prose form — must resolve to a real skill or agent; every `references/<file>.md` the body names must exist on disk; and `metadata.version` must be present and three-part semver (ADR-0022). That last one is FAIL rather than SUGGESTION because the `skill-frontmatter` pre-commit hook rejects the file without it — an audit grading it lower would report ready-to-ship on a file the commit gate refuses. Three are SUGGESTIONs: a missing boundary clause, a Gotchas section over five entries, and a Gotchas section over 25% of the body. The resolution universe for boundary targets is derived by walking up from the audited `SKILL.md` — the authoring root above it, its own apm package, and that package's declared `apm.yml` dependencies — so a fresh clone and a machine that has run `apm install` return the same verdict. When no universe can be determined the check prints `INFO ... DID NOT RUN` and does not silently pass.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
/skill-audit
|
||||
```
|
||||
|
||||
Provide the path to the skill directory to audit when invoking.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description presence and length, `metadata.version` presence and semver shape (ADR-0022), body-only word count, line and whole-file word ceilings, boundary-clause presence, boundary-target resolution, `references/` pointer existence, Gotchas entry count and body share, placeholder detection, script executable bit, and interactive-prompt detection |
|
||||
| `scripts/validate-provenance.sh` | Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, upstream research doc alignment, and (check 9, INFO only) whether a slug's `Description` or `Contributing files` text has changed since a base ref — `--base-ref=<ref>` or `VALIDATE_PROVENANCE_BASE_REF`, defaulting to the merge base with `origin/main` |
|
||||
| `scripts/vale-wrap.sh` | Vale prefilter wrapper — runs the bundled `Kyberforge` Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
|
||||
| `assets/vale/.vale.ini` | Vale configuration — points Vale at the bundled `Kyberforge` style path, self-located relative to `vale-wrap.sh` |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags non-imperative "This..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Vale rule — flags generic "see references/" padding phrasing in conditional references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Vale rule — flags body sentences starting with "There is"/"There are" |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
|
||||
| `references/finding-criteria.md` | Every dimension's FAIL and SUGGESTION criteria — the one Step 3 file loaded on every run; it decides which rubrics below are worth loading |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — why the description is the expensive part, the hand-invoked (`disable-model-invocation`) contract, the three-part shape, when an indirect trigger is warranted, near-miss exclusions, and a before/after pair |
|
||||
| `references/body-discipline.md` | Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
|
||||
| `references/patterns.md` | Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
|
||||
| `references/file-structure.md` | Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
|
||||
| `references/formatting-and-scripts.md` | Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
|
||||
| `references/validation-scripts.md` | Step 1 troubleshooting — the manual structural fallback when `validate.sh` cannot run, and the script exit codes that are easy to misread (loaded on a script failure, and on any exit-0 run that printed something — `validate-provenance.sh`'s check 9 is INFO-only, so its findings arrive that way) |
|
||||
| `references/sources.md` | Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
|
||||
| `tests/validate.bats` | (source-only) Bats test suite for validate.sh |
|
||||
| `tests/validate-provenance.bats` | (source-only) Bats test suite for validate-provenance.sh |
|
||||
| `tests/README.md` | (source-only) Setup instructions for bats-support and bats-assert test dependencies |
|
||||
|
||||
Rows marked **(source-only)** exist in the authoring source (`.apm/skills/skill-audit/`) but are
|
||||
not present in an installed plugin: `scripts/sync-plugin-content.sh` strips
|
||||
`<category>/<name>/tests` when it generates the flat mirror, because these are dev-time fixtures no
|
||||
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.
|
||||
@@ -7,7 +7,7 @@ description: >
|
||||
skill-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
version: "1.0.1"
|
||||
version: "1.0.2"
|
||||
category: factory
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
@@ -50,7 +50,7 @@ Read `references/validation-scripts.md` when any of the three cannot run or exit
|
||||
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md`, `README.md`, and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
@@ -64,7 +64,7 @@ Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGE
|
||||
| file-structure, internal-consistency | `references/file-structure.md` |
|
||||
| formatting, scripts | `references/formatting-and-scripts.md` |
|
||||
|
||||
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget (ADR-0020). Cite file and line number for every finding.
|
||||
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ source_keys:
|
||||
# Body Discipline Reference
|
||||
|
||||
Upstream source: agentskills.io — skill-authoring, best-practices.
|
||||
House contract: ADR-0020, the context budget.
|
||||
House contract: the context budget.
|
||||
|
||||
## The core test
|
||||
|
||||
@@ -31,7 +31,7 @@ Include content the agent lacks:
|
||||
Move to `references/`, behind an explicit "If X, read `references/<file>.md`" trigger — the literal
|
||||
conditional form, never a generic pointer. Write the real filename in the skill under audit; the
|
||||
angle brackets are a placeholder here, and a literal `references/file.md` in a body is an ERROR
|
||||
from the ADR-0020 gate because no such file exists on disk.
|
||||
from the gate because no such file exists on disk.
|
||||
|
||||
**A dispatch table satisfies this requirement on its own.** A table row already pairs a condition
|
||||
with a target, which is exactly what the literal form encodes; restating each row underneath as a
|
||||
@@ -60,7 +60,7 @@ Do not conflate these, and do not report them as one finding.
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| Body budget (house, ADR-0020) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Body budget (house) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Spec conformance (agentskills.io) | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 2,770-word ceiling is a token-conformance backstop calibrated to the densest prose in the
|
||||
@@ -134,7 +134,7 @@ Constraints:
|
||||
|
||||
Worked negative example — **`git-commits` v0.1.2 at commit `5e23250`, a fixed pre-retrofit
|
||||
snapshot, not the current file.** The live skill is v0.1.3 and matches none of the citations below;
|
||||
they are quoted as they stood before the ADR-0020 retrofit, and are not to be refreshed against
|
||||
they are quoted as they stood in that snapshot, and are not to be refreshed against
|
||||
`HEAD`. The snapshot is reachable only from a checkout of the authoring repo — an installed plugin
|
||||
cache holds no git history and no such path — so read the citations below as quoted rather than
|
||||
going to look for the file. From a checkout:
|
||||
|
||||
@@ -7,7 +7,7 @@ source_keys:
|
||||
# Description Quality Reference
|
||||
|
||||
Upstream source: agentskills.io — optimizing-descriptions, specification.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than the spec
|
||||
House contract: the context budget. The house contract is narrower than the spec
|
||||
rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## Why the description is the expensive part
|
||||
|
||||
@@ -19,7 +19,6 @@ knows to look at. Flag any other directory as a FAIL.
|
||||
`test_*.sh`) there are a FAIL — they belong in `tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
- `README.md` is present and describes the skill and its files accurately.
|
||||
|
||||
## Cross-plugin path references
|
||||
|
||||
@@ -41,7 +40,7 @@ Resolve before flagging, twice over:
|
||||
**Referring to another skill's file.** There is one sanctioned spelling, and it is possessive:
|
||||
`skill-audit's references/validation-scripts.md`. Write the skill by name and let the reader
|
||||
resolve it — do not spell the repo path. The full path is the thing this section forbids, and
|
||||
`references/validation-scripts.md` on its own is a hard ERROR from the ADR-0020 gate, which
|
||||
`references/validation-scripts.md` on its own is a hard ERROR from the gate, which
|
||||
requires an unqualified `references/` pointer to exist in the skill's OWN directory. The
|
||||
possessive form is the only spelling both rules accept; the gate recognises it and skips the
|
||||
on-disk check. Flag any other spelling of a cross-skill reference.
|
||||
@@ -59,17 +58,12 @@ Two directories are exempt, and the exemptions are structural rather than discre
|
||||
|
||||
## Internal consistency
|
||||
|
||||
The skill has to agree with itself. Three checks:
|
||||
The skill has to agree with itself. Two checks:
|
||||
|
||||
- `SKILL.md`'s steps match what the scripts actually do — the arguments, the exit codes, and the
|
||||
output shape it tells the agent to expect.
|
||||
- `README.md`'s file table lists every file that exists, with no missing rows and no stale rows for
|
||||
files since deleted.
|
||||
- Placeholder READMEs inside `scripts/`, `references/` and `assets/` say the same thing about each
|
||||
- Placeholder READMEs inside `scripts/`, `tests/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
A stale README row is the most common finding here and the easiest to miss from inside an
|
||||
authoring pass, because the author knows what was intended and reads it into the gap.
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
|
||||
@@ -111,13 +111,11 @@ Flag as FAIL if:
|
||||
- A path that resolves outside the skill directory appears outside the two exempt locations, in
|
||||
prose rather than in a fenced example
|
||||
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
||||
- `README.md` is absent, or its file table has a missing or stale row
|
||||
- `SKILL.md` describes a script invocation the script does not accept
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
|
||||
|
||||
## formatting and scripts — `references/formatting-and-scripts.md`
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ If the API returns a non-200 status, read `references/api-errors.md`.
|
||||
```
|
||||
|
||||
That block is fenced because the filename in it is illustrative — an unfenced `references/` pointer
|
||||
in a `SKILL.md` body must resolve on disk or the ADR-0020 gate reports a hard ERROR. The generic
|
||||
in a `SKILL.md` body must resolve on disk or the gate reports a hard ERROR. The generic
|
||||
form — pointing at the directory and hoping — defeats
|
||||
progressive disclosure, because the agent either loads everything or loads nothing.
|
||||
`Kyberforge.PaddingPhrase` catches the common generic phrasing deterministically; other malformed
|
||||
|
||||
@@ -21,7 +21,7 @@ have checked, and the Step 4 coverage line then names a dimension nothing actual
|
||||
## Manual structural fallback
|
||||
|
||||
`validate.sh` needs `python3` **and** PyYAML, and refuses to start without either — the description
|
||||
value has to be measured after YAML folding is resolved, so skipping the ADR-0020 gates would be a
|
||||
value has to be measured after YAML folding is resolved, so skipping these gates would be a
|
||||
vacuous pass rather than a partial one. The two are checked separately, so the message already names
|
||||
the right one — report it verbatim rather than diagnosing further:
|
||||
|
||||
@@ -40,14 +40,14 @@ by hand and file the results under `### Structure` exactly as the script's outpu
|
||||
session, so a skill without one can never be routed to.
|
||||
- **Description length**, measured on the folded YAML value with newlines collapsed to single
|
||||
spaces — not on the raw block scalar, which counts indentation. 250 characters SUGGESTION, 400
|
||||
FAIL (ADR-0020), 1,024 FAIL (agentskills.io spec).
|
||||
FAIL (house), 1,024 FAIL (agentskills.io spec).
|
||||
- **Body length**, counting everything after the frontmatter's closing `---`. 600 words
|
||||
SUGGESTION, 900 FAIL (ADR-0020).
|
||||
SUGGESTION, 900 FAIL (house).
|
||||
- **Whole-file ceilings**, counting the file including frontmatter: 500 lines FAIL, 2,770 words
|
||||
FAIL (agentskills.io spec). These are a different measurement from the two above — report them
|
||||
as separate findings, never merged.
|
||||
- **A boundary clause is present** — either the prose form (`do not` / `instead` / `rather than` /
|
||||
`not for`) or ADR-0020's compressed `Not <thing> -> <name>` arrow. **SUGGESTION**, not FAIL:
|
||||
`not for`) or the compressed `Not <thing> -> <name>` arrow. **SUGGESTION**, not FAIL:
|
||||
the absence is deterministic, but whether this skill warrants one is the auditor's call.
|
||||
- **Boundary targets resolve** — **FAIL** on a name that resolves to nothing. See the section
|
||||
below; resolving these by hand is the one item on this list with a procedure of its own.
|
||||
|
||||
Reference in New Issue
Block a user