feat(kyberforge): enforce the ADR-0020 context contract for skills and agents
Skill name+description pairs are preloaded into every session, costing ~6,200 tokens across 39 skills before any skill is invoked. The authoring rules mandated that growth: skill-author:104 and description-quality.md:21 both required padding, while skill-author:102 (the deflating rule) had no FAIL condition behind it. Gates (blocking, no baseline file): - description 250 chars SUGGESTION / 400 FAIL, measured on the folded YAML value - body-only 600 words SUGGESTION / 900 FAIL, independent of the unchanged whole-file 2770-word / 500-line spec backstop - every boundary-clause routing target must resolve to a real skill or agent; catches skill-improve, neuledge-context and gitea-labels - agents take the description gates but deliberately no body gate; a test pins that absence Vale: DescriptionOpener widened to ^This\b, new CompositionNote rule banning architecture notes from descriptions. 10 hits, 0 false positives. Kyberforge's own four skills retrofitted: descriptions 3,364 -> 938 chars (-72%), bodies 8,306 -> 2,487 words (-70%), all via the apm-workflow dispatch pattern. Fixes the skill-improve dangling route and the agent-author misroute to manual review. Also fixes a pre-existing false positive where any line-initial 'read ' was flagged as interactive input, which had already caused two scripts to be rewritten around it. Refs: ADR-0020
This commit is contained in:
@@ -1,13 +1,15 @@
|
||||
# skill-audit
|
||||
|
||||
Audit a skill directory against the agentskills.io specification. Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags known-bad description openers, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across seven dimensions
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to /skill-improve
|
||||
3. Applies qualitative checks across six dimension groups, loading one rubric from `references/` per group
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to `skill-author`
|
||||
|
||||
`validate.sh` enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words, plus resolvable boundary targets).
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -26,12 +28,16 @@ Provide the path to the skill directory to audit when invoking.
|
||||
| `scripts/validate-provenance.sh` | Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, and upstream research doc alignment |
|
||||
| `scripts/vale-wrap.sh` | Vale prefilter wrapper — runs the bundled `Kyberforge` Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
|
||||
| `assets/vale/.vale.ini` | Vale configuration — points Vale at the bundled `Kyberforge` style path, self-located relative to `vale-wrap.sh` |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags literal "This skill..."/"This agent..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags non-imperative "This..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Vale rule — flags generic "see references/" padding phrasing in conditional references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Vale rule — flags body sentences starting with "There is"/"There are" |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
|
||||
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
|
||||
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked (`disable-model-invocation`) contract, and the internal-mechanics FAIL |
|
||||
| `references/body-discipline.md` | Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
|
||||
| `references/patterns.md` | Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
|
||||
| `references/file-structure.md` | Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
|
||||
| `references/formatting-and-scripts.md` | Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
|
||||
| `references/sources.md` | Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
|
||||
| `tests/validate.bats` | (source-only) Bats test suite for validate.sh |
|
||||
| `tests/validate-provenance.bats` | (source-only) Bats test suite for validate-provenance.sh |
|
||||
|
||||
@@ -1,19 +1,10 @@
|
||||
---
|
||||
name: skill-audit
|
||||
description: >
|
||||
Use when the user wants to review a skill they wrote, says "audit this skill",
|
||||
"check if my skill follows best practices", "review my SKILL.md", or wants to
|
||||
know if a skill is ready to ship — even if they don't use the word "audit".
|
||||
Also invoke proactively after directly hand-editing a skill's files outside
|
||||
skill-author — an unaudited hand-edit is the same risk as unreviewed code.
|
||||
Audits a skill directory against the agentskills.io specification — structural
|
||||
checks plus qualitative review of description quality, body discipline, patterns,
|
||||
formatting, file structure, scripts, and internal consistency, plus a provenance
|
||||
chain check. Produces a compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
|
||||
handoff to /skill-improve or human auditability. Do not use to fix application
|
||||
code bugs or perform general code review unrelated to skill quality.
|
||||
Do not use when the user wants improvements applied — use /skill-improve instead.
|
||||
Use when the user wants a skill directory audited against the agentskills.io
|
||||
spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or
|
||||
after hand-editing a skill outside skill-author. Not applying fixes ->
|
||||
skill-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -27,9 +18,14 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not output PASS/FAIL per check while auditing — gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Structural validation
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
@@ -37,98 +33,47 @@ bash scripts/validate-provenance.sh <skill-dir>
|
||||
scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines and ≤2770 words (the word count is a proxy for the ~5,000-token ceiling, and blocks a commit exactly like the line count does), no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both. If it cannot run at all (no `python3`, Bash denied), report that as an INFO finding rather than guessing; what it measures is not reproducible by reading.
|
||||
|
||||
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
|
||||
`validate-provenance.sh` prints nothing on success. Its FAIL and INFO findings become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim.
|
||||
|
||||
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". It applies that config's `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns/Body dimensions below, not a replacement for Step 3. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one citing its rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 3 judgment for the dimensions it would have covered.
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture.
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md`, `README.md`, and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Work through each dimension internally. Collect findings only; report them in Step 4. Cite file and line number for every finding.
|
||||
Load a dimension's rubric before judging that dimension. Each is self-contained, and each is grounded in the agentskills.io specification plus the house context-budget contract (ADR-0020).
|
||||
|
||||
### Description
|
||||
| Dimension | Read |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body-discipline | `references/body-discipline.md` |
|
||||
| patterns | `references/patterns.md` |
|
||||
| file-structure, internal-consistency | `references/file-structure.md` |
|
||||
| formatting, scripts | `references/formatting-and-scripts.md` |
|
||||
|
||||
Vale's `Kyberforge.DescriptionOpener` ("This skill..." openers) and `Kyberforge.VagueWording` (filler like "helps with", "utilize") alerts from Step 1 — both FAILs — cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
|
||||
|
||||
- **Action-verb opening**: does the description start with a verb ("Audits...", "Reviews...", "Validates...")? Vale's `Kyberforge.DescriptionOpener` alert only catches the literal "This skill..." pattern — confirming an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
|
||||
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
|
||||
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
|
||||
- **Length**: under 1024 characters?
|
||||
|
||||
If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`.
|
||||
|
||||
### Body discipline
|
||||
|
||||
For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding.
|
||||
|
||||
- **Defaults not menus**: every decision point gives one default + one escape hatch, not a list of options
|
||||
- **Why rationale**: include/exclude rules explain why, not just what
|
||||
- **Control calibration**: prescriptive for fragile or critical sequences (e.g. a script invocation where flag order or exact arguments must not change); flexible where multiple approaches are valid
|
||||
|
||||
Vale's `Kyberforge.SentenceOpenerThereIs` alert from Step 1 (FAIL — sentences starting with "There is"/"There are") covers pattern-matchable body-wide filler directly; report it as a finding without re-deriving by judgment.
|
||||
|
||||
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
|
||||
|
||||
### Patterns
|
||||
|
||||
Check each pattern is appropriate and correctly formed:
|
||||
|
||||
- **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip
|
||||
- **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block
|
||||
- **Checklists**: used for multi-step workflows, not single steps
|
||||
- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment.
|
||||
- **Output templates**: present when the agent must produce a specific format; absent otherwise
|
||||
|
||||
### File structure
|
||||
|
||||
- Permitted directories: `scripts/`, `references/`, `assets/`, `tests/`; flag any other unlisted directory as FAIL — the spec allows additional dirs but this skill permits only these four to keep skills focused
|
||||
- `scripts/` contains only executable code agents can run; test files (`.bats`, `*_test.*`, `test_*.sh`) in `scripts/` are a FAIL — they belong in `tests/`
|
||||
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
|
||||
- Optional directories contain real content — not just unfilled placeholder READMEs
|
||||
- `README.md` present and accurately describes the skill and its files
|
||||
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`, or its APM-native equivalent `.apm/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
|
||||
- `references/sources.md` is exempt from the cross-plugin path check — `Research doc:` fields are development-only provenance pointers, not runtime references; they intentionally reference paths outside the skill directory and are expected to be non-resolvable after plugin install; `validate-provenance.sh` handles this gracefully by silently skipping upstream checks when those paths don't resolve
|
||||
- `tests/` is exempt from the cross-plugin path check — test files are dev-only and may reference repo-level test infrastructure (e.g. a shared `tests/test_helper/`). This dependency must be declared in `tests/README.md`; flag if tests exist but `tests/README.md` is absent or does not document the dependency
|
||||
|
||||
### Formatting
|
||||
|
||||
- Heading levels consistent: H2 for main sections, H3 for subsections
|
||||
- Code blocks fenced with a language tag where applicable (`bash`, `markdown`, `python`)
|
||||
- Consistent whitespace: blank line between sections, consistent list indentation
|
||||
- No broken relative paths in file references
|
||||
|
||||
### Scripts
|
||||
|
||||
- No interactive TTY prompts (`read`, `input()`, `readline`)
|
||||
- `--help` exposed with concise usage
|
||||
- Data to stdout, diagnostics to stderr
|
||||
- Idempotent ("create if not exists")
|
||||
- Meaningful exit codes documented in `--help`
|
||||
- `--dry-run` present for destructive operations
|
||||
|
||||
### Internal consistency
|
||||
|
||||
- SKILL.md steps match what scripts actually do
|
||||
- `README.md` file table lists every file that exists — no missing entries, no stale entries
|
||||
- Placeholder READMEs in `scripts/`, `references/`, `assets/` consistent with what SKILL.md says about each directory
|
||||
Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with a coverage line listing every dimension checked:
|
||||
Open with a coverage line naming every dimension checked:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely — their absence confirms they passed.
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
For each finding:
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
@@ -136,18 +81,4 @@ FAIL/SUGGESTION <finding> — file:line
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a result block:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS (N suggestions)
|
||||
PASS · P info
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /skill-improve to address findings.
|
||||
```
|
||||
|
||||
INFO findings are observational — do not affect PASS/FAIL. Omit `· P info` when there are no INFO findings. Omit the `/skill-improve` line when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -4,4 +4,4 @@ level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This (skill|agent)\b'
|
||||
- '^This\b'
|
||||
|
||||
@@ -6,31 +6,114 @@ source_keys:
|
||||
|
||||
# Body Discipline Reference
|
||||
|
||||
Source: agentskills.io — skill-authoring
|
||||
Upstream source: agentskills.io — skill-authoring, best-practices.
|
||||
House contract: ADR-0020, the context budget.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and dilutes the signal of what matters.
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## What belongs in the body
|
||||
## What the body is for
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when.
|
||||
|
||||
Include content the agent lacks:
|
||||
|
||||
- Project-specific conventions and domain procedures it cannot infer
|
||||
- Non-obvious edge cases and environment-specific gotchas
|
||||
- The specific tools or sequences to use (not the full range of options)
|
||||
- The specific tools or sequences to use — not the full range of options
|
||||
- One default per decision point with one escape hatch
|
||||
|
||||
Do not include:
|
||||
Move to `references/`, behind an explicit "If X, read `references/file.md`" trigger — the literal
|
||||
conditional form, never a generic pointer:
|
||||
|
||||
- Lookup tables and spec restatements
|
||||
- Output schemas, templates and example blocks
|
||||
- Rationale and justification prose
|
||||
- Anything only one branch of the procedure ever reaches
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description — it's already in context
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Two length families, measured differently
|
||||
|
||||
Do not conflate these, and do not report them as one finding.
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| Body budget (house, ADR-0020) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Spec conformance (agentskills.io) | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 2,770-word ceiling is a token-conformance backstop calibrated to the densest prose in the
|
||||
corpus; it says nothing about quality and a file can sit a thousand words inside it while failing
|
||||
the body budget. The 900-word ceiling is the quality gate: a body is loaded into the caller's live
|
||||
context and competes with the conversation already there. `validate.sh` reports both. Cite whichever
|
||||
one actually fired.
|
||||
|
||||
A word count cannot detect the defect it stands in for. Treat both numbers as backstops to the
|
||||
dispatch rule and the Gotchas constraint below, never as a substitute for them.
|
||||
|
||||
## Dispatch is mandatory at two or more mutually exclusive flows
|
||||
|
||||
If a skill handles two or more flows that a single invocation cannot both take — separate
|
||||
subcommands, separate input types, separate lifecycle stages — the body carries a **dispatch
|
||||
table** plus the gates common to every branch, and each flow lives in its own self-contained
|
||||
`references/` file. Inlining all of them is a FAIL regardless of word count, because every
|
||||
invocation then pays for every branch it did not take.
|
||||
|
||||
The reference shape in this repo is `apm-workflow`: a 554-word body dispatching to roughly 3,000
|
||||
words of references across five mutually exclusive invocations.
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
The highest-value construct in a body, and the easiest to fill with noise. A Gotcha must state a
|
||||
fact that **contradicts a reasonable default** — something the agent gets wrong precisely by acting
|
||||
sensibly.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- **Maximum five entries.** Past five, the section is a summary of the body rather than a set of
|
||||
traps, and the agent stops reading it as a warning.
|
||||
- **A Gotcha that paraphrases a step in the body below it is a FAIL.** It has no independent
|
||||
content, and it teaches the agent that Gotchas can be skimmed because the real instruction is
|
||||
coming.
|
||||
- **A Gotchas section exceeding 25% of the body is a SUGGESTION** — the body has been inverted into
|
||||
a preamble.
|
||||
- Place the section near the top. A gotcha read after the mistake is worthless, which is also why
|
||||
Gotchas is the one construct exempt from moving to `references/`.
|
||||
|
||||
Worked negative example — `git-commits` carries thirteen entries, of which four restate content
|
||||
that already appears below or in the description:
|
||||
|
||||
| Gotcha | Restates |
|
||||
|---|---|
|
||||
| `:31` "Communicates SemVer impact" | the description |
|
||||
| `:32` "Confirmation gates are mandatory for destructive operations" | step 9 at `:52` |
|
||||
| `:33` "Never skip hooks with `--no-verify`" | step 9 at `:52` |
|
||||
| `:36` "Never commit secrets" | step 2 at `:45` |
|
||||
|
||||
All four are FAILs under this rule, and the section as a whole breaches the five-entry maximum. It
|
||||
also passes every plausible word gate, which is the point of auditing the construct directly.
|
||||
|
||||
## Calibrating control
|
||||
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed:
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be
|
||||
followed:
|
||||
|
||||
```markdown
|
||||
Run exactly:
|
||||
\`\`\`bash
|
||||
@@ -39,11 +122,13 @@ python scripts/migrate.py --verify --backup
|
||||
Do not modify the command or add additional flags.
|
||||
```
|
||||
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives — agents make better decisions when they understand the purpose.
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives —
|
||||
agents make better decisions when they understand the purpose.
|
||||
|
||||
## Defaults not menus
|
||||
|
||||
Never present a list of equivalent options — pick one and mention the alternative briefly:
|
||||
|
||||
```markdown
|
||||
# Too many options
|
||||
Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
@@ -52,37 +137,22 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
|
||||
```
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
Highest value content — environment-specific facts that defy reasonable assumptions. Place near the top of the body so the agent reads them before encountering the situation.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Each entry must be a specific, surprising fact — not a general tip or reminder.
|
||||
|
||||
## Progressive disclosure
|
||||
|
||||
Keep `SKILL.md` under 500 lines. When more content is needed, move it to `references/` and load conditionally:
|
||||
|
||||
```markdown
|
||||
If the API returns a non-200 status, read `references/api-errors.md`.
|
||||
```
|
||||
|
||||
"If X, read Y" is more useful than "see references/ for details." The agent loads on demand rather than up front.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- A sentence answers "no" to the core test (would agent get this wrong without it?) — it is padding
|
||||
- Decision points present a menu of options with no default
|
||||
- Instructions repeat content already in the description
|
||||
- Prescriptive sequences are used where flexibility is fine, or vice versa
|
||||
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
|
||||
- Two or more mutually exclusive flows are inlined instead of dispatched
|
||||
- A Gotcha paraphrases a step in the body below it, or the section exceeds five entries
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- A rationale is missing from an include/exclude rule (present but unexplained)
|
||||
|
||||
- The body exceeds 600 words counted body-only but stays at or under 900
|
||||
- The Gotchas section exceeds 25% of the body
|
||||
- A rationale is missing from an include/exclude rule — present but unexplained
|
||||
- Gotchas are correct but placed late in the body rather than near the top
|
||||
- A conditional reference trigger is vague ("see references/") rather than specific ("If X, read Y")
|
||||
- Content that only one branch reaches is inlined where a `references/` file would serve
|
||||
|
||||
@@ -6,49 +6,112 @@ source_keys:
|
||||
|
||||
# Description Quality Reference
|
||||
|
||||
Source: agentskills.io — optimizing-descriptions
|
||||
Upstream source: agentskills.io — optimizing-descriptions, specification.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than the spec
|
||||
rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## How triggering works
|
||||
## Why the description is the expensive part
|
||||
|
||||
At startup, agents load only the `name` and `description` of each skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. **The description carries the entire triggering burden** — the body is never seen until after triggering.
|
||||
At startup an agent loads only the `name` and `description` of every installed skill. The body is
|
||||
never seen until the skill triggers. The description therefore carries the entire triggering
|
||||
burden **and** is paid for in every session, whether the skill fires or not.
|
||||
|
||||
Agents typically consult skills only for tasks requiring knowledge beyond their defaults. Specialized knowledge — unfamiliar APIs, domain-specific workflows, uncommon formats — is where description wording makes the difference.
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the agent takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## What a good description does
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
- **Imperative phrasing** — "Use when..." not "This skill does...". The agent is deciding whether to act.
|
||||
- **User intent, not mechanics** — describe what the user is trying to achieve, not how the skill works internally.
|
||||
- **Err toward being pushy** — explicitly name contexts where the skill applies, including cases where the user doesn't name the domain: "even if they don't mention X explicitly."
|
||||
- **Specificity over vagueness** — "parses and validates OpenAPI specs" beats "helps with APIs."
|
||||
- **Near-miss exclusions** — add "Do not use when..." only if a near-miss skill exists that could steal activations. Use strong near-misses (queries that share keywords but need something different), not weak ones ("write a fibonacci function").
|
||||
- **Hard limit: 1024 characters** — descriptions grow during revision; check length before finalising.
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
- **`disable-model-invocation: true`** — the skill is hand-invoked. Its description is never
|
||||
matched against user intent, so it is not a routing string. It carries **one plain human-facing
|
||||
sentence** stating what the skill does. Audit it for that and nothing else. Reporting a missing
|
||||
trigger clause, a missing boundary clause or absent indirect triggers on a hand-invoked skill is
|
||||
a wrong finding, not a strict one.
|
||||
- **No such flag** — the skill is model-invoked and the rest of this file applies.
|
||||
|
||||
## The three-part shape
|
||||
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This skill ...` —
|
||||
the agent is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source; `validate.sh` checks that
|
||||
deterministically and a dangling target already surfaces as a Structure FAIL.
|
||||
|
||||
Everything else belongs in the body or in `README.md`.
|
||||
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on a skill whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Near-miss exclusions
|
||||
|
||||
Add a boundary clause only where a sibling skill could plausibly steal the activation. Use strong
|
||||
near-misses — queries that share keywords but need something different — not weak ones ("write a
|
||||
fibonacci function"). One boundary clause per genuine near-miss; a list of four is enumeration
|
||||
wearing a boundary's clothes.
|
||||
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# Weak
|
||||
description: Process CSV files.
|
||||
|
||||
# Strong
|
||||
# FAIL — enumeration first, mechanics as the opener, a blanket indirect trigger,
|
||||
# and 300+ characters of it preloaded into every session forever.
|
||||
description: >
|
||||
Analyze CSV and tabular data files — compute summary statistics,
|
||||
add derived columns, generate charts, and clean messy data. Use when
|
||||
the user has a CSV, TSV, or Excel file and wants to explore, transform,
|
||||
or visualize the data, even if they don't explicitly mention "CSV" or
|
||||
"analysis."
|
||||
Analyze CSV and tabular data files — compute summary statistics, add derived
|
||||
columns, generate charts, and clean messy data. Use when the user has a CSV,
|
||||
TSV, or Excel file and wants to explore, transform, or visualize the data,
|
||||
even if they don't explicitly mention "CSV" or "analysis."
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The four verbs the FAIL
|
||||
# version enumerates are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when the user has a CSV, TSV, or Excel file and wants it explored,
|
||||
transformed, or charted. Not schema design -> data-model.
|
||||
```
|
||||
|
||||
The strong version names capabilities precisely and broadens applicability beyond explicit keyword matches.
|
||||
(`data-model` is illustrative. In a real description the target has to resolve.)
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- Phrasing is descriptive ("This skill...") not imperative ("Use when...")
|
||||
- Capabilities are vague ("helps with APIs") — require precise verbs and nouns
|
||||
- No indirect trigger coverage when indirect cases clearly exist
|
||||
- No near-miss exclusions when a sibling skill could plausibly steal activations
|
||||
- Length exceeds 1024 characters
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared skill", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
||||
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
|
||||
file in the authoring source. `validate.sh` reports the unresolved name.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0.
|
||||
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
||||
of the 400-character house ceiling above.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- Indirect trigger coverage exists but could be more specific
|
||||
- Near-miss exclusions are present but target weak near-misses only
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
---
|
||||
|
||||
# File Structure and Internal Consistency Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (optional directories, file references).
|
||||
|
||||
Read this when judging the **file-structure** and **internal-consistency** dimensions.
|
||||
|
||||
## Permitted directories
|
||||
|
||||
Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification permits additional
|
||||
directories; this house does not, because an unlisted directory is content no auditor and no host
|
||||
knows to look at. Flag any other directory as a FAIL.
|
||||
|
||||
- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`,
|
||||
`test_*.sh`) there are a FAIL — they belong in `tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
- `README.md` is present and describes the skill and its files accurately.
|
||||
|
||||
## Cross-plugin path references
|
||||
|
||||
A plugin is copied to a cache on install, and a path that climbs out of the skill directory stops
|
||||
resolving there. Flag any `../`, `../../`, or absolute repo path (`plugins/<plugin>/skills/<other>/`
|
||||
and its APM-native equivalent `.apm/skills/<other>/`) appearing in `SKILL.md`, `scripts/`,
|
||||
`references/` or `assets/`.
|
||||
|
||||
Two directories are exempt, and the exemptions are structural rather than discretionary:
|
||||
|
||||
- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers,
|
||||
not runtime references. They are expected to be unresolvable after install, and
|
||||
`validate-provenance.sh` handles that by skipping upstream checks silently when the path is
|
||||
absent. Flagging them would make every correctly-provenanced skill fail.
|
||||
- **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared
|
||||
`tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/`
|
||||
exists and `tests/README.md` is absent or does not document it, that is a FAIL.
|
||||
|
||||
## Internal consistency
|
||||
|
||||
The skill has to agree with itself. Three checks:
|
||||
|
||||
- `SKILL.md`'s steps match what the scripts actually do — the arguments, the exit codes, and the
|
||||
output shape it tells the agent to expect.
|
||||
- `README.md`'s file table lists every file that exists, with no missing rows and no stale rows for
|
||||
files since deleted.
|
||||
- Placeholder READMEs inside `scripts/`, `references/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
A stale README row is the most common finding here and the easiest to miss from inside an
|
||||
authoring pass, because the author knows what was intended and reads it into the gap.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A directory outside the four permitted ones exists
|
||||
- Test files sit in `scripts/`
|
||||
- A non-spec file sits at the skill root
|
||||
- A cross-plugin or parent-relative path appears outside the two exempt locations
|
||||
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
||||
- `README.md` is absent, or its file table has a missing or stale row
|
||||
- `SKILL.md` describes a script invocation the script does not accept
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Formatting and Scripts Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (body content), using-scripts (designing scripts
|
||||
for agentic use).
|
||||
|
||||
Read this when judging the **formatting** and **scripts** dimensions. Both are checklists of static
|
||||
criteria that never vary by skill, which is exactly why they live here rather than in the body.
|
||||
|
||||
## Formatting
|
||||
|
||||
- Heading levels are consistent: H2 for main sections, H3 for subsections. A body that jumps from
|
||||
H2 to H4, or opens on H3, reads as a fragment of a larger document.
|
||||
- Code blocks carry a language tag wherever one applies — `bash`, `markdown`, `python`, `yaml`,
|
||||
`text`. An untagged block loses syntax highlighting and, more importantly, loses the signal of
|
||||
what the agent is meant to do with it.
|
||||
- Whitespace is consistent: a blank line between sections, one list-indentation style throughout.
|
||||
- No broken relative paths in file references. Every `references/…`, `scripts/…` and `assets/…`
|
||||
path named in the body resolves against the skill directory.
|
||||
|
||||
## Scripts
|
||||
|
||||
A script in a skill is run by an agent with no terminal and no human to answer it. The criteria
|
||||
follow from that:
|
||||
|
||||
- **No interactive TTY prompts** — no `read`, no `input()`, no `readline`. A script that blocks on
|
||||
a prompt hangs the run with no diagnostic. `validate.sh` detects the common forms and reports
|
||||
them under Structure; the judgment call is any prompt it cannot pattern-match. What counts is
|
||||
where stdin comes from, not the word `read`: a `read` fed by a here-string, a here-doc, a pipe,
|
||||
or a redirect from a file never touches a terminal and is not a finding. `validate.sh` excludes
|
||||
those forms, so do not rewrite a working `read -r A B <<< "$line"` into parameter expansion to
|
||||
satisfy this rule.
|
||||
- **`--help` is exposed** and gives concise usage.
|
||||
- **Data to stdout, diagnostics to stderr.** A caller piping the script has to be able to separate
|
||||
the result from the commentary.
|
||||
- **Idempotent** — "create if not exists" rather than "create", so a re-run after a partial failure
|
||||
is safe.
|
||||
- **Meaningful exit codes, documented in `--help`.** An agent branches on the exit code; an
|
||||
undocumented one is a coin flip.
|
||||
- **`--dry-run` present for destructive operations.**
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A script prompts interactively, in any form
|
||||
- A script exposes no `--help`
|
||||
- A destructive script has no `--dry-run`
|
||||
- Data and diagnostics share a stream, so the output cannot be piped
|
||||
- A relative path named in the body does not resolve
|
||||
- Heading levels are inconsistent enough to break the document's structure
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Exit codes are meaningful but undocumented in `--help`
|
||||
- A code block is untagged where a language applies
|
||||
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
|
||||
- List indentation or section spacing is inconsistent without breaking the render
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
---
|
||||
|
||||
# Patterns Reference
|
||||
|
||||
Upstream source: agentskills.io — best-practices (instruction patterns), specification.
|
||||
|
||||
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
|
||||
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
|
||||
*content* is judged in `references/body-discipline.md`.
|
||||
|
||||
## The constructs and when each is right
|
||||
|
||||
| Construct | Right when | Wrong when |
|
||||
|---|---|---|
|
||||
| Gotchas | An environment fact contradicts a reasonable default | Used as a summary of the steps below |
|
||||
| Prescriptive sequence | The operation is fragile and flag order or exact arguments must not change | Several approaches are equally valid |
|
||||
| Checklist | A multi-step workflow the agent must complete in order | A single step dressed up as a list |
|
||||
| Conditional reference | Detail is needed on one branch only | The reference is needed on every run and is loaded blind |
|
||||
| Output template | The agent must emit a specific format a caller consumes | The output is prose nobody parses |
|
||||
|
||||
## Formation rules
|
||||
|
||||
**Gotchas** sit near the top of the body, before the steps that would otherwise walk into them.
|
||||
Placement late in the body is a SUGGESTION, not a FAIL — the content is still correct, it is just
|
||||
read after the mistake.
|
||||
|
||||
**Prescriptive sequences** that quote a fenced block inside another markdown block must escape the
|
||||
inner fence as `` \`\`\` ``. An unescaped inner fence terminates the outer block and the remaining
|
||||
instructions render as prose.
|
||||
|
||||
**Conditional references** state a specific trigger: "If the API returns a non-200 status, read
|
||||
`references/api-errors.md`." The generic form — pointing at the directory and hoping — defeats
|
||||
progressive disclosure, because the agent either loads everything or loads nothing.
|
||||
`Kyberforge.PaddingPhrase` catches the common generic phrasing deterministically; other malformed
|
||||
forms are judgment.
|
||||
|
||||
**Output templates** belong in the body when the agent must emit them on every run, and in
|
||||
`references/` when only one dispatch branch produces that output. A template inlined for a branch
|
||||
most invocations never take is body-discipline padding.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
|
||||
assumption
|
||||
- An inner code fence is unescaped inside a markdown block, breaking the render
|
||||
- A checklist wraps a single step
|
||||
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
|
||||
- The agent must produce a specific format and no output template is given
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Gotchas are correctly formed but placed late in the body
|
||||
- An output template is present but permissive where the consumer needs it exact
|
||||
- A conditional reference names a trigger that is real but broader than the branch it guards
|
||||
@@ -15,7 +15,7 @@
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
@@ -23,7 +23,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
@@ -47,7 +47,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
@@ -11,7 +11,7 @@ Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
EOF
|
||||
}
|
||||
@@ -32,6 +32,7 @@ python3 -u - "$1" <<'PYTHON'
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
@@ -44,6 +45,7 @@ with open(skill_md) as f:
|
||||
content = f.read()
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
@@ -53,6 +55,13 @@ def fail(msg):
|
||||
print(f"FAIL {msg}")
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
|
||||
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
|
||||
if not fm_match:
|
||||
@@ -66,16 +75,58 @@ body_start = fm_match.end()
|
||||
name_m = re.search(r'^name:\s*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — inline or block scalar (> or |)
|
||||
desc = ""
|
||||
desc_m = re.search(r'^description:\s*([>|])\n((?:[ \t]+.+\n?)+)', fm, re.MULTILINE)
|
||||
if desc_m:
|
||||
raw = desc_m.group(2)
|
||||
desc = re.sub(r'\s+', ' ', raw).strip()
|
||||
else:
|
||||
desc_inline = re.search(r'^description:\s*(.+)', fm, re.MULTILINE)
|
||||
if desc_inline:
|
||||
desc = desc_inline.group(1).strip()
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first. PyYAML is used
|
||||
# when importable (it is a real parser); the fallback recognises exactly the
|
||||
# shapes this corpus uses — an inline scalar, optionally quoted and optionally
|
||||
# continued on following indented lines, and a `>`/`|` block scalar with
|
||||
# optional indentation and chomping indicators.
|
||||
|
||||
def normalize(value):
|
||||
return re.sub(r'\s+', ' ', value).strip()
|
||||
|
||||
def fold_description_fallback(fm_text):
|
||||
lines = fm_text.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^description:[ \t]*(.*)$', line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1).strip()
|
||||
block = bool(re.match(r'^[>|][0-9]*[-+]?$|^[>|][-+]?[0-9]*$', head))
|
||||
parts = [] if block else [head]
|
||||
for nxt in lines[i + 1:]:
|
||||
if not nxt.strip():
|
||||
parts.append('')
|
||||
continue
|
||||
if not re.match(r'^[ \t]', nxt):
|
||||
break
|
||||
parts.append(nxt.strip())
|
||||
value = ' '.join(parts)
|
||||
if not block:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in '"\'':
|
||||
value = value[1:-1]
|
||||
return value
|
||||
return ''
|
||||
|
||||
def extract_description(fm_text):
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(fm_text)
|
||||
if isinstance(data, dict):
|
||||
value = data.get('description')
|
||||
if isinstance(value, str):
|
||||
return normalize(value)
|
||||
if value is not None:
|
||||
return normalize(str(value))
|
||||
return ''
|
||||
except Exception:
|
||||
pass
|
||||
return normalize(fold_description_fallback(fm_text))
|
||||
|
||||
desc = extract_description(fm)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
@@ -114,11 +165,13 @@ if desc:
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (limit: 1024)")
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
@@ -146,6 +199,26 @@ MAX_LINES = 500
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
|
||||
# description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
@@ -160,14 +233,220 @@ if word_count <= MAX_WORDS:
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
# Body unfilled placeholders
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# A boundary clause names another skill — or an agent, which is an equally
|
||||
# valid routing target (git-workflow routes to the git-orchestrate agent). Every
|
||||
# named target is resolved against the AUTHORING SOURCE, plugins/*/.apm/skills/
|
||||
# and plugins/*/.apm/agents/, so the check works offline and before an
|
||||
# `apm install` has deployed anything into .claude/skills/.
|
||||
#
|
||||
# False positives are the design constraint here, not recall. Two rules do the
|
||||
# work:
|
||||
# * A BARE hyphenated word is read as a routing target only inside a boundary
|
||||
# sentence (one carrying "do not"/"instead"/"rather than"/"not for").
|
||||
# Without that, pc-run's "run pre-commit hooks" reads as a route to a
|
||||
# non-existent `pre-commit` skill.
|
||||
# * A BARE arrow target counts only in ADR-0020's compressed boundary form,
|
||||
# `Not <thing> -> <skill-name>`. Without that, diagnose's process chain
|
||||
# "fix -> regression-test" reads as a route to `regression-test`.
|
||||
# Backticked and /slash-command targets are unambiguous and always count. Tool
|
||||
# names (Read, Write, Edit) are excluded by the lowercase-only name pattern;
|
||||
# MCP tool names (issue_write, pull_request_write) by its rejection of
|
||||
# underscores; file names by its rejection of dots and slashes.
|
||||
|
||||
NAME_ANY = r"[a-z0-9]+(?:-[a-z0-9]+)*"
|
||||
NAME_HYPH = r"[a-z0-9]+(?:-[a-z0-9]+)+"
|
||||
ROUTE_VERB = (r"(?:use|uses|using|run|runs|invoke|invokes|invoking|try|see"
|
||||
r"|that'?s|compose|composes|call|calls)")
|
||||
MARKED_TARGET = r"(?:`/?(%s)`|(?<![\w./*-])/(%s)\b)" % (NAME_ANY, NAME_ANY)
|
||||
ANY_TARGET = r"(?:%s|(%s)\b)" % (MARKED_TARGET, NAME_HYPH)
|
||||
ROUTE_MARKED = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, MARKED_TARGET), re.I)
|
||||
ROUTE_ANY = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, ANY_TARGET), re.I)
|
||||
CONT_MARKED = re.compile(r"\s*(?:or|and|/|,)\s*%s" % MARKED_TARGET)
|
||||
CONT_ANY = re.compile(r"\s*(?:or|and|/|,)\s*%s" % ANY_TARGET)
|
||||
ARROW_MARKED = re.compile(r"(?:->|→)\s*%s" % MARKED_TARGET)
|
||||
ARROW_BOUNDARY = re.compile(r"\bnot\b[^.;]*?(?:->|→)\s*(%s)\b" % NAME_HYPH, re.I)
|
||||
BACKTICK = re.compile(r"`(%s)`" % NAME_HYPH)
|
||||
BOUNDARY_MARKER = re.compile(r"\b(?:do\s+not|instead|rather\s+than|not\s+for)\b", re.I)
|
||||
SENTENCE_SPLIT = re.compile('(?<=[.!?])\\s+(?=[A-Z"“(])')
|
||||
|
||||
def _first_group(groups):
|
||||
for g in groups:
|
||||
if g:
|
||||
return g
|
||||
return None
|
||||
|
||||
def _scan_routes(text, route_re, cont_re, out):
|
||||
for m in route_re.finditer(text):
|
||||
target = _first_group(m.groups())
|
||||
if not target:
|
||||
continue
|
||||
out.append(target)
|
||||
# Conjoined targets: "use git-history or git-branches instead",
|
||||
# "use gitea-issues / gitea-prs".
|
||||
pos = m.end()
|
||||
while True:
|
||||
cm = cont_re.match(text, pos)
|
||||
if not cm:
|
||||
break
|
||||
nxt = _first_group(cm.groups())
|
||||
if nxt:
|
||||
out.append(nxt)
|
||||
pos = cm.end()
|
||||
|
||||
def boundary_targets(description):
|
||||
out = []
|
||||
for sentence in SENTENCE_SPLIT.split(description):
|
||||
boundary = bool(BOUNDARY_MARKER.search(sentence))
|
||||
_scan_routes(sentence,
|
||||
ROUTE_ANY if boundary else ROUTE_MARKED,
|
||||
CONT_ANY if boundary else CONT_MARKED,
|
||||
out)
|
||||
for m in ARROW_MARKED.finditer(sentence):
|
||||
target = _first_group(m.groups())
|
||||
if target:
|
||||
out.append(target)
|
||||
for m in ARROW_BOUNDARY.finditer(sentence):
|
||||
out.append(m.group(1))
|
||||
out.extend(BACKTICK.findall(sentence))
|
||||
return sorted(set(out))
|
||||
|
||||
def known_targets(start_dir):
|
||||
names = set()
|
||||
# Sibling skills/agents. This is the branch that works in a cache-installed
|
||||
# plugin and in a deployed .claude/skills/ tree, neither of which has a
|
||||
# plugins/ directory above it.
|
||||
parent = os.path.dirname(os.path.abspath(start_dir))
|
||||
if os.path.basename(parent) == 'skills' and os.path.isdir(parent):
|
||||
for entry in os.listdir(parent):
|
||||
if os.path.isdir(os.path.join(parent, entry)):
|
||||
names.add(entry)
|
||||
agents_dir = os.path.join(os.path.dirname(parent), 'agents')
|
||||
if os.path.isdir(agents_dir):
|
||||
for entry in os.listdir(agents_dir):
|
||||
if entry.endswith('.agent.md'):
|
||||
names.add(entry[:-len('.agent.md')])
|
||||
elif entry.endswith('.md'):
|
||||
names.add(entry[:-len('.md')])
|
||||
# Walk up for a monorepo root (plugins/*/.apm/) or a plugin root (.apm/).
|
||||
# Capped at ten levels so a pathological path can't become a filesystem
|
||||
# crawl; that covers every real layout by a wide margin.
|
||||
current = os.path.abspath(start_dir)
|
||||
for _ in range(10):
|
||||
# Never glob the filesystem root: a stray /.apm/skills/ (a scaffolding
|
||||
# test's leftover, say) would otherwise become part of every skill's
|
||||
# resolution universe on that machine.
|
||||
if os.path.dirname(current) == current:
|
||||
break
|
||||
for pattern in ('plugins/*/.apm/skills/*/', '.apm/skills/*/'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path.rstrip('/')))
|
||||
for pattern in ('plugins/*/.apm/agents/*.agent.md', '.apm/agents/*.agent.md'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path)[:-len('.agent.md')])
|
||||
current = os.path.dirname(current)
|
||||
return names
|
||||
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
# An empty universe means no authoring source was found anywhere above this
|
||||
# skill — reporting every target as dangling there would be noise, not a
|
||||
# finding, so the check declines to run rather than guessing.
|
||||
if routing_targets and known:
|
||||
unresolved = [t for t in routing_targets if t not in known]
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill under "
|
||||
f"plugins/*/.apm/skills/ and no agent under plugins/*/.apm/agents/ — "
|
||||
f"a boundary clause naming a non-existent target sends the router nowhere")
|
||||
if not unresolved:
|
||||
ok(f"all {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(routing_targets)}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
for line in source.splitlines():
|
||||
stripped = line.strip()
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
@@ -177,9 +456,10 @@ if os.path.isdir(scripts_dir):
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
with open(fpath) as f:
|
||||
sc = f.read()
|
||||
# Interactive prompt heuristic
|
||||
if re.search(r'^\s*(read\s|input\()', sc, re.MULTILINE):
|
||||
fail(f"scripts/{fname}: may use interactive input (read/input detected)")
|
||||
interactive = interactive_reads(sc)
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
else:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
@@ -190,8 +470,17 @@ if os.path.isdir(scripts_dir):
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
print("All checks passed.")
|
||||
if suggestions:
|
||||
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
|
||||
@@ -25,6 +25,39 @@ description: A valid skill description that is well within the limit.
|
||||
Do the thing.
|
||||
EOF
|
||||
}
|
||||
|
||||
# Helper: create a skill directory with an exact description length and an
|
||||
# exact body word count. <desc> is used verbatim; <body_words> "word"
|
||||
# tokens follow the frontmatter. Used by the ADR-0020 boundary tests.
|
||||
make_sized_skill() {
|
||||
local dir="$1" desc="$2" body_words="$3"
|
||||
local name
|
||||
name="$(basename "$dir")"
|
||||
mkdir -p "$dir"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: $name"
|
||||
echo "description: $desc"
|
||||
echo "---"
|
||||
echo ""
|
||||
python3 -c "print(' '.join(['word'] * $body_words))"
|
||||
} > "$dir/SKILL.md"
|
||||
}
|
||||
|
||||
# Helper: build a self-contained fixture plugin tree so the boundary-target
|
||||
# resolver has a real authoring source to resolve against, independent of
|
||||
# this repo's live skills. Echoes the subject skill's directory.
|
||||
#
|
||||
# <root>/plugins/fixture-plugin/.apm/skills/<subject>/SKILL.md
|
||||
# <root>/plugins/fixture-plugin/.apm/skills/fixture-sibling-skill/
|
||||
# <root>/plugins/fixture-plugin/.apm/agents/fixture-sibling-agent.agent.md
|
||||
make_fixture_tree() {
|
||||
local root="$1" subject="$2"
|
||||
local apm="$root/plugins/fixture-plugin/.apm"
|
||||
mkdir -p "$apm/skills/$subject" "$apm/skills/fixture-sibling-skill" "$apm/agents"
|
||||
touch "$apm/agents/fixture-sibling-agent.agent.md"
|
||||
echo "$apm/skills/$subject"
|
||||
}
|
||||
}
|
||||
|
||||
teardown() {
|
||||
@@ -64,7 +97,7 @@ teardown() {
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes at exactly 1024-char description" {
|
||||
@test "the 1024-char agentskills.io spec backstop is unchanged and separate from the ADR-0020 ceiling" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
local name
|
||||
name="$(basename "$skill")"
|
||||
@@ -82,7 +115,12 @@ description: $desc
|
||||
Do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
# Two independent gates on one value: the spec limit still PASSES at
|
||||
# exactly 1024 (its own boundary is unmoved), while ADR-0020's 400-char
|
||||
# ceiling FAILs. The run fails on the second, not the first.
|
||||
assert_output --partial "description length 1024 chars (agentskills.io spec limit: 1024)"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "passes at exactly 500 lines" {
|
||||
@@ -170,6 +208,61 @@ EOF
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "fails when a script reads a variable with no redirect" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r ANSWER\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "fails when an interactive prompt string contains an angle bracket" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -p "enter <name>: " NAME\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a here-string" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nLINE="a b"\nread -r X Y <<< "$LINE"\n' \
|
||||
> "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a here-doc" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r X <<EOF\nvalue\nEOF\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a file redirect" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r LINE < "$1"\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a pipe continued onto the next line" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nprintf %%s "$1" |\n read -r X\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "fails when name contains consecutive hyphens" {
|
||||
local skill="$TMPDIR/my--skill"
|
||||
make_valid_skill "$skill"
|
||||
@@ -196,3 +289,222 @@ EOF
|
||||
run bash "$SCRIPT"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — description budget (250 SUGGESTION / 400 FAIL)
|
||||
#
|
||||
# These sit UNDER the agentskills.io 1024-character spec backstop above, which
|
||||
# is unchanged. Both ceilings are inclusive: exactly at the number passes that
|
||||
# tier, one past it trips.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: description of exactly 250 chars raises no suggestion" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 250)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of 251 chars raises a SUGGESTION and still exits 0" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 251)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
assert_output --partial "description is 251 chars"
|
||||
assert_output --partial "All checks passed (1 suggestion(s))."
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of exactly 400 chars is a SUGGESTION, not a FAIL" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 400)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of 401 chars FAILs and exits non-zero" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 401)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "description is 401 chars"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description length is measured after YAML folding is resolved" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
mkdir -p "$skill"
|
||||
# A >-folded block scalar: 11 lines of 40 chars folded with 10 joining
|
||||
# spaces = 450 characters. Measured off its raw `description: >` line it is
|
||||
# 1 character and passes; measured as the folded VALUE it must FAIL. This
|
||||
# is exactly the case a line-wise regex gets wrong.
|
||||
{
|
||||
echo "---"
|
||||
echo "name: my-skill"
|
||||
echo "description: >"
|
||||
python3 -c "print('\n'.join([' ' + 'x' * 40] * 11))"
|
||||
echo "---"
|
||||
echo ""
|
||||
echo "Do the thing."
|
||||
} > "$skill/SKILL.md"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "description is 450 chars"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — body budget (600 SUGGESTION / 900 FAIL), body ONLY
|
||||
#
|
||||
# Distinct from the 2,770-word whole-file spec ceiling above, which counts
|
||||
# frontmatter too and is unchanged. Do not unify them.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: body of exactly 600 words raises no suggestion" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 600
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of 601 words raises a SUGGESTION and still exits 0" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 601
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "body is 601 words"
|
||||
assert_output --partial "All checks passed (1 suggestion(s))."
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of exactly 900 words is a SUGGESTION, not a FAIL" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 900
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "body is 900 words"
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of 901 words FAILs and exits non-zero" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 901
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "body is 901 words"
|
||||
assert_output --partial "900-word ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: the body gate counts the body only — frontmatter words do not count toward it" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
# 895 body words plus a description long enough that the WHOLE FILE is well
|
||||
# over 900 words. The body gate must stay silent; the 2,770-word whole-file
|
||||
# ceiling is a separate measurement and is nowhere near tripping.
|
||||
make_sized_skill "$skill" "$(python3 -c "print(' '.join(['w'] * 100))")" 895
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "900-word ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — resolvable boundary targets
|
||||
#
|
||||
# Resolved against the AUTHORING SOURCE (plugins/*/.apm/skills/ and
|
||||
# plugins/*/.apm/agents/), never .claude/skills/, so the check works offline and
|
||||
# before an apm install. Every fixture below builds its own plugin tree rather
|
||||
# than leaning on this repo's live skills.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: a boundary target naming an existing sibling skill resolves" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use fixture-sibling-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "boundary target(s) resolve"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a boundary target naming a non-existent skill FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use fixture-missing-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-skill'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a boundary target naming an AGENT file resolves (agents are valid routing targets)" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use when the caller is an agent — invoke fixture-sibling-agent instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "boundary target(s) resolve"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a /slash-command boundary target that does not resolve FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use when improvements are wanted — use /fixture-missing-improve instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-improve'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a backticked name that does not resolve FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Composes \`fixture-missing-helper\` for the shared part." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-helper'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a bare hyphenated word outside a boundary sentence is not read as a routing target" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
# "run pre-commit hooks" is pc-run's real phrasing. A naive extractor reads
|
||||
# it as a route to a non-existent `pre-commit` skill.
|
||||
make_sized_skill "$skill" "Use when the user wants to run pre-commit hooks or install git hooks." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "pre-commit"
|
||||
}
|
||||
|
||||
@test "ADR-0020: an arrow chain outside a boundary clause is not read as a routing target" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
# diagnose's real process chain. Only ADR-0020's `Not <thing> -> <skill>`
|
||||
# form makes a bare arrow target a route.
|
||||
make_sized_skill "$skill" "Reproduce → minimise → instrument → fix → regression-test. Use when a bug is reported." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "regression-test"
|
||||
}
|
||||
|
||||
@test "ADR-0020: MCP tool names and capitalised tool names are not read as routing targets" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when writing issues. Do not use for local files (use Read/Write/Edit) — that write goes through \`issue_write\`/\`pull_request_write\` instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "routes to"
|
||||
}
|
||||
|
||||
@test "ADR-0020: ADR's compressed boundary form (Not <thing> -> <skill>) is checked" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Not the other thing → fixture-missing-target." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-target'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: the boundary check declines rather than false-FAILs when no authoring source is found" {
|
||||
local skill="$TMPDIR/orphan/my-skill"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use some-other-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "routes to"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user