feat(kyberforge): enforce the ADR-0020 context contract for skills and agents
Skill name+description pairs are preloaded into every session, costing ~6,200 tokens across 39 skills before any skill is invoked. The authoring rules mandated that growth: skill-author:104 and description-quality.md:21 both required padding, while skill-author:102 (the deflating rule) had no FAIL condition behind it. Gates (blocking, no baseline file): - description 250 chars SUGGESTION / 400 FAIL, measured on the folded YAML value - body-only 600 words SUGGESTION / 900 FAIL, independent of the unchanged whole-file 2770-word / 500-line spec backstop - every boundary-clause routing target must resolve to a real skill or agent; catches skill-improve, neuledge-context and gitea-labels - agents take the description gates but deliberately no body gate; a test pins that absence Vale: DescriptionOpener widened to ^This\b, new CompositionNote rule banning architecture notes from descriptions. 10 hits, 0 false positives. Kyberforge's own four skills retrofitted: descriptions 3,364 -> 938 chars (-72%), bodies 8,306 -> 2,487 words (-70%), all via the apm-workflow dispatch pattern. Fixes the skill-improve dangling route and the agent-author misroute to manual review. Also fixes a pre-existing false positive where any line-initial 'read ' was flagged as interactive input, which had already caused two scripts to be rewritten around it. Refs: ADR-0020
This commit is contained in:
@@ -255,12 +255,21 @@ repos:
|
||||
|
||||
- id: skill-size-check
|
||||
stages: ['pre-commit']
|
||||
name: SKILL.md size ceiling
|
||||
description: Enforce agentskills.io's 500-line/5,000-token SKILL.md size ceiling
|
||||
name: SKILL.md size and context-budget ceilings
|
||||
description: Enforce agentskills.io's 500-line/2,770-whole-file-word spec ceilings AND ADR-0020's context budget -- description 250 chars SUGGESTION / 400 FAIL, body-only 600 words SUGGESTION / 900 FAIL, and every boundary-clause routing target resolving to a real skill or agent under plugins/*/.apm/
|
||||
entry: scripts/skill-size-check.sh
|
||||
language: script
|
||||
files: '^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$'
|
||||
pass_filenames: true
|
||||
verbose: true
|
||||
# verbose so the SUGGESTION tier is audible. ADR-0020 depends on it:
|
||||
# "A ceiling does not produce an average ... The halving depends
|
||||
# entirely on the 250-character SUGGESTION tier being visible and
|
||||
# respected." pre-commit prints nothing at all for a passing hook, and
|
||||
# a SUGGESTION deliberately does not fail, so without verbose every
|
||||
# suggestion would be swallowed -- the exact invisibility ADR-0013
|
||||
# records for Vale warnings. Costs nothing on a clean file: the script
|
||||
# prints only findings.
|
||||
|
||||
- id: vale-audit-prefilter-skill
|
||||
stages: ['pre-commit']
|
||||
|
||||
@@ -13,8 +13,11 @@
|
||||
files: '(^|/)agents/[^/]+\.md$|\.agent\.md$'
|
||||
|
||||
- id: kyberforge-skill-size-check
|
||||
name: SKILL.md size ceiling
|
||||
description: Enforce agentskills.io's 500-line/5,000-token SKILL.md size ceiling
|
||||
name: SKILL.md size and context-budget ceilings
|
||||
description: Enforce agentskills.io's 500-line/2,770-whole-file-word spec ceilings plus ADR-0020's context budget (description 250 chars SUGGESTION / 400 FAIL, body-only 600 words SUGGESTION / 900 FAIL, resolvable boundary-clause routing targets)
|
||||
entry: scripts/skill-size-check.sh
|
||||
language: script
|
||||
files: '(^|/)SKILL\.md$'
|
||||
# verbose so the SUGGESTION tier reaches a human -- pre-commit prints
|
||||
# nothing for a passing hook, and a SUGGESTION deliberately does not fail.
|
||||
verbose: true
|
||||
|
||||
@@ -43,6 +43,9 @@ Fall back to raw shell only when no skill covers it.
|
||||
- Install git hooks via `pc-run`, wiring all three stages — this repo's `.pre-commit-config.yaml` has no `default_install_hook_types`, so a plain install silently skips `commit-msg` (Conventional Commits) and `pre-push` (the 14-hook gate described below).
|
||||
- Install the `apm` CLI — four pre-push hooks shell out to it: `apm-marketplace-check`, `apm-audit-ci`, `apm-pack-check-clean`, and `check-plugin-content-sync` (via `scripts/sync-plugin-content.sh`, which wraps `apm pack`). `apm-marketplace-check` and `apm-pack-check-clean` are bare `apm …` hook entries and `apm-audit-ci` is a `bash -c` loop calling `apm` once per package, so without it the push dies with an unhelpful "command not found". Use `apm-install`, or `curl -sSL https://aka.ms/apm-unix | sh`; verify with `apm --version`.
|
||||
- Install `jq` — required by `scripts/check-manifests.sh` and `scripts/sync-plugin-content.sh`, both pre-push. These at least fail loudly (`Error: jq is required but not installed`).
|
||||
- Install `python3` — required by `scripts/skill-size-check.sh`, the `skill-size-check` pre-commit hook. It measures the *folded* `description` value: most descriptions here are `>`-block scalars, so a regex over the raw lines measures indentation and newlines instead of the value. Missing it fails the hook with an install pointer rather than skipping the ADR-0020 checks, which would be a vacuous green. In practice it is already present — pre-commit is itself a Python application. PyYAML is used when importable and is genuinely optional; a fallback reader covers the frontmatter shapes this corpus uses.
|
||||
- That hook enforces **two independent gate families** over `plugins/*/.apm/skills/*/SKILL.md`, and neither replaced the other. The agentskills.io spec backstop is unchanged: 500 lines and 2,770 words, counted over the **whole file including frontmatter**. ADR-0020 adds a context budget measured differently — `description` 250 chars SUGGESTION / 400 FAIL (it is preloaded into every session whether the skill fires or not), **body-only** word count 600 SUGGESTION / 900 FAIL (everything after the frontmatter's closing `---`), and every boundary-clause routing target resolving to a real skill or agent under `plugins/*/.apm/`. A file can sit well inside one family and fail the other. The hook is `verbose: true` so the SUGGESTION tier is audible — pre-commit prints nothing at all for a passing hook, and a SUGGESTION deliberately does not fail. `skill-audit`'s `validate.sh` holds a second copy of the four ADR-0020 constants; `tests/test-skill-size-check.sh` asserts the copies agree.
|
||||
- **Those ADR-0020 gates ship hot, with no baseline file.** 26 of 39 descriptions and 9 of 39 bodies currently exceed their FAIL tier, so editing one of those skills *for any reason* means retrofitting it to the contract first — a one-line fix to `gitea-prs` cannot be committed until that skill complies. This is deliberate, and the retrofit is tracked as Gitea issue #99. Check where a skill stands before starting: `pre-commit run skill-size-check --all-files`.
|
||||
- Install the `vale` binary — required by the `vale-audit-prefilter-skill`/`-agent` pre-commit hooks. Their `files:` patterns are `.apm/`-scoped: `^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$` and `^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$`. Only the authoring source triggers them — a `SKILL.md` in the generated mirror matches neither pattern, so prose findings surface only when you edit the file you are supposed to be editing. Without the binary the hooks fail with a bare "command not found" and no install pointer. `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or see https://vale.sh/docs/vale-cli/installation/. No `vale sync` needed — the `Kyberforge` styles are committed under `plugins/kyberforge/.apm/skills/{skill-audit,agent-audit}/assets/vale/styles/`, not downloaded packages (see ADR-0014).
|
||||
- `vale` is also a **pre-push** dependency, not only pre-commit. `check-vale-style-sync` runs six glob-coverage probes by invoking `vale --config` — they are the only assertions in it that catch a `.vale.ini` glob typo, the failure mode where every text-level check stays clean while vale lints zero files. Missing `vale` is therefore a hard failure there. The opt-out is `CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1`, and it is **not** `SKIP=`: the hook still runs and still asserts everything verifiable from file text, but the six probes do not, and its summary says so explicitly — `Vale style sync check passed (text-level only, vale unavailable): … 0 glob probe(s) verified`. Use it only on a machine that genuinely cannot install `vale`, and read that summary line as "the glob axis was not checked", not as a pass.
|
||||
- Run `bash tests/run-tests.sh` before considering any change done — it runs every `test-*.sh` script in the repo plus the bats suite (`--bats-only` for just bats). First run auto-initializes the bats submodules; no manual `git submodule update` needed.
|
||||
@@ -50,7 +53,7 @@ Fall back to raw shell only when no skill covers it.
|
||||
- `tests/run-bats.sh` derives the set of `.bats` files it expects from `git ls-files`, so a `.bats` file deleted from the worktree but still tracked in the index fails the run rather than silently shrinking the suite. Remove one with `git rm` (or stage the deletion) when the removal is intentional; an untracked new `.bats` file is picked up and needs no ceremony. Both discovery walks (`tests/run-bats.sh` and `tests/run-tests.sh`) exclude `apm_modules/`: `apm install` materializes a full copy of every plugin there, and running a dependency's copy of a `.bats` file breaks its relative path to the bats helpers — 167 spurious failures before the exclusion landed.
|
||||
- Pushing runs 14 repo-defined pre-push hooks, not just the test suite — `run-tests` and `check-manifests`, plus generated-content drift gates (`check-plugin-content-sync`, `check-marketplace-mirror-sync`, `check-vale-style-sync`, `check-scope-walkup-sync`, `check-executables-allow-sync`), artifact validators (`check-apm-agents-valid`, which runs agent-audit's `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md`), apm's own gates (`apm-marketplace-check`, `apm-audit-ci`, `apm-pack-check-clean`), host validators (`validate-plugins`, `validate-marketplace`, both needing the `claude` CLI), and `check-release-needed`. `check-executables-allow-sync` is the odd one in that first group — it guards a silent failure rather than drift in generated text. apm gates a package's `hooks/` and `bin/` on an exact `<package>#<version>` lookup in root `apm.yml`'s `executables.allow`, with no wildcard and no version-less form, so bumping `plugins/kyberforge/apm.yml`'s `version:` without bumping the key errors nowhere: the entry simply stops matching, kyberforge's `SessionStart` hook stops deploying, and the install goes quietly stale — the failure ADR-0019 records as live. Run `pre-commit run --hook-stage pre-push --all-files` locally — one command, the whole gate. That command reports **16**, not 14: pre-commit's own `meta` hooks, `check-hooks-apply` and `check-useless-excludes`, declare no `stages:` and so run at every stage including this one.
|
||||
- `apm-audit-ci` runs `apm audit --ci` once per manifest — the root one and each of the six plugin packages — because the root-only invocation audits the marketplace manifest and **nothing else**, and `apm-pack-check-clean` does not parse plugin `dependencies:` blocks either (verified: a malformed one passes `apm pack --check-versions --check-clean --dry-run` and fails `apm audit --ci` in that package's directory). It verifies two things and claims no more: each `apm.yml` parses as a valid APM manifest, and any package declaring dependencies has a consistent `apm.lock.yaml`. It does **not** enforce an org policy — apm discovers one from the git remote and only understands github.com and Azure DevOps, so against this repo's self-hosted Gitea remote it prints `No org policy found at unknown; enforcement skipped`. Do **not** "fix" that with `policy.fetch_failure_default: block` in `apm.yml`: it was tested and rejected, because with no reachable policy source it makes the hook exit 1 on every push forever.
|
||||
- `check-apm-agents-valid` derives its expected agent-file set from `git ls-files` (same pattern as `tests/run-bats.sh`), so an agent file deleted from the worktree but still tracked fails the run, and discovering zero agent files is an error rather than a pass. An untracked new agent file is still validated — the derivation is one-directional on purpose, so uncommitted work is not blocked but also cannot bypass the gate.
|
||||
- `check-apm-agents-valid` derives its expected agent-file set from `git ls-files` (same pattern as `tests/run-bats.sh`), so an agent file deleted from the worktree but still tracked fails the run, and discovering zero agent files is an error rather than a pass. An untracked new agent file is still validated — the derivation is one-directional on purpose, so uncommitted work is not blocked but also cannot bypass the gate. Agents take the ADR-0020 description gates (`agent-audit`'s `validate.sh` holds its own copy of those two constants) and, deliberately, **no** body word gate: an agent body becomes the system prompt of a fresh context rather than competing with the caller's live conversation, so the 900-word FAIL does not transfer. A bats test pins that absence — adding a body gate there contradicts the ADR rather than fixing an inconsistency.
|
||||
- **Two** pre-push hooks need the network, for one shared reason: root `apm.yml`'s `marketplace.packages[]` contains exactly one remote entry (`mattpocock-skills`, `source: mattpocock/skills`), and resolving it needs a `git ls-remote`. `apm-marketplace-check` resolves every entry and is `always_run`, so it fails with `No cached refs (offline)`. `apm-pack-check-clean` (`apm pack --check-versions --check-clean --dry-run`) re-resolves the same entry and fails with `Error: Git network timeout during ls-remote`. Pinning the entry to an exact version does **not** remove the call — an exact pin still ls-remotes. `--offline` rescues neither. To push without a network, skip both using pre-commit's own mechanism: `SKIP=apm-marketplace-check,apm-pack-check-clean git push`. Skip those two alone — verified under `unshare -rn`, the other twelve pre-push hooks pass offline because they are real local checks (`check-executables-allow-sync` landed after that run, but reads two local manifests and makes no network call), and adding one of them to `SKIP` disarms it silently. `apm-audit-ci` calls `apm` too but stays local: its org-policy discovery resolves nothing on this remote before any network call, so it does not join the pair above.
|
||||
- Author commits with `git-commits` — it validates Conventional Commits (enforced at `commit-msg`) for you.
|
||||
|
||||
|
||||
26
LESSONS.md
26
LESSONS.md
@@ -177,3 +177,29 @@ Across one review round, four fixes specified by the orchestrating reviewer were
|
||||
Mutation testing a review round's own fixes found repeatedly that a passing test was pinning nothing. Deleting `sync_dir`'s stale-directory wipe, its check-mode stale branch, or three of five `MIRROR_DIRS` entries each left the suite at 18/18 green; so did replacing the hooks trailing-newline normalisation with plain `cp`. A pair of concurrency assertions written to guard a reentrancy defect caught it 0 times in 10 runs against the deliberately broken script — and one of them was structurally incapable of ever catching it, because the broken code wrote to the system temp dir while the assertion inspected `$TMPDIR`. A fixture-leak fix ran green with and without the fix, verified only by external observation. Two manifest fixtures passed with the canonicalisation they claimed to cover deleted, rescued by an unrelated name-matching axis. In each case the test named the right behaviour in its description and asserted something adjacent to it. The cheap discipline that finds all of these: for every assertion, construct the revert it is supposed to catch and confirm it fails — and when an assertion survives every revert you can think of, that is not reassurance, it is the finding (one test only revealed itself as decoration once a sixth, differently-targeted revert was built for it). Fix: treat "which revert does this fail against?" as a required answer at the time an assertion is written, and record it where the assertion lives, since a test's own description is exactly the artifact that made the gap invisible.
|
||||
|
||||
Graduation candidate: this overlaps 2026-08-09's "an assertion written to cover an accepted residual tends to assert the residual's presence rather than the behaviour it costs" and the same date's "assert on the expected members, so a derivation whose input vanished fails loudly instead of quietly covering less." Three entries circling one pattern — human review for promotion to `core/instructions/testing.md`.
|
||||
|
||||
## 2026-08-14 — Vale's `existence` extension concatenates `raw:` entries, it does not alternate them
|
||||
|
||||
A new `Kyberforge.CompositionNote` rule was first written with seven `raw:` entries, one per banned
|
||||
phrasing. Vale loaded it without a diagnostic and it matched **zero of 43 files** — an outcome
|
||||
indistinguishable from a clean corpus, and the exact shape of 2026-08-08's "a clean linter result can
|
||||
mean nothing was checked". The cause is that `existence` joins multiple `raw:` entries into one
|
||||
pattern rather than OR-ing them, so the rule was searching for all seven phrases concatenated. Every
|
||||
pre-existing rule in this style has exactly one `raw:` entry, so nothing in the repo demonstrated the
|
||||
difference, and the multi-entry form looks natural beside them. `tokens:` is the alternated form,
|
||||
which is why `VagueWording` uses it. Fix: a new Vale rule is not landed until it has been shown to
|
||||
*fire* — the standing revert-check applies to linter rules as much as to tests, and the revert here
|
||||
is the broken multi-`raw:` form, which `tests/test-vale-hooks-consumer.sh` now fails against.
|
||||
|
||||
## 2026-08-14 — Un-anchoring a description rule to reach mid-sentence text is unshippable
|
||||
|
||||
Widening `DescriptionOpener` to catch `gitea-workflow`'s mid-description "This is the human-facing
|
||||
entry point…" looked like a one-character change. Under `scope: text.frontmatter.description`, `^`
|
||||
anchors to the start of the whole description value — and `vale-wrap.sh` has already flattened that
|
||||
value to one physical line, so `(?m)` changes nothing. Un-anchoring is therefore the only route to
|
||||
mid-description text, and measured across the corpus it scores 5 hits and 5 false positives: skills
|
||||
legitimately quote user phrasings (`says "audit this skill"`) and write boundary clauses (`do not use
|
||||
this skill to manage label definitions`). That is the `Kyberforge.VagueQualifier` deletion repeating.
|
||||
Fix: keep the opener rule opener-anchored and give mid-description prose its own rule with its own
|
||||
token list. A rule's scope anchor is part of its contract, not an implementation detail to relax when
|
||||
a new case does not fit.
|
||||
|
||||
@@ -82,9 +82,11 @@ The body carries the **decision procedure only**: ordered steps, decision branch
|
||||
reference to load when. Lookup tables, spec restatements, output schemas, templates, and rationale
|
||||
prose move to `references/` behind an explicit "read X when Y" trigger.
|
||||
|
||||
- **600 words SUGGESTION, 900 FAIL.** The 2,770-word / 500-line spec backstop is unchanged and keeps
|
||||
its existing meaning: conformance, not quality. These are two different gates measuring two
|
||||
different things, and conflating them is what produced the current state.
|
||||
- **600 words SUGGESTION, 900 FAIL, counted body-only** — everything after the closing `---` of the
|
||||
frontmatter. The 2,770-word / 500-line spec backstop is unchanged, keeps its existing meaning
|
||||
(conformance, not quality), and keeps counting the **whole file including frontmatter**. These are
|
||||
two different gates measuring two different things, and conflating them is what produced the
|
||||
current state.
|
||||
- **Dispatch is mandatory at two or more mutually exclusive flows.** The body carries the dispatch
|
||||
table and the gates that apply to every branch; each flow lives in its own self-contained
|
||||
`references/` file. This is `apm-workflow/SKILL.md:33-41` promoted from accident to rule.
|
||||
@@ -149,8 +151,12 @@ Three pre-existing contradictions are fixed in the same change, because they are
|
||||
|
||||
## Consequences
|
||||
|
||||
**Editing any non-compliant skill now requires retrofitting it first.** 30 of 39 descriptions exceed
|
||||
400 characters and 14 of 39 bodies exceed 900 words. With the gate hot and no baseline, a one-line
|
||||
**Editing any non-compliant skill now requires retrofitting it first.** At decision time, 30 of 39
|
||||
descriptions exceeded 400 characters and 13 of 39 bodies exceeded 900 words — the latter counted
|
||||
body-only, which is what the new gate measures; the pre-existing 2,770-word gate counts the whole
|
||||
file including frontmatter, and the two must not be conflated. The change that carries this ADR also
|
||||
retrofits kyberforge's own four author/audit skills, so the figures on landing are **26 and 9**.
|
||||
With the gate hot and no baseline, a one-line
|
||||
fix to `gitea-prs` cannot be committed until that skill meets the contract. This is deliberate — it
|
||||
guarantees convergence and avoids a half-state — but it means the retrofit is lazy and *mandatory*
|
||||
rather than deferred. The follow-up retrofit issue should be prioritised accordingly, and the risk it
|
||||
|
||||
@@ -1,32 +1,40 @@
|
||||
# agent-audit
|
||||
|
||||
Audits an agent definition for correctness and quality — a single vendor-neutral file at
|
||||
Audits an agent definition for correctness and quality against the Claude Code and Copilot agent
|
||||
references and the house context-budget contract (ADR-0020) — a single vendor-neutral file at
|
||||
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
|
||||
|
||||
## What it does
|
||||
|
||||
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
|
||||
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
|
||||
vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both harnesses and an
|
||||
unsafe field can't be silently dropped for just one of them. The allowlist itself lives in the
|
||||
`apm-agent-allowlist` section of `references/field-inventory.md` and is read from there as data —
|
||||
consult that section rather than any restatement of it, including this one. As of 2026-08-14 it
|
||||
admits `name`, `description`, `model`, `source_keys`, and `disallowedTools`; `source_keys` is
|
||||
provenance metadata checked separately by `validate-provenance.sh` against `sources.md`, and
|
||||
`disallowedTools` is admitted because a denylist survives verbatim copy where the `tools` allowlist
|
||||
does not (ADR-0016 and its 2026-08-14 amendment).
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance
|
||||
checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags
|
||||
non-imperative description openers, composition and architecture notes, vague wording, padding
|
||||
phrases, "There is/are" sentence openers, and CC-specific "Use proactively" phrasing in a
|
||||
Copilot or vendor-neutral description
|
||||
2. Reads the agent file, and its counterpart when one exists, then loads the contract for its scope
|
||||
3. Applies qualitative checks across description, body, delegation and comment discipline, loading
|
||||
one rubric from `references/` per group
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix —
|
||||
and a result block with handoff to `agent-author`
|
||||
|
||||
At **project/user scope**, accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives
|
||||
the counterpart automatically, and validates both. Runs structural checks via `validate.sh`
|
||||
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
|
||||
Copilot-only fields in the CC file), provenance chain validation via `validate-provenance.sh`
|
||||
(checks `source_keys` against `sources.md` at the plugin root — plugin/APM scope only), then
|
||||
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
|
||||
Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the
|
||||
`Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert
|
||||
is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is
|
||||
unavailable or reports `0 files` scanned. Produces a compact findings report in the same format
|
||||
as `skill-audit`.
|
||||
Two things follow from ADR-0020 and are easy to get backwards. Agents take the **same** description
|
||||
gates a skill takes — 250 characters SUGGESTION, 400 FAIL, since a `name` + `description` is
|
||||
preloaded into every session either way — and **no body word gate at all**, because an agent body
|
||||
becomes the system prompt of a fresh context rather than competing with the caller's live
|
||||
conversation. Body length is judged through the delegation check instead: an agent body that
|
||||
restates a procedure owned by a skill it can invoke is a FAIL, because a plugin-scope agent has no
|
||||
sibling `references/` directory to disclose to and can only delegate.
|
||||
|
||||
At **plugin/APM scope** the audit accepts the single `.apm/agents/<name>.agent.md` file — there is
|
||||
no counterpart, and pair consistency does not apply. `validate.sh` hard-`FAIL`s any frontmatter
|
||||
field outside the vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both
|
||||
harnesses and an unsafe field cannot be silently dropped for just one of them. The allowlist lives
|
||||
in the `apm-agent-allowlist` section of `references/field-inventory.md`, is read from there as data
|
||||
by the script, and is deliberately not restated anywhere else in this skill (ADR-0009).
|
||||
|
||||
At **project/user scope** the audit accepts either file in a CC `.md` / Copilot `.agent.md` pair,
|
||||
derives the counterpart automatically, and validates both, including the field-leakage checks in
|
||||
each direction.
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -42,18 +50,23 @@ Pass the path to either agent file as the argument.
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `assets/vale/.vale.ini` | Vale config: scopes `Kyberforge` to `**/agents/*.md`, `Kyberforge`+`KyberforgeCopilot` to `**/*.agent.md` |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This skill/agent" instead of an imperative "Use when..." |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Flags composition and architecture notes in a description ("cross-cutting", "entry point", "composes", "rather than duplicating") that belong in README.md |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This..." instead of an imperative "Use when..." |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Flags generic "see references/ for info" pointers instead of specific file references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Flags sentences opening with "There is/are" instead of naming the subject directly |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
|
||||
| `assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml` | Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
|
||||
| `references/README.md` | Directory documentation for references/ |
|
||||
| `references/description-quality.md` | Qualitative guide for borderline description findings |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked contract, and the internal-mechanics FAIL |
|
||||
| `references/body-and-delegation.md` | Rubric for the body, delegation and comment-discipline dimensions — the delegation FAIL and why agents take no body word gate |
|
||||
| `references/scope-plugin-apm.md` | Scope contract for a single vendor-neutral APM agent file — allowlist, dimension routing, and the dimensions that do not apply |
|
||||
| `references/scope-project-user.md` | Scope contract for a CC / Copilot pair — counterpart derivation, provider field rules, pair consistency |
|
||||
| `references/validation-scripts.md` | Loaded only when a Step 1 script fails or cannot run — scope-detection walk-up, manual fallback checks, known script failures |
|
||||
| `references/field-inventory.md` | Authoritative field lists read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM-scope allowlist |
|
||||
| `references/sources.md` | Research provenance for skill content |
|
||||
| `scripts/README.md` | Directory documentation for scripts/ |
|
||||
| `scripts/validate.sh` | Structural validation script for agent file pairs |
|
||||
| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `sources.md` (plugin root) |
|
||||
| `scripts/validate.sh` | Structural validator — required fields, name format, placeholder detection, the ADR-0020 description budget, and the field rules for the detected scope |
|
||||
| `scripts/validate-provenance.sh` | Provenance chain validation against `sources.md` at the package root (plugin/APM scope only) |
|
||||
| `scripts/vale-wrap.sh` | Drop-in `vale` wrapper that works around a frontmatter-description NLP scope limitation |
|
||||
| `tests/README.md` | (source-only) Bats test dependency and run instructions |
|
||||
| `tests/validate.bats` | (source-only) Bats tests for validate.sh |
|
||||
|
||||
@@ -1,18 +1,10 @@
|
||||
---
|
||||
name: agent-audit
|
||||
description: >
|
||||
Use when the user wants to review an agent definition they wrote, says "audit this
|
||||
agent", "check if my agent follows best practices", "review my agent file", or wants
|
||||
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
||||
Also invoke proactively after directly hand-editing an agent file pair outside
|
||||
agent-author — an unaudited hand-edit is the same risk as unreviewed code.
|
||||
Audits a Claude Code .md and Copilot .agent.md agent file pair across six dimensions:
|
||||
structural validation, provider safety, description quality, body quality, comment
|
||||
discipline, and pair consistency — plus provenance chain validation. Produces a
|
||||
compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
|
||||
files — use /agent-author instead. Do not use to audit SKILL.md files — use
|
||||
/skill-audit instead.
|
||||
Use when the user wants an agent definition audited — "audit this agent",
|
||||
"review my agent file", "is this ready to ship" — or after hand-editing an
|
||||
agent outside agent-author. Not applying fixes -> agent-author. Not a skill
|
||||
directory -> skill-audit.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -26,80 +18,67 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- The unit of authoring at project/user scope is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention at those scopes — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures. **At plugin/APM scope there is no pair** — the unit of authoring is a single vendor-neutral `.apm/agents/<name>.agent.md` file, and Pair Consistency does not apply there at all (see below).
|
||||
- Scope is detected by walking up from the agent file's directory: at each level, if `apm.yml` exists AND contains a top-level `type: instructions|skill|hybrid|prompts` line, that directory is an APM package root — plugin/APM scope. A `type:`-less `apm.yml` is marketplace-only (see `docs/research/docs/microsoft-apm/monorepo-and-repo-shapes.md`) — skip it and keep walking up. Otherwise, if `.git` is a directory at that level, stop there — project scope. If neither is found before the filesystem root, fall back to user scope at `$HOME`. `plugin.json`/`.claude-plugin/plugin.json` are no longer scope signals for this skill — a directory with only a `plugin.json` and no `apm.yml` falls through to project (or user) scope.
|
||||
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
||||
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Agents take the same 250/400-character description gates as skills and **no body word gate at all** — an agent body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer. Judge an over-long agent body through the delegation check, never by word count.
|
||||
- At plugin/APM scope the agent is a single vendor-neutral file by design: never raise a pair-consistency finding there, and provider safety stops meaning Claude-Code-versus-Copilot field leakage.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Run structural validation
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <path-to-agent-file>
|
||||
bash scripts/validate-provenance.sh <path-to-agent-file>
|
||||
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file> # project/user scope
|
||||
scripts/vale-wrap.sh <path-to-apm-agent-file> # plugin/APM scope — single file
|
||||
bash scripts/validate.sh <agent-file>
|
||||
bash scripts/validate-provenance.sh <agent-file>
|
||||
scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
|
||||
```
|
||||
|
||||
The script accepts either the CC file, the Copilot file, or (at plugin/APM scope) the single `.apm/agents/<name>.agent.md` file. It detects provider from extension and scope from the walk-up above, then runs the checks for that scope.
|
||||
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the ADR-0020 description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
|
||||
|
||||
At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
|
||||
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, `references/field-inventory.md` missing — read `references/validation-scripts.md`; what these scripts measure is not reproducible by reading.
|
||||
|
||||
At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md`. Read that section for the current list rather than reciting one here; it is the authoritative source and it changes. As of 2026-08-14 it is `name`, `description`, `model`, `source_keys`, `disallowedTools` — `source_keys` is provenance metadata, not a provider-specific field, and is validated separately by `validate-provenance.sh` against `sources.md`; `disallowedTools` is a denylist, admitted because denying a tool by name is safe under `apm compile`'s verbatim copy in a way the `tools` allowlist is not (ADR-0016's 2026-08-14 amendment, and the rationale recorded alongside the list itself). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design.
|
||||
`validate-provenance.sh` prints nothing on success and runs at plugin/APM scope only, exiting 0 silently elsewhere. Its FAIL findings become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim.
|
||||
|
||||
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
`validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
|
||||
|
||||
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside the allowlist — read the `apm-agent-allowlist` section of `references/field-inventory.md` for it, do not work from memory (`source_keys` carries provenance metadata, checked separately by `validate-provenance.sh` against `sources.md`).
|
||||
## Step 2 — Read the agent and load its scope contract
|
||||
|
||||
## Step 2 — Qualitative checks
|
||||
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
|
||||
|
||||
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
||||
| Scope | Read |
|
||||
|---|---|
|
||||
| plugin/APM | `references/scope-plugin-apm.md` |
|
||||
| project, user | `references/scope-project-user.md` |
|
||||
|
||||
**Description (both files):**
|
||||
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those as FAILs without re-deriving by judgment.
|
||||
- `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment.
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
If a description finding is borderline, read `references/description-quality.md`.
|
||||
Load a dimension's rubric before judging that dimension.
|
||||
|
||||
**Body:**
|
||||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||||
- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment
|
||||
- Sentences that open with "There is"/"There are": Vale's `Kyberforge.SentenceOpenerThereIs` alert flags this directly — report it without re-deriving by judgment
|
||||
| Dimension | Read |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body, delegation, comment-discipline | `references/body-and-delegation.md` |
|
||||
|
||||
**Body/Frontmatter comments:**
|
||||
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.
|
||||
- Look for patterns like `# Optional. <long explanation>` or extensive inline guidance (more than 1–2 lines per field) that should be condensed or removed before shipping.
|
||||
- This mirrors skill-audit's body-discipline check but applies to template documentation in the frontmatter — template guidance belongs in development; agent-ready files should have minimal comments.
|
||||
Cite file and line number for every finding.
|
||||
|
||||
**Pair consistency (cross-file) — project/user scope only:**
|
||||
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
|
||||
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
|
||||
- **Does not apply at plugin/APM scope** — there is only one file, by design; do not raise a Pair Consistency finding there under any circumstance.
|
||||
## Step 4 — Report
|
||||
|
||||
**Unexpressable Claude-only behavior — plugin/APM scope only:**
|
||||
- Read the description and body. If either implies a need the vendor-neutral frontmatter can no longer express — tool restriction, `isolation`, `memory`, or another Claude-only behavior that a hand-authored CC file could have declared — flag it as a SUGGESTION, never a FAIL. This is a known upstream schema limitation (APM's agent primitive has no per-target compile integrator, so `tools:`/`isolation`/etc. can't be emitted safely to both CC and Copilot — ADR-0016), not an authoring mistake. The finding exists to give the author visibility into the gap, not to imply the schema can be made to do something it can't.
|
||||
- Example: a body that says "only use Read and Grep, never Edit" but the frontmatter has no `tools` field to enforce it — SUGGESTION, not FAIL.
|
||||
|
||||
## Step 3 — Report
|
||||
|
||||
Open with a coverage line. At project/user scope:
|
||||
Open with a coverage line naming every dimension checked. At project/user scope:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · comment-discipline · pair-consistency · provenance
|
||||
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
|
||||
```
|
||||
|
||||
At plugin/APM scope, omit `pair-consistency` — it does not apply when there is no pair:
|
||||
At plugin/APM scope, drop `pair-consistency` — there is no pair to check.
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · comment-discipline · provenance
|
||||
```
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
|
||||
|
||||
For each finding:
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
@@ -107,18 +86,4 @@ FAIL/SUGGESTION <finding> — file:line
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS · P info
|
||||
PASS (N suggestions)
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /agent-author to address findings.
|
||||
```
|
||||
|
||||
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run agent-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -4,4 +4,4 @@ level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This (skill|agent)\b'
|
||||
- '^This\b'
|
||||
|
||||
@@ -10,6 +10,10 @@ Additional documentation agents load on demand.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `description-quality.md` | Qualitative guide for borderline description findings — action-verb rules, specificity criteria, proactive-use caveat, length limits. |
|
||||
| `field-inventory.md` | Canonical list of valid CC and Copilot agent definition fields. Load when the script needs authoritative field lists for structural validation. |
|
||||
| `description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked contract, and the internal-mechanics FAIL. |
|
||||
| `body-and-delegation.md` | Rubric for the body, delegation and comment-discipline dimensions — the delegation FAIL, why agents take no body word gate, and what an agent body is for. |
|
||||
| `scope-plugin-apm.md` | Contract for a single vendor-neutral `.apm/agents/<name>.agent.md` file — allowlist, dimension routing, and the dimensions that do not apply. |
|
||||
| `scope-project-user.md` | Contract for a Claude Code / Copilot file pair — counterpart derivation, provider field rules, and pair consistency. |
|
||||
| `validation-scripts.md` | Loaded only when a Step 1 script fails or cannot run — scope-detection walk-up, manual fallback checks, and known script failures. |
|
||||
| `field-inventory.md` | Authoritative field lists, read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM allowlist. |
|
||||
| `sources.md` | Research provenance records for skill content. Load only when tracing the origin of a specific rule or field constraint. |
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Body, Delegation and Comment Discipline Reference
|
||||
|
||||
Upstream source: Claude Code subagent and plugin references, GitHub Copilot custom-agents
|
||||
configuration. House contract: ADR-0020, the context budget.
|
||||
|
||||
Read this when judging the **body**, **delegation** and **comment-discipline** dimensions.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## Agents take no body word gate
|
||||
|
||||
ADR-0020 gates a skill body at 600 words SUGGESTION / 900 FAIL and deliberately gates an agent body
|
||||
at nothing. The two are not the same construct: a skill body is loaded into the caller's live
|
||||
context and competes with the conversation already there, while an agent body *becomes* the system
|
||||
prompt of a fresh context that has nothing else in it. The rationale for the 900-word ceiling does
|
||||
not transfer, so:
|
||||
|
||||
- **Never report an agent body as too long on a word count.** There is no number to cite.
|
||||
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate.bats` pins its absence with a
|
||||
body far past 900 words that must still pass, and adding one would contradict the ADR.
|
||||
- The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which
|
||||
`validate.sh` already reports as a SUGGESTION because content past it is silently truncated.
|
||||
|
||||
Length is judged through the delegation check below instead, which is the defect a word count was
|
||||
standing in for anyway.
|
||||
|
||||
## The delegation check
|
||||
|
||||
A plugin-scope agent is a single `.apm/agents/<name>.agent.md` file with no sibling `references/`
|
||||
directory. It cannot progressively disclose to itself — it can only delegate to skills. So a
|
||||
procedure spelled out in an agent body that a skill the agent invokes already owns is not a
|
||||
shortcut: it is a second copy of that procedure, and the second copy drifts. This is the
|
||||
characteristic agent defect, the way a stale README row is the characteristic skill defect.
|
||||
|
||||
**An agent body that restates a procedure owned by a skill it can invoke is a FAIL.** The Fix is
|
||||
always the same shape: invoke `<skill>` instead.
|
||||
|
||||
How to apply it: for each procedural block in the body — a rule list, a numbered sequence, a
|
||||
constraint table — ask which skill owns that procedure. If the agent names that skill anywhere (its
|
||||
dispatch table, its routing prose, its frontmatter), the block is a restatement and the skill is
|
||||
already there to be invoked.
|
||||
|
||||
Worked example. The three `*-orchestrate` agents exist to compose domain skills — `git-orchestrate`
|
||||
(933 body words), `gitea-orchestrate` (1,199) and `apm-orchestrate` (1,080) — so any step they
|
||||
spell out that the composed skill already owns is the defect. `git-orchestrate:24-31` carries a
|
||||
"Hard rules" list (Conventional Commits types, atomic commits, never commit secrets, git trailers)
|
||||
that `git-commits` owns and that `git-orchestrate:44` routes to by name; `:39` concedes the point
|
||||
outright, noting the sub-skills "carry their own local copies of these rules". Two copies, one
|
||||
authority, and nothing keeping them in step.
|
||||
|
||||
What is **not** a finding under this rule, because no skill owns it:
|
||||
|
||||
- The dispatch table itself — which operation routes to which skill.
|
||||
- Safety gates the agent enforces before dispatching, and refusals it makes on its own authority.
|
||||
- The input contract and the structured output the agent's caller consumes.
|
||||
- Session state the agent carries across skill invocations.
|
||||
|
||||
## What the body is for
|
||||
|
||||
Include what the fresh context lacks:
|
||||
|
||||
- A direct role instruction opening the prompt: `You are a [role]. When invoked, [action].`
|
||||
- One bounded job, stated so the agent knows what it must refuse.
|
||||
- The dispatch, gates, inputs and outputs listed above.
|
||||
- Non-obvious environment facts and project-specific conventions it cannot infer.
|
||||
- One default per decision point with one escape hatch.
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Comment discipline
|
||||
|
||||
Inspect every comment block in the YAML frontmatter and apply the core test to each: *would the
|
||||
agent get this wrong without this comment?* Template scaffolding — `# Optional. <long
|
||||
explanation>`, more than a line or two of inline guidance per field — belongs to development, not
|
||||
to a shipped file. At plugin/APM scope the stakes are higher than tidiness: `apm compile` copies
|
||||
frontmatter verbatim to every target, `<!-- ... -->` is not valid YAML, and `validate.sh` FAILs a
|
||||
frontmatter block that still contains one.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- The body restates a procedure owned by a skill the agent can invoke — Fix: invoke `<skill>`
|
||||
instead
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- Frontmatter comments are template scaffolding rather than instruction, or are HTML comments at
|
||||
plugin/APM scope
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- The body does not open with a direct role instruction
|
||||
- The job the agent describes is unbounded, or bounded only implicitly
|
||||
- A rationale is missing from a rule the agent is expected to enforce — present but unexplained
|
||||
- Comments are useful but verbose enough to bury the field they annotate
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
@@ -9,41 +8,120 @@ source_keys:
|
||||
|
||||
# Agent Description Quality Reference
|
||||
|
||||
Load this file when a description finding is borderline and you need to make a precise call.
|
||||
Upstream source: Claude Code subagent reference, GitHub Copilot custom-agents configuration.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than either
|
||||
platform's schema rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## Action-verb opening
|
||||
## Why the description is the expensive part
|
||||
|
||||
The description must open with an imperative or present-tense verb that describes what the agent does ("Reviews...", "Audits...", "Generates...", "Analyzes..."). Avoid:
|
||||
- Noun phrases: "An agent that..." — no verb
|
||||
- "This agent..." or "Use this when..." — passive framing
|
||||
- "Helps with..." — too vague to be a clear verb
|
||||
At startup an agent loads only the `name` and `description` of every installed skill and agent.
|
||||
The body is never seen until the agent is invoked. The description therefore carries the entire
|
||||
triggering burden **and** is paid for in every session, whether the agent fires or not.
|
||||
|
||||
**Borderline call:** "Validates and reviews..." is acceptable — two verbs is fine if both are specific. "Assists in reviewing..." is not — "assists" is vague filler.
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the caller takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## Specificity of trigger condition
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
The description must state what specifically triggers the agent. Generic phrasing fails:
|
||||
- Too vague: "when the user needs help with agents"
|
||||
- Acceptable: "when the user says 'audit this agent', 'check if my agent follows best practices', or wants to know if an agent pair is ready to ship"
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
Include indirect triggers: "even if they don't use the word 'audit'" or "even if the user doesn't phrase it as a review request". If the agent should activate on a recognisable user goal (not just literal keyword matches), name that goal.
|
||||
- **`disable-model-invocation: true` or `user-invocable: false`** — the agent is hand-invoked. Its
|
||||
description is never matched against user intent, so it is not a routing string. It carries **one
|
||||
plain human-facing sentence** stating what the agent does. Audit it for that and nothing else.
|
||||
Reporting a missing trigger clause, a missing boundary clause or absent indirect triggers on a
|
||||
hand-invoked agent is a wrong finding, not a strict one. Both fields are Copilot-only and neither
|
||||
is on the vendor-neutral APM allowlist, so this case arises in a Copilot `.agent.md` at
|
||||
project/user scope and nowhere else. Its Claude Code counterpart has no equivalent field and stays
|
||||
model-invoked, so the two halves of the pair carrying differently shaped descriptions is expected
|
||||
there rather than a pair-consistency finding.
|
||||
- **No such flag** — the agent is model-invoked and the rest of this file applies.
|
||||
|
||||
**Borderline call:** If the description covers direct triggers but omits common indirect phrasings that a user would plausibly use, mark as SUGGESTION (not FAIL) — the agent still activates, just less reliably.
|
||||
## The three-part shape
|
||||
|
||||
## `Use proactively`
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
For CC files: including "Use proactively" signals the CC runtime to offer the agent unprompted when conditions are met. This is CC-specific — use it when the agent should activate without an explicit user request.
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This agent ...` —
|
||||
the caller is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source.
|
||||
|
||||
For Copilot files: this phrase has no effect. Use `user-invocable: false` / `disable-model-invocation: true` for equivalent Copilot behavior. Flag `Use proactively` in a Copilot description as a SUGGESTION (not FAIL) — it causes no harm, just has no effect.
|
||||
Everything else belongs in the body or in the plugin's `README.md`.
|
||||
|
||||
## Length and hard limits
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
- CC agent descriptions: no documented character limit, but keep under 500 characters to avoid truncation in UI contexts.
|
||||
- Copilot agent descriptions: no separate documented limit, but the overall 30,000-character body limit applies to the full file.
|
||||
- Skill descriptions (SKILL.md): hard 1024-character limit enforced by the platform.
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on an agent whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Do not use when
|
||||
## Near-miss exclusions
|
||||
|
||||
Include a "Do not use when..." clause only if a near-miss agent or skill exists that could steal activations. Omitting it is not a finding. Including it is correct when there is a real confusion risk (e.g., `/agent-audit` vs `/skill-audit`).
|
||||
Add a boundary clause only where a sibling skill or agent could plausibly steal the activation. Use
|
||||
strong near-misses — queries that share keywords but need something different — not weak ones. One
|
||||
boundary clause per genuine near-miss; a list of four is enumeration wearing a boundary's clothes.
|
||||
|
||||
**Borderline call:** If the "Do not use when" clause is present but the exclusion described is already obvious from context, mark as SUGGESTION to tighten or remove — not FAIL.
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# FAIL — a noun-phrase opener rather than a trigger, capability enumeration in
|
||||
# place of one capability clause, and no boundary clause at all, preloaded into
|
||||
# every session forever. (The live git-orchestrate description, 254 chars.)
|
||||
description: Orchestrates git workflow operations for other agents. Invoke when a
|
||||
caller needs a multi-step or destructive git operation (rebase, force-push, branch
|
||||
deletion) coordinated across domain skills with safety gates, session context, and
|
||||
structured results.
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The operation list and the
|
||||
# safety-gate mechanics are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when an agent caller needs a multi-step or destructive git operation
|
||||
dispatched and safety-gated. Not conversational git help -> git-workflow.
|
||||
```
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut. Agent
|
||||
descriptions have no platform-documented ceiling of their own — unlike a skill's 1,024-character
|
||||
spec limit, the 400-character house ceiling is the only hard limit there is, so do not go looking
|
||||
for a backstop behind it.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared agent", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This agent ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`. There is no action-verb rule
|
||||
here and never was a defensible one: an `Orchestrates ...` or `Audits ...` opener is a catalogue
|
||||
entry, not a trigger.
|
||||
- **Vague capabilities** ("helps with agents" where "audits an agent definition pair" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
|
||||
file in the authoring source. No script checks this for an agent file — `validate.sh` resolves
|
||||
boundary targets for skills only, so resolve the name yourself against `plugins/*/.apm/skills/`
|
||||
and `plugins/*/.apm/agents/`.
|
||||
- **`Use proactively` in a Copilot or vendor-neutral description.**
|
||||
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
|
||||
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Plugin/APM Scope Contract
|
||||
|
||||
Read this when the agent file sits at `<package>/.apm/agents/<name>.agent.md` — a single
|
||||
vendor-neutral file inside an APM package, with no counterpart anywhere.
|
||||
|
||||
## What is different here
|
||||
|
||||
`apm compile` copies an agent's frontmatter **verbatim** to every target harness. There is no
|
||||
per-target integrator to reconcile a Claude-Code-only field with a Copilot-only one, so the file
|
||||
cannot carry either (ADR-0016). That single fact drives everything below.
|
||||
|
||||
## Frontmatter allowlist
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `references/field-inventory.md`. Read
|
||||
them from there. Do not recite the list in a finding, do not work from memory, and do not trust any
|
||||
restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it
|
||||
has changed before, and `validate.sh` parses that same section at load time, so a recitation is a
|
||||
copy that can disagree with the check the agent just ran.
|
||||
|
||||
`field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
|
||||
is not. Read that note before arguing with a finding about it.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at this scope:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| any frontmatter key outside the allowlist; body over the 30,000-character Copilot limit | Provider safety |
|
||||
| everything else — missing or malformed field, `name` not matching the filename stem, empty body, absent frontmatter, template HTML comments, description length | Structure |
|
||||
| — | Pair consistency never applies |
|
||||
|
||||
**Provider safety means something else here.** At project/user scope it asks whether a field leaked
|
||||
across the Claude Code / Copilot boundary. At this scope there is no boundary and no pair: it asks
|
||||
whether every field survives a verbatim copy to *every* target. Report it in those terms — a
|
||||
finding phrased as "CC-only field in a Copilot file" is the wrong finding here.
|
||||
|
||||
**Pair consistency never applies.** There is one file by design. `validate.sh` never emits a
|
||||
missing-counterpart FAIL at this scope, and neither do you, under any circumstance. Drop
|
||||
`pair-consistency` from the Step 4 coverage line rather than reporting it clean.
|
||||
|
||||
## Behaviour the schema cannot express
|
||||
|
||||
Read the description and body. If either implies a need the vendor-neutral frontmatter can no
|
||||
longer express — a tool restriction, `isolation`, `memory`, or another Claude-only behaviour a
|
||||
hand-authored CC file could have declared — flag it as a **SUGGESTION, never a FAIL**. This is a
|
||||
known upstream schema limitation (ADR-0016), not an authoring mistake, and the finding exists to
|
||||
give the author visibility into the gap rather than to imply the schema can be made to close it.
|
||||
|
||||
Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it.
|
||||
A denylist-shaped restriction is the available half of that — see `field-inventory.md`.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Project and User Scope Contract
|
||||
|
||||
Read this when the agent file is not under `.apm/agents/` — a Claude Code `.md` and Copilot CLI
|
||||
`.agent.md` **pair**, at project scope (`<repo>/.claude/agents/` and `<repo>/.github/agents/`) or
|
||||
user scope (`~/.claude/agents/` and `~/.copilot/agents/`). `validate.sh` derives the counterpart
|
||||
from whichever half it was handed; audit both.
|
||||
|
||||
## The pair is a house convention
|
||||
|
||||
Neither platform requires a counterpart file. The pair is a kyberforge convention (ADR-0005), so a
|
||||
missing counterpart is a FAIL against **this repo's** convention and must be labelled that way in
|
||||
the finding, not presented as a platform spec failure.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at these scopes:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| a Claude-Code-only field in the Copilot file, a Copilot-only field in the CC file, a tool the runtime withholds from subagents, body over the 30,000-character Copilot limit | Provider safety |
|
||||
| counterpart file not found | Pair consistency |
|
||||
| everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure |
|
||||
|
||||
The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of
|
||||
`references/field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
|
||||
those same sections, so any restatement is a copy that can disagree with the check (ADR-0009).
|
||||
|
||||
## Field and naming rules that differ by provider
|
||||
|
||||
- `name` must match the filename stem in a **Copilot CLI** `.agent.md`. Claude Code imposes no such
|
||||
rule, so a CC file whose `name` differs from its filename is not a finding.
|
||||
- A Copilot **cloud/IDE** agent — one under `.github/copilot/agents/` — may omit `name` entirely.
|
||||
If it carries one, it still has to be kebab-case.
|
||||
- `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent
|
||||
unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags
|
||||
it. The Copilot equivalent is `disable-model-invocation` / `user-invocable`, which changes the
|
||||
description contract entirely — see `references/description-quality.md`, Step 0.
|
||||
|
||||
## Pair consistency
|
||||
|
||||
Check that:
|
||||
|
||||
- Both files exist.
|
||||
- Both system prompt bodies are non-empty (`validate.sh` covers this; do it by hand only when the
|
||||
script could not run).
|
||||
- The two files describe the **same job**. Divergent capability claims across the pair mean one
|
||||
half was edited and the other was not, which is the defect this dimension exists to catch.
|
||||
- Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is
|
||||
the Step 0 case in `references/description-quality.md`, not a pair-consistency finding.
|
||||
|
||||
Keep `pair-consistency` in the Step 4 coverage line at these scopes.
|
||||
@@ -14,7 +14,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
@@ -22,7 +22,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
@@ -30,7 +30,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
@@ -38,7 +38,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
@@ -46,7 +46,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Validation Scripts Reference
|
||||
|
||||
Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting.
|
||||
Nothing here is needed on a clean run.
|
||||
|
||||
## Report the gap, do not guess
|
||||
|
||||
If a script cannot run at all — Bash denied, `python3` unavailable, `vale` not installed — say so
|
||||
as an **INFO** finding naming the script and the missing dependency, then fall back to the manual
|
||||
checks below. An INFO never changes PASS/FAIL. Silently omitting the dimension a script would have
|
||||
covered reports a clean audit that checked less than it claims to have checked.
|
||||
|
||||
## How the scripts detect scope
|
||||
|
||||
`validate.sh` and `validate-provenance.sh` walk up from the agent file's directory and stop at the
|
||||
first of these:
|
||||
|
||||
1. An `apm.yml` carrying a top-level `type: instructions|skill|hybrid|prompts` line — **plugin/APM
|
||||
scope**, and that directory is the package root. An `apm.yml` with no `type:` is a
|
||||
marketplace-only manifest: skip it and keep walking.
|
||||
2. `$HOME` — **user scope**, checked before `.git` so a dotfiles-managed home directory that is its
|
||||
own repo cannot shadow it.
|
||||
3. A `.git` directory or file — **project scope**.
|
||||
4. The filesystem root — **project scope**.
|
||||
|
||||
`plugin.json` and `.claude-plugin/plugin.json` are not scope signals. A directory holding only a
|
||||
`plugin.json` and no `apm.yml` falls through to project or user scope.
|
||||
|
||||
`validate-provenance.sh` exits 0 silently when that walk does not land on a package root, and again
|
||||
when the package has no provenance data. Silence from it is a pass, not a skip you need to
|
||||
investigate.
|
||||
|
||||
## Manual fallback
|
||||
|
||||
**Every scope:** required fields present (`name`, `description`, non-empty body); `name` is
|
||||
kebab-case; no `FILL IN:` placeholders in the description or body; the description at or under 400
|
||||
characters measured on the folded YAML value.
|
||||
|
||||
**Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter;
|
||||
no frontmatter key outside the `apm-agent-allowlist` section of `references/field-inventory.md` —
|
||||
open that file, do not work from memory.
|
||||
|
||||
**Project/user scope:** the counterpart file exists; `name` matches the filename stem in the
|
||||
Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in
|
||||
the Copilot file and none from `copilot-only-fields` in the CC file, both read from
|
||||
`references/field-inventory.md`.
|
||||
|
||||
## Script-specific failures
|
||||
|
||||
- **`Error: field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
|
||||
`references/field-inventory.md` at load time and refuses to run without it, rather than falling
|
||||
back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not
|
||||
work around it.
|
||||
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full
|
||||
Step 3 judgment for the dimensions it would have covered. The `Kyberforge` style is scoped to
|
||||
`**/agents/*.md` and `**/*.agent.md`, and `KyberforgeCopilot` to `**/*.agent.md` alone — a file
|
||||
outside those globs is silently not linted.
|
||||
- **`E100 Runtime error ... does not exist` (exit 2) from `vale-wrap.sh`.** An explicit relative
|
||||
`--config` was passed. Pass none: the wrapper locates its own `assets/vale/.vale.ini` from its
|
||||
own path. Do not read this exit code as vale being unavailable.
|
||||
- **A path argument that does not exist is a hard error** in `vale-wrap.sh`, deliberately: bare
|
||||
`vale` would fall back to reading stdin and print a clean-looking `0 errors ... in stdin`, which
|
||||
the `0 files` guard above does not catch.
|
||||
@@ -90,6 +90,23 @@ SUBAGENT_UNAVAILABLE_TOOLS = {
|
||||
# Copilot body length limit (chars) — content beyond this is silently truncated
|
||||
COPILOT_BODY_LIMIT = 30000
|
||||
|
||||
# ADR-0020 description budget. An agent's name + description is preloaded into
|
||||
# every session exactly like a skill's, so agents take the SAME description
|
||||
# gates. These two constants are DUPLICATED from scripts/skill-size-check.sh
|
||||
# and skill-audit/scripts/validate.sh rather than shared from one file: a
|
||||
# cache-installed plugin's scripts cannot read files outside their own plugin
|
||||
# directory, so there is no single source to share (same rationale as
|
||||
# vale-wrap.sh's per-plugin duplication). tests/test-skill-size-check.sh
|
||||
# asserts all copies agree, so drift fails CI rather than silently diverging.
|
||||
#
|
||||
# Agents deliberately take NO body word gate, and adding one here would
|
||||
# contradict ADR-0020: a skill body is loaded into the caller's context and
|
||||
# competes with the live conversation, while an agent body becomes the system
|
||||
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
|
||||
# not transfer. Agent body length falls out of the delegation rule instead.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
|
||||
# --- Helpers (shared by every scope) ---
|
||||
failed = False
|
||||
suggestions = []
|
||||
@@ -122,6 +139,80 @@ def get_frontmatter_keys(fm):
|
||||
keys.add(m.group(1))
|
||||
return keys
|
||||
|
||||
def normalize_scalar(value):
|
||||
return re.sub(r'\s+', ' ', value).strip()
|
||||
|
||||
def fold_description_fallback(fm_text):
|
||||
"""Resolve `description:` without PyYAML.
|
||||
|
||||
Not a YAML parser — it recognises exactly the shapes agent frontmatter
|
||||
uses: an inline scalar (optionally quoted, optionally continued on
|
||||
following indented lines) and a `>`/`|` block scalar with optional
|
||||
indentation and chomping indicators.
|
||||
"""
|
||||
lines = fm_text.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^description:[ \t]*(.*)$', line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1).strip()
|
||||
block = bool(re.match(r'^[>|][0-9]*[-+]?$|^[>|][-+]?[0-9]*$', head))
|
||||
parts = [] if block else [head]
|
||||
for nxt in lines[i + 1:]:
|
||||
if not nxt.strip():
|
||||
parts.append('')
|
||||
continue
|
||||
if not re.match(r'^[ \t]', nxt):
|
||||
break
|
||||
parts.append(nxt.strip())
|
||||
value = ' '.join(parts)
|
||||
if not block:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in '"\'':
|
||||
value = value[1:-1]
|
||||
return value
|
||||
return ''
|
||||
|
||||
def description_value(fm):
|
||||
"""The description VALUE with YAML folding resolved.
|
||||
|
||||
extract_field() reads one raw line, which is the right shape for the
|
||||
presence and placeholder checks but the wrong one for a length gate: a
|
||||
`>`-folded description measured off its first raw line is not the value the
|
||||
host preloads. Parse instead of regexing the raw text.
|
||||
"""
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(fm)
|
||||
if isinstance(data, dict):
|
||||
value = data.get('description')
|
||||
if isinstance(value, str):
|
||||
return normalize_scalar(value)
|
||||
if value is not None:
|
||||
return normalize_scalar(str(value))
|
||||
return ''
|
||||
except Exception:
|
||||
pass
|
||||
return normalize_scalar(fold_description_fallback(fm))
|
||||
|
||||
def check_description_budget(fm, local_fname):
|
||||
"""ADR-0020 description gates — identical for every scope."""
|
||||
value = description_value(fm)
|
||||
if not value:
|
||||
return
|
||||
dlen = len(value)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body — {local_fname}")
|
||||
elif dlen > DESC_SUGGEST_CHARS:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers "
|
||||
f"— {local_fname}")
|
||||
|
||||
def extract_tools_list(fm):
|
||||
"""Extract tool names from the tools frontmatter field (space or comma separated)."""
|
||||
val = extract_field(fm, 'tools')
|
||||
@@ -273,6 +364,7 @@ def check_apm_agent_file(fpath, allowlist, stem):
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(desc_val):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
check_description_budget(fm, local_fname)
|
||||
|
||||
# body — required, non-empty, no placeholder; same Copilot truncation risk
|
||||
# applies since this file compiles verbatim into a real Copilot file downstream.
|
||||
@@ -346,6 +438,7 @@ def check_file(fpath, file_provider):
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(desc_val):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
check_description_budget(fm, local_fname)
|
||||
|
||||
# body
|
||||
if not body.strip():
|
||||
|
||||
@@ -31,6 +31,26 @@ description: A valid agent description.
|
||||
${extra_frontmatter}
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
}
|
||||
|
||||
# Helper: same shape as make_apm_agent, but the description is supplied
|
||||
# verbatim — used by the ADR-0020 description-budget tests.
|
||||
make_apm_agent_with_desc() {
|
||||
local root="$1" name="$2" desc="$3"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
|
||||
---
|
||||
name: ${name}
|
||||
description: ${desc}
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
}
|
||||
@@ -607,6 +627,125 @@ EOF
|
||||
refute_output --partial "FAIL"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — description budget (250 SUGGESTION / 400 FAIL)
|
||||
#
|
||||
# Agents take the SAME description gates as skills: name + description is
|
||||
# preloaded into every session identically. Agents take NO body word gate — see
|
||||
# the final test in this block, which pins that asymmetry.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: agent description of exactly 250 chars raises no suggestion" {
|
||||
local root="$TMPDIR/pkg"
|
||||
make_apm_agent_with_desc "$root" "my-agent" "$(python3 -c "print('x' * 250)")"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_success
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: agent description of 251 chars raises a SUGGESTION and still exits 0" {
|
||||
local root="$TMPDIR/pkg"
|
||||
make_apm_agent_with_desc "$root" "my-agent" "$(python3 -c "print('x' * 251)")"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
assert_output --partial "description is 251 chars"
|
||||
}
|
||||
|
||||
@test "ADR-0020: agent description of exactly 400 chars is a SUGGESTION, not a FAIL" {
|
||||
local root="$TMPDIR/pkg"
|
||||
make_apm_agent_with_desc "$root" "my-agent" "$(python3 -c "print('x' * 400)")"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: agent description of 401 chars FAILs and exits non-zero" {
|
||||
local root="$TMPDIR/pkg"
|
||||
make_apm_agent_with_desc "$root" "my-agent" "$(python3 -c "print('x' * 401)")"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "description is 401 chars"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: agent description length is measured after YAML folding is resolved" {
|
||||
local root="$TMPDIR/pkg"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
# 11 folded lines of 40 chars + 10 joining spaces = 450 characters. Read off
|
||||
# the raw `description: >` line it is 1 character and passes.
|
||||
{
|
||||
echo "---"
|
||||
echo "name: my-agent"
|
||||
echo "description: >"
|
||||
python3 -c "print('\n'.join([' ' + 'x' * 40] * 11))"
|
||||
echo "---"
|
||||
echo ""
|
||||
echo "You are a test agent."
|
||||
} > "$root/.apm/agents/my-agent.agent.md"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "description is 450 chars"
|
||||
}
|
||||
|
||||
@test "ADR-0020: the description gate applies at project scope too" {
|
||||
local root="$TMPDIR/project"
|
||||
local desc
|
||||
desc="$(python3 -c "print('x' * 401)")"
|
||||
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
|
||||
cat > "$root/.claude/agents/my-agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: $desc
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: agents take NO body word gate — a body far over the 900-word skill ceiling passes" {
|
||||
local root="$TMPDIR/pkg"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
# Deliberate asymmetry, not an oversight: a skill body is loaded into the
|
||||
# caller's context and competes with the live conversation, while an agent
|
||||
# body becomes the system prompt of a fresh context. ADR-0020 gates the
|
||||
# former at 900 words and explicitly declines to gate the latter. If a body
|
||||
# word gate is ever added here, it contradicts the ADR.
|
||||
{
|
||||
echo "---"
|
||||
echo "name: my-agent"
|
||||
echo "description: A valid agent description."
|
||||
echo "---"
|
||||
echo ""
|
||||
python3 -c "print(' '.join(['word'] * 1500))"
|
||||
} > "$root/.apm/agents/my-agent.agent.md"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_success
|
||||
refute_output --partial "FAIL"
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "a bare plugin.json with no apm.yml is no longer plugin scope — falls through to project scope" {
|
||||
local root="$TMPDIR/proj-legacy-plugin-json"
|
||||
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
|
||||
|
||||
@@ -30,10 +30,15 @@ bash scripts/new-agent.sh security-reviewer ~
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `SKILL.md` | Skill instructions for agents — gotchas, the create/improve dispatch table, the scope dispatch table, the shared gates, and validation/close |
|
||||
| `scripts/new-agent.sh` | Scaffolds agent definition file(s) from templates — a single `.apm/agents/<name>.agent.md` at plugin/APM scope, or a Claude Code + Copilot CLI pair at project/user scope |
|
||||
| `references/deployment-modes.md` | Plugin/APM vs project vs user scope: restrictions, scoped identifiers, path conventions |
|
||||
| `references/scripts.md` | Conventions for new-agent.sh and any future scripts: contract, template variables, file placement, error messages |
|
||||
| `references/create.md` | Create flow: prerequisites, scaffold and scope walk-up, what to fill in, package-root `sources.md` |
|
||||
| `references/improve.md` | Improve flow: signal verification, root-cause grouping, generalizing, delegation over growth, ADR-0020 retrofit |
|
||||
| `references/contract.md` | Description and body contract: three-part description shape, 250/400 tiers, delegation rule in place of a body word gate, invocation axis |
|
||||
| `references/plugin-scope.md` | Plugin/APM scope field rules for the single vendor-neutral file, plus its pre-audit checklist |
|
||||
| `references/project-user-scope.md` | Project/user scope field rules for the Claude Code + Copilot pair, both Copilot formats, plus its pre-audit checklist |
|
||||
| `references/deployment-modes.md` | Scope hierarchy and precedence, scoped identifiers, cache isolation, path conventions |
|
||||
| `references/scripts.md` | Conventions for new-agent.sh and the templates it copies: contract, template variables, file placement, error messages |
|
||||
| `references/sources.md` | Research provenance — sources that informed this skill |
|
||||
| `assets/templates/claude-code.md` | Annotated Claude Code agent definition template (project/user scope) |
|
||||
| `assets/templates/copilot.agent.md.template` | Annotated Copilot CLI agent definition template (project/user scope) |
|
||||
|
||||
@@ -1,18 +1,9 @@
|
||||
---
|
||||
name: agent-author
|
||||
description: >
|
||||
Use when the user wants to create a new agent definition file from scratch
|
||||
("write an agent for X", "build a subagent that does Y", "create an agent
|
||||
definition for Z"), or improve an existing one. Handles agent definitions at
|
||||
plugin/APM, project, and user scope. Project and user scope always generate
|
||||
a Claude Code (`.md`) + Copilot CLI (`.agent.md`) file pair in one pass;
|
||||
plugin/APM scope generates a single vendor-neutral `.apm/agents/<name>.agent.md`
|
||||
file instead (no per-target Claude Code / Copilot split). Also use when the
|
||||
user provides inline feedback about an agent's behavior and wants it applied,
|
||||
or when a grill session has produced findings the user wants acted on — even
|
||||
if they don't say "improve" explicitly. Do not use for read-only review —
|
||||
examine agent files manually or run a grill session to generate improvement
|
||||
signals. Do not use to author skills — use /skill-author instead.
|
||||
Use when the user wants to create a new agent definition file from scratch, or
|
||||
apply grill findings, audit findings, or inline feedback to an existing one.
|
||||
Not read-only review -> `agent-audit`. Not skills -> `skill-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -20,240 +11,52 @@ metadata:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
- github-cli-plugin-reference
|
||||
- github-plugins-creating
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- At plugin/APM scope, bump the resolved package's `apm.yml` `version` after every change — minor for a new agent, patch for a fix. Consumers compare this version to detect updates; skipping it hides the change.
|
||||
- At plugin/APM scope, `tools` and all Claude-only fields (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) are omitted entirely, not merely restricted (ADR-0016: `apm compile` copies frontmatter verbatim to both harnesses with no per-target integrator, so a harness-specific value is wrong on at least one). Only project/user scope supports these fields.
|
||||
- `disallowedTools` is the one exception, on **shape**, not favouritism. `tools` is an *allowlist* whose vocabulary differs per harness (Claude tool names vs Copilot's `execute`/`read`/`edit`/`search`/`agent`/`web`), so verbatim copy makes one value wrong on one target. A *denylist* cannot fail that way: an unrecognised name denies nothing, so the worst case is a missing fence, never a wrong grant. Claude Code honours it for plugin subagents — `docs/research/docs/claude-code-plugins/agent-definition.md:99` lists the three fields plugin agents ignore (`hooks`, `mcpServers`, `permissionMode`) and this is not one. Write it on every read-only plugin-scope agent (ADR-0016's 2026-08-14 amendment).
|
||||
- That fence is partial: it denies only the tools it names. It does not deny `Bash`, which a plugin-scope agent with no `tools` inherits, so a shell redirect still writes. Say the agent is read-only in the body too.
|
||||
- An `apm.yml` with no top-level `type:` field is a marketplace-only manifest, not a package root — the walk-up skips it and keeps going.
|
||||
- `AskUserQuestion`, `EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup`, and `WaitForMcpServers` are never available to any subagent regardless of the `tools` field. Exception: `ExitPlanMode` is available when the parent session runs in `permissionMode: plan`.
|
||||
- Duplicate `name` values in the same scope: Claude Code silently discards one without warning. Always verify uniqueness before shipping.
|
||||
- Plugin agents in subdirectories get scoped identifiers (`plugin:folder:name`) — keep agents flat in `agents/` to avoid this. Applies to project/user-scope Claude Code agents only.
|
||||
- Copilot CLI agent files **must** use the `.agent.md` extension — a plain `.md` file isn't picked up. The plugin/APM-scope single file also ends in `.agent.md` by convention, but it's vendor-neutral, not Copilot-only — it compiles to Claude Code too.
|
||||
- Copilot has no `permissionMode`, `maxTurns`, `isolation`, or `memory` fields — do not include them in project/user-scope Copilot files.
|
||||
- `model` resolution order for Claude Code: `CLAUDE_CODE_SUBAGENT_MODEL` env var → per-invocation parameter → frontmatter `model` → main session model. The frontmatter value is a low-priority default, not a guarantee.
|
||||
- At plugin/APM scope `tools` and every Claude-only field are omitted entirely, not merely ignored: `apm compile` copies frontmatter verbatim to both harnesses, so fencing a read-only agent with `tools:` is wrong on one of them. `disallowedTools` is the one restriction that survives (ADR-0016).
|
||||
- That fence is partial. It denies only the tools it names, never `Bash`, which a plugin-scope agent inherits — a shell redirect still writes. State the read-only boundary in the body too.
|
||||
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is an `agent-audit` FAIL.
|
||||
- Duplicate `name` values in one scope: Claude Code discards one silently. Verify uniqueness before shipping.
|
||||
|
||||
## Route
|
||||
## Step 1 — Dispatch
|
||||
|
||||
If the destination resolves to plugin/APM scope (scope detection in Step 1 finds a `type:`-bearing `apm.yml` at or above the root), read `references/deployment-modes.md`.
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| No agent file at the target path(s) | Create | `references/create.md` |
|
||||
| A file exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| A file exists, no signals | Stop and ask | — |
|
||||
|
||||
Determine which flow before touching the filesystem:
|
||||
Signals: grill output, `agent-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
|
||||
- **Neither `<name>.md` nor `<name>.agent.md` exist at the target paths** → follow **Creating a new agent**
|
||||
- **At least one file exists + improvement signals present** → follow **Improving an existing agent**
|
||||
- **At least one file exists + no signals** → ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
Read only the reference for the resolved flow. Capture `git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
Signals: grill session output, inline user feedback, session context describing what went wrong.
|
||||
## Step 2 — Scope
|
||||
|
||||
## Creating a new agent
|
||||
Scope decides which fields exist, so resolve it first. `scripts/new-agent.sh` walks up for a `type:`-bearing `apm.yml` and prints the scope it chose — read that output.
|
||||
|
||||
### Prerequisites
|
||||
| Resolved scope | Emits | Read |
|
||||
|---|---|---|
|
||||
| plugin/APM | one vendor-neutral `.apm/agents/<name>.agent.md` | `references/plugin-scope.md` |
|
||||
| project or user | a Claude Code `.md` + Copilot `.agent.md` pair | `references/project-user-scope.md` |
|
||||
|
||||
Before touching the filesystem, confirm you have:
|
||||
- [ ] Agent name (kebab-case, e.g. `code-reviewer`)
|
||||
- [ ] Root directory (a path inside a package for plugin/APM scope, project root, or `~` for user scope)
|
||||
- [ ] Agent purpose — one sentence describing the task this agent handles
|
||||
- [ ] Trigger condition — when should the runtime delegate to this agent?
|
||||
Read only the file for the resolved scope; the other describes fields this run cannot use. If precedence, cache isolation or path conventions matter, read `references/deployment-modes.md`.
|
||||
|
||||
If any are missing, stop and ask before proceeding. Then capture `git log --oneline -1` before touching the filesystem — Step 5 needs it to verify a real commit landed.
|
||||
## Step 3 — Contract
|
||||
|
||||
Verify `kyberforge:agent-audit` is available — it ships with the kyberforge plugin and is co-installed with this skill. If unavailable, stop and tell the user to install the kyberforge plugin before continuing.
|
||||
Before writing or editing a `description`, or restructuring a body, read `references/contract.md` — the three-part shape, banned content, the delegation rule and the body pattern.
|
||||
|
||||
### Step 1 — Scaffold
|
||||
Gates `agent-audit` enforces at every scope:
|
||||
|
||||
Run the scaffold script with the agent name and root directory:
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <name>` that resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only: an agent's `name` and `description` is preloaded into every session exactly as a skill's is.
|
||||
- **Body** — no word gate, and a delegation check in its place: name the skill to invoke rather than restating what it does.
|
||||
- **Invocation** — decide whether the agent is model-delegated or reached only by name. Only Copilot's cloud/IDE format expresses that in frontmatter (`disable-model-invocation`, `user-invocable`).
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh <name> <root>
|
||||
```
|
||||
## Step 4 — Validate and close
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
|
||||
bash scripts/new-agent.sh deploy-assistant .
|
||||
bash scripts/new-agent.sh security-reviewer ~
|
||||
```
|
||||
Invoke `agent-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
|
||||
|
||||
**Scope detection (script handles this automatically).** The script walks up from `<root>` for a package boundary — same shape `agent-audit`'s `validate.sh` uses:
|
||||
- Nearest ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) → **plugin/APM scope** → `<package-root>/.apm/agents/<name>.agent.md` (single vendor-neutral file). A `type:`-less `apm.yml` is marketplace-only — skipped, walk continues upward.
|
||||
- No such `apm.yml`, `<root>` is a project directory → **project scope** (unchanged) → `<root>/.claude/agents/<name>.md` + `<root>/.github/agents/<name>.agent.md`
|
||||
- `<root>` is exactly `~` (checked directly, no walk-up) → **user scope** (unchanged) → `~/.claude/agents/<name>.md` + `~/.copilot/agents/<name>.agent.md`
|
||||
At plugin/APM scope bump the resolved package's `apm.yml` `version` — **minor** on create, **patch** on improve — because consumers compare it to detect updates. Project and user scope have no manifest.
|
||||
|
||||
A bare `plugin.json` with no `apm.yml` no longer signals plugin scope — that path is fully replaced, not dual-mode; it falls through to project scope.
|
||||
|
||||
The script is file-by-file no-op — it skips any file that already exists.
|
||||
|
||||
### Step 2 — Fill in the agent file(s)
|
||||
|
||||
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Its frontmatter allowlist is the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, read from there as data: `name`, `description`, `model`, `source_keys` (provenance metadata, not a runtime field — see the template), and `disallowedTools` for a read-only agent. Never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill those in plus the system prompt body per the guidance below; the rest of this step's field-by-field guidance is project/user scope only. Skip Step 3 and go to Step 4.
|
||||
|
||||
**At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file.
|
||||
|
||||
Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Remove all template documentation comments from the YAML frontmatter after filling in required fields** — these are marked with `<!--` and `-->` and must be deleted before shipping.
|
||||
|
||||
**`name`** — lowercase letters and hyphens only. Must be unique within the scope.
|
||||
|
||||
**`description`** — the most important field for autonomous delegation:
|
||||
- Start with an action verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
- If this agent should trigger without explicit user direction, include "Use proactively" in the description
|
||||
- Specific about the triggering condition and expertise domain
|
||||
- Under 300 characters preferred
|
||||
|
||||
**`tools`** (project/user scope only — never at plugin/APM scope) — restrict to what the agent actually needs. Omit to inherit all tools. Use `Agent(type1,type2)` to limit which subagent types this agent can spawn; omit `Agent` entirely to prevent spawning.
|
||||
|
||||
**`disallowedTools`** (all scopes, including plugin/APM) — denylist applied before `tools` and taking precedence over it; supports `mcp__<server>`, `mcp__<server>__*`, and `mcp__*` globs. `api-reference.md:40` types it `string / list` and `agent-definition.md:71` types it `string[]`, so a YAML list or a delimited string both work; this repo's plugin-scope agents use the comma-separated string (`disallowedTools: Edit, Write, NotebookEdit`) — match that.
|
||||
|
||||
**Optional fields worth considering (project/user scope only — never at plugin/APM scope, with the exception of `model`, which is allowed at every scope):**
|
||||
- `model`: set when this agent needs a different capability tier (`haiku` for fast tasks, `opus` for deep reasoning)
|
||||
- `maxTurns`: set a cap to prevent runaway agents on bounded tasks
|
||||
- `effort`: set to `low` for single-lookup tasks, `high` or above for deep reasoning or multi-file analysis — overrides session effort level; omit to inherit
|
||||
- `memory`: `user`, `project`, or `local` — only when cross-session state is genuinely needed
|
||||
- `isolation: worktree` — only when the agent modifies files and needs an isolated copy
|
||||
- `skills`: list of skill names preloaded at agent startup — different from the `source_keys` metadata field
|
||||
- `color`: UI color for the agent tile (`red`, `blue`, `green`, `yellow`, `purple`, `orange`, `pink`, `cyan`)
|
||||
- `initialPrompt`: auto-submitted as the first turn when this agent activates as the main session thread; only set when this agent is intended for main-thread activation
|
||||
- `background`: set `true` to force background execution
|
||||
|
||||
**`source_keys`** — top-level list of research source slugs that informed this agent. Add only when research sources were used (i.e. entries with `` `extracted` `` status are in context from a prior `/research` session). Each slug must match an H2 heading in `sources.md` — see Step 4 for where that file lives (plugin/APM scope only). Omit entirely when no research was used.
|
||||
|
||||
```yaml
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
```
|
||||
|
||||
**System prompt body** — write as a direct role instruction:
|
||||
- Open with: "You are a [role]. When invoked, [primary action]."
|
||||
- Cover: inputs expected, process steps, output format, error handling
|
||||
- One job per agent
|
||||
|
||||
### Step 3 — Fill in the Copilot agent file (project/user scope only)
|
||||
|
||||
Skip this step entirely at plugin/APM scope — there is no separate Copilot file there. The single `.apm/agents/<name>.agent.md` file from Step 2 already compiles to both Claude Code and Copilot CLI via `apm compile`.
|
||||
|
||||
**Two distinct Copilot agent formats** exist, with different paths and field sets. Choose one based on the deployment target:
|
||||
|
||||
**CLI format** (default — what the scaffold creates):
|
||||
- Path: `.github/agents/<name>.agent.md` (project) or `~/.copilot/agents/<name>.agent.md` (user)
|
||||
- Extension: **must be `.agent.md`**
|
||||
- Supported fields: `name` (required), `description` (required), `tools` (optional)
|
||||
- `tools` uses Copilot aliases: `execute` (shell), `read`, `edit`, `search`, `agent`, `web`
|
||||
- Body length limit: **30,000 characters** — content beyond this is silently truncated
|
||||
|
||||
**Cloud/IDE format** (use when targeting Copilot Chat in VS Code or GitHub.com):
|
||||
- Path: `.github/copilot/agents/<name>.md` (note: plain `.md`, different directory)
|
||||
- Additional fields available: `target` (`vscode`, `github-copilot`, or omit for both), `user-invocable` (set `false` to hide from manual invocation), `disable-model-invocation` (set `true` to require explicit user invocation), `mcp-servers` (MCP server config — processed by cloud runtime, ignored in VS Code)
|
||||
- Body length limit: **30,000 characters** — silently truncated
|
||||
|
||||
**Do not include Claude Code-only fields in either format**: `maxTurns`, `isolation`, `memory`, `permissionMode`, `effort`, `hooks`, `mcpServers`.
|
||||
|
||||
**`source_keys`** — add the same top-level list as the CC file when research sources were used. Omit when no research was used.
|
||||
|
||||
**Delete the `<!-- -->` template comments from the frontmatter**, as in Step 2.
|
||||
|
||||
The system prompt body should match the Claude Code version — the agent's task definition is the same across providers.
|
||||
|
||||
### Step 4 — Populate or delete `sources.md` (plugin/APM scope only)
|
||||
|
||||
Skip at project/user scope. The file lives at the package root (alongside `apm.yml`), not inside `.apm/agents/` — otherwise tooling that scans that directory for agent definitions would treat it as an agent needing frontmatter (ADR-0010).
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
1. Filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, identify which agent file it contributed to.
|
||||
3. Write `sources.md` at the package root using the format below. Paths in `Contributing files:` are relative to the package root.
|
||||
|
||||
```markdown
|
||||
# Sources
|
||||
|
||||
## slug-name
|
||||
|
||||
- **URL:** <source URL>
|
||||
- **Research doc:** <path/to/research/sources.md relative to repo root>
|
||||
- **Description:** <what this source covers>
|
||||
- **Contributing files:** .apm/agents/<name>.agent.md
|
||||
- **Status:** `extracted`
|
||||
```
|
||||
|
||||
Each slug must match an H2 heading, and each slug must also appear in the `source_keys` list of the file listed under `Contributing files:`.
|
||||
|
||||
If no research sources are in context, delete `sources.md`.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Run this checklist before invoking the audit.
|
||||
|
||||
**Every file, at every scope:**
|
||||
- [ ] `name` present and kebab-case; `description` present
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] No `FILL IN:` placeholders and no `<!-- -->` template comments remain
|
||||
|
||||
**Plugin/APM scope — single file (`<name>.agent.md`):**
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
- [ ] Every frontmatter field is in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the single source of truth, read as data by `validate.sh`. As of 2026-08-14: `name`, `description`, `model`, `source_keys`, `disallowedTools`. Nothing else — in particular no `tools`
|
||||
- [ ] A read-only agent carries `disallowedTools` **and** says so in the body
|
||||
|
||||
**Project/user scope — Claude Code file (`<name>.md`):**
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
|
||||
**Project/user scope — Copilot CLI file (`<name>.agent.md`):**
|
||||
- [ ] File extension is `.agent.md` (not `.md`)
|
||||
- [ ] `name` matches the filename stem (e.g. `name: my-agent` in `my-agent.agent.md`)
|
||||
- [ ] No Claude Code-only fields (`maxTurns`, `isolation`, `memory`, `permissionMode`, `effort`, `hooks`, `mcpServers`)
|
||||
- [ ] Body does not exceed 30,000 characters
|
||||
|
||||
At plugin/APM scope, apply a **minor bump** to the resolved package's `apm.yml` `version` (single manifest, e.g. `1.0.4` → `1.1.0`).
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the created file(s) before closing — validates the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` for the new agent files — do not stop at staging. Then confirm `git log --oneline -1` differs from the hash captured before Step 1. A non-empty `git diff --stat` is not proof of completion: staged-but-uncommitted work is part of no commit and can be silently lost if the working tree is cleaned up. Only report the agent as done once the hash has actually changed.
|
||||
|
||||
## Improving an existing agent
|
||||
|
||||
### Step 1 — Verify inputs
|
||||
|
||||
Confirm the agent files exist and at least one improvement signal is present in the conversation or a referenced file.
|
||||
|
||||
If no signals: "This skill applies existing signals to an agent. For a blind review, examine the files manually or run a grill session first."
|
||||
|
||||
Verify `kyberforge:agent-audit` is available, as in the create flow's Prerequisites. Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
|
||||
**Partial state (project/user scope only)** — if one provider file exists but not the other, scaffold the missing one (`bash scripts/new-agent.sh <name> <root>`, file-by-file no-op) then continue. Doesn't apply at plugin/APM scope — single file, no partial-pair state.
|
||||
|
||||
### Step 2 — Gather and group signals
|
||||
|
||||
Read the current agent file(s). Collect all signals from the conversation.
|
||||
|
||||
Group by **root cause**, not symptom. One root cause → one fix.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- User feedback: agent keeps trying to push to remote
|
||||
- Session context: no scope boundary in system prompt
|
||||
→ Root cause: system prompt lacks git scope constraint → fix: add explicit boundary
|
||||
```
|
||||
|
||||
### Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state which root causes were identified, what evidence supports each, and which files will change. Then proceed — edits are reversible via git.
|
||||
|
||||
### Step 4 — Apply changes
|
||||
|
||||
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced. For a plugin/APM-scope single file, verify every field is still in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, and that an existing `disallowedTools` fence was not dropped by the edit.
|
||||
|
||||
If the edit adds or removes research-sourced content, update `source_keys` in the edited file(s) and the corresponding entry in `sources.md` per Create flow's Step 4.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Re-run the validation checklist from the create flow's Step 5 on any edited file.
|
||||
|
||||
At plugin/APM scope, apply a **patch bump** to the resolved package's `apm.yml` `version` (e.g. `1.0.4` → `1.0.5`).
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the edited file(s) to confirm no regressions — the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Exactly as in the create flow's Step 5, against the hash captured at Step 1: commit the changed files once the audit is clean, and only report the improvement as done once `git log --oneline -1` shows a different hash.
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` — do not stop at staging. Re-run `git log --oneline -1` and confirm the hash changed from Step 1's. A non-empty `git diff --stat` is not proof: staged-but-uncommitted work is part of no commit and is lost if the tree is cleaned up. Report done only once the hash has changed.
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
## templates/
|
||||
|
||||
Annotated agent definition templates copied by `scripts/new-agent.sh` when scaffolding a new agent.
|
||||
All three scaffold the `description` in the three-part ADR-0020 shape — a `Use when` trigger clause, at most one capability clause, and a boundary clause — rather than the deleted action-verb opener, and each carries a delegate-don't-restate note in the body.
|
||||
|
||||
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
|
||||
- **`copilot.agent.md.template`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
|
||||
|
||||
@@ -27,11 +27,17 @@
|
||||
name: AGENT_NAME
|
||||
<!-- Required. Lowercase letters and hyphens only. Must be unique within the scope. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. The primary signal for autonomous delegation.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Be specific about the triggering condition and expertise domain.
|
||||
Example: "Reviews pull request diffs for security issues. Use proactively after code changes." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. The primary signal for autonomous delegation, and preloaded into every
|
||||
session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was
|
||||
deleted. Add "Use proactively" only if the runtime should delegate here without
|
||||
the user naming this agent.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- model: sonnet
|
||||
Optional. Aliases: sonnet, opus, haiku, fable. Or full model ID.
|
||||
@@ -58,6 +64,11 @@ FILL IN: System prompt body. Write as a direct role instruction.
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -7,12 +7,17 @@ name: AGENT_NAME
|
||||
<!-- Required. Lowercase letters and hyphens only. Must be unique within the scope.
|
||||
Duplicate names are silently discarded — no warning is emitted. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. The primary signal for autonomous delegation.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Include "Use proactively" to trigger automatic invocation.
|
||||
Be specific about the triggering condition and domain.
|
||||
Example: "Reviews pull request diffs for security issues. Use proactively after code changes." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. The primary signal for autonomous delegation, and preloaded into every
|
||||
session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was
|
||||
deleted. Add "Use proactively" only if the runtime should delegate here without
|
||||
the user naming this agent.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- tools: Read, Bash, Grep
|
||||
Optional. Allowlist of tool names: a comma-separated string or a YAML list.
|
||||
@@ -78,6 +83,11 @@ FILL IN: System prompt body. Write as a direct role instruction.
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -12,10 +12,16 @@
|
||||
name: AGENT_NAME
|
||||
<!-- Required. Kebab-case identifier. Home-directory version wins on name collision. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. Used by the runtime for automatic agent selection — quality matters.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Example: "Reviews pull request diffs for security issues." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. Used by the runtime for automatic agent selection, and preloaded into
|
||||
every session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was deleted.
|
||||
Keep it identical in wording to the Claude Code half of the pair.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- tools: ["read", "search", "edit"]
|
||||
Optional. Array of tool names. Omit = all available tools. [] = no tools.
|
||||
@@ -46,6 +52,11 @@ FILL IN: System prompt body. Should match the Claude Code version — the agent'
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -4,14 +4,52 @@ source_keys: []
|
||||
|
||||
# references/
|
||||
|
||||
## create.md
|
||||
|
||||
The create flow, loaded from SKILL.md Step 1 when no agent file exists at the target path.
|
||||
Covers: prerequisites, the scaffold script and its scope walk-up, what to fill in at every scope,
|
||||
and populating or deleting the package-root `sources.md`.
|
||||
|
||||
## improve.md
|
||||
|
||||
The improve flow, loaded from SKILL.md Step 1 when a file exists and at least one improvement
|
||||
signal is present. Covers: signal verification, partial-pair recovery, root-cause grouping,
|
||||
generalizing rather than patching, delegation over growth, and the ADR-0020 retrofit rule.
|
||||
|
||||
## contract.md
|
||||
|
||||
The description and body contract, loaded from SKILL.md Step 3 before any description is written
|
||||
or any body restructured. Covers: the three-part description shape, banned description content,
|
||||
boundary-target resolution, the 250/400 length tiers, the body role-instruction pattern, the
|
||||
delegation rule that replaces a body word gate, and the invocation axis.
|
||||
|
||||
## plugin-scope.md
|
||||
|
||||
Field rules and the pre-audit checklist for the single vendor-neutral `.apm/agents/<name>.agent.md`
|
||||
file. Loaded from SKILL.md Step 2 when the scaffold resolves plugin/APM scope.
|
||||
|
||||
## project-user-scope.md
|
||||
|
||||
Field rules and the pre-audit checklist for the Claude Code `.md` + Copilot `.agent.md` pair,
|
||||
including the two distinct Copilot formats. Loaded from SKILL.md Step 2 when the scaffold resolves
|
||||
project or user scope.
|
||||
|
||||
## deployment-modes.md
|
||||
|
||||
Agent scope hierarchy, precedence rules, and per-scope restrictions. Covers: which frontmatter fields a plugin/APM-scope agent may carry and the allowlist-vs-denylist shape rule that decides it (deferring to `agent-audit`'s `references/field-inventory.md` for the list itself), scoped identifiers for plugin subdirectory agents, cache isolation behaviour, and Copilot CLI path conventions. Loaded conditionally from SKILL.md when the destination is a plugin directory.
|
||||
Scope hierarchy and precedence, scoped identifiers for plugin subdirectory agents, cache isolation
|
||||
behaviour, and Copilot CLI path conventions. Loaded from SKILL.md Step 2 when precedence, paths or
|
||||
cache isolation matter to the run.
|
||||
|
||||
## scripts.md
|
||||
|
||||
Conventions for the `new-agent.sh` scaffold script and any future scripts added to this skill. Covers: what scripts should and should not do, file placement, error handling, template variable conventions, and the no-interactive-prompts rule.
|
||||
Conventions for the `new-agent.sh` scaffold script, the templates it copies, and any future script
|
||||
in this skill. Loaded from `create.md` Step 1 when the script or a template has to change. Covers:
|
||||
the no-interactive-prompts rule, structured output, idempotency, template variables, file
|
||||
placement, error messages, and the no-restated-field-roster rule that `tests/new-agent.bats`
|
||||
enforces.
|
||||
|
||||
## sources.md
|
||||
|
||||
Research provenance record for this skill. Lists the upstream research sources (claude-code-plugins and github-copilot-plugins research docs) that informed SKILL.md, the templates, and the deployment-modes reference. Used by `skill-audit` to validate the provenance chain.
|
||||
Research provenance record for this skill. Lists the upstream research sources
|
||||
(claude-code-plugins and github-copilot-plugins research docs) that informed SKILL.md and the
|
||||
reference files. Used by `skill-audit` to validate the provenance chain.
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# The agent description and body contract
|
||||
|
||||
House contract, set by ADR-0020. The counts and the boundary targets are enforced by
|
||||
`agent-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
|
||||
judgment calls by its reference files.
|
||||
|
||||
## Why the budget exists
|
||||
|
||||
An agent's `name` and `description` is loaded into every session's context at startup, whether or
|
||||
not the agent is ever delegated to — the same cost a skill's description carries, so agents take
|
||||
the same numbers. The body is different: it is not loaded into the caller's conversation at all,
|
||||
it *becomes the system prompt of a fresh context* when the agent runs. That is why the body has no
|
||||
word gate here and a skill body has one.
|
||||
|
||||
## Description
|
||||
|
||||
A description carries exactly three things:
|
||||
|
||||
1. **Trigger clause** — when to delegate, imperative: "Use when …", never "This agent …". Describe
|
||||
the user's intent and the triggering condition, not the agent's internal mechanics.
|
||||
2. **At most one capability clause** — what it does, one clause, no enumeration. Be specific
|
||||
("reviews a diff for injected credentials", not "helps with security").
|
||||
3. **Boundary clause** — form: `Not <thing> -> <name>.` Add one only where a near-miss agent or
|
||||
skill could steal delegations.
|
||||
|
||||
Banned from a description; move it to the body or to `README.md`:
|
||||
|
||||
- Capability enumeration or feature lists
|
||||
- Per-scope emission mechanics — which files the author skill writes at which scope changes no
|
||||
delegation decision
|
||||
- Output-format detail ("Produces a compact findings report with Why and Fix per finding")
|
||||
- Composition or architecture notes ("composes X rather than duplicating Y", "cross-cutting")
|
||||
- Implementation detail ("Self-validates via a bundled deterministic script")
|
||||
- Restating the same trigger twice in two registers — a verb list, then the same verbs re-quoted
|
||||
as user phrasings. This is a FAIL, not a suggestion.
|
||||
|
||||
**Do not open with an action verb.** "Reviews…", "Analyzes…", "Generates…" was the old house rule
|
||||
and ADR-0020 deleted it: the opener is `Use when`, matching every skill in this corpus, so one
|
||||
router reads one shape.
|
||||
|
||||
**"Use proactively" is conditional.** Add it only where the runtime should delegate without the
|
||||
user naming the agent — an agent invoked by name does not need it, and it costs activations
|
||||
elsewhere when added by reflex. The same conditional governs indirect triggers ("even if the user
|
||||
doesn't say X"): add one only where the user's natural phrasing genuinely omits the domain word.
|
||||
|
||||
**Boundary targets must resolve.** The name after the arrow is checked against real skills under
|
||||
`plugins/*/.apm/skills/<name>/` and real agents under `plugins/*/.apm/agents/<name>.agent.md`. A
|
||||
target that does not exist sends the router nowhere. Verify it before writing it — do not invent a
|
||||
plausible sibling.
|
||||
|
||||
**Length.** 250 characters SUGGESTION, 400 characters FAIL, counting the frontmatter value only
|
||||
with YAML folding resolved. Treat 250 as the target: the SUGGESTION tier is what moves the corpus
|
||||
average, the FAIL tier only stops outliers.
|
||||
|
||||
## Body
|
||||
|
||||
Write the body as a direct role instruction, addressed to the agent:
|
||||
|
||||
````markdown
|
||||
You are a <role>. When invoked, <primary action>.
|
||||
|
||||
## Inputs
|
||||
<what the agent is given: files, context, parameters>
|
||||
|
||||
## Process
|
||||
<ordered steps; be explicit where ordering matters>
|
||||
|
||||
## Output
|
||||
<what it produces: format, location, structure>
|
||||
````
|
||||
|
||||
One job per agent. An agent covering two jobs gets delegated to for the wrong one.
|
||||
|
||||
**Delegation discipline replaces the word gate.** A plugin/APM agent is a single file with no
|
||||
sibling `references/` directory: it cannot disclose progressively to itself, so its only way to
|
||||
stay short is to *invoke* rather than *restate*. A body that transcribes a procedure a skill it
|
||||
can invoke already owns is an `agent-audit` FAIL, and the fix is one line — "invoke `<skill>`".
|
||||
|
||||
- Restating: "To commit, check the message against Conventional Commits: type, scope,
|
||||
description; header under 100 chars; …"
|
||||
- Delegating: "Author commits with `git-commits`."
|
||||
|
||||
The same holds for a procedure another agent owns. What belongs in the body is what no invocable
|
||||
skill covers: the agent's role, its boundaries, the order it works in, and the format it returns.
|
||||
|
||||
**State a read-only boundary in prose, not only in frontmatter.** `disallowedTools` denies the
|
||||
tools it names and nothing else — never `Bash`, which an agent with no `tools` field inherits — so
|
||||
an agent fenced only in frontmatter can still write through a shell redirect.
|
||||
|
||||
## Invocation axis
|
||||
|
||||
Decide before writing the description whether the agent is model-delegated (the runtime picks it)
|
||||
or reached only by name (`@agent-<name>`).
|
||||
|
||||
Only Copilot's cloud/IDE format expresses that in frontmatter: `disable-model-invocation: true`
|
||||
requires explicit invocation, and `user-invocable: false` hides an agent from manual invocation.
|
||||
Both live in `.github/copilot/agents/<name>.md` and are inert in the CLI format. Claude Code has
|
||||
no equivalent field, and neither does the vendor-neutral plugin/APM file, so at those scopes a
|
||||
name-invoked agent still needs a description precise enough not to steal delegations — the
|
||||
boundary clause is doing that work.
|
||||
|
||||
## One gate, two measurements
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| description | 250 chars | 400 chars | the `description:` value only |
|
||||
| body (Copilot limit) | 30,000 chars | — | the body only; content past it is truncated silently |
|
||||
|
||||
The 30,000-character Copilot ceiling is a runtime truncation limit, not a quality target, and it
|
||||
applies to a plugin/APM file too — that file compiles into a real Copilot agent downstream. An
|
||||
agent body long enough to approach it has a delegation defect, not a length problem.
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- github-plugins-creating
|
||||
---
|
||||
|
||||
# Creating a new agent
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 3 below is done — validation, the version bump and commit
|
||||
verification are shared with the improve flow and are not repeated here.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before touching the filesystem, confirm you have:
|
||||
|
||||
- [ ] Agent name (kebab-case, e.g. `code-reviewer`)
|
||||
- [ ] Root directory — a path inside a package for plugin/APM scope, a project root, or `~` for
|
||||
user scope
|
||||
- [ ] Agent purpose — one sentence describing the task this agent handles
|
||||
- [ ] Trigger condition — when should the runtime delegate to this agent?
|
||||
|
||||
If any are missing, stop and ask before proceeding.
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
|
||||
is co-installed with this skill; if it is unavailable, stop and ask the user to install
|
||||
kyberforge before continuing.
|
||||
|
||||
Design for one job per agent. An agent covering two jobs is delegated to for the wrong one.
|
||||
|
||||
## Step 1 — Scaffold
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh <name> <root>
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
|
||||
bash scripts/new-agent.sh deploy-assistant .
|
||||
bash scripts/new-agent.sh security-reviewer ~
|
||||
```
|
||||
|
||||
The script resolves scope itself and prints which one it used and every path it wrote — read that
|
||||
output rather than predicting it. It walks up from `<root>` for the nearest ancestor `apm.yml`
|
||||
carrying a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`), which marks a
|
||||
package root and means plugin/APM scope. An `apm.yml` with no `type:` is a marketplace-only
|
||||
manifest: the walk skips it and keeps going. With no such manifest found, `<root>` being exactly
|
||||
`~` (checked directly, no walk-up) is user scope and anything else is project scope. A bare
|
||||
`plugin.json` no longer signals plugin scope — that path was replaced outright, not made
|
||||
dual-mode, and falls through to project scope.
|
||||
|
||||
The script is file-by-file no-op: it skips any file that already exists, so re-running it to
|
||||
complete a partial pair is safe.
|
||||
|
||||
If the script or a template under `assets/templates/` has to change to support this agent — a new
|
||||
scope, a new scaffolded field, different output — read `references/scripts.md` first. Its
|
||||
conventions are asserted by `tests/new-agent.bats`, and an edit that ignores them fails the suite.
|
||||
|
||||
## Step 2 — Fill in the file(s)
|
||||
|
||||
Take the scope the script reported and read the matching reference — `SKILL.md` Step 2 has the
|
||||
table. That file carries the field rules and the pre-audit checklist for this scope; the other one
|
||||
describes fields this run cannot use.
|
||||
|
||||
Every scaffolded file, at every scope:
|
||||
|
||||
1. Replace each `FILL IN:` placeholder.
|
||||
2. Delete every `<!-- ... -->` template comment from the frontmatter. `apm compile` copies plugin
|
||||
frontmatter verbatim and HTML comments are not valid YAML, so a leftover comment breaks the
|
||||
file downstream on both harnesses.
|
||||
3. Write the `description` against `references/contract.md` and the system prompt body against its
|
||||
Body section.
|
||||
|
||||
## Step 3 — Populate or delete `sources.md`
|
||||
|
||||
Plugin/APM scope only — skip at project and user scope, which have no package root to hold the
|
||||
file.
|
||||
|
||||
The scaffold writes a commented `sources.md` skeleton at the package root, alongside `apm.yml` and
|
||||
not inside `.apm/agents/`, so that tooling scanning that directory for agent definitions does not
|
||||
treat it as an agent missing its frontmatter (ADR-0010).
|
||||
|
||||
If a research `sources.md` is present in the conversation context, filter it to entries with
|
||||
`` `extracted` `` status, work out which agent file each one contributed to, and fill in the
|
||||
skeleton following the commented format already in the file. Paths in `Contributing files:` are
|
||||
relative to the package root. Each slug must match an H2 heading and must also appear in the
|
||||
`source_keys` list of every file named under its `Contributing files:`.
|
||||
|
||||
If no research sources are in context, delete `sources.md`.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -22,13 +22,14 @@ Agent definitions deploy at three scopes and behave differently at each. The sco
|
||||
|
||||
When the same agent `name` appears at multiple scopes, **user scope wins over project scope wins over plugin scope** in Claude Code. In Copilot CLI, repo-level agents override enterprise and org-level; home-directory (user) agents override repo-level on name collision.
|
||||
|
||||
## Plugin scope restrictions
|
||||
## Which fields exist where
|
||||
|
||||
Plugin/APM agents (`.apm/agents/<name>.agent.md`) carry only the fields in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`. That section is the authoritative list — `agent-audit`'s `validate.sh` reads it from there as data, and it changes — so consult it rather than any restatement of it. `apm compile` copies this frontmatter verbatim to both the Claude Code and Copilot CLI compile targets with no per-target integrator, so a harness-specific value is guaranteed wrong on at least one target (ADR-0016).
|
||||
|
||||
**The rule is about a field's shape, not a fixed roster.** `tools` is an **allowlist** whose vocabulary differs per harness — Claude Code names its own tools, Copilot CLI uses aliases (`execute`/`read`/`edit`/`search`/`agent`/`web`) — so under verbatim copy one value is wrong on one target. It stays out. `disallowedTools` is a **denylist**, and denying by name has no such conflict: a name the other harness does not recognise denies nothing, so the worst case is that the fence is absent there, never that a capability is wrongly granted. That asymmetry is why the denylist is admitted where the allowlist is not (ADR-0016's 2026-08-14 amendment). Claude Code honours it for plugin subagents — `docs/research/docs/claude-code-plugins/agent-definition.md:99` names the three fields plugin agents silently ignore (`hooks`, `mcpServers`, `permissionMode`) and `disallowedTools` is not among them. It is a partial fence: it denies only the tools it names, not `Bash`, which a plugin-scope agent with no `tools` inherits — so state read-only intent in the body too.
|
||||
|
||||
This makes the old "silently ignored at plugin scope" framing moot for the excluded fields. It's not that `hooks`, `mcpServers`, `permissionMode`, `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `skills`, `color`, `initialPrompt`, or `background` are merely ignored at this scope — they are never written to the file at all. Copy the agent to `.claude/agents/` (project scope) or `~/.claude/agents/` (user scope) to use any of them.
|
||||
Field rules are per scope and live with the scope: `references/plugin-scope.md` for the single
|
||||
vendor-neutral file, `references/project-user-scope.md` for the Claude Code / Copilot pair. Read
|
||||
one, not both. The short version is that plugin/APM frontmatter is an allowlist read from
|
||||
`agent-audit`'s `references/field-inventory.md`, narrow because `apm compile` copies frontmatter
|
||||
verbatim to every target (ADR-0016), while project and user scope carry the full per-provider
|
||||
field sets.
|
||||
|
||||
## Scoped identifiers (Claude Code plugin agents only)
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
---
|
||||
|
||||
# Improving an existing agent
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 4 below is done — validation, the version bump and commit
|
||||
verification are shared with the create flow and are not repeated here.
|
||||
|
||||
## Step 1 — Verify inputs
|
||||
|
||||
Confirm the agent file (or, at project and user scope, the pair) exists and that at least one
|
||||
improvement signal is present in the conversation or in a referenced file.
|
||||
|
||||
If no signals are present, stop: "This skill applies existing signals to an agent. For a blind
|
||||
review, run `agent-audit` instead."
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
|
||||
it is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
|
||||
|
||||
**Partial pair — project and user scope only.** If one provider file exists and the other does
|
||||
not, scaffold the missing one with `bash scripts/new-agent.sh <name> <root>` (file-by-file no-op)
|
||||
and continue. Plugin/APM scope is a single file and has no partial state.
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current file(s), then collect every signal from the conversation and from any path the
|
||||
user referenced.
|
||||
|
||||
Group signals by **root cause**, not by symptom. Patching per symptom is the default failure mode:
|
||||
three complaints often trace to one missing instruction. Ask: "What single gap in this agent
|
||||
causes this cluster?" One root cause, one fix.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- User feedback: the agent keeps trying to push to the remote
|
||||
- Session context: no scope boundary in the system prompt
|
||||
→ Root cause: the system prompt has no git scope constraint → fix: add an explicit boundary
|
||||
```
|
||||
|
||||
## Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state which root causes were identified, what evidence supports each, and which
|
||||
files will change. Then proceed — edits are reversible via git, so no approval checkpoint is
|
||||
needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit whichever file the signals point to.
|
||||
|
||||
**Generalize, do not patch.** Fix the underlying gap, not the one example that failed. A fix
|
||||
scoped to the cases you have seen overfits and performs worse on new input.
|
||||
|
||||
**Delegate rather than grow.** An agent body has no word ceiling, but a body that restates a
|
||||
procedure a skill it can invoke already owns is an `agent-audit` FAIL. When a signal reports a
|
||||
missing procedure, check first whether an installed skill owns it and name that skill instead of
|
||||
transcribing it. See `references/contract.md`.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. A rule written in
|
||||
all caps (ALWAYS/NEVER) is usually better reframed as why the behaviour matters, so the agent can
|
||||
apply judgment at the edges.
|
||||
|
||||
**Retrofit before extending.** Any agent predating ADR-0020 has to meet the description contract
|
||||
before any other edit lands — the gates are hot and carry no baseline file, so a one-line fix to a
|
||||
non-compliant agent cannot be committed until its description meets `references/contract.md`.
|
||||
Treat that retrofit as part of the same change, not a follow-up.
|
||||
|
||||
**Re-check the scope rules.** Read the reference for the resolved scope (`SKILL.md` Step 2) and
|
||||
confirm the edit introduced no field that scope forbids, and dropped no `disallowedTools` fence
|
||||
that was already there.
|
||||
|
||||
If the edit adds or removes research-sourced content, update `source_keys` in the edited file and
|
||||
the matching `sources.md` entry — the create flow's Step 3 has the rules.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
---
|
||||
|
||||
# Plugin/APM scope — the single vendor-neutral file
|
||||
|
||||
One file, no counterpart: `<package-root>/.apm/agents/<name>.agent.md`. `apm compile` emits it to
|
||||
both the Claude Code and the Copilot CLI target. The `.agent.md` extension here is convention, not
|
||||
a Copilot marker — the file is vendor-neutral.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `agent-audit`'s
|
||||
`references/field-inventory.md`. Read them from there as data — that section is the single source
|
||||
of truth, `agent-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
|
||||
the roster, here or in a template or in script output, goes stale one step further out than the
|
||||
list itself.
|
||||
|
||||
- `name` — kebab-case, must equal the filename stem, unique within the scope.
|
||||
- `description` — write it against `references/contract.md`.
|
||||
- Everything else — check the allowlist section before adding a key. A key outside it fails the
|
||||
audit.
|
||||
|
||||
**Why the list is narrow.** `apm compile` copies frontmatter verbatim to every target with no
|
||||
per-target integrator, so a harness-specific value is wrong on at least one of them (ADR-0016).
|
||||
The rule is about a field's *shape*, not a fixed roster:
|
||||
|
||||
- `tools` is an **allowlist** whose vocabulary differs per harness — Claude Code names its own
|
||||
tools, Copilot CLI uses aliases (`execute`/`read`/`edit`/`search`/`agent`/`web`) — so one value
|
||||
is wrong on one target. It stays out. Omitting it means inherit-all-tools on both, which is
|
||||
never wrong.
|
||||
- `disallowedTools` is a **denylist**, and denying by name cannot fail that way: a name the other
|
||||
harness does not recognise denies nothing, so the worst case is a missing fence, never a wrongly
|
||||
granted capability. That asymmetry is the whole exception (ADR-0016's 2026-08-14 amendment).
|
||||
Claude Code honours it for plugin subagents; the three fields plugin agents do silently ignore
|
||||
are `hooks`, `mcpServers` and `permissionMode`, and this is not one of them. Copilot's handling
|
||||
of the key is unconfirmed, which ADR-0016 accepts as a stated risk.
|
||||
- The Claude-only knobs (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `skills`,
|
||||
`color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) have no Copilot equivalent and
|
||||
are never written to this file at all. "Silently ignored at plugin scope" is the wrong framing:
|
||||
they are absent, not tolerated. To use any of them, copy the agent to `.claude/agents/`
|
||||
(project scope) or `~/.claude/agents/` (user scope).
|
||||
|
||||
Write `disallowedTools` on every read-only plugin-scope agent — and say the agent is read-only in
|
||||
the body as well, because the fence does not cover the inherited `Bash` tool.
|
||||
|
||||
`source_keys` is provenance metadata, not a runtime field: both harnesses ignore it. Add it only
|
||||
when research sources informed the agent, with slugs matching H2 headings in the package root's
|
||||
`sources.md`.
|
||||
|
||||
## Body
|
||||
|
||||
Follow the Body section of `references/contract.md`: role instruction, one job, and delegation
|
||||
to installed skills instead of transcribed procedure.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
|
||||
- [ ] `name` kebab-case, matching the filename stem, unique in scope
|
||||
- [ ] `description` written to `references/contract.md`
|
||||
- [ ] Every frontmatter key present in the `apm-agent-allowlist` section — in particular no `tools`
|
||||
- [ ] No `FILL IN:` placeholder and no `<!-- ... -->` template comment anywhere in the file
|
||||
- [ ] System prompt body non-empty, and a read-only agent says so in prose as well as in
|
||||
`disallowedTools`
|
||||
|
||||
Then return to the flow reference you came from.
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
- github-cli-plugin-reference
|
||||
---
|
||||
|
||||
# Project and user scope — the Claude Code / Copilot pair
|
||||
|
||||
Two files per agent, written in one pass and kept in step: a Claude Code `.md` and a Copilot CLI
|
||||
`.agent.md`. The system prompt body is the same in both — the agent's task does not change with
|
||||
the provider. The frontmatter is not.
|
||||
|
||||
| Scope | Claude Code | Copilot CLI |
|
||||
|---|---|---|
|
||||
| Project | `.claude/agents/<name>.md` | `.github/agents/<name>.agent.md` |
|
||||
| User | `~/.claude/agents/<name>.md` | `~/.copilot/agents/<name>.agent.md` |
|
||||
|
||||
## Claude Code file
|
||||
|
||||
**`name`** — lowercase letters and hyphens only, unique within the scope. Claude Code discards a
|
||||
duplicate silently.
|
||||
|
||||
**`description`** — write it against `references/contract.md`. It is the primary signal for
|
||||
autonomous delegation.
|
||||
|
||||
**`tools`** — an allowlist; omit it to inherit every tool from the parent. Use `Agent(type1,type2)`
|
||||
to restrict which subagent types this agent may spawn, and omit `Agent` entirely to stop it
|
||||
spawning any. Five tools reach no subagent whatever this field says — `AskUserQuestion`,
|
||||
`EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup` and `WaitForMcpServers` — so listing one buys
|
||||
nothing. The single exception is `ExitPlanMode`, available when the parent session runs
|
||||
`permissionMode: plan`.
|
||||
|
||||
**`disallowedTools`** — a denylist, applied before `tools` and taking precedence over it. Supports
|
||||
`mcp__<server>`, `mcp__<server>__*` and `mcp__*` globs. Both a YAML list and a delimited string
|
||||
are accepted; this repo writes the comma-separated string form (`disallowedTools: Edit, Write,
|
||||
NotebookEdit`) — match it.
|
||||
|
||||
**`model`** — set it when the agent needs a different capability tier (`haiku` for fast lookups,
|
||||
`opus` for deep reasoning). Resolution order is `CLAUDE_CODE_SUBAGENT_MODEL` → the per-invocation
|
||||
parameter → this field → the main session model, so the frontmatter value is a low-priority
|
||||
default rather than a guarantee.
|
||||
|
||||
Optional fields worth considering, none of which exist at plugin/APM scope:
|
||||
|
||||
- `maxTurns` — cap agentic turns on a bounded task, to stop a runaway
|
||||
- `effort` — `low` for a single lookup, `high` or above for multi-file analysis; omit to inherit
|
||||
- `memory` — `user`, `project` or `local`; only when cross-session state is genuinely needed
|
||||
- `isolation: worktree` — only when the agent modifies files and needs an isolated copy
|
||||
- `skills` — skill names preloaded at agent startup; unrelated to the `source_keys` metadata field
|
||||
- `color` — the UI tile colour (`red`, `blue`, `green`, `yellow`, `purple`, `orange`, `pink`,
|
||||
`cyan`)
|
||||
- `background` — `true` forces background execution
|
||||
- `initialPrompt` — auto-submitted as the first turn when the agent activates as the main session
|
||||
thread; set it only for a main-thread agent, never for a subagent
|
||||
|
||||
`hooks`, `mcpServers` and `permissionMode` are honoured at these two scopes and nowhere else — a
|
||||
plugin agent carrying them is ignored silently.
|
||||
|
||||
A subdirectory under `agents/` does not affect the agent's name at these scopes; it does at plugin
|
||||
scope, which is one reason `references/deployment-modes.md` recommends keeping agents flat.
|
||||
|
||||
## Copilot file
|
||||
|
||||
Two Copilot formats exist, with different paths and different field sets. Pick one:
|
||||
|
||||
**CLI format** — what the scaffold writes.
|
||||
|
||||
- Path: `.github/agents/<name>.agent.md` (project) or `~/.copilot/agents/<name>.agent.md` (user)
|
||||
- The `.agent.md` extension is mandatory: Copilot CLI does not pick up a plain `.md` file in
|
||||
`agents/`, and fails silently rather than reporting it
|
||||
- Fields: `name` (required, must equal the filename stem), `description` (required), `tools`
|
||||
(optional)
|
||||
- `tools` uses Copilot aliases, not Claude tool names: `execute` (shell), `read`, `edit`,
|
||||
`search`, `agent`, `web`; MCP tools as `server-name/tool-name` or `server-name/*`
|
||||
|
||||
**Cloud/IDE format** — for Copilot Chat in VS Code or on GitHub.com.
|
||||
|
||||
- Path: `.github/copilot/agents/<name>.md` — a plain `.md`, in a different directory
|
||||
- Adds `target` (`vscode`, `github-copilot`, or omit for both), `user-invocable`,
|
||||
`disable-model-invocation` and `mcp-servers` (processed by the cloud runtime, ignored in VS
|
||||
Code). These four are inert in the CLI format — do not write them there
|
||||
- This is the only format that can express the invocation axis in frontmatter; see the Invocation
|
||||
axis section of `references/contract.md`
|
||||
|
||||
Both formats truncate a body past **30,000 characters** silently.
|
||||
|
||||
Copilot has no `permissionMode`, `maxTurns`, `isolation`, `memory`, `effort`, `hooks` or
|
||||
`mcpServers`. Never let those cross over from the Claude Code file.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
|
||||
Both files:
|
||||
|
||||
- [ ] `name` present and kebab-case; `description` written to `references/contract.md`
|
||||
- [ ] System prompt body present, non-empty and equivalent across the pair
|
||||
- [ ] No `FILL IN:` placeholder and no `<!-- ... -->` template comment left
|
||||
|
||||
Copilot file only:
|
||||
|
||||
- [ ] Extension is `.agent.md` (CLI format), and `name` matches the filename stem
|
||||
- [ ] No Claude Code-only field present
|
||||
- [ ] Body under 30,000 characters
|
||||
|
||||
Then return to the flow reference you came from.
|
||||
@@ -19,7 +19,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/deployment-modes.md, references/plugin-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
@@ -27,7 +27,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md, references/plugin-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
@@ -35,7 +35,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md, references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
@@ -43,7 +43,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
@@ -51,7 +51,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/contract.md, references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
@@ -59,7 +59,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-creating
|
||||
@@ -67,7 +67,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-finding-installing
|
||||
|
||||
@@ -157,9 +157,11 @@ find_package_root() {
|
||||
done
|
||||
}
|
||||
|
||||
# `read` consumes a single line, so kind and path are emitted on one
|
||||
# space-separated line rather than two `echo`s — kind first (never contains
|
||||
# spaces), path last (absorbs any spaces in the path safely).
|
||||
# kind and path are emitted on one space-separated line rather than two
|
||||
# `echo`s — kind first (never contains spaces), path last (absorbs any spaces
|
||||
# in the path safely). `mapfile`/`readarray` would need bash 4.0+, which
|
||||
# macOS's stock /bin/bash 3.2 is not; a here-string `read` splits the single
|
||||
# line without it. Same form as skill-author's new-skill.sh, deliberately.
|
||||
WALK_RESULT="$(find_package_root "$ROOT")"
|
||||
read -r WALK_KIND WALK_ROOT <<< "$WALK_RESULT"
|
||||
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
# skill-audit
|
||||
|
||||
Audit a skill directory against the agentskills.io specification. Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags known-bad description openers, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across seven dimensions
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to /skill-improve
|
||||
3. Applies qualitative checks across six dimension groups, loading one rubric from `references/` per group
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to `skill-author`
|
||||
|
||||
`validate.sh` enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words, plus resolvable boundary targets).
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -26,12 +28,16 @@ Provide the path to the skill directory to audit when invoking.
|
||||
| `scripts/validate-provenance.sh` | Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, and upstream research doc alignment |
|
||||
| `scripts/vale-wrap.sh` | Vale prefilter wrapper — runs the bundled `Kyberforge` Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
|
||||
| `assets/vale/.vale.ini` | Vale configuration — points Vale at the bundled `Kyberforge` style path, self-located relative to `vale-wrap.sh` |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags literal "This skill..."/"This agent..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags non-imperative "This..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Vale rule — flags generic "see references/" padding phrasing in conditional references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Vale rule — flags body sentences starting with "There is"/"There are" |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
|
||||
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
|
||||
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked (`disable-model-invocation`) contract, and the internal-mechanics FAIL |
|
||||
| `references/body-discipline.md` | Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
|
||||
| `references/patterns.md` | Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
|
||||
| `references/file-structure.md` | Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
|
||||
| `references/formatting-and-scripts.md` | Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
|
||||
| `references/sources.md` | Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
|
||||
| `tests/validate.bats` | (source-only) Bats test suite for validate.sh |
|
||||
| `tests/validate-provenance.bats` | (source-only) Bats test suite for validate-provenance.sh |
|
||||
|
||||
@@ -1,19 +1,10 @@
|
||||
---
|
||||
name: skill-audit
|
||||
description: >
|
||||
Use when the user wants to review a skill they wrote, says "audit this skill",
|
||||
"check if my skill follows best practices", "review my SKILL.md", or wants to
|
||||
know if a skill is ready to ship — even if they don't use the word "audit".
|
||||
Also invoke proactively after directly hand-editing a skill's files outside
|
||||
skill-author — an unaudited hand-edit is the same risk as unreviewed code.
|
||||
Audits a skill directory against the agentskills.io specification — structural
|
||||
checks plus qualitative review of description quality, body discipline, patterns,
|
||||
formatting, file structure, scripts, and internal consistency, plus a provenance
|
||||
chain check. Produces a compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
|
||||
handoff to /skill-improve or human auditability. Do not use to fix application
|
||||
code bugs or perform general code review unrelated to skill quality.
|
||||
Do not use when the user wants improvements applied — use /skill-improve instead.
|
||||
Use when the user wants a skill directory audited against the agentskills.io
|
||||
spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or
|
||||
after hand-editing a skill outside skill-author. Not applying fixes ->
|
||||
skill-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -27,9 +18,14 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not output PASS/FAIL per check while auditing — gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Structural validation
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
@@ -37,98 +33,47 @@ bash scripts/validate-provenance.sh <skill-dir>
|
||||
scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines and ≤2770 words (the word count is a proxy for the ~5,000-token ceiling, and blocks a commit exactly like the line count does), no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both. If it cannot run at all (no `python3`, Bash denied), report that as an INFO finding rather than guessing; what it measures is not reproducible by reading.
|
||||
|
||||
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
|
||||
`validate-provenance.sh` prints nothing on success. Its FAIL and INFO findings become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim.
|
||||
|
||||
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". It applies that config's `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns/Body dimensions below, not a replacement for Step 3. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one citing its rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 3 judgment for the dimensions it would have covered.
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture.
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md`, `README.md`, and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Work through each dimension internally. Collect findings only; report them in Step 4. Cite file and line number for every finding.
|
||||
Load a dimension's rubric before judging that dimension. Each is self-contained, and each is grounded in the agentskills.io specification plus the house context-budget contract (ADR-0020).
|
||||
|
||||
### Description
|
||||
| Dimension | Read |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body-discipline | `references/body-discipline.md` |
|
||||
| patterns | `references/patterns.md` |
|
||||
| file-structure, internal-consistency | `references/file-structure.md` |
|
||||
| formatting, scripts | `references/formatting-and-scripts.md` |
|
||||
|
||||
Vale's `Kyberforge.DescriptionOpener` ("This skill..." openers) and `Kyberforge.VagueWording` (filler like "helps with", "utilize") alerts from Step 1 — both FAILs — cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
|
||||
|
||||
- **Action-verb opening**: does the description start with a verb ("Audits...", "Reviews...", "Validates...")? Vale's `Kyberforge.DescriptionOpener` alert only catches the literal "This skill..." pattern — confirming an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
|
||||
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
|
||||
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
|
||||
- **Length**: under 1024 characters?
|
||||
|
||||
If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`.
|
||||
|
||||
### Body discipline
|
||||
|
||||
For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding.
|
||||
|
||||
- **Defaults not menus**: every decision point gives one default + one escape hatch, not a list of options
|
||||
- **Why rationale**: include/exclude rules explain why, not just what
|
||||
- **Control calibration**: prescriptive for fragile or critical sequences (e.g. a script invocation where flag order or exact arguments must not change); flexible where multiple approaches are valid
|
||||
|
||||
Vale's `Kyberforge.SentenceOpenerThereIs` alert from Step 1 (FAIL — sentences starting with "There is"/"There are") covers pattern-matchable body-wide filler directly; report it as a finding without re-deriving by judgment.
|
||||
|
||||
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
|
||||
|
||||
### Patterns
|
||||
|
||||
Check each pattern is appropriate and correctly formed:
|
||||
|
||||
- **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip
|
||||
- **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block
|
||||
- **Checklists**: used for multi-step workflows, not single steps
|
||||
- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment.
|
||||
- **Output templates**: present when the agent must produce a specific format; absent otherwise
|
||||
|
||||
### File structure
|
||||
|
||||
- Permitted directories: `scripts/`, `references/`, `assets/`, `tests/`; flag any other unlisted directory as FAIL — the spec allows additional dirs but this skill permits only these four to keep skills focused
|
||||
- `scripts/` contains only executable code agents can run; test files (`.bats`, `*_test.*`, `test_*.sh`) in `scripts/` are a FAIL — they belong in `tests/`
|
||||
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
|
||||
- Optional directories contain real content — not just unfilled placeholder READMEs
|
||||
- `README.md` present and accurately describes the skill and its files
|
||||
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`, or its APM-native equivalent `.apm/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
|
||||
- `references/sources.md` is exempt from the cross-plugin path check — `Research doc:` fields are development-only provenance pointers, not runtime references; they intentionally reference paths outside the skill directory and are expected to be non-resolvable after plugin install; `validate-provenance.sh` handles this gracefully by silently skipping upstream checks when those paths don't resolve
|
||||
- `tests/` is exempt from the cross-plugin path check — test files are dev-only and may reference repo-level test infrastructure (e.g. a shared `tests/test_helper/`). This dependency must be declared in `tests/README.md`; flag if tests exist but `tests/README.md` is absent or does not document the dependency
|
||||
|
||||
### Formatting
|
||||
|
||||
- Heading levels consistent: H2 for main sections, H3 for subsections
|
||||
- Code blocks fenced with a language tag where applicable (`bash`, `markdown`, `python`)
|
||||
- Consistent whitespace: blank line between sections, consistent list indentation
|
||||
- No broken relative paths in file references
|
||||
|
||||
### Scripts
|
||||
|
||||
- No interactive TTY prompts (`read`, `input()`, `readline`)
|
||||
- `--help` exposed with concise usage
|
||||
- Data to stdout, diagnostics to stderr
|
||||
- Idempotent ("create if not exists")
|
||||
- Meaningful exit codes documented in `--help`
|
||||
- `--dry-run` present for destructive operations
|
||||
|
||||
### Internal consistency
|
||||
|
||||
- SKILL.md steps match what scripts actually do
|
||||
- `README.md` file table lists every file that exists — no missing entries, no stale entries
|
||||
- Placeholder READMEs in `scripts/`, `references/`, `assets/` consistent with what SKILL.md says about each directory
|
||||
Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with a coverage line listing every dimension checked:
|
||||
Open with a coverage line naming every dimension checked:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely — their absence confirms they passed.
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
For each finding:
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
@@ -136,18 +81,4 @@ FAIL/SUGGESTION <finding> — file:line
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a result block:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS (N suggestions)
|
||||
PASS · P info
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /skill-improve to address findings.
|
||||
```
|
||||
|
||||
INFO findings are observational — do not affect PASS/FAIL. Omit `· P info` when there are no INFO findings. Omit the `/skill-improve` line when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -4,4 +4,4 @@ level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This (skill|agent)\b'
|
||||
- '^This\b'
|
||||
|
||||
@@ -6,31 +6,114 @@ source_keys:
|
||||
|
||||
# Body Discipline Reference
|
||||
|
||||
Source: agentskills.io — skill-authoring
|
||||
Upstream source: agentskills.io — skill-authoring, best-practices.
|
||||
House contract: ADR-0020, the context budget.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and dilutes the signal of what matters.
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## What belongs in the body
|
||||
## What the body is for
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when.
|
||||
|
||||
Include content the agent lacks:
|
||||
|
||||
- Project-specific conventions and domain procedures it cannot infer
|
||||
- Non-obvious edge cases and environment-specific gotchas
|
||||
- The specific tools or sequences to use (not the full range of options)
|
||||
- The specific tools or sequences to use — not the full range of options
|
||||
- One default per decision point with one escape hatch
|
||||
|
||||
Do not include:
|
||||
Move to `references/`, behind an explicit "If X, read `references/file.md`" trigger — the literal
|
||||
conditional form, never a generic pointer:
|
||||
|
||||
- Lookup tables and spec restatements
|
||||
- Output schemas, templates and example blocks
|
||||
- Rationale and justification prose
|
||||
- Anything only one branch of the procedure ever reaches
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description — it's already in context
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Two length families, measured differently
|
||||
|
||||
Do not conflate these, and do not report them as one finding.
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| Body budget (house, ADR-0020) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Spec conformance (agentskills.io) | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 2,770-word ceiling is a token-conformance backstop calibrated to the densest prose in the
|
||||
corpus; it says nothing about quality and a file can sit a thousand words inside it while failing
|
||||
the body budget. The 900-word ceiling is the quality gate: a body is loaded into the caller's live
|
||||
context and competes with the conversation already there. `validate.sh` reports both. Cite whichever
|
||||
one actually fired.
|
||||
|
||||
A word count cannot detect the defect it stands in for. Treat both numbers as backstops to the
|
||||
dispatch rule and the Gotchas constraint below, never as a substitute for them.
|
||||
|
||||
## Dispatch is mandatory at two or more mutually exclusive flows
|
||||
|
||||
If a skill handles two or more flows that a single invocation cannot both take — separate
|
||||
subcommands, separate input types, separate lifecycle stages — the body carries a **dispatch
|
||||
table** plus the gates common to every branch, and each flow lives in its own self-contained
|
||||
`references/` file. Inlining all of them is a FAIL regardless of word count, because every
|
||||
invocation then pays for every branch it did not take.
|
||||
|
||||
The reference shape in this repo is `apm-workflow`: a 554-word body dispatching to roughly 3,000
|
||||
words of references across five mutually exclusive invocations.
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
The highest-value construct in a body, and the easiest to fill with noise. A Gotcha must state a
|
||||
fact that **contradicts a reasonable default** — something the agent gets wrong precisely by acting
|
||||
sensibly.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- **Maximum five entries.** Past five, the section is a summary of the body rather than a set of
|
||||
traps, and the agent stops reading it as a warning.
|
||||
- **A Gotcha that paraphrases a step in the body below it is a FAIL.** It has no independent
|
||||
content, and it teaches the agent that Gotchas can be skimmed because the real instruction is
|
||||
coming.
|
||||
- **A Gotchas section exceeding 25% of the body is a SUGGESTION** — the body has been inverted into
|
||||
a preamble.
|
||||
- Place the section near the top. A gotcha read after the mistake is worthless, which is also why
|
||||
Gotchas is the one construct exempt from moving to `references/`.
|
||||
|
||||
Worked negative example — `git-commits` carries thirteen entries, of which four restate content
|
||||
that already appears below or in the description:
|
||||
|
||||
| Gotcha | Restates |
|
||||
|---|---|
|
||||
| `:31` "Communicates SemVer impact" | the description |
|
||||
| `:32` "Confirmation gates are mandatory for destructive operations" | step 9 at `:52` |
|
||||
| `:33` "Never skip hooks with `--no-verify`" | step 9 at `:52` |
|
||||
| `:36` "Never commit secrets" | step 2 at `:45` |
|
||||
|
||||
All four are FAILs under this rule, and the section as a whole breaches the five-entry maximum. It
|
||||
also passes every plausible word gate, which is the point of auditing the construct directly.
|
||||
|
||||
## Calibrating control
|
||||
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed:
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be
|
||||
followed:
|
||||
|
||||
```markdown
|
||||
Run exactly:
|
||||
\`\`\`bash
|
||||
@@ -39,11 +122,13 @@ python scripts/migrate.py --verify --backup
|
||||
Do not modify the command or add additional flags.
|
||||
```
|
||||
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives — agents make better decisions when they understand the purpose.
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives —
|
||||
agents make better decisions when they understand the purpose.
|
||||
|
||||
## Defaults not menus
|
||||
|
||||
Never present a list of equivalent options — pick one and mention the alternative briefly:
|
||||
|
||||
```markdown
|
||||
# Too many options
|
||||
Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
@@ -52,37 +137,22 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
|
||||
```
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
Highest value content — environment-specific facts that defy reasonable assumptions. Place near the top of the body so the agent reads them before encountering the situation.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Each entry must be a specific, surprising fact — not a general tip or reminder.
|
||||
|
||||
## Progressive disclosure
|
||||
|
||||
Keep `SKILL.md` under 500 lines. When more content is needed, move it to `references/` and load conditionally:
|
||||
|
||||
```markdown
|
||||
If the API returns a non-200 status, read `references/api-errors.md`.
|
||||
```
|
||||
|
||||
"If X, read Y" is more useful than "see references/ for details." The agent loads on demand rather than up front.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- A sentence answers "no" to the core test (would agent get this wrong without it?) — it is padding
|
||||
- Decision points present a menu of options with no default
|
||||
- Instructions repeat content already in the description
|
||||
- Prescriptive sequences are used where flexibility is fine, or vice versa
|
||||
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
|
||||
- Two or more mutually exclusive flows are inlined instead of dispatched
|
||||
- A Gotcha paraphrases a step in the body below it, or the section exceeds five entries
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- A rationale is missing from an include/exclude rule (present but unexplained)
|
||||
|
||||
- The body exceeds 600 words counted body-only but stays at or under 900
|
||||
- The Gotchas section exceeds 25% of the body
|
||||
- A rationale is missing from an include/exclude rule — present but unexplained
|
||||
- Gotchas are correct but placed late in the body rather than near the top
|
||||
- A conditional reference trigger is vague ("see references/") rather than specific ("If X, read Y")
|
||||
- Content that only one branch reaches is inlined where a `references/` file would serve
|
||||
|
||||
@@ -6,49 +6,112 @@ source_keys:
|
||||
|
||||
# Description Quality Reference
|
||||
|
||||
Source: agentskills.io — optimizing-descriptions
|
||||
Upstream source: agentskills.io — optimizing-descriptions, specification.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than the spec
|
||||
rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## How triggering works
|
||||
## Why the description is the expensive part
|
||||
|
||||
At startup, agents load only the `name` and `description` of each skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. **The description carries the entire triggering burden** — the body is never seen until after triggering.
|
||||
At startup an agent loads only the `name` and `description` of every installed skill. The body is
|
||||
never seen until the skill triggers. The description therefore carries the entire triggering
|
||||
burden **and** is paid for in every session, whether the skill fires or not.
|
||||
|
||||
Agents typically consult skills only for tasks requiring knowledge beyond their defaults. Specialized knowledge — unfamiliar APIs, domain-specific workflows, uncommon formats — is where description wording makes the difference.
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the agent takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## What a good description does
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
- **Imperative phrasing** — "Use when..." not "This skill does...". The agent is deciding whether to act.
|
||||
- **User intent, not mechanics** — describe what the user is trying to achieve, not how the skill works internally.
|
||||
- **Err toward being pushy** — explicitly name contexts where the skill applies, including cases where the user doesn't name the domain: "even if they don't mention X explicitly."
|
||||
- **Specificity over vagueness** — "parses and validates OpenAPI specs" beats "helps with APIs."
|
||||
- **Near-miss exclusions** — add "Do not use when..." only if a near-miss skill exists that could steal activations. Use strong near-misses (queries that share keywords but need something different), not weak ones ("write a fibonacci function").
|
||||
- **Hard limit: 1024 characters** — descriptions grow during revision; check length before finalising.
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
- **`disable-model-invocation: true`** — the skill is hand-invoked. Its description is never
|
||||
matched against user intent, so it is not a routing string. It carries **one plain human-facing
|
||||
sentence** stating what the skill does. Audit it for that and nothing else. Reporting a missing
|
||||
trigger clause, a missing boundary clause or absent indirect triggers on a hand-invoked skill is
|
||||
a wrong finding, not a strict one.
|
||||
- **No such flag** — the skill is model-invoked and the rest of this file applies.
|
||||
|
||||
## The three-part shape
|
||||
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This skill ...` —
|
||||
the agent is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source; `validate.sh` checks that
|
||||
deterministically and a dangling target already surfaces as a Structure FAIL.
|
||||
|
||||
Everything else belongs in the body or in `README.md`.
|
||||
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on a skill whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Near-miss exclusions
|
||||
|
||||
Add a boundary clause only where a sibling skill could plausibly steal the activation. Use strong
|
||||
near-misses — queries that share keywords but need something different — not weak ones ("write a
|
||||
fibonacci function"). One boundary clause per genuine near-miss; a list of four is enumeration
|
||||
wearing a boundary's clothes.
|
||||
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# Weak
|
||||
description: Process CSV files.
|
||||
|
||||
# Strong
|
||||
# FAIL — enumeration first, mechanics as the opener, a blanket indirect trigger,
|
||||
# and 300+ characters of it preloaded into every session forever.
|
||||
description: >
|
||||
Analyze CSV and tabular data files — compute summary statistics,
|
||||
add derived columns, generate charts, and clean messy data. Use when
|
||||
the user has a CSV, TSV, or Excel file and wants to explore, transform,
|
||||
or visualize the data, even if they don't explicitly mention "CSV" or
|
||||
"analysis."
|
||||
Analyze CSV and tabular data files — compute summary statistics, add derived
|
||||
columns, generate charts, and clean messy data. Use when the user has a CSV,
|
||||
TSV, or Excel file and wants to explore, transform, or visualize the data,
|
||||
even if they don't explicitly mention "CSV" or "analysis."
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The four verbs the FAIL
|
||||
# version enumerates are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when the user has a CSV, TSV, or Excel file and wants it explored,
|
||||
transformed, or charted. Not schema design -> data-model.
|
||||
```
|
||||
|
||||
The strong version names capabilities precisely and broadens applicability beyond explicit keyword matches.
|
||||
(`data-model` is illustrative. In a real description the target has to resolve.)
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- Phrasing is descriptive ("This skill...") not imperative ("Use when...")
|
||||
- Capabilities are vague ("helps with APIs") — require precise verbs and nouns
|
||||
- No indirect trigger coverage when indirect cases clearly exist
|
||||
- No near-miss exclusions when a sibling skill could plausibly steal activations
|
||||
- Length exceeds 1024 characters
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared skill", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
||||
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
|
||||
file in the authoring source. `validate.sh` reports the unresolved name.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0.
|
||||
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
||||
of the 400-character house ceiling above.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- Indirect trigger coverage exists but could be more specific
|
||||
- Near-miss exclusions are present but target weak near-misses only
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
---
|
||||
|
||||
# File Structure and Internal Consistency Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (optional directories, file references).
|
||||
|
||||
Read this when judging the **file-structure** and **internal-consistency** dimensions.
|
||||
|
||||
## Permitted directories
|
||||
|
||||
Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification permits additional
|
||||
directories; this house does not, because an unlisted directory is content no auditor and no host
|
||||
knows to look at. Flag any other directory as a FAIL.
|
||||
|
||||
- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`,
|
||||
`test_*.sh`) there are a FAIL — they belong in `tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
- `README.md` is present and describes the skill and its files accurately.
|
||||
|
||||
## Cross-plugin path references
|
||||
|
||||
A plugin is copied to a cache on install, and a path that climbs out of the skill directory stops
|
||||
resolving there. Flag any `../`, `../../`, or absolute repo path (`plugins/<plugin>/skills/<other>/`
|
||||
and its APM-native equivalent `.apm/skills/<other>/`) appearing in `SKILL.md`, `scripts/`,
|
||||
`references/` or `assets/`.
|
||||
|
||||
Two directories are exempt, and the exemptions are structural rather than discretionary:
|
||||
|
||||
- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers,
|
||||
not runtime references. They are expected to be unresolvable after install, and
|
||||
`validate-provenance.sh` handles that by skipping upstream checks silently when the path is
|
||||
absent. Flagging them would make every correctly-provenanced skill fail.
|
||||
- **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared
|
||||
`tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/`
|
||||
exists and `tests/README.md` is absent or does not document it, that is a FAIL.
|
||||
|
||||
## Internal consistency
|
||||
|
||||
The skill has to agree with itself. Three checks:
|
||||
|
||||
- `SKILL.md`'s steps match what the scripts actually do — the arguments, the exit codes, and the
|
||||
output shape it tells the agent to expect.
|
||||
- `README.md`'s file table lists every file that exists, with no missing rows and no stale rows for
|
||||
files since deleted.
|
||||
- Placeholder READMEs inside `scripts/`, `references/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
A stale README row is the most common finding here and the easiest to miss from inside an
|
||||
authoring pass, because the author knows what was intended and reads it into the gap.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A directory outside the four permitted ones exists
|
||||
- Test files sit in `scripts/`
|
||||
- A non-spec file sits at the skill root
|
||||
- A cross-plugin or parent-relative path appears outside the two exempt locations
|
||||
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
||||
- `README.md` is absent, or its file table has a missing or stale row
|
||||
- `SKILL.md` describes a script invocation the script does not accept
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Formatting and Scripts Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (body content), using-scripts (designing scripts
|
||||
for agentic use).
|
||||
|
||||
Read this when judging the **formatting** and **scripts** dimensions. Both are checklists of static
|
||||
criteria that never vary by skill, which is exactly why they live here rather than in the body.
|
||||
|
||||
## Formatting
|
||||
|
||||
- Heading levels are consistent: H2 for main sections, H3 for subsections. A body that jumps from
|
||||
H2 to H4, or opens on H3, reads as a fragment of a larger document.
|
||||
- Code blocks carry a language tag wherever one applies — `bash`, `markdown`, `python`, `yaml`,
|
||||
`text`. An untagged block loses syntax highlighting and, more importantly, loses the signal of
|
||||
what the agent is meant to do with it.
|
||||
- Whitespace is consistent: a blank line between sections, one list-indentation style throughout.
|
||||
- No broken relative paths in file references. Every `references/…`, `scripts/…` and `assets/…`
|
||||
path named in the body resolves against the skill directory.
|
||||
|
||||
## Scripts
|
||||
|
||||
A script in a skill is run by an agent with no terminal and no human to answer it. The criteria
|
||||
follow from that:
|
||||
|
||||
- **No interactive TTY prompts** — no `read`, no `input()`, no `readline`. A script that blocks on
|
||||
a prompt hangs the run with no diagnostic. `validate.sh` detects the common forms and reports
|
||||
them under Structure; the judgment call is any prompt it cannot pattern-match. What counts is
|
||||
where stdin comes from, not the word `read`: a `read` fed by a here-string, a here-doc, a pipe,
|
||||
or a redirect from a file never touches a terminal and is not a finding. `validate.sh` excludes
|
||||
those forms, so do not rewrite a working `read -r A B <<< "$line"` into parameter expansion to
|
||||
satisfy this rule.
|
||||
- **`--help` is exposed** and gives concise usage.
|
||||
- **Data to stdout, diagnostics to stderr.** A caller piping the script has to be able to separate
|
||||
the result from the commentary.
|
||||
- **Idempotent** — "create if not exists" rather than "create", so a re-run after a partial failure
|
||||
is safe.
|
||||
- **Meaningful exit codes, documented in `--help`.** An agent branches on the exit code; an
|
||||
undocumented one is a coin flip.
|
||||
- **`--dry-run` present for destructive operations.**
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A script prompts interactively, in any form
|
||||
- A script exposes no `--help`
|
||||
- A destructive script has no `--dry-run`
|
||||
- Data and diagnostics share a stream, so the output cannot be piped
|
||||
- A relative path named in the body does not resolve
|
||||
- Heading levels are inconsistent enough to break the document's structure
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Exit codes are meaningful but undocumented in `--help`
|
||||
- A code block is untagged where a language applies
|
||||
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
|
||||
- List indentation or section spacing is inconsistent without breaking the render
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
---
|
||||
|
||||
# Patterns Reference
|
||||
|
||||
Upstream source: agentskills.io — best-practices (instruction patterns), specification.
|
||||
|
||||
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
|
||||
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
|
||||
*content* is judged in `references/body-discipline.md`.
|
||||
|
||||
## The constructs and when each is right
|
||||
|
||||
| Construct | Right when | Wrong when |
|
||||
|---|---|---|
|
||||
| Gotchas | An environment fact contradicts a reasonable default | Used as a summary of the steps below |
|
||||
| Prescriptive sequence | The operation is fragile and flag order or exact arguments must not change | Several approaches are equally valid |
|
||||
| Checklist | A multi-step workflow the agent must complete in order | A single step dressed up as a list |
|
||||
| Conditional reference | Detail is needed on one branch only | The reference is needed on every run and is loaded blind |
|
||||
| Output template | The agent must emit a specific format a caller consumes | The output is prose nobody parses |
|
||||
|
||||
## Formation rules
|
||||
|
||||
**Gotchas** sit near the top of the body, before the steps that would otherwise walk into them.
|
||||
Placement late in the body is a SUGGESTION, not a FAIL — the content is still correct, it is just
|
||||
read after the mistake.
|
||||
|
||||
**Prescriptive sequences** that quote a fenced block inside another markdown block must escape the
|
||||
inner fence as `` \`\`\` ``. An unescaped inner fence terminates the outer block and the remaining
|
||||
instructions render as prose.
|
||||
|
||||
**Conditional references** state a specific trigger: "If the API returns a non-200 status, read
|
||||
`references/api-errors.md`." The generic form — pointing at the directory and hoping — defeats
|
||||
progressive disclosure, because the agent either loads everything or loads nothing.
|
||||
`Kyberforge.PaddingPhrase` catches the common generic phrasing deterministically; other malformed
|
||||
forms are judgment.
|
||||
|
||||
**Output templates** belong in the body when the agent must emit them on every run, and in
|
||||
`references/` when only one dispatch branch produces that output. A template inlined for a branch
|
||||
most invocations never take is body-discipline padding.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
|
||||
assumption
|
||||
- An inner code fence is unescaped inside a markdown block, breaking the render
|
||||
- A checklist wraps a single step
|
||||
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
|
||||
- The agent must produce a specific format and no output template is given
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Gotchas are correctly formed but placed late in the body
|
||||
- An output template is present but permissive where the consumer needs it exact
|
||||
- A conditional reference names a trigger that is real but broader than the branch it guards
|
||||
@@ -15,7 +15,7 @@
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
@@ -23,7 +23,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
@@ -47,7 +47,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
@@ -11,7 +11,7 @@ Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
EOF
|
||||
}
|
||||
@@ -32,6 +32,7 @@ python3 -u - "$1" <<'PYTHON'
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
@@ -44,6 +45,7 @@ with open(skill_md) as f:
|
||||
content = f.read()
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
@@ -53,6 +55,13 @@ def fail(msg):
|
||||
print(f"FAIL {msg}")
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
|
||||
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
|
||||
if not fm_match:
|
||||
@@ -66,16 +75,58 @@ body_start = fm_match.end()
|
||||
name_m = re.search(r'^name:\s*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — inline or block scalar (> or |)
|
||||
desc = ""
|
||||
desc_m = re.search(r'^description:\s*([>|])\n((?:[ \t]+.+\n?)+)', fm, re.MULTILINE)
|
||||
if desc_m:
|
||||
raw = desc_m.group(2)
|
||||
desc = re.sub(r'\s+', ' ', raw).strip()
|
||||
else:
|
||||
desc_inline = re.search(r'^description:\s*(.+)', fm, re.MULTILINE)
|
||||
if desc_inline:
|
||||
desc = desc_inline.group(1).strip()
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first. PyYAML is used
|
||||
# when importable (it is a real parser); the fallback recognises exactly the
|
||||
# shapes this corpus uses — an inline scalar, optionally quoted and optionally
|
||||
# continued on following indented lines, and a `>`/`|` block scalar with
|
||||
# optional indentation and chomping indicators.
|
||||
|
||||
def normalize(value):
|
||||
return re.sub(r'\s+', ' ', value).strip()
|
||||
|
||||
def fold_description_fallback(fm_text):
|
||||
lines = fm_text.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^description:[ \t]*(.*)$', line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1).strip()
|
||||
block = bool(re.match(r'^[>|][0-9]*[-+]?$|^[>|][-+]?[0-9]*$', head))
|
||||
parts = [] if block else [head]
|
||||
for nxt in lines[i + 1:]:
|
||||
if not nxt.strip():
|
||||
parts.append('')
|
||||
continue
|
||||
if not re.match(r'^[ \t]', nxt):
|
||||
break
|
||||
parts.append(nxt.strip())
|
||||
value = ' '.join(parts)
|
||||
if not block:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in '"\'':
|
||||
value = value[1:-1]
|
||||
return value
|
||||
return ''
|
||||
|
||||
def extract_description(fm_text):
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(fm_text)
|
||||
if isinstance(data, dict):
|
||||
value = data.get('description')
|
||||
if isinstance(value, str):
|
||||
return normalize(value)
|
||||
if value is not None:
|
||||
return normalize(str(value))
|
||||
return ''
|
||||
except Exception:
|
||||
pass
|
||||
return normalize(fold_description_fallback(fm_text))
|
||||
|
||||
desc = extract_description(fm)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
@@ -114,11 +165,13 @@ if desc:
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (limit: 1024)")
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
@@ -146,6 +199,26 @@ MAX_LINES = 500
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
|
||||
# description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
@@ -160,14 +233,220 @@ if word_count <= MAX_WORDS:
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
# Body unfilled placeholders
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# A boundary clause names another skill — or an agent, which is an equally
|
||||
# valid routing target (git-workflow routes to the git-orchestrate agent). Every
|
||||
# named target is resolved against the AUTHORING SOURCE, plugins/*/.apm/skills/
|
||||
# and plugins/*/.apm/agents/, so the check works offline and before an
|
||||
# `apm install` has deployed anything into .claude/skills/.
|
||||
#
|
||||
# False positives are the design constraint here, not recall. Two rules do the
|
||||
# work:
|
||||
# * A BARE hyphenated word is read as a routing target only inside a boundary
|
||||
# sentence (one carrying "do not"/"instead"/"rather than"/"not for").
|
||||
# Without that, pc-run's "run pre-commit hooks" reads as a route to a
|
||||
# non-existent `pre-commit` skill.
|
||||
# * A BARE arrow target counts only in ADR-0020's compressed boundary form,
|
||||
# `Not <thing> -> <skill-name>`. Without that, diagnose's process chain
|
||||
# "fix -> regression-test" reads as a route to `regression-test`.
|
||||
# Backticked and /slash-command targets are unambiguous and always count. Tool
|
||||
# names (Read, Write, Edit) are excluded by the lowercase-only name pattern;
|
||||
# MCP tool names (issue_write, pull_request_write) by its rejection of
|
||||
# underscores; file names by its rejection of dots and slashes.
|
||||
|
||||
NAME_ANY = r"[a-z0-9]+(?:-[a-z0-9]+)*"
|
||||
NAME_HYPH = r"[a-z0-9]+(?:-[a-z0-9]+)+"
|
||||
ROUTE_VERB = (r"(?:use|uses|using|run|runs|invoke|invokes|invoking|try|see"
|
||||
r"|that'?s|compose|composes|call|calls)")
|
||||
MARKED_TARGET = r"(?:`/?(%s)`|(?<![\w./*-])/(%s)\b)" % (NAME_ANY, NAME_ANY)
|
||||
ANY_TARGET = r"(?:%s|(%s)\b)" % (MARKED_TARGET, NAME_HYPH)
|
||||
ROUTE_MARKED = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, MARKED_TARGET), re.I)
|
||||
ROUTE_ANY = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, ANY_TARGET), re.I)
|
||||
CONT_MARKED = re.compile(r"\s*(?:or|and|/|,)\s*%s" % MARKED_TARGET)
|
||||
CONT_ANY = re.compile(r"\s*(?:or|and|/|,)\s*%s" % ANY_TARGET)
|
||||
ARROW_MARKED = re.compile(r"(?:->|→)\s*%s" % MARKED_TARGET)
|
||||
ARROW_BOUNDARY = re.compile(r"\bnot\b[^.;]*?(?:->|→)\s*(%s)\b" % NAME_HYPH, re.I)
|
||||
BACKTICK = re.compile(r"`(%s)`" % NAME_HYPH)
|
||||
BOUNDARY_MARKER = re.compile(r"\b(?:do\s+not|instead|rather\s+than|not\s+for)\b", re.I)
|
||||
SENTENCE_SPLIT = re.compile('(?<=[.!?])\\s+(?=[A-Z"“(])')
|
||||
|
||||
def _first_group(groups):
|
||||
for g in groups:
|
||||
if g:
|
||||
return g
|
||||
return None
|
||||
|
||||
def _scan_routes(text, route_re, cont_re, out):
|
||||
for m in route_re.finditer(text):
|
||||
target = _first_group(m.groups())
|
||||
if not target:
|
||||
continue
|
||||
out.append(target)
|
||||
# Conjoined targets: "use git-history or git-branches instead",
|
||||
# "use gitea-issues / gitea-prs".
|
||||
pos = m.end()
|
||||
while True:
|
||||
cm = cont_re.match(text, pos)
|
||||
if not cm:
|
||||
break
|
||||
nxt = _first_group(cm.groups())
|
||||
if nxt:
|
||||
out.append(nxt)
|
||||
pos = cm.end()
|
||||
|
||||
def boundary_targets(description):
|
||||
out = []
|
||||
for sentence in SENTENCE_SPLIT.split(description):
|
||||
boundary = bool(BOUNDARY_MARKER.search(sentence))
|
||||
_scan_routes(sentence,
|
||||
ROUTE_ANY if boundary else ROUTE_MARKED,
|
||||
CONT_ANY if boundary else CONT_MARKED,
|
||||
out)
|
||||
for m in ARROW_MARKED.finditer(sentence):
|
||||
target = _first_group(m.groups())
|
||||
if target:
|
||||
out.append(target)
|
||||
for m in ARROW_BOUNDARY.finditer(sentence):
|
||||
out.append(m.group(1))
|
||||
out.extend(BACKTICK.findall(sentence))
|
||||
return sorted(set(out))
|
||||
|
||||
def known_targets(start_dir):
|
||||
names = set()
|
||||
# Sibling skills/agents. This is the branch that works in a cache-installed
|
||||
# plugin and in a deployed .claude/skills/ tree, neither of which has a
|
||||
# plugins/ directory above it.
|
||||
parent = os.path.dirname(os.path.abspath(start_dir))
|
||||
if os.path.basename(parent) == 'skills' and os.path.isdir(parent):
|
||||
for entry in os.listdir(parent):
|
||||
if os.path.isdir(os.path.join(parent, entry)):
|
||||
names.add(entry)
|
||||
agents_dir = os.path.join(os.path.dirname(parent), 'agents')
|
||||
if os.path.isdir(agents_dir):
|
||||
for entry in os.listdir(agents_dir):
|
||||
if entry.endswith('.agent.md'):
|
||||
names.add(entry[:-len('.agent.md')])
|
||||
elif entry.endswith('.md'):
|
||||
names.add(entry[:-len('.md')])
|
||||
# Walk up for a monorepo root (plugins/*/.apm/) or a plugin root (.apm/).
|
||||
# Capped at ten levels so a pathological path can't become a filesystem
|
||||
# crawl; that covers every real layout by a wide margin.
|
||||
current = os.path.abspath(start_dir)
|
||||
for _ in range(10):
|
||||
# Never glob the filesystem root: a stray /.apm/skills/ (a scaffolding
|
||||
# test's leftover, say) would otherwise become part of every skill's
|
||||
# resolution universe on that machine.
|
||||
if os.path.dirname(current) == current:
|
||||
break
|
||||
for pattern in ('plugins/*/.apm/skills/*/', '.apm/skills/*/'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path.rstrip('/')))
|
||||
for pattern in ('plugins/*/.apm/agents/*.agent.md', '.apm/agents/*.agent.md'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path)[:-len('.agent.md')])
|
||||
current = os.path.dirname(current)
|
||||
return names
|
||||
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
# An empty universe means no authoring source was found anywhere above this
|
||||
# skill — reporting every target as dangling there would be noise, not a
|
||||
# finding, so the check declines to run rather than guessing.
|
||||
if routing_targets and known:
|
||||
unresolved = [t for t in routing_targets if t not in known]
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill under "
|
||||
f"plugins/*/.apm/skills/ and no agent under plugins/*/.apm/agents/ — "
|
||||
f"a boundary clause naming a non-existent target sends the router nowhere")
|
||||
if not unresolved:
|
||||
ok(f"all {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(routing_targets)}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
for line in source.splitlines():
|
||||
stripped = line.strip()
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
@@ -177,9 +456,10 @@ if os.path.isdir(scripts_dir):
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
with open(fpath) as f:
|
||||
sc = f.read()
|
||||
# Interactive prompt heuristic
|
||||
if re.search(r'^\s*(read\s|input\()', sc, re.MULTILINE):
|
||||
fail(f"scripts/{fname}: may use interactive input (read/input detected)")
|
||||
interactive = interactive_reads(sc)
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
else:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
@@ -190,8 +470,17 @@ if os.path.isdir(scripts_dir):
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
print("All checks passed.")
|
||||
if suggestions:
|
||||
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
|
||||
@@ -25,6 +25,39 @@ description: A valid skill description that is well within the limit.
|
||||
Do the thing.
|
||||
EOF
|
||||
}
|
||||
|
||||
# Helper: create a skill directory with an exact description length and an
|
||||
# exact body word count. <desc> is used verbatim; <body_words> "word"
|
||||
# tokens follow the frontmatter. Used by the ADR-0020 boundary tests.
|
||||
make_sized_skill() {
|
||||
local dir="$1" desc="$2" body_words="$3"
|
||||
local name
|
||||
name="$(basename "$dir")"
|
||||
mkdir -p "$dir"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: $name"
|
||||
echo "description: $desc"
|
||||
echo "---"
|
||||
echo ""
|
||||
python3 -c "print(' '.join(['word'] * $body_words))"
|
||||
} > "$dir/SKILL.md"
|
||||
}
|
||||
|
||||
# Helper: build a self-contained fixture plugin tree so the boundary-target
|
||||
# resolver has a real authoring source to resolve against, independent of
|
||||
# this repo's live skills. Echoes the subject skill's directory.
|
||||
#
|
||||
# <root>/plugins/fixture-plugin/.apm/skills/<subject>/SKILL.md
|
||||
# <root>/plugins/fixture-plugin/.apm/skills/fixture-sibling-skill/
|
||||
# <root>/plugins/fixture-plugin/.apm/agents/fixture-sibling-agent.agent.md
|
||||
make_fixture_tree() {
|
||||
local root="$1" subject="$2"
|
||||
local apm="$root/plugins/fixture-plugin/.apm"
|
||||
mkdir -p "$apm/skills/$subject" "$apm/skills/fixture-sibling-skill" "$apm/agents"
|
||||
touch "$apm/agents/fixture-sibling-agent.agent.md"
|
||||
echo "$apm/skills/$subject"
|
||||
}
|
||||
}
|
||||
|
||||
teardown() {
|
||||
@@ -64,7 +97,7 @@ teardown() {
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes at exactly 1024-char description" {
|
||||
@test "the 1024-char agentskills.io spec backstop is unchanged and separate from the ADR-0020 ceiling" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
local name
|
||||
name="$(basename "$skill")"
|
||||
@@ -82,7 +115,12 @@ description: $desc
|
||||
Do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
# Two independent gates on one value: the spec limit still PASSES at
|
||||
# exactly 1024 (its own boundary is unmoved), while ADR-0020's 400-char
|
||||
# ceiling FAILs. The run fails on the second, not the first.
|
||||
assert_output --partial "description length 1024 chars (agentskills.io spec limit: 1024)"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "passes at exactly 500 lines" {
|
||||
@@ -170,6 +208,61 @@ EOF
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "fails when a script reads a variable with no redirect" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r ANSWER\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "fails when an interactive prompt string contains an angle bracket" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -p "enter <name>: " NAME\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a here-string" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nLINE="a b"\nread -r X Y <<< "$LINE"\n' \
|
||||
> "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a here-doc" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r X <<EOF\nvalue\nEOF\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a file redirect" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nread -r LINE < "$1"\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "passes when a script reads from a pipe continued onto the next line" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
printf '#!/usr/bin/env bash\nprintf %%s "$1" |\n read -r X\n' > "$skill/scripts/helper.sh"
|
||||
chmod +x "$skill/scripts/helper.sh"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
}
|
||||
|
||||
@test "fails when name contains consecutive hyphens" {
|
||||
local skill="$TMPDIR/my--skill"
|
||||
make_valid_skill "$skill"
|
||||
@@ -196,3 +289,222 @@ EOF
|
||||
run bash "$SCRIPT"
|
||||
assert_failure
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — description budget (250 SUGGESTION / 400 FAIL)
|
||||
#
|
||||
# These sit UNDER the agentskills.io 1024-character spec backstop above, which
|
||||
# is unchanged. Both ceilings are inclusive: exactly at the number passes that
|
||||
# tier, one past it trips.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: description of exactly 250 chars raises no suggestion" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 250)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of 251 chars raises a SUGGESTION and still exits 0" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 251)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
assert_output --partial "description is 251 chars"
|
||||
assert_output --partial "All checks passed (1 suggestion(s))."
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of exactly 400 chars is a SUGGESTION, not a FAIL" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 400)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description of 401 chars FAILs and exits non-zero" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "$(python3 -c "print('x' * 401)")" 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "description is 401 chars"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: description length is measured after YAML folding is resolved" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
mkdir -p "$skill"
|
||||
# A >-folded block scalar: 11 lines of 40 chars folded with 10 joining
|
||||
# spaces = 450 characters. Measured off its raw `description: >` line it is
|
||||
# 1 character and passes; measured as the folded VALUE it must FAIL. This
|
||||
# is exactly the case a line-wise regex gets wrong.
|
||||
{
|
||||
echo "---"
|
||||
echo "name: my-skill"
|
||||
echo "description: >"
|
||||
python3 -c "print('\n'.join([' ' + 'x' * 40] * 11))"
|
||||
echo "---"
|
||||
echo ""
|
||||
echo "Do the thing."
|
||||
} > "$skill/SKILL.md"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "description is 450 chars"
|
||||
assert_output --partial "400-character ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — body budget (600 SUGGESTION / 900 FAIL), body ONLY
|
||||
#
|
||||
# Distinct from the 2,770-word whole-file spec ceiling above, which counts
|
||||
# frontmatter too and is unchanged. Do not unify them.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: body of exactly 600 words raises no suggestion" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 600
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "SUGGESTION"
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of 601 words raises a SUGGESTION and still exits 0" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 601
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "body is 601 words"
|
||||
assert_output --partial "All checks passed (1 suggestion(s))."
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of exactly 900 words is a SUGGESTION, not a FAIL" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 900
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "body is 900 words"
|
||||
}
|
||||
|
||||
@test "ADR-0020: body of 901 words FAILs and exits non-zero" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_sized_skill "$skill" "A short valid description." 901
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "body is 901 words"
|
||||
assert_output --partial "900-word ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
@test "ADR-0020: the body gate counts the body only — frontmatter words do not count toward it" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
# 895 body words plus a description long enough that the WHOLE FILE is well
|
||||
# over 900 words. The body gate must stay silent; the 2,770-word whole-file
|
||||
# ceiling is a separate measurement and is nowhere near tripping.
|
||||
make_sized_skill "$skill" "$(python3 -c "print(' '.join(['w'] * 100))")" 895
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "900-word ADR-0020 ceiling"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 — resolvable boundary targets
|
||||
#
|
||||
# Resolved against the AUTHORING SOURCE (plugins/*/.apm/skills/ and
|
||||
# plugins/*/.apm/agents/), never .claude/skills/, so the check works offline and
|
||||
# before an apm install. Every fixture below builds its own plugin tree rather
|
||||
# than leaning on this repo's live skills.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "ADR-0020: a boundary target naming an existing sibling skill resolves" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use fixture-sibling-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "boundary target(s) resolve"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a boundary target naming a non-existent skill FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use fixture-missing-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-skill'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a boundary target naming an AGENT file resolves (agents are valid routing targets)" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use when the caller is an agent — invoke fixture-sibling-agent instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
assert_output --partial "boundary target(s) resolve"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a /slash-command boundary target that does not resolve FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use when improvements are wanted — use /fixture-missing-improve instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-improve'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a backticked name that does not resolve FAILs" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Composes \`fixture-missing-helper\` for the shared part." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-helper'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: a bare hyphenated word outside a boundary sentence is not read as a routing target" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
# "run pre-commit hooks" is pc-run's real phrasing. A naive extractor reads
|
||||
# it as a route to a non-existent `pre-commit` skill.
|
||||
make_sized_skill "$skill" "Use when the user wants to run pre-commit hooks or install git hooks." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "pre-commit"
|
||||
}
|
||||
|
||||
@test "ADR-0020: an arrow chain outside a boundary clause is not read as a routing target" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
# diagnose's real process chain. Only ADR-0020's `Not <thing> -> <skill>`
|
||||
# form makes a bare arrow target a route.
|
||||
make_sized_skill "$skill" "Reproduce → minimise → instrument → fix → regression-test. Use when a bug is reported." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "regression-test"
|
||||
}
|
||||
|
||||
@test "ADR-0020: MCP tool names and capitalised tool names are not read as routing targets" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when writing issues. Do not use for local files (use Read/Write/Edit) — that write goes through \`issue_write\`/\`pull_request_write\` instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "routes to"
|
||||
}
|
||||
|
||||
@test "ADR-0020: ADR's compressed boundary form (Not <thing> -> <skill>) is checked" {
|
||||
local skill
|
||||
skill="$(make_fixture_tree "$TMPDIR/tree" "my-skill")"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Not the other thing → fixture-missing-target." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_failure
|
||||
assert_output --partial "routes to 'fixture-missing-target'"
|
||||
}
|
||||
|
||||
@test "ADR-0020: the boundary check declines rather than false-FAILs when no authoring source is found" {
|
||||
local skill="$TMPDIR/orphan/my-skill"
|
||||
make_sized_skill "$skill" "Use when doing the thing. Do not use for the other thing — use some-other-skill instead." 10
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "routes to"
|
||||
}
|
||||
|
||||
@@ -6,6 +6,14 @@ Author and refine skills conforming to the [agentskills.io](https://agentskills.
|
||||
|
||||
Routes to one of two flows based on context: if no skill directory exists at the target path, it scaffolds the directory from annotated templates, fills in `SKILL.md` and supporting files, and validates the result. If an existing skill directory and improvement signals are both present, it groups those signals by root cause and applies targeted edits, then re-validates. In both flows, bumps the skill's `metadata.version` when present (minor for create, patch for improve).
|
||||
|
||||
`SKILL.md` itself carries only the dispatch table, the invocation-axis decision, the contract gates and the shared close; each flow lives in its own self-contained reference file, per ADR-0020.
|
||||
|
||||
## The contract it teaches
|
||||
|
||||
Authored skills are held to the ADR-0020 context budget. A description carries a trigger clause, at most one capability clause, and a boundary clause of the form `Not <thing> -> <skill-name>` whose target must resolve to a real skill or agent — 250 characters target, 400 hard ceiling. A body carries the decision procedure only — 600 words target, 900 hard ceiling, counting the body alone, which is a separate measurement from the 2,770-word / 500-line whole-file spec backstop. Skills with two or more mutually exclusive flows must dispatch. `references/contract.md` holds the full rules; `assets/templates/SKILL.md` encodes them as a fill-in skeleton.
|
||||
|
||||
Before a description is written, the skill asks whether the target is model-invoked or hand-invoked. A hand-invoked skill sets `disable-model-invocation: true` and carries one plain human-facing sentence with no trigger list.
|
||||
|
||||
## Before you start
|
||||
|
||||
- Run `/grill-me` to resolve design decisions before creating a new skill
|
||||
@@ -14,7 +22,7 @@ Routes to one of two flows based on context: if no skill directory exists at the
|
||||
|
||||
## Placement
|
||||
|
||||
`scripts/new-skill.sh` resolves the mode automatically by walking up from the given path — see `SKILL.md` Step 1 for the full algorithm.
|
||||
`scripts/new-skill.sh` resolves the mode automatically by walking up from the given path — see `references/create.md` Step 1 for the full algorithm.
|
||||
|
||||
| Mode | Path | Chosen when |
|
||||
|------|------|-------------|
|
||||
@@ -36,10 +44,13 @@ If the destination resolves inside an APM package, read `references/deployment-m
|
||||
| `README.md` | Human-readable overview of the skill and its files |
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `scripts/new-skill.sh` | Walks up from the given path to resolve package vs standalone mode, then copies annotated templates to the resolved destination |
|
||||
| `references/create.md` | The create flow end to end — prerequisites, package-intent gate, scaffold, frontmatter, scripts, references, sources (loaded on demand) |
|
||||
| `references/improve.md` | The improve flow end to end — signal verification, root-cause grouping, announcement, edits (loaded on demand) |
|
||||
| `references/contract.md` | The ADR-0020 description and body contract, the Gotchas constraint, the two size gates, body patterns, and org-policy embedding (loaded on demand) |
|
||||
| `references/deployment-modes.md` | APM package vs standalone differences and self-containment/cache-isolation rules (loaded on demand) |
|
||||
| `references/scripts.md` | Package runners, inline dependency patterns, and full script contract (loaded on demand) |
|
||||
| `references/sources.md` | Upstream research sources and which skill files each contributed to |
|
||||
| `assets/templates/SKILL.md` | Annotated SKILL.md template |
|
||||
| `assets/templates/SKILL.md` | Annotated SKILL.md template — emits an ADR-0020-compliant description and body skeleton |
|
||||
| `assets/templates/README.md` | Annotated README template for the new skill |
|
||||
| `assets/templates/scripts/README.md` | Placeholder for bundled scripts |
|
||||
| `assets/templates/references/README.md` | Placeholder for reference docs |
|
||||
|
||||
@@ -1,14 +1,9 @@
|
||||
---
|
||||
name: skill-author
|
||||
description: >
|
||||
Use when the user wants to create a new skill from scratch ("write a skill
|
||||
for X", "build a skill that does Y", "create a SKILL.md for Z") or improve
|
||||
an existing one ("improve this skill", "fix based on feedback", "apply these
|
||||
audit findings", "update based on grill output"). Also use when the user provides inline feedback
|
||||
about a skill's behavior and wants it applied, or when a grill session, eval
|
||||
run, or audit has produced findings the user wants acted on — even if they
|
||||
don't say "improve" explicitly. Do not use for read-only review — use
|
||||
/skill-audit instead. Do not use to author agent definition files.
|
||||
Use when the user wants to create a new skill from scratch, or apply audit
|
||||
findings, grill output, eval results, or inline feedback to an existing one.
|
||||
Not read-only review -> `skill-audit`. Not agent files -> `agent-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -24,283 +19,44 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Patching per symptom is the default failure mode. Three eval failures may all trace to one missing instruction — always identify the root cause before editing.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing scripts from scratch requires transcript analysis that is out of scope here; flag the opportunity as a suggestion instead.
|
||||
- Never spawn a subagent to audit or recheck your own work during an authoring pass. Run `/skill-audit` yourself, inline, in the same context as the edits you just made. A *separate* independent recheck via a clean-context subagent is the `/forge` skill's outer-loop responsibility exclusively — delegating it inward here duplicates that layer and introduces a race: a stray self-spawned subagent can have its worktree torn down by concurrent cleanup, destroying an uncommitted draft before it was ever safe.
|
||||
- A skill's `name` and `description` are preloaded into every agent's context every session, invoked or not; the body loads only on invocation. The description is the scarce budget.
|
||||
- The word gates are two different measurements, not one rule with two tiers. The 2,770-word / 500-line spec backstop counts the whole file including frontmatter; Step 3's gate counts the body alone. A file can sit well inside one and fail the other, so never unify them.
|
||||
- Never spawn a subagent to audit or recheck your own work here. Run `/skill-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent can have its worktree torn down by concurrent cleanup, destroying an uncommitted draft.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing one from scratch requires transcript analysis that is out of scope here — flag the opportunity as a suggestion instead.
|
||||
|
||||
## Route
|
||||
## Step 1 — Dispatch
|
||||
|
||||
Determine which flow to follow before touching the filesystem:
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| No skill directory at the target path | Create | `references/create.md` |
|
||||
| Directory exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| Directory exists, no signals | Stop and ask | — |
|
||||
|
||||
- **No skill directory at the target path** → follow **Creating a new skill**
|
||||
- **Directory exists + at least one improvement signal present** → follow **Improving an existing skill**
|
||||
- **Directory exists + no signals present** → ask: "No improvement signals found. Did you mean to create a new skill, or do you have feedback to apply?"
|
||||
Signals: grill output, `/skill-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new skill, or do you have feedback to apply?"
|
||||
|
||||
Signals include: grill session output, `/skill-audit` findings (PASS/FAIL punch list), inline user feedback, session context describing what went wrong.
|
||||
Read only the reference matching the resolved flow — each is self-contained. Capture `git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
**Before running the scaffold script**, judge whether the destination is meant to be inside an APM package — the script can't tell "no package here" apart from "package not scaffolded yet":
|
||||
## Step 2 — Invocation axis
|
||||
|
||||
- Package intent but no `type:`-bearing `apm.yml` found at/above the destination (e.g. "add to my apm package", or a sibling `.apm/`/`apm.yml` exists nearby) → **stop**, tell the user to run `/apm-workflow configure` (`apm plugin init`, from inside the package directory) first, then retry. Don't fall through to standalone mode.
|
||||
- Otherwise (a `~/`-rooted destination, or no package context implied) → run `scripts/new-skill.sh`; it resolves package vs. standalone automatically (see Step 1).
|
||||
Decide before writing any description: model-invoked or hand-invoked?
|
||||
|
||||
## Creating a new skill
|
||||
- **Hand-invoked** — the user types `/name` and no agent should route to it. Set `disable-model-invocation: true` and write one plain human-facing sentence: no trigger list, no boundary clause. Worked example: `plugins/bin/.apm/skills/zoom-out/SKILL.md`. Skip Step 3's description rules.
|
||||
- **Model-invoked** — the default.
|
||||
|
||||
### Prerequisites
|
||||
## Step 3 — Contract
|
||||
|
||||
Run `/grill-me` on the skill's design and research the target domain first.
|
||||
Share those outputs in this conversation: grill context, research docs, examples, constraints.
|
||||
Before writing or editing a description, or restructuring a body, read `references/contract.md` — the banned-content list, boundary form, include/exclude rubric and body patterns.
|
||||
|
||||
Design for one coherent user intent — skills too narrow force multiple loads per task; too broad are hard to activate precisely.
|
||||
Gates `/skill-audit` enforces in both flows:
|
||||
|
||||
**Before touching the filesystem, verify you have:**
|
||||
- [ ] A clear purpose — what specific task will this skill handle?
|
||||
- [ ] Trigger scenarios — when should an agent activate it, including indirect cases?
|
||||
- [ ] Skill name (kebab-case) and destination path
|
||||
- [ ] Capture `git log --oneline -1` now, before touching the filesystem — Step 7 needs it to verify a real commit landed
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <skill-name>` whose target resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only.
|
||||
- **Body** — decision procedure only: ordered steps, branches, gates, and which reference to load when. 600 words SUGGESTION, 900 FAIL, body only. At two or more mutually exclusive flows a dispatch table is mandatory and each flow gets its own self-contained `references/` file.
|
||||
- **Gotchas** — at most five, each contradicting a reasonable default. A Gotcha paraphrasing a step below it is a FAIL.
|
||||
|
||||
If any are missing, stop and ask the user before proceeding.
|
||||
## Step 4 — Validate and close
|
||||
|
||||
**Requires `/skill-audit`** — used in Step 7 for final validation. Both skills ship in the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
|
||||
Run `/skill-audit` on the resolved skill directory. It checks name-to-directory match, description presence, leftover `FILL IN:` placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those first. Resolve every FAIL before reporting done.
|
||||
|
||||
### Step 1 — Scaffold
|
||||
With `metadata.version` present, bump the **minor** version on create (new skills start at `0.1.0`) and the **patch** version on improve.
|
||||
|
||||
Run the copy script with the skill name and a path inside or at the target:
|
||||
|
||||
```bash
|
||||
bash scripts/new-skill.sh <skill-name> <path>
|
||||
```
|
||||
|
||||
The script walks up from `<path>` for a package boundary: an ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) means **package mode** — scaffolds into `<package-root>/.apm/skills/<skill-name>/`, not under `<path>` (a subdirectory of the package works fine as `<path>`). A `type:`-less `apm.yml` is a marketplace-only manifest, skipped. Hitting `.git` or the filesystem root first means **standalone mode** — scaffolds directly into `<path>/<skill-name>/`, same as before.
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
# Package mode — packages/my-pkg/apm.yml already has `type: skill`
|
||||
bash scripts/new-skill.sh my-tool packages/my-pkg/
|
||||
|
||||
# Standalone mode — no apm.yml/.git above ~/.agents/skills/
|
||||
bash scripts/new-skill.sh my-tool ~/.agents/skills/
|
||||
```
|
||||
|
||||
The script prints which mode it used and where the skill landed — read its output.
|
||||
|
||||
In package mode, read `references/deployment-modes.md` before adding any file references to SKILL.md.
|
||||
|
||||
### Step 2 — Update `apm.yml` includes (package mode only)
|
||||
|
||||
Skip in standalone mode. In package mode, check the resolved package's `apm.yml`: if `includes:` is an explicit list (not `auto`), append `.apm/skills/<skill-name>/` to it if not already present, preserving YAML formatting. If `includes: auto` or the field is absent, do nothing — `auto` already covers the new skill. Use Read/Edit directly on `apm.yml`; this isn't part of `scripts/new-skill.sh`.
|
||||
|
||||
### Step 3 — Fill in SKILL.md
|
||||
|
||||
Open the new skill's `SKILL.md` (the path Step 1 printed). Replace every `FILL IN:` placeholder.
|
||||
|
||||
**Frontmatter**
|
||||
|
||||
**`name`** — already set by the scaffold script. Must exactly match the directory name. Format: 1–64 characters, lowercase letters/numbers/hyphens only, no leading, trailing, or consecutive hyphens (`--`).
|
||||
|
||||
**`description`** — carries the entire triggering burden. Rules:
|
||||
- Imperative: "Use when..." not "This skill..."
|
||||
- Focus on user intent, not implementation — describe what the user is trying to achieve, not the skill's internal mechanics
|
||||
- Specific about capabilities ("parses and validates OpenAPI specs", not "helps with APIs")
|
||||
- Include indirect triggers: "even if the user doesn't mention X explicitly"
|
||||
- Add "Do not use when..." only if a near-miss skill exists that could steal activations
|
||||
- Hard limit: 1024 characters — count before finalizing
|
||||
|
||||
**Optional fields** — uncomment and fill in or remove entirely:
|
||||
- `license` — include when distributing the skill externally
|
||||
- `compatibility` — include if the skill requires specific tools, runtimes, or network access (max 500 characters)
|
||||
- `metadata` — key-value map; use `author`, `version`, `category`; add `source_keys` now (see below) if research sources are in context
|
||||
- `allowed-tools` — space-separated pre-approved tools; reduces permission prompts (experimental — support varies by client)
|
||||
|
||||
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs here as you write the body; don't defer this to Step 6. Agents that fill in source_keys late tend to omit it entirely. Example:
|
||||
```yaml
|
||||
metadata:
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
- another-slug
|
||||
```
|
||||
|
||||
**Embedding org-specific policy** — if a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`), inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing to the file's path. Plugins must be self-contained and portable — the org file may not exist wherever the plugin is installed, and in this repo such files are meant to be deleted once their content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the same `references/sources.md` schema as Step 6, noting in the `Research doc:` field that the source is an org convention rather than a plugin research corpus entry, so provenance survives after the source file is gone.
|
||||
|
||||
**Body — include only what the agent lacks**
|
||||
|
||||
Rename the placeholder section heading to one that fits the skill's structure — `## Step 1`, `## Workflow`, `## Instructions`, etc.
|
||||
|
||||
Ask of every sentence: "Would the agent get this wrong without it?" Cut anything that answers "no."
|
||||
|
||||
**Include:**
|
||||
- Non-obvious sequences or ordering constraints — the agent may skip or reorder steps without this
|
||||
- Domain conventions the agent cannot infer from general knowledge — this is the core value a skill adds
|
||||
- One default per decision point, plus one escape hatch — never a menu; menus cause the agent to pause or pick arbitrarily
|
||||
- Gotchas — facts that defy reasonable assumptions; the agent will get these wrong every time without them
|
||||
|
||||
**Exclude:**
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works) — adds tokens without changing behavior
|
||||
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads agents to follow unproductive paths
|
||||
- Restatements of the description — it's already in context; repeating it wastes the token budget
|
||||
|
||||
**Patterns**
|
||||
|
||||
**Gotchas** — highest value; place near the top:
|
||||
````markdown
|
||||
## Gotchas
|
||||
- <Fact that defies a reasonable assumption>
|
||||
- <Non-obvious naming discrepancy or hidden constraint>
|
||||
````
|
||||
|
||||
**Default with escape hatch** (not a menu):
|
||||
````markdown
|
||||
Use <X> for <task>. For <edge case>, use <Y> instead.
|
||||
````
|
||||
|
||||
**Prescriptive sequence** (when order is critical or fragile):
|
||||
````markdown
|
||||
Run exactly:
|
||||
```bash
|
||||
<command>
|
||||
```
|
||||
Do not modify flags.
|
||||
````
|
||||
|
||||
**Checklist** (multi-step workflows):
|
||||
````markdown
|
||||
- [ ] Step 1: ...
|
||||
- [ ] Step 2: ...
|
||||
````
|
||||
|
||||
**Conditional reference** (progressive disclosure — load only when needed):
|
||||
````markdown
|
||||
If <condition>, read `references/<file>.md`.
|
||||
````
|
||||
|
||||
**Output format template** (when the skill produces structured output):
|
||||
````markdown
|
||||
Output format:
|
||||
```
|
||||
<field>: <value>
|
||||
<field>: <value>
|
||||
```
|
||||
````
|
||||
For longer templates, place in `assets/<name>.md` and reference conditionally.
|
||||
|
||||
**Size budget**
|
||||
|
||||
Keep `SKILL.md` under 500 lines; 5,000 tokens is the recommended body budget. When approaching the limit:
|
||||
- Move reference material to `references/<topic>.md` and load it conditionally
|
||||
- Bundle repeated executable logic into `scripts/` rather than reinventing each run
|
||||
|
||||
### Step 4 — Add scripts (if needed)
|
||||
|
||||
Place executable scripts in `scripts/`. Critical rule: **no interactive prompts** — agents run non-interactive; blocking on TTY input hangs indefinitely. Accept all input via flags, env vars, or stdin.
|
||||
|
||||
If adding a script, read `references/scripts.md` first — it covers the full contract: structured output, pinned versions, self-contained deps, idempotency, exit codes, dry-run, error messages, and output size limits.
|
||||
|
||||
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
|
||||
|
||||
### Step 5 — Add references, assets, and tests (if needed)
|
||||
|
||||
**`references/`** — additional documentation loaded on demand. One topic per file.
|
||||
Reference conditionally from SKILL.md: `If <condition>, read references/<file>.md`.
|
||||
Keep reference chains one level deep — a reference file that references another reference file is rarely loaded correctly.
|
||||
|
||||
**`assets/`** — static resources: templates, schemas, lookup tables.
|
||||
Reference by relative path from SKILL.md.
|
||||
|
||||
**`tests/`** — test files for scripts in `scripts/`. Use when scripts are complex
|
||||
enough to break silently. Test infrastructure (`.bats`, `*_test.*`) belongs here,
|
||||
not in `scripts/`. See `tests/README.md` for setup instructions.
|
||||
|
||||
If not needed, delete the placeholder READMEs and their directories.
|
||||
|
||||
### Step 6 — Populate or delete `references/sources.md`
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
|
||||
1. Read it and filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, determine which skill files it contributed to (SKILL.md and any files in references/ that drew from it). Update `Contributing files` accordingly — list skill files, not research topic files.
|
||||
3. Write the updated content to `references/sources.md`. For each entry, include `- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the plugin-level research sources file this entry was drawn from (e.g. `plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
|
||||
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of sources that informed it.
|
||||
5. For each file in `references/` that was informed by research sources, add `source_keys` frontmatter (same format as research topic files) listing the relevant slugs.
|
||||
|
||||
If no research `sources.md` is in context, delete `references/sources.md`.
|
||||
|
||||
### Step 7 — Validate and close
|
||||
|
||||
Before running the audit, confirm:
|
||||
- [ ] Skill name matches the directory name exactly
|
||||
- [ ] `description` field is present and non-empty
|
||||
- [ ] Body has at least one non-empty section
|
||||
- [ ] No `FILL IN:` placeholders remain in any file
|
||||
|
||||
Run `/skill-audit` on the skill directory Step 1 reported — either `<package-root>/.apm/skills/<skill-name>/` or `<path>/<skill-name>/`.
|
||||
|
||||
All FAIL findings must be resolved before the skill is considered done.
|
||||
|
||||
If the skill is versioned (`metadata.version`), set it to the next **minor** version (e.g. `0.2.0` → `0.3.0`). New skills without a prior version start at `0.1.0`.
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` before Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the new skill files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the skill as done once the hash has actually changed.
|
||||
|
||||
## Improving an existing skill
|
||||
|
||||
### Step 1 — Verify inputs
|
||||
|
||||
Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.
|
||||
|
||||
If the skill dir is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
|
||||
Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
Also verify the `name` field in frontmatter matches the skill's directory name exactly.
|
||||
|
||||
### Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/, tests/). Then collect all signals from the conversation and any file paths the user has referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- Session context: output format is wrong on every run
|
||||
- Audit finding: no output template defined
|
||||
- User feedback: "I always have to ask it to format the output"
|
||||
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
|
||||
```
|
||||
|
||||
### Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state:
|
||||
- Which root causes were identified and what evidence supports each
|
||||
- Which files will be changed and what will change in each
|
||||
|
||||
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
### Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, tests/, README.md.
|
||||
|
||||
**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs.
|
||||
|
||||
**Keep it lean.** Remove instructions that aren't pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
|
||||
|
||||
If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Before running the audit, confirm:
|
||||
- [ ] Skill name still matches the directory name
|
||||
- [ ] No `FILL IN:` placeholders were introduced
|
||||
- [ ] No previously-passing audit checks were broken by the edits
|
||||
|
||||
Run `/skill-audit` on the skill directory. Resolve any FAIL findings before considering the improvement complete.
|
||||
|
||||
If the skill is versioned (`metadata.version`), bump the **patch** version (e.g. `0.1.0` → `0.1.1`).
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` at the start of Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the changed files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the improvement as done once the hash has actually changed.
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` — do not stop at staging. Re-run `git log --oneline -1` and confirm the hash changed from the one captured at Step 1. A non-empty `git diff --stat` is not proof: staged-but-uncommitted work is part of no commit and is silently lost if the tree is cleaned up first. Report done only once the hash has changed.
|
||||
|
||||
@@ -10,11 +10,24 @@ name: SKILL_NAME
|
||||
# Examples: my-tool, data-analyzer, pdf-processor
|
||||
|
||||
description: >
|
||||
FILL IN: What does this skill do? State capabilities specifically
|
||||
Use when FILL IN: trigger — when should an agent activate this skill?
|
||||
FILL IN: at most ONE capability clause, stated specifically
|
||||
(e.g. "parses and validates OpenAPI specs", not "helps with APIs").
|
||||
Use when FILL IN: when should an agent activate this skill?
|
||||
Include indirect triggers: even if the user doesn't mention X explicitly.
|
||||
Do not use when FILL IN: near-miss exclusions — remove this line if none apply.
|
||||
Not FILL IN: near-miss case -> FILL IN: real sibling skill name.
|
||||
# Required. Preloaded into EVERY session whether or not the skill is invoked.
|
||||
# Exactly three parts, in this order: trigger clause, at most one capability
|
||||
# clause, boundary clause. Drop the boundary line if no near-miss skill exists.
|
||||
# Budget: 250 characters target, 400 hard ceiling (counting this value only).
|
||||
# Banned here: capability lists, output-format detail, composition notes,
|
||||
# implementation detail, and restating one trigger twice in two registers.
|
||||
# The boundary target must resolve to a real skill or agent — it is checked.
|
||||
# Add "even if the user doesn't mention X explicitly" ONLY when the user's
|
||||
# natural phrasing genuinely omits the domain word.
|
||||
|
||||
# disable-model-invocation: true
|
||||
# Optional. Hand-invoked skills only (reached solely by the user typing
|
||||
# /SKILL_NAME). With this set, replace the description above with ONE plain
|
||||
# human-facing sentence — no trigger clause, no boundary clause.
|
||||
|
||||
# license: MIT
|
||||
# Optional. License name (e.g. MIT, Apache-2.0) or relative path to a bundled
|
||||
@@ -42,29 +55,30 @@ description: >
|
||||
---
|
||||
|
||||
<!-- ============================================================
|
||||
SKILL BODY
|
||||
SKILL BODY — decision procedure ONLY
|
||||
|
||||
Include only what the agent lacks:
|
||||
- Domain conventions the agent cannot infer from general knowledge
|
||||
- Non-obvious sequences or ordering constraints
|
||||
- One default per decision point + one escape hatch (never a menu)
|
||||
- Gotchas — facts that defy reasonable assumptions
|
||||
Keep here: ordered steps, decision branches, gates, and which reference
|
||||
file to load when.
|
||||
|
||||
Omit:
|
||||
- Concepts the agent already knows
|
||||
- Exhaustive option lists
|
||||
- Steps the agent handles independently
|
||||
- Restatements of the description
|
||||
Move to references/: lookup tables, spec restatements, output schemas,
|
||||
templates, example blocks, rationale prose, and anything only one branch
|
||||
reaches. Wire each one with the literal conditional form
|
||||
"If <condition>, read `references/<file>.md`." — a generic
|
||||
"see references/ for details" is a lint error.
|
||||
|
||||
Size budget: under 500 lines / 5000 tokens.
|
||||
Move reference material to references/ and load it conditionally.
|
||||
Bundle repeated executable logic into scripts/.
|
||||
Budget: 600 words target, 900 hard ceiling, counting THIS BODY ONLY
|
||||
(everything after the closing --- above). Separate from the whole-file
|
||||
spec backstop of 2,770 words / 500 lines — do not conflate them.
|
||||
|
||||
Delete this comment block before shipping.
|
||||
============================================================ -->
|
||||
|
||||
<!-- OPTIONAL: Gotchas section — highest-value content. Place near the top.
|
||||
Add facts that defy reasonable assumptions or non-obvious constraints.
|
||||
<!-- OPTIONAL but high-value: Gotchas. Place near the top — a gotcha read
|
||||
after the mistake is worthless.
|
||||
|
||||
Each entry states a fact that CONTRADICTS a reasonable default:
|
||||
something the agent gets wrong by acting sensibly. Maximum 5 entries.
|
||||
An entry that paraphrases a step below it is a failure, not a gotcha.
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -72,7 +86,22 @@ description: >
|
||||
- FILL IN: non-obvious naming discrepancy or hidden constraint
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Multi-step workflow checklist.
|
||||
<!-- DISPATCH — MANDATORY when this skill has two or more mutually exclusive
|
||||
flows. Keep only the dispatch table plus the gates common to every
|
||||
branch in this body; give each flow its own self-contained
|
||||
references/ file. Delete this block for a single-flow skill.
|
||||
|
||||
## Step 1 — Dispatch
|
||||
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| FILL IN: condition | FILL IN: flow | `references/FILL IN.md` |
|
||||
| FILL IN: condition | FILL IN: flow | `references/FILL IN.md` |
|
||||
|
||||
Read only the reference matching the resolved flow — each is self-contained.
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: single-flow workflow checklist. Delete if the skill dispatches.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -81,23 +110,12 @@ description: >
|
||||
- [ ] Step 3: FILL IN
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Output format template — use when the agent must produce a specific format.
|
||||
<!-- OPTIONAL: gates that apply to every branch — validation, versioning,
|
||||
closing checks. Keep these in the body even when flows are dispatched.
|
||||
|
||||
## Output format
|
||||
## Step N — Validate and close
|
||||
|
||||
Use this structure:
|
||||
|
||||
```markdown
|
||||
# [FILL IN: Title]
|
||||
|
||||
## FILL IN: Section
|
||||
FILL IN: what goes here
|
||||
```
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Conditional reference — load documentation only when needed.
|
||||
|
||||
If FILL IN: condition, read `references/FILL IN: filename.md`.
|
||||
FILL IN: the check that must pass before this skill reports done.
|
||||
-->
|
||||
|
||||
## FILL IN: <section-name (e.g. Step 1, Workflow, Instructions)>
|
||||
|
||||
@@ -5,9 +5,17 @@ without bloating its core context.
|
||||
|
||||
## When to add a reference file
|
||||
|
||||
Move content here when SKILL.md is approaching 500 lines, or when a topic
|
||||
is only relevant in specific circumstances (error handling, edge cases,
|
||||
domain-specific sub-procedures).
|
||||
The SKILL.md body carries the decision procedure only. Everything else lives
|
||||
here: lookup tables, spec restatements, output schemas, templates, example
|
||||
blocks, rationale prose, and anything only one branch reaches.
|
||||
|
||||
Two triggers make a reference file mandatory rather than optional:
|
||||
|
||||
- The body is over its 600-word target (900 is a hard failure), counting the
|
||||
body only — everything after the frontmatter's closing `---`.
|
||||
- The skill has two or more mutually exclusive flows. The body then keeps only
|
||||
a dispatch table plus the gates common to every branch, and each flow gets
|
||||
its own self-contained file here (e.g. `create.md`, `improve.md`).
|
||||
|
||||
## How to reference from SKILL.md
|
||||
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
---
|
||||
|
||||
# The description and body contract
|
||||
|
||||
House contract, set by ADR-0020. Every rule here is enforced by `/skill-audit` —
|
||||
`scripts/validate.sh` for the counts and the boundary targets, the bundled Vale styles for the
|
||||
prose patterns, and its reference files for the judgment calls.
|
||||
|
||||
## Why the budget exists
|
||||
|
||||
A skill's `name` and `description` are loaded into every agent's context at the start of every
|
||||
session, whether or not the skill is ever invoked. The body is loaded only on invocation, and then
|
||||
competes with the caller's live conversation. Those are two different costs, so they get two
|
||||
different ceilings — and a fat description is not merely expensive. A description that summarizes
|
||||
the workflow gets followed *instead of* the body: a description saying "code review between tasks"
|
||||
produced one review from a skill whose flowchart specified two.
|
||||
|
||||
## Description
|
||||
|
||||
A description carries exactly three things:
|
||||
|
||||
1. **Trigger clause** — when to invoke, imperative: "Use when ...", never "This skill ...".
|
||||
Focus on user intent, not the skill's internal mechanics.
|
||||
2. **At most one capability clause** — what it does, one clause, no enumeration. Be specific
|
||||
("parses and validates OpenAPI specs", not "helps with APIs").
|
||||
3. **Boundary clause** — form: `Not <thing> -> <skill-name>.` Add one only where a near-miss skill
|
||||
could steal activations.
|
||||
|
||||
Banned from a description; move it to the body or to `README.md`:
|
||||
|
||||
- Capability enumeration or feature lists
|
||||
- Output-format detail ("Produces a compact findings report with Why and Fix per finding")
|
||||
- Composition or architecture notes ("composes X rather than duplicating Y", "This is a
|
||||
cross-cutting shared skill", "the human-facing entry point")
|
||||
- Implementation detail ("Self-validates via a bundled deterministic script")
|
||||
- Restating the same trigger twice in two registers — a verb list, then the same verbs re-quoted
|
||||
as user phrasings. This is a FAIL, not a suggestion.
|
||||
|
||||
**Indirect triggers are conditional, not mandatory.** Add "even if the user doesn't mention X
|
||||
explicitly" only where the user's natural phrasing genuinely omits the domain word — true for the
|
||||
`gitea-*` family, because people say "create an issue" rather than "create a Gitea issue"; false
|
||||
for `git-commits`, where the user says "commit". Adding one everywhere is what inflated this
|
||||
corpus, and it was deleted as a blanket rule.
|
||||
|
||||
**Boundary targets must resolve.** The name after the arrow is checked against real skill
|
||||
directories under `plugins/*/.apm/skills/<name>/` and real agents under
|
||||
`plugins/*/.apm/agents/<name>.agent.md`. A boundary clause naming a target that does not exist
|
||||
sends the router nowhere and fails the audit. Check the target exists before writing it — do not
|
||||
invent a plausible sibling name.
|
||||
|
||||
**Length.** 250 characters SUGGESTION, 400 characters FAIL, counting the frontmatter value only
|
||||
with YAML folding resolved. The agentskills.io 1,024-character spec limit is unchanged and sits
|
||||
above both. The SUGGESTION tier is the one that moves the average; treat 250 as the target and 400
|
||||
as the outlier stop.
|
||||
|
||||
**Hand-invoked skills are exempt.** A skill carrying `disable-model-invocation: true` is absent
|
||||
from the model-visible listing and is reached only by the user typing `/name`. It takes one plain
|
||||
human-facing sentence — no trigger clause, no boundary clause, no indirect triggers. Worked
|
||||
example: `plugins/bin/.apm/skills/zoom-out/SKILL.md`.
|
||||
|
||||
## Body
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when. Everything else moves to `references/`.
|
||||
|
||||
Ask of every sentence: "Would the agent get this wrong without it?" Cut anything that answers "no."
|
||||
|
||||
Include:
|
||||
|
||||
- Non-obvious sequences or ordering constraints — the agent may skip or reorder steps without this
|
||||
- Domain conventions the agent cannot infer from general knowledge — the core value a skill adds
|
||||
- One default per decision point, plus one escape hatch — never a menu; menus cause the agent to
|
||||
pause or pick arbitrarily
|
||||
- Gotchas — facts that defy reasonable assumptions
|
||||
|
||||
Exclude:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works) — tokens without behavior change
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads agents down unproductive paths
|
||||
- Restatements of the description — it is already in context
|
||||
|
||||
Move to `references/`: lookup tables, spec restatements, output schemas, templates, example
|
||||
blocks, rationale prose, and any content only one branch reaches. Each reference file is
|
||||
self-contained for its concern, and every one is wired from the body with the literal conditional
|
||||
form:
|
||||
|
||||
````markdown
|
||||
If <condition>, read `references/<file>.md`.
|
||||
````
|
||||
|
||||
A generic pointer ("see references/ for details") is a Vale error — the agent cannot act on it.
|
||||
|
||||
**Dispatch is mandatory at two or more mutually exclusive flows.** The body carries the dispatch
|
||||
table and the gates common to every branch; each flow gets its own self-contained `references/`
|
||||
file. Exemplar: `plugins/kyberforge/.apm/skills/apm-workflow/SKILL.md` — a 554-word body
|
||||
dispatching to 3,006 words of references.
|
||||
|
||||
**Length.** 600 words SUGGESTION, 900 words FAIL, counting the **body only** — everything after
|
||||
the frontmatter's closing `---`.
|
||||
|
||||
## Gotchas section
|
||||
|
||||
- Each entry must state a fact that **contradicts a reasonable default** — something the agent
|
||||
gets wrong by acting sensibly. "Never commit secrets" is not one; the agent already knows.
|
||||
- Maximum five entries.
|
||||
- A Gotcha that paraphrases a step in the body below it is a **FAIL**. If the rule is already a
|
||||
step, it is not a gotcha.
|
||||
- A Gotchas section exceeding 25% of the body is a SUGGESTION.
|
||||
- Place the section near the top — a gotcha read after the mistake is worthless.
|
||||
|
||||
## Two size gates, two measurements
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| description | 250 chars | 400 chars | the `description:` value only |
|
||||
| body | 600 words | 900 words | the body only, after the closing `---` |
|
||||
| spec backstop | — | 1,024 chars | the `description:` value only |
|
||||
| spec backstop | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 600/900 pair and the 2,770/500 pair are not the same measurement and must not be unified: the
|
||||
first is a quality gate on what the caller's context absorbs, the second a conformance backstop on
|
||||
the file. A skill can sit well inside one and fail the other.
|
||||
|
||||
When a body approaches its ceiling, relocate rather than delete — move reference material to
|
||||
`references/<topic>.md` behind a conditional trigger, and bundle repeated executable logic into
|
||||
`scripts/` rather than reinventing it each run.
|
||||
|
||||
## Body patterns
|
||||
|
||||
**Default with escape hatch** (not a menu):
|
||||
|
||||
````markdown
|
||||
Use <X> for <task>. For <edge case>, use <Y> instead.
|
||||
````
|
||||
|
||||
**Prescriptive sequence** (when order is critical or fragile):
|
||||
|
||||
````markdown
|
||||
Run exactly:
|
||||
```bash
|
||||
<command>
|
||||
```
|
||||
Do not modify flags.
|
||||
````
|
||||
|
||||
**Checklist** (multi-step workflows):
|
||||
|
||||
````markdown
|
||||
- [ ] Step 1: ...
|
||||
- [ ] Step 2: ...
|
||||
````
|
||||
|
||||
**Dispatch table** (two or more mutually exclusive flows):
|
||||
|
||||
````markdown
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| <condition> | <flow> | `references/<file>.md` |
|
||||
````
|
||||
|
||||
**Output format template** (when the skill produces structured output):
|
||||
|
||||
````markdown
|
||||
Output format:
|
||||
```
|
||||
<field>: <value>
|
||||
```
|
||||
````
|
||||
|
||||
For longer templates, place them in `assets/<name>.md` and reference conditionally.
|
||||
|
||||
## Embedding org-specific policy
|
||||
|
||||
If a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`),
|
||||
inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing
|
||||
to the file's path. Plugins must be self-contained and portable — the org file may not exist
|
||||
wherever the plugin is installed, and in this repo such files are meant to be deleted once their
|
||||
content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the
|
||||
same `references/sources.md` schema as the create flow's Step 6, noting in the `Research doc:`
|
||||
field that the source is an org convention rather than a plugin research corpus entry, so
|
||||
provenance survives after the source file is gone.
|
||||
173
plugins/kyberforge/.apm/skills/skill-author/references/create.md
Normal file
173
plugins/kyberforge/.apm/skills/skill-author/references/create.md
Normal file
@@ -0,0 +1,173 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-quickstart
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Creating a new skill
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 6 below is done — validation, versioning and commit
|
||||
verification are shared with the improve flow and are not repeated here.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Run `/grill-me` on the skill's design and research the target domain first. Share those outputs
|
||||
in this conversation: grill context, research docs, examples, constraints.
|
||||
|
||||
Design for one coherent user intent — skills too narrow force multiple loads per task; too broad
|
||||
are hard to activate precisely.
|
||||
|
||||
Before touching the filesystem, verify you have:
|
||||
|
||||
- [ ] A clear purpose — what specific task will this skill handle?
|
||||
- [ ] Trigger scenarios — when should an agent activate it?
|
||||
- [ ] Skill name (kebab-case) and destination path
|
||||
|
||||
If any are missing, stop and ask the user before proceeding.
|
||||
|
||||
**Requires `/skill-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
|
||||
the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the
|
||||
user to install the kyberforge plugin before continuing.
|
||||
|
||||
## Package-intent gate
|
||||
|
||||
Judge whether the destination is meant to be inside an APM package before running the scaffold
|
||||
script — the script cannot tell "no package here" apart from "package not scaffolded yet":
|
||||
|
||||
- Package intent but no `type:`-bearing `apm.yml` found at or above the destination (e.g. "add to
|
||||
my apm package", or a sibling `.apm/`/`apm.yml` exists nearby) → **stop**, tell the user to run
|
||||
`/apm-workflow configure` (`apm plugin init`, from inside the package directory) first, then
|
||||
retry. Do not fall through to standalone mode.
|
||||
- Otherwise (a `~/`-rooted destination, or no package context implied) → continue to Step 1.
|
||||
|
||||
## Step 1 — Scaffold
|
||||
|
||||
Run the copy script with the skill name and a path inside or at the target:
|
||||
|
||||
```bash
|
||||
bash scripts/new-skill.sh <skill-name> <path>
|
||||
```
|
||||
|
||||
The script walks up from `<path>` for a package boundary: an ancestor `apm.yml` with a top-level
|
||||
`type:` field (`instructions`/`skill`/`hybrid`/`prompts`) means **package mode** — scaffolds into
|
||||
`<package-root>/.apm/skills/<skill-name>/`, not under `<path>` (a subdirectory of the package
|
||||
works fine as `<path>`). A `type:`-less `apm.yml` is a marketplace-only manifest, skipped. Hitting
|
||||
`.git` or the filesystem root first means **standalone mode** — scaffolds directly into
|
||||
`<path>/<skill-name>/`.
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Package mode — packages/my-pkg/apm.yml already has `type: skill`
|
||||
bash scripts/new-skill.sh my-tool packages/my-pkg/
|
||||
|
||||
# Standalone mode — no apm.yml/.git above ~/.agents/skills/
|
||||
bash scripts/new-skill.sh my-tool ~/.agents/skills/
|
||||
```
|
||||
|
||||
The script prints which mode it used and where the skill landed — read its output.
|
||||
|
||||
In package mode, read `references/deployment-modes.md` before adding any file references to
|
||||
SKILL.md.
|
||||
|
||||
## Step 2 — Update `apm.yml` includes (package mode only)
|
||||
|
||||
Skip in standalone mode. In package mode, check the resolved package's `apm.yml`: if `includes:`
|
||||
is an explicit list (not `auto`), append `.apm/skills/<skill-name>/` to it if not already present,
|
||||
preserving YAML formatting. If `includes: auto` or the field is absent, do nothing — `auto`
|
||||
already covers the new skill. Use Read/Edit directly on `apm.yml`; this is not part of
|
||||
`scripts/new-skill.sh`.
|
||||
|
||||
## Step 3 — Fill in SKILL.md
|
||||
|
||||
Open the new skill's `SKILL.md` (the path Step 1 printed) and replace every `FILL IN:`
|
||||
placeholder. The scaffold template already carries the compliant frontmatter and body skeleton —
|
||||
fill it rather than restructuring it.
|
||||
|
||||
**`name`** — already set by the scaffold script. Must exactly match the directory name. Format:
|
||||
1–64 characters, lowercase letters, numbers and hyphens only; no leading, trailing or consecutive
|
||||
hyphens (`--`).
|
||||
|
||||
**`description`** — carries the entire triggering burden and is preloaded every session. Write it
|
||||
against `references/contract.md`, which holds the three-part shape, the banned content, the
|
||||
boundary-clause form and the length tiers. A hand-invoked skill (`SKILL.md` Step 2) takes one
|
||||
plain sentence and `disable-model-invocation: true` instead.
|
||||
|
||||
**Optional frontmatter** — uncomment and fill in, or remove entirely:
|
||||
|
||||
- `license` — include when distributing the skill externally
|
||||
- `compatibility` — include if the skill requires specific tools, runtimes, or network access
|
||||
(max 500 characters)
|
||||
- `metadata` — key-value map; use `author`, `version`, `category`; add `source_keys` now (Step 6)
|
||||
if research sources are in context
|
||||
- `allowed-tools` — space-separated pre-approved tools; reduces permission prompts (experimental —
|
||||
support varies by client)
|
||||
- `disable-model-invocation` — hand-invoked skills only
|
||||
|
||||
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs as you
|
||||
write the body; do not defer this to Step 6. Agents that fill in `source_keys` late tend to omit
|
||||
it entirely. Example:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
- another-slug
|
||||
```
|
||||
|
||||
**Body** — write the decision procedure only, following the body rules and patterns in
|
||||
`references/contract.md`. Rename the placeholder section headings to ones that fit the skill's
|
||||
structure.
|
||||
|
||||
## Step 4 — Add scripts (if needed)
|
||||
|
||||
Place executable scripts in `scripts/`. Critical rule: **no interactive prompts** — agents run
|
||||
non-interactive, and blocking on TTY input hangs indefinitely. Accept all input via flags, env
|
||||
vars, or stdin.
|
||||
|
||||
If adding a script, read `references/scripts.md` first — it covers the full contract: structured
|
||||
output, pinned versions, self-contained deps, idempotency, exit codes, dry-run, error messages,
|
||||
and output size limits.
|
||||
|
||||
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
|
||||
|
||||
## Step 5 — Add references, assets, and tests (if needed)
|
||||
|
||||
**`references/`** — additional documentation loaded on demand. One topic per file. Reference
|
||||
conditionally from SKILL.md with the literal form ``If <condition>, read `references/<file>.md` ``.
|
||||
Keep reference chains one level deep — a reference file that references another reference file is
|
||||
rarely loaded correctly.
|
||||
|
||||
**`assets/`** — static resources: templates, schemas, lookup tables. Reference by relative path
|
||||
from SKILL.md.
|
||||
|
||||
**`tests/`** — test files for scripts in `scripts/`. Use when scripts are complex enough to break
|
||||
silently. Test infrastructure (`.bats`, `*_test.*`) belongs here, not in `scripts/`. See
|
||||
`tests/README.md` for setup instructions.
|
||||
|
||||
If not needed, delete the placeholder READMEs and their directories.
|
||||
|
||||
## Step 6 — Populate or delete `references/sources.md`
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
|
||||
1. Read it and filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, determine which skill files it contributed to (SKILL.md and any files in
|
||||
`references/` that drew from it). Update `Contributing files` accordingly — list skill files,
|
||||
not research topic files.
|
||||
3. Write the updated content to `references/sources.md`. For each entry, include
|
||||
`- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the
|
||||
plugin-level research sources file this entry was drawn from (e.g.
|
||||
`plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every
|
||||
entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
|
||||
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of
|
||||
sources that informed it.
|
||||
5. For each file in `references/` that was informed by research sources, add `source_keys`
|
||||
frontmatter (same format as research topic files) listing the relevant slugs.
|
||||
|
||||
If no research `sources.md` is in context, delete `references/sources.md`.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-best-practices
|
||||
- agentskills-evaluating-skills
|
||||
- agentskills-optimizing-descriptions
|
||||
---
|
||||
|
||||
# Improving an existing skill
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 4 below is done — validation, versioning and commit
|
||||
verification are shared with the create flow and are not repeated here.
|
||||
|
||||
## Step 1 — Verify inputs
|
||||
|
||||
Confirm the skill directory path exists and that at least one improvement signal is present in the
|
||||
conversation or a referenced file.
|
||||
|
||||
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
|
||||
existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
Also verify the `name` field in frontmatter matches the skill's directory name exactly.
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in `scripts/`, `references/`, `assets/`,
|
||||
`tests/`). Then collect all signals from the conversation and any file paths the user has
|
||||
referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Patching per symptom is the default failure mode:
|
||||
three eval failures may all trace to one missing instruction. Ask: "What single gap in the skill
|
||||
causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each
|
||||
symptom.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- Session context: output format is wrong on every run
|
||||
- Audit finding: no output template defined
|
||||
- User feedback: "I always have to ask it to format the output"
|
||||
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
|
||||
```
|
||||
|
||||
## Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state:
|
||||
|
||||
- Which root causes were identified and what evidence supports each
|
||||
- Which files will be changed and what will change in each
|
||||
|
||||
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, `scripts/`,
|
||||
`references/`, `assets/`, `tests/`, README.md.
|
||||
|
||||
**Generalize, do not patch.** Find the underlying gap, not the specific example that failed. A fix
|
||||
scoped only to the test cases you have seen will overfit and perform worse on new inputs.
|
||||
|
||||
**Keep it lean.** Remove instructions that are not pulling their weight. For every sentence you
|
||||
add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently
|
||||
outperforms an exhaustive one.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself
|
||||
writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the
|
||||
agent can apply judgment in edge cases.
|
||||
|
||||
**Retrofit before extending.** Any edit to a skill that predates ADR-0020 has to bring it into the
|
||||
contract first — the gates are hot and carry no baseline file, so a one-line fix to a
|
||||
non-compliant skill cannot be committed until the description and body meet
|
||||
`references/contract.md`. Treat that retrofit as part of the same change, not a follow-up.
|
||||
|
||||
If a signal points to a script or reference file, edit that file directly rather than adding a
|
||||
workaround in SKILL.md.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -18,7 +18,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/home.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-spec
|
||||
@@ -26,7 +26,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/contract.md, references/deployment-modes.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
@@ -34,7 +34,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
@@ -42,7 +42,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-evaluating-skills
|
||||
@@ -50,7 +50,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/improve.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-using-scripts
|
||||
@@ -58,7 +58,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md, references/scripts.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
@@ -66,5 +66,5 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/quickstart.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -180,7 +180,8 @@ else
|
||||
fi
|
||||
echo "" >&2
|
||||
echo "Next steps:" >&2
|
||||
echo " 1. Fill in $TARGET/SKILL.md — replace all FILL IN: placeholders" >&2
|
||||
echo " 1. Fill in $TARGET/SKILL.md — replace all FILL IN: placeholders." >&2
|
||||
echo " Description: 250 chars target / 400 ceiling. Body: 600 / 900, body only." >&2
|
||||
echo " 2. Add scripts to scripts/ if needed (or delete the directory)" >&2
|
||||
echo " 3. Add docs to references/ if needed (or delete the directory)" >&2
|
||||
echo " 4. Add resources to assets/ if needed (or delete the directory)" >&2
|
||||
|
||||
@@ -1,32 +1,40 @@
|
||||
# agent-audit
|
||||
|
||||
Audits an agent definition for correctness and quality — a single vendor-neutral file at
|
||||
Audits an agent definition for correctness and quality against the Claude Code and Copilot agent
|
||||
references and the house context-budget contract (ADR-0020) — a single vendor-neutral file at
|
||||
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
|
||||
|
||||
## What it does
|
||||
|
||||
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
|
||||
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
|
||||
vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both harnesses and an
|
||||
unsafe field can't be silently dropped for just one of them. The allowlist itself lives in the
|
||||
`apm-agent-allowlist` section of `references/field-inventory.md` and is read from there as data —
|
||||
consult that section rather than any restatement of it, including this one. As of 2026-08-14 it
|
||||
admits `name`, `description`, `model`, `source_keys`, and `disallowedTools`; `source_keys` is
|
||||
provenance metadata checked separately by `validate-provenance.sh` against `sources.md`, and
|
||||
`disallowedTools` is admitted because a denylist survives verbatim copy where the `tools` allowlist
|
||||
does not (ADR-0016 and its 2026-08-14 amendment).
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance
|
||||
checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags
|
||||
non-imperative description openers, composition and architecture notes, vague wording, padding
|
||||
phrases, "There is/are" sentence openers, and CC-specific "Use proactively" phrasing in a
|
||||
Copilot or vendor-neutral description
|
||||
2. Reads the agent file, and its counterpart when one exists, then loads the contract for its scope
|
||||
3. Applies qualitative checks across description, body, delegation and comment discipline, loading
|
||||
one rubric from `references/` per group
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix —
|
||||
and a result block with handoff to `agent-author`
|
||||
|
||||
At **project/user scope**, accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives
|
||||
the counterpart automatically, and validates both. Runs structural checks via `validate.sh`
|
||||
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
|
||||
Copilot-only fields in the CC file), provenance chain validation via `validate-provenance.sh`
|
||||
(checks `source_keys` against `sources.md` at the plugin root — plugin/APM scope only), then
|
||||
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
|
||||
Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the
|
||||
`Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert
|
||||
is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is
|
||||
unavailable or reports `0 files` scanned. Produces a compact findings report in the same format
|
||||
as `skill-audit`.
|
||||
Two things follow from ADR-0020 and are easy to get backwards. Agents take the **same** description
|
||||
gates a skill takes — 250 characters SUGGESTION, 400 FAIL, since a `name` + `description` is
|
||||
preloaded into every session either way — and **no body word gate at all**, because an agent body
|
||||
becomes the system prompt of a fresh context rather than competing with the caller's live
|
||||
conversation. Body length is judged through the delegation check instead: an agent body that
|
||||
restates a procedure owned by a skill it can invoke is a FAIL, because a plugin-scope agent has no
|
||||
sibling `references/` directory to disclose to and can only delegate.
|
||||
|
||||
At **plugin/APM scope** the audit accepts the single `.apm/agents/<name>.agent.md` file — there is
|
||||
no counterpart, and pair consistency does not apply. `validate.sh` hard-`FAIL`s any frontmatter
|
||||
field outside the vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both
|
||||
harnesses and an unsafe field cannot be silently dropped for just one of them. The allowlist lives
|
||||
in the `apm-agent-allowlist` section of `references/field-inventory.md`, is read from there as data
|
||||
by the script, and is deliberately not restated anywhere else in this skill (ADR-0009).
|
||||
|
||||
At **project/user scope** the audit accepts either file in a CC `.md` / Copilot `.agent.md` pair,
|
||||
derives the counterpart automatically, and validates both, including the field-leakage checks in
|
||||
each direction.
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -42,18 +50,23 @@ Pass the path to either agent file as the argument.
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `assets/vale/.vale.ini` | Vale config: scopes `Kyberforge` to `**/agents/*.md`, `Kyberforge`+`KyberforgeCopilot` to `**/*.agent.md` |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This skill/agent" instead of an imperative "Use when..." |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Flags composition and architecture notes in a description ("cross-cutting", "entry point", "composes", "rather than duplicating") that belong in README.md |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This..." instead of an imperative "Use when..." |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Flags generic "see references/ for info" pointers instead of specific file references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Flags sentences opening with "There is/are" instead of naming the subject directly |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
|
||||
| `assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml` | Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
|
||||
| `references/README.md` | Directory documentation for references/ |
|
||||
| `references/description-quality.md` | Qualitative guide for borderline description findings |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked contract, and the internal-mechanics FAIL |
|
||||
| `references/body-and-delegation.md` | Rubric for the body, delegation and comment-discipline dimensions — the delegation FAIL and why agents take no body word gate |
|
||||
| `references/scope-plugin-apm.md` | Scope contract for a single vendor-neutral APM agent file — allowlist, dimension routing, and the dimensions that do not apply |
|
||||
| `references/scope-project-user.md` | Scope contract for a CC / Copilot pair — counterpart derivation, provider field rules, pair consistency |
|
||||
| `references/validation-scripts.md` | Loaded only when a Step 1 script fails or cannot run — scope-detection walk-up, manual fallback checks, known script failures |
|
||||
| `references/field-inventory.md` | Authoritative field lists read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM-scope allowlist |
|
||||
| `references/sources.md` | Research provenance for skill content |
|
||||
| `scripts/README.md` | Directory documentation for scripts/ |
|
||||
| `scripts/validate.sh` | Structural validation script for agent file pairs |
|
||||
| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `sources.md` (plugin root) |
|
||||
| `scripts/validate.sh` | Structural validator — required fields, name format, placeholder detection, the ADR-0020 description budget, and the field rules for the detected scope |
|
||||
| `scripts/validate-provenance.sh` | Provenance chain validation against `sources.md` at the package root (plugin/APM scope only) |
|
||||
| `scripts/vale-wrap.sh` | Drop-in `vale` wrapper that works around a frontmatter-description NLP scope limitation |
|
||||
| `tests/README.md` | (source-only) Bats test dependency and run instructions |
|
||||
| `tests/validate.bats` | (source-only) Bats tests for validate.sh |
|
||||
|
||||
@@ -1,18 +1,10 @@
|
||||
---
|
||||
name: agent-audit
|
||||
description: >
|
||||
Use when the user wants to review an agent definition they wrote, says "audit this
|
||||
agent", "check if my agent follows best practices", "review my agent file", or wants
|
||||
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
||||
Also invoke proactively after directly hand-editing an agent file pair outside
|
||||
agent-author — an unaudited hand-edit is the same risk as unreviewed code.
|
||||
Audits a Claude Code .md and Copilot .agent.md agent file pair across six dimensions:
|
||||
structural validation, provider safety, description quality, body quality, comment
|
||||
discipline, and pair consistency — plus provenance chain validation. Produces a
|
||||
compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
|
||||
files — use /agent-author instead. Do not use to audit SKILL.md files — use
|
||||
/skill-audit instead.
|
||||
Use when the user wants an agent definition audited — "audit this agent",
|
||||
"review my agent file", "is this ready to ship" — or after hand-editing an
|
||||
agent outside agent-author. Not applying fixes -> agent-author. Not a skill
|
||||
directory -> skill-audit.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -26,80 +18,67 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- The unit of authoring at project/user scope is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention at those scopes — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures. **At plugin/APM scope there is no pair** — the unit of authoring is a single vendor-neutral `.apm/agents/<name>.agent.md` file, and Pair Consistency does not apply there at all (see below).
|
||||
- Scope is detected by walking up from the agent file's directory: at each level, if `apm.yml` exists AND contains a top-level `type: instructions|skill|hybrid|prompts` line, that directory is an APM package root — plugin/APM scope. A `type:`-less `apm.yml` is marketplace-only (see `docs/research/docs/microsoft-apm/monorepo-and-repo-shapes.md`) — skip it and keep walking up. Otherwise, if `.git` is a directory at that level, stop there — project scope. If neither is found before the filesystem root, fall back to user scope at `$HOME`. `plugin.json`/`.claude-plugin/plugin.json` are no longer scope signals for this skill — a directory with only a `plugin.json` and no `apm.yml` falls through to project (or user) scope.
|
||||
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
||||
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Agents take the same 250/400-character description gates as skills and **no body word gate at all** — an agent body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer. Judge an over-long agent body through the delegation check, never by word count.
|
||||
- At plugin/APM scope the agent is a single vendor-neutral file by design: never raise a pair-consistency finding there, and provider safety stops meaning Claude-Code-versus-Copilot field leakage.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Run structural validation
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <path-to-agent-file>
|
||||
bash scripts/validate-provenance.sh <path-to-agent-file>
|
||||
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file> # project/user scope
|
||||
scripts/vale-wrap.sh <path-to-apm-agent-file> # plugin/APM scope — single file
|
||||
bash scripts/validate.sh <agent-file>
|
||||
bash scripts/validate-provenance.sh <agent-file>
|
||||
scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
|
||||
```
|
||||
|
||||
The script accepts either the CC file, the Copilot file, or (at plugin/APM scope) the single `.apm/agents/<name>.agent.md` file. It detects provider from extension and scope from the walk-up above, then runs the checks for that scope.
|
||||
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the ADR-0020 description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
|
||||
|
||||
At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
|
||||
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, `references/field-inventory.md` missing — read `references/validation-scripts.md`; what these scripts measure is not reproducible by reading.
|
||||
|
||||
At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md`. Read that section for the current list rather than reciting one here; it is the authoritative source and it changes. As of 2026-08-14 it is `name`, `description`, `model`, `source_keys`, `disallowedTools` — `source_keys` is provenance metadata, not a provider-specific field, and is validated separately by `validate-provenance.sh` against `sources.md`; `disallowedTools` is a denylist, admitted because denying a tool by name is safe under `apm compile`'s verbatim copy in a way the `tools` allowlist is not (ADR-0016's 2026-08-14 amendment, and the rationale recorded alongside the list itself). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design.
|
||||
`validate-provenance.sh` prints nothing on success and runs at plugin/APM scope only, exiting 0 silently elsewhere. Its FAIL findings become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim.
|
||||
|
||||
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
`validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
|
||||
|
||||
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside the allowlist — read the `apm-agent-allowlist` section of `references/field-inventory.md` for it, do not work from memory (`source_keys` carries provenance metadata, checked separately by `validate-provenance.sh` against `sources.md`).
|
||||
## Step 2 — Read the agent and load its scope contract
|
||||
|
||||
## Step 2 — Qualitative checks
|
||||
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
|
||||
|
||||
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
||||
| Scope | Read |
|
||||
|---|---|
|
||||
| plugin/APM | `references/scope-plugin-apm.md` |
|
||||
| project, user | `references/scope-project-user.md` |
|
||||
|
||||
**Description (both files):**
|
||||
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those as FAILs without re-deriving by judgment.
|
||||
- `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment.
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
If a description finding is borderline, read `references/description-quality.md`.
|
||||
Load a dimension's rubric before judging that dimension.
|
||||
|
||||
**Body:**
|
||||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||||
- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment
|
||||
- Sentences that open with "There is"/"There are": Vale's `Kyberforge.SentenceOpenerThereIs` alert flags this directly — report it without re-deriving by judgment
|
||||
| Dimension | Read |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body, delegation, comment-discipline | `references/body-and-delegation.md` |
|
||||
|
||||
**Body/Frontmatter comments:**
|
||||
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.
|
||||
- Look for patterns like `# Optional. <long explanation>` or extensive inline guidance (more than 1–2 lines per field) that should be condensed or removed before shipping.
|
||||
- This mirrors skill-audit's body-discipline check but applies to template documentation in the frontmatter — template guidance belongs in development; agent-ready files should have minimal comments.
|
||||
Cite file and line number for every finding.
|
||||
|
||||
**Pair consistency (cross-file) — project/user scope only:**
|
||||
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
|
||||
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
|
||||
- **Does not apply at plugin/APM scope** — there is only one file, by design; do not raise a Pair Consistency finding there under any circumstance.
|
||||
## Step 4 — Report
|
||||
|
||||
**Unexpressable Claude-only behavior — plugin/APM scope only:**
|
||||
- Read the description and body. If either implies a need the vendor-neutral frontmatter can no longer express — tool restriction, `isolation`, `memory`, or another Claude-only behavior that a hand-authored CC file could have declared — flag it as a SUGGESTION, never a FAIL. This is a known upstream schema limitation (APM's agent primitive has no per-target compile integrator, so `tools:`/`isolation`/etc. can't be emitted safely to both CC and Copilot — ADR-0016), not an authoring mistake. The finding exists to give the author visibility into the gap, not to imply the schema can be made to do something it can't.
|
||||
- Example: a body that says "only use Read and Grep, never Edit" but the frontmatter has no `tools` field to enforce it — SUGGESTION, not FAIL.
|
||||
|
||||
## Step 3 — Report
|
||||
|
||||
Open with a coverage line. At project/user scope:
|
||||
Open with a coverage line naming every dimension checked. At project/user scope:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · comment-discipline · pair-consistency · provenance
|
||||
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
|
||||
```
|
||||
|
||||
At plugin/APM scope, omit `pair-consistency` — it does not apply when there is no pair:
|
||||
At plugin/APM scope, drop `pair-consistency` — there is no pair to check.
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · comment-discipline · provenance
|
||||
```
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
|
||||
|
||||
For each finding:
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
@@ -107,18 +86,4 @@ FAIL/SUGGESTION <finding> — file:line
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS · P info
|
||||
PASS (N suggestions)
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /agent-author to address findings.
|
||||
```
|
||||
|
||||
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run agent-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -4,4 +4,4 @@ level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This (skill|agent)\b'
|
||||
- '^This\b'
|
||||
|
||||
@@ -10,6 +10,10 @@ Additional documentation agents load on demand.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `description-quality.md` | Qualitative guide for borderline description findings — action-verb rules, specificity criteria, proactive-use caveat, length limits. |
|
||||
| `field-inventory.md` | Canonical list of valid CC and Copilot agent definition fields. Load when the script needs authoritative field lists for structural validation. |
|
||||
| `description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked contract, and the internal-mechanics FAIL. |
|
||||
| `body-and-delegation.md` | Rubric for the body, delegation and comment-discipline dimensions — the delegation FAIL, why agents take no body word gate, and what an agent body is for. |
|
||||
| `scope-plugin-apm.md` | Contract for a single vendor-neutral `.apm/agents/<name>.agent.md` file — allowlist, dimension routing, and the dimensions that do not apply. |
|
||||
| `scope-project-user.md` | Contract for a Claude Code / Copilot file pair — counterpart derivation, provider field rules, and pair consistency. |
|
||||
| `validation-scripts.md` | Loaded only when a Step 1 script fails or cannot run — scope-detection walk-up, manual fallback checks, and known script failures. |
|
||||
| `field-inventory.md` | Authoritative field lists, read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM allowlist. |
|
||||
| `sources.md` | Research provenance records for skill content. Load only when tracing the origin of a specific rule or field constraint. |
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Body, Delegation and Comment Discipline Reference
|
||||
|
||||
Upstream source: Claude Code subagent and plugin references, GitHub Copilot custom-agents
|
||||
configuration. House contract: ADR-0020, the context budget.
|
||||
|
||||
Read this when judging the **body**, **delegation** and **comment-discipline** dimensions.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## Agents take no body word gate
|
||||
|
||||
ADR-0020 gates a skill body at 600 words SUGGESTION / 900 FAIL and deliberately gates an agent body
|
||||
at nothing. The two are not the same construct: a skill body is loaded into the caller's live
|
||||
context and competes with the conversation already there, while an agent body *becomes* the system
|
||||
prompt of a fresh context that has nothing else in it. The rationale for the 900-word ceiling does
|
||||
not transfer, so:
|
||||
|
||||
- **Never report an agent body as too long on a word count.** There is no number to cite.
|
||||
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate.bats` pins its absence with a
|
||||
body far past 900 words that must still pass, and adding one would contradict the ADR.
|
||||
- The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which
|
||||
`validate.sh` already reports as a SUGGESTION because content past it is silently truncated.
|
||||
|
||||
Length is judged through the delegation check below instead, which is the defect a word count was
|
||||
standing in for anyway.
|
||||
|
||||
## The delegation check
|
||||
|
||||
A plugin-scope agent is a single `.apm/agents/<name>.agent.md` file with no sibling `references/`
|
||||
directory. It cannot progressively disclose to itself — it can only delegate to skills. So a
|
||||
procedure spelled out in an agent body that a skill the agent invokes already owns is not a
|
||||
shortcut: it is a second copy of that procedure, and the second copy drifts. This is the
|
||||
characteristic agent defect, the way a stale README row is the characteristic skill defect.
|
||||
|
||||
**An agent body that restates a procedure owned by a skill it can invoke is a FAIL.** The Fix is
|
||||
always the same shape: invoke `<skill>` instead.
|
||||
|
||||
How to apply it: for each procedural block in the body — a rule list, a numbered sequence, a
|
||||
constraint table — ask which skill owns that procedure. If the agent names that skill anywhere (its
|
||||
dispatch table, its routing prose, its frontmatter), the block is a restatement and the skill is
|
||||
already there to be invoked.
|
||||
|
||||
Worked example. The three `*-orchestrate` agents exist to compose domain skills — `git-orchestrate`
|
||||
(933 body words), `gitea-orchestrate` (1,199) and `apm-orchestrate` (1,080) — so any step they
|
||||
spell out that the composed skill already owns is the defect. `git-orchestrate:24-31` carries a
|
||||
"Hard rules" list (Conventional Commits types, atomic commits, never commit secrets, git trailers)
|
||||
that `git-commits` owns and that `git-orchestrate:44` routes to by name; `:39` concedes the point
|
||||
outright, noting the sub-skills "carry their own local copies of these rules". Two copies, one
|
||||
authority, and nothing keeping them in step.
|
||||
|
||||
What is **not** a finding under this rule, because no skill owns it:
|
||||
|
||||
- The dispatch table itself — which operation routes to which skill.
|
||||
- Safety gates the agent enforces before dispatching, and refusals it makes on its own authority.
|
||||
- The input contract and the structured output the agent's caller consumes.
|
||||
- Session state the agent carries across skill invocations.
|
||||
|
||||
## What the body is for
|
||||
|
||||
Include what the fresh context lacks:
|
||||
|
||||
- A direct role instruction opening the prompt: `You are a [role]. When invoked, [action].`
|
||||
- One bounded job, stated so the agent knows what it must refuse.
|
||||
- The dispatch, gates, inputs and outputs listed above.
|
||||
- Non-obvious environment facts and project-specific conventions it cannot infer.
|
||||
- One default per decision point with one escape hatch.
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Comment discipline
|
||||
|
||||
Inspect every comment block in the YAML frontmatter and apply the core test to each: *would the
|
||||
agent get this wrong without this comment?* Template scaffolding — `# Optional. <long
|
||||
explanation>`, more than a line or two of inline guidance per field — belongs to development, not
|
||||
to a shipped file. At plugin/APM scope the stakes are higher than tidiness: `apm compile` copies
|
||||
frontmatter verbatim to every target, `<!-- ... -->` is not valid YAML, and `validate.sh` FAILs a
|
||||
frontmatter block that still contains one.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- The body restates a procedure owned by a skill the agent can invoke — Fix: invoke `<skill>`
|
||||
instead
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- Frontmatter comments are template scaffolding rather than instruction, or are HTML comments at
|
||||
plugin/APM scope
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- The body does not open with a direct role instruction
|
||||
- The job the agent describes is unbounded, or bounded only implicitly
|
||||
- A rationale is missing from a rule the agent is expected to enforce — present but unexplained
|
||||
- Comments are useful but verbose enough to bury the field they annotate
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
@@ -9,41 +8,120 @@ source_keys:
|
||||
|
||||
# Agent Description Quality Reference
|
||||
|
||||
Load this file when a description finding is borderline and you need to make a precise call.
|
||||
Upstream source: Claude Code subagent reference, GitHub Copilot custom-agents configuration.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than either
|
||||
platform's schema rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## Action-verb opening
|
||||
## Why the description is the expensive part
|
||||
|
||||
The description must open with an imperative or present-tense verb that describes what the agent does ("Reviews...", "Audits...", "Generates...", "Analyzes..."). Avoid:
|
||||
- Noun phrases: "An agent that..." — no verb
|
||||
- "This agent..." or "Use this when..." — passive framing
|
||||
- "Helps with..." — too vague to be a clear verb
|
||||
At startup an agent loads only the `name` and `description` of every installed skill and agent.
|
||||
The body is never seen until the agent is invoked. The description therefore carries the entire
|
||||
triggering burden **and** is paid for in every session, whether the agent fires or not.
|
||||
|
||||
**Borderline call:** "Validates and reviews..." is acceptable — two verbs is fine if both are specific. "Assists in reviewing..." is not — "assists" is vague filler.
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the caller takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## Specificity of trigger condition
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
The description must state what specifically triggers the agent. Generic phrasing fails:
|
||||
- Too vague: "when the user needs help with agents"
|
||||
- Acceptable: "when the user says 'audit this agent', 'check if my agent follows best practices', or wants to know if an agent pair is ready to ship"
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
Include indirect triggers: "even if they don't use the word 'audit'" or "even if the user doesn't phrase it as a review request". If the agent should activate on a recognisable user goal (not just literal keyword matches), name that goal.
|
||||
- **`disable-model-invocation: true` or `user-invocable: false`** — the agent is hand-invoked. Its
|
||||
description is never matched against user intent, so it is not a routing string. It carries **one
|
||||
plain human-facing sentence** stating what the agent does. Audit it for that and nothing else.
|
||||
Reporting a missing trigger clause, a missing boundary clause or absent indirect triggers on a
|
||||
hand-invoked agent is a wrong finding, not a strict one. Both fields are Copilot-only and neither
|
||||
is on the vendor-neutral APM allowlist, so this case arises in a Copilot `.agent.md` at
|
||||
project/user scope and nowhere else. Its Claude Code counterpart has no equivalent field and stays
|
||||
model-invoked, so the two halves of the pair carrying differently shaped descriptions is expected
|
||||
there rather than a pair-consistency finding.
|
||||
- **No such flag** — the agent is model-invoked and the rest of this file applies.
|
||||
|
||||
**Borderline call:** If the description covers direct triggers but omits common indirect phrasings that a user would plausibly use, mark as SUGGESTION (not FAIL) — the agent still activates, just less reliably.
|
||||
## The three-part shape
|
||||
|
||||
## `Use proactively`
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
For CC files: including "Use proactively" signals the CC runtime to offer the agent unprompted when conditions are met. This is CC-specific — use it when the agent should activate without an explicit user request.
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This agent ...` —
|
||||
the caller is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source.
|
||||
|
||||
For Copilot files: this phrase has no effect. Use `user-invocable: false` / `disable-model-invocation: true` for equivalent Copilot behavior. Flag `Use proactively` in a Copilot description as a SUGGESTION (not FAIL) — it causes no harm, just has no effect.
|
||||
Everything else belongs in the body or in the plugin's `README.md`.
|
||||
|
||||
## Length and hard limits
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
- CC agent descriptions: no documented character limit, but keep under 500 characters to avoid truncation in UI contexts.
|
||||
- Copilot agent descriptions: no separate documented limit, but the overall 30,000-character body limit applies to the full file.
|
||||
- Skill descriptions (SKILL.md): hard 1024-character limit enforced by the platform.
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on an agent whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Do not use when
|
||||
## Near-miss exclusions
|
||||
|
||||
Include a "Do not use when..." clause only if a near-miss agent or skill exists that could steal activations. Omitting it is not a finding. Including it is correct when there is a real confusion risk (e.g., `/agent-audit` vs `/skill-audit`).
|
||||
Add a boundary clause only where a sibling skill or agent could plausibly steal the activation. Use
|
||||
strong near-misses — queries that share keywords but need something different — not weak ones. One
|
||||
boundary clause per genuine near-miss; a list of four is enumeration wearing a boundary's clothes.
|
||||
|
||||
**Borderline call:** If the "Do not use when" clause is present but the exclusion described is already obvious from context, mark as SUGGESTION to tighten or remove — not FAIL.
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# FAIL — a noun-phrase opener rather than a trigger, capability enumeration in
|
||||
# place of one capability clause, and no boundary clause at all, preloaded into
|
||||
# every session forever. (The live git-orchestrate description, 254 chars.)
|
||||
description: Orchestrates git workflow operations for other agents. Invoke when a
|
||||
caller needs a multi-step or destructive git operation (rebase, force-push, branch
|
||||
deletion) coordinated across domain skills with safety gates, session context, and
|
||||
structured results.
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The operation list and the
|
||||
# safety-gate mechanics are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when an agent caller needs a multi-step or destructive git operation
|
||||
dispatched and safety-gated. Not conversational git help -> git-workflow.
|
||||
```
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut. Agent
|
||||
descriptions have no platform-documented ceiling of their own — unlike a skill's 1,024-character
|
||||
spec limit, the 400-character house ceiling is the only hard limit there is, so do not go looking
|
||||
for a backstop behind it.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared agent", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This agent ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`. There is no action-verb rule
|
||||
here and never was a defensible one: an `Orchestrates ...` or `Audits ...` opener is a catalogue
|
||||
entry, not a trigger.
|
||||
- **Vague capabilities** ("helps with agents" where "audits an agent definition pair" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
|
||||
file in the authoring source. No script checks this for an agent file — `validate.sh` resolves
|
||||
boundary targets for skills only, so resolve the name yourself against `plugins/*/.apm/skills/`
|
||||
and `plugins/*/.apm/agents/`.
|
||||
- **`Use proactively` in a Copilot or vendor-neutral description.**
|
||||
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
|
||||
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Plugin/APM Scope Contract
|
||||
|
||||
Read this when the agent file sits at `<package>/.apm/agents/<name>.agent.md` — a single
|
||||
vendor-neutral file inside an APM package, with no counterpart anywhere.
|
||||
|
||||
## What is different here
|
||||
|
||||
`apm compile` copies an agent's frontmatter **verbatim** to every target harness. There is no
|
||||
per-target integrator to reconcile a Claude-Code-only field with a Copilot-only one, so the file
|
||||
cannot carry either (ADR-0016). That single fact drives everything below.
|
||||
|
||||
## Frontmatter allowlist
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `references/field-inventory.md`. Read
|
||||
them from there. Do not recite the list in a finding, do not work from memory, and do not trust any
|
||||
restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it
|
||||
has changed before, and `validate.sh` parses that same section at load time, so a recitation is a
|
||||
copy that can disagree with the check the agent just ran.
|
||||
|
||||
`field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
|
||||
is not. Read that note before arguing with a finding about it.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at this scope:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| any frontmatter key outside the allowlist; body over the 30,000-character Copilot limit | Provider safety |
|
||||
| everything else — missing or malformed field, `name` not matching the filename stem, empty body, absent frontmatter, template HTML comments, description length | Structure |
|
||||
| — | Pair consistency never applies |
|
||||
|
||||
**Provider safety means something else here.** At project/user scope it asks whether a field leaked
|
||||
across the Claude Code / Copilot boundary. At this scope there is no boundary and no pair: it asks
|
||||
whether every field survives a verbatim copy to *every* target. Report it in those terms — a
|
||||
finding phrased as "CC-only field in a Copilot file" is the wrong finding here.
|
||||
|
||||
**Pair consistency never applies.** There is one file by design. `validate.sh` never emits a
|
||||
missing-counterpart FAIL at this scope, and neither do you, under any circumstance. Drop
|
||||
`pair-consistency` from the Step 4 coverage line rather than reporting it clean.
|
||||
|
||||
## Behaviour the schema cannot express
|
||||
|
||||
Read the description and body. If either implies a need the vendor-neutral frontmatter can no
|
||||
longer express — a tool restriction, `isolation`, `memory`, or another Claude-only behaviour a
|
||||
hand-authored CC file could have declared — flag it as a **SUGGESTION, never a FAIL**. This is a
|
||||
known upstream schema limitation (ADR-0016), not an authoring mistake, and the finding exists to
|
||||
give the author visibility into the gap rather than to imply the schema can be made to close it.
|
||||
|
||||
Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it.
|
||||
A denylist-shaped restriction is the available half of that — see `field-inventory.md`.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Project and User Scope Contract
|
||||
|
||||
Read this when the agent file is not under `.apm/agents/` — a Claude Code `.md` and Copilot CLI
|
||||
`.agent.md` **pair**, at project scope (`<repo>/.claude/agents/` and `<repo>/.github/agents/`) or
|
||||
user scope (`~/.claude/agents/` and `~/.copilot/agents/`). `validate.sh` derives the counterpart
|
||||
from whichever half it was handed; audit both.
|
||||
|
||||
## The pair is a house convention
|
||||
|
||||
Neither platform requires a counterpart file. The pair is a kyberforge convention (ADR-0005), so a
|
||||
missing counterpart is a FAIL against **this repo's** convention and must be labelled that way in
|
||||
the finding, not presented as a platform spec failure.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at these scopes:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| a Claude-Code-only field in the Copilot file, a Copilot-only field in the CC file, a tool the runtime withholds from subagents, body over the 30,000-character Copilot limit | Provider safety |
|
||||
| counterpart file not found | Pair consistency |
|
||||
| everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure |
|
||||
|
||||
The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of
|
||||
`references/field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
|
||||
those same sections, so any restatement is a copy that can disagree with the check (ADR-0009).
|
||||
|
||||
## Field and naming rules that differ by provider
|
||||
|
||||
- `name` must match the filename stem in a **Copilot CLI** `.agent.md`. Claude Code imposes no such
|
||||
rule, so a CC file whose `name` differs from its filename is not a finding.
|
||||
- A Copilot **cloud/IDE** agent — one under `.github/copilot/agents/` — may omit `name` entirely.
|
||||
If it carries one, it still has to be kebab-case.
|
||||
- `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent
|
||||
unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags
|
||||
it. The Copilot equivalent is `disable-model-invocation` / `user-invocable`, which changes the
|
||||
description contract entirely — see `references/description-quality.md`, Step 0.
|
||||
|
||||
## Pair consistency
|
||||
|
||||
Check that:
|
||||
|
||||
- Both files exist.
|
||||
- Both system prompt bodies are non-empty (`validate.sh` covers this; do it by hand only when the
|
||||
script could not run).
|
||||
- The two files describe the **same job**. Divergent capability claims across the pair mean one
|
||||
half was edited and the other was not, which is the defect this dimension exists to catch.
|
||||
- Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is
|
||||
the Step 0 case in `references/description-quality.md`, not a pair-consistency finding.
|
||||
|
||||
Keep `pair-consistency` in the Step 4 coverage line at these scopes.
|
||||
@@ -14,7 +14,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
@@ -22,7 +22,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
@@ -30,7 +30,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
@@ -38,7 +38,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
@@ -46,7 +46,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Validation Scripts Reference
|
||||
|
||||
Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting.
|
||||
Nothing here is needed on a clean run.
|
||||
|
||||
## Report the gap, do not guess
|
||||
|
||||
If a script cannot run at all — Bash denied, `python3` unavailable, `vale` not installed — say so
|
||||
as an **INFO** finding naming the script and the missing dependency, then fall back to the manual
|
||||
checks below. An INFO never changes PASS/FAIL. Silently omitting the dimension a script would have
|
||||
covered reports a clean audit that checked less than it claims to have checked.
|
||||
|
||||
## How the scripts detect scope
|
||||
|
||||
`validate.sh` and `validate-provenance.sh` walk up from the agent file's directory and stop at the
|
||||
first of these:
|
||||
|
||||
1. An `apm.yml` carrying a top-level `type: instructions|skill|hybrid|prompts` line — **plugin/APM
|
||||
scope**, and that directory is the package root. An `apm.yml` with no `type:` is a
|
||||
marketplace-only manifest: skip it and keep walking.
|
||||
2. `$HOME` — **user scope**, checked before `.git` so a dotfiles-managed home directory that is its
|
||||
own repo cannot shadow it.
|
||||
3. A `.git` directory or file — **project scope**.
|
||||
4. The filesystem root — **project scope**.
|
||||
|
||||
`plugin.json` and `.claude-plugin/plugin.json` are not scope signals. A directory holding only a
|
||||
`plugin.json` and no `apm.yml` falls through to project or user scope.
|
||||
|
||||
`validate-provenance.sh` exits 0 silently when that walk does not land on a package root, and again
|
||||
when the package has no provenance data. Silence from it is a pass, not a skip you need to
|
||||
investigate.
|
||||
|
||||
## Manual fallback
|
||||
|
||||
**Every scope:** required fields present (`name`, `description`, non-empty body); `name` is
|
||||
kebab-case; no `FILL IN:` placeholders in the description or body; the description at or under 400
|
||||
characters measured on the folded YAML value.
|
||||
|
||||
**Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter;
|
||||
no frontmatter key outside the `apm-agent-allowlist` section of `references/field-inventory.md` —
|
||||
open that file, do not work from memory.
|
||||
|
||||
**Project/user scope:** the counterpart file exists; `name` matches the filename stem in the
|
||||
Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in
|
||||
the Copilot file and none from `copilot-only-fields` in the CC file, both read from
|
||||
`references/field-inventory.md`.
|
||||
|
||||
## Script-specific failures
|
||||
|
||||
- **`Error: field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
|
||||
`references/field-inventory.md` at load time and refuses to run without it, rather than falling
|
||||
back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not
|
||||
work around it.
|
||||
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full
|
||||
Step 3 judgment for the dimensions it would have covered. The `Kyberforge` style is scoped to
|
||||
`**/agents/*.md` and `**/*.agent.md`, and `KyberforgeCopilot` to `**/*.agent.md` alone — a file
|
||||
outside those globs is silently not linted.
|
||||
- **`E100 Runtime error ... does not exist` (exit 2) from `vale-wrap.sh`.** An explicit relative
|
||||
`--config` was passed. Pass none: the wrapper locates its own `assets/vale/.vale.ini` from its
|
||||
own path. Do not read this exit code as vale being unavailable.
|
||||
- **A path argument that does not exist is a hard error** in `vale-wrap.sh`, deliberately: bare
|
||||
`vale` would fall back to reading stdin and print a clean-looking `0 errors ... in stdin`, which
|
||||
the `0 files` guard above does not catch.
|
||||
@@ -90,6 +90,23 @@ SUBAGENT_UNAVAILABLE_TOOLS = {
|
||||
# Copilot body length limit (chars) — content beyond this is silently truncated
|
||||
COPILOT_BODY_LIMIT = 30000
|
||||
|
||||
# ADR-0020 description budget. An agent's name + description is preloaded into
|
||||
# every session exactly like a skill's, so agents take the SAME description
|
||||
# gates. These two constants are DUPLICATED from scripts/skill-size-check.sh
|
||||
# and skill-audit/scripts/validate.sh rather than shared from one file: a
|
||||
# cache-installed plugin's scripts cannot read files outside their own plugin
|
||||
# directory, so there is no single source to share (same rationale as
|
||||
# vale-wrap.sh's per-plugin duplication). tests/test-skill-size-check.sh
|
||||
# asserts all copies agree, so drift fails CI rather than silently diverging.
|
||||
#
|
||||
# Agents deliberately take NO body word gate, and adding one here would
|
||||
# contradict ADR-0020: a skill body is loaded into the caller's context and
|
||||
# competes with the live conversation, while an agent body becomes the system
|
||||
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
|
||||
# not transfer. Agent body length falls out of the delegation rule instead.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
|
||||
# --- Helpers (shared by every scope) ---
|
||||
failed = False
|
||||
suggestions = []
|
||||
@@ -122,6 +139,80 @@ def get_frontmatter_keys(fm):
|
||||
keys.add(m.group(1))
|
||||
return keys
|
||||
|
||||
def normalize_scalar(value):
|
||||
return re.sub(r'\s+', ' ', value).strip()
|
||||
|
||||
def fold_description_fallback(fm_text):
|
||||
"""Resolve `description:` without PyYAML.
|
||||
|
||||
Not a YAML parser — it recognises exactly the shapes agent frontmatter
|
||||
uses: an inline scalar (optionally quoted, optionally continued on
|
||||
following indented lines) and a `>`/`|` block scalar with optional
|
||||
indentation and chomping indicators.
|
||||
"""
|
||||
lines = fm_text.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^description:[ \t]*(.*)$', line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1).strip()
|
||||
block = bool(re.match(r'^[>|][0-9]*[-+]?$|^[>|][-+]?[0-9]*$', head))
|
||||
parts = [] if block else [head]
|
||||
for nxt in lines[i + 1:]:
|
||||
if not nxt.strip():
|
||||
parts.append('')
|
||||
continue
|
||||
if not re.match(r'^[ \t]', nxt):
|
||||
break
|
||||
parts.append(nxt.strip())
|
||||
value = ' '.join(parts)
|
||||
if not block:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in '"\'':
|
||||
value = value[1:-1]
|
||||
return value
|
||||
return ''
|
||||
|
||||
def description_value(fm):
|
||||
"""The description VALUE with YAML folding resolved.
|
||||
|
||||
extract_field() reads one raw line, which is the right shape for the
|
||||
presence and placeholder checks but the wrong one for a length gate: a
|
||||
`>`-folded description measured off its first raw line is not the value the
|
||||
host preloads. Parse instead of regexing the raw text.
|
||||
"""
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(fm)
|
||||
if isinstance(data, dict):
|
||||
value = data.get('description')
|
||||
if isinstance(value, str):
|
||||
return normalize_scalar(value)
|
||||
if value is not None:
|
||||
return normalize_scalar(str(value))
|
||||
return ''
|
||||
except Exception:
|
||||
pass
|
||||
return normalize_scalar(fold_description_fallback(fm))
|
||||
|
||||
def check_description_budget(fm, local_fname):
|
||||
"""ADR-0020 description gates — identical for every scope."""
|
||||
value = description_value(fm)
|
||||
if not value:
|
||||
return
|
||||
dlen = len(value)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body — {local_fname}")
|
||||
elif dlen > DESC_SUGGEST_CHARS:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers "
|
||||
f"— {local_fname}")
|
||||
|
||||
def extract_tools_list(fm):
|
||||
"""Extract tool names from the tools frontmatter field (space or comma separated)."""
|
||||
val = extract_field(fm, 'tools')
|
||||
@@ -273,6 +364,7 @@ def check_apm_agent_file(fpath, allowlist, stem):
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(desc_val):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
check_description_budget(fm, local_fname)
|
||||
|
||||
# body — required, non-empty, no placeholder; same Copilot truncation risk
|
||||
# applies since this file compiles verbatim into a real Copilot file downstream.
|
||||
@@ -346,6 +438,7 @@ def check_file(fpath, file_provider):
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(desc_val):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
check_description_budget(fm, local_fname)
|
||||
|
||||
# body
|
||||
if not body.strip():
|
||||
|
||||
@@ -30,10 +30,15 @@ bash scripts/new-agent.sh security-reviewer ~
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `SKILL.md` | Skill instructions for agents — gotchas, the create/improve dispatch table, the scope dispatch table, the shared gates, and validation/close |
|
||||
| `scripts/new-agent.sh` | Scaffolds agent definition file(s) from templates — a single `.apm/agents/<name>.agent.md` at plugin/APM scope, or a Claude Code + Copilot CLI pair at project/user scope |
|
||||
| `references/deployment-modes.md` | Plugin/APM vs project vs user scope: restrictions, scoped identifiers, path conventions |
|
||||
| `references/scripts.md` | Conventions for new-agent.sh and any future scripts: contract, template variables, file placement, error messages |
|
||||
| `references/create.md` | Create flow: prerequisites, scaffold and scope walk-up, what to fill in, package-root `sources.md` |
|
||||
| `references/improve.md` | Improve flow: signal verification, root-cause grouping, generalizing, delegation over growth, ADR-0020 retrofit |
|
||||
| `references/contract.md` | Description and body contract: three-part description shape, 250/400 tiers, delegation rule in place of a body word gate, invocation axis |
|
||||
| `references/plugin-scope.md` | Plugin/APM scope field rules for the single vendor-neutral file, plus its pre-audit checklist |
|
||||
| `references/project-user-scope.md` | Project/user scope field rules for the Claude Code + Copilot pair, both Copilot formats, plus its pre-audit checklist |
|
||||
| `references/deployment-modes.md` | Scope hierarchy and precedence, scoped identifiers, cache isolation, path conventions |
|
||||
| `references/scripts.md` | Conventions for new-agent.sh and the templates it copies: contract, template variables, file placement, error messages |
|
||||
| `references/sources.md` | Research provenance — sources that informed this skill |
|
||||
| `assets/templates/claude-code.md` | Annotated Claude Code agent definition template (project/user scope) |
|
||||
| `assets/templates/copilot.agent.md.template` | Annotated Copilot CLI agent definition template (project/user scope) |
|
||||
|
||||
@@ -1,18 +1,9 @@
|
||||
---
|
||||
name: agent-author
|
||||
description: >
|
||||
Use when the user wants to create a new agent definition file from scratch
|
||||
("write an agent for X", "build a subagent that does Y", "create an agent
|
||||
definition for Z"), or improve an existing one. Handles agent definitions at
|
||||
plugin/APM, project, and user scope. Project and user scope always generate
|
||||
a Claude Code (`.md`) + Copilot CLI (`.agent.md`) file pair in one pass;
|
||||
plugin/APM scope generates a single vendor-neutral `.apm/agents/<name>.agent.md`
|
||||
file instead (no per-target Claude Code / Copilot split). Also use when the
|
||||
user provides inline feedback about an agent's behavior and wants it applied,
|
||||
or when a grill session has produced findings the user wants acted on — even
|
||||
if they don't say "improve" explicitly. Do not use for read-only review —
|
||||
examine agent files manually or run a grill session to generate improvement
|
||||
signals. Do not use to author skills — use /skill-author instead.
|
||||
Use when the user wants to create a new agent definition file from scratch, or
|
||||
apply grill findings, audit findings, or inline feedback to an existing one.
|
||||
Not read-only review -> `agent-audit`. Not skills -> `skill-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -20,240 +11,52 @@ metadata:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
- github-cli-plugin-reference
|
||||
- github-plugins-creating
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- At plugin/APM scope, bump the resolved package's `apm.yml` `version` after every change — minor for a new agent, patch for a fix. Consumers compare this version to detect updates; skipping it hides the change.
|
||||
- At plugin/APM scope, `tools` and all Claude-only fields (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) are omitted entirely, not merely restricted (ADR-0016: `apm compile` copies frontmatter verbatim to both harnesses with no per-target integrator, so a harness-specific value is wrong on at least one). Only project/user scope supports these fields.
|
||||
- `disallowedTools` is the one exception, on **shape**, not favouritism. `tools` is an *allowlist* whose vocabulary differs per harness (Claude tool names vs Copilot's `execute`/`read`/`edit`/`search`/`agent`/`web`), so verbatim copy makes one value wrong on one target. A *denylist* cannot fail that way: an unrecognised name denies nothing, so the worst case is a missing fence, never a wrong grant. Claude Code honours it for plugin subagents — `docs/research/docs/claude-code-plugins/agent-definition.md:99` lists the three fields plugin agents ignore (`hooks`, `mcpServers`, `permissionMode`) and this is not one. Write it on every read-only plugin-scope agent (ADR-0016's 2026-08-14 amendment).
|
||||
- That fence is partial: it denies only the tools it names. It does not deny `Bash`, which a plugin-scope agent with no `tools` inherits, so a shell redirect still writes. Say the agent is read-only in the body too.
|
||||
- An `apm.yml` with no top-level `type:` field is a marketplace-only manifest, not a package root — the walk-up skips it and keeps going.
|
||||
- `AskUserQuestion`, `EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup`, and `WaitForMcpServers` are never available to any subagent regardless of the `tools` field. Exception: `ExitPlanMode` is available when the parent session runs in `permissionMode: plan`.
|
||||
- Duplicate `name` values in the same scope: Claude Code silently discards one without warning. Always verify uniqueness before shipping.
|
||||
- Plugin agents in subdirectories get scoped identifiers (`plugin:folder:name`) — keep agents flat in `agents/` to avoid this. Applies to project/user-scope Claude Code agents only.
|
||||
- Copilot CLI agent files **must** use the `.agent.md` extension — a plain `.md` file isn't picked up. The plugin/APM-scope single file also ends in `.agent.md` by convention, but it's vendor-neutral, not Copilot-only — it compiles to Claude Code too.
|
||||
- Copilot has no `permissionMode`, `maxTurns`, `isolation`, or `memory` fields — do not include them in project/user-scope Copilot files.
|
||||
- `model` resolution order for Claude Code: `CLAUDE_CODE_SUBAGENT_MODEL` env var → per-invocation parameter → frontmatter `model` → main session model. The frontmatter value is a low-priority default, not a guarantee.
|
||||
- At plugin/APM scope `tools` and every Claude-only field are omitted entirely, not merely ignored: `apm compile` copies frontmatter verbatim to both harnesses, so fencing a read-only agent with `tools:` is wrong on one of them. `disallowedTools` is the one restriction that survives (ADR-0016).
|
||||
- That fence is partial. It denies only the tools it names, never `Bash`, which a plugin-scope agent inherits — a shell redirect still writes. State the read-only boundary in the body too.
|
||||
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is an `agent-audit` FAIL.
|
||||
- Duplicate `name` values in one scope: Claude Code discards one silently. Verify uniqueness before shipping.
|
||||
|
||||
## Route
|
||||
## Step 1 — Dispatch
|
||||
|
||||
If the destination resolves to plugin/APM scope (scope detection in Step 1 finds a `type:`-bearing `apm.yml` at or above the root), read `references/deployment-modes.md`.
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| No agent file at the target path(s) | Create | `references/create.md` |
|
||||
| A file exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| A file exists, no signals | Stop and ask | — |
|
||||
|
||||
Determine which flow before touching the filesystem:
|
||||
Signals: grill output, `agent-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
|
||||
- **Neither `<name>.md` nor `<name>.agent.md` exist at the target paths** → follow **Creating a new agent**
|
||||
- **At least one file exists + improvement signals present** → follow **Improving an existing agent**
|
||||
- **At least one file exists + no signals** → ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
Read only the reference for the resolved flow. Capture `git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
Signals: grill session output, inline user feedback, session context describing what went wrong.
|
||||
## Step 2 — Scope
|
||||
|
||||
## Creating a new agent
|
||||
Scope decides which fields exist, so resolve it first. `scripts/new-agent.sh` walks up for a `type:`-bearing `apm.yml` and prints the scope it chose — read that output.
|
||||
|
||||
### Prerequisites
|
||||
| Resolved scope | Emits | Read |
|
||||
|---|---|---|
|
||||
| plugin/APM | one vendor-neutral `.apm/agents/<name>.agent.md` | `references/plugin-scope.md` |
|
||||
| project or user | a Claude Code `.md` + Copilot `.agent.md` pair | `references/project-user-scope.md` |
|
||||
|
||||
Before touching the filesystem, confirm you have:
|
||||
- [ ] Agent name (kebab-case, e.g. `code-reviewer`)
|
||||
- [ ] Root directory (a path inside a package for plugin/APM scope, project root, or `~` for user scope)
|
||||
- [ ] Agent purpose — one sentence describing the task this agent handles
|
||||
- [ ] Trigger condition — when should the runtime delegate to this agent?
|
||||
Read only the file for the resolved scope; the other describes fields this run cannot use. If precedence, cache isolation or path conventions matter, read `references/deployment-modes.md`.
|
||||
|
||||
If any are missing, stop and ask before proceeding. Then capture `git log --oneline -1` before touching the filesystem — Step 5 needs it to verify a real commit landed.
|
||||
## Step 3 — Contract
|
||||
|
||||
Verify `kyberforge:agent-audit` is available — it ships with the kyberforge plugin and is co-installed with this skill. If unavailable, stop and tell the user to install the kyberforge plugin before continuing.
|
||||
Before writing or editing a `description`, or restructuring a body, read `references/contract.md` — the three-part shape, banned content, the delegation rule and the body pattern.
|
||||
|
||||
### Step 1 — Scaffold
|
||||
Gates `agent-audit` enforces at every scope:
|
||||
|
||||
Run the scaffold script with the agent name and root directory:
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <name>` that resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only: an agent's `name` and `description` is preloaded into every session exactly as a skill's is.
|
||||
- **Body** — no word gate, and a delegation check in its place: name the skill to invoke rather than restating what it does.
|
||||
- **Invocation** — decide whether the agent is model-delegated or reached only by name. Only Copilot's cloud/IDE format expresses that in frontmatter (`disable-model-invocation`, `user-invocable`).
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh <name> <root>
|
||||
```
|
||||
## Step 4 — Validate and close
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
|
||||
bash scripts/new-agent.sh deploy-assistant .
|
||||
bash scripts/new-agent.sh security-reviewer ~
|
||||
```
|
||||
Invoke `agent-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
|
||||
|
||||
**Scope detection (script handles this automatically).** The script walks up from `<root>` for a package boundary — same shape `agent-audit`'s `validate.sh` uses:
|
||||
- Nearest ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) → **plugin/APM scope** → `<package-root>/.apm/agents/<name>.agent.md` (single vendor-neutral file). A `type:`-less `apm.yml` is marketplace-only — skipped, walk continues upward.
|
||||
- No such `apm.yml`, `<root>` is a project directory → **project scope** (unchanged) → `<root>/.claude/agents/<name>.md` + `<root>/.github/agents/<name>.agent.md`
|
||||
- `<root>` is exactly `~` (checked directly, no walk-up) → **user scope** (unchanged) → `~/.claude/agents/<name>.md` + `~/.copilot/agents/<name>.agent.md`
|
||||
At plugin/APM scope bump the resolved package's `apm.yml` `version` — **minor** on create, **patch** on improve — because consumers compare it to detect updates. Project and user scope have no manifest.
|
||||
|
||||
A bare `plugin.json` with no `apm.yml` no longer signals plugin scope — that path is fully replaced, not dual-mode; it falls through to project scope.
|
||||
|
||||
The script is file-by-file no-op — it skips any file that already exists.
|
||||
|
||||
### Step 2 — Fill in the agent file(s)
|
||||
|
||||
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Its frontmatter allowlist is the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, read from there as data: `name`, `description`, `model`, `source_keys` (provenance metadata, not a runtime field — see the template), and `disallowedTools` for a read-only agent. Never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill those in plus the system prompt body per the guidance below; the rest of this step's field-by-field guidance is project/user scope only. Skip Step 3 and go to Step 4.
|
||||
|
||||
**At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file.
|
||||
|
||||
Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Remove all template documentation comments from the YAML frontmatter after filling in required fields** — these are marked with `<!--` and `-->` and must be deleted before shipping.
|
||||
|
||||
**`name`** — lowercase letters and hyphens only. Must be unique within the scope.
|
||||
|
||||
**`description`** — the most important field for autonomous delegation:
|
||||
- Start with an action verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
- If this agent should trigger without explicit user direction, include "Use proactively" in the description
|
||||
- Specific about the triggering condition and expertise domain
|
||||
- Under 300 characters preferred
|
||||
|
||||
**`tools`** (project/user scope only — never at plugin/APM scope) — restrict to what the agent actually needs. Omit to inherit all tools. Use `Agent(type1,type2)` to limit which subagent types this agent can spawn; omit `Agent` entirely to prevent spawning.
|
||||
|
||||
**`disallowedTools`** (all scopes, including plugin/APM) — denylist applied before `tools` and taking precedence over it; supports `mcp__<server>`, `mcp__<server>__*`, and `mcp__*` globs. `api-reference.md:40` types it `string / list` and `agent-definition.md:71` types it `string[]`, so a YAML list or a delimited string both work; this repo's plugin-scope agents use the comma-separated string (`disallowedTools: Edit, Write, NotebookEdit`) — match that.
|
||||
|
||||
**Optional fields worth considering (project/user scope only — never at plugin/APM scope, with the exception of `model`, which is allowed at every scope):**
|
||||
- `model`: set when this agent needs a different capability tier (`haiku` for fast tasks, `opus` for deep reasoning)
|
||||
- `maxTurns`: set a cap to prevent runaway agents on bounded tasks
|
||||
- `effort`: set to `low` for single-lookup tasks, `high` or above for deep reasoning or multi-file analysis — overrides session effort level; omit to inherit
|
||||
- `memory`: `user`, `project`, or `local` — only when cross-session state is genuinely needed
|
||||
- `isolation: worktree` — only when the agent modifies files and needs an isolated copy
|
||||
- `skills`: list of skill names preloaded at agent startup — different from the `source_keys` metadata field
|
||||
- `color`: UI color for the agent tile (`red`, `blue`, `green`, `yellow`, `purple`, `orange`, `pink`, `cyan`)
|
||||
- `initialPrompt`: auto-submitted as the first turn when this agent activates as the main session thread; only set when this agent is intended for main-thread activation
|
||||
- `background`: set `true` to force background execution
|
||||
|
||||
**`source_keys`** — top-level list of research source slugs that informed this agent. Add only when research sources were used (i.e. entries with `` `extracted` `` status are in context from a prior `/research` session). Each slug must match an H2 heading in `sources.md` — see Step 4 for where that file lives (plugin/APM scope only). Omit entirely when no research was used.
|
||||
|
||||
```yaml
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
```
|
||||
|
||||
**System prompt body** — write as a direct role instruction:
|
||||
- Open with: "You are a [role]. When invoked, [primary action]."
|
||||
- Cover: inputs expected, process steps, output format, error handling
|
||||
- One job per agent
|
||||
|
||||
### Step 3 — Fill in the Copilot agent file (project/user scope only)
|
||||
|
||||
Skip this step entirely at plugin/APM scope — there is no separate Copilot file there. The single `.apm/agents/<name>.agent.md` file from Step 2 already compiles to both Claude Code and Copilot CLI via `apm compile`.
|
||||
|
||||
**Two distinct Copilot agent formats** exist, with different paths and field sets. Choose one based on the deployment target:
|
||||
|
||||
**CLI format** (default — what the scaffold creates):
|
||||
- Path: `.github/agents/<name>.agent.md` (project) or `~/.copilot/agents/<name>.agent.md` (user)
|
||||
- Extension: **must be `.agent.md`**
|
||||
- Supported fields: `name` (required), `description` (required), `tools` (optional)
|
||||
- `tools` uses Copilot aliases: `execute` (shell), `read`, `edit`, `search`, `agent`, `web`
|
||||
- Body length limit: **30,000 characters** — content beyond this is silently truncated
|
||||
|
||||
**Cloud/IDE format** (use when targeting Copilot Chat in VS Code or GitHub.com):
|
||||
- Path: `.github/copilot/agents/<name>.md` (note: plain `.md`, different directory)
|
||||
- Additional fields available: `target` (`vscode`, `github-copilot`, or omit for both), `user-invocable` (set `false` to hide from manual invocation), `disable-model-invocation` (set `true` to require explicit user invocation), `mcp-servers` (MCP server config — processed by cloud runtime, ignored in VS Code)
|
||||
- Body length limit: **30,000 characters** — silently truncated
|
||||
|
||||
**Do not include Claude Code-only fields in either format**: `maxTurns`, `isolation`, `memory`, `permissionMode`, `effort`, `hooks`, `mcpServers`.
|
||||
|
||||
**`source_keys`** — add the same top-level list as the CC file when research sources were used. Omit when no research was used.
|
||||
|
||||
**Delete the `<!-- -->` template comments from the frontmatter**, as in Step 2.
|
||||
|
||||
The system prompt body should match the Claude Code version — the agent's task definition is the same across providers.
|
||||
|
||||
### Step 4 — Populate or delete `sources.md` (plugin/APM scope only)
|
||||
|
||||
Skip at project/user scope. The file lives at the package root (alongside `apm.yml`), not inside `.apm/agents/` — otherwise tooling that scans that directory for agent definitions would treat it as an agent needing frontmatter (ADR-0010).
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
1. Filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, identify which agent file it contributed to.
|
||||
3. Write `sources.md` at the package root using the format below. Paths in `Contributing files:` are relative to the package root.
|
||||
|
||||
```markdown
|
||||
# Sources
|
||||
|
||||
## slug-name
|
||||
|
||||
- **URL:** <source URL>
|
||||
- **Research doc:** <path/to/research/sources.md relative to repo root>
|
||||
- **Description:** <what this source covers>
|
||||
- **Contributing files:** .apm/agents/<name>.agent.md
|
||||
- **Status:** `extracted`
|
||||
```
|
||||
|
||||
Each slug must match an H2 heading, and each slug must also appear in the `source_keys` list of the file listed under `Contributing files:`.
|
||||
|
||||
If no research sources are in context, delete `sources.md`.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Run this checklist before invoking the audit.
|
||||
|
||||
**Every file, at every scope:**
|
||||
- [ ] `name` present and kebab-case; `description` present
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] No `FILL IN:` placeholders and no `<!-- -->` template comments remain
|
||||
|
||||
**Plugin/APM scope — single file (`<name>.agent.md`):**
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
- [ ] Every frontmatter field is in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the single source of truth, read as data by `validate.sh`. As of 2026-08-14: `name`, `description`, `model`, `source_keys`, `disallowedTools`. Nothing else — in particular no `tools`
|
||||
- [ ] A read-only agent carries `disallowedTools` **and** says so in the body
|
||||
|
||||
**Project/user scope — Claude Code file (`<name>.md`):**
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
|
||||
**Project/user scope — Copilot CLI file (`<name>.agent.md`):**
|
||||
- [ ] File extension is `.agent.md` (not `.md`)
|
||||
- [ ] `name` matches the filename stem (e.g. `name: my-agent` in `my-agent.agent.md`)
|
||||
- [ ] No Claude Code-only fields (`maxTurns`, `isolation`, `memory`, `permissionMode`, `effort`, `hooks`, `mcpServers`)
|
||||
- [ ] Body does not exceed 30,000 characters
|
||||
|
||||
At plugin/APM scope, apply a **minor bump** to the resolved package's `apm.yml` `version` (single manifest, e.g. `1.0.4` → `1.1.0`).
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the created file(s) before closing — validates the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` for the new agent files — do not stop at staging. Then confirm `git log --oneline -1` differs from the hash captured before Step 1. A non-empty `git diff --stat` is not proof of completion: staged-but-uncommitted work is part of no commit and can be silently lost if the working tree is cleaned up. Only report the agent as done once the hash has actually changed.
|
||||
|
||||
## Improving an existing agent
|
||||
|
||||
### Step 1 — Verify inputs
|
||||
|
||||
Confirm the agent files exist and at least one improvement signal is present in the conversation or a referenced file.
|
||||
|
||||
If no signals: "This skill applies existing signals to an agent. For a blind review, examine the files manually or run a grill session first."
|
||||
|
||||
Verify `kyberforge:agent-audit` is available, as in the create flow's Prerequisites. Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
|
||||
**Partial state (project/user scope only)** — if one provider file exists but not the other, scaffold the missing one (`bash scripts/new-agent.sh <name> <root>`, file-by-file no-op) then continue. Doesn't apply at plugin/APM scope — single file, no partial-pair state.
|
||||
|
||||
### Step 2 — Gather and group signals
|
||||
|
||||
Read the current agent file(s). Collect all signals from the conversation.
|
||||
|
||||
Group by **root cause**, not symptom. One root cause → one fix.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- User feedback: agent keeps trying to push to remote
|
||||
- Session context: no scope boundary in system prompt
|
||||
→ Root cause: system prompt lacks git scope constraint → fix: add explicit boundary
|
||||
```
|
||||
|
||||
### Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state which root causes were identified, what evidence supports each, and which files will change. Then proceed — edits are reversible via git.
|
||||
|
||||
### Step 4 — Apply changes
|
||||
|
||||
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced. For a plugin/APM-scope single file, verify every field is still in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, and that an existing `disallowedTools` fence was not dropped by the edit.
|
||||
|
||||
If the edit adds or removes research-sourced content, update `source_keys` in the edited file(s) and the corresponding entry in `sources.md` per Create flow's Step 4.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Re-run the validation checklist from the create flow's Step 5 on any edited file.
|
||||
|
||||
At plugin/APM scope, apply a **patch bump** to the resolved package's `apm.yml` `version` (e.g. `1.0.4` → `1.0.5`).
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the edited file(s) to confirm no regressions — the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Exactly as in the create flow's Step 5, against the hash captured at Step 1: commit the changed files once the audit is clean, and only report the improvement as done once `git log --oneline -1` shows a different hash.
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` — do not stop at staging. Re-run `git log --oneline -1` and confirm the hash changed from Step 1's. A non-empty `git diff --stat` is not proof: staged-but-uncommitted work is part of no commit and is lost if the tree is cleaned up. Report done only once the hash has changed.
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
## templates/
|
||||
|
||||
Annotated agent definition templates copied by `scripts/new-agent.sh` when scaffolding a new agent.
|
||||
All three scaffold the `description` in the three-part ADR-0020 shape — a `Use when` trigger clause, at most one capability clause, and a boundary clause — rather than the deleted action-verb opener, and each carries a delegate-don't-restate note in the body.
|
||||
|
||||
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
|
||||
- **`copilot.agent.md.template`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
|
||||
|
||||
@@ -27,11 +27,17 @@
|
||||
name: AGENT_NAME
|
||||
<!-- Required. Lowercase letters and hyphens only. Must be unique within the scope. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. The primary signal for autonomous delegation.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Be specific about the triggering condition and expertise domain.
|
||||
Example: "Reviews pull request diffs for security issues. Use proactively after code changes." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. The primary signal for autonomous delegation, and preloaded into every
|
||||
session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was
|
||||
deleted. Add "Use proactively" only if the runtime should delegate here without
|
||||
the user naming this agent.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- model: sonnet
|
||||
Optional. Aliases: sonnet, opus, haiku, fable. Or full model ID.
|
||||
@@ -58,6 +64,11 @@ FILL IN: System prompt body. Write as a direct role instruction.
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -7,12 +7,17 @@ name: AGENT_NAME
|
||||
<!-- Required. Lowercase letters and hyphens only. Must be unique within the scope.
|
||||
Duplicate names are silently discarded — no warning is emitted. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. The primary signal for autonomous delegation.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Include "Use proactively" to trigger automatic invocation.
|
||||
Be specific about the triggering condition and domain.
|
||||
Example: "Reviews pull request diffs for security issues. Use proactively after code changes." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. The primary signal for autonomous delegation, and preloaded into every
|
||||
session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was
|
||||
deleted. Add "Use proactively" only if the runtime should delegate here without
|
||||
the user naming this agent.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- tools: Read, Bash, Grep
|
||||
Optional. Allowlist of tool names: a comma-separated string or a YAML list.
|
||||
@@ -78,6 +83,11 @@ FILL IN: System prompt body. Write as a direct role instruction.
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -12,10 +12,16 @@
|
||||
name: AGENT_NAME
|
||||
<!-- Required. Kebab-case identifier. Home-directory version wins on name collision. -->
|
||||
|
||||
description: FILL IN: Action-first description of what this agent does and when to invoke it.
|
||||
<!-- Required. Used by the runtime for automatic agent selection — quality matters.
|
||||
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
|
||||
Example: "Reviews pull request diffs for security issues." -->
|
||||
description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -> <name>.
|
||||
<!-- Required. Used by the runtime for automatic agent selection, and preloaded into
|
||||
every session whether or not this agent is ever used. Three parts, nothing else:
|
||||
a trigger clause opening "Use when", at most one capability clause, and a
|
||||
boundary clause naming a real sibling skill or agent.
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was deleted.
|
||||
Keep it identical in wording to the Claude Code half of the pair.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not general code review -> code-reviewer." -->
|
||||
|
||||
<!-- tools: ["read", "search", "edit"]
|
||||
Optional. Array of tool names. Omit = all available tools. [] = no tools.
|
||||
@@ -46,6 +52,11 @@ FILL IN: System prompt body. Should match the Claude Code version — the agent'
|
||||
|
||||
You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
FILL IN: What inputs does this agent expect? (files, context, parameters)
|
||||
|
||||
@@ -4,14 +4,52 @@ source_keys: []
|
||||
|
||||
# references/
|
||||
|
||||
## create.md
|
||||
|
||||
The create flow, loaded from SKILL.md Step 1 when no agent file exists at the target path.
|
||||
Covers: prerequisites, the scaffold script and its scope walk-up, what to fill in at every scope,
|
||||
and populating or deleting the package-root `sources.md`.
|
||||
|
||||
## improve.md
|
||||
|
||||
The improve flow, loaded from SKILL.md Step 1 when a file exists and at least one improvement
|
||||
signal is present. Covers: signal verification, partial-pair recovery, root-cause grouping,
|
||||
generalizing rather than patching, delegation over growth, and the ADR-0020 retrofit rule.
|
||||
|
||||
## contract.md
|
||||
|
||||
The description and body contract, loaded from SKILL.md Step 3 before any description is written
|
||||
or any body restructured. Covers: the three-part description shape, banned description content,
|
||||
boundary-target resolution, the 250/400 length tiers, the body role-instruction pattern, the
|
||||
delegation rule that replaces a body word gate, and the invocation axis.
|
||||
|
||||
## plugin-scope.md
|
||||
|
||||
Field rules and the pre-audit checklist for the single vendor-neutral `.apm/agents/<name>.agent.md`
|
||||
file. Loaded from SKILL.md Step 2 when the scaffold resolves plugin/APM scope.
|
||||
|
||||
## project-user-scope.md
|
||||
|
||||
Field rules and the pre-audit checklist for the Claude Code `.md` + Copilot `.agent.md` pair,
|
||||
including the two distinct Copilot formats. Loaded from SKILL.md Step 2 when the scaffold resolves
|
||||
project or user scope.
|
||||
|
||||
## deployment-modes.md
|
||||
|
||||
Agent scope hierarchy, precedence rules, and per-scope restrictions. Covers: which frontmatter fields a plugin/APM-scope agent may carry and the allowlist-vs-denylist shape rule that decides it (deferring to `agent-audit`'s `references/field-inventory.md` for the list itself), scoped identifiers for plugin subdirectory agents, cache isolation behaviour, and Copilot CLI path conventions. Loaded conditionally from SKILL.md when the destination is a plugin directory.
|
||||
Scope hierarchy and precedence, scoped identifiers for plugin subdirectory agents, cache isolation
|
||||
behaviour, and Copilot CLI path conventions. Loaded from SKILL.md Step 2 when precedence, paths or
|
||||
cache isolation matter to the run.
|
||||
|
||||
## scripts.md
|
||||
|
||||
Conventions for the `new-agent.sh` scaffold script and any future scripts added to this skill. Covers: what scripts should and should not do, file placement, error handling, template variable conventions, and the no-interactive-prompts rule.
|
||||
Conventions for the `new-agent.sh` scaffold script, the templates it copies, and any future script
|
||||
in this skill. Loaded from `create.md` Step 1 when the script or a template has to change. Covers:
|
||||
the no-interactive-prompts rule, structured output, idempotency, template variables, file
|
||||
placement, error messages, and the no-restated-field-roster rule that `tests/new-agent.bats`
|
||||
enforces.
|
||||
|
||||
## sources.md
|
||||
|
||||
Research provenance record for this skill. Lists the upstream research sources (claude-code-plugins and github-copilot-plugins research docs) that informed SKILL.md, the templates, and the deployment-modes reference. Used by `skill-audit` to validate the provenance chain.
|
||||
Research provenance record for this skill. Lists the upstream research sources
|
||||
(claude-code-plugins and github-copilot-plugins research docs) that informed SKILL.md and the
|
||||
reference files. Used by `skill-audit` to validate the provenance chain.
|
||||
|
||||
117
plugins/kyberforge/skills/agent-author/references/contract.md
Normal file
117
plugins/kyberforge/skills/agent-author/references/contract.md
Normal file
@@ -0,0 +1,117 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# The agent description and body contract
|
||||
|
||||
House contract, set by ADR-0020. The counts and the boundary targets are enforced by
|
||||
`agent-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
|
||||
judgment calls by its reference files.
|
||||
|
||||
## Why the budget exists
|
||||
|
||||
An agent's `name` and `description` is loaded into every session's context at startup, whether or
|
||||
not the agent is ever delegated to — the same cost a skill's description carries, so agents take
|
||||
the same numbers. The body is different: it is not loaded into the caller's conversation at all,
|
||||
it *becomes the system prompt of a fresh context* when the agent runs. That is why the body has no
|
||||
word gate here and a skill body has one.
|
||||
|
||||
## Description
|
||||
|
||||
A description carries exactly three things:
|
||||
|
||||
1. **Trigger clause** — when to delegate, imperative: "Use when …", never "This agent …". Describe
|
||||
the user's intent and the triggering condition, not the agent's internal mechanics.
|
||||
2. **At most one capability clause** — what it does, one clause, no enumeration. Be specific
|
||||
("reviews a diff for injected credentials", not "helps with security").
|
||||
3. **Boundary clause** — form: `Not <thing> -> <name>.` Add one only where a near-miss agent or
|
||||
skill could steal delegations.
|
||||
|
||||
Banned from a description; move it to the body or to `README.md`:
|
||||
|
||||
- Capability enumeration or feature lists
|
||||
- Per-scope emission mechanics — which files the author skill writes at which scope changes no
|
||||
delegation decision
|
||||
- Output-format detail ("Produces a compact findings report with Why and Fix per finding")
|
||||
- Composition or architecture notes ("composes X rather than duplicating Y", "cross-cutting")
|
||||
- Implementation detail ("Self-validates via a bundled deterministic script")
|
||||
- Restating the same trigger twice in two registers — a verb list, then the same verbs re-quoted
|
||||
as user phrasings. This is a FAIL, not a suggestion.
|
||||
|
||||
**Do not open with an action verb.** "Reviews…", "Analyzes…", "Generates…" was the old house rule
|
||||
and ADR-0020 deleted it: the opener is `Use when`, matching every skill in this corpus, so one
|
||||
router reads one shape.
|
||||
|
||||
**"Use proactively" is conditional.** Add it only where the runtime should delegate without the
|
||||
user naming the agent — an agent invoked by name does not need it, and it costs activations
|
||||
elsewhere when added by reflex. The same conditional governs indirect triggers ("even if the user
|
||||
doesn't say X"): add one only where the user's natural phrasing genuinely omits the domain word.
|
||||
|
||||
**Boundary targets must resolve.** The name after the arrow is checked against real skills under
|
||||
`plugins/*/.apm/skills/<name>/` and real agents under `plugins/*/.apm/agents/<name>.agent.md`. A
|
||||
target that does not exist sends the router nowhere. Verify it before writing it — do not invent a
|
||||
plausible sibling.
|
||||
|
||||
**Length.** 250 characters SUGGESTION, 400 characters FAIL, counting the frontmatter value only
|
||||
with YAML folding resolved. Treat 250 as the target: the SUGGESTION tier is what moves the corpus
|
||||
average, the FAIL tier only stops outliers.
|
||||
|
||||
## Body
|
||||
|
||||
Write the body as a direct role instruction, addressed to the agent:
|
||||
|
||||
````markdown
|
||||
You are a <role>. When invoked, <primary action>.
|
||||
|
||||
## Inputs
|
||||
<what the agent is given: files, context, parameters>
|
||||
|
||||
## Process
|
||||
<ordered steps; be explicit where ordering matters>
|
||||
|
||||
## Output
|
||||
<what it produces: format, location, structure>
|
||||
````
|
||||
|
||||
One job per agent. An agent covering two jobs gets delegated to for the wrong one.
|
||||
|
||||
**Delegation discipline replaces the word gate.** A plugin/APM agent is a single file with no
|
||||
sibling `references/` directory: it cannot disclose progressively to itself, so its only way to
|
||||
stay short is to *invoke* rather than *restate*. A body that transcribes a procedure a skill it
|
||||
can invoke already owns is an `agent-audit` FAIL, and the fix is one line — "invoke `<skill>`".
|
||||
|
||||
- Restating: "To commit, check the message against Conventional Commits: type, scope,
|
||||
description; header under 100 chars; …"
|
||||
- Delegating: "Author commits with `git-commits`."
|
||||
|
||||
The same holds for a procedure another agent owns. What belongs in the body is what no invocable
|
||||
skill covers: the agent's role, its boundaries, the order it works in, and the format it returns.
|
||||
|
||||
**State a read-only boundary in prose, not only in frontmatter.** `disallowedTools` denies the
|
||||
tools it names and nothing else — never `Bash`, which an agent with no `tools` field inherits — so
|
||||
an agent fenced only in frontmatter can still write through a shell redirect.
|
||||
|
||||
## Invocation axis
|
||||
|
||||
Decide before writing the description whether the agent is model-delegated (the runtime picks it)
|
||||
or reached only by name (`@agent-<name>`).
|
||||
|
||||
Only Copilot's cloud/IDE format expresses that in frontmatter: `disable-model-invocation: true`
|
||||
requires explicit invocation, and `user-invocable: false` hides an agent from manual invocation.
|
||||
Both live in `.github/copilot/agents/<name>.md` and are inert in the CLI format. Claude Code has
|
||||
no equivalent field, and neither does the vendor-neutral plugin/APM file, so at those scopes a
|
||||
name-invoked agent still needs a description precise enough not to steal delegations — the
|
||||
boundary clause is doing that work.
|
||||
|
||||
## One gate, two measurements
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| description | 250 chars | 400 chars | the `description:` value only |
|
||||
| body (Copilot limit) | 30,000 chars | — | the body only; content past it is truncated silently |
|
||||
|
||||
The 30,000-character Copilot ceiling is a runtime truncation limit, not a quality target, and it
|
||||
applies to a plugin/APM file too — that file compiles into a real Copilot agent downstream. An
|
||||
agent body long enough to approach it has a delegation defect, not a length problem.
|
||||
93
plugins/kyberforge/skills/agent-author/references/create.md
Normal file
93
plugins/kyberforge/skills/agent-author/references/create.md
Normal file
@@ -0,0 +1,93 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- github-plugins-creating
|
||||
---
|
||||
|
||||
# Creating a new agent
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 3 below is done — validation, the version bump and commit
|
||||
verification are shared with the improve flow and are not repeated here.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before touching the filesystem, confirm you have:
|
||||
|
||||
- [ ] Agent name (kebab-case, e.g. `code-reviewer`)
|
||||
- [ ] Root directory — a path inside a package for plugin/APM scope, a project root, or `~` for
|
||||
user scope
|
||||
- [ ] Agent purpose — one sentence describing the task this agent handles
|
||||
- [ ] Trigger condition — when should the runtime delegate to this agent?
|
||||
|
||||
If any are missing, stop and ask before proceeding.
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
|
||||
is co-installed with this skill; if it is unavailable, stop and ask the user to install
|
||||
kyberforge before continuing.
|
||||
|
||||
Design for one job per agent. An agent covering two jobs is delegated to for the wrong one.
|
||||
|
||||
## Step 1 — Scaffold
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh <name> <root>
|
||||
```
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
|
||||
bash scripts/new-agent.sh deploy-assistant .
|
||||
bash scripts/new-agent.sh security-reviewer ~
|
||||
```
|
||||
|
||||
The script resolves scope itself and prints which one it used and every path it wrote — read that
|
||||
output rather than predicting it. It walks up from `<root>` for the nearest ancestor `apm.yml`
|
||||
carrying a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`), which marks a
|
||||
package root and means plugin/APM scope. An `apm.yml` with no `type:` is a marketplace-only
|
||||
manifest: the walk skips it and keeps going. With no such manifest found, `<root>` being exactly
|
||||
`~` (checked directly, no walk-up) is user scope and anything else is project scope. A bare
|
||||
`plugin.json` no longer signals plugin scope — that path was replaced outright, not made
|
||||
dual-mode, and falls through to project scope.
|
||||
|
||||
The script is file-by-file no-op: it skips any file that already exists, so re-running it to
|
||||
complete a partial pair is safe.
|
||||
|
||||
If the script or a template under `assets/templates/` has to change to support this agent — a new
|
||||
scope, a new scaffolded field, different output — read `references/scripts.md` first. Its
|
||||
conventions are asserted by `tests/new-agent.bats`, and an edit that ignores them fails the suite.
|
||||
|
||||
## Step 2 — Fill in the file(s)
|
||||
|
||||
Take the scope the script reported and read the matching reference — `SKILL.md` Step 2 has the
|
||||
table. That file carries the field rules and the pre-audit checklist for this scope; the other one
|
||||
describes fields this run cannot use.
|
||||
|
||||
Every scaffolded file, at every scope:
|
||||
|
||||
1. Replace each `FILL IN:` placeholder.
|
||||
2. Delete every `<!-- ... -->` template comment from the frontmatter. `apm compile` copies plugin
|
||||
frontmatter verbatim and HTML comments are not valid YAML, so a leftover comment breaks the
|
||||
file downstream on both harnesses.
|
||||
3. Write the `description` against `references/contract.md` and the system prompt body against its
|
||||
Body section.
|
||||
|
||||
## Step 3 — Populate or delete `sources.md`
|
||||
|
||||
Plugin/APM scope only — skip at project and user scope, which have no package root to hold the
|
||||
file.
|
||||
|
||||
The scaffold writes a commented `sources.md` skeleton at the package root, alongside `apm.yml` and
|
||||
not inside `.apm/agents/`, so that tooling scanning that directory for agent definitions does not
|
||||
treat it as an agent missing its frontmatter (ADR-0010).
|
||||
|
||||
If a research `sources.md` is present in the conversation context, filter it to entries with
|
||||
`` `extracted` `` status, work out which agent file each one contributed to, and fill in the
|
||||
skeleton following the commented format already in the file. Paths in `Contributing files:` are
|
||||
relative to the package root. Each slug must match an H2 heading and must also appear in the
|
||||
`source_keys` list of every file named under its `Contributing files:`.
|
||||
|
||||
If no research sources are in context, delete `sources.md`.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -22,13 +22,14 @@ Agent definitions deploy at three scopes and behave differently at each. The sco
|
||||
|
||||
When the same agent `name` appears at multiple scopes, **user scope wins over project scope wins over plugin scope** in Claude Code. In Copilot CLI, repo-level agents override enterprise and org-level; home-directory (user) agents override repo-level on name collision.
|
||||
|
||||
## Plugin scope restrictions
|
||||
## Which fields exist where
|
||||
|
||||
Plugin/APM agents (`.apm/agents/<name>.agent.md`) carry only the fields in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`. That section is the authoritative list — `agent-audit`'s `validate.sh` reads it from there as data, and it changes — so consult it rather than any restatement of it. `apm compile` copies this frontmatter verbatim to both the Claude Code and Copilot CLI compile targets with no per-target integrator, so a harness-specific value is guaranteed wrong on at least one target (ADR-0016).
|
||||
|
||||
**The rule is about a field's shape, not a fixed roster.** `tools` is an **allowlist** whose vocabulary differs per harness — Claude Code names its own tools, Copilot CLI uses aliases (`execute`/`read`/`edit`/`search`/`agent`/`web`) — so under verbatim copy one value is wrong on one target. It stays out. `disallowedTools` is a **denylist**, and denying by name has no such conflict: a name the other harness does not recognise denies nothing, so the worst case is that the fence is absent there, never that a capability is wrongly granted. That asymmetry is why the denylist is admitted where the allowlist is not (ADR-0016's 2026-08-14 amendment). Claude Code honours it for plugin subagents — `docs/research/docs/claude-code-plugins/agent-definition.md:99` names the three fields plugin agents silently ignore (`hooks`, `mcpServers`, `permissionMode`) and `disallowedTools` is not among them. It is a partial fence: it denies only the tools it names, not `Bash`, which a plugin-scope agent with no `tools` inherits — so state read-only intent in the body too.
|
||||
|
||||
This makes the old "silently ignored at plugin scope" framing moot for the excluded fields. It's not that `hooks`, `mcpServers`, `permissionMode`, `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `skills`, `color`, `initialPrompt`, or `background` are merely ignored at this scope — they are never written to the file at all. Copy the agent to `.claude/agents/` (project scope) or `~/.claude/agents/` (user scope) to use any of them.
|
||||
Field rules are per scope and live with the scope: `references/plugin-scope.md` for the single
|
||||
vendor-neutral file, `references/project-user-scope.md` for the Claude Code / Copilot pair. Read
|
||||
one, not both. The short version is that plugin/APM frontmatter is an allowlist read from
|
||||
`agent-audit`'s `references/field-inventory.md`, narrow because `apm compile` copies frontmatter
|
||||
verbatim to every target (ADR-0016), while project and user scope carry the full per-provider
|
||||
field sets.
|
||||
|
||||
## Scoped identifiers (Claude Code plugin agents only)
|
||||
|
||||
|
||||
76
plugins/kyberforge/skills/agent-author/references/improve.md
Normal file
76
plugins/kyberforge/skills/agent-author/references/improve.md
Normal file
@@ -0,0 +1,76 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
---
|
||||
|
||||
# Improving an existing agent
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 4 below is done — validation, the version bump and commit
|
||||
verification are shared with the create flow and are not repeated here.
|
||||
|
||||
## Step 1 — Verify inputs
|
||||
|
||||
Confirm the agent file (or, at project and user scope, the pair) exists and that at least one
|
||||
improvement signal is present in the conversation or in a referenced file.
|
||||
|
||||
If no signals are present, stop: "This skill applies existing signals to an agent. For a blind
|
||||
review, run `agent-audit` instead."
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
|
||||
it is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
|
||||
|
||||
**Partial pair — project and user scope only.** If one provider file exists and the other does
|
||||
not, scaffold the missing one with `bash scripts/new-agent.sh <name> <root>` (file-by-file no-op)
|
||||
and continue. Plugin/APM scope is a single file and has no partial state.
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current file(s), then collect every signal from the conversation and from any path the
|
||||
user referenced.
|
||||
|
||||
Group signals by **root cause**, not by symptom. Patching per symptom is the default failure mode:
|
||||
three complaints often trace to one missing instruction. Ask: "What single gap in this agent
|
||||
causes this cluster?" One root cause, one fix.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- User feedback: the agent keeps trying to push to the remote
|
||||
- Session context: no scope boundary in the system prompt
|
||||
→ Root cause: the system prompt has no git scope constraint → fix: add an explicit boundary
|
||||
```
|
||||
|
||||
## Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state which root causes were identified, what evidence supports each, and which
|
||||
files will change. Then proceed — edits are reversible via git, so no approval checkpoint is
|
||||
needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit whichever file the signals point to.
|
||||
|
||||
**Generalize, do not patch.** Fix the underlying gap, not the one example that failed. A fix
|
||||
scoped to the cases you have seen overfits and performs worse on new input.
|
||||
|
||||
**Delegate rather than grow.** An agent body has no word ceiling, but a body that restates a
|
||||
procedure a skill it can invoke already owns is an `agent-audit` FAIL. When a signal reports a
|
||||
missing procedure, check first whether an installed skill owns it and name that skill instead of
|
||||
transcribing it. See `references/contract.md`.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. A rule written in
|
||||
all caps (ALWAYS/NEVER) is usually better reframed as why the behaviour matters, so the agent can
|
||||
apply judgment at the edges.
|
||||
|
||||
**Retrofit before extending.** Any agent predating ADR-0020 has to meet the description contract
|
||||
before any other edit lands — the gates are hot and carry no baseline file, so a one-line fix to a
|
||||
non-compliant agent cannot be committed until its description meets `references/contract.md`.
|
||||
Treat that retrofit as part of the same change, not a follow-up.
|
||||
|
||||
**Re-check the scope rules.** Read the reference for the resolved scope (`SKILL.md` Step 2) and
|
||||
confirm the edit introduced no field that scope forbids, and dropped no `disallowedTools` fence
|
||||
that was already there.
|
||||
|
||||
If the edit adds or removes research-sourced content, update `source_keys` in the edited file and
|
||||
the matching `sources.md` entry — the create flow's Step 3 has the rules.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
---
|
||||
|
||||
# Plugin/APM scope — the single vendor-neutral file
|
||||
|
||||
One file, no counterpart: `<package-root>/.apm/agents/<name>.agent.md`. `apm compile` emits it to
|
||||
both the Claude Code and the Copilot CLI target. The `.agent.md` extension here is convention, not
|
||||
a Copilot marker — the file is vendor-neutral.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `agent-audit`'s
|
||||
`references/field-inventory.md`. Read them from there as data — that section is the single source
|
||||
of truth, `agent-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
|
||||
the roster, here or in a template or in script output, goes stale one step further out than the
|
||||
list itself.
|
||||
|
||||
- `name` — kebab-case, must equal the filename stem, unique within the scope.
|
||||
- `description` — write it against `references/contract.md`.
|
||||
- Everything else — check the allowlist section before adding a key. A key outside it fails the
|
||||
audit.
|
||||
|
||||
**Why the list is narrow.** `apm compile` copies frontmatter verbatim to every target with no
|
||||
per-target integrator, so a harness-specific value is wrong on at least one of them (ADR-0016).
|
||||
The rule is about a field's *shape*, not a fixed roster:
|
||||
|
||||
- `tools` is an **allowlist** whose vocabulary differs per harness — Claude Code names its own
|
||||
tools, Copilot CLI uses aliases (`execute`/`read`/`edit`/`search`/`agent`/`web`) — so one value
|
||||
is wrong on one target. It stays out. Omitting it means inherit-all-tools on both, which is
|
||||
never wrong.
|
||||
- `disallowedTools` is a **denylist**, and denying by name cannot fail that way: a name the other
|
||||
harness does not recognise denies nothing, so the worst case is a missing fence, never a wrongly
|
||||
granted capability. That asymmetry is the whole exception (ADR-0016's 2026-08-14 amendment).
|
||||
Claude Code honours it for plugin subagents; the three fields plugin agents do silently ignore
|
||||
are `hooks`, `mcpServers` and `permissionMode`, and this is not one of them. Copilot's handling
|
||||
of the key is unconfirmed, which ADR-0016 accepts as a stated risk.
|
||||
- The Claude-only knobs (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `skills`,
|
||||
`color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) have no Copilot equivalent and
|
||||
are never written to this file at all. "Silently ignored at plugin scope" is the wrong framing:
|
||||
they are absent, not tolerated. To use any of them, copy the agent to `.claude/agents/`
|
||||
(project scope) or `~/.claude/agents/` (user scope).
|
||||
|
||||
Write `disallowedTools` on every read-only plugin-scope agent — and say the agent is read-only in
|
||||
the body as well, because the fence does not cover the inherited `Bash` tool.
|
||||
|
||||
`source_keys` is provenance metadata, not a runtime field: both harnesses ignore it. Add it only
|
||||
when research sources informed the agent, with slugs matching H2 headings in the package root's
|
||||
`sources.md`.
|
||||
|
||||
## Body
|
||||
|
||||
Follow the Body section of `references/contract.md`: role instruction, one job, and delegation
|
||||
to installed skills instead of transcribed procedure.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
|
||||
- [ ] `name` kebab-case, matching the filename stem, unique in scope
|
||||
- [ ] `description` written to `references/contract.md`
|
||||
- [ ] Every frontmatter key present in the `apm-agent-allowlist` section — in particular no `tools`
|
||||
- [ ] No `FILL IN:` placeholder and no `<!-- ... -->` template comment anywhere in the file
|
||||
- [ ] System prompt body non-empty, and a read-only agent says so in prose as well as in
|
||||
`disallowedTools`
|
||||
|
||||
Then return to the flow reference you came from.
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
- github-cli-plugin-reference
|
||||
---
|
||||
|
||||
# Project and user scope — the Claude Code / Copilot pair
|
||||
|
||||
Two files per agent, written in one pass and kept in step: a Claude Code `.md` and a Copilot CLI
|
||||
`.agent.md`. The system prompt body is the same in both — the agent's task does not change with
|
||||
the provider. The frontmatter is not.
|
||||
|
||||
| Scope | Claude Code | Copilot CLI |
|
||||
|---|---|---|
|
||||
| Project | `.claude/agents/<name>.md` | `.github/agents/<name>.agent.md` |
|
||||
| User | `~/.claude/agents/<name>.md` | `~/.copilot/agents/<name>.agent.md` |
|
||||
|
||||
## Claude Code file
|
||||
|
||||
**`name`** — lowercase letters and hyphens only, unique within the scope. Claude Code discards a
|
||||
duplicate silently.
|
||||
|
||||
**`description`** — write it against `references/contract.md`. It is the primary signal for
|
||||
autonomous delegation.
|
||||
|
||||
**`tools`** — an allowlist; omit it to inherit every tool from the parent. Use `Agent(type1,type2)`
|
||||
to restrict which subagent types this agent may spawn, and omit `Agent` entirely to stop it
|
||||
spawning any. Five tools reach no subagent whatever this field says — `AskUserQuestion`,
|
||||
`EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup` and `WaitForMcpServers` — so listing one buys
|
||||
nothing. The single exception is `ExitPlanMode`, available when the parent session runs
|
||||
`permissionMode: plan`.
|
||||
|
||||
**`disallowedTools`** — a denylist, applied before `tools` and taking precedence over it. Supports
|
||||
`mcp__<server>`, `mcp__<server>__*` and `mcp__*` globs. Both a YAML list and a delimited string
|
||||
are accepted; this repo writes the comma-separated string form (`disallowedTools: Edit, Write,
|
||||
NotebookEdit`) — match it.
|
||||
|
||||
**`model`** — set it when the agent needs a different capability tier (`haiku` for fast lookups,
|
||||
`opus` for deep reasoning). Resolution order is `CLAUDE_CODE_SUBAGENT_MODEL` → the per-invocation
|
||||
parameter → this field → the main session model, so the frontmatter value is a low-priority
|
||||
default rather than a guarantee.
|
||||
|
||||
Optional fields worth considering, none of which exist at plugin/APM scope:
|
||||
|
||||
- `maxTurns` — cap agentic turns on a bounded task, to stop a runaway
|
||||
- `effort` — `low` for a single lookup, `high` or above for multi-file analysis; omit to inherit
|
||||
- `memory` — `user`, `project` or `local`; only when cross-session state is genuinely needed
|
||||
- `isolation: worktree` — only when the agent modifies files and needs an isolated copy
|
||||
- `skills` — skill names preloaded at agent startup; unrelated to the `source_keys` metadata field
|
||||
- `color` — the UI tile colour (`red`, `blue`, `green`, `yellow`, `purple`, `orange`, `pink`,
|
||||
`cyan`)
|
||||
- `background` — `true` forces background execution
|
||||
- `initialPrompt` — auto-submitted as the first turn when the agent activates as the main session
|
||||
thread; set it only for a main-thread agent, never for a subagent
|
||||
|
||||
`hooks`, `mcpServers` and `permissionMode` are honoured at these two scopes and nowhere else — a
|
||||
plugin agent carrying them is ignored silently.
|
||||
|
||||
A subdirectory under `agents/` does not affect the agent's name at these scopes; it does at plugin
|
||||
scope, which is one reason `references/deployment-modes.md` recommends keeping agents flat.
|
||||
|
||||
## Copilot file
|
||||
|
||||
Two Copilot formats exist, with different paths and different field sets. Pick one:
|
||||
|
||||
**CLI format** — what the scaffold writes.
|
||||
|
||||
- Path: `.github/agents/<name>.agent.md` (project) or `~/.copilot/agents/<name>.agent.md` (user)
|
||||
- The `.agent.md` extension is mandatory: Copilot CLI does not pick up a plain `.md` file in
|
||||
`agents/`, and fails silently rather than reporting it
|
||||
- Fields: `name` (required, must equal the filename stem), `description` (required), `tools`
|
||||
(optional)
|
||||
- `tools` uses Copilot aliases, not Claude tool names: `execute` (shell), `read`, `edit`,
|
||||
`search`, `agent`, `web`; MCP tools as `server-name/tool-name` or `server-name/*`
|
||||
|
||||
**Cloud/IDE format** — for Copilot Chat in VS Code or on GitHub.com.
|
||||
|
||||
- Path: `.github/copilot/agents/<name>.md` — a plain `.md`, in a different directory
|
||||
- Adds `target` (`vscode`, `github-copilot`, or omit for both), `user-invocable`,
|
||||
`disable-model-invocation` and `mcp-servers` (processed by the cloud runtime, ignored in VS
|
||||
Code). These four are inert in the CLI format — do not write them there
|
||||
- This is the only format that can express the invocation axis in frontmatter; see the Invocation
|
||||
axis section of `references/contract.md`
|
||||
|
||||
Both formats truncate a body past **30,000 characters** silently.
|
||||
|
||||
Copilot has no `permissionMode`, `maxTurns`, `isolation`, `memory`, `effort`, `hooks` or
|
||||
`mcpServers`. Never let those cross over from the Claude Code file.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
|
||||
Both files:
|
||||
|
||||
- [ ] `name` present and kebab-case; `description` written to `references/contract.md`
|
||||
- [ ] System prompt body present, non-empty and equivalent across the pair
|
||||
- [ ] No `FILL IN:` placeholder and no `<!-- ... -->` template comment left
|
||||
|
||||
Copilot file only:
|
||||
|
||||
- [ ] Extension is `.agent.md` (CLI format), and `name` matches the filename stem
|
||||
- [ ] No Claude Code-only field present
|
||||
- [ ] Body under 30,000 characters
|
||||
|
||||
Then return to the flow reference you came from.
|
||||
@@ -19,7 +19,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/deployment-modes.md, references/plugin-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
@@ -27,7 +27,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md, references/plugin-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
@@ -35,7 +35,7 @@ source_keys:
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md, references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
@@ -43,7 +43,7 @@ source_keys:
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
@@ -51,7 +51,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/contract.md, references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
@@ -59,7 +59,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** references/deployment-modes.md, references/project-user-scope.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-creating
|
||||
@@ -67,7 +67,7 @@ source_keys:
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-finding-installing
|
||||
|
||||
@@ -157,9 +157,11 @@ find_package_root() {
|
||||
done
|
||||
}
|
||||
|
||||
# `read` consumes a single line, so kind and path are emitted on one
|
||||
# space-separated line rather than two `echo`s — kind first (never contains
|
||||
# spaces), path last (absorbs any spaces in the path safely).
|
||||
# kind and path are emitted on one space-separated line rather than two
|
||||
# `echo`s — kind first (never contains spaces), path last (absorbs any spaces
|
||||
# in the path safely). `mapfile`/`readarray` would need bash 4.0+, which
|
||||
# macOS's stock /bin/bash 3.2 is not; a here-string `read` splits the single
|
||||
# line without it. Same form as skill-author's new-skill.sh, deliberately.
|
||||
WALK_RESULT="$(find_package_root "$ROOT")"
|
||||
read -r WALK_KIND WALK_ROOT <<< "$WALK_RESULT"
|
||||
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
# skill-audit
|
||||
|
||||
Audit a skill directory against the agentskills.io specification. Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
Audit a skill directory against the agentskills.io specification and the house context-budget contract (ADR-0020). Runs structural validation then a qualitative review across description quality, body discipline, patterns, formatting, file structure, scripts, and internal consistency, plus a provenance chain check.
|
||||
|
||||
## What it does
|
||||
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags known-bad description openers, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags non-imperative description openers, composition and architecture notes, vague wording, padding phrases, and "There is/are" sentence openers
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across seven dimensions
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to /skill-improve
|
||||
3. Applies qualitative checks across six dimension groups, loading one rubric from `references/` per group
|
||||
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to `skill-author`
|
||||
|
||||
`validate.sh` enforces two independent length families that must not be conflated: the agentskills.io spec conformance ceilings (500 lines, 2,770 words, both counting the whole file) and the ADR-0020 context budget (250/400 description characters, 600/900 body-only words, plus resolvable boundary targets).
|
||||
|
||||
## Usage
|
||||
|
||||
@@ -26,12 +28,16 @@ Provide the path to the skill directory to audit when invoking.
|
||||
| `scripts/validate-provenance.sh` | Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, and upstream research doc alignment |
|
||||
| `scripts/vale-wrap.sh` | Vale prefilter wrapper — runs the bundled `Kyberforge` Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
|
||||
| `assets/vale/.vale.ini` | Vale configuration — points Vale at the bundled `Kyberforge` style path, self-located relative to `vale-wrap.sh` |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags literal "This skill..."/"This agent..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Vale rule — flags composition and architecture notes in a description (e.g. "cross-cutting", "entry point", "rather than duplicating") |
|
||||
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags non-imperative "This..." description openers |
|
||||
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Vale rule — flags generic "see references/" padding phrasing in conditional references |
|
||||
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Vale rule — flags body sentences starting with "There is"/"There are" |
|
||||
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
|
||||
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
|
||||
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
|
||||
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked (`disable-model-invocation`) contract, and the internal-mechanics FAIL |
|
||||
| `references/body-discipline.md` | Rubric for the body-discipline dimension — the core test, the 600/900 body-only budget against the 2,770-word whole-file backstop, the mandatory-dispatch rule, and the Gotchas constraints |
|
||||
| `references/patterns.md` | Rubric for the patterns dimension — which instruction construct fits which job, and how each is correctly formed |
|
||||
| `references/file-structure.md` | Rubric for the file-structure and internal-consistency dimensions — permitted directories, cross-plugin path rules and their two structural exemptions, README drift |
|
||||
| `references/formatting-and-scripts.md` | Rubric for the formatting and scripts dimensions — heading and fencing conventions, and the agentic-use criteria for bundled scripts |
|
||||
| `references/sources.md` | Provenance record — agentskills.io sources that informed this skill and which files each contributed to |
|
||||
| `tests/validate.bats` | (source-only) Bats test suite for validate.sh |
|
||||
| `tests/validate-provenance.bats` | (source-only) Bats test suite for validate-provenance.sh |
|
||||
|
||||
@@ -1,19 +1,10 @@
|
||||
---
|
||||
name: skill-audit
|
||||
description: >
|
||||
Use when the user wants to review a skill they wrote, says "audit this skill",
|
||||
"check if my skill follows best practices", "review my SKILL.md", or wants to
|
||||
know if a skill is ready to ship — even if they don't use the word "audit".
|
||||
Also invoke proactively after directly hand-editing a skill's files outside
|
||||
skill-author — an unaudited hand-edit is the same risk as unreviewed code.
|
||||
Audits a skill directory against the agentskills.io specification — structural
|
||||
checks plus qualitative review of description quality, body discipline, patterns,
|
||||
formatting, file structure, scripts, and internal consistency, plus a provenance
|
||||
chain check. Produces a compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
|
||||
handoff to /skill-improve or human auditability. Do not use to fix application
|
||||
code bugs or perform general code review unrelated to skill quality.
|
||||
Do not use when the user wants improvements applied — use /skill-improve instead.
|
||||
Use when the user wants a skill directory audited against the agentskills.io
|
||||
spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or
|
||||
after hand-editing a skill outside skill-author. Not applying fixes ->
|
||||
skill-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -27,9 +18,14 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not output PASS/FAIL per check while auditing — gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Structural validation
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
@@ -37,98 +33,47 @@ bash scripts/validate-provenance.sh <skill-dir>
|
||||
scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines and ≤2770 words (the word count is a proxy for the ~5,000-token ceiling, and blocks a commit exactly like the line count does), no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both. If it cannot run at all (no `python3`, Bash denied), report that as an INFO finding rather than guessing; what it measures is not reproducible by reading.
|
||||
|
||||
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
|
||||
`validate-provenance.sh` prints nothing on success. Its FAIL and INFO findings become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim.
|
||||
|
||||
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". It applies that config's `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns/Body dimensions below, not a replacement for Step 3. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one citing its rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 3 judgment for the dimensions it would have covered.
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture.
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md`, `README.md`, and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Work through each dimension internally. Collect findings only; report them in Step 4. Cite file and line number for every finding.
|
||||
Load a dimension's rubric before judging that dimension. Each is self-contained, and each is grounded in the agentskills.io specification plus the house context-budget contract (ADR-0020).
|
||||
|
||||
### Description
|
||||
| Dimension | Read |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body-discipline | `references/body-discipline.md` |
|
||||
| patterns | `references/patterns.md` |
|
||||
| file-structure, internal-consistency | `references/file-structure.md` |
|
||||
| formatting, scripts | `references/formatting-and-scripts.md` |
|
||||
|
||||
Vale's `Kyberforge.DescriptionOpener` ("This skill..." openers) and `Kyberforge.VagueWording` (filler like "helps with", "utilize") alerts from Step 1 — both FAILs — cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
|
||||
|
||||
- **Action-verb opening**: does the description start with a verb ("Audits...", "Reviews...", "Validates...")? Vale's `Kyberforge.DescriptionOpener` alert only catches the literal "This skill..." pattern — confirming an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
|
||||
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
|
||||
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
|
||||
- **Length**: under 1024 characters?
|
||||
|
||||
If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`.
|
||||
|
||||
### Body discipline
|
||||
|
||||
For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding.
|
||||
|
||||
- **Defaults not menus**: every decision point gives one default + one escape hatch, not a list of options
|
||||
- **Why rationale**: include/exclude rules explain why, not just what
|
||||
- **Control calibration**: prescriptive for fragile or critical sequences (e.g. a script invocation where flag order or exact arguments must not change); flexible where multiple approaches are valid
|
||||
|
||||
Vale's `Kyberforge.SentenceOpenerThereIs` alert from Step 1 (FAIL — sentences starting with "There is"/"There are") covers pattern-matchable body-wide filler directly; report it as a finding without re-deriving by judgment.
|
||||
|
||||
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
|
||||
|
||||
### Patterns
|
||||
|
||||
Check each pattern is appropriate and correctly formed:
|
||||
|
||||
- **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip
|
||||
- **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block
|
||||
- **Checklists**: used for multi-step workflows, not single steps
|
||||
- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment.
|
||||
- **Output templates**: present when the agent must produce a specific format; absent otherwise
|
||||
|
||||
### File structure
|
||||
|
||||
- Permitted directories: `scripts/`, `references/`, `assets/`, `tests/`; flag any other unlisted directory as FAIL — the spec allows additional dirs but this skill permits only these four to keep skills focused
|
||||
- `scripts/` contains only executable code agents can run; test files (`.bats`, `*_test.*`, `test_*.sh`) in `scripts/` are a FAIL — they belong in `tests/`
|
||||
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
|
||||
- Optional directories contain real content — not just unfilled placeholder READMEs
|
||||
- `README.md` present and accurately describes the skill and its files
|
||||
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`, or its APM-native equivalent `.apm/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
|
||||
- `references/sources.md` is exempt from the cross-plugin path check — `Research doc:` fields are development-only provenance pointers, not runtime references; they intentionally reference paths outside the skill directory and are expected to be non-resolvable after plugin install; `validate-provenance.sh` handles this gracefully by silently skipping upstream checks when those paths don't resolve
|
||||
- `tests/` is exempt from the cross-plugin path check — test files are dev-only and may reference repo-level test infrastructure (e.g. a shared `tests/test_helper/`). This dependency must be declared in `tests/README.md`; flag if tests exist but `tests/README.md` is absent or does not document the dependency
|
||||
|
||||
### Formatting
|
||||
|
||||
- Heading levels consistent: H2 for main sections, H3 for subsections
|
||||
- Code blocks fenced with a language tag where applicable (`bash`, `markdown`, `python`)
|
||||
- Consistent whitespace: blank line between sections, consistent list indentation
|
||||
- No broken relative paths in file references
|
||||
|
||||
### Scripts
|
||||
|
||||
- No interactive TTY prompts (`read`, `input()`, `readline`)
|
||||
- `--help` exposed with concise usage
|
||||
- Data to stdout, diagnostics to stderr
|
||||
- Idempotent ("create if not exists")
|
||||
- Meaningful exit codes documented in `--help`
|
||||
- `--dry-run` present for destructive operations
|
||||
|
||||
### Internal consistency
|
||||
|
||||
- SKILL.md steps match what scripts actually do
|
||||
- `README.md` file table lists every file that exists — no missing entries, no stale entries
|
||||
- Placeholder READMEs in `scripts/`, `references/`, `assets/` consistent with what SKILL.md says about each directory
|
||||
Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with a coverage line listing every dimension checked:
|
||||
Open with a coverage line naming every dimension checked:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely — their absence confirms they passed.
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
For each finding:
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
@@ -136,18 +81,4 @@ FAIL/SUGGESTION <finding> — file:line
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a result block:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS (N suggestions)
|
||||
PASS · P info
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /skill-improve to address findings.
|
||||
```
|
||||
|
||||
INFO findings are observational — do not affect PASS/FAIL. Omit `· P info` when there are no INFO findings. Omit the `/skill-improve` line when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -4,4 +4,4 @@ level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This (skill|agent)\b'
|
||||
- '^This\b'
|
||||
|
||||
@@ -6,31 +6,114 @@ source_keys:
|
||||
|
||||
# Body Discipline Reference
|
||||
|
||||
Source: agentskills.io — skill-authoring
|
||||
Upstream source: agentskills.io — skill-authoring, best-practices.
|
||||
House contract: ADR-0020, the context budget.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and dilutes the signal of what matters.
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## What belongs in the body
|
||||
## What the body is for
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when.
|
||||
|
||||
Include content the agent lacks:
|
||||
|
||||
- Project-specific conventions and domain procedures it cannot infer
|
||||
- Non-obvious edge cases and environment-specific gotchas
|
||||
- The specific tools or sequences to use (not the full range of options)
|
||||
- The specific tools or sequences to use — not the full range of options
|
||||
- One default per decision point with one escape hatch
|
||||
|
||||
Do not include:
|
||||
Move to `references/`, behind an explicit "If X, read `references/file.md`" trigger — the literal
|
||||
conditional form, never a generic pointer:
|
||||
|
||||
- Lookup tables and spec restatements
|
||||
- Output schemas, templates and example blocks
|
||||
- Rationale and justification prose
|
||||
- Anything only one branch of the procedure ever reaches
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description — it's already in context
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Two length families, measured differently
|
||||
|
||||
Do not conflate these, and do not report them as one finding.
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| Body budget (house, ADR-0020) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Spec conformance (agentskills.io) | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 2,770-word ceiling is a token-conformance backstop calibrated to the densest prose in the
|
||||
corpus; it says nothing about quality and a file can sit a thousand words inside it while failing
|
||||
the body budget. The 900-word ceiling is the quality gate: a body is loaded into the caller's live
|
||||
context and competes with the conversation already there. `validate.sh` reports both. Cite whichever
|
||||
one actually fired.
|
||||
|
||||
A word count cannot detect the defect it stands in for. Treat both numbers as backstops to the
|
||||
dispatch rule and the Gotchas constraint below, never as a substitute for them.
|
||||
|
||||
## Dispatch is mandatory at two or more mutually exclusive flows
|
||||
|
||||
If a skill handles two or more flows that a single invocation cannot both take — separate
|
||||
subcommands, separate input types, separate lifecycle stages — the body carries a **dispatch
|
||||
table** plus the gates common to every branch, and each flow lives in its own self-contained
|
||||
`references/` file. Inlining all of them is a FAIL regardless of word count, because every
|
||||
invocation then pays for every branch it did not take.
|
||||
|
||||
The reference shape in this repo is `apm-workflow`: a 554-word body dispatching to roughly 3,000
|
||||
words of references across five mutually exclusive invocations.
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
The highest-value construct in a body, and the easiest to fill with noise. A Gotcha must state a
|
||||
fact that **contradicts a reasonable default** — something the agent gets wrong precisely by acting
|
||||
sensibly.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- **Maximum five entries.** Past five, the section is a summary of the body rather than a set of
|
||||
traps, and the agent stops reading it as a warning.
|
||||
- **A Gotcha that paraphrases a step in the body below it is a FAIL.** It has no independent
|
||||
content, and it teaches the agent that Gotchas can be skimmed because the real instruction is
|
||||
coming.
|
||||
- **A Gotchas section exceeding 25% of the body is a SUGGESTION** — the body has been inverted into
|
||||
a preamble.
|
||||
- Place the section near the top. A gotcha read after the mistake is worthless, which is also why
|
||||
Gotchas is the one construct exempt from moving to `references/`.
|
||||
|
||||
Worked negative example — `git-commits` carries thirteen entries, of which four restate content
|
||||
that already appears below or in the description:
|
||||
|
||||
| Gotcha | Restates |
|
||||
|---|---|
|
||||
| `:31` "Communicates SemVer impact" | the description |
|
||||
| `:32` "Confirmation gates are mandatory for destructive operations" | step 9 at `:52` |
|
||||
| `:33` "Never skip hooks with `--no-verify`" | step 9 at `:52` |
|
||||
| `:36` "Never commit secrets" | step 2 at `:45` |
|
||||
|
||||
All four are FAILs under this rule, and the section as a whole breaches the five-entry maximum. It
|
||||
also passes every plausible word gate, which is the point of auditing the construct directly.
|
||||
|
||||
## Calibrating control
|
||||
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed:
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be
|
||||
followed:
|
||||
|
||||
```markdown
|
||||
Run exactly:
|
||||
\`\`\`bash
|
||||
@@ -39,11 +122,13 @@ python scripts/migrate.py --verify --backup
|
||||
Do not modify the command or add additional flags.
|
||||
```
|
||||
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives — agents make better decisions when they understand the purpose.
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives —
|
||||
agents make better decisions when they understand the purpose.
|
||||
|
||||
## Defaults not menus
|
||||
|
||||
Never present a list of equivalent options — pick one and mention the alternative briefly:
|
||||
|
||||
```markdown
|
||||
# Too many options
|
||||
Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
@@ -52,37 +137,22 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
|
||||
```
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
Highest value content — environment-specific facts that defy reasonable assumptions. Place near the top of the body so the agent reads them before encountering the situation.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Each entry must be a specific, surprising fact — not a general tip or reminder.
|
||||
|
||||
## Progressive disclosure
|
||||
|
||||
Keep `SKILL.md` under 500 lines. When more content is needed, move it to `references/` and load conditionally:
|
||||
|
||||
```markdown
|
||||
If the API returns a non-200 status, read `references/api-errors.md`.
|
||||
```
|
||||
|
||||
"If X, read Y" is more useful than "see references/ for details." The agent loads on demand rather than up front.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- A sentence answers "no" to the core test (would agent get this wrong without it?) — it is padding
|
||||
- Decision points present a menu of options with no default
|
||||
- Instructions repeat content already in the description
|
||||
- Prescriptive sequences are used where flexibility is fine, or vice versa
|
||||
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
|
||||
- Two or more mutually exclusive flows are inlined instead of dispatched
|
||||
- A Gotcha paraphrases a step in the body below it, or the section exceeds five entries
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- A rationale is missing from an include/exclude rule (present but unexplained)
|
||||
|
||||
- The body exceeds 600 words counted body-only but stays at or under 900
|
||||
- The Gotchas section exceeds 25% of the body
|
||||
- A rationale is missing from an include/exclude rule — present but unexplained
|
||||
- Gotchas are correct but placed late in the body rather than near the top
|
||||
- A conditional reference trigger is vague ("see references/") rather than specific ("If X, read Y")
|
||||
- Content that only one branch reaches is inlined where a `references/` file would serve
|
||||
|
||||
@@ -6,49 +6,112 @@ source_keys:
|
||||
|
||||
# Description Quality Reference
|
||||
|
||||
Source: agentskills.io — optimizing-descriptions
|
||||
Upstream source: agentskills.io — optimizing-descriptions, specification.
|
||||
House contract: ADR-0020, the context budget. The house contract is narrower than the spec
|
||||
rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## How triggering works
|
||||
## Why the description is the expensive part
|
||||
|
||||
At startup, agents load only the `name` and `description` of each skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. **The description carries the entire triggering burden** — the body is never seen until after triggering.
|
||||
At startup an agent loads only the `name` and `description` of every installed skill. The body is
|
||||
never seen until the skill triggers. The description therefore carries the entire triggering
|
||||
burden **and** is paid for in every session, whether the skill fires or not.
|
||||
|
||||
Agents typically consult skills only for tasks requiring knowledge beyond their defaults. Specialized knowledge — unfamiliar APIs, domain-specific workflows, uncommon formats — is where description wording makes the difference.
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the agent takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## What a good description does
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
- **Imperative phrasing** — "Use when..." not "This skill does...". The agent is deciding whether to act.
|
||||
- **User intent, not mechanics** — describe what the user is trying to achieve, not how the skill works internally.
|
||||
- **Err toward being pushy** — explicitly name contexts where the skill applies, including cases where the user doesn't name the domain: "even if they don't mention X explicitly."
|
||||
- **Specificity over vagueness** — "parses and validates OpenAPI specs" beats "helps with APIs."
|
||||
- **Near-miss exclusions** — add "Do not use when..." only if a near-miss skill exists that could steal activations. Use strong near-misses (queries that share keywords but need something different), not weak ones ("write a fibonacci function").
|
||||
- **Hard limit: 1024 characters** — descriptions grow during revision; check length before finalising.
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
- **`disable-model-invocation: true`** — the skill is hand-invoked. Its description is never
|
||||
matched against user intent, so it is not a routing string. It carries **one plain human-facing
|
||||
sentence** stating what the skill does. Audit it for that and nothing else. Reporting a missing
|
||||
trigger clause, a missing boundary clause or absent indirect triggers on a hand-invoked skill is
|
||||
a wrong finding, not a strict one.
|
||||
- **No such flag** — the skill is model-invoked and the rest of this file applies.
|
||||
|
||||
## The three-part shape
|
||||
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This skill ...` —
|
||||
the agent is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source; `validate.sh` checks that
|
||||
deterministically and a dangling target already surfaces as a Structure FAIL.
|
||||
|
||||
Everything else belongs in the body or in `README.md`.
|
||||
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on a skill whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Near-miss exclusions
|
||||
|
||||
Add a boundary clause only where a sibling skill could plausibly steal the activation. Use strong
|
||||
near-misses — queries that share keywords but need something different — not weak ones ("write a
|
||||
fibonacci function"). One boundary clause per genuine near-miss; a list of four is enumeration
|
||||
wearing a boundary's clothes.
|
||||
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# Weak
|
||||
description: Process CSV files.
|
||||
|
||||
# Strong
|
||||
# FAIL — enumeration first, mechanics as the opener, a blanket indirect trigger,
|
||||
# and 300+ characters of it preloaded into every session forever.
|
||||
description: >
|
||||
Analyze CSV and tabular data files — compute summary statistics,
|
||||
add derived columns, generate charts, and clean messy data. Use when
|
||||
the user has a CSV, TSV, or Excel file and wants to explore, transform,
|
||||
or visualize the data, even if they don't explicitly mention "CSV" or
|
||||
"analysis."
|
||||
Analyze CSV and tabular data files — compute summary statistics, add derived
|
||||
columns, generate charts, and clean messy data. Use when the user has a CSV,
|
||||
TSV, or Excel file and wants to explore, transform, or visualize the data,
|
||||
even if they don't explicitly mention "CSV" or "analysis."
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The four verbs the FAIL
|
||||
# version enumerates are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when the user has a CSV, TSV, or Excel file and wants it explored,
|
||||
transformed, or charted. Not schema design -> data-model.
|
||||
```
|
||||
|
||||
The strong version names capabilities precisely and broadens applicability beyond explicit keyword matches.
|
||||
(`data-model` is illustrative. In a real description the target has to resolve.)
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
- Phrasing is descriptive ("This skill...") not imperative ("Use when...")
|
||||
- Capabilities are vague ("helps with APIs") — require precise verbs and nouns
|
||||
- No indirect trigger coverage when indirect cases clearly exist
|
||||
- No near-miss exclusions when a sibling skill could plausibly steal activations
|
||||
- Length exceeds 1024 characters
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared skill", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
||||
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
|
||||
file in the authoring source. `validate.sh` reports the unresolved name.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0.
|
||||
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
||||
of the 400-character house ceiling above.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
- Indirect trigger coverage exists but could be more specific
|
||||
- Near-miss exclusions are present but target weak near-misses only
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
---
|
||||
|
||||
# File Structure and Internal Consistency Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (optional directories, file references).
|
||||
|
||||
Read this when judging the **file-structure** and **internal-consistency** dimensions.
|
||||
|
||||
## Permitted directories
|
||||
|
||||
Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification permits additional
|
||||
directories; this house does not, because an unlisted directory is content no auditor and no host
|
||||
knows to look at. Flag any other directory as a FAIL.
|
||||
|
||||
- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`,
|
||||
`test_*.sh`) there are a FAIL — they belong in `tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
- `README.md` is present and describes the skill and its files accurately.
|
||||
|
||||
## Cross-plugin path references
|
||||
|
||||
A plugin is copied to a cache on install, and a path that climbs out of the skill directory stops
|
||||
resolving there. Flag any `../`, `../../`, or absolute repo path (`plugins/<plugin>/skills/<other>/`
|
||||
and its APM-native equivalent `.apm/skills/<other>/`) appearing in `SKILL.md`, `scripts/`,
|
||||
`references/` or `assets/`.
|
||||
|
||||
Two directories are exempt, and the exemptions are structural rather than discretionary:
|
||||
|
||||
- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers,
|
||||
not runtime references. They are expected to be unresolvable after install, and
|
||||
`validate-provenance.sh` handles that by skipping upstream checks silently when the path is
|
||||
absent. Flagging them would make every correctly-provenanced skill fail.
|
||||
- **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared
|
||||
`tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/`
|
||||
exists and `tests/README.md` is absent or does not document it, that is a FAIL.
|
||||
|
||||
## Internal consistency
|
||||
|
||||
The skill has to agree with itself. Three checks:
|
||||
|
||||
- `SKILL.md`'s steps match what the scripts actually do — the arguments, the exit codes, and the
|
||||
output shape it tells the agent to expect.
|
||||
- `README.md`'s file table lists every file that exists, with no missing rows and no stale rows for
|
||||
files since deleted.
|
||||
- Placeholder READMEs inside `scripts/`, `references/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
A stale README row is the most common finding here and the easiest to miss from inside an
|
||||
authoring pass, because the author knows what was intended and reads it into the gap.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A directory outside the four permitted ones exists
|
||||
- Test files sit in `scripts/`
|
||||
- A non-spec file sits at the skill root
|
||||
- A cross-plugin or parent-relative path appears outside the two exempt locations
|
||||
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
||||
- `README.md` is absent, or its file table has a missing or stale row
|
||||
- `SKILL.md` describes a script invocation the script does not accept
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Formatting and Scripts Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (body content), using-scripts (designing scripts
|
||||
for agentic use).
|
||||
|
||||
Read this when judging the **formatting** and **scripts** dimensions. Both are checklists of static
|
||||
criteria that never vary by skill, which is exactly why they live here rather than in the body.
|
||||
|
||||
## Formatting
|
||||
|
||||
- Heading levels are consistent: H2 for main sections, H3 for subsections. A body that jumps from
|
||||
H2 to H4, or opens on H3, reads as a fragment of a larger document.
|
||||
- Code blocks carry a language tag wherever one applies — `bash`, `markdown`, `python`, `yaml`,
|
||||
`text`. An untagged block loses syntax highlighting and, more importantly, loses the signal of
|
||||
what the agent is meant to do with it.
|
||||
- Whitespace is consistent: a blank line between sections, one list-indentation style throughout.
|
||||
- No broken relative paths in file references. Every `references/…`, `scripts/…` and `assets/…`
|
||||
path named in the body resolves against the skill directory.
|
||||
|
||||
## Scripts
|
||||
|
||||
A script in a skill is run by an agent with no terminal and no human to answer it. The criteria
|
||||
follow from that:
|
||||
|
||||
- **No interactive TTY prompts** — no `read`, no `input()`, no `readline`. A script that blocks on
|
||||
a prompt hangs the run with no diagnostic. `validate.sh` detects the common forms and reports
|
||||
them under Structure; the judgment call is any prompt it cannot pattern-match. What counts is
|
||||
where stdin comes from, not the word `read`: a `read` fed by a here-string, a here-doc, a pipe,
|
||||
or a redirect from a file never touches a terminal and is not a finding. `validate.sh` excludes
|
||||
those forms, so do not rewrite a working `read -r A B <<< "$line"` into parameter expansion to
|
||||
satisfy this rule.
|
||||
- **`--help` is exposed** and gives concise usage.
|
||||
- **Data to stdout, diagnostics to stderr.** A caller piping the script has to be able to separate
|
||||
the result from the commentary.
|
||||
- **Idempotent** — "create if not exists" rather than "create", so a re-run after a partial failure
|
||||
is safe.
|
||||
- **Meaningful exit codes, documented in `--help`.** An agent branches on the exit code; an
|
||||
undocumented one is a coin flip.
|
||||
- **`--dry-run` present for destructive operations.**
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A script prompts interactively, in any form
|
||||
- A script exposes no `--help`
|
||||
- A destructive script has no `--dry-run`
|
||||
- Data and diagnostics share a stream, so the output cannot be piped
|
||||
- A relative path named in the body does not resolve
|
||||
- Heading levels are inconsistent enough to break the document's structure
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Exit codes are meaningful but undocumented in `--help`
|
||||
- A code block is untagged where a language applies
|
||||
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
|
||||
- List indentation or section spacing is inconsistent without breaking the render
|
||||
60
plugins/kyberforge/skills/skill-audit/references/patterns.md
Normal file
60
plugins/kyberforge/skills/skill-audit/references/patterns.md
Normal file
@@ -0,0 +1,60 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
---
|
||||
|
||||
# Patterns Reference
|
||||
|
||||
Upstream source: agentskills.io — best-practices (instruction patterns), specification.
|
||||
|
||||
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
|
||||
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
|
||||
*content* is judged in `references/body-discipline.md`.
|
||||
|
||||
## The constructs and when each is right
|
||||
|
||||
| Construct | Right when | Wrong when |
|
||||
|---|---|---|
|
||||
| Gotchas | An environment fact contradicts a reasonable default | Used as a summary of the steps below |
|
||||
| Prescriptive sequence | The operation is fragile and flag order or exact arguments must not change | Several approaches are equally valid |
|
||||
| Checklist | A multi-step workflow the agent must complete in order | A single step dressed up as a list |
|
||||
| Conditional reference | Detail is needed on one branch only | The reference is needed on every run and is loaded blind |
|
||||
| Output template | The agent must emit a specific format a caller consumes | The output is prose nobody parses |
|
||||
|
||||
## Formation rules
|
||||
|
||||
**Gotchas** sit near the top of the body, before the steps that would otherwise walk into them.
|
||||
Placement late in the body is a SUGGESTION, not a FAIL — the content is still correct, it is just
|
||||
read after the mistake.
|
||||
|
||||
**Prescriptive sequences** that quote a fenced block inside another markdown block must escape the
|
||||
inner fence as `` \`\`\` ``. An unescaped inner fence terminates the outer block and the remaining
|
||||
instructions render as prose.
|
||||
|
||||
**Conditional references** state a specific trigger: "If the API returns a non-200 status, read
|
||||
`references/api-errors.md`." The generic form — pointing at the directory and hoping — defeats
|
||||
progressive disclosure, because the agent either loads everything or loads nothing.
|
||||
`Kyberforge.PaddingPhrase` catches the common generic phrasing deterministically; other malformed
|
||||
forms are judgment.
|
||||
|
||||
**Output templates** belong in the body when the agent must emit them on every run, and in
|
||||
`references/` when only one dispatch branch produces that output. A template inlined for a branch
|
||||
most invocations never take is body-discipline padding.
|
||||
|
||||
## Auditing guidance
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
|
||||
assumption
|
||||
- An inner code fence is unescaped inside a markdown block, breaking the render
|
||||
- A checklist wraps a single step
|
||||
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
|
||||
- The agent must produce a specific format and no output template is given
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Gotchas are correctly formed but placed late in the body
|
||||
- An output template is present but permissive where the consumer needs it exact
|
||||
- A conditional reference names a trigger that is real but broader than the branch it guards
|
||||
@@ -15,7 +15,7 @@
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
@@ -23,7 +23,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
@@ -47,7 +47,7 @@
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
@@ -11,7 +11,7 @@ Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
EOF
|
||||
}
|
||||
@@ -32,6 +32,7 @@ python3 -u - "$1" <<'PYTHON'
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
@@ -44,6 +45,7 @@ with open(skill_md) as f:
|
||||
content = f.read()
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
@@ -53,6 +55,13 @@ def fail(msg):
|
||||
print(f"FAIL {msg}")
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
|
||||
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
|
||||
if not fm_match:
|
||||
@@ -66,16 +75,58 @@ body_start = fm_match.end()
|
||||
name_m = re.search(r'^name:\s*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — inline or block scalar (> or |)
|
||||
desc = ""
|
||||
desc_m = re.search(r'^description:\s*([>|])\n((?:[ \t]+.+\n?)+)', fm, re.MULTILINE)
|
||||
if desc_m:
|
||||
raw = desc_m.group(2)
|
||||
desc = re.sub(r'\s+', ' ', raw).strip()
|
||||
else:
|
||||
desc_inline = re.search(r'^description:\s*(.+)', fm, re.MULTILINE)
|
||||
if desc_inline:
|
||||
desc = desc_inline.group(1).strip()
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first. PyYAML is used
|
||||
# when importable (it is a real parser); the fallback recognises exactly the
|
||||
# shapes this corpus uses — an inline scalar, optionally quoted and optionally
|
||||
# continued on following indented lines, and a `>`/`|` block scalar with
|
||||
# optional indentation and chomping indicators.
|
||||
|
||||
def normalize(value):
|
||||
return re.sub(r'\s+', ' ', value).strip()
|
||||
|
||||
def fold_description_fallback(fm_text):
|
||||
lines = fm_text.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
m = re.match(r'^description:[ \t]*(.*)$', line)
|
||||
if not m:
|
||||
continue
|
||||
head = m.group(1).strip()
|
||||
block = bool(re.match(r'^[>|][0-9]*[-+]?$|^[>|][-+]?[0-9]*$', head))
|
||||
parts = [] if block else [head]
|
||||
for nxt in lines[i + 1:]:
|
||||
if not nxt.strip():
|
||||
parts.append('')
|
||||
continue
|
||||
if not re.match(r'^[ \t]', nxt):
|
||||
break
|
||||
parts.append(nxt.strip())
|
||||
value = ' '.join(parts)
|
||||
if not block:
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in '"\'':
|
||||
value = value[1:-1]
|
||||
return value
|
||||
return ''
|
||||
|
||||
def extract_description(fm_text):
|
||||
try:
|
||||
import yaml
|
||||
data = yaml.safe_load(fm_text)
|
||||
if isinstance(data, dict):
|
||||
value = data.get('description')
|
||||
if isinstance(value, str):
|
||||
return normalize(value)
|
||||
if value is not None:
|
||||
return normalize(str(value))
|
||||
return ''
|
||||
except Exception:
|
||||
pass
|
||||
return normalize(fold_description_fallback(fm_text))
|
||||
|
||||
desc = extract_description(fm)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
@@ -114,11 +165,13 @@ if desc:
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (limit: 1024)")
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
@@ -146,6 +199,26 @@ MAX_LINES = 500
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
|
||||
# description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
@@ -160,14 +233,220 @@ if word_count <= MAX_WORDS:
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
# Body unfilled placeholders
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# A boundary clause names another skill — or an agent, which is an equally
|
||||
# valid routing target (git-workflow routes to the git-orchestrate agent). Every
|
||||
# named target is resolved against the AUTHORING SOURCE, plugins/*/.apm/skills/
|
||||
# and plugins/*/.apm/agents/, so the check works offline and before an
|
||||
# `apm install` has deployed anything into .claude/skills/.
|
||||
#
|
||||
# False positives are the design constraint here, not recall. Two rules do the
|
||||
# work:
|
||||
# * A BARE hyphenated word is read as a routing target only inside a boundary
|
||||
# sentence (one carrying "do not"/"instead"/"rather than"/"not for").
|
||||
# Without that, pc-run's "run pre-commit hooks" reads as a route to a
|
||||
# non-existent `pre-commit` skill.
|
||||
# * A BARE arrow target counts only in ADR-0020's compressed boundary form,
|
||||
# `Not <thing> -> <skill-name>`. Without that, diagnose's process chain
|
||||
# "fix -> regression-test" reads as a route to `regression-test`.
|
||||
# Backticked and /slash-command targets are unambiguous and always count. Tool
|
||||
# names (Read, Write, Edit) are excluded by the lowercase-only name pattern;
|
||||
# MCP tool names (issue_write, pull_request_write) by its rejection of
|
||||
# underscores; file names by its rejection of dots and slashes.
|
||||
|
||||
NAME_ANY = r"[a-z0-9]+(?:-[a-z0-9]+)*"
|
||||
NAME_HYPH = r"[a-z0-9]+(?:-[a-z0-9]+)+"
|
||||
ROUTE_VERB = (r"(?:use|uses|using|run|runs|invoke|invokes|invoking|try|see"
|
||||
r"|that'?s|compose|composes|call|calls)")
|
||||
MARKED_TARGET = r"(?:`/?(%s)`|(?<![\w./*-])/(%s)\b)" % (NAME_ANY, NAME_ANY)
|
||||
ANY_TARGET = r"(?:%s|(%s)\b)" % (MARKED_TARGET, NAME_HYPH)
|
||||
ROUTE_MARKED = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, MARKED_TARGET), re.I)
|
||||
ROUTE_ANY = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, ANY_TARGET), re.I)
|
||||
CONT_MARKED = re.compile(r"\s*(?:or|and|/|,)\s*%s" % MARKED_TARGET)
|
||||
CONT_ANY = re.compile(r"\s*(?:or|and|/|,)\s*%s" % ANY_TARGET)
|
||||
ARROW_MARKED = re.compile(r"(?:->|→)\s*%s" % MARKED_TARGET)
|
||||
ARROW_BOUNDARY = re.compile(r"\bnot\b[^.;]*?(?:->|→)\s*(%s)\b" % NAME_HYPH, re.I)
|
||||
BACKTICK = re.compile(r"`(%s)`" % NAME_HYPH)
|
||||
BOUNDARY_MARKER = re.compile(r"\b(?:do\s+not|instead|rather\s+than|not\s+for)\b", re.I)
|
||||
SENTENCE_SPLIT = re.compile('(?<=[.!?])\\s+(?=[A-Z"“(])')
|
||||
|
||||
def _first_group(groups):
|
||||
for g in groups:
|
||||
if g:
|
||||
return g
|
||||
return None
|
||||
|
||||
def _scan_routes(text, route_re, cont_re, out):
|
||||
for m in route_re.finditer(text):
|
||||
target = _first_group(m.groups())
|
||||
if not target:
|
||||
continue
|
||||
out.append(target)
|
||||
# Conjoined targets: "use git-history or git-branches instead",
|
||||
# "use gitea-issues / gitea-prs".
|
||||
pos = m.end()
|
||||
while True:
|
||||
cm = cont_re.match(text, pos)
|
||||
if not cm:
|
||||
break
|
||||
nxt = _first_group(cm.groups())
|
||||
if nxt:
|
||||
out.append(nxt)
|
||||
pos = cm.end()
|
||||
|
||||
def boundary_targets(description):
|
||||
out = []
|
||||
for sentence in SENTENCE_SPLIT.split(description):
|
||||
boundary = bool(BOUNDARY_MARKER.search(sentence))
|
||||
_scan_routes(sentence,
|
||||
ROUTE_ANY if boundary else ROUTE_MARKED,
|
||||
CONT_ANY if boundary else CONT_MARKED,
|
||||
out)
|
||||
for m in ARROW_MARKED.finditer(sentence):
|
||||
target = _first_group(m.groups())
|
||||
if target:
|
||||
out.append(target)
|
||||
for m in ARROW_BOUNDARY.finditer(sentence):
|
||||
out.append(m.group(1))
|
||||
out.extend(BACKTICK.findall(sentence))
|
||||
return sorted(set(out))
|
||||
|
||||
def known_targets(start_dir):
|
||||
names = set()
|
||||
# Sibling skills/agents. This is the branch that works in a cache-installed
|
||||
# plugin and in a deployed .claude/skills/ tree, neither of which has a
|
||||
# plugins/ directory above it.
|
||||
parent = os.path.dirname(os.path.abspath(start_dir))
|
||||
if os.path.basename(parent) == 'skills' and os.path.isdir(parent):
|
||||
for entry in os.listdir(parent):
|
||||
if os.path.isdir(os.path.join(parent, entry)):
|
||||
names.add(entry)
|
||||
agents_dir = os.path.join(os.path.dirname(parent), 'agents')
|
||||
if os.path.isdir(agents_dir):
|
||||
for entry in os.listdir(agents_dir):
|
||||
if entry.endswith('.agent.md'):
|
||||
names.add(entry[:-len('.agent.md')])
|
||||
elif entry.endswith('.md'):
|
||||
names.add(entry[:-len('.md')])
|
||||
# Walk up for a monorepo root (plugins/*/.apm/) or a plugin root (.apm/).
|
||||
# Capped at ten levels so a pathological path can't become a filesystem
|
||||
# crawl; that covers every real layout by a wide margin.
|
||||
current = os.path.abspath(start_dir)
|
||||
for _ in range(10):
|
||||
# Never glob the filesystem root: a stray /.apm/skills/ (a scaffolding
|
||||
# test's leftover, say) would otherwise become part of every skill's
|
||||
# resolution universe on that machine.
|
||||
if os.path.dirname(current) == current:
|
||||
break
|
||||
for pattern in ('plugins/*/.apm/skills/*/', '.apm/skills/*/'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path.rstrip('/')))
|
||||
for pattern in ('plugins/*/.apm/agents/*.agent.md', '.apm/agents/*.agent.md'):
|
||||
for path in glob.glob(os.path.join(current, pattern)):
|
||||
names.add(os.path.basename(path)[:-len('.agent.md')])
|
||||
current = os.path.dirname(current)
|
||||
return names
|
||||
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
# An empty universe means no authoring source was found anywhere above this
|
||||
# skill — reporting every target as dangling there would be noise, not a
|
||||
# finding, so the check declines to run rather than guessing.
|
||||
if routing_targets and known:
|
||||
unresolved = [t for t in routing_targets if t not in known]
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill under "
|
||||
f"plugins/*/.apm/skills/ and no agent under plugins/*/.apm/agents/ — "
|
||||
f"a boundary clause naming a non-existent target sends the router nowhere")
|
||||
if not unresolved:
|
||||
ok(f"all {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(routing_targets)}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
for line in source.splitlines():
|
||||
stripped = line.strip()
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
@@ -177,9 +456,10 @@ if os.path.isdir(scripts_dir):
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
with open(fpath) as f:
|
||||
sc = f.read()
|
||||
# Interactive prompt heuristic
|
||||
if re.search(r'^\s*(read\s|input\()', sc, re.MULTILINE):
|
||||
fail(f"scripts/{fname}: may use interactive input (read/input detected)")
|
||||
interactive = interactive_reads(sc)
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
else:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
@@ -190,8 +470,17 @@ if os.path.isdir(scripts_dir):
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
print("All checks passed.")
|
||||
if suggestions:
|
||||
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
|
||||
@@ -6,6 +6,14 @@ Author and refine skills conforming to the [agentskills.io](https://agentskills.
|
||||
|
||||
Routes to one of two flows based on context: if no skill directory exists at the target path, it scaffolds the directory from annotated templates, fills in `SKILL.md` and supporting files, and validates the result. If an existing skill directory and improvement signals are both present, it groups those signals by root cause and applies targeted edits, then re-validates. In both flows, bumps the skill's `metadata.version` when present (minor for create, patch for improve).
|
||||
|
||||
`SKILL.md` itself carries only the dispatch table, the invocation-axis decision, the contract gates and the shared close; each flow lives in its own self-contained reference file, per ADR-0020.
|
||||
|
||||
## The contract it teaches
|
||||
|
||||
Authored skills are held to the ADR-0020 context budget. A description carries a trigger clause, at most one capability clause, and a boundary clause of the form `Not <thing> -> <skill-name>` whose target must resolve to a real skill or agent — 250 characters target, 400 hard ceiling. A body carries the decision procedure only — 600 words target, 900 hard ceiling, counting the body alone, which is a separate measurement from the 2,770-word / 500-line whole-file spec backstop. Skills with two or more mutually exclusive flows must dispatch. `references/contract.md` holds the full rules; `assets/templates/SKILL.md` encodes them as a fill-in skeleton.
|
||||
|
||||
Before a description is written, the skill asks whether the target is model-invoked or hand-invoked. A hand-invoked skill sets `disable-model-invocation: true` and carries one plain human-facing sentence with no trigger list.
|
||||
|
||||
## Before you start
|
||||
|
||||
- Run `/grill-me` to resolve design decisions before creating a new skill
|
||||
@@ -14,7 +22,7 @@ Routes to one of two flows based on context: if no skill directory exists at the
|
||||
|
||||
## Placement
|
||||
|
||||
`scripts/new-skill.sh` resolves the mode automatically by walking up from the given path — see `SKILL.md` Step 1 for the full algorithm.
|
||||
`scripts/new-skill.sh` resolves the mode automatically by walking up from the given path — see `references/create.md` Step 1 for the full algorithm.
|
||||
|
||||
| Mode | Path | Chosen when |
|
||||
|------|------|-------------|
|
||||
@@ -36,10 +44,13 @@ If the destination resolves inside an APM package, read `references/deployment-m
|
||||
| `README.md` | Human-readable overview of the skill and its files |
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `scripts/new-skill.sh` | Walks up from the given path to resolve package vs standalone mode, then copies annotated templates to the resolved destination |
|
||||
| `references/create.md` | The create flow end to end — prerequisites, package-intent gate, scaffold, frontmatter, scripts, references, sources (loaded on demand) |
|
||||
| `references/improve.md` | The improve flow end to end — signal verification, root-cause grouping, announcement, edits (loaded on demand) |
|
||||
| `references/contract.md` | The ADR-0020 description and body contract, the Gotchas constraint, the two size gates, body patterns, and org-policy embedding (loaded on demand) |
|
||||
| `references/deployment-modes.md` | APM package vs standalone differences and self-containment/cache-isolation rules (loaded on demand) |
|
||||
| `references/scripts.md` | Package runners, inline dependency patterns, and full script contract (loaded on demand) |
|
||||
| `references/sources.md` | Upstream research sources and which skill files each contributed to |
|
||||
| `assets/templates/SKILL.md` | Annotated SKILL.md template |
|
||||
| `assets/templates/SKILL.md` | Annotated SKILL.md template — emits an ADR-0020-compliant description and body skeleton |
|
||||
| `assets/templates/README.md` | Annotated README template for the new skill |
|
||||
| `assets/templates/scripts/README.md` | Placeholder for bundled scripts |
|
||||
| `assets/templates/references/README.md` | Placeholder for reference docs |
|
||||
|
||||
@@ -1,14 +1,9 @@
|
||||
---
|
||||
name: skill-author
|
||||
description: >
|
||||
Use when the user wants to create a new skill from scratch ("write a skill
|
||||
for X", "build a skill that does Y", "create a SKILL.md for Z") or improve
|
||||
an existing one ("improve this skill", "fix based on feedback", "apply these
|
||||
audit findings", "update based on grill output"). Also use when the user provides inline feedback
|
||||
about a skill's behavior and wants it applied, or when a grill session, eval
|
||||
run, or audit has produced findings the user wants acted on — even if they
|
||||
don't say "improve" explicitly. Do not use for read-only review — use
|
||||
/skill-audit instead. Do not use to author agent definition files.
|
||||
Use when the user wants to create a new skill from scratch, or apply audit
|
||||
findings, grill output, eval results, or inline feedback to an existing one.
|
||||
Not read-only review -> `skill-audit`. Not agent files -> `agent-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -24,283 +19,44 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Patching per symptom is the default failure mode. Three eval failures may all trace to one missing instruction — always identify the root cause before editing.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing scripts from scratch requires transcript analysis that is out of scope here; flag the opportunity as a suggestion instead.
|
||||
- Never spawn a subagent to audit or recheck your own work during an authoring pass. Run `/skill-audit` yourself, inline, in the same context as the edits you just made. A *separate* independent recheck via a clean-context subagent is the `/forge` skill's outer-loop responsibility exclusively — delegating it inward here duplicates that layer and introduces a race: a stray self-spawned subagent can have its worktree torn down by concurrent cleanup, destroying an uncommitted draft before it was ever safe.
|
||||
- A skill's `name` and `description` are preloaded into every agent's context every session, invoked or not; the body loads only on invocation. The description is the scarce budget.
|
||||
- The word gates are two different measurements, not one rule with two tiers. The 2,770-word / 500-line spec backstop counts the whole file including frontmatter; Step 3's gate counts the body alone. A file can sit well inside one and fail the other, so never unify them.
|
||||
- Never spawn a subagent to audit or recheck your own work here. Run `/skill-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent can have its worktree torn down by concurrent cleanup, destroying an uncommitted draft.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing one from scratch requires transcript analysis that is out of scope here — flag the opportunity as a suggestion instead.
|
||||
|
||||
## Route
|
||||
## Step 1 — Dispatch
|
||||
|
||||
Determine which flow to follow before touching the filesystem:
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| No skill directory at the target path | Create | `references/create.md` |
|
||||
| Directory exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| Directory exists, no signals | Stop and ask | — |
|
||||
|
||||
- **No skill directory at the target path** → follow **Creating a new skill**
|
||||
- **Directory exists + at least one improvement signal present** → follow **Improving an existing skill**
|
||||
- **Directory exists + no signals present** → ask: "No improvement signals found. Did you mean to create a new skill, or do you have feedback to apply?"
|
||||
Signals: grill output, `/skill-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new skill, or do you have feedback to apply?"
|
||||
|
||||
Signals include: grill session output, `/skill-audit` findings (PASS/FAIL punch list), inline user feedback, session context describing what went wrong.
|
||||
Read only the reference matching the resolved flow — each is self-contained. Capture `git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
**Before running the scaffold script**, judge whether the destination is meant to be inside an APM package — the script can't tell "no package here" apart from "package not scaffolded yet":
|
||||
## Step 2 — Invocation axis
|
||||
|
||||
- Package intent but no `type:`-bearing `apm.yml` found at/above the destination (e.g. "add to my apm package", or a sibling `.apm/`/`apm.yml` exists nearby) → **stop**, tell the user to run `/apm-workflow configure` (`apm plugin init`, from inside the package directory) first, then retry. Don't fall through to standalone mode.
|
||||
- Otherwise (a `~/`-rooted destination, or no package context implied) → run `scripts/new-skill.sh`; it resolves package vs. standalone automatically (see Step 1).
|
||||
Decide before writing any description: model-invoked or hand-invoked?
|
||||
|
||||
## Creating a new skill
|
||||
- **Hand-invoked** — the user types `/name` and no agent should route to it. Set `disable-model-invocation: true` and write one plain human-facing sentence: no trigger list, no boundary clause. Worked example: `plugins/bin/.apm/skills/zoom-out/SKILL.md`. Skip Step 3's description rules.
|
||||
- **Model-invoked** — the default.
|
||||
|
||||
### Prerequisites
|
||||
## Step 3 — Contract
|
||||
|
||||
Run `/grill-me` on the skill's design and research the target domain first.
|
||||
Share those outputs in this conversation: grill context, research docs, examples, constraints.
|
||||
Before writing or editing a description, or restructuring a body, read `references/contract.md` — the banned-content list, boundary form, include/exclude rubric and body patterns.
|
||||
|
||||
Design for one coherent user intent — skills too narrow force multiple loads per task; too broad are hard to activate precisely.
|
||||
Gates `/skill-audit` enforces in both flows:
|
||||
|
||||
**Before touching the filesystem, verify you have:**
|
||||
- [ ] A clear purpose — what specific task will this skill handle?
|
||||
- [ ] Trigger scenarios — when should an agent activate it, including indirect cases?
|
||||
- [ ] Skill name (kebab-case) and destination path
|
||||
- [ ] Capture `git log --oneline -1` now, before touching the filesystem — Step 7 needs it to verify a real commit landed
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <skill-name>` whose target resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only.
|
||||
- **Body** — decision procedure only: ordered steps, branches, gates, and which reference to load when. 600 words SUGGESTION, 900 FAIL, body only. At two or more mutually exclusive flows a dispatch table is mandatory and each flow gets its own self-contained `references/` file.
|
||||
- **Gotchas** — at most five, each contradicting a reasonable default. A Gotcha paraphrasing a step below it is a FAIL.
|
||||
|
||||
If any are missing, stop and ask the user before proceeding.
|
||||
## Step 4 — Validate and close
|
||||
|
||||
**Requires `/skill-audit`** — used in Step 7 for final validation. Both skills ship in the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
|
||||
Run `/skill-audit` on the resolved skill directory. It checks name-to-directory match, description presence, leftover `FILL IN:` placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those first. Resolve every FAIL before reporting done.
|
||||
|
||||
### Step 1 — Scaffold
|
||||
With `metadata.version` present, bump the **minor** version on create (new skills start at `0.1.0`) and the **patch** version on improve.
|
||||
|
||||
Run the copy script with the skill name and a path inside or at the target:
|
||||
|
||||
```bash
|
||||
bash scripts/new-skill.sh <skill-name> <path>
|
||||
```
|
||||
|
||||
The script walks up from `<path>` for a package boundary: an ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) means **package mode** — scaffolds into `<package-root>/.apm/skills/<skill-name>/`, not under `<path>` (a subdirectory of the package works fine as `<path>`). A `type:`-less `apm.yml` is a marketplace-only manifest, skipped. Hitting `.git` or the filesystem root first means **standalone mode** — scaffolds directly into `<path>/<skill-name>/`, same as before.
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
# Package mode — packages/my-pkg/apm.yml already has `type: skill`
|
||||
bash scripts/new-skill.sh my-tool packages/my-pkg/
|
||||
|
||||
# Standalone mode — no apm.yml/.git above ~/.agents/skills/
|
||||
bash scripts/new-skill.sh my-tool ~/.agents/skills/
|
||||
```
|
||||
|
||||
The script prints which mode it used and where the skill landed — read its output.
|
||||
|
||||
In package mode, read `references/deployment-modes.md` before adding any file references to SKILL.md.
|
||||
|
||||
### Step 2 — Update `apm.yml` includes (package mode only)
|
||||
|
||||
Skip in standalone mode. In package mode, check the resolved package's `apm.yml`: if `includes:` is an explicit list (not `auto`), append `.apm/skills/<skill-name>/` to it if not already present, preserving YAML formatting. If `includes: auto` or the field is absent, do nothing — `auto` already covers the new skill. Use Read/Edit directly on `apm.yml`; this isn't part of `scripts/new-skill.sh`.
|
||||
|
||||
### Step 3 — Fill in SKILL.md
|
||||
|
||||
Open the new skill's `SKILL.md` (the path Step 1 printed). Replace every `FILL IN:` placeholder.
|
||||
|
||||
**Frontmatter**
|
||||
|
||||
**`name`** — already set by the scaffold script. Must exactly match the directory name. Format: 1–64 characters, lowercase letters/numbers/hyphens only, no leading, trailing, or consecutive hyphens (`--`).
|
||||
|
||||
**`description`** — carries the entire triggering burden. Rules:
|
||||
- Imperative: "Use when..." not "This skill..."
|
||||
- Focus on user intent, not implementation — describe what the user is trying to achieve, not the skill's internal mechanics
|
||||
- Specific about capabilities ("parses and validates OpenAPI specs", not "helps with APIs")
|
||||
- Include indirect triggers: "even if the user doesn't mention X explicitly"
|
||||
- Add "Do not use when..." only if a near-miss skill exists that could steal activations
|
||||
- Hard limit: 1024 characters — count before finalizing
|
||||
|
||||
**Optional fields** — uncomment and fill in or remove entirely:
|
||||
- `license` — include when distributing the skill externally
|
||||
- `compatibility` — include if the skill requires specific tools, runtimes, or network access (max 500 characters)
|
||||
- `metadata` — key-value map; use `author`, `version`, `category`; add `source_keys` now (see below) if research sources are in context
|
||||
- `allowed-tools` — space-separated pre-approved tools; reduces permission prompts (experimental — support varies by client)
|
||||
|
||||
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs here as you write the body; don't defer this to Step 6. Agents that fill in source_keys late tend to omit it entirely. Example:
|
||||
```yaml
|
||||
metadata:
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
- another-slug
|
||||
```
|
||||
|
||||
**Embedding org-specific policy** — if a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`), inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing to the file's path. Plugins must be self-contained and portable — the org file may not exist wherever the plugin is installed, and in this repo such files are meant to be deleted once their content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the same `references/sources.md` schema as Step 6, noting in the `Research doc:` field that the source is an org convention rather than a plugin research corpus entry, so provenance survives after the source file is gone.
|
||||
|
||||
**Body — include only what the agent lacks**
|
||||
|
||||
Rename the placeholder section heading to one that fits the skill's structure — `## Step 1`, `## Workflow`, `## Instructions`, etc.
|
||||
|
||||
Ask of every sentence: "Would the agent get this wrong without it?" Cut anything that answers "no."
|
||||
|
||||
**Include:**
|
||||
- Non-obvious sequences or ordering constraints — the agent may skip or reorder steps without this
|
||||
- Domain conventions the agent cannot infer from general knowledge — this is the core value a skill adds
|
||||
- One default per decision point, plus one escape hatch — never a menu; menus cause the agent to pause or pick arbitrarily
|
||||
- Gotchas — facts that defy reasonable assumptions; the agent will get these wrong every time without them
|
||||
|
||||
**Exclude:**
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works) — adds tokens without changing behavior
|
||||
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads agents to follow unproductive paths
|
||||
- Restatements of the description — it's already in context; repeating it wastes the token budget
|
||||
|
||||
**Patterns**
|
||||
|
||||
**Gotchas** — highest value; place near the top:
|
||||
````markdown
|
||||
## Gotchas
|
||||
- <Fact that defies a reasonable assumption>
|
||||
- <Non-obvious naming discrepancy or hidden constraint>
|
||||
````
|
||||
|
||||
**Default with escape hatch** (not a menu):
|
||||
````markdown
|
||||
Use <X> for <task>. For <edge case>, use <Y> instead.
|
||||
````
|
||||
|
||||
**Prescriptive sequence** (when order is critical or fragile):
|
||||
````markdown
|
||||
Run exactly:
|
||||
```bash
|
||||
<command>
|
||||
```
|
||||
Do not modify flags.
|
||||
````
|
||||
|
||||
**Checklist** (multi-step workflows):
|
||||
````markdown
|
||||
- [ ] Step 1: ...
|
||||
- [ ] Step 2: ...
|
||||
````
|
||||
|
||||
**Conditional reference** (progressive disclosure — load only when needed):
|
||||
````markdown
|
||||
If <condition>, read `references/<file>.md`.
|
||||
````
|
||||
|
||||
**Output format template** (when the skill produces structured output):
|
||||
````markdown
|
||||
Output format:
|
||||
```
|
||||
<field>: <value>
|
||||
<field>: <value>
|
||||
```
|
||||
````
|
||||
For longer templates, place in `assets/<name>.md` and reference conditionally.
|
||||
|
||||
**Size budget**
|
||||
|
||||
Keep `SKILL.md` under 500 lines; 5,000 tokens is the recommended body budget. When approaching the limit:
|
||||
- Move reference material to `references/<topic>.md` and load it conditionally
|
||||
- Bundle repeated executable logic into `scripts/` rather than reinventing each run
|
||||
|
||||
### Step 4 — Add scripts (if needed)
|
||||
|
||||
Place executable scripts in `scripts/`. Critical rule: **no interactive prompts** — agents run non-interactive; blocking on TTY input hangs indefinitely. Accept all input via flags, env vars, or stdin.
|
||||
|
||||
If adding a script, read `references/scripts.md` first — it covers the full contract: structured output, pinned versions, self-contained deps, idempotency, exit codes, dry-run, error messages, and output size limits.
|
||||
|
||||
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
|
||||
|
||||
### Step 5 — Add references, assets, and tests (if needed)
|
||||
|
||||
**`references/`** — additional documentation loaded on demand. One topic per file.
|
||||
Reference conditionally from SKILL.md: `If <condition>, read references/<file>.md`.
|
||||
Keep reference chains one level deep — a reference file that references another reference file is rarely loaded correctly.
|
||||
|
||||
**`assets/`** — static resources: templates, schemas, lookup tables.
|
||||
Reference by relative path from SKILL.md.
|
||||
|
||||
**`tests/`** — test files for scripts in `scripts/`. Use when scripts are complex
|
||||
enough to break silently. Test infrastructure (`.bats`, `*_test.*`) belongs here,
|
||||
not in `scripts/`. See `tests/README.md` for setup instructions.
|
||||
|
||||
If not needed, delete the placeholder READMEs and their directories.
|
||||
|
||||
### Step 6 — Populate or delete `references/sources.md`
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
|
||||
1. Read it and filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, determine which skill files it contributed to (SKILL.md and any files in references/ that drew from it). Update `Contributing files` accordingly — list skill files, not research topic files.
|
||||
3. Write the updated content to `references/sources.md`. For each entry, include `- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the plugin-level research sources file this entry was drawn from (e.g. `plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
|
||||
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of sources that informed it.
|
||||
5. For each file in `references/` that was informed by research sources, add `source_keys` frontmatter (same format as research topic files) listing the relevant slugs.
|
||||
|
||||
If no research `sources.md` is in context, delete `references/sources.md`.
|
||||
|
||||
### Step 7 — Validate and close
|
||||
|
||||
Before running the audit, confirm:
|
||||
- [ ] Skill name matches the directory name exactly
|
||||
- [ ] `description` field is present and non-empty
|
||||
- [ ] Body has at least one non-empty section
|
||||
- [ ] No `FILL IN:` placeholders remain in any file
|
||||
|
||||
Run `/skill-audit` on the skill directory Step 1 reported — either `<package-root>/.apm/skills/<skill-name>/` or `<path>/<skill-name>/`.
|
||||
|
||||
All FAIL findings must be resolved before the skill is considered done.
|
||||
|
||||
If the skill is versioned (`metadata.version`), set it to the next **minor** version (e.g. `0.2.0` → `0.3.0`). New skills without a prior version start at `0.1.0`.
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` before Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the new skill files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the skill as done once the hash has actually changed.
|
||||
|
||||
## Improving an existing skill
|
||||
|
||||
### Step 1 — Verify inputs
|
||||
|
||||
Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.
|
||||
|
||||
If the skill dir is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
|
||||
Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
Also verify the `name` field in frontmatter matches the skill's directory name exactly.
|
||||
|
||||
### Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/, tests/). Then collect all signals from the conversation and any file paths the user has referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- Session context: output format is wrong on every run
|
||||
- Audit finding: no output template defined
|
||||
- User feedback: "I always have to ask it to format the output"
|
||||
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
|
||||
```
|
||||
|
||||
### Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state:
|
||||
- Which root causes were identified and what evidence supports each
|
||||
- Which files will be changed and what will change in each
|
||||
|
||||
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
### Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, tests/, README.md.
|
||||
|
||||
**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs.
|
||||
|
||||
**Keep it lean.** Remove instructions that aren't pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
|
||||
|
||||
If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Before running the audit, confirm:
|
||||
- [ ] Skill name still matches the directory name
|
||||
- [ ] No `FILL IN:` placeholders were introduced
|
||||
- [ ] No previously-passing audit checks were broken by the edits
|
||||
|
||||
Run `/skill-audit` on the skill directory. Resolve any FAIL findings before considering the improvement complete.
|
||||
|
||||
If the skill is versioned (`metadata.version`), bump the **patch** version (e.g. `0.1.0` → `0.1.1`).
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` at the start of Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the changed files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the improvement as done once the hash has actually changed.
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` — do not stop at staging. Re-run `git log --oneline -1` and confirm the hash changed from the one captured at Step 1. A non-empty `git diff --stat` is not proof: staged-but-uncommitted work is part of no commit and is silently lost if the tree is cleaned up first. Report done only once the hash has changed.
|
||||
|
||||
@@ -10,11 +10,24 @@ name: SKILL_NAME
|
||||
# Examples: my-tool, data-analyzer, pdf-processor
|
||||
|
||||
description: >
|
||||
FILL IN: What does this skill do? State capabilities specifically
|
||||
Use when FILL IN: trigger — when should an agent activate this skill?
|
||||
FILL IN: at most ONE capability clause, stated specifically
|
||||
(e.g. "parses and validates OpenAPI specs", not "helps with APIs").
|
||||
Use when FILL IN: when should an agent activate this skill?
|
||||
Include indirect triggers: even if the user doesn't mention X explicitly.
|
||||
Do not use when FILL IN: near-miss exclusions — remove this line if none apply.
|
||||
Not FILL IN: near-miss case -> FILL IN: real sibling skill name.
|
||||
# Required. Preloaded into EVERY session whether or not the skill is invoked.
|
||||
# Exactly three parts, in this order: trigger clause, at most one capability
|
||||
# clause, boundary clause. Drop the boundary line if no near-miss skill exists.
|
||||
# Budget: 250 characters target, 400 hard ceiling (counting this value only).
|
||||
# Banned here: capability lists, output-format detail, composition notes,
|
||||
# implementation detail, and restating one trigger twice in two registers.
|
||||
# The boundary target must resolve to a real skill or agent — it is checked.
|
||||
# Add "even if the user doesn't mention X explicitly" ONLY when the user's
|
||||
# natural phrasing genuinely omits the domain word.
|
||||
|
||||
# disable-model-invocation: true
|
||||
# Optional. Hand-invoked skills only (reached solely by the user typing
|
||||
# /SKILL_NAME). With this set, replace the description above with ONE plain
|
||||
# human-facing sentence — no trigger clause, no boundary clause.
|
||||
|
||||
# license: MIT
|
||||
# Optional. License name (e.g. MIT, Apache-2.0) or relative path to a bundled
|
||||
@@ -42,29 +55,30 @@ description: >
|
||||
---
|
||||
|
||||
<!-- ============================================================
|
||||
SKILL BODY
|
||||
SKILL BODY — decision procedure ONLY
|
||||
|
||||
Include only what the agent lacks:
|
||||
- Domain conventions the agent cannot infer from general knowledge
|
||||
- Non-obvious sequences or ordering constraints
|
||||
- One default per decision point + one escape hatch (never a menu)
|
||||
- Gotchas — facts that defy reasonable assumptions
|
||||
Keep here: ordered steps, decision branches, gates, and which reference
|
||||
file to load when.
|
||||
|
||||
Omit:
|
||||
- Concepts the agent already knows
|
||||
- Exhaustive option lists
|
||||
- Steps the agent handles independently
|
||||
- Restatements of the description
|
||||
Move to references/: lookup tables, spec restatements, output schemas,
|
||||
templates, example blocks, rationale prose, and anything only one branch
|
||||
reaches. Wire each one with the literal conditional form
|
||||
"If <condition>, read `references/<file>.md`." — a generic
|
||||
"see references/ for details" is a lint error.
|
||||
|
||||
Size budget: under 500 lines / 5000 tokens.
|
||||
Move reference material to references/ and load it conditionally.
|
||||
Bundle repeated executable logic into scripts/.
|
||||
Budget: 600 words target, 900 hard ceiling, counting THIS BODY ONLY
|
||||
(everything after the closing --- above). Separate from the whole-file
|
||||
spec backstop of 2,770 words / 500 lines — do not conflate them.
|
||||
|
||||
Delete this comment block before shipping.
|
||||
============================================================ -->
|
||||
|
||||
<!-- OPTIONAL: Gotchas section — highest-value content. Place near the top.
|
||||
Add facts that defy reasonable assumptions or non-obvious constraints.
|
||||
<!-- OPTIONAL but high-value: Gotchas. Place near the top — a gotcha read
|
||||
after the mistake is worthless.
|
||||
|
||||
Each entry states a fact that CONTRADICTS a reasonable default:
|
||||
something the agent gets wrong by acting sensibly. Maximum 5 entries.
|
||||
An entry that paraphrases a step below it is a failure, not a gotcha.
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -72,7 +86,22 @@ description: >
|
||||
- FILL IN: non-obvious naming discrepancy or hidden constraint
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Multi-step workflow checklist.
|
||||
<!-- DISPATCH — MANDATORY when this skill has two or more mutually exclusive
|
||||
flows. Keep only the dispatch table plus the gates common to every
|
||||
branch in this body; give each flow its own self-contained
|
||||
references/ file. Delete this block for a single-flow skill.
|
||||
|
||||
## Step 1 — Dispatch
|
||||
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| FILL IN: condition | FILL IN: flow | `references/FILL IN.md` |
|
||||
| FILL IN: condition | FILL IN: flow | `references/FILL IN.md` |
|
||||
|
||||
Read only the reference matching the resolved flow — each is self-contained.
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: single-flow workflow checklist. Delete if the skill dispatches.
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -81,23 +110,12 @@ description: >
|
||||
- [ ] Step 3: FILL IN
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Output format template — use when the agent must produce a specific format.
|
||||
<!-- OPTIONAL: gates that apply to every branch — validation, versioning,
|
||||
closing checks. Keep these in the body even when flows are dispatched.
|
||||
|
||||
## Output format
|
||||
## Step N — Validate and close
|
||||
|
||||
Use this structure:
|
||||
|
||||
```markdown
|
||||
# [FILL IN: Title]
|
||||
|
||||
## FILL IN: Section
|
||||
FILL IN: what goes here
|
||||
```
|
||||
-->
|
||||
|
||||
<!-- OPTIONAL: Conditional reference — load documentation only when needed.
|
||||
|
||||
If FILL IN: condition, read `references/FILL IN: filename.md`.
|
||||
FILL IN: the check that must pass before this skill reports done.
|
||||
-->
|
||||
|
||||
## FILL IN: <section-name (e.g. Step 1, Workflow, Instructions)>
|
||||
|
||||
@@ -5,9 +5,17 @@ without bloating its core context.
|
||||
|
||||
## When to add a reference file
|
||||
|
||||
Move content here when SKILL.md is approaching 500 lines, or when a topic
|
||||
is only relevant in specific circumstances (error handling, edge cases,
|
||||
domain-specific sub-procedures).
|
||||
The SKILL.md body carries the decision procedure only. Everything else lives
|
||||
here: lookup tables, spec restatements, output schemas, templates, example
|
||||
blocks, rationale prose, and anything only one branch reaches.
|
||||
|
||||
Two triggers make a reference file mandatory rather than optional:
|
||||
|
||||
- The body is over its 600-word target (900 is a hard failure), counting the
|
||||
body only — everything after the frontmatter's closing `---`.
|
||||
- The skill has two or more mutually exclusive flows. The body then keeps only
|
||||
a dispatch table plus the gates common to every branch, and each flow gets
|
||||
its own self-contained file here (e.g. `create.md`, `improve.md`).
|
||||
|
||||
## How to reference from SKILL.md
|
||||
|
||||
|
||||
187
plugins/kyberforge/skills/skill-author/references/contract.md
Normal file
187
plugins/kyberforge/skills/skill-author/references/contract.md
Normal file
@@ -0,0 +1,187 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
---
|
||||
|
||||
# The description and body contract
|
||||
|
||||
House contract, set by ADR-0020. Every rule here is enforced by `/skill-audit` —
|
||||
`scripts/validate.sh` for the counts and the boundary targets, the bundled Vale styles for the
|
||||
prose patterns, and its reference files for the judgment calls.
|
||||
|
||||
## Why the budget exists
|
||||
|
||||
A skill's `name` and `description` are loaded into every agent's context at the start of every
|
||||
session, whether or not the skill is ever invoked. The body is loaded only on invocation, and then
|
||||
competes with the caller's live conversation. Those are two different costs, so they get two
|
||||
different ceilings — and a fat description is not merely expensive. A description that summarizes
|
||||
the workflow gets followed *instead of* the body: a description saying "code review between tasks"
|
||||
produced one review from a skill whose flowchart specified two.
|
||||
|
||||
## Description
|
||||
|
||||
A description carries exactly three things:
|
||||
|
||||
1. **Trigger clause** — when to invoke, imperative: "Use when ...", never "This skill ...".
|
||||
Focus on user intent, not the skill's internal mechanics.
|
||||
2. **At most one capability clause** — what it does, one clause, no enumeration. Be specific
|
||||
("parses and validates OpenAPI specs", not "helps with APIs").
|
||||
3. **Boundary clause** — form: `Not <thing> -> <skill-name>.` Add one only where a near-miss skill
|
||||
could steal activations.
|
||||
|
||||
Banned from a description; move it to the body or to `README.md`:
|
||||
|
||||
- Capability enumeration or feature lists
|
||||
- Output-format detail ("Produces a compact findings report with Why and Fix per finding")
|
||||
- Composition or architecture notes ("composes X rather than duplicating Y", "This is a
|
||||
cross-cutting shared skill", "the human-facing entry point")
|
||||
- Implementation detail ("Self-validates via a bundled deterministic script")
|
||||
- Restating the same trigger twice in two registers — a verb list, then the same verbs re-quoted
|
||||
as user phrasings. This is a FAIL, not a suggestion.
|
||||
|
||||
**Indirect triggers are conditional, not mandatory.** Add "even if the user doesn't mention X
|
||||
explicitly" only where the user's natural phrasing genuinely omits the domain word — true for the
|
||||
`gitea-*` family, because people say "create an issue" rather than "create a Gitea issue"; false
|
||||
for `git-commits`, where the user says "commit". Adding one everywhere is what inflated this
|
||||
corpus, and it was deleted as a blanket rule.
|
||||
|
||||
**Boundary targets must resolve.** The name after the arrow is checked against real skill
|
||||
directories under `plugins/*/.apm/skills/<name>/` and real agents under
|
||||
`plugins/*/.apm/agents/<name>.agent.md`. A boundary clause naming a target that does not exist
|
||||
sends the router nowhere and fails the audit. Check the target exists before writing it — do not
|
||||
invent a plausible sibling name.
|
||||
|
||||
**Length.** 250 characters SUGGESTION, 400 characters FAIL, counting the frontmatter value only
|
||||
with YAML folding resolved. The agentskills.io 1,024-character spec limit is unchanged and sits
|
||||
above both. The SUGGESTION tier is the one that moves the average; treat 250 as the target and 400
|
||||
as the outlier stop.
|
||||
|
||||
**Hand-invoked skills are exempt.** A skill carrying `disable-model-invocation: true` is absent
|
||||
from the model-visible listing and is reached only by the user typing `/name`. It takes one plain
|
||||
human-facing sentence — no trigger clause, no boundary clause, no indirect triggers. Worked
|
||||
example: `plugins/bin/.apm/skills/zoom-out/SKILL.md`.
|
||||
|
||||
## Body
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when. Everything else moves to `references/`.
|
||||
|
||||
Ask of every sentence: "Would the agent get this wrong without it?" Cut anything that answers "no."
|
||||
|
||||
Include:
|
||||
|
||||
- Non-obvious sequences or ordering constraints — the agent may skip or reorder steps without this
|
||||
- Domain conventions the agent cannot infer from general knowledge — the core value a skill adds
|
||||
- One default per decision point, plus one escape hatch — never a menu; menus cause the agent to
|
||||
pause or pick arbitrarily
|
||||
- Gotchas — facts that defy reasonable assumptions
|
||||
|
||||
Exclude:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works) — tokens without behavior change
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads agents down unproductive paths
|
||||
- Restatements of the description — it is already in context
|
||||
|
||||
Move to `references/`: lookup tables, spec restatements, output schemas, templates, example
|
||||
blocks, rationale prose, and any content only one branch reaches. Each reference file is
|
||||
self-contained for its concern, and every one is wired from the body with the literal conditional
|
||||
form:
|
||||
|
||||
````markdown
|
||||
If <condition>, read `references/<file>.md`.
|
||||
````
|
||||
|
||||
A generic pointer ("see references/ for details") is a Vale error — the agent cannot act on it.
|
||||
|
||||
**Dispatch is mandatory at two or more mutually exclusive flows.** The body carries the dispatch
|
||||
table and the gates common to every branch; each flow gets its own self-contained `references/`
|
||||
file. Exemplar: `plugins/kyberforge/.apm/skills/apm-workflow/SKILL.md` — a 554-word body
|
||||
dispatching to 3,006 words of references.
|
||||
|
||||
**Length.** 600 words SUGGESTION, 900 words FAIL, counting the **body only** — everything after
|
||||
the frontmatter's closing `---`.
|
||||
|
||||
## Gotchas section
|
||||
|
||||
- Each entry must state a fact that **contradicts a reasonable default** — something the agent
|
||||
gets wrong by acting sensibly. "Never commit secrets" is not one; the agent already knows.
|
||||
- Maximum five entries.
|
||||
- A Gotcha that paraphrases a step in the body below it is a **FAIL**. If the rule is already a
|
||||
step, it is not a gotcha.
|
||||
- A Gotchas section exceeding 25% of the body is a SUGGESTION.
|
||||
- Place the section near the top — a gotcha read after the mistake is worthless.
|
||||
|
||||
## Two size gates, two measurements
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| description | 250 chars | 400 chars | the `description:` value only |
|
||||
| body | 600 words | 900 words | the body only, after the closing `---` |
|
||||
| spec backstop | — | 1,024 chars | the `description:` value only |
|
||||
| spec backstop | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 600/900 pair and the 2,770/500 pair are not the same measurement and must not be unified: the
|
||||
first is a quality gate on what the caller's context absorbs, the second a conformance backstop on
|
||||
the file. A skill can sit well inside one and fail the other.
|
||||
|
||||
When a body approaches its ceiling, relocate rather than delete — move reference material to
|
||||
`references/<topic>.md` behind a conditional trigger, and bundle repeated executable logic into
|
||||
`scripts/` rather than reinventing it each run.
|
||||
|
||||
## Body patterns
|
||||
|
||||
**Default with escape hatch** (not a menu):
|
||||
|
||||
````markdown
|
||||
Use <X> for <task>. For <edge case>, use <Y> instead.
|
||||
````
|
||||
|
||||
**Prescriptive sequence** (when order is critical or fragile):
|
||||
|
||||
````markdown
|
||||
Run exactly:
|
||||
```bash
|
||||
<command>
|
||||
```
|
||||
Do not modify flags.
|
||||
````
|
||||
|
||||
**Checklist** (multi-step workflows):
|
||||
|
||||
````markdown
|
||||
- [ ] Step 1: ...
|
||||
- [ ] Step 2: ...
|
||||
````
|
||||
|
||||
**Dispatch table** (two or more mutually exclusive flows):
|
||||
|
||||
````markdown
|
||||
| Condition | Flow | Reference |
|
||||
|---|---|---|
|
||||
| <condition> | <flow> | `references/<file>.md` |
|
||||
````
|
||||
|
||||
**Output format template** (when the skill produces structured output):
|
||||
|
||||
````markdown
|
||||
Output format:
|
||||
```
|
||||
<field>: <value>
|
||||
```
|
||||
````
|
||||
|
||||
For longer templates, place them in `assets/<name>.md` and reference conditionally.
|
||||
|
||||
## Embedding org-specific policy
|
||||
|
||||
If a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`),
|
||||
inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing
|
||||
to the file's path. Plugins must be self-contained and portable — the org file may not exist
|
||||
wherever the plugin is installed, and in this repo such files are meant to be deleted once their
|
||||
content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the
|
||||
same `references/sources.md` schema as the create flow's Step 6, noting in the `Research doc:`
|
||||
field that the source is an org convention rather than a plugin research corpus entry, so
|
||||
provenance survives after the source file is gone.
|
||||
173
plugins/kyberforge/skills/skill-author/references/create.md
Normal file
173
plugins/kyberforge/skills/skill-author/references/create.md
Normal file
@@ -0,0 +1,173 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-quickstart
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Creating a new skill
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 6 below is done — validation, versioning and commit
|
||||
verification are shared with the improve flow and are not repeated here.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Run `/grill-me` on the skill's design and research the target domain first. Share those outputs
|
||||
in this conversation: grill context, research docs, examples, constraints.
|
||||
|
||||
Design for one coherent user intent — skills too narrow force multiple loads per task; too broad
|
||||
are hard to activate precisely.
|
||||
|
||||
Before touching the filesystem, verify you have:
|
||||
|
||||
- [ ] A clear purpose — what specific task will this skill handle?
|
||||
- [ ] Trigger scenarios — when should an agent activate it?
|
||||
- [ ] Skill name (kebab-case) and destination path
|
||||
|
||||
If any are missing, stop and ask the user before proceeding.
|
||||
|
||||
**Requires `/skill-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
|
||||
the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the
|
||||
user to install the kyberforge plugin before continuing.
|
||||
|
||||
## Package-intent gate
|
||||
|
||||
Judge whether the destination is meant to be inside an APM package before running the scaffold
|
||||
script — the script cannot tell "no package here" apart from "package not scaffolded yet":
|
||||
|
||||
- Package intent but no `type:`-bearing `apm.yml` found at or above the destination (e.g. "add to
|
||||
my apm package", or a sibling `.apm/`/`apm.yml` exists nearby) → **stop**, tell the user to run
|
||||
`/apm-workflow configure` (`apm plugin init`, from inside the package directory) first, then
|
||||
retry. Do not fall through to standalone mode.
|
||||
- Otherwise (a `~/`-rooted destination, or no package context implied) → continue to Step 1.
|
||||
|
||||
## Step 1 — Scaffold
|
||||
|
||||
Run the copy script with the skill name and a path inside or at the target:
|
||||
|
||||
```bash
|
||||
bash scripts/new-skill.sh <skill-name> <path>
|
||||
```
|
||||
|
||||
The script walks up from `<path>` for a package boundary: an ancestor `apm.yml` with a top-level
|
||||
`type:` field (`instructions`/`skill`/`hybrid`/`prompts`) means **package mode** — scaffolds into
|
||||
`<package-root>/.apm/skills/<skill-name>/`, not under `<path>` (a subdirectory of the package
|
||||
works fine as `<path>`). A `type:`-less `apm.yml` is a marketplace-only manifest, skipped. Hitting
|
||||
`.git` or the filesystem root first means **standalone mode** — scaffolds directly into
|
||||
`<path>/<skill-name>/`.
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
# Package mode — packages/my-pkg/apm.yml already has `type: skill`
|
||||
bash scripts/new-skill.sh my-tool packages/my-pkg/
|
||||
|
||||
# Standalone mode — no apm.yml/.git above ~/.agents/skills/
|
||||
bash scripts/new-skill.sh my-tool ~/.agents/skills/
|
||||
```
|
||||
|
||||
The script prints which mode it used and where the skill landed — read its output.
|
||||
|
||||
In package mode, read `references/deployment-modes.md` before adding any file references to
|
||||
SKILL.md.
|
||||
|
||||
## Step 2 — Update `apm.yml` includes (package mode only)
|
||||
|
||||
Skip in standalone mode. In package mode, check the resolved package's `apm.yml`: if `includes:`
|
||||
is an explicit list (not `auto`), append `.apm/skills/<skill-name>/` to it if not already present,
|
||||
preserving YAML formatting. If `includes: auto` or the field is absent, do nothing — `auto`
|
||||
already covers the new skill. Use Read/Edit directly on `apm.yml`; this is not part of
|
||||
`scripts/new-skill.sh`.
|
||||
|
||||
## Step 3 — Fill in SKILL.md
|
||||
|
||||
Open the new skill's `SKILL.md` (the path Step 1 printed) and replace every `FILL IN:`
|
||||
placeholder. The scaffold template already carries the compliant frontmatter and body skeleton —
|
||||
fill it rather than restructuring it.
|
||||
|
||||
**`name`** — already set by the scaffold script. Must exactly match the directory name. Format:
|
||||
1–64 characters, lowercase letters, numbers and hyphens only; no leading, trailing or consecutive
|
||||
hyphens (`--`).
|
||||
|
||||
**`description`** — carries the entire triggering burden and is preloaded every session. Write it
|
||||
against `references/contract.md`, which holds the three-part shape, the banned content, the
|
||||
boundary-clause form and the length tiers. A hand-invoked skill (`SKILL.md` Step 2) takes one
|
||||
plain sentence and `disable-model-invocation: true` instead.
|
||||
|
||||
**Optional frontmatter** — uncomment and fill in, or remove entirely:
|
||||
|
||||
- `license` — include when distributing the skill externally
|
||||
- `compatibility` — include if the skill requires specific tools, runtimes, or network access
|
||||
(max 500 characters)
|
||||
- `metadata` — key-value map; use `author`, `version`, `category`; add `source_keys` now (Step 6)
|
||||
if research sources are in context
|
||||
- `allowed-tools` — space-separated pre-approved tools; reduces permission prompts (experimental —
|
||||
support varies by client)
|
||||
- `disable-model-invocation` — hand-invoked skills only
|
||||
|
||||
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs as you
|
||||
write the body; do not defer this to Step 6. Agents that fill in `source_keys` late tend to omit
|
||||
it entirely. Example:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
source_keys:
|
||||
- my-source-slug
|
||||
- another-slug
|
||||
```
|
||||
|
||||
**Body** — write the decision procedure only, following the body rules and patterns in
|
||||
`references/contract.md`. Rename the placeholder section headings to ones that fit the skill's
|
||||
structure.
|
||||
|
||||
## Step 4 — Add scripts (if needed)
|
||||
|
||||
Place executable scripts in `scripts/`. Critical rule: **no interactive prompts** — agents run
|
||||
non-interactive, and blocking on TTY input hangs indefinitely. Accept all input via flags, env
|
||||
vars, or stdin.
|
||||
|
||||
If adding a script, read `references/scripts.md` first — it covers the full contract: structured
|
||||
output, pinned versions, self-contained deps, idempotency, exit codes, dry-run, error messages,
|
||||
and output size limits.
|
||||
|
||||
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
|
||||
|
||||
## Step 5 — Add references, assets, and tests (if needed)
|
||||
|
||||
**`references/`** — additional documentation loaded on demand. One topic per file. Reference
|
||||
conditionally from SKILL.md with the literal form ``If <condition>, read `references/<file>.md` ``.
|
||||
Keep reference chains one level deep — a reference file that references another reference file is
|
||||
rarely loaded correctly.
|
||||
|
||||
**`assets/`** — static resources: templates, schemas, lookup tables. Reference by relative path
|
||||
from SKILL.md.
|
||||
|
||||
**`tests/`** — test files for scripts in `scripts/`. Use when scripts are complex enough to break
|
||||
silently. Test infrastructure (`.bats`, `*_test.*`) belongs here, not in `scripts/`. See
|
||||
`tests/README.md` for setup instructions.
|
||||
|
||||
If not needed, delete the placeholder READMEs and their directories.
|
||||
|
||||
## Step 6 — Populate or delete `references/sources.md`
|
||||
|
||||
If a research `sources.md` is present in the conversation context:
|
||||
|
||||
1. Read it and filter to entries with `` `extracted` `` status only.
|
||||
2. For each entry, determine which skill files it contributed to (SKILL.md and any files in
|
||||
`references/` that drew from it). Update `Contributing files` accordingly — list skill files,
|
||||
not research topic files.
|
||||
3. Write the updated content to `references/sources.md`. For each entry, include
|
||||
`- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the
|
||||
plugin-level research sources file this entry was drawn from (e.g.
|
||||
`plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every
|
||||
entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
|
||||
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of
|
||||
sources that informed it.
|
||||
5. For each file in `references/` that was informed by research sources, add `source_keys`
|
||||
frontmatter (same format as research topic files) listing the relevant slugs.
|
||||
|
||||
If no research `sources.md` is in context, delete `references/sources.md`.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
82
plugins/kyberforge/skills/skill-author/references/improve.md
Normal file
82
plugins/kyberforge/skills/skill-author/references/improve.md
Normal file
@@ -0,0 +1,82 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-best-practices
|
||||
- agentskills-evaluating-skills
|
||||
- agentskills-optimizing-descriptions
|
||||
---
|
||||
|
||||
# Improving an existing skill
|
||||
|
||||
Return to `SKILL.md` Step 4 once Step 4 below is done — validation, versioning and commit
|
||||
verification are shared with the create flow and are not repeated here.
|
||||
|
||||
## Step 1 — Verify inputs
|
||||
|
||||
Confirm the skill directory path exists and that at least one improvement signal is present in the
|
||||
conversation or a referenced file.
|
||||
|
||||
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
|
||||
existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
Also verify the `name` field in frontmatter matches the skill's directory name exactly.
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in `scripts/`, `references/`, `assets/`,
|
||||
`tests/`). Then collect all signals from the conversation and any file paths the user has
|
||||
referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Patching per symptom is the default failure mode:
|
||||
three eval failures may all trace to one missing instruction. Ask: "What single gap in the skill
|
||||
causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each
|
||||
symptom.
|
||||
|
||||
```text
|
||||
Example:
|
||||
- Session context: output format is wrong on every run
|
||||
- Audit finding: no output template defined
|
||||
- User feedback: "I always have to ask it to format the output"
|
||||
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
|
||||
```
|
||||
|
||||
## Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state:
|
||||
|
||||
- Which root causes were identified and what evidence supports each
|
||||
- Which files will be changed and what will change in each
|
||||
|
||||
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, `scripts/`,
|
||||
`references/`, `assets/`, `tests/`, README.md.
|
||||
|
||||
**Generalize, do not patch.** Find the underlying gap, not the specific example that failed. A fix
|
||||
scoped only to the test cases you have seen will overfit and perform worse on new inputs.
|
||||
|
||||
**Keep it lean.** Remove instructions that are not pulling their weight. For every sentence you
|
||||
add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently
|
||||
outperforms an exhaustive one.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself
|
||||
writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the
|
||||
agent can apply judgment in edge cases.
|
||||
|
||||
**Retrofit before extending.** Any edit to a skill that predates ADR-0020 has to bring it into the
|
||||
contract first — the gates are hot and carry no baseline file, so a one-line fix to a
|
||||
non-compliant skill cannot be committed until the description and body meet
|
||||
`references/contract.md`. Treat that retrofit as part of the same change, not a follow-up.
|
||||
|
||||
If a signal points to a script or reference file, edit that file directly rather than adding a
|
||||
workaround in SKILL.md.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -18,7 +18,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/home.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-spec
|
||||
@@ -26,7 +26,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/deployment-modes.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/contract.md, references/deployment-modes.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
@@ -34,7 +34,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
@@ -42,7 +42,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-evaluating-skills
|
||||
@@ -50,7 +50,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/improve.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-using-scripts
|
||||
@@ -58,7 +58,7 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md, references/scripts.md
|
||||
- **Contributing files:** SKILL.md, references/create.md, references/scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
@@ -66,5 +66,5 @@ source_keys:
|
||||
- **URL:** https://agentskills.io/skill-creation/quickstart.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Contributing files:** SKILL.md, references/create.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user