Defame1297
4a5c3c0cff
Skill name+description pairs are preloaded into every session, costing ~6,200 tokens across 39 skills before any skill is invoked. The authoring rules mandated that growth: skill-author:104 and description-quality.md:21 both required padding, while skill-author:102 (the deflating rule) had no FAIL condition behind it. Gates (blocking, no baseline file): - description 250 chars SUGGESTION / 400 FAIL, measured on the folded YAML value - body-only 600 words SUGGESTION / 900 FAIL, independent of the unchanged whole-file 2770-word / 500-line spec backstop - every boundary-clause routing target must resolve to a real skill or agent; catches skill-improve, neuledge-context and gitea-labels - agents take the description gates but deliberately no body gate; a test pins that absence Vale: DescriptionOpener widened to ^This\b, new CompositionNote rule banning architecture notes from descriptions. 10 hits, 0 false positives. Kyberforge's own four skills retrofitted: descriptions 3,364 -> 938 chars (-72%), bodies 8,306 -> 2,487 words (-70%), all via the apm-workflow dispatch pattern. Fixes the skill-improve dangling route and the agent-author misroute to manual review. Also fixes a pre-existing false positive where any line-initial 'read ' was flagged as interactive input, which had already caused two scripts to be rewritten around it. Refs: ADR-0020
79 lines
6.0 KiB
Markdown
79 lines
6.0 KiB
Markdown
# agent-audit
|
|
|
|
Audits an agent definition for correctness and quality against the Claude Code and Copilot agent
|
|
references and the house context-budget contract (ADR-0020) — a single vendor-neutral file at
|
|
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
|
|
|
|
## What it does
|
|
|
|
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance
|
|
checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags
|
|
non-imperative description openers, composition and architecture notes, vague wording, padding
|
|
phrases, "There is/are" sentence openers, and CC-specific "Use proactively" phrasing in a
|
|
Copilot or vendor-neutral description
|
|
2. Reads the agent file, and its counterpart when one exists, then loads the contract for its scope
|
|
3. Applies qualitative checks across description, body, delegation and comment discipline, loading
|
|
one rubric from `references/` per group
|
|
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix —
|
|
and a result block with handoff to `agent-author`
|
|
|
|
Two things follow from ADR-0020 and are easy to get backwards. Agents take the **same** description
|
|
gates a skill takes — 250 characters SUGGESTION, 400 FAIL, since a `name` + `description` is
|
|
preloaded into every session either way — and **no body word gate at all**, because an agent body
|
|
becomes the system prompt of a fresh context rather than competing with the caller's live
|
|
conversation. Body length is judged through the delegation check instead: an agent body that
|
|
restates a procedure owned by a skill it can invoke is a FAIL, because a plugin-scope agent has no
|
|
sibling `references/` directory to disclose to and can only delegate.
|
|
|
|
At **plugin/APM scope** the audit accepts the single `.apm/agents/<name>.agent.md` file — there is
|
|
no counterpart, and pair consistency does not apply. `validate.sh` hard-`FAIL`s any frontmatter
|
|
field outside the vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both
|
|
harnesses and an unsafe field cannot be silently dropped for just one of them. The allowlist lives
|
|
in the `apm-agent-allowlist` section of `references/field-inventory.md`, is read from there as data
|
|
by the script, and is deliberately not restated anywhere else in this skill (ADR-0009).
|
|
|
|
At **project/user scope** the audit accepts either file in a CC `.md` / Copilot `.agent.md` pair,
|
|
derives the counterpart automatically, and validates both, including the field-leakage checks in
|
|
each direction.
|
|
|
|
## Usage
|
|
|
|
```
|
|
/agent-audit
|
|
```
|
|
|
|
Pass the path to either agent file as the argument.
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `SKILL.md` | Skill instructions for agents |
|
|
| `assets/vale/.vale.ini` | Vale config: scopes `Kyberforge` to `**/agents/*.md`, `Kyberforge`+`KyberforgeCopilot` to `**/*.agent.md` |
|
|
| `assets/vale/styles/Kyberforge/CompositionNote.yml` | Flags composition and architecture notes in a description ("cross-cutting", "entry point", "composes", "rather than duplicating") that belong in README.md |
|
|
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This..." instead of an imperative "Use when..." |
|
|
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Flags generic "see references/ for info" pointers instead of specific file references |
|
|
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Flags sentences opening with "There is/are" instead of naming the subject directly |
|
|
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
|
|
| `assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml` | Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
|
|
| `references/README.md` | Directory documentation for references/ |
|
|
| `references/description-quality.md` | Rubric for the description dimension — three-part shape, the 250/400-character budget, the hand-invoked contract, and the internal-mechanics FAIL |
|
|
| `references/body-and-delegation.md` | Rubric for the body, delegation and comment-discipline dimensions — the delegation FAIL and why agents take no body word gate |
|
|
| `references/scope-plugin-apm.md` | Scope contract for a single vendor-neutral APM agent file — allowlist, dimension routing, and the dimensions that do not apply |
|
|
| `references/scope-project-user.md` | Scope contract for a CC / Copilot pair — counterpart derivation, provider field rules, pair consistency |
|
|
| `references/validation-scripts.md` | Loaded only when a Step 1 script fails or cannot run — scope-detection walk-up, manual fallback checks, known script failures |
|
|
| `references/field-inventory.md` | Authoritative field lists read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM-scope allowlist |
|
|
| `references/sources.md` | Research provenance for skill content |
|
|
| `scripts/README.md` | Directory documentation for scripts/ |
|
|
| `scripts/validate.sh` | Structural validator — required fields, name format, placeholder detection, the ADR-0020 description budget, and the field rules for the detected scope |
|
|
| `scripts/validate-provenance.sh` | Provenance chain validation against `sources.md` at the package root (plugin/APM scope only) |
|
|
| `scripts/vale-wrap.sh` | Drop-in `vale` wrapper that works around a frontmatter-description NLP scope limitation |
|
|
| `tests/README.md` | (source-only) Bats test dependency and run instructions |
|
|
| `tests/validate.bats` | (source-only) Bats tests for validate.sh |
|
|
| `tests/validate-provenance.bats` | (source-only) Bats tests for validate-provenance.sh |
|
|
|
|
Rows marked **(source-only)** exist in the authoring source (`.apm/skills/agent-audit/`) but are
|
|
not present in an installed plugin: `scripts/sync-plugin-content.sh` strips
|
|
`<category>/<name>/tests` when it generates the flat mirror, because these are dev-time fixtures no
|
|
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.
|