Defame1297
f4bb1cf4e5
The previous round taught agent-audit's validator to permit disallowedTools but left the skill that writes agents still forbidding it, in six places. Running agent-author on any of the three fenced orchestrators would have stripped the fence, and nothing would have caught it: the validator's allowlist is a permit list, so an absent field passes. The template was the worst of them, since its comment is copied verbatim into every new plugin-scope agent. Where a list had to be restated it is now a pointer to field-inventory.md's apm-agent-allowlist instead -- the same data validate.sh reads -- because a roster copied into a template goes stale one step further out than the roster itself. Where the text has to teach something it teaches the shape rule rather than the exception: tools is an allowlist whose vocabulary differs per harness, so verbatim copy makes one value wrong on one target; disallowedTools is a denylist, where an unrecognised name denies nothing, so the worst case is a missing fence rather than a wrongly granted capability. ADR-0016's amendment claimed an unrecognised key is inert on Copilot while the same ADR's Context says that behaviour is unconfirmed by research -- asserting as settled the exact thing it flags as unknown, and justifying it with apm's compile-time behaviour, which says nothing about Copilot's runtime. It is rewritten into labelled tiers: confirmed for Claude Code with citations, inferred by analogy for Copilot with the analogy's limits stated, unverified where it is unverified, and the residual risk accepted explicitly with its blast radius. It also no longer claims to restore a write sandbox: the denylist does not deny Bash, which these agents inherit and legitimately need. docs/hooks.md called the old root hooks.json a stale sync artifact -- it was added in the plugin's creating commit and pointed at by main's Copilot manifest -- and claimed both ecosystems now resolve hooks/hooks.json. Copilot does not: its hooks field has no default and no compiled manifest declares one, so it resolves nothing. Recorded as the gap it is, with re-injection noted as a follow-up rather than asserted away. Its event list is marked partial. Also: new-agent.bats asserted a hardcoded four-field allowlist and would have rejected a scaffolded agent carrying the field the ADR now blesses; it reads field-inventory.md too. And ADR-0016's premise that Claude's tools: is space-separated was wrong -- it takes a comma-separated string or a YAML list. The incompatibility with Copilot is the vocabulary, not the punctuation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
4.4 KiB
agent-audit
Audits an agent definition for correctness and quality — a single vendor-neutral file at plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
What it does
At plugin/APM scope, accepts the single .apm/agents/<name>.agent.md file — there is no
counterpart. Structural checks via validate.sh hard-FAIL any frontmatter field outside the
vendor-neutral allowlist, since apm compile copies frontmatter verbatim to both harnesses and an
unsafe field can't be silently dropped for just one of them. The allowlist itself lives in the
apm-agent-allowlist section of references/field-inventory.md and is read from there as data —
consult that section rather than any restatement of it, including this one. As of 2026-08-14 it
admits name, description, model, source_keys, and disallowedTools; source_keys is
provenance metadata checked separately by validate-provenance.sh against sources.md, and
disallowedTools is admitted because a denylist survives verbatim copy where the tools allowlist
does not (ADR-0016 and its 2026-08-14 amendment).
At project/user scope, accepts either file in a CC .md / Copilot .agent.md pair, derives
the counterpart automatically, and validates both. Runs structural checks via validate.sh
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
Copilot-only fields in the CC file), provenance chain validation via validate-provenance.sh
(checks source_keys against sources.md at the plugin root — plugin/APM scope only), then
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
Vale-based prose sub-check via vale-wrap.sh against both files of the pair, using the
Kyberforge style (both files) and KyberforgeCopilot style (Copilot file only) — every alert
is a FAIL, cited by rule ID — falling back to Step 2 judgment when the vale binary is
unavailable or reports 0 files scanned. Produces a compact findings report in the same format
as skill-audit.
Usage
/agent-audit
Pass the path to either agent file as the argument.
Files
| File | Purpose |
|---|---|
SKILL.md |
Skill instructions for agents |
assets/vale/.vale.ini |
Vale config: scopes Kyberforge to **/agents/*.md, Kyberforge+KyberforgeCopilot to **/*.agent.md |
assets/vale/styles/Kyberforge/DescriptionOpener.yml |
Flags descriptions opening with "This skill/agent" instead of an imperative "Use when..." |
assets/vale/styles/Kyberforge/PaddingPhrase.yml |
Flags generic "see references/ for info" pointers instead of specific file references |
assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml |
Flags sentences opening with "There is/are" instead of naming the subject directly |
assets/vale/styles/Kyberforge/VagueWording.yml |
Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml |
Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
references/README.md |
Directory documentation for references/ |
references/description-quality.md |
Qualitative guide for borderline description findings |
references/field-inventory.md |
Authoritative field lists read as data by validate.sh: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM-scope allowlist |
references/sources.md |
Research provenance for skill content |
scripts/README.md |
Directory documentation for scripts/ |
scripts/validate.sh |
Structural validation script for agent file pairs |
scripts/validate-provenance.sh |
Provenance chain validation script for agent pairs against sources.md (plugin root) |
scripts/vale-wrap.sh |
Drop-in vale wrapper that works around a frontmatter-description NLP scope limitation |
tests/README.md |
(source-only) Bats test dependency and run instructions |
tests/validate.bats |
(source-only) Bats tests for validate.sh |
tests/validate-provenance.bats |
(source-only) Bats tests for validate-provenance.sh |
Rows marked (source-only) exist in the authoring source (.apm/skills/agent-audit/) but are
not present in an installed plugin: scripts/sync-plugin-content.sh strips
<category>/<name>/tests when it generates the flat mirror, because these are dev-time fixtures no
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.