Defame1297
f4bb1cf4e5
The previous round taught agent-audit's validator to permit disallowedTools but left the skill that writes agents still forbidding it, in six places. Running agent-author on any of the three fenced orchestrators would have stripped the fence, and nothing would have caught it: the validator's allowlist is a permit list, so an absent field passes. The template was the worst of them, since its comment is copied verbatim into every new plugin-scope agent. Where a list had to be restated it is now a pointer to field-inventory.md's apm-agent-allowlist instead -- the same data validate.sh reads -- because a roster copied into a template goes stale one step further out than the roster itself. Where the text has to teach something it teaches the shape rule rather than the exception: tools is an allowlist whose vocabulary differs per harness, so verbatim copy makes one value wrong on one target; disallowedTools is a denylist, where an unrecognised name denies nothing, so the worst case is a missing fence rather than a wrongly granted capability. ADR-0016's amendment claimed an unrecognised key is inert on Copilot while the same ADR's Context says that behaviour is unconfirmed by research -- asserting as settled the exact thing it flags as unknown, and justifying it with apm's compile-time behaviour, which says nothing about Copilot's runtime. It is rewritten into labelled tiers: confirmed for Claude Code with citations, inferred by analogy for Copilot with the analogy's limits stated, unverified where it is unverified, and the residual risk accepted explicitly with its blast radius. It also no longer claims to restore a write sandbox: the denylist does not deny Bash, which these agents inherit and legitimately need. docs/hooks.md called the old root hooks.json a stale sync artifact -- it was added in the plugin's creating commit and pointed at by main's Copilot manifest -- and claimed both ecosystems now resolve hooks/hooks.json. Copilot does not: its hooks field has no default and no compiled manifest declares one, so it resolves nothing. Recorded as the gap it is, with re-injection noted as a follow-up rather than asserted away. Its event list is marked partial. Also: new-agent.bats asserted a hardcoded four-field allowlist and would have rejected a scaffolded agent carrying the field the ADR now blesses; it reads field-inventory.md too. And ADR-0016's premise that Claude's tools: is space-separated was wrong -- it takes a comma-separated string or a YAML list. The incompatibility with Copilot is the vocabulary, not the punctuation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
66 lines
4.4 KiB
Markdown
66 lines
4.4 KiB
Markdown
# agent-audit
|
|
|
|
Audits an agent definition for correctness and quality — a single vendor-neutral file at
|
|
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
|
|
|
|
## What it does
|
|
|
|
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
|
|
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
|
|
vendor-neutral allowlist, since `apm compile` copies frontmatter verbatim to both harnesses and an
|
|
unsafe field can't be silently dropped for just one of them. The allowlist itself lives in the
|
|
`apm-agent-allowlist` section of `references/field-inventory.md` and is read from there as data —
|
|
consult that section rather than any restatement of it, including this one. As of 2026-08-14 it
|
|
admits `name`, `description`, `model`, `source_keys`, and `disallowedTools`; `source_keys` is
|
|
provenance metadata checked separately by `validate-provenance.sh` against `sources.md`, and
|
|
`disallowedTools` is admitted because a denylist survives verbatim copy where the `tools` allowlist
|
|
does not (ADR-0016 and its 2026-08-14 amendment).
|
|
|
|
At **project/user scope**, accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives
|
|
the counterpart automatically, and validates both. Runs structural checks via `validate.sh`
|
|
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
|
|
Copilot-only fields in the CC file), provenance chain validation via `validate-provenance.sh`
|
|
(checks `source_keys` against `sources.md` at the plugin root — plugin/APM scope only), then
|
|
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
|
|
Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the
|
|
`Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert
|
|
is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is
|
|
unavailable or reports `0 files` scanned. Produces a compact findings report in the same format
|
|
as `skill-audit`.
|
|
|
|
## Usage
|
|
|
|
```
|
|
/agent-audit
|
|
```
|
|
|
|
Pass the path to either agent file as the argument.
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `SKILL.md` | Skill instructions for agents |
|
|
| `assets/vale/.vale.ini` | Vale config: scopes `Kyberforge` to `**/agents/*.md`, `Kyberforge`+`KyberforgeCopilot` to `**/*.agent.md` |
|
|
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This skill/agent" instead of an imperative "Use when..." |
|
|
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Flags generic "see references/ for info" pointers instead of specific file references |
|
|
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Flags sentences opening with "There is/are" instead of naming the subject directly |
|
|
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
|
|
| `assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml` | Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
|
|
| `references/README.md` | Directory documentation for references/ |
|
|
| `references/description-quality.md` | Qualitative guide for borderline description findings |
|
|
| `references/field-inventory.md` | Authoritative field lists read as data by `validate.sh`: valid CC and Copilot agent fields, and the vendor-neutral plugin/APM-scope allowlist |
|
|
| `references/sources.md` | Research provenance for skill content |
|
|
| `scripts/README.md` | Directory documentation for scripts/ |
|
|
| `scripts/validate.sh` | Structural validation script for agent file pairs |
|
|
| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `sources.md` (plugin root) |
|
|
| `scripts/vale-wrap.sh` | Drop-in `vale` wrapper that works around a frontmatter-description NLP scope limitation |
|
|
| `tests/README.md` | (source-only) Bats test dependency and run instructions |
|
|
| `tests/validate.bats` | (source-only) Bats tests for validate.sh |
|
|
| `tests/validate-provenance.bats` | (source-only) Bats tests for validate-provenance.sh |
|
|
|
|
Rows marked **(source-only)** exist in the authoring source (`.apm/skills/agent-audit/`) but are
|
|
not present in an installed plugin: `scripts/sync-plugin-content.sh` strips
|
|
`<category>/<name>/tests` when it generates the flat mirror, because these are dev-time fixtures no
|
|
plugin host needs to discover (ADR-0017). Run them from a repo checkout, not from an install.
|