## Why agent-author produces paired agent definition files (Claude Code .md + Copilot .agent.md) but had no companion audit skill to validate them. agent-audit fills that gap, giving the same structured PASS/FAIL report that skill-audit provides for SKILL.md files. ## Implementation Notes - validate.sh uses scope detection (walk up for plugin.json / .git) to locate the counterpart file and determine whether plugin-silently-ignored fields (hooks, mcpServers, permissionMode) should be flagged - CC-only and silently-ignored field lists are read from references/field-inventory.md at runtime rather than hardcoded — provenance back to the research corpus; see ADR-0019 - Single-file invocation (pass either file, counterpart derived) chosen over directory or name+root — see ADR-0018 - 12 bats tests cover provider detection, scope detection, all FAIL paths, and clean-pair pass ## Impact - kyberforge bumped to v1.1.2 - agent-author close step should be updated to reference agent-audit (#11) - Provenance/sources chain check deferred to #60 ADR: docs/adr/0018-agent-audit-single-file-invocation.md ADR: docs/adr/0019-agent-audit-field-inventory-reference.md Refs: #11 Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
3.9 KiB
name, description, allowed-tools, metadata
| name | description | allowed-tools | metadata | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| agent-audit | Use when the user wants to review an agent definition they wrote, says "audit this agent", "check if my agent follows best practices", "review my agent file", or wants to know if an agent pair is ready to ship — even if they don't use the word "audit". Audits a Claude Code .md and Copilot .agent.md agent file pair — structural validation via validate.sh plus qualitative checks on description and system prompt. Produces a compact findings report (findings only, no PASS noise) with Why and Fix per finding, in the same format as skill-audit. Do not use to fix agent files — use /agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead. | Bash Read |
|
Gotchas
- The unit of authoring is always a pair (CC
.md+ Copilot.agent.md). A missing counterpart is always a FAIL, not a warning. - Plugin scope is detected by the presence of
plugin.jsonin the directory tree — not by the file path pattern. Walk up, don't guess. references/field-inventory.mdmust exist forvalidate.shto run. The script exits with an error if it is missing.- Do not output findings while auditing — gather internally, surface in Step 3 report.
Step 1 — Run structural validation
bash scripts/validate.sh <path-to-agent-file>
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the ### Structure and ### Provider safety report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → ### Structure. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → ### Provider safety.
If the script cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (name, description, non-empty body), name is kebab-case, name matches filename stem, no FILL IN: placeholders, no CC-only fields in Copilot file.
Step 2 — Qualitative checks
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
Description (both files):
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
Use proactivelyin a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
Body:
- Direct role instruction: system prompt opens with
You are a [role]. When invoked, [action].— SUGGESTION if absent - One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
Pair consistency (cross-file):
namefield matches between CC and Copilot files — FAIL if mismatch- Both system prompt bodies non-empty — FAIL if either is empty
Step 3 — Report
Open with a coverage line:
Checked: structure · provider-safety · description · body · pair-consistency
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely.
For each finding:
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
Close with:
## Result
PASS
PASS (N suggestions)
FAIL (N fails · M suggestions)
Run /agent-author to address findings.
Omit Run /agent-author to address findings. when there are no findings at all. Do not apply fixes — report and propose only.