Defame1297
1333d2c1b1
## Why agent-author produces agents/sources.md at plugin scope to record which research sources informed which agent files. agent-audit had no way to validate this chain, leaving stale or missing provenance undetected. ## Implementation Notes Validation is per-pair (the given agent file + its counterpart) rather than plugin-wide, keeping the scope consistent with validate.sh. The script exits 0 silently for non-plugin-scope agents. source_keys is top-level in both CC .md and Copilot .agent.md files (not under metadata:) to avoid conflict with Copilot's own metadata field semantics. Checks 0, 1, 2, 4, 5, 6 mirror the skill provenance set; upstream research-doc cross-reference checks (7, 8) are deferred. agent-author Steps 2, 3, and 4 updated to formally specify the agents/sources.md format and instruct authors to add source_keys to both files when research sources are in context. Refs: #60 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
4.4 KiB
name, description, allowed-tools, metadata
| name | description | allowed-tools | metadata | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| agent-audit | Use when the user wants to review an agent definition they wrote, says "audit this agent", "check if my agent follows best practices", "review my agent file", or wants to know if an agent pair is ready to ship — even if they don't use the word "audit". Audits a Claude Code .md and Copilot .agent.md agent file pair — structural validation via validate.sh plus qualitative checks on description and system prompt. Produces a compact findings report (findings only, no PASS noise) with Why and Fix per finding, in the same format as skill-audit. Do not use to fix agent files — use /agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead. | Bash Read |
|
Gotchas
- The unit of authoring is always a pair (CC
.md+ Copilot.agent.md). A missing counterpart is always a FAIL, not a warning. - Plugin scope is detected by the presence of
plugin.jsonin the directory tree — not by the file path pattern. Walk up, don't guess. references/field-inventory.mdmust exist forvalidate.shto run. The script exits with an error if it is missing.- Do not output findings while auditing — gather internally, surface in Step 3 report.
Step 1 — Run structural validation
bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the ### Structure and ### Provider safety report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → ### Structure. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → ### Provider safety.
validate-provenance.sh validates the provenance chain between the agent pair's source_keys and the plugin-scoped agents/sources.md. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the ### Provenance dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (name, description, non-empty body), name is kebab-case, name matches filename stem, no FILL IN: placeholders, no CC-only fields in Copilot file.
Step 2 — Qualitative checks
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
Description (both files):
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
Use proactivelyin a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
Body:
- Direct role instruction: system prompt opens with
You are a [role]. When invoked, [action].— SUGGESTION if absent - One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
Pair consistency (cross-file):
namefield matches between CC and Copilot files — FAIL if mismatch- Both system prompt bodies non-empty — FAIL if either is empty
Step 3 — Report
Open with a coverage line:
Checked: structure · provider-safety · description · body · pair-consistency · provenance
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. ### Provenance findings are sourced verbatim from validate-provenance.sh output — copy them without rephrasing.
For each finding:
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
Close with:
## Result
PASS
PASS (N suggestions)
FAIL (N fails · M suggestions)
Run /agent-author to address findings.
Omit Run /agent-author to address findings. when there are no findings at all. Do not apply fixes — report and propose only.