Defame1297
1333d2c1b1
## Why agent-author produces agents/sources.md at plugin scope to record which research sources informed which agent files. agent-audit had no way to validate this chain, leaving stale or missing provenance undetected. ## Implementation Notes Validation is per-pair (the given agent file + its counterpart) rather than plugin-wide, keeping the scope consistent with validate.sh. The script exits 0 silently for non-plugin-scope agents. source_keys is top-level in both CC .md and Copilot .agent.md files (not under metadata:) to avoid conflict with Copilot's own metadata field semantics. Checks 0, 1, 2, 4, 5, 6 mirror the skill provenance set; upstream research-doc cross-reference checks (7, 8) are deferred. agent-author Steps 2, 3, and 4 updated to formally specify the agents/sources.md format and instruct authors to add source_keys to both files when research sources are in context. Refs: #60 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
90 lines
4.4 KiB
Markdown
90 lines
4.4 KiB
Markdown
---
|
|
name: agent-audit
|
|
description: >
|
|
Use when the user wants to review an agent definition they wrote, says "audit this
|
|
agent", "check if my agent follows best practices", "review my agent file", or wants
|
|
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
|
Audits a Claude Code .md and Copilot .agent.md agent file pair — structural
|
|
validation via validate.sh plus qualitative checks on description and system prompt.
|
|
Produces a compact findings report (findings only, no PASS noise) with Why and Fix
|
|
per finding, in the same format as skill-audit. Do not use to fix agent files — use
|
|
/agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead.
|
|
allowed-tools: Bash Read
|
|
metadata:
|
|
category: factory
|
|
source_keys:
|
|
- context7-websites-code-claude
|
|
- claude-code-plugins-docs
|
|
- claude-code-subagents-docs
|
|
- context7-github-en-copilot
|
|
- github-custom-agents-configuration
|
|
---
|
|
|
|
## Gotchas
|
|
|
|
- The unit of authoring is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is always a FAIL, not a warning.
|
|
- Plugin scope is detected by the presence of `plugin.json` in the directory tree — not by the file path pattern. Walk up, don't guess.
|
|
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
|
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
|
|
|
## Step 1 — Run structural validation
|
|
|
|
```bash
|
|
bash scripts/validate.sh <path-to-agent-file>
|
|
bash scripts/validate-provenance.sh <path-to-agent-file>
|
|
```
|
|
|
|
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
|
|
|
|
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
|
|
|
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
|
|
|
|
## Step 2 — Qualitative checks
|
|
|
|
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
|
|
|
**Description (both files):**
|
|
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
|
|
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
|
|
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
|
|
|
|
**Body:**
|
|
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
|
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
|
|
|
**Pair consistency (cross-file):**
|
|
- `name` field matches between CC and Copilot files — FAIL if mismatch
|
|
- Both system prompt bodies non-empty — FAIL if either is empty
|
|
|
|
## Step 3 — Report
|
|
|
|
Open with a coverage line:
|
|
|
|
```text
|
|
Checked: structure · provider-safety · description · body · pair-consistency · provenance
|
|
```
|
|
|
|
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
|
|
|
|
For each finding:
|
|
|
|
```text
|
|
FAIL/SUGGESTION <finding> — file:line
|
|
Why: <why this is a problem>
|
|
Fix: <exact change — quote before/after where applicable>
|
|
```
|
|
|
|
Close with:
|
|
|
|
```text
|
|
## Result
|
|
|
|
PASS
|
|
PASS (N suggestions)
|
|
FAIL (N fails · M suggestions)
|
|
Run /agent-author to address findings.
|
|
```
|
|
|
|
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|