- agent-author: convert template comments from YAML (#) to HTML (<!-- -->) - Easier to spot and distinguish from functional comments - Add explicit "Delete template comments before shipping" reminders - Update SKILL.md Steps 2-3 with removal instruction - agent-audit: add comment-discipline check - Flag excessive frontmatter documentation comments as padding - Mirrors skill-audit's body-discipline principle - Update coverage line to include comment-discipline dimension This ensures agents follow the same comment-cleanup discipline as skills, preventing template documentation from shipping with agent definitions. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
101 lines
5.9 KiB
Markdown
101 lines
5.9 KiB
Markdown
---
|
||
name: agent-audit
|
||
description: >
|
||
Use when the user wants to review an agent definition they wrote, says "audit this
|
||
agent", "check if my agent follows best practices", "review my agent file", or wants
|
||
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
||
Audits a Claude Code .md and Copilot .agent.md agent file pair across five dimensions:
|
||
structural validation, provider safety, description quality, body quality, and pair
|
||
consistency — plus provenance chain validation. Produces a compact findings report
|
||
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
|
||
files — use /agent-author instead. Do not use to audit SKILL.md files — use
|
||
/skill-audit instead.
|
||
allowed-tools: Bash Read
|
||
metadata:
|
||
category: factory
|
||
source_keys:
|
||
- context7-websites-code-claude
|
||
- claude-code-plugins-docs
|
||
- claude-code-subagents-docs
|
||
- context7-github-en-copilot
|
||
- github-custom-agents-configuration
|
||
---
|
||
|
||
## Gotchas
|
||
|
||
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
|
||
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
|
||
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
||
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
||
|
||
## Step 1 — Run structural validation
|
||
|
||
```bash
|
||
bash scripts/validate.sh <path-to-agent-file>
|
||
bash scripts/validate-provenance.sh <path-to-agent-file>
|
||
```
|
||
|
||
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`.
|
||
|
||
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
||
|
||
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
|
||
|
||
## Step 2 — Qualitative checks
|
||
|
||
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
||
|
||
**Description (both files):**
|
||
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
|
||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
|
||
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
|
||
|
||
If a description finding is borderline, read `references/description-quality.md`.
|
||
|
||
**Body:**
|
||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||
|
||
**Body/Frontmatter comments:**
|
||
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.
|
||
- Look for patterns like `# Optional. <long explanation>` or extensive inline guidance (more than 1–2 lines per field) that should be condensed or removed before shipping.
|
||
- This mirrors skill-audit's body-discipline check but applies to template documentation in the frontmatter — template guidance belongs in development; agent-ready files should have minimal comments.
|
||
|
||
**Pair consistency (cross-file):**
|
||
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
|
||
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
|
||
|
||
## Step 3 — Report
|
||
|
||
Open with a coverage line:
|
||
|
||
```text
|
||
Checked: structure · provider-safety · description · body · comment-discipline · pair-consistency · provenance
|
||
```
|
||
|
||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
|
||
|
||
For each finding:
|
||
|
||
```text
|
||
FAIL/SUGGESTION <finding> — file:line
|
||
Why: <why this is a problem>
|
||
Fix: <exact change — quote before/after where applicable>
|
||
```
|
||
|
||
Close with:
|
||
|
||
```text
|
||
## Result
|
||
|
||
PASS
|
||
PASS · P info
|
||
PASS (N suggestions)
|
||
PASS (N suggestions) · P info
|
||
FAIL (N fails · M suggestions)
|
||
FAIL (N fails · M suggestions) · P info
|
||
Run /agent-author to address findings.
|
||
```
|
||
|
||
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|