Remove phantom cross-file name-match claim from SKILL.md manual fallback list (validate.sh has no such check). Fix broken bats stem-mismatch test to overwrite the Copilot file instead of the CC file, exercising the actual Copilot stem check. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
96 lines
5.3 KiB
Markdown
96 lines
5.3 KiB
Markdown
---
|
|
name: agent-audit
|
|
description: >
|
|
Use when the user wants to review an agent definition they wrote, says "audit this
|
|
agent", "check if my agent follows best practices", "review my agent file", or wants
|
|
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
|
Audits a Claude Code .md and Copilot .agent.md agent file pair across five dimensions:
|
|
structural validation, provider safety, description quality, body quality, and pair
|
|
consistency — plus provenance chain validation. Produces a compact findings report
|
|
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
|
|
files — use /agent-author instead. Do not use to audit SKILL.md files — use
|
|
/skill-audit instead.
|
|
allowed-tools: Bash Read
|
|
metadata:
|
|
category: factory
|
|
source_keys:
|
|
- context7-websites-code-claude
|
|
- claude-code-plugins-docs
|
|
- claude-code-subagents-docs
|
|
- context7-github-en-copilot
|
|
- github-custom-agents-configuration
|
|
---
|
|
|
|
## Gotchas
|
|
|
|
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
|
|
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
|
|
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
|
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
|
|
|
## Step 1 — Run structural validation
|
|
|
|
```bash
|
|
bash scripts/validate.sh <path-to-agent-file>
|
|
bash scripts/validate-provenance.sh <path-to-agent-file>
|
|
```
|
|
|
|
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`.
|
|
|
|
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
|
|
|
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
|
|
|
|
## Step 2 — Qualitative checks
|
|
|
|
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
|
|
|
**Description (both files):**
|
|
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
|
|
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
|
|
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
|
|
|
|
If a description finding is borderline, read `references/description-quality.md`.
|
|
|
|
**Body:**
|
|
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
|
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
|
|
|
**Pair consistency (cross-file):**
|
|
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
|
|
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
|
|
|
|
## Step 3 — Report
|
|
|
|
Open with a coverage line:
|
|
|
|
```text
|
|
Checked: structure · provider-safety · description · body · pair-consistency · provenance
|
|
```
|
|
|
|
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
|
|
|
|
For each finding:
|
|
|
|
```text
|
|
FAIL/SUGGESTION <finding> — file:line
|
|
Why: <why this is a problem>
|
|
Fix: <exact change — quote before/after where applicable>
|
|
```
|
|
|
|
Close with:
|
|
|
|
```text
|
|
## Result
|
|
|
|
PASS
|
|
PASS · P info
|
|
PASS (N suggestions)
|
|
PASS (N suggestions) · P info
|
|
FAIL (N fails · M suggestions)
|
|
FAIL (N fails · M suggestions) · P info
|
|
Run /agent-author to address findings.
|
|
```
|
|
|
|
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|