feat(kyberforge): improve agent-audit skill with platform-accurate checks and description quality reference

- Fix validate.sh: remove incorrect name==stem check for CC files (CC docs say filename need not match name field); keep check for Copilot CLI only
- Fix validate.sh: plugin scope detection now checks both plugin.json and .claude-plugin/plugin.json
- Fix validate.sh: Copilot cloud/IDE agents (.github/copilot/agents/) have name as optional; path-based guard added
- Add validate.sh checks: Copilot body length >30,000 chars (SUGGESTION), Copilot-only fields in CC files (FAIL), subagent-unavailable tools in tools field (SUGGESTION)
- Add references/description-quality.md as conditional escape hatch for borderline description findings
- SKILL.md: name five audit dimensions in description; sharpen indirect-trigger phrasing
- SKILL.md: label pair-mandate as kyberforge project convention, not platform requirement
- SKILL.md: scope redundant name-match/body-empty checks to manual fallback only
- SKILL.md: add conditional reference to description-quality.md; update provider-safety description for new check categories; fix plugin scope gotcha to mention .claude-plugin/plugin.json
- SKILL.md: add INFO tier to result block template
- Add source_keys frontmatter to references/README.md; update sources.md to add description-quality.md to contributing files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
This commit is contained in:
2026-07-04 11:10:32 +00:00
parent f9b22322a1
commit 8463c87dfc
7 changed files with 142 additions and 30 deletions

View File

@@ -4,11 +4,12 @@ description: >
Use when the user wants to review an agent definition they wrote, says "audit this
agent", "check if my agent follows best practices", "review my agent file", or wants
to know if an agent pair is ready to ship — even if they don't use the word "audit".
Audits a Claude Code .md and Copilot .agent.md agent file pair — structural
validation via validate.sh plus qualitative checks on description and system prompt.
Produces a compact findings report (findings only, no PASS noise) with Why and Fix
per finding, in the same format as skill-audit. Do not use to fix agent files — use
/agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead.
Audits a Claude Code .md and Copilot .agent.md agent file pair across five dimensions:
structural validation, provider safety, description quality, body quality, and pair
consistency — plus provenance chain validation. Produces a compact findings report
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
files — use /agent-author instead. Do not use to audit SKILL.md files — use
/skill-audit instead.
allowed-tools: Bash Read
metadata:
category: factory
@@ -22,8 +23,8 @@ metadata:
## Gotchas
- The unit of authoring is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is always a FAIL, not a warning.
- Plugin scope is detected by the presence of `plugin.json` in the directory tree — not by the file path pattern. Walk up, don't guess.
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
- Do not output findings while auditing — gather internally, surface in Step 3 report.
@@ -34,11 +35,11 @@ bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
## Step 2 — Qualitative checks
@@ -49,13 +50,15 @@ Read both agent files. Work through each dimension internally. Collect findings
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
If a description finding is borderline, read `references/description-quality.md`.
**Body:**
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
**Pair consistency (cross-file):**
- `name` field matches between CC and Copilot files — FAIL if mismatch
- Both system prompt bodies non-empty — FAIL if either is empty
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: `name` field matches between CC and Copilot files — FAIL if mismatch; both system prompt bodies non-empty — FAIL if either is empty
## Step 3 — Report
@@ -81,8 +84,11 @@ Close with:
## Result
PASS
PASS · P info
PASS (N suggestions)
PASS (N suggestions) · P info
FAIL (N fails · M suggestions)
FAIL (N fails · M suggestions) · P info
Run /agent-author to address findings.
```