feat(kyberforge): improve agent-audit skill with platform-accurate checks and description quality reference
- Fix validate.sh: remove incorrect name==stem check for CC files (CC docs say filename need not match name field); keep check for Copilot CLI only - Fix validate.sh: plugin scope detection now checks both plugin.json and .claude-plugin/plugin.json - Fix validate.sh: Copilot cloud/IDE agents (.github/copilot/agents/) have name as optional; path-based guard added - Add validate.sh checks: Copilot body length >30,000 chars (SUGGESTION), Copilot-only fields in CC files (FAIL), subagent-unavailable tools in tools field (SUGGESTION) - Add references/description-quality.md as conditional escape hatch for borderline description findings - SKILL.md: name five audit dimensions in description; sharpen indirect-trigger phrasing - SKILL.md: label pair-mandate as kyberforge project convention, not platform requirement - SKILL.md: scope redundant name-match/body-empty checks to manual fallback only - SKILL.md: add conditional reference to description-quality.md; update provider-safety description for new check categories; fix plugin scope gotcha to mention .claude-plugin/plugin.json - SKILL.md: add INFO tier to result block template - Add source_keys frontmatter to references/README.md; update sources.md to add description-quality.md to contributing files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
This commit is contained in:
@@ -4,11 +4,12 @@ description: >
|
||||
Use when the user wants to review an agent definition they wrote, says "audit this
|
||||
agent", "check if my agent follows best practices", "review my agent file", or wants
|
||||
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
||||
Audits a Claude Code .md and Copilot .agent.md agent file pair — structural
|
||||
validation via validate.sh plus qualitative checks on description and system prompt.
|
||||
Produces a compact findings report (findings only, no PASS noise) with Why and Fix
|
||||
per finding, in the same format as skill-audit. Do not use to fix agent files — use
|
||||
/agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead.
|
||||
Audits a Claude Code .md and Copilot .agent.md agent file pair across five dimensions:
|
||||
structural validation, provider safety, description quality, body quality, and pair
|
||||
consistency — plus provenance chain validation. Produces a compact findings report
|
||||
(findings only, no PASS noise) with Why and Fix per finding. Do not use to fix agent
|
||||
files — use /agent-author instead. Do not use to audit SKILL.md files — use
|
||||
/skill-audit instead.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -22,8 +23,8 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- The unit of authoring is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is always a FAIL, not a warning.
|
||||
- Plugin scope is detected by the presence of `plugin.json` in the directory tree — not by the file path pattern. Walk up, don't guess.
|
||||
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
|
||||
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
|
||||
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
||||
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
||||
|
||||
@@ -34,11 +35,11 @@ bash scripts/validate.sh <path-to-agent-file>
|
||||
bash scripts/validate-provenance.sh <path-to-agent-file>
|
||||
```
|
||||
|
||||
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
|
||||
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`.
|
||||
|
||||
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
||||
|
||||
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
|
||||
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
|
||||
|
||||
## Step 2 — Qualitative checks
|
||||
|
||||
@@ -49,13 +50,15 @@ Read both agent files. Work through each dimension internally. Collect findings
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
|
||||
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
|
||||
|
||||
If a description finding is borderline, read `references/description-quality.md`.
|
||||
|
||||
**Body:**
|
||||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||||
|
||||
**Pair consistency (cross-file):**
|
||||
- `name` field matches between CC and Copilot files — FAIL if mismatch
|
||||
- Both system prompt bodies non-empty — FAIL if either is empty
|
||||
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
|
||||
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: `name` field matches between CC and Copilot files — FAIL if mismatch; both system prompt bodies non-empty — FAIL if either is empty
|
||||
|
||||
## Step 3 — Report
|
||||
|
||||
@@ -81,8 +84,11 @@ Close with:
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS · P info
|
||||
PASS (N suggestions)
|
||||
PASS (N suggestions) · P info
|
||||
FAIL (N fails · M suggestions)
|
||||
FAIL (N fails · M suggestions) · P info
|
||||
Run /agent-author to address findings.
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user