feat(kyberforge): add agent-audit skill (closes #11)

## Why

agent-author produces paired agent definition files (Claude Code .md +
Copilot .agent.md) but had no companion audit skill to validate them.
agent-audit fills that gap, giving the same structured PASS/FAIL report
that skill-audit provides for SKILL.md files.

## Implementation Notes

- validate.sh uses scope detection (walk up for plugin.json / .git) to
  locate the counterpart file and determine whether plugin-silently-ignored
  fields (hooks, mcpServers, permissionMode) should be flagged
- CC-only and silently-ignored field lists are read from
  references/field-inventory.md at runtime rather than hardcoded —
  provenance back to the research corpus; see ADR-0019
- Single-file invocation (pass either file, counterpart derived) chosen
  over directory or name+root — see ADR-0018
- 12 bats tests cover provider detection, scope detection, all FAIL paths,
  and clean-pair pass

## Impact

- kyberforge bumped to v1.1.2
- agent-author close step should be updated to reference agent-audit (#11)
- Provenance/sources chain check deferred to #60

ADR: docs/adr/0018-agent-audit-single-file-invocation.md
ADR: docs/adr/0019-agent-audit-field-inventory-reference.md
Refs: #11
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
This commit is contained in:
2026-07-04 10:06:37 +00:00
parent e3e43502db
commit 92c13b997f
13 changed files with 794 additions and 2 deletions

View File

@@ -0,0 +1,86 @@
---
name: agent-audit
description: >
Use when the user wants to review an agent definition they wrote, says "audit this
agent", "check if my agent follows best practices", "review my agent file", or wants
to know if an agent pair is ready to ship — even if they don't use the word "audit".
Audits a Claude Code .md and Copilot .agent.md agent file pair — structural
validation via validate.sh plus qualitative checks on description and system prompt.
Produces a compact findings report (findings only, no PASS noise) with Why and Fix
per finding, in the same format as skill-audit. Do not use to fix agent files — use
/agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead.
allowed-tools: Bash Read
metadata:
category: factory
source_keys:
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
## Gotchas
- The unit of authoring is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is always a FAIL, not a warning.
- Plugin scope is detected by the presence of `plugin.json` in the directory tree — not by the file path pattern. Walk up, don't guess.
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
- Do not output findings while auditing — gather internally, surface in Step 3 report.
## Step 1 — Run structural validation
```bash
bash scripts/validate.sh <path-to-agent-file>
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
If the script cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
## Step 2 — Qualitative checks
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
**Description (both files):**
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
**Body:**
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
**Pair consistency (cross-file):**
- `name` field matches between CC and Copilot files — FAIL if mismatch
- Both system prompt bodies non-empty — FAIL if either is empty
## Step 3 — Report
Open with a coverage line:
```text
Checked: structure · provider-safety · description · body · pair-consistency
```
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely.
For each finding:
```text
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
```
Close with:
```text
## Result
PASS
PASS (N suggestions)
FAIL (N fails · M suggestions)
Run /agent-author to address findings.
```
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.