feat(kyberforge): add agent-audit skill (closes #11)
## Why agent-author produces paired agent definition files (Claude Code .md + Copilot .agent.md) but had no companion audit skill to validate them. agent-audit fills that gap, giving the same structured PASS/FAIL report that skill-audit provides for SKILL.md files. ## Implementation Notes - validate.sh uses scope detection (walk up for plugin.json / .git) to locate the counterpart file and determine whether plugin-silently-ignored fields (hooks, mcpServers, permissionMode) should be flagged - CC-only and silently-ignored field lists are read from references/field-inventory.md at runtime rather than hardcoded — provenance back to the research corpus; see ADR-0019 - Single-file invocation (pass either file, counterpart derived) chosen over directory or name+root — see ADR-0018 - 12 bats tests cover provider detection, scope detection, all FAIL paths, and clean-pair pass ## Impact - kyberforge bumped to v1.1.2 - agent-author close step should be updated to reference agent-audit (#11) - Provenance/sources chain check deferred to #60 ADR: docs/adr/0018-agent-audit-single-file-invocation.md ADR: docs/adr/0019-agent-audit-field-inventory-reference.md Refs: #11 Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
This commit is contained in:
86
plugins/kyberforge/skills/agent-audit/SKILL.md
Normal file
86
plugins/kyberforge/skills/agent-audit/SKILL.md
Normal file
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: agent-audit
|
||||
description: >
|
||||
Use when the user wants to review an agent definition they wrote, says "audit this
|
||||
agent", "check if my agent follows best practices", "review my agent file", or wants
|
||||
to know if an agent pair is ready to ship — even if they don't use the word "audit".
|
||||
Audits a Claude Code .md and Copilot .agent.md agent file pair — structural
|
||||
validation via validate.sh plus qualitative checks on description and system prompt.
|
||||
Produces a compact findings report (findings only, no PASS noise) with Why and Fix
|
||||
per finding, in the same format as skill-audit. Do not use to fix agent files — use
|
||||
/agent-author instead. Do not use to audit SKILL.md files — use /skill-audit instead.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- The unit of authoring is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is always a FAIL, not a warning.
|
||||
- Plugin scope is detected by the presence of `plugin.json` in the directory tree — not by the file path pattern. Walk up, don't guess.
|
||||
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
|
||||
- Do not output findings while auditing — gather internally, surface in Step 3 report.
|
||||
|
||||
## Step 1 — Run structural validation
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <path-to-agent-file>
|
||||
```
|
||||
|
||||
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
|
||||
|
||||
If the script cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
|
||||
|
||||
## Step 2 — Qualitative checks
|
||||
|
||||
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
|
||||
|
||||
**Description (both files):**
|
||||
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
|
||||
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
|
||||
|
||||
**Body:**
|
||||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||||
|
||||
**Pair consistency (cross-file):**
|
||||
- `name` field matches between CC and Copilot files — FAIL if mismatch
|
||||
- Both system prompt bodies non-empty — FAIL if either is empty
|
||||
|
||||
## Step 3 — Report
|
||||
|
||||
Open with a coverage line:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · pair-consistency
|
||||
```
|
||||
|
||||
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely.
|
||||
|
||||
For each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with:
|
||||
|
||||
```text
|
||||
## Result
|
||||
|
||||
PASS
|
||||
PASS (N suggestions)
|
||||
FAIL (N fails · M suggestions)
|
||||
Run /agent-author to address findings.
|
||||
```
|
||||
|
||||
Omit `Run /agent-author to address findings.` when there are no findings at all. Do not apply fixes — report and propose only.
|
||||
Reference in New Issue
Block a user