Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

kyberforge: agent-audit skill (agent-author follow-ons) #11

New Issue
Closed
opened 2026-06-27 14:53:02 +00:00 by Claude · 1 comment
Claude commented 2026-06-27 14:53:02 +00:00
Collaborator
Copy Link

Context

Follow-ons deferred from the agent-author workstream. Both items were explicitly out of scope for the initial implementation shipped in 8f7945b.

Related: #10 (parent workstream) · ADR-0015 (docs/adr/0015-agent-author-dual-provider-scaffold.md)

1. agent-audit skill

A companion audit skill for agent definition files, analogous to how skill-audit validates SKILL.md files.

Why deferred: The agent definition spec is simpler than agentskills.io — fewer auditable rules. Inline validation in the agent-author close step is sufficient for MVP. A dedicated audit skill becomes worthwhile once agent-author is in active use and producing files that need systematic review.

Acceptance criteria:

  • agent-audit <path> validates a Claude Code .md or Copilot .agent.md agent file
  • Checks: required fields present (name, description, system prompt body), name format valid (kebab-case), no Claude Code-only fields in Copilot file, silently-ignored fields flagged (hooks/mcpServers/permissionMode for plugin agents)
  • Reports FAILs and SUGGESTIONs in the same format as skill-audit
  • agent-author close step updated to reference agent-audit
## Context Follow-ons deferred from the `agent-author` workstream. Both items were explicitly out of scope for the initial implementation shipped in `8f7945b`. **Related:** #10 (parent workstream) · ADR-0015 (`docs/adr/0015-agent-author-dual-provider-scaffold.md`) --- ## 1. `agent-audit` skill A companion audit skill for agent definition files, analogous to how `skill-audit` validates `SKILL.md` files. **Why deferred:** The agent definition spec is simpler than agentskills.io — fewer auditable rules. Inline validation in the `agent-author` close step is sufficient for MVP. A dedicated audit skill becomes worthwhile once `agent-author` is in active use and producing files that need systematic review. **Acceptance criteria:** - [x] `agent-audit <path>` validates a Claude Code `.md` or Copilot `.agent.md` agent file - [x] Checks: required fields present (`name`, `description`, system prompt body), name format valid (kebab-case), no Claude Code-only fields in Copilot file, silently-ignored fields flagged (`hooks`/`mcpServers`/`permissionMode` for plugin agents) - [x] Reports FAILs and SUGGESTIONs in the same format as `skill-audit` - [x] `agent-author` close step updated to reference `agent-audit`
Claude changed title from feat(kyberforge): agent-audit skill and non-standard Copilot path override for agent-author to kyberforge: agent-audit skill + non-standard Copilot path override for new-agent.sh 2026-06-27 14:57:09 +00:00
Claude added the Kind/Feature
Priority
Low
4
 labels 2026-06-27 14:57:12 +00:00
Claude changed title from kyberforge: agent-audit skill + non-standard Copilot path override for new-agent.sh to kyberforge: agent-audit skill + Copilot path override (agent-author follow-ons) 2026-06-27 15:00:52 +00:00
Defame1297 added this to the Kyberforge basics milestone 2026-06-27 15:11:03 +00:00
Defame1297 changed title from kyberforge: agent-audit skill + Copilot path override (agent-author follow-ons) to kyberforge: agent-audit skill (agent-author follow-ons) 2026-07-04 09:18:31 +00:00
Claude commented 2026-07-04 09:39:53 +00:00
Author
Collaborator
Copy Link

Grill session design decisions

Invocation

agent-audit <path-to-either-file> — pass either the Claude Code .md or Copilot .agent.md file. The skill derives the counterpart using scope detection. ADR-0018 documents this decision.

Scope detection

Walk up the directory tree from the input file:

  • plugin.json present → plugin scope (both files flat in agents/)
  • .git without plugin.json → project scope (CC in .claude/agents/, Copilot in .github/agents/)
  • ~ → user scope

Script (scripts/validate.sh) — structural checks

  1. Provider detection (.agent.md → Copilot, .md → Claude Code)
  2. Scope detection (as above)
  3. Counterpart file exists — FAIL if missing (the pair is the unit of authoring)
  4. Required fields present: name, description, body non-empty
  5. name is kebab-case
  6. name matches filename stem
  7. No unfilled FILL IN: placeholders
  8. No Claude Code-only fields in Copilot file (full list read from references/field-inventory.md)
  9. Silently-ignored fields (hooks, mcpServers, permissionMode) in a plugin-scoped CC file — FAIL

references/field-inventory.md

Machine-parseable markdown listing valid CC fields, CC-only fields, and silently-ignored plugin fields. Script reads this at runtime rather than hardcoding. ADR-0019 documents this decision.

Skill body — qualitative checks

  • Cross-file consistency: name matches between pair, both bodies non-empty
  • Description action-verb check (FAIL if absent)
  • You are a [role]. opening in system prompt (SUGGESTION if absent)

Report format

Same as skill-audit — coverage line, H3 dimensions, FAILs before SUGGESTIONs, Result block.

Dimensions: structure · provider-safety · description · body · pair-consistency

Tests

tests/validate.bats covering: provider detection, scope detection, counterpart-not-found FAIL, CC-only field in Copilot FAIL, silently-ignored field in plugin agent FAIL, clean pair passes.

Out of scope

Provenance/sources chain check deferred to #60.

agent-author update

Step 5 (close step) to reference agent-audit — run /agent-audit on the created files before closing.

## Grill session design decisions ### Invocation `agent-audit <path-to-either-file>` — pass either the Claude Code `.md` or Copilot `.agent.md` file. The skill derives the counterpart using scope detection. ADR-0018 documents this decision. ### Scope detection Walk up the directory tree from the input file: - `plugin.json` present → plugin scope (both files flat in `agents/`) - `.git` without `plugin.json` → project scope (CC in `.claude/agents/`, Copilot in `.github/agents/`) - `~` → user scope ### Script (`scripts/validate.sh`) — structural checks 1. Provider detection (`.agent.md` → Copilot, `.md` → Claude Code) 2. Scope detection (as above) 3. Counterpart file exists — **FAIL if missing** (the pair is the unit of authoring) 4. Required fields present: `name`, `description`, body non-empty 5. `name` is kebab-case 6. `name` matches filename stem 7. No unfilled `FILL IN:` placeholders 8. No Claude Code-only fields in Copilot file (full list read from `references/field-inventory.md`) 9. Silently-ignored fields (`hooks`, `mcpServers`, `permissionMode`) in a plugin-scoped CC file — FAIL ### `references/field-inventory.md` Machine-parseable markdown listing valid CC fields, CC-only fields, and silently-ignored plugin fields. Script reads this at runtime rather than hardcoding. ADR-0019 documents this decision. ### Skill body — qualitative checks - Cross-file consistency: `name` matches between pair, both bodies non-empty - Description action-verb check (FAIL if absent) - `You are a [role].` opening in system prompt (SUGGESTION if absent) ### Report format Same as `skill-audit` — coverage line, H3 dimensions, FAILs before SUGGESTIONs, Result block. Dimensions: `structure · provider-safety · description · body · pair-consistency` ### Tests `tests/validate.bats` covering: provider detection, scope detection, counterpart-not-found FAIL, CC-only field in Copilot FAIL, silently-ignored field in plugin agent FAIL, clean pair passes. ### Out of scope Provenance/sources chain check deferred to #60. ### agent-author update Step 5 (close step) to reference `agent-audit` — run `/agent-audit` on the created files before closing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No Label Kind/Feature
Priority
Low
4
Milestone
No items
No Milestone Kyberforge basics
Projects
Clear projects
No project
Assignees
Clear assignees
Claude renovate-bot Defame1297
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Defame1297/holocron#11
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?