Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat(kyberforge): add agent-audit provenance chain validation (closes #60)

Browse Source 
## Why

agent-author produces agents/sources.md at plugin scope to record which
research sources informed which agent files. agent-audit had no way to
validate this chain, leaving stale or missing provenance undetected.

## Implementation Notes

Validation is per-pair (the given agent file + its counterpart) rather
than plugin-wide, keeping the scope consistent with validate.sh. The
script exits 0 silently for non-plugin-scope agents.

source_keys is top-level in both CC .md and Copilot .agent.md files
(not under metadata:) to avoid conflict with Copilot's own metadata
field semantics. Checks 0, 1, 2, 4, 5, 6 mirror the skill provenance
set; upstream research-doc cross-reference checks (7, 8) are deferred.

agent-author Steps 2, 3, and 4 updated to formally specify the
agents/sources.md format and instruct authors to add source_keys to
both files when research sources are in context.

Refs: #60

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
This commit is contained in:
Defame1297
2026-07-04 10:37:44 +00:00
parent a4235d197d
commit 1333d2c1b1
7 changed files with 703 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
plugins/kyberforge/skills/agent-audit/SKILL.md
View File

@@ -31,11 +31,14 @@ metadata:
```bash
bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file or plugin-silently-ignored fields in a CC file → `### Provider safety`.
If the script cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, `name` matches filename stem, no `FILL IN:` placeholders, no CC-only fields in Copilot file.
## Step 2 — Qualitative checks
@@ -59,10 +62,10 @@ Read both agent files. Work through each dimension internally. Collect findings
Open with a coverage line:
```text
Checked: structure · provider-safety · description · body · pair-consistency
Checked: structure · provider-safety · description · body · pair-consistency · provenance
```
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely.
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
For each finding: