Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat(kyberforge): wire Vale as deterministic prefilter for skill-audit/agent-audit

Browse Source 
Adds repo-root .vale.ini plus a custom Kyberforge style (description-opener,
vague-wording, and generic reference-pointer padding rules) and a
KyberforgeCopilot style scoped to .agent.md files (Use proactively check).
skill-audit and agent-audit Step 1 now run vale against the specific file(s)
being audited and defer the corresponding Description/Patterns/Body checks
to its output instead of re-deriving them by LLM judgment, per the split
proposed in issue #84.

Closes #84

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxG5T8EJDgkABXxuneuFfn
This commit is contained in:
Defame1297
2026-07-23 20:39:25 +00:00
parent f326df4861
commit cbc33d952e
8 changed files with 60 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
plugins/kyberforge/skills/agent-audit/SKILL.md
View File

@@ -36,10 +36,13 @@ metadata:
```bash
bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
vale --config .vale.ini <path-to-cc-file> <path-to-copilot-file>
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
Run `vale` from the repo root, against both files of the pair (not just the one passed in), using the repo-root `.vale.ini`. It applies the `Kyberforge` style to both files and the `KyberforgeCopilot` style to the `.agent.md` file only — that split is how the Copilot-only `Use proactively` check stays scoped to the Copilot file without a manual per-file judgment call. Map `error` alerts to `FAIL` and `warning`/`suggestion` alerts to `SUGGESTION` in the `### Description` / `### Body` dimensions below, citing the rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). If `vale` is not installed or `.vale.ini` is missing, skip this and fall back to the manual judgment calls in Step 2 — do not block the audit on tooling absence.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
@@ -49,15 +52,16 @@ If the scripts cannot run (Bash denied, python3 unavailable), perform checks man
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
**Description (both files):**
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment.
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those without re-deriving by judgment.
- `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment.
If a description finding is borderline, read `references/description-quality.md`.
**Body:**
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment
**Body/Frontmatter comments:**
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.