feat(kyberforge): wire Vale as deterministic prefilter for skill-audit/agent-audit

Adds repo-root .vale.ini plus a custom Kyberforge style (description-opener,
vague-wording, and generic reference-pointer padding rules) and a
KyberforgeCopilot style scoped to .agent.md files (Use proactively check).
skill-audit and agent-audit Step 1 now run vale against the specific file(s)
being audited and defer the corresponding Description/Patterns/Body checks
to its output instead of re-deriving them by LLM judgment, per the split
proposed in issue #84.

Closes #84

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxG5T8EJDgkABXxuneuFfn
This commit is contained in:
2026-07-23 20:39:25 +00:00
parent f326df4861
commit cbc33d952e
8 changed files with 60 additions and 7 deletions

11
.vale.ini Normal file
View File

@@ -0,0 +1,11 @@
StylesPath = styles
MinAlertLevel = suggestion
[plugins/*/skills/*/SKILL.md]
BasedOnStyles = Kyberforge
[plugins/*/agents/*.md]
BasedOnStyles = Kyberforge
[plugins/*/agents/*.agent.md]
BasedOnStyles = Kyberforge, KyberforgeCopilot

View File

@@ -67,7 +67,10 @@ A skill pair in the `core` plugin for writing, updating, and reviewing a repo's
A companion skill (`core` plugin) that detects a target repo's provider-specific instruction file (`CLAUDE.md`, `.cursor/rules/*.mdc`, `copilot-instructions.md`, etc.) and, where it duplicates content AGENTS.md should own, converts it into a thin adapter that imports AGENTS.md — mirroring this repo's own ADR-0002/ADR-0003 two-tier adapter pattern. Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`: checks for an import reference, no duplicated headings, size threshold) rather than a separate paired audit skill — the check is mechanical, so a script suffices per governance.md's "prefer deterministic code for repeatable tasks." `agentsmd-author` calls this skill via skill composition when it detects an existing provider file with overlapping content. A companion skill (`core` plugin) that detects a target repo's provider-specific instruction file (`CLAUDE.md`, `.cursor/rules/*.mdc`, `copilot-instructions.md`, etc.) and, where it duplicates content AGENTS.md should own, converts it into a thin adapter that imports AGENTS.md — mirroring this repo's own ADR-0002/ADR-0003 two-tier adapter pattern. Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`: checks for an import reference, no duplicated headings, size threshold) rather than a separate paired audit skill — the check is mechanical, so a script suffices per governance.md's "prefer deterministic code for repeatable tasks." `agentsmd-author` calls this skill via skill composition when it detects an existing provider file with overlapping content.
### lint plugin ### lint plugin
A standalone, repo-agnostic plugin (`plugins/lint/`) for configuring and running linters — not scoped to kyberforge's own meta-tooling, and not (yet) wired into `skill-audit`/`agent-audit`. First linter is Vale (prose style linting), split into two skills per the git/gitea per-concern pattern: `vale-config` (setup — `.vale.ini`, `StylesPath`, styles) and `vale-run` (invoke Vale, interpret/report findings). A `lint-runner` agent composes these for isolated-context lint sweeps; it is report-only (no `Edit` tool) — it flags findings, it does not rewrite prose. Wiring Vale into the audit pipeline as a prefilter (the original motivation captured in branch `feat/84-vale-audit-prefilter`) is deferred to a follow-up task once this plugin exists standalone. Vale's research docs (`docs/research/docs/vale/`) moved from `plugins/kyberforge/` to `plugins/lint/` to keep the provenance chain same-plugin. A standalone, repo-agnostic plugin (`plugins/lint/`) for configuring and running linters — not scoped to kyberforge's own meta-tooling. First linter is Vale (prose style linting), split into two skills per the git/gitea per-concern pattern: `vale-config` (setup — `.vale.ini`, `StylesPath`, styles) and `vale-run` (invoke Vale, interpret/report findings). A `lint-runner` agent composes these for isolated-context lint sweeps; it is report-only (no `Edit` tool) — it flags findings, it does not rewrite prose. Vale's research docs (`docs/research/docs/vale/`) moved from `plugins/kyberforge/` to `plugins/lint/` to keep the provenance chain same-plugin.
### Vale audit prefilter (skill-audit / agent-audit)
Wiring Vale as a deterministic prefilter for `skill-audit`/`agent-audit`'s Description dimension (ADR motivation: issue #84) is repo-specific, not part of the generic `lint` plugin, so its config lives at the repo root rather than inside `plugins/lint/`: `.vale.ini` plus a custom `Kyberforge` style (`styles/Kyberforge/`) covering description-opener banning ("This skill/agent..."), vague-capability wording ("helps with", "utilize", ...), and generic "see references/ for details" padding — and a `KyberforgeCopilot` style (`styles/KyberforgeCopilot/`) scoped only to `.agent.md` files for the Copilot-only "Use proactively has no effect" check. Both skills' Step 1 run `vale --config .vale.ini <target>` against the specific file(s) being audited (never a repo-wide sweep) — Vale's glob matching crosses directory boundaries (`plugins/*/agents/*.md` matches nested `docs/research/examples/**/agents/*.md` too), so scoping every invocation to a known target file/dir is what keeps research-example files out of the audit's lint pass rather than the glob pattern itself. `error` alerts map to FAIL, `warning`/`suggestion` map to SUGGESTION. Vale only replaces the specific pattern-matchable sub-checks named in issue #84 (imperative opener, vague filler, `Use proactively`, generic reference-pointer padding) — body discipline, near-miss exclusion strength, and control calibration stay LLM judgment per the issue's explicit non-goals.
### LESSONS.md ### LESSONS.md
Long-loop feedback log for patterns observed across sessions. Three or more entries on the same pattern graduate to the relevant standing file (e.g. a coding convention, a governance rule). Updated by the session-handoff skill or directly by the human. Lives at the repo root. Long-loop feedback log for patterns observed across sessions. Three or more entries on the same pattern graduate to the relevant standing file (e.g. a coding convention, a governance rule). Updated by the session-handoff skill or directly by the human. Lives at the repo root.

View File

@@ -36,10 +36,13 @@ metadata:
```bash ```bash
bash scripts/validate.sh <path-to-agent-file> bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file> bash scripts/validate-provenance.sh <path-to-agent-file>
vale --config .vale.ini <path-to-cc-file> <path-to-copilot-file>
``` ```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`. The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
Run `vale` from the repo root, against both files of the pair (not just the one passed in), using the repo-root `.vale.ini`. It applies the `Kyberforge` style to both files and the `KyberforgeCopilot` style to the `.agent.md` file only — that split is how the Copilot-only `Use proactively` check stays scoped to the Copilot file without a manual per-file judgment call. Map `error` alerts to `FAIL` and `warning`/`suggestion` alerts to `SUGGESTION` in the `### Description` / `### Body` dimensions below, citing the rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). If `vale` is not installed or `.vale.ini` is missing, skip this and fall back to the manual judgment calls in Step 2 — do not block the audit on tooling absence.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix. `validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
@@ -49,15 +52,16 @@ If the scripts cannot run (Bash denied, python3 unavailable), perform checks man
Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3. Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3.
**Description (both files):** **Description (both files):**
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent - Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment.
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague - Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those without re-deriving by judgment.
- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove - `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment.
If a description finding is borderline, read `references/description-quality.md`. If a description finding is borderline, read `references/description-quality.md`.
**Body:** **Body:**
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent - Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded - One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment
**Body/Frontmatter comments:** **Body/Frontmatter comments:**
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding. - Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.

View File

@@ -34,12 +34,15 @@ metadata:
```bash ```bash
bash scripts/validate.sh <skill-dir> bash scripts/validate.sh <skill-dir>
bash scripts/validate-provenance.sh <skill-dir> bash scripts/validate-provenance.sh <skill-dir>
vale --config .vale.ini <skill-dir>/SKILL.md
``` ```
Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts. Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim. Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
`vale` runs from the repo root against the target `SKILL.md` using the repo-root `.vale.ini` (custom `Kyberforge` style) — it is a deterministic prefilter for a subset of the Description and Patterns dimensions below, not a replacement for Step 3's qualitative pass. Map `error` alerts to `FAIL` and `warning`/`suggestion` alerts to `SUGGESTION` in the `### Description` / `### Patterns` report dimensions, citing the rule ID (e.g. `Kyberforge.DescriptionOpener`) as the finding. If `vale` is not installed or `.vale.ini` is missing, skip this and fall back to the manual judgment calls described in Step 3 — do not block the audit on tooling absence.
## Step 2 — Read all skill files ## Step 2 — Read all skill files
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture. Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture.
@@ -50,8 +53,9 @@ Work through each dimension internally. Collect findings only; report them in St
### Description ### Description
- **Imperative phrasing**: does it use "Use when..." not "This skill..."? Vale's `Kyberforge.DescriptionOpener` (FAIL — "This skill..." openers) and `Kyberforge.VagueWording` (SUGGESTION — filler like "helps with", "utilize") alerts from Step 1 cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
- **Specificity**: are capabilities stated precisely ("parses OpenAPI specs") or vaguely ("helps with APIs")?
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly? - **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations? - **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
- **Length**: under 1024 characters? - **Length**: under 1024 characters?
@@ -75,7 +79,7 @@ Check each pattern is appropriate and correctly formed:
- **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip - **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip
- **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block - **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block
- **Checklists**: used for multi-step workflows, not single steps - **Checklists**: used for multi-step workflows, not single steps
- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/" - **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment.
- **Output templates**: present when the agent must produce a specific format; absent otherwise - **Output templates**: present when the agent must produce a specific format; absent otherwise
### File structure ### File structure

View File

@@ -0,0 +1,7 @@
extends: existence
message: "Description opens with '%s' — use an imperative 'Use when...' opener instead"
level: error
scope: text.frontmatter.description
ignorecase: true
raw:
- '^This (skill|agent)\b'

View File

@@ -0,0 +1,7 @@
extends: existence
message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead"
level: warning
scope: text
ignorecase: true
raw:
- 'see references?/? for (more )?(info|information|details)\b'

View File

@@ -0,0 +1,10 @@
extends: existence
message: "Vague capability wording: '%s' — state the capability precisely instead"
level: warning
scope: text.frontmatter.description
ignorecase: true
tokens:
- helps with
- utilize
- assists with
- used for

View File

@@ -0,0 +1,7 @@
extends: existence
message: "'%s' is CC-specific phrasing with no effect in Copilot descriptions — remove it"
level: warning
scope: text.frontmatter.description
ignorecase: true
tokens:
- Use proactively