From cbc33d952efc4aaf605a2f3a2d447fa7af0adef6 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Thu, 23 Jul 2026 20:39:25 +0000 Subject: [PATCH] feat(kyberforge): wire Vale as deterministic prefilter for skill-audit/agent-audit Adds repo-root .vale.ini plus a custom Kyberforge style (description-opener, vague-wording, and generic reference-pointer padding rules) and a KyberforgeCopilot style scoped to .agent.md files (Use proactively check). skill-audit and agent-audit Step 1 now run vale against the specific file(s) being audited and defer the corresponding Description/Patterns/Body checks to its output instead of re-deriving them by LLM judgment, per the split proposed in issue #84. Closes #84 Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_01FxG5T8EJDgkABXxuneuFfn --- .vale.ini | 11 +++++++++++ CONTEXT.md | 5 ++++- plugins/kyberforge/skills/agent-audit/SKILL.md | 10 +++++++--- plugins/kyberforge/skills/skill-audit/SKILL.md | 10 +++++++--- styles/Kyberforge/DescriptionOpener.yml | 7 +++++++ styles/Kyberforge/PaddingPhrase.yml | 7 +++++++ styles/Kyberforge/VagueWording.yml | 10 ++++++++++ styles/KyberforgeCopilot/ProactivePhrase.yml | 7 +++++++ 8 files changed, 60 insertions(+), 7 deletions(-) create mode 100644 .vale.ini create mode 100644 styles/Kyberforge/DescriptionOpener.yml create mode 100644 styles/Kyberforge/PaddingPhrase.yml create mode 100644 styles/Kyberforge/VagueWording.yml create mode 100644 styles/KyberforgeCopilot/ProactivePhrase.yml diff --git a/.vale.ini b/.vale.ini new file mode 100644 index 0000000..c0d0410 --- /dev/null +++ b/.vale.ini @@ -0,0 +1,11 @@ +StylesPath = styles +MinAlertLevel = suggestion + +[plugins/*/skills/*/SKILL.md] +BasedOnStyles = Kyberforge + +[plugins/*/agents/*.md] +BasedOnStyles = Kyberforge + +[plugins/*/agents/*.agent.md] +BasedOnStyles = Kyberforge, KyberforgeCopilot diff --git a/CONTEXT.md b/CONTEXT.md index e2cc184..b9c1320 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -67,7 +67,10 @@ A skill pair in the `core` plugin for writing, updating, and reviewing a repo's A companion skill (`core` plugin) that detects a target repo's provider-specific instruction file (`CLAUDE.md`, `.cursor/rules/*.mdc`, `copilot-instructions.md`, etc.) and, where it duplicates content AGENTS.md should own, converts it into a thin adapter that imports AGENTS.md — mirroring this repo's own ADR-0002/ADR-0003 two-tier adapter pattern. Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`: checks for an import reference, no duplicated headings, size threshold) rather than a separate paired audit skill — the check is mechanical, so a script suffices per governance.md's "prefer deterministic code for repeatable tasks." `agentsmd-author` calls this skill via skill composition when it detects an existing provider file with overlapping content. ### lint plugin -A standalone, repo-agnostic plugin (`plugins/lint/`) for configuring and running linters — not scoped to kyberforge's own meta-tooling, and not (yet) wired into `skill-audit`/`agent-audit`. First linter is Vale (prose style linting), split into two skills per the git/gitea per-concern pattern: `vale-config` (setup — `.vale.ini`, `StylesPath`, styles) and `vale-run` (invoke Vale, interpret/report findings). A `lint-runner` agent composes these for isolated-context lint sweeps; it is report-only (no `Edit` tool) — it flags findings, it does not rewrite prose. Wiring Vale into the audit pipeline as a prefilter (the original motivation captured in branch `feat/84-vale-audit-prefilter`) is deferred to a follow-up task once this plugin exists standalone. Vale's research docs (`docs/research/docs/vale/`) moved from `plugins/kyberforge/` to `plugins/lint/` to keep the provenance chain same-plugin. +A standalone, repo-agnostic plugin (`plugins/lint/`) for configuring and running linters — not scoped to kyberforge's own meta-tooling. First linter is Vale (prose style linting), split into two skills per the git/gitea per-concern pattern: `vale-config` (setup — `.vale.ini`, `StylesPath`, styles) and `vale-run` (invoke Vale, interpret/report findings). A `lint-runner` agent composes these for isolated-context lint sweeps; it is report-only (no `Edit` tool) — it flags findings, it does not rewrite prose. Vale's research docs (`docs/research/docs/vale/`) moved from `plugins/kyberforge/` to `plugins/lint/` to keep the provenance chain same-plugin. + +### Vale audit prefilter (skill-audit / agent-audit) +Wiring Vale as a deterministic prefilter for `skill-audit`/`agent-audit`'s Description dimension (ADR motivation: issue #84) is repo-specific, not part of the generic `lint` plugin, so its config lives at the repo root rather than inside `plugins/lint/`: `.vale.ini` plus a custom `Kyberforge` style (`styles/Kyberforge/`) covering description-opener banning ("This skill/agent..."), vague-capability wording ("helps with", "utilize", ...), and generic "see references/ for details" padding — and a `KyberforgeCopilot` style (`styles/KyberforgeCopilot/`) scoped only to `.agent.md` files for the Copilot-only "Use proactively has no effect" check. Both skills' Step 1 run `vale --config .vale.ini ` against the specific file(s) being audited (never a repo-wide sweep) — Vale's glob matching crosses directory boundaries (`plugins/*/agents/*.md` matches nested `docs/research/examples/**/agents/*.md` too), so scoping every invocation to a known target file/dir is what keeps research-example files out of the audit's lint pass rather than the glob pattern itself. `error` alerts map to FAIL, `warning`/`suggestion` map to SUGGESTION. Vale only replaces the specific pattern-matchable sub-checks named in issue #84 (imperative opener, vague filler, `Use proactively`, generic reference-pointer padding) — body discipline, near-miss exclusion strength, and control calibration stay LLM judgment per the issue's explicit non-goals. ### LESSONS.md Long-loop feedback log for patterns observed across sessions. Three or more entries on the same pattern graduate to the relevant standing file (e.g. a coding convention, a governance rule). Updated by the session-handoff skill or directly by the human. Lives at the repo root. diff --git a/plugins/kyberforge/skills/agent-audit/SKILL.md b/plugins/kyberforge/skills/agent-audit/SKILL.md index 894a823..2897dd3 100644 --- a/plugins/kyberforge/skills/agent-audit/SKILL.md +++ b/plugins/kyberforge/skills/agent-audit/SKILL.md @@ -36,10 +36,13 @@ metadata: ```bash bash scripts/validate.sh bash scripts/validate-provenance.sh +vale --config .vale.ini ``` The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`. +Run `vale` from the repo root, against both files of the pair (not just the one passed in), using the repo-root `.vale.ini`. It applies the `Kyberforge` style to both files and the `KyberforgeCopilot` style to the `.agent.md` file only — that split is how the Copilot-only `Use proactively` check stays scoped to the Copilot file without a manual per-file judgment call. Map `error` alerts to `FAIL` and `warning`/`suggestion` alerts to `SUGGESTION` in the `### Description` / `### Body` dimensions below, citing the rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). If `vale` is not installed or `.vale.ini` is missing, skip this and fall back to the manual judgment calls in Step 2 — do not block the audit on tooling absence. + `validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix. If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). @@ -49,15 +52,16 @@ If the scripts cannot run (Bash denied, python3 unavailable), perform checks man Read both agent files. Work through each dimension internally. Collect findings only; report in Step 3. **Description (both files):** -- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent -- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague -- `Use proactively` in a Copilot description: CC-specific phrasing, has no effect in Copilot — SUGGESTION to remove +- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment. +- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those without re-deriving by judgment. +- `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment. If a description finding is borderline, read `references/description-quality.md`. **Body:** - Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent - One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded +- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment **Body/Frontmatter comments:** - Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding. diff --git a/plugins/kyberforge/skills/skill-audit/SKILL.md b/plugins/kyberforge/skills/skill-audit/SKILL.md index eac4a0f..8923bb0 100644 --- a/plugins/kyberforge/skills/skill-audit/SKILL.md +++ b/plugins/kyberforge/skills/skill-audit/SKILL.md @@ -34,12 +34,15 @@ metadata: ```bash bash scripts/validate.sh bash scripts/validate-provenance.sh +vale --config .vale.ini /SKILL.md ``` Note any structural FAILs — they will appear in the report as a `### Structure` dimension. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts. Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim. +`vale` runs from the repo root against the target `SKILL.md` using the repo-root `.vale.ini` (custom `Kyberforge` style) — it is a deterministic prefilter for a subset of the Description and Patterns dimensions below, not a replacement for Step 3's qualitative pass. Map `error` alerts to `FAIL` and `warning`/`suggestion` alerts to `SUGGESTION` in the `### Description` / `### Patterns` report dimensions, citing the rule ID (e.g. `Kyberforge.DescriptionOpener`) as the finding. If `vale` is not installed or `.vale.ini` is missing, skip this and fall back to the manual judgment calls described in Step 3 — do not block the audit on tooling absence. + ## Step 2 — Read all skill files Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, `assets/`, and `tests/`. Skip binary files only. Do not skip text files — internal consistency checks require the full picture. @@ -50,8 +53,9 @@ Work through each dimension internally. Collect findings only; report them in St ### Description -- **Imperative phrasing**: does it use "Use when..." not "This skill..."? -- **Specificity**: are capabilities stated precisely ("parses OpenAPI specs") or vaguely ("helps with APIs")? +Vale's `Kyberforge.DescriptionOpener` (FAIL — "This skill..." openers) and `Kyberforge.VagueWording` (SUGGESTION — filler like "helps with", "utilize") alerts from Step 1 cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call: + +- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")? - **Indirect triggers**: does it cover cases where the user doesn't name the domain directly? - **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations? - **Length**: under 1024 characters? @@ -75,7 +79,7 @@ Check each pattern is appropriate and correctly formed: - **Gotchas**: placed near the top; each entry is a specific fact that defies a reasonable assumption — not a general tip - **Prescriptive sequence**: inner code fences escaped as `\`\`\`` when nested inside a markdown block - **Checklists**: used for multi-step workflows, not single steps -- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/" +- **Conditional references**: specific trigger stated ("If X, read `references/file.md`") — not a generic "see references/". Vale's `Kyberforge.PaddingPhrase` alert from Step 1 flags the generic phrasing directly; other malformed conditional-reference forms still require judgment. - **Output templates**: present when the agent must produce a specific format; absent otherwise ### File structure diff --git a/styles/Kyberforge/DescriptionOpener.yml b/styles/Kyberforge/DescriptionOpener.yml new file mode 100644 index 0000000..d978d73 --- /dev/null +++ b/styles/Kyberforge/DescriptionOpener.yml @@ -0,0 +1,7 @@ +extends: existence +message: "Description opens with '%s' — use an imperative 'Use when...' opener instead" +level: error +scope: text.frontmatter.description +ignorecase: true +raw: + - '^This (skill|agent)\b' diff --git a/styles/Kyberforge/PaddingPhrase.yml b/styles/Kyberforge/PaddingPhrase.yml new file mode 100644 index 0000000..189cc66 --- /dev/null +++ b/styles/Kyberforge/PaddingPhrase.yml @@ -0,0 +1,7 @@ +extends: existence +message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead" +level: warning +scope: text +ignorecase: true +raw: + - 'see references?/? for (more )?(info|information|details)\b' diff --git a/styles/Kyberforge/VagueWording.yml b/styles/Kyberforge/VagueWording.yml new file mode 100644 index 0000000..7490aaf --- /dev/null +++ b/styles/Kyberforge/VagueWording.yml @@ -0,0 +1,10 @@ +extends: existence +message: "Vague capability wording: '%s' — state the capability precisely instead" +level: warning +scope: text.frontmatter.description +ignorecase: true +tokens: + - helps with + - utilize + - assists with + - used for diff --git a/styles/KyberforgeCopilot/ProactivePhrase.yml b/styles/KyberforgeCopilot/ProactivePhrase.yml new file mode 100644 index 0000000..d23f8b7 --- /dev/null +++ b/styles/KyberforgeCopilot/ProactivePhrase.yml @@ -0,0 +1,7 @@ +extends: existence +message: "'%s' is CC-specific phrasing with no effect in Copilot descriptions — remove it" +level: warning +scope: text.frontmatter.description +ignorecase: true +tokens: + - Use proactively