Defame1297
149d564f6a
Round-3 review of PR #85 found the "enforcing" pre-commit hook enforced nothing. Vale's exit code keys on error-level alerts alone: five of the six rules were level: warning, so they exited 0, and pre-commit hides output from a passing hook — the alerts were invisible and blocked nothing. ADR-0013 rejected a report-only trial tier and then shipped one by accident. Flatten every rule to level: error. Vale's own exit code is then correct, so the hook entry drops to a bare vale-wrap.sh call and the graded error->FAIL / warning->SUGGESTION mapping disappears from both audit skills: every alert is a FAIL, in the gate and the audit alike. No ignorable tier, matching shellcheck, the test suite and conventional-pre-commit. Delete Kyberforge.VagueQualifier. Measured against the 41 skill/agent files as they stood before the rule ever ran: 2 hits. One marginal ("very different" -> "fundamentally different"), one an unfixable false positive — caveman/SKILL.md quotes "of course" as an example of filler, a mention not a use — which forced the only Vale suppression comments in the repo. Those four lines go with it; two of them were dead anyway, suppressing a frontmatter-scoped rule on a body line. Held-out prose (273 files) fired 15 times, 9 inside out-of-scope research examples and the rest one word in two idioms in a single doc. SentenceOpenerThereIs survives: 22 held-out hits, both in-corpus hits clean rewrites, zero suppressions. Widen .vale.ini's globs to [**/SKILL.md], [**/agents/*.md] and [**/*.agent.md]. The plugins/*/-prefixed globs scoped nothing — Vale's * crosses /, so they already matched docs/research/examples/**/agents/*.md and assets/templates/SKILL.md, the two paths CONTEXT.md claimed they excluded. Scoping is and was the hook's files: regex. The old globs also hid a silent false negative: a skill outside plugins/ matched no section, so Vale reported 0 files and exited 0, which both audits read as clean. They now treat a 0-file run as NOT RUN and fall back to full judgment. Also: - vale-wrap.sh resolves relative --config values and file arguments against the caller's cwd, as vale does, instead of the repo root, which hard-errored from a subdirectory and silently skipped flattening for file args that did not resolve from the root. Absolute paths inside the cwd are relativized so reports cite resolvable paths, not scratch ones. - vale-run's exit-code model was documented backwards ("exits non-zero whenever it finds an alert at or above MinAlertLevel") and would have led anyone following it to build a gate that passes everything. Its Markdown suppression syntax was MDX-only and does not suppress in .md; corrected in the skill and its troubleshooting reference, with backtick/fence exemption documented as the first resort. - skill-size-check.sh fails only above 500 lines, agreeing with skill-audit's validate.sh <= 500 pass. - ADR-0013 and CONTEXT.md amended to match, recording why graded severities cannot gate. Verified: 9 test scripts / 15 vale-wrap cases pass; vale-audit-prefilter, skill-size-check and shellcheck pass --all-files; check-manifests and claude plugin validate --strict clean. New tests fail against the old script (3 of them) and pass against the new one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
8.9 KiB
Vale audit prefilter expands into a plugin-content harness, scoped to prose-pattern rules only
Issue #84 wired Vale as a deterministic prefilter for skill-audit/agent-audit, scoped to
exactly four pattern-matchable checks (imperative description opener, vague capability wording,
generic reference-pointer padding, Copilot's dead Use proactively phrasing), documented only in
CONTEXT.md's "Vale audit prefilter" section — never its own ADR — and explicitly excluding body
discipline, near-miss exclusion strength, and control calibration as non-goals. This ADR records a
deferred PR #85 review item to broaden that coverage, retroactively captures #84's own rationale
(since it was never recorded as a decision in its own right), and layers the expansion on top
without reversing or weakening the original four rules.
File scope stays the same. SKILL.md plus agent files (**/agents/*.md,
**/*.agent.md) only — matching the existing prefilter's globs. Skill-level
README.md files and plugin.json manifests are not added: README.md files are navigational, not
spec-governed content, and plugin.json is JSON, not prose Vale can meaningfully lint.
Rule categories are prose-pattern-matchable only. Structural, schema, and security concerns
stay out of this Vale-based harness because this repo already has dedicated tools for them:
skill-frontmatter (required frontmatter fields), validate-plugins/validate-marketplace
(claude plugin validate --strict, schema), and gitleaks/detect-private-key (secrets).
Duplicating those concerns as Vale rules would fight tools that already own them better.
Governance docs are excluded as a rule source. docs/research/governance_principles/CONTROLS.md
and governance.md were investigated and found to contribute nothing minable: CONTROLS.md is
org/CI-infrastructure controls (secret scanning, dependency/license scanning, agent permission
scoping, audit logging, human approval gates, periodic reviews) — none of it is a prose pattern
expressible as a Vale rule against SKILL.md/agent-file text, and what it does cover is either
already handled elsewhere (gitleaks) or genuinely out of scope for a plugin-content prose harness
(dependency/license scanning is a code-dependency concern, not skill authoring).
Spec-derived custom rules stay mostly as-is. Re-reading agentskills.io's
optimizing-descriptions.md and skill-authoring.md, plus claude-code-plugins/agent-definition.md
and github-copilot-plugins/agent-definition.md, found that the existing four Kyberforge rules
already cover the pattern-matchable surface those specs describe. The remaining spec guidance —
calibrating control vs. giving freedom, avoiding menus of options, coherent skill scope, moderate
detail level — is semantic judgment, already skill-audit's job via LLM review, not new lintable
rules. One confirmation surfaced: Claude Code's Use proactively phrasing is meaningful for .md
agent files (it triggers auto-invocation), unlike Copilot's .agent.md files where it's dead
phrasing — so KyberforgeCopilot/ProactivePhrase's existing .agent.md-only scope is correct and
must not be extended to .md files.
write-good/alex are trialed, not adopted wholesale. These built-in/third-party Vale
packages are tuned for general blog-style prose (passive voice, weasel words, wordy phrases) and
are expected to be noisy against this repo's terse, imperative instruction-file corpus. Only
individual rules proven low-noise against the existing corpus get cherry-picked into
styles/Kyberforge; the packages are never referenced wholesale in BasedOnStyles.
A new non-Vale check closes a real gap. skill-authoring.md states SKILL.md should stay
under 500 lines / 5,000 tokens — currently unenforced anywhere in this repo. This is a whole-file
length ceiling, not a text pattern, so it isn't a Vale rule — it becomes a new deterministic script
and pre-commit hook, sibling to the existing skill-frontmatter hook.
Rules land directly in styles/Kyberforge, enforcing immediately. No trial/report-only tier
is introduced (see Considered Options). "Enforcing immediately" holds only because every rule in
both styles is level: error: Vale's exit code keys on error-level alerts alone, so a
warning- or suggestion-level rule prints an alert and still exits 0, and pre-commit suppresses
output from hooks that pass — such a rule is invisible and blocks nothing. Every Vale alert is
therefore a FAIL, in the audit skills and in the blocking pre-commit hook alike, with no ignorable
tier; that matches every other gate in this repo (shellcheck, the test suite,
conventional-pre-commit). The implementation pass finalizes the cherry-picked
write-good/alex rules and any new spec-derived rule wording, runs the full set against the
existing SKILL.md/agent-file corpus, fixes any resulting violations across that corpus, and lands
the rule changes and the corpus fixes as one atomic commit — the same enforcement model as the
original four rules, never a partial or opt-in state.
Considered options
Phased rollout via a separate trial style + config (rejected). A styles/KyberforgeTrial/
directory plus a parallel .vale.trial.ini (mirroring the root config's globs but with
BasedOnStyles = Kyberforge, KyberforgeTrial) would let new rules be swept report-only via
lint-runner/vale-run before promotion into the enforcing styles/Kyberforge + root
.vale.ini. This was considered because BasedOnStyles = Kyberforge activates every rule file
under that directory automatically — there's no partial/opt-in application within a style, so a
rule dropped straight into styles/Kyberforge goes live in the blocking pre-commit hook
immediately. Rejected in favor of finalizing rules directly and fixing violations via subagent
before committing: simpler, no new trial-config machinery to build or maintain — at the cost of no
standing report-only tier for future candidate rules. Note that the first implementation shipped
graded severities (error/warning/suggestion) and thereby recreated the rejected option by
accident: the five non-error rules never affected an exit code and never surfaced output through
a passing pre-commit hook, so they were a report-only tier that reported to nobody. Flattening
every rule to level: error is what actually implements this decision.
Consequences
styles/Kyberforge/gained one new rule file, cherry-picked fromwrite-good/alexas low-noise against this repo's corpus:SentenceOpenerThereIs.yml(22 hits across 273 held-out markdown files; both in-corpus hits were clean rewrites, needing no suppression).- A second candidate,
VagueQualifier.yml, was cherry-picked and then dropped. Against the 41 skill/agent files it hit twice: one marginal real finding (prototype/SKILL.md, "very different" → "fundamentally different") and one false positive (caveman/SKILL.md, which quotesof courseas an example of filler — a mention, not a use) that no rewrite could clear, forcing the repo's only Vale suppression comments. Of its 15 held-out hits, 9 were indocs/research/examples/(out-of-scope upstream material) and the remaining 6 were the word "very" in two idioms in a single research doc, each already adjacent to the hard number carrying the fact. One marginal catch does not pay for a permanent suppression, so the rule is deleted and this ADR's "cherry-picked rules" is one rule, not two. - A new pre-commit hook,
skill-size-check(scripts/skill-size-check.sh), enforces the 500-line/5,000-tokenSKILL.mdceiling, sibling toskill-frontmatter. It fails only above 500 lines, matchingskill-audit/scripts/validate.sh's long-standing<= 500pass. styles/KyberforgeTrial/and.vale.trial.iniwere deliberately not created — noted here so a future reader doesn't wonder if a trial tier was forgotten.- The styles-portability question — whether
styles/and.vale.inishould move intoplugins/lint/so the prefilter also works for repos that installkyberforge@holocronas an external plugin, rather than living at this repo's root — was deliberately deferred, not fixed, in this pass. This repo-root placement remains intentional: this ADR's "File scope stays the same" framing is specific to Kyberforge's own authoring conventions in this repo, not a genericlint-plugin feature. Portability is a known limitation, tracked for a separate future session, not silently forgotten.
What this ADR's implementation pass did: synced and trialed write-good/alex against the
existing SKILL.md/agent-file corpus, cherry-picked the one low-noise rule above into
styles/Kyberforge, wrote scripts/skill-size-check.sh and its pre-commit hook, fixed the
resulting corpus violations, and landed the rule changes and corpus fixes as one atomic commit —
matching the enforcement model described above (no partial or opt-in state), with every rule at
level: error so that model is real rather than nominal.