fix(lint): make the Vale gate actually gate, drop VagueQualifier
Round-3 review of PR #85 found the "enforcing" pre-commit hook enforced nothing. Vale's exit code keys on error-level alerts alone: five of the six rules were level: warning, so they exited 0, and pre-commit hides output from a passing hook — the alerts were invisible and blocked nothing. ADR-0013 rejected a report-only trial tier and then shipped one by accident. Flatten every rule to level: error. Vale's own exit code is then correct, so the hook entry drops to a bare vale-wrap.sh call and the graded error->FAIL / warning->SUGGESTION mapping disappears from both audit skills: every alert is a FAIL, in the gate and the audit alike. No ignorable tier, matching shellcheck, the test suite and conventional-pre-commit. Delete Kyberforge.VagueQualifier. Measured against the 41 skill/agent files as they stood before the rule ever ran: 2 hits. One marginal ("very different" -> "fundamentally different"), one an unfixable false positive — caveman/SKILL.md quotes "of course" as an example of filler, a mention not a use — which forced the only Vale suppression comments in the repo. Those four lines go with it; two of them were dead anyway, suppressing a frontmatter-scoped rule on a body line. Held-out prose (273 files) fired 15 times, 9 inside out-of-scope research examples and the rest one word in two idioms in a single doc. SentenceOpenerThereIs survives: 22 held-out hits, both in-corpus hits clean rewrites, zero suppressions. Widen .vale.ini's globs to [**/SKILL.md], [**/agents/*.md] and [**/*.agent.md]. The plugins/*/-prefixed globs scoped nothing — Vale's * crosses /, so they already matched docs/research/examples/**/agents/*.md and assets/templates/SKILL.md, the two paths CONTEXT.md claimed they excluded. Scoping is and was the hook's files: regex. The old globs also hid a silent false negative: a skill outside plugins/ matched no section, so Vale reported 0 files and exited 0, which both audits read as clean. They now treat a 0-file run as NOT RUN and fall back to full judgment. Also: - vale-wrap.sh resolves relative --config values and file arguments against the caller's cwd, as vale does, instead of the repo root, which hard-errored from a subdirectory and silently skipped flattening for file args that did not resolve from the root. Absolute paths inside the cwd are relativized so reports cite resolvable paths, not scratch ones. - vale-run's exit-code model was documented backwards ("exits non-zero whenever it finds an alert at or above MinAlertLevel") and would have led anyone following it to build a gate that passes everything. Its Markdown suppression syntax was MDX-only and does not suppress in .md; corrected in the skill and its troubleshooting reference, with backtick/fence exemption documented as the first resort. - skill-size-check.sh fails only above 500 lines, agreeing with skill-audit's validate.sh <= 500 pass. - ADR-0013 and CONTEXT.md amended to match, recording why graded severities cannot gate. Verified: 9 test scripts / 15 vale-wrap cases pass; vale-audit-prefilter, skill-size-check and shellcheck pass --all-files; check-manifests and claude plugin validate --strict clean. New tests fail against the old script (3 of them) and pass against the new one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
This commit is contained in:
@@ -15,12 +15,7 @@ ACTIVE EVERY RESPONSE once triggered. No revert after many turns. No filler drif
|
||||
|
||||
## Rules
|
||||
|
||||
<!-- vale Kyberforge.VagueQualifier = NO -->
|
||||
<!-- vale Kyberforge.VagueWording = NO -->
|
||||
Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging.
|
||||
<!-- vale Kyberforge.VagueQualifier = YES -->
|
||||
<!-- vale Kyberforge.VagueWording = YES -->
|
||||
Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Abbreviate common terms (DB/auth/config/req/res/fn/impl). Strip conjunctions. Use arrows for causality (X -> Y). One word when one word enough.
|
||||
Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Abbreviate common terms (DB/auth/config/req/res/fn/impl). Strip conjunctions. Use arrows for causality (X -> Y). One word when one word enough.
|
||||
|
||||
Technical terms stay exact. Code blocks unchanged. Errors quoted exact.
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ bash scripts/validate-provenance.sh <path-to-agent-file>
|
||||
|
||||
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
|
||||
|
||||
`vale-wrap.sh` resolves its own path and `.vale.ini` via `git rev-parse --show-toplevel`, so it runs correctly regardless of the caller's cwd. Run it against both files of the pair (not just the one passed in). `Kyberforge` applies to both files; `KyberforgeCopilot` applies to the `.agent.md` file only, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a Copilot description — there's nothing to flag in the CC file, so it isn't scoped there. Map `error` → `FAIL` and `warning`/`suggestion` → `SUGGESTION` in the `### Description` / `### Body` dimensions, citing the rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if vale or `.vale.ini` is unavailable.
|
||||
`vale-wrap.sh` resolves its own path and `.vale.ini` via `git rev-parse --show-toplevel`; pass the absolute paths shown above so the invocation holds from any cwd. Run it against both files of the pair (not just the one passed in). `Kyberforge` applies to both files; `KyberforgeCopilot` applies to the `.agent.md` file only, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a Copilot description — there's nothing to flag in the CC file, so it isn't scoped there. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if vale or `.vale.ini` is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
|
||||
|
||||
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
|
||||
|
||||
@@ -53,7 +53,7 @@ Read both agent files. Work through each dimension internally. Collect findings
|
||||
|
||||
**Description (both files):**
|
||||
- Action-verb opening: description starts with a verb ("Reviews...", "Analyzes...", "Generates...") — FAIL if absent. Vale's `Kyberforge.DescriptionOpener` alert flags the specific known-bad "This agent..." opener directly; verifying an arbitrary opening word is genuinely a strong verb still requires judgment.
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those without re-deriving by judgment.
|
||||
- Specificity: is the trigger condition stated precisely? — SUGGESTION if vague. Vale's `Kyberforge.VagueWording` alert covers known filler ("helps with", "utilize", ...) directly; report those as FAILs without re-deriving by judgment.
|
||||
- `Use proactively` in a Copilot description: Vale's `KyberforgeCopilot.ProactivePhrase` alert (Copilot file only) flags this directly — report it without re-deriving by judgment.
|
||||
|
||||
If a description finding is borderline, read `references/description-quality.md`.
|
||||
@@ -62,7 +62,7 @@ If a description finding is borderline, read `references/description-quality.md`
|
||||
- Direct role instruction: system prompt opens with `You are a [role]. When invoked, [action].` — SUGGESTION if absent
|
||||
- One job per agent: system prompt describes a single bounded task — SUGGESTION if scope appears unbounded
|
||||
- Generic, non-specific reference pointers to the `references/` directory: Vale's `Kyberforge.PaddingPhrase` alert flags this directly — report it without re-deriving by judgment
|
||||
- Vague filler wording and sentences that open with "There is"/"There are": Vale's `Kyberforge.VagueQualifier` and `Kyberforge.SentenceOpenerThereIs` alerts flag this directly — report them without re-deriving by judgment
|
||||
- Sentences that open with "There is"/"There are": Vale's `Kyberforge.SentenceOpenerThereIs` alert flags this directly — report it without re-deriving by judgment
|
||||
|
||||
**Body/Frontmatter comments:**
|
||||
- Inspect each comment block in the YAML frontmatter. For each comment, apply: *"Would the agent get this wrong without this comment?"* Flag any that answer "no" as padding.
|
||||
|
||||
@@ -41,7 +41,7 @@ Note any structural FAILs — they will appear in the report as a `### Structure
|
||||
|
||||
Note any Provenance FAILs and INFO findings from `validate-provenance.sh` — they surface in the report as a `### Provenance` dimension (separate from `### Structure`). The script embeds full FAIL/INFO format with Why and Fix per finding; surface them verbatim.
|
||||
|
||||
`vale-wrap.sh` resolves its own path and `.vale.ini` via `git rev-parse --show-toplevel`, so it runs correctly regardless of the caller's cwd, using `.vale.ini`'s `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns dimensions below, not a replacement for Step 3. Map `error` → `FAIL` and `warning`/`suggestion` → `SUGGESTION` in those dimensions, citing the rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if vale or `.vale.ini` is unavailable.
|
||||
`vale-wrap.sh` resolves its own path and `.vale.ini` via `git rev-parse --show-toplevel`; pass the absolute paths shown above so the invocation holds from any cwd. It applies `.vale.ini`'s `Kyberforge` style — a deterministic prefilter for a subset of the Description/Patterns/Body dimensions below, not a replacement for Step 3. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one citing its rule ID (e.g. `Kyberforge.DescriptionOpener`). Skip and fall back to Step 3 judgment if vale or `.vale.ini` is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 3 judgment for the dimensions it would have covered.
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
|
||||
@@ -53,7 +53,7 @@ Work through each dimension internally. Collect findings only; report them in St
|
||||
|
||||
### Description
|
||||
|
||||
Vale's `Kyberforge.DescriptionOpener` (FAIL — "This skill..." openers) and `Kyberforge.VagueWording` (SUGGESTION — filler like "helps with", "utilize") alerts from Step 1 cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
|
||||
Vale's `Kyberforge.DescriptionOpener` ("This skill..." openers) and `Kyberforge.VagueWording` (filler like "helps with", "utilize") alerts from Step 1 — both FAILs — cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
|
||||
|
||||
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
|
||||
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
|
||||
@@ -70,7 +70,7 @@ For each sentence in the body, apply: *"Would the agent get this wrong without t
|
||||
- **Why rationale**: include/exclude rules explain why, not just what
|
||||
- **Control calibration**: prescriptive for fragile or critical sequences (e.g. a script invocation where flag order or exact arguments must not change); flexible where multiple approaches are valid
|
||||
|
||||
Vale's `Kyberforge.VagueQualifier` (SUGGESTION — vague filler like "clearly", "obviously") and `Kyberforge.SentenceOpenerThereIs` (SUGGESTION — sentences starting with "There is"/"There are") alerts from Step 1 cover pattern-matchable body-wide filler directly; report them as findings without re-deriving by judgment.
|
||||
Vale's `Kyberforge.SentenceOpenerThereIs` alert from Step 1 (FAIL — sentences starting with "There is"/"There are") covers pattern-matchable body-wide filler directly; report it as a finding without re-deriving by judgment.
|
||||
|
||||
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
|
||||
|
||||
|
||||
@@ -19,9 +19,9 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Vale exits non-zero whenever it finds an alert at or above `MinAlertLevel` — that's what makes it usable as a CI gate, not a sign the invocation failed. Read the output before concluding the command errored.
|
||||
- Vale's exit code is driven by `error`-level alerts only. `warning` and `suggestion` alerts are reported but still exit `0`. `MinAlertLevel` and `--minAlertLevel` control display, never the exit code — no flag makes warnings fail. A rule that must gate CI or a commit hook has to be `level: error`. This is the single most common way a Vale gate silently passes everything.
|
||||
- `vale ls-config` prints the fully-resolved, currently active configuration as JSON — the fastest way to check why a rule "isn't applying" is what's actually active, not what's written in `.vale.ini`.
|
||||
- Inline suppression syntax is format-specific: Markdown/MDX uses `{/* vale off */}` / `{/* vale on */}`, Org mode uses `# vale off` / `# vale on`. Don't assume one syntax works across formats.
|
||||
- Inline suppression syntax is format-specific: Markdown uses HTML comments `<!-- vale off -->` / `<!-- vale on -->`, MDX uses `{/* vale off */}` / `{/* vale on */}`, Org mode uses `# vale off` / `# vale on`. The MDX form does nothing in a plain `.md` file — the alert still fires. Don't assume one syntax works across formats.
|
||||
- Before calling the `vale` binary directly, check whether the target repo documents its own wrapper script for Vale (look in its README, CONTRIBUTING docs, pre-commit config, or a `scripts/` directory). Some projects wrap `vale` to work around real bugs — e.g. a scope that silently stops matching multi-line YAML block-scalar frontmatter fields — and calling bare `vale` in a repo that has such a wrapper silently skips whatever the wrapper works around. If a wrapper is documented, invoke it with the same arguments instead of calling `vale` directly; otherwise fall back to the default below.
|
||||
|
||||
## Running vale
|
||||
@@ -37,8 +37,8 @@ Key flags:
|
||||
| Flag | Purpose |
|
||||
|---|---|
|
||||
| `--output=<style>` | Output format/template: `CLI` (default, human-readable), `line` (compact, one alert per line, good for grep/piping), `JSON` (for programmatic parsing), or a custom template. |
|
||||
| `--minAlertLevel=<suggestion\|warning\|error>` | Overrides `MinAlertLevel` from `.vale.ini` for this run only, without editing config. |
|
||||
| `--no-exit` | Forces exit code `0` regardless of findings. Use in CI stages that should surface lint output without hard-failing the build. |
|
||||
| `--minAlertLevel=<suggestion\|warning\|error>` | Overrides `MinAlertLevel` from `.vale.ini` for this run only, without editing config. Filters what is displayed; does not affect the exit code. |
|
||||
| `--no-exit` | Suppresses the nonzero exit that `error`-level alerts would otherwise cause; a no-op when no rule is `error`-level. Use in CI stages that should surface lint output without hard-failing the build. |
|
||||
| `--ignore-syntax` | Treats input as plain text, skipping format-aware parsing — use when a file's syntax-aware parser produces noisy or wrong results. |
|
||||
|
||||
`vale sync` downloads the packages/styles declared in `.vale.ini` — that's a one-time-per-change setup step (vale-config's territory), not part of a normal lint run. If a run behaves as though no styles are active, that's a sign `vale sync` hasn't been run yet, not a `vale-run` problem.
|
||||
@@ -49,14 +49,15 @@ Prefer `--output=JSON` whenever the caller (a script, a CI step, another agent)
|
||||
|
||||
Scope the fix as narrowly as possible, in this order:
|
||||
|
||||
1. **One-off**: inline-suppress the specific text run with the format's `vale off`/`vale on` markup.
|
||||
2. **Recurring known-exception string, one rule**: disable that specific rule for that specific match inline (e.g. `{/* vale Style.Redundancy["ACT test","OTHER"] = NO */}` ... `= YES`), rather than the whole rule.
|
||||
3. **Known project term failing spell check**: add it to the style's `ignore` list, not an inline suppression.
|
||||
1. **Mentioning banned phrasing rather than using it**: wrap it in backticks or a fenced code block. Vale skips code spans and fences, so no suppression is needed at all. Try this before any suppression markup.
|
||||
2. **One-off**: inline-suppress the specific text run with the format's `vale off`/`vale on` markup.
|
||||
3. **Recurring known-exception string, one rule**: disable that specific rule for that specific match inline (e.g. `<!-- vale Style.Redundancy["ACT test","OTHER"] = NO -->` ... `= YES`), rather than the whole rule.
|
||||
4. **Known project term failing spell check**: add it to the style's `ignore` list, not an inline suppression.
|
||||
|
||||
Never disable a rule project-wide to fix one false positive — editing `.vale.ini`/`BasedOnStyles` is vale-config's job, and it silences the rule everywhere, not just the false-positive case.
|
||||
|
||||
If output looks wrong because Vale mis-parsed a file's format, rerun with `--ignore-syntax` before assuming the rule itself is broken.
|
||||
|
||||
For CI that fails solely because Vale returned non-zero on found alerts — not because the content is wrong for that pipeline stage — add `--no-exit` rather than disabling the rule.
|
||||
For CI that fails solely because Vale returned non-zero on `error`-level alerts — not because the content is wrong for that pipeline stage — add `--no-exit` rather than disabling the rule. If the failing alerts are warnings or suggestions, Vale is not what failed the build; look elsewhere.
|
||||
|
||||
If setting up Vale as a pre-commit hook or need the full inline-suppression/spelling-ignore syntax reference, read `references/troubleshooting.md`.
|
||||
|
||||
@@ -7,7 +7,14 @@ source_keys:
|
||||
|
||||
## Inline suppression syntax by format
|
||||
|
||||
Markdown/MDX:
|
||||
Markdown uses HTML comments — the MDX `{/* */}` form does not suppress anything in a plain `.md` file:
|
||||
```markdown
|
||||
<!-- vale off -->
|
||||
This text will be ignored.
|
||||
<!-- vale on -->
|
||||
```
|
||||
|
||||
MDX:
|
||||
```mdx
|
||||
{/* vale off */}
|
||||
This text will be ignored.
|
||||
@@ -25,6 +32,14 @@ This text will be ignored.
|
||||
|
||||
Targets one rule and specific known-exception strings, then re-enables — the preferred fix for a recurring false positive on a specific term, since it keeps the rule active everywhere else:
|
||||
|
||||
Markdown:
|
||||
```markdown
|
||||
<!-- vale Style.Redundancy["ACT test","OTHER"] = NO -->
|
||||
This is some text ACT test
|
||||
<!-- vale Style.Redundancy["ACT test","OTHER"] = YES -->
|
||||
```
|
||||
|
||||
MDX:
|
||||
```mdx
|
||||
{/* vale Style.Redundancy["ACT test","OTHER"] = NO */}
|
||||
This is some text ACT test
|
||||
@@ -50,7 +65,7 @@ If a file's syntax-aware parsing produces noisy or incorrect results (an unsuppo
|
||||
|
||||
## CI failing unexpectedly
|
||||
|
||||
If a CI job fails solely because Vale returns a non-zero exit code on found alerts — not because the content is actually wrong for that pipeline stage — add `--no-exit` rather than suppressing the rule itself. This preserves the lint output while not gating the build on it.
|
||||
Only `error`-level alerts make Vale exit non-zero; `warning` and `suggestion` alerts are printed but exit `0`. If a CI job fails solely because of `error`-level alerts — not because the content is actually wrong for that pipeline stage — add `--no-exit` rather than suppressing the rule itself. This preserves the lint output while not gating the build on it. If the alerts are warnings or suggestions, Vale did not fail the job — look elsewhere.
|
||||
|
||||
## pre-commit integration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user