fix(lint): make the Vale gate actually gate, drop VagueQualifier
Round-3 review of PR #85 found the "enforcing" pre-commit hook enforced nothing. Vale's exit code keys on error-level alerts alone: five of the six rules were level: warning, so they exited 0, and pre-commit hides output from a passing hook — the alerts were invisible and blocked nothing. ADR-0013 rejected a report-only trial tier and then shipped one by accident. Flatten every rule to level: error. Vale's own exit code is then correct, so the hook entry drops to a bare vale-wrap.sh call and the graded error->FAIL / warning->SUGGESTION mapping disappears from both audit skills: every alert is a FAIL, in the gate and the audit alike. No ignorable tier, matching shellcheck, the test suite and conventional-pre-commit. Delete Kyberforge.VagueQualifier. Measured against the 41 skill/agent files as they stood before the rule ever ran: 2 hits. One marginal ("very different" -> "fundamentally different"), one an unfixable false positive — caveman/SKILL.md quotes "of course" as an example of filler, a mention not a use — which forced the only Vale suppression comments in the repo. Those four lines go with it; two of them were dead anyway, suppressing a frontmatter-scoped rule on a body line. Held-out prose (273 files) fired 15 times, 9 inside out-of-scope research examples and the rest one word in two idioms in a single doc. SentenceOpenerThereIs survives: 22 held-out hits, both in-corpus hits clean rewrites, zero suppressions. Widen .vale.ini's globs to [**/SKILL.md], [**/agents/*.md] and [**/*.agent.md]. The plugins/*/-prefixed globs scoped nothing — Vale's * crosses /, so they already matched docs/research/examples/**/agents/*.md and assets/templates/SKILL.md, the two paths CONTEXT.md claimed they excluded. Scoping is and was the hook's files: regex. The old globs also hid a silent false negative: a skill outside plugins/ matched no section, so Vale reported 0 files and exited 0, which both audits read as clean. They now treat a 0-file run as NOT RUN and fall back to full judgment. Also: - vale-wrap.sh resolves relative --config values and file arguments against the caller's cwd, as vale does, instead of the repo root, which hard-errored from a subdirectory and silently skipped flattening for file args that did not resolve from the root. Absolute paths inside the cwd are relativized so reports cite resolvable paths, not scratch ones. - vale-run's exit-code model was documented backwards ("exits non-zero whenever it finds an alert at or above MinAlertLevel") and would have led anyone following it to build a gate that passes everything. Its Markdown suppression syntax was MDX-only and does not suppress in .md; corrected in the skill and its troubleshooting reference, with backtick/fence exemption documented as the first resort. - skill-size-check.sh fails only above 500 lines, agreeing with skill-audit's validate.sh <= 500 pass. - ADR-0013 and CONTEXT.md amended to match, recording why graded severities cannot gate. Verified: 9 test scripts / 15 vale-wrap cases pass; vale-audit-prefilter, skill-size-check and shellcheck pass --all-files; check-manifests and claude plugin validate --strict clean. New tests fail against the old script (3 of them) and pass against the new one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
This commit is contained in:
@@ -1,11 +1,18 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Enforces agentskills.io's skill-authoring.md guidance: keep SKILL.md under 500
|
||||
# lines and roughly 5,000 tokens, so the full body doesn't crowd out conversation
|
||||
# history and other active skills once loaded into context. Vale can't express
|
||||
# a whole-file length ceiling (its checks operate on text patterns, not raw
|
||||
# file size), so this is a plain script instead of a Vale rule.
|
||||
# Enforces agentskills.io's skill-authoring.md guidance: keep SKILL.md within
|
||||
# 500 lines and roughly 5,000 tokens, so the full body doesn't crowd out
|
||||
# conversation history and other active skills once loaded into context. Vale
|
||||
# can't express a whole-file length ceiling (its checks operate on text
|
||||
# patterns, not raw file size), so this is a plain script instead of a Vale
|
||||
# rule.
|
||||
#
|
||||
# Both ceilings are inclusive: a file at exactly MAX_LINES or MAX_WORDS passes,
|
||||
# and only one past it fails. That matches skill-audit/scripts/validate.sh,
|
||||
# which has always used `line_count <= 500` as its pass condition — the two
|
||||
# previously disagreed at exactly 500 lines, so a SKILL.md could pass its own
|
||||
# audit and still be blocked by the commit hook.
|
||||
#
|
||||
# Token counts aren't computed exactly here — word count (`wc -w`) is used as
|
||||
# a proxy. This repo's own SKILL.md corpus measures ~5.7-6.5 characters per
|
||||
@@ -26,8 +33,8 @@ for f in "$@"; do
|
||||
# Python's splitlines() semantics (used by skill-audit/scripts/validate.sh
|
||||
# for its own line count) — `wc -l` undercounts by 1 in that case.
|
||||
lines=$(awk 'END{print NR}' "$f")
|
||||
if (( lines >= MAX_LINES )); then
|
||||
echo "ERROR: $f has $lines lines, at or over the $MAX_LINES-line ceiling (agentskills.io skill-authoring.md)" >&2
|
||||
if (( lines > MAX_LINES )); then
|
||||
echo "ERROR: $f has $lines lines, exceeding the $MAX_LINES-line ceiling (agentskills.io skill-authoring.md)" >&2
|
||||
FAIL=1
|
||||
fi
|
||||
|
||||
|
||||
@@ -8,56 +8,104 @@ set -euo pipefail
|
||||
# (padding with blank lines so every other line number is unchanged), then runs
|
||||
# the real `vale` binary against the copies. Drop-in replacement for calling
|
||||
# `vale` directly: same args, same exit code.
|
||||
#
|
||||
# "Same args" means relative paths — `--config` values and file arguments alike
|
||||
# — resolve against the caller's current directory, exactly as bare `vale`
|
||||
# resolves them. (An earlier version resolved them against the repo root, an
|
||||
# invented convention that hard-errored on `--config ../../.vale.ini` from a
|
||||
# subdirectory and, worse, silently dropped file arguments that didn't happen to
|
||||
# resolve from the repo root — skipping the flattening this script exists for.)
|
||||
#
|
||||
# Vale prints each file path exactly as it was handed to it, so the scratch tree
|
||||
# mirrors the caller's absolute cwd: a relative file argument is passed through
|
||||
# verbatim and resolves to its flattened copy, keeping the report byte-identical
|
||||
# to bare `vale`'s. An absolute file argument inside the cwd is relativized to
|
||||
# keep that property. Only an absolute path outside the cwd is rewritten to its
|
||||
# scratch copy and so reports a scratch path — unavoidable, since a file can
|
||||
# only be read from where it actually is.
|
||||
|
||||
repo_root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
|
||||
cwd="$(pwd -P)"
|
||||
|
||||
vale_args=()
|
||||
files=()
|
||||
file_args=()
|
||||
config_next=false
|
||||
for arg in "$@"; do
|
||||
if [[ "$config_next" == true ]]; then
|
||||
config_next=false
|
||||
if [[ "$arg" == /* ]]; then
|
||||
vale_args+=("$arg")
|
||||
else
|
||||
vale_args+=("$repo_root/$arg")
|
||||
vale_args+=("$cwd/$arg")
|
||||
fi
|
||||
config_next=false
|
||||
continue
|
||||
fi
|
||||
if [[ "$arg" == "--config" ]]; then
|
||||
vale_args+=("$arg")
|
||||
config_next=true
|
||||
continue
|
||||
fi
|
||||
if [[ "$arg" == --config=* ]]; then
|
||||
cfg="${arg#--config=}"
|
||||
if [[ "$cfg" == /* ]]; then
|
||||
vale_args+=("--config=$cfg")
|
||||
case "$arg" in
|
||||
--config)
|
||||
vale_args+=("$arg")
|
||||
config_next=true
|
||||
continue
|
||||
;;
|
||||
--config=/*)
|
||||
vale_args+=("$arg")
|
||||
continue
|
||||
;;
|
||||
--config=*)
|
||||
vale_args+=("--config=$cwd/${arg#--config=}")
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
# `-f` resolves relative paths against the caller's cwd, same as vale does.
|
||||
if [[ "$arg" != -* && -f "$arg" ]]; then
|
||||
# An absolute path inside the caller's cwd is relativized so the report cites
|
||||
# a path that resolves against the real tree. Left absolute, it would be
|
||||
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
|
||||
# path to a file that is deleted on exit, which reads as a bug in any report
|
||||
# quoting it. Absolute paths outside the cwd have no relative form and keep
|
||||
# the scratch-path behaviour documented above.
|
||||
if [[ "$arg" == "$cwd"/* ]]; then
|
||||
file_args+=("${arg#"$cwd"/}")
|
||||
else
|
||||
vale_args+=("--config=$repo_root/$cfg")
|
||||
file_args+=("$arg")
|
||||
fi
|
||||
continue
|
||||
fi
|
||||
if [[ "$arg" != -* && -f "$repo_root/$arg" ]]; then
|
||||
files+=("$arg")
|
||||
elif [[ "$arg" == /* && -f "$arg" && "$arg" == "$repo_root"/* ]]; then
|
||||
files+=("${arg#"$repo_root"/}")
|
||||
else
|
||||
vale_args+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ ${#files[@]} -eq 0 ]]; then
|
||||
if [[ ${#file_args[@]} -eq 0 ]]; then
|
||||
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
|
||||
# block waiting on a pipe that will never carry content.
|
||||
exec vale "${vale_args[@]}" < /dev/null
|
||||
fi
|
||||
|
||||
tmpdir="$(mktemp -d)"
|
||||
tmpdir="$(realpath -m "$(mktemp -d)")"
|
||||
trap 'rm -rf "$tmpdir"' EXIT
|
||||
|
||||
for rel in "${files[@]}"; do
|
||||
dest="$tmpdir/$rel"
|
||||
# Mirror of the caller's cwd inside the scratch tree; relative file arguments
|
||||
# are resolved from here.
|
||||
mirror="$tmpdir$cwd"
|
||||
mkdir -p "$mirror"
|
||||
|
||||
argv_files=()
|
||||
for arg in "${file_args[@]}"; do
|
||||
if [[ "$arg" == /* ]]; then
|
||||
dest="$tmpdir$arg"
|
||||
else
|
||||
dest="$mirror/$arg"
|
||||
fi
|
||||
dest="$(realpath -m "$dest")"
|
||||
# A file argument with enough leading `..` to climb past the mirror root would
|
||||
# write outside the scratch dir. The real filesystem clamps such a path at
|
||||
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
|
||||
case "$dest" in
|
||||
"$tmpdir"/*) ;;
|
||||
*)
|
||||
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
mkdir -p "$(dirname "$dest")"
|
||||
python3 - "$repo_root/$rel" "$dest" <<'PYTHON'
|
||||
python3 - "$arg" "$dest" <<'PYTHON'
|
||||
import re
|
||||
import sys
|
||||
|
||||
@@ -118,7 +166,12 @@ if fm_match:
|
||||
with open(dest, 'w') as fh:
|
||||
fh.write(content)
|
||||
PYTHON
|
||||
if [[ "$arg" == /* ]]; then
|
||||
argv_files+=("$dest")
|
||||
else
|
||||
argv_files+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
cd "$tmpdir"
|
||||
vale "${vale_args[@]}" "${files[@]}"
|
||||
cd "$mirror"
|
||||
vale "${vale_args[@]}" "${argv_files[@]}"
|
||||
|
||||
Reference in New Issue
Block a user