fix(scripts): fail the vale and scope gates when they cannot verify
check-vale-style-sync.sh's glob-coverage probe silently self-disabled when vale was absent from PATH, exiting 0 on the one-character glob typo it exists to catch. pre-commit swallows a passing hook's output, so the pre-push hook reported Passed. The script already hard-fails on a bad REPO_ROOT for exactly this reason -- "a clean exit 0 here would read as 'checked, in sync' when nothing ran at all" -- and six of its assertions are vale invocations. Absence now fails; the opt-out is an env var that must be set deliberately, and it downgrades the run to text-level assertions while saying so in the summary. Neither script had a floor on its rewritten .apm/ paths, so relocating .apm/ made both exit 0 -- and this PR's whole change to them was a path rewrite, the exact edit that failure mode survives. A third gap the directory check could not see: relocating only assets/vale/ left both audit skill directories in place while every probe continued past its missing .vale.ini, skipping the whole table with FAIL=0. A zero-probe run is now an error. Both test suites encoded the vacuous pass as a passing case. Those cases are now scoped to "no plugins/kyberforge at all" and assert the fixture really lacks it, with new counterparts covering the drift shape and new positive cases requiring each script to report a non-zero inspected-target count. Also removes the HOOK_REGEX_CACHE memoization: every call site was a command substitution, so the writes happened in a subshell and the lookup always missed. Measured at 14ms of an ~870ms run, all of which is the six vale invocations. Deleted rather than repaired -- 35 lines claiming a benefit they never delivered is worse than no cache -- with a comment recording why. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
This commit is contained in:
@@ -31,8 +31,19 @@ NEW_SKILL="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-author/scripts/new-sk
|
||||
VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh"
|
||||
VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh"
|
||||
|
||||
# Floor on the four hardcoded `plugins/kyberforge/.apm/...` paths above. A
|
||||
# missing target is only a legitimate no-op for a repo that has no kyberforge
|
||||
# plugin at all; if `plugins/kyberforge/` IS here and the `.apm/` script under it
|
||||
# is not, these paths have gone stale and every fixture below silently does not
|
||||
# run. The whole exit is 0 either way, so a stale path is indistinguishable from
|
||||
# "all four implementations agree" — and a path rewrite is exactly the kind of
|
||||
# edit that would slip through it. Same reasoning as the REPO_ROOT guard above.
|
||||
for f in "$NEW_AGENT" "$NEW_SKILL" "$VALIDATE" "$VALIDATE_PROVENANCE"; do
|
||||
if [[ ! -f "$f" ]]; then
|
||||
if [[ -d "$REPO_ROOT/plugins/kyberforge" ]]; then
|
||||
echo "Scope walk-up sync check failed: $REPO_ROOT/plugins/kyberforge exists but $f does not — this script's .apm/ paths have gone stale, so none of the walk-up fixtures ran. Update them to wherever the agent-author/agent-audit/skill-author scripts now live." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "Scope walk-up sync check: $f not found — kyberforge agent-author/agent-audit/skill-author skills not present, nothing to check." >&2
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -28,7 +28,19 @@ err() { echo " FAIL: $1" >&2; FAIL=$((FAIL + 1)); }
|
||||
SKILL_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit"
|
||||
AGENT_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit"
|
||||
|
||||
# Floor on the hardcoded `plugins/kyberforge/.apm/...` paths above. Neither copy
|
||||
# present is only a legitimate no-op for a repo that has no kyberforge plugin at
|
||||
# all. If `plugins/kyberforge/` IS here and the `.apm/` targets under it are not,
|
||||
# the paths in this script have gone stale — a rename or relocation of `.apm/`
|
||||
# would otherwise turn every assertion below into a silent exit 0, which reads as
|
||||
# "checked, in sync" exactly like the REPO_ROOT case above. That matters most for
|
||||
# the change that introduced these paths: a path rewrite is precisely the edit
|
||||
# this would survive unnoticed.
|
||||
if [[ ! -d "$SKILL_AUDIT" && ! -d "$AGENT_AUDIT" ]]; then
|
||||
if [[ -d "$REPO_ROOT/plugins/kyberforge" ]]; then
|
||||
echo "Vale style sync check failed: $REPO_ROOT/plugins/kyberforge exists but neither $SKILL_AUDIT nor $AGENT_AUDIT does — this script's .apm/ paths have gone stale, so nothing was checked. Update them to wherever the audit skills now live." >&2
|
||||
exit 1
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -183,49 +195,31 @@ fi
|
||||
# it — silently masking exactly the kind of hook-rescoping drift this script
|
||||
# exists to catch.
|
||||
#
|
||||
# Cached per (skill, manifest) pair (parallel HOOK_REGEX_CACHE_KEYS/_VALS
|
||||
# arrays, populated lazily) because the final validation loop below probes
|
||||
# agent-audit's two manifests across three probe shapes; without the cache,
|
||||
# each repeated (skill, manifest) pairing would re-parse the same manifest
|
||||
# file from scratch for no new information. Plain indexed arrays, not
|
||||
# `declare -A`: associative arrays are bash 4.0+ and this script must run on
|
||||
# macOS's stock bash 3.2. Only ${#arr[@]} (always safe on an empty/unset array
|
||||
# under `set -u`) and index access are used below — never a bare `${arr[@]}`
|
||||
# expansion, which aborts on bash < 4.4 under nounset.
|
||||
HOOK_REGEX_CACHE_KEYS=()
|
||||
HOOK_REGEX_CACHE_VALS=()
|
||||
# Deliberately NOT memoized. The probe loop below calls this 12 times over the
|
||||
# same two small manifests, which measures at 14ms against a ~870ms run (the six
|
||||
# vale invocations are the wall clock). A previous memoization attempt was inert
|
||||
# anyway: every call site is `x="$(hook_file_regexes ...)"`, a command
|
||||
# substitution, so the cache writes landed in a subshell and the lookup never
|
||||
# hit. Re-parsing is the honest, working version of a saving too small to buy.
|
||||
hook_file_regexes() {
|
||||
local skill="$1" manifest="$2" raw result idx=0
|
||||
local cache_key="$skill|$manifest"
|
||||
while [[ $idx -lt ${#HOOK_REGEX_CACHE_KEYS[@]} ]]; do
|
||||
if [[ "${HOOK_REGEX_CACHE_KEYS[$idx]}" == "$cache_key" ]]; then
|
||||
printf '%s' "${HOOK_REGEX_CACHE_VALS[$idx]}"
|
||||
return
|
||||
fi
|
||||
idx=$((idx + 1))
|
||||
done
|
||||
result="$(
|
||||
if [[ -f "$manifest" ]]; then
|
||||
awk -v skill="$skill" '
|
||||
function flush() {
|
||||
if (entry ~ skill "/scripts/vale-wrap.sh" && files != "") print files
|
||||
entry = ""; files = ""
|
||||
}
|
||||
/^[ \t]*-[ \t]*id:/ { flush() }
|
||||
/^[ \t]*entry:/ { entry = $0 }
|
||||
/^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) }
|
||||
END { flush() }
|
||||
' "$manifest" | while IFS= read -r raw; do
|
||||
# Strip the surrounding YAML quotes; the regex itself never carries them.
|
||||
raw="${raw%\'}"; raw="${raw#\'}"
|
||||
raw="${raw%\"}"; raw="${raw#\"}"
|
||||
printf '%s\n' "$raw"
|
||||
done
|
||||
fi
|
||||
)"
|
||||
HOOK_REGEX_CACHE_KEYS[${#HOOK_REGEX_CACHE_KEYS[@]}]="$cache_key"
|
||||
HOOK_REGEX_CACHE_VALS[${#HOOK_REGEX_CACHE_VALS[@]}]="$result"
|
||||
printf '%s' "$result"
|
||||
local skill="$1" manifest="$2" raw
|
||||
if [[ -f "$manifest" ]]; then
|
||||
awk -v skill="$skill" '
|
||||
function flush() {
|
||||
if (entry ~ skill "/scripts/vale-wrap.sh" && files != "") print files
|
||||
entry = ""; files = ""
|
||||
}
|
||||
/^[ \t]*-[ \t]*id:/ { flush() }
|
||||
/^[ \t]*entry:/ { entry = $0 }
|
||||
/^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) }
|
||||
END { flush() }
|
||||
' "$manifest" | while IFS= read -r raw; do
|
||||
# Strip the surrounding YAML quotes; the regex itself never carries them.
|
||||
raw="${raw%\'}"; raw="${raw#\'}"
|
||||
raw="${raw%\"}"; raw="${raw#\"}"
|
||||
printf '%s\n' "$raw"
|
||||
done
|
||||
fi
|
||||
}
|
||||
|
||||
# True if $1 matches at least one newline-delimited regex in $2.
|
||||
@@ -265,10 +259,28 @@ vale_flags_path() {
|
||||
printf '%s\n' "$out" | grep -qF "Kyberforge.VagueWording"
|
||||
}
|
||||
|
||||
# Missing vale is a HARD FAILURE, not a warning. Six of this script's assertions
|
||||
# — one glob probe per path below — are `vale --config` invocations, and they are
|
||||
# the only ones that catch the defect the whole `.vale.ini coverage` section was
|
||||
# written for: the one-character glob typo (`[**/SKILL.md]` -> `[**/SKILLS.md]`)
|
||||
# that leaves every text-level assertion clean while vale lints zero files. As a
|
||||
# warning this self-disabled on exactly that mutation and exited 0, and since
|
||||
# pre-commit swallows a passing hook's output the stderr line was never seen —
|
||||
# the pre-push hook reported `Passed`. That is the same "clean exit 0 reads as
|
||||
# 'checked, in sync' when nothing ran" failure the REPO_ROOT guard at the top of
|
||||
# this file already refuses to allow.
|
||||
#
|
||||
# The opt-out exists for a machine that genuinely cannot install vale, and it is
|
||||
# an env var that has to be set on purpose — never mere absence of the binary.
|
||||
# Setting it downgrades the run to text-level assertions only and says so.
|
||||
VALE_AVAILABLE=true
|
||||
if ! command -v vale >/dev/null 2>&1; then
|
||||
VALE_AVAILABLE=false
|
||||
echo " WARNING: vale is not installed — .vale.ini glob coverage was NOT verified. Install it (https://vale.sh/docs/vale-cli/installation/) before trusting a clean run." >&2
|
||||
if [[ "${CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE:-}" == "1" ]]; then
|
||||
echo " WARNING: vale is not installed and CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 — .vale.ini glob coverage was NOT verified, only the text-level assertions ran. A clean result here does not mean the globs cover what their hooks lint." >&2
|
||||
else
|
||||
err "vale is not installed, so none of the .vale.ini glob-coverage probes ran — a glob typo that silently lints zero files is invisible without them. Install it (https://vale.sh/docs/vale-cli/installation/), or set CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 to accept a text-only run"
|
||||
fi
|
||||
fi
|
||||
|
||||
# One representative path per file shape the prefilter is supposed to cover,
|
||||
@@ -284,11 +296,13 @@ fi
|
||||
# narrow out of sync with .pre-commit-hooks.yaml's without either manifest's
|
||||
# own hook breaking (each still matches real files on its own), so nothing
|
||||
# else would catch it.
|
||||
PROBES_CHECKED=0
|
||||
while IFS='|' read -r skill rel scope; do
|
||||
[[ -n "$skill" ]] || continue
|
||||
dir="$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill"
|
||||
ini="$dir/assets/vale/.vale.ini"
|
||||
[[ -f "$ini" ]] || continue
|
||||
PROBES_CHECKED=$((PROBES_CHECKED + 1))
|
||||
|
||||
hooks_regexes="$(hook_file_regexes "$skill" "$REPO_ROOT/.pre-commit-hooks.yaml")"
|
||||
config_regexes="$(hook_file_regexes "$skill" "$REPO_ROOT/.pre-commit-config.yaml")"
|
||||
@@ -334,7 +348,26 @@ agent-audit|.claude/agents/demo.md|hooks-only
|
||||
agent-audit|copilot/demo.agent.md|hooks-only
|
||||
EOF_PROBE
|
||||
|
||||
# Second floor, on the probe table rather than the directory paths: every probe
|
||||
# `continue`s when its skill's `.vale.ini` is absent, so a relocation of
|
||||
# `assets/vale/` alone — with both audit skill directories still in place, which
|
||||
# the guard at the top of this file therefore cannot see — would skip the entire
|
||||
# table and leave FAIL at 0. Zero probes checked is never a valid result.
|
||||
if [[ $PROBES_CHECKED -eq 0 ]]; then
|
||||
err "no probe path was checked — every probe skill's assets/vale/.vale.ini is missing, so the glob-coverage section verified nothing at all"
|
||||
fi
|
||||
|
||||
if [[ $FAIL -gt 0 ]]; then
|
||||
echo "Vale style sync check failed: $FAIL error(s). For a drifted wrapper or style, agent-audit's copy is canonical — run scripts/sync-vale-styles.sh to regenerate skill-audit's copy, then commit both. A .vale.ini finding is not drift and sync-vale-styles.sh will not fix it: edit that file's own StylesPath, BasedOnStyles or glob sections." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A clean run says what it actually inspected. Silence is what let the vacuous
|
||||
# passes above look identical to real ones, and it is what made "did this script
|
||||
# do any work against the real repo?" untestable from outside — the counts below
|
||||
# are what tests/test-check-vale-style-sync.sh asserts a non-zero floor on.
|
||||
if [[ "$VALE_AVAILABLE" == true ]]; then
|
||||
echo "Vale style sync check passed: 2 .vale.ini file(s) checked, $PROBES_CHECKED glob probe(s) verified with vale."
|
||||
else
|
||||
echo "Vale style sync check passed (text-level only, vale unavailable): 2 .vale.ini file(s) checked, 0 glob probe(s) verified."
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user