test(gates): pin hook wiring and enforce the bats TAP plan

The repo's gates were not pinned to their wiring. Deleting the
check-skill-version-bump block from .pre-commit-config.yaml left the
whole suite green; deleting eight blocks at once, run-tests among them,
also left it green. Only 4 of 20 hook ids had their wiring pinned
anywhere, so a merge conflict resolved badly could stop the suite
running at pre-push forever while every test still reported green.

test-adr0020-contract now derives the repo-authored hooks from the
repo: local entries and pins each one's id, entry and stages against an
explicit expected set, both directions, with the same non-vacuity
guards the file already applies to its own fixtures. Upstream hooks and
their rev: values are untouched, so a rev bump does not churn the test.
Mutation-checked: a removed block, a repointed entry and a hook moved
off pre-push each go red; a rev bump, a comment edit and reordering
stay green. 29 -> 44 assertions.

run-bats computed each file's TAP plan and then discarded it, so a
process printing "1..10", three ok lines and exit 0 was counted as
"3 tests, 0 failures" with seven tests silently gone. That is exactly
the wrapper-swallows-the-status case the runner's own comment puts in
its threat model, and the plan was the only surviving signal. The plan
is now enforced in both directions when a file emits exactly one.

Also: test-no-pipefail-early-exit-grep's live-tree floor goes from 20 to
50 against an actual 57, matching test-vale-wrap's per-glob discipline,
and test-vale-wrap's header names the real path to vale-wrap.sh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2
This commit is contained in:
2026-09-20 12:34:07 +00:00
parent 8cfd54f925
commit 384756b343
5 changed files with 271 additions and 10 deletions

View File

@@ -180,6 +180,15 @@ for f in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
# file bats really did run, so it has to be distinguishable from a file that
# produced nothing whatsoever.
file_plan="$(grep -c '^1\.\.[0-9]' "$SCRATCH_ROOT/$i.log" || true)"
# The planned count itself, extracted only when the file emitted exactly one
# plan line. Zero plans is the broken-harness case the aggregate guard below
# names, and two or more means the log is not one file's TAP stream at all --
# in neither case does "the planned count" mean anything, so the per-file
# comparison is skipped and the previous behaviour stands.
file_planned=""
if [[ "$file_plan" -eq 1 ]]; then
file_planned="$(sed -n 's/^1\.\.\([0-9][0-9]*\).*$/\1/p' "$SCRATCH_ROOT/$i.log")"
fi
# String-compared below, not `-ne`. `-ne` is arithmetic and bash evaluates an
# empty string as 0 there -- `[[ "" -ne 0 ]]` is false -- so an *empty* status
# file read as a clean exit. The `|| echo 1` fallback only covers a *missing*
@@ -189,14 +198,28 @@ for f in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
TOTAL_OK=$((TOTAL_OK + file_ok))
TOTAL_NOT_OK=$((TOTAL_NOT_OK + file_not_ok))
TOTAL_PLANS=$((TOTAL_PLANS + file_plan))
# Two independent failure signals, deliberately OR-ed: a file can report `not
# ok` lines while its process still exits 0 (a bats formatter or wrapper that
# swallows the status), and a file can exit non-zero having emitted no `not
# ok` at all (a crash, a timeout, an unbound variable in setup_file). Real
# bats normally emits both at once, so each signal masks the other and
# dropping either half is invisible without tests that produce one without
# the other -- tests/test-run-bats.sh has those.
if [[ "$file_not_ok" -gt 0 || "$status" != "0" ]]; then
# Three independent failure signals, deliberately OR-ed: a file can report
# `not ok` lines while its process still exits 0 (a bats formatter or wrapper
# that swallows the status), a file can exit non-zero having emitted no `not
# ok` at all (a crash, a timeout, an unbound variable in setup_file), and a
# file can emit FEWER results than its own plan line promised. Real bats
# normally emits all three consistently, so each signal masks the others and
# dropping any one of them is invisible without tests that produce one without
# the rest -- tests/test-run-bats.sh has those.
#
# The plan is the third signal and it is now enforced, not merely counted. It
# is the one that survives precisely the wrapper-swallows-the-status case
# named above: a process printing `1..10`, three `ok` lines and exit 0 used to
# be counted as "3 tests, 0 failures" and go green with seven tests silently
# gone, because the plan was computed for the aggregate zero-count guard below
# and then discarded. Mismatch either way is a failure -- more results than
# planned is as broken a TAP stream as fewer.
file_short=false
if [[ -n "$file_planned" && $((file_ok + file_not_ok)) -ne "$file_planned" ]]; then
file_short=true
echo "Error: $rel planned $file_planned test(s) but emitted $((file_ok + file_not_ok)) result line(s) — the run was truncated, or its exit status was swallowed" >&2
fi
if [[ "$file_not_ok" -gt 0 || "$status" != "0" || "$file_short" == true ]]; then
FAIL=1
fi
done