test(gates): pin hook wiring and enforce the bats TAP plan
The repo's gates were not pinned to their wiring. Deleting the check-skill-version-bump block from .pre-commit-config.yaml left the whole suite green; deleting eight blocks at once, run-tests among them, also left it green. Only 4 of 20 hook ids had their wiring pinned anywhere, so a merge conflict resolved badly could stop the suite running at pre-push forever while every test still reported green. test-adr0020-contract now derives the repo-authored hooks from the repo: local entries and pins each one's id, entry and stages against an explicit expected set, both directions, with the same non-vacuity guards the file already applies to its own fixtures. Upstream hooks and their rev: values are untouched, so a rev bump does not churn the test. Mutation-checked: a removed block, a repointed entry and a hook moved off pre-push each go red; a rev bump, a comment edit and reordering stay green. 29 -> 44 assertions. run-bats computed each file's TAP plan and then discarded it, so a process printing "1..10", three ok lines and exit 0 was counted as "3 tests, 0 failures" with seven tests silently gone. That is exactly the wrapper-swallows-the-status case the runner's own comment puts in its threat model, and the plan was the only surviving signal. The plan is now enforced in both directions when a file emits exactly one. Also: test-no-pipefail-early-exit-grep's live-tree floor goes from 20 to 50 against an actual 57, matching test-vale-wrap's per-glob discipline, and test-vale-wrap's header names the real path to vale-wrap.sh. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2
This commit is contained in:
@@ -180,6 +180,15 @@ for f in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
|
||||
# file bats really did run, so it has to be distinguishable from a file that
|
||||
# produced nothing whatsoever.
|
||||
file_plan="$(grep -c '^1\.\.[0-9]' "$SCRATCH_ROOT/$i.log" || true)"
|
||||
# The planned count itself, extracted only when the file emitted exactly one
|
||||
# plan line. Zero plans is the broken-harness case the aggregate guard below
|
||||
# names, and two or more means the log is not one file's TAP stream at all --
|
||||
# in neither case does "the planned count" mean anything, so the per-file
|
||||
# comparison is skipped and the previous behaviour stands.
|
||||
file_planned=""
|
||||
if [[ "$file_plan" -eq 1 ]]; then
|
||||
file_planned="$(sed -n 's/^1\.\.\([0-9][0-9]*\).*$/\1/p' "$SCRATCH_ROOT/$i.log")"
|
||||
fi
|
||||
# String-compared below, not `-ne`. `-ne` is arithmetic and bash evaluates an
|
||||
# empty string as 0 there -- `[[ "" -ne 0 ]]` is false -- so an *empty* status
|
||||
# file read as a clean exit. The `|| echo 1` fallback only covers a *missing*
|
||||
@@ -189,14 +198,28 @@ for f in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
|
||||
TOTAL_OK=$((TOTAL_OK + file_ok))
|
||||
TOTAL_NOT_OK=$((TOTAL_NOT_OK + file_not_ok))
|
||||
TOTAL_PLANS=$((TOTAL_PLANS + file_plan))
|
||||
# Two independent failure signals, deliberately OR-ed: a file can report `not
|
||||
# ok` lines while its process still exits 0 (a bats formatter or wrapper that
|
||||
# swallows the status), and a file can exit non-zero having emitted no `not
|
||||
# ok` at all (a crash, a timeout, an unbound variable in setup_file). Real
|
||||
# bats normally emits both at once, so each signal masks the other and
|
||||
# dropping either half is invisible without tests that produce one without
|
||||
# the other -- tests/test-run-bats.sh has those.
|
||||
if [[ "$file_not_ok" -gt 0 || "$status" != "0" ]]; then
|
||||
# Three independent failure signals, deliberately OR-ed: a file can report
|
||||
# `not ok` lines while its process still exits 0 (a bats formatter or wrapper
|
||||
# that swallows the status), a file can exit non-zero having emitted no `not
|
||||
# ok` at all (a crash, a timeout, an unbound variable in setup_file), and a
|
||||
# file can emit FEWER results than its own plan line promised. Real bats
|
||||
# normally emits all three consistently, so each signal masks the others and
|
||||
# dropping any one of them is invisible without tests that produce one without
|
||||
# the rest -- tests/test-run-bats.sh has those.
|
||||
#
|
||||
# The plan is the third signal and it is now enforced, not merely counted. It
|
||||
# is the one that survives precisely the wrapper-swallows-the-status case
|
||||
# named above: a process printing `1..10`, three `ok` lines and exit 0 used to
|
||||
# be counted as "3 tests, 0 failures" and go green with seven tests silently
|
||||
# gone, because the plan was computed for the aggregate zero-count guard below
|
||||
# and then discarded. Mismatch either way is a failure -- more results than
|
||||
# planned is as broken a TAP stream as fewer.
|
||||
file_short=false
|
||||
if [[ -n "$file_planned" && $((file_ok + file_not_ok)) -ne "$file_planned" ]]; then
|
||||
file_short=true
|
||||
echo "Error: $rel planned $file_planned test(s) but emitted $((file_ok + file_not_ok)) result line(s) — the run was truncated, or its exit status was swallowed" >&2
|
||||
fi
|
||||
if [[ "$file_not_ok" -gt 0 || "$status" != "0" || "$file_short" == true ]]; then
|
||||
FAIL=1
|
||||
fi
|
||||
done
|
||||
|
||||
Reference in New Issue
Block a user