test: fail the gate when a suite is skipped or never reports

Three gaps left by the previous round, all the same shape: a gate reporting
success having verified less than it appears to.

run_bats() hard-failed on a missing or non-executable runner but never checked
that the runner produced anything. An empty, executable run-bats.sh exits 0,
and the dispatcher printed a green summary with 166 bats tests silently absent.
It now requires an "N tests, M failures" line with a non-zero count.

run-tests.sh's skip listing is swallowed by pre-commit on a pass, so on a
machine without vale three suites exited 77 and the pre-push gate went green
having run 14 of 17. The hook now invokes it as --strict, where a skip fails
and the error names each suite and the reason it skipped. An ad-hoc local run
still skips gracefully -- at pre-push a skip means a documented dependency is
missing, which is a setup error, not a legitimate state. Deliberately not wired
to the vale downgrade's env var: one flag must not disarm two gates.

BATS_FILE_FLOOR is replaced by an expectation derived from git ls-files. A
floor of 8 against a real count of 10 let two files and eleven tests disappear
green, and the number needed an edit whenever a plugin was added. The derived
set needs no number, and catches an addition as well as a removal -- a .bats
file staged into the index and deleted from disk is now demanded back.

The vale opt-out announced its downgrade to nobody: pre-commit prints nothing
for a passing hook, so the summary line AGENTS.md tells the reader to check was
unreachable in exactly the situation it exists for. The hook is now verbose.

Also corrects the PROBES_CHECKED guard, whose commit message described a state
that cannot occur -- the .vale.ini loop errs first. Its two reachable triggers,
a gutted probe heredoc and a probe row naming a missing directory, had no test;
they do now, each asserting the guard is the sole cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT

tests/run-bats.sh

@@ -32,23 +32,74 @@ done < <(
         | sort
 )
 
-# A floor on the discovered file count, not merely a zero check, and a hard error
-# rather than the `exit 0` this used to be. Zero discovered files was the likelier
-# of the two silent-green failures -- a moved tests/ tree, a renamed skill
-# directory, or the `-not -path` exclusions above widening -- and it exited 0 with
-# a note on stderr nobody reads, while the zero-*result* guard further down was
-# already a hard error. A floor rather than `-gt 0` because the count collapsing
-# to 1 or 2 is the same failure as collapsing to 0 and only a floor names it.
-# Same reasoning, and the same "set it a little under the current count" rule, as
-# the per-glob floors in tests/test-vale-wrap.sh -- ordinary file churn does not
-# trip it, a broken or renamed path does.
+# The expected set of files is DERIVED from the index, not guessed at with a
+# hardcoded floor. This was `BATS_FILE_FLOOR=8` against a real count of 10, and
+# two files of slack is not a hypothetical margin -- deleting two .bats files
+# (agentsmd-audit/tests/ alone holds three) left the run reporting
+# "155 tests, 0 failures" and exiting 0 with 11 tests silently gone.
 #
-# BATS_FILE_FLOOR overrides it. That override exists for the fixture repos in
-# tests/test-run-bats.sh and tests/test-run-tests.sh, which hold one or two .bats
-# files by design; it is not an escape hatch for a real run.
-BATS_FILE_FLOOR="${BATS_FILE_FLOOR:-8}"
-if [[ ${#TEST_FILES[@]} -lt $BATS_FILE_FLOOR ]]; then
-    echo "Error: found ${#TEST_FILES[@]} .bats file(s) under $REPO_ROOT, below the floor of $BATS_FILE_FLOOR — the search path is wrong or the suite has been gutted" >&2
+# `git ls-files` gives the exact set for free. It catches a *removal* (a tracked
+# file gone from the worktree) and an *addition* the walk above missed (a tracked
+# file the `find` exclusions or a moved search root no longer reach), it needs no
+# magic number, and it needs no edit when a plugin is added or removed -- a newly
+# `git add`ed .bats file joins the expectation immediately, where a floor only
+# ever grows more slack as the suite grows.
+#
+# Direction matters: every tracked file must have been discovered, but a
+# discovered file need NOT be tracked. An untracked, not-yet-committed .bats file
+# is ordinary work in progress, and a file removed deliberately with `git rm` (or
+# a staged deletion) leaves the index, so an intentional removal passes while an
+# accidental disappearance fails. The same `-not -path` exclusions are reapplied
+# to the index listing so the two sides are compared over the same universe.
+#
+# The exact-equality check on `--show-toplevel` is what keeps this off the
+# fixture repos in tests/test-run-bats.sh and tests/test-run-tests.sh: those are
+# mktemp trees holding one or two .bats files by design, and git resolves no
+# worktree for them. That degradation is announced rather than silent, and the
+# zero-file check below is unconditional, so a non-git checkout still cannot run
+# on an empty set.
+EXPECTED_FILES=()
+DERIVED=false
+GIT_TOPLEVEL="$(git -C "$REPO_ROOT" rev-parse --show-toplevel 2>/dev/null || true)"
+if [[ -n "$GIT_TOPLEVEL" && "$GIT_TOPLEVEL" == "$REPO_ROOT" ]]; then
+    DERIVED=true
+    while IFS= read -r f; do
+        [[ -n "$f" ]] && EXPECTED_FILES+=("$REPO_ROOT/$f")
+    done < <(
+        git -C "$REPO_ROOT" ls-files -- '*.bats' \
+            | grep -Ev '(^|/)tests/bats/|(^|/)test_helper/|(^|/)\.claude/worktrees/' \
+            | sort || true
+    )
+else
+    echo "Note: $REPO_ROOT is not a git worktree root, so the expected .bats file set could not be derived from the index — only the zero-file check below applies" >&2
+fi
+
+if [[ "$DERIVED" == true && ${#EXPECTED_FILES[@]} -gt 0 ]]; then
+    MISSING=()
+    for expected in ${EXPECTED_FILES[@]+"${EXPECTED_FILES[@]}"}; do
+        found=false
+        for actual in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
+            if [[ "$actual" == "$expected" ]]; then
+                found=true
+                break
+            fi
+        done
+        [[ "$found" == true ]] || MISSING+=("${expected#"$REPO_ROOT"/}")
+    done
+    if [[ ${#MISSING[@]} -gt 0 ]]; then
+        echo "Error: ${#MISSING[@]} of ${#EXPECTED_FILES[@]} tracked .bats file(s) were not discovered under $REPO_ROOT — they were deleted without being removed from the index, or the search path/exclusions above no longer reach them:" >&2
+        for m in ${MISSING[@]+"${MISSING[@]}"}; do
+            echo "  $m" >&2
+        done
+        exit 1
+    fi
+fi
+
+# Unconditional, and separate from the derived check above: a tree with nothing
+# tracked (a tarball export, a fresh scaffold) still must not run on an empty set
+# and call it green. This was `exit 0` with a note on stderr nobody reads.
+if [[ ${#TEST_FILES[@]} -eq 0 ]]; then
+    echo "Error: found 0 .bats file(s) under $REPO_ROOT — the search path is wrong or the suite has been gutted" >&2
     exit 1
 fi
 
@@ -119,8 +170,9 @@ for f in ${TEST_FILES[@]+"${TEST_FILES[@]}"}; do
     fi
 done
 
-# Zero counted tests is never a clean run: enough files were found (a count under
-# the floor, zero included, exits non-zero above), so nothing was executed. Without this, a `bats` that emits
+# Zero counted tests is never a clean run: files were found (zero discovered
+# files, and any tracked file that went missing, exit non-zero above), so nothing
+# was executed. Without this, a `bats` that emits
 # nothing and exits 0 -- a broken binary, a formatter change, or a wholesale
 # `@test` removal -- reports "0 tests, 0 failures" and exits green, silently
 # turning a total harness failure into a pass.