test: fail the gate when a suite is skipped or never reports

Three gaps left by the previous round, all the same shape: a gate reporting
success having verified less than it appears to.

run_bats() hard-failed on a missing or non-executable runner but never checked
that the runner produced anything. An empty, executable run-bats.sh exits 0,
and the dispatcher printed a green summary with 166 bats tests silently absent.
It now requires an "N tests, M failures" line with a non-zero count.

run-tests.sh's skip listing is swallowed by pre-commit on a pass, so on a
machine without vale three suites exited 77 and the pre-push gate went green
having run 14 of 17. The hook now invokes it as --strict, where a skip fails
and the error names each suite and the reason it skipped. An ad-hoc local run
still skips gracefully -- at pre-push a skip means a documented dependency is
missing, which is a setup error, not a legitimate state. Deliberately not wired
to the vale downgrade's env var: one flag must not disarm two gates.

BATS_FILE_FLOOR is replaced by an expectation derived from git ls-files. A
floor of 8 against a real count of 10 let two files and eleven tests disappear
green, and the number needed an edit whenever a plugin was added. The derived
set needs no number, and catches an addition as well as a removal -- a .bats
file staged into the index and deleted from disk is now demanded back.

The vale opt-out announced its downgrade to nobody: pre-commit prints nothing
for a passing hook, so the summary line AGENTS.md tells the reader to check was
unreachable in exactly the situation it exists for. The hook is now verbose.

Also corrects the PROBES_CHECKED guard, whose commit message described a state
that cannot occur -- the .vale.ini loop errs first. Its two reachable triggers,
a gutted probe heredoc and a probe row naming a missing directory, had no test;
they do now, each asserting the guard is the sole cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT

tests/test-run-bats.sh

@@ -70,20 +70,33 @@ seed_bats_files() {
 
 # Runs the fixture's run-bats.sh, capturing output and exit code separately.
 #
-# BATS_FILE_FLOOR is lowered to 1 for every case that is not specifically about
-# the floor: these fixtures hold one or two .bats files by design, which is well
-# under the real repo's floor. FAKE_FLOOR lets the floor cases opt back into the
-# script's own default. TMPDIR is a private per-run directory so a stub can find
-# the scratch dir run-bats.sh mktemp'd for itself -- see case 7.
+# No file-count knob is passed any more, and none is needed: the expected file
+# set is derived from `git ls-files`, and a mktemp fixture is not a git worktree
+# root, so run-bats.sh announces that it could not derive an expectation and
+# falls back to the unconditional zero-file check. Cases 8-8c drive the derived
+# path deliberately by `git init`-ing their fixtures.
+#
+# TMPDIR is a private per-run directory so a stub can find the scratch dir
+# run-bats.sh mktemp'd for itself -- see case 7.
 FAKE_OUT=""
 FAKE_RC=0
-FAKE_FLOOR=""
 run_fake() {
   local priv
   priv="$(mktemp -d)"
   FIXTURES+=("$priv")
   FAKE_RC=0
-  FAKE_OUT="$(TMPDIR="$priv" BATS_FILE_FLOOR="${FAKE_FLOOR:-1}" bash "$1/tests/run-bats.sh" 2>&1)" || FAKE_RC=$?
+  FAKE_OUT="$(TMPDIR="$priv" bash "$1/tests/run-bats.sh" 2>&1)" || FAKE_RC=$?
+}
+
+# A fixture whose root IS a git worktree root, so run-bats.sh derives its
+# expected set from the index instead of degrading. Only `git add` is used --
+# `git ls-files` reads the index, so nothing needs committing and no user
+# identity is required.
+make_git_fake_repo() {
+  local dir
+  dir="$(make_fake_repo)"
+  git -C "$dir" init -q
+  echo "$dir"
 }
 
 # --- 1. A stub emitting nothing at all is a broken harness, not a clean run ---
@@ -277,13 +290,15 @@ else
   pass "an empty status file fails the run despite a healthy TAP stream"
 fi
 
-# --- 8. Too few discovered .bats files is a hard error, not a green run. Zero
-# files used to print a note to stderr and `exit 0`; the floor makes both zero
-# and a collapsed-but-nonzero count fail, because a moved tests/ tree or a
-# widened `-not -path` exclusion produces exactly that and nothing else notices.
+# --- 8. A tracked .bats file the walk did not discover is a hard error. This
+# replaces a `BATS_FILE_FLOOR=8` guess against a real count of 10 -- two files of
+# slack, which is not hypothetical: deleting two real .bats files left the suite
+# reporting "155 tests, 0 failures" and exiting 0 with 11 tests silently gone.
+# The expectation is now derived from `git ls-files`, so it is exact and needs no
+# magic number.
 echo ""
-echo "--- a discovered-file count under the floor fails the run ---"
-DIR8="$(make_fake_repo)"
+echo "--- a tracked .bats file missing from the walk fails the run and names it ---"
+DIR8="$(make_git_fake_repo)"
 FIXTURES+=("$DIR8")
 seed_bats_files "$DIR8"
 install_stub_bats "$DIR8" <<'EOF'
@@ -292,20 +307,92 @@ echo "1..1"
 echo "ok 1 first"
 exit 0
 EOF
-FAKE_FLOOR=8
+git -C "$DIR8" add tests/a.bats tests/b.bats
+rm "$DIR8/tests/b.bats"
 run_fake "$DIR8"
-FAKE_FLOOR=""
 if [[ $FAKE_RC -eq 0 ]]; then
-  fail "2 .bats files under a floor of 8 passed — a collapsed search path reads as green"
-elif echo "$FAKE_OUT" | grep -q "below the floor of 8"; then
-  pass "a file count under the floor fails and names the floor it missed"
+  fail "a tracked .bats file gone from the worktree passed — a deleted suite reads as green"
+elif ! echo "$FAKE_OUT" | grep -q "tracked .bats file(s) were not discovered"; then
+  fail "the run failed but not with the undiscovered-tracked-file message: $FAKE_OUT"
+elif echo "$FAKE_OUT" | grep -q "^  tests/b.bats$"; then
+  pass "a tracked .bats file missing from the walk fails the run and names the file"
 else
-  fail "the run failed but not with the floor message: $FAKE_OUT"
+  fail "the run failed without naming the missing file: $FAKE_OUT"
 fi
 
-# --- 9. Zero discovered .bats files is the same hard error. Kept separate from
-# case 8 because it is the state the old `exit 0` branch handled by name, and it
-# is the one a path change actually produces.
+# --- 8b. An UNTRACKED .bats file is not a finding. The derived expectation runs
+# one way only: every tracked file must have been discovered, but a discovered
+# file need not be tracked. Without this the check would fail on ordinary
+# not-yet-committed work, which is how a correct guard gets disabled.
+echo ""
+echo "--- an untracked new .bats file does not fail the run ---"
+DIR8B="$(make_git_fake_repo)"
+FIXTURES+=("$DIR8B")
+seed_bats_files "$DIR8B"
+install_stub_bats "$DIR8B" <<'EOF'
+#!/usr/bin/env bash
+echo "1..1"
+echo "ok 1 first"
+exit 0
+EOF
+git -C "$DIR8B" add tests/a.bats
+run_fake "$DIR8B"
+if [[ $FAKE_RC -ne 0 ]]; then
+  fail "an untracked .bats file was reported as a finding: $FAKE_OUT"
+elif echo "$FAKE_OUT" | grep -q "^2 tests, 0 failures$"; then
+  pass "an untracked .bats file is run without being demanded of the index"
+else
+  fail "the untracked-file run passed with the wrong count: $FAKE_OUT"
+fi
+
+# --- 8c. A newly added .bats file joins the expectation immediately. This is the
+# half a floor can never have: adding files only ever widens a floor's slack,
+# while `git add` alone makes the new file required from the next run on, with no
+# edit to this script and no number to bump.
+echo ""
+echo "--- a newly git-added .bats file is required from the next run on ---"
+git -C "$DIR8B" add tests/b.bats
+rm "$DIR8B/tests/b.bats"
+run_fake "$DIR8B"
+if [[ $FAKE_RC -eq 0 ]]; then
+  fail "the .bats file added to the index a moment ago was not demanded back: $FAKE_OUT"
+elif echo "$FAKE_OUT" | grep -q "^  tests/b.bats$"; then
+  pass "a file added to the index joins the expected set with no floor to bump"
+else
+  fail "the run failed but did not name the newly tracked file: $FAKE_OUT"
+fi
+
+# --- 8d. Outside a git worktree the run still works, and says the expectation
+# could not be derived. That degradation is what every other fixture here relies
+# on, and it must be announced rather than silent -- an unannounced fallback is
+# how a derived check quietly becomes no check at all on a tarball export.
+echo ""
+echo "--- a non-git tree runs, and announces that no expectation could be derived ---"
+DIR8D="$(make_fake_repo)"
+FIXTURES+=("$DIR8D")
+seed_bats_files "$DIR8D"
+install_stub_bats "$DIR8D" <<'EOF'
+#!/usr/bin/env bash
+echo "1..1"
+echo "ok 1 first"
+exit 0
+EOF
+run_fake "$DIR8D"
+if [[ $FAKE_RC -ne 0 ]]; then
+  fail "a non-git tree failed the run: $FAKE_OUT"
+elif ! echo "$FAKE_OUT" | grep -q "not a git worktree root"; then
+  fail "a non-git tree silently skipped the derived expectation with no note: $FAKE_OUT"
+elif echo "$FAKE_OUT" | grep -q "^2 tests, 0 failures$"; then
+  pass "a non-git tree runs the suite and says the expected set could not be derived"
+else
+  fail "the non-git run passed with the wrong count: $FAKE_OUT"
+fi
+
+# --- 9. Zero discovered .bats files is a hard error regardless, and is checked
+# unconditionally rather than through the derived set: a tree with nothing
+# tracked at all (a tarball export, a fresh scaffold) must still not run on an
+# empty set and call it green. It is the state the old `exit 0` branch handled by
+# name, and the one a path change actually produces.
 echo ""
 echo "--- zero discovered .bats files fails the run ---"
 DIR9="$(make_fake_repo)"