Three gaps left by the previous round, all the same shape: a gate reporting
success having verified less than it appears to.
run_bats() hard-failed on a missing or non-executable runner but never checked
that the runner produced anything. An empty, executable run-bats.sh exits 0,
and the dispatcher printed a green summary with 166 bats tests silently absent.
It now requires an "N tests, M failures" line with a non-zero count.
run-tests.sh's skip listing is swallowed by pre-commit on a pass, so on a
machine without vale three suites exited 77 and the pre-push gate went green
having run 14 of 17. The hook now invokes it as --strict, where a skip fails
and the error names each suite and the reason it skipped. An ad-hoc local run
still skips gracefully -- at pre-push a skip means a documented dependency is
missing, which is a setup error, not a legitimate state. Deliberately not wired
to the vale downgrade's env var: one flag must not disarm two gates.
BATS_FILE_FLOOR is replaced by an expectation derived from git ls-files. A
floor of 8 against a real count of 10 let two files and eleven tests disappear
green, and the number needed an edit whenever a plugin was added. The derived
set needs no number, and catches an addition as well as a removal -- a .bats
file staged into the index and deleted from disk is now demanded back.
The vale opt-out announced its downgrade to nobody: pre-commit prints nothing
for a passing hook, so the summary line AGENTS.md tells the reader to check was
unreachable in exactly the situation it exists for. The hook is now verbose.
Also corrects the PROBES_CHECKED guard, whose commit message described a state
that cannot occur -- the .vale.ini loop errs first. Its two reachable triggers,
a gutted probe heredoc and a probe row naming a missing directory, had no test;
they do now, each asserting the guard is the sole cause.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
run_bats() was an if with no else, so a missing or non-executable run-bats.sh
made all 166 bats tests disappear with no diagnostic while the run printed a
green summary and exited 0. --bats-only became a total no-op. This is the same
defect the PR fixed one level down, left open in the dispatcher that pre-push
actually invokes -- and nothing tested run-tests.sh at all.
run-bats.sh's aggregation was asserted by nothing. Three separate mutations to
its failure-detection line all survived the existing suite, because real bats
emits both a nonzero exit and "not ok" lines, so each signal masked the other.
The new cases produce each signal without the other; all three mutants now die.
Also in this pass:
- zero discovered .bats files exited 0, so a widened path exclusion retired the
suite silently. Replaced with a file-count floor: a collapse to one or two
files is the same failure as a collapse to zero
- an existing-but-empty status file counted as a pass, because [[ "" -eq 0 ]]
is arithmetic-true and the || echo 1 fallback only covered a missing file.
The repro is deterministic: the stub truncates its own status file, then
kill -9s its parent so the real exit-code write never happens
Adds tests/test-run-tests.sh, 9 cases pinning the exit 0 / 1 / 77 three-way
split against a single summary line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
`tests/run-tests.sh` declared `source=lib/batch-run.sh`, which resolves
to neither the repo root nor the script's own directory. A directive
that does not resolve is silent: it blinds test-vale-wrap.sh's
`sourced_files()` seeding exemption, and shellcheck's own SC1091 is
`info` while .pre-commit-config.yaml pins `--severity=warning`.
Issue #97 names run-bats.sh's `../scripts/lib/batch-run.sh` as the
correct spelling. It is not. Directives resolve against the source-path,
which under pre-commit is the repo root, so `../scripts/...` escapes the
repo and trips SC1091 exactly as `lib/...` does -- verified directly.
The spelling satisfying both shellcheck and `sourced_files()`'s
two-candidate rule is repo-root-relative, matching scripts/install.sh.
Fixes all three: run-tests.sh, run-bats.sh, and check-manifests.sh,
the last unmentioned by the issue. Every directive in the repo now
resolves, which the previous commit's case 27 asserts.
Refs #97
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
Four repo gates reported success in states they exist to reject.
`check-vale-style-sync.sh` passed while a Kyberforge lint rule was silenced. The
check matched a blocklist of severity values, but Vale's semantic is an allowlist:
anything that is not exactly YES/error/warning/suggestion disables the rule. So
`= false`, `= 0`, `= garbage`, an empty value and — worst — a lowercase `= yes` all
killed enforcement while reading as "enabled" to a human. Inverted to an allowlist.
Two sibling holes: dropping `KyberforgeCopilot` from `BasedOnStyles` unloaded the
Copilot-only check silently, and narrowing a section glob to a location made Vale
lint zero files, which is the "0 files, hook Passed" failure the script's own
comment says it exists to catch.
`sync-marketplace-mirror.sh --check` failed open when its source was missing, while
its sibling correctly errored in the same state.
`check-scope-walkup-sync.sh` wrote to hardcoded `/tmp/fN.out` paths and read one
back, making it non-reentrant — a concurrent instance can flip a verdict, and this
branch made the test runner concurrent. Now per-run `mktemp -d`.
`check-manifests.sh` had no disk-to-marketplace pass, so a plugin directory absent
from `marketplace.json` passed every gate while the `validate-plugins` hook globbed
it. The "listed" match is restricted to remote-source entry names; matching any
entry name let a genuine orphan through on a name coincidence.
`run-bats.sh` reported an empty TAP stream as `0 tests, 0 failures`, exit 0 — a
total harness failure reading as a pass.
The test-side changes are the larger half, because the guards were the real problem.
`test-sync-marketplace-mirror.sh` could overwrite the live tracked mirror under an
inherited GIT_DIR, which is precisely the git-hook context it runs in. The bash-3.2
scan hand-maintained its file list, omitting the new shared runner, and had no rule
for `wait -n` or `nproc` — the two hazards the previous review round found live. It
now derives 43 files across three globs with per-glob floors. Several assertions
were decoration: the concurrency checks caught the reentrancy defect 0 times in 10,
the leak fix was green either way, and two manifest fixtures passed with the code
they claimed to cover deleted. Every assertion now has a revert it provably fails
against.
Refs: #90
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
Review findings #5 and #7 on PR #95 flagged two bash-3.2-incompatible
patterns despite the surrounding scripts claiming 3.2 safety:
- tests/run-bats.sh used `mapfile` (bash 4.0+), which fails immediately
under macOS's stock bash 3.2 before any batching logic runs. Replaced
with the `while read` loop already established in tests/run-tests.sh,
and guarded the two downstream `${TEST_FILES[@]}` expansions with
`${arr[@]+"${arr[@]}"}` to match that file's convention.
- `trap 'rm -rf "${CLEANUP_DIRS[@]}"' EXIT` was unguarded in
tests/test-sync-marketplace-mirror.sh and
tests/test-sync-plugin-content.sh: under `set -u`, if `mktemp -d`
fails before the array is populated, the trap itself throws an
unbound-variable error that masks the real test failure. A repo-wide
grep for the same pattern turned up a third, unreviewed instance in
tests/test-check-release-needed.sh. Fixed all three with the guarded
idiom already used elsewhere in the repo.
Extended the existing bash-3.2-hazard static check (test 16 in
tests/test-vale-wrap.sh) to scan all four fixed files going forward,
so a regression of either pattern fails the suite instead of only
surfacing on a real bash 3.2 host.
Refs: PR #95
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
--check's throwaway pack copy seeded .claude-plugin/plugin.json and
.github/plugin/plugin.json from the real plugin dir, then packed without
--force -- apm pack silently skips regenerating a plugin.json that already
exists, so the diff always compared the copy against itself and never caught
drift in the compiled name/version/description/mcpServers. --force is now
always passed; in check mode it forces regeneration inside the throwaway copy
only, which sync_plugin_manifest() then diffs against the real committed
manifest.
sync_hooks_json() returned early whenever .apm/hooks/ was missing, without
checking whether a stale hooks.json was still sitting at the plugin root from
a prior sync -- unlike sync_dir(), which already detects that kind of orphaned
mirrored output. It now mirrors sync_dir()'s shape: flagged as drift in
--check, removed on a real sync.
Running the corrected --check --all against this repo's own plugins surfaced
3 real orphans: plugins/{git,gitea,core}/hooks.json, empty stubs added in
4edaaac only to satisfy an old plugin.json pointer-field check that no longer
exists (their compiled plugin.json has never had a hooks field, and none of
the three ever had .apm/hooks/). Removed as part of this fix since they're
exactly the drift the corrected check now catches -- leaving them would break
the sync-plugin-content pre-push gate on this branch.
Also extracts two shared helpers into scripts/lib/, sourced by this script and
others so a future bug fix doesn't need hand-applying three times:
- marketplace-plugins.sh: walks marketplace.json for local plugin dirs (this
script's --all branch and check-manifests.sh had near-identical copies)
- batch-run.sh: the bounded-batch concurrent job runner (this script,
tests/run-tests.sh, and tests/run-bats.sh each hand-rolled the same
core-count-capped wait loop independently)
Extended tests/test-sync-plugin-content.sh with coverage for both drift cases
(plugin.json version-bump drift, orphaned-hooks.json drift), including that a
re-sync clears each.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
run-bats.sh's new bounded parallel dispatcher used nproc/wait -n,
which are bash 4.3+/GNU-only and silently drop the concurrency cap on
macOS's stock bash 3.2 (the wait -n error is swallowed by `|| true`).
Its sibling tests/run-tests.sh, changed in the same PR and explicitly
bash-3.2-safe, already solves this with getconf + a batched wait.
Ported that same pattern here for consistency and to actually meet the
compatibility goal.
Refs: #95
Both runners spawned one process at a time and let it finish before starting
the next, so wall time scaled with file count even though each file's own
work (bats: mostly repeated apm/git subprocess startup; the vale-heavy
shell scripts: repeated vale binary startup) rarely used a full core.
run-bats.sh now backgrounds one `bats` invocation per file, bounded by core
count, buffering each file's TAP output so concurrent streams can't
interleave, then flushes in stable sorted order once every job finishes.
run-tests.sh does the same for test-*.sh in fixed-size batches (plain `wait`,
not `wait -n`, to stay on the bash-3.2-safe path test-vale-wrap.sh already
enforces for this file) -- verified beforehand that every test-*.sh keeps its
fixtures under its own mktemp dir rather than mutating the live repo tree, so
running them concurrently is safe.
Confirmed correct on both the pass and fail paths (a deliberately failing
bats test and a scratch TEST_DIR with pass/fail/skip-77 scripts) before
measuring: full bats suite 26-30s -> 7.5s, tests/run-tests.sh 68s -> ~45-50s,
full pre-push gate 2m40s -> 1m12s.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
