fix(scripts): close gates that passed while the thing they guard was disabled

Four repo gates reported success in states they exist to reject.

`check-vale-style-sync.sh` passed while a Kyberforge lint rule was silenced. The
check matched a blocklist of severity values, but Vale's semantic is an allowlist:
anything that is not exactly YES/error/warning/suggestion disables the rule. So
`= false`, `= 0`, `= garbage`, an empty value and — worst — a lowercase `= yes` all
killed enforcement while reading as "enabled" to a human. Inverted to an allowlist.
Two sibling holes: dropping `KyberforgeCopilot` from `BasedOnStyles` unloaded the
Copilot-only check silently, and narrowing a section glob to a location made Vale
lint zero files, which is the "0 files, hook Passed" failure the script's own
comment says it exists to catch.

`sync-marketplace-mirror.sh --check` failed open when its source was missing, while
its sibling correctly errored in the same state.

`check-scope-walkup-sync.sh` wrote to hardcoded `/tmp/fN.out` paths and read one
back, making it non-reentrant — a concurrent instance can flip a verdict, and this
branch made the test runner concurrent. Now per-run `mktemp -d`.

`check-manifests.sh` had no disk-to-marketplace pass, so a plugin directory absent
from `marketplace.json` passed every gate while the `validate-plugins` hook globbed
it. The "listed" match is restricted to remote-source entry names; matching any
entry name let a genuine orphan through on a name coincidence.

`run-bats.sh` reported an empty TAP stream as `0 tests, 0 failures`, exit 0 — a
total harness failure reading as a pass.

The test-side changes are the larger half, because the guards were the real problem.
`test-sync-marketplace-mirror.sh` could overwrite the live tracked mirror under an
inherited GIT_DIR, which is precisely the git-hook context it runs in. The bash-3.2
scan hand-maintained its file list, omitting the new shared runner, and had no rule
for `wait -n` or `nproc` — the two hazards the previous review round found live. It
now derives 43 files across three globs with per-glob floors. Several assertions
were decoration: the concurrency checks caught the reentrancy defect 0 times in 10,
the leak fix was green either way, and two manifest fixtures passed with the code
they claimed to cover deleted. Every assertion now has a revert it provably fails
against.

Refs: #90

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
This commit is contained in:
2026-08-14 01:52:56 +00:00
parent d4fa4b7153
commit 413a750819
11 changed files with 1198 additions and 90 deletions

View File

@@ -71,6 +71,13 @@ for ini in "$SKILL_INI" "$AGENT_INI"; do
err "$rel_ini is missing — without it vale falls back to an upward config search and lints with whatever it finds"
continue
fi
# Present but unreadable is its own case: every assertion below is a grep, and
# grep exits 2 on a read error. The override capture swallows that into an
# empty result, which would read as "no findings" rather than "not checked".
if [[ ! -r "$ini" ]]; then
err "$rel_ini is not readable — none of its assertions could run, and an unreadable file cannot be distinguished from a clean one downstream"
continue
fi
# StylesPath is resolved relative to the .vale.ini, which is the only reason
# the bundled styles are found from a consuming repo's clone prefix.
if ! grep -Eq '^[[:space:]]*StylesPath[[:space:]]*=[[:space:]]*styles[[:space:]]*$' "$ini"; then
@@ -81,8 +88,71 @@ for ini in "$SKILL_INI" "$AGENT_INI"; do
if ! grep -Eq '^[[:space:]]*BasedOnStyles[[:space:]]*=.*Kyberforge([[:space:],]|$)' "$ini"; then
err "$rel_ini has no section whose BasedOnStyles names Kyberforge — every rule the audit prefilters on lives in that style"
fi
# Per-rule overrides are the third way to retire a rule without touching a
# style file or a glob. CONTEXT.md's "Vale audit prefilter" entry: "Every rule
# is `level: error` and every alert is a FAIL — no ignorable tier". Vale's exit
# code keys on `error` alerts alone, so any override that leaves a rule at
# anything other than `error` still lints the file, still exits 0, and still
# shows `Passed` in pre-commit. The glob probe below cannot backstop this: it
# keys on one `Kyberforge.VagueWording` alert, so DescriptionOpener,
# PaddingPhrase, SentenceOpenerThereIs and ProactivePhrase can each be retired
# underneath a passing probe.
#
# Asserted as an ALLOWLIST, not a blocklist of `NO|warning|suggestion`, because
# that is vale 3.15.2's own semantic: only the exact tokens `YES` and `error`
# keep a rule blocking. `warning`/`suggestion` downgrade it (alert still
# printed, exit 0 — invisible, since pre-commit swallows a passing hook's
# output); every other value — `NO`, `false`, `0`, `off`, `n`, empty,
# `garbage`, and lowercase `yes`, `true`, `1`, `on` — silences the rule
# outright. Lowercase `yes` is the trap a blocklist cannot cover: it reads as
# "enabled" to a human and disables the rule. Verified by enumerating the
# value space against vale 3.15.2.
#
# The allowlist demands a BARE `YES`/`error` with nothing after it, which also
# rejects `error # note` and `error ; note`. Vale itself strips those — a
# whitespace-preceded `#` or `;` comment is removed and the rule stays live —
# so rejecting them is deliberately stricter than vale, not a workaround for
# it. Uniformity is worth more here than the ability to annotate a line that
# should not exist: no shipped `.vale.ini` has any override line at all, and
# the failure mode is a loud false positive rather than a silent pass. The
# genuine hazard is the no-space form — `error# note` and `error; note` are
# NOT stripped and silence the rule outright — and a rule that demands a bare
# token catches those without having to reimplement vale's comment parsing.
#
# `[A-Za-z0-9_-]` on both halves of the name, not `[A-Za-z]`: a rule named
# `Kyberforge.Vague2` is genuinely silenced by `= NO` (verified: 1 error ->
# 0 errors), so an alpha-only class would let a digit-bearing rule name slip
# past the gate. All five current rule names are pure alpha, so this is
# forward cover, not a live hole.
bad_overrides="$(
grep -E '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=' "$ini" \
| grep -Ev '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=[[:space:]]*(YES|error)[[:space:]]*$' \
|| true
)"
# No `grep -q` in that pipeline on purpose: `-q` exits on its first match, and
# under `set -o pipefail` the resulting SIGPIPE on the upstream grep would make
# the whole pipeline report 141 and read as "no findings".
if [[ -n "$bad_overrides" ]]; then
err "$rel_ini overrides a Kyberforge rule to something other than a bare YES or error (first: '${bad_overrides%%$'\n'*}') — every rule in this prefilter is level: error and every alert is a FAIL, and any other value downgrades or silences the rule while vale still exits 0. A trailing comment is rejected too: vale strips a spaced '# ...' but not 'error# ...', so this asks for the bare token rather than guessing which form you meant"
fi
done
# KyberforgeCopilot is agent-audit's alone — CONTEXT.md describes it as "scoped
# only to `.agent.md` files for the Copilot-only 'Use proactively has no effect'
# check". The loop above deliberately asserts only `Kyberforge`, since
# skill-audit's copy legitimately has no Copilot style, so dropping
# `, KyberforgeCopilot` from agent-audit's `[**/*.agent.md]` section unloaded the
# whole style silently: no glob broke, the styles/ diff above stayed clean (the
# style directory is still shipped, just never loaded), the two .vale.ini files
# are deliberately unequal so no equality check applies, and the probe below
# still passed because it keys on a Kyberforge alert. Assert the style is loaded
# whenever it is shipped.
if [[ -d "$AGENT_AUDIT/assets/vale/styles/KyberforgeCopilot" && -f "$AGENT_INI" ]]; then
if ! grep -Eq '^[[:space:]]*BasedOnStyles[[:space:]]*=.*KyberforgeCopilot([[:space:],]|$)' "$AGENT_INI"; then
err "${AGENT_INI#"$REPO_ROOT"/} ships a styles/KyberforgeCopilot style but no section's BasedOnStyles names it — the style is never loaded, so its Copilot-only rules lint nothing"
fi
fi
# Prints the `files:` regex of every hook, in ONE manifest ($2), whose entry
# is $1's vale-wrap.sh. Records are delimited by their `- id:` line, so the
# check does not depend on `entry:` preceding `files:` within a record.
@@ -227,10 +297,24 @@ while IFS='|' read -r skill rel scope; do
# shape — per ADR-0016 every `.apm/agents/*` file is named `*.agent.md`, so
# `.pre-commit-config.yaml`'s regex correctly no longer matches it and that's
# not drift. `demo.agent.md` is the real, current shape and is `shared`.
#
# The two `.claude/`-prefixed probes carry the location-independence CONTEXT.md
# asserts: "A `SKILL.md` outside `plugins/` (e.g. project-scope
# `.claude/skills/foo/SKILL.md`) still matches `[**/SKILL.md]` and gets linted
# normally — the globs constrain filename shape, not location." Every other
# probe here starts with `plugins/`, so narrowing a glob to a `plugins/`-shaped
# path (`[**/SKILL.md]` -> `[**/.apm/skills/*/SKILL.md]`) left all of them
# matching while the project-scope shape started linting as `0 errors ... in 0
# files` — the exact "vale lints zero files, hook shows Passed" failure the
# comment at the top of this section describes. Both are `hooks-only`: only
# `.pre-commit-hooks.yaml` is layout-agnostic, and `.pre-commit-config.yaml`
# pinning this repo's own `plugins/**/.apm/` layout is by design, not drift.
done <<'EOF_PROBE'
skill-audit|plugins/demo/.apm/skills/demo/SKILL.md|shared
skill-audit|.claude/skills/demo/SKILL.md|hooks-only
agent-audit|plugins/demo/.apm/agents/demo.md|hooks-only
agent-audit|plugins/demo/.apm/agents/demo.agent.md|shared
agent-audit|.claude/agents/demo.md|hooks-only
agent-audit|copilot/demo.agent.md|hooks-only
EOF_PROBE