fix(scripts): close gates that passed while the thing they guard was disabled
Four repo gates reported success in states they exist to reject. `check-vale-style-sync.sh` passed while a Kyberforge lint rule was silenced. The check matched a blocklist of severity values, but Vale's semantic is an allowlist: anything that is not exactly YES/error/warning/suggestion disables the rule. So `= false`, `= 0`, `= garbage`, an empty value and — worst — a lowercase `= yes` all killed enforcement while reading as "enabled" to a human. Inverted to an allowlist. Two sibling holes: dropping `KyberforgeCopilot` from `BasedOnStyles` unloaded the Copilot-only check silently, and narrowing a section glob to a location made Vale lint zero files, which is the "0 files, hook Passed" failure the script's own comment says it exists to catch. `sync-marketplace-mirror.sh --check` failed open when its source was missing, while its sibling correctly errored in the same state. `check-scope-walkup-sync.sh` wrote to hardcoded `/tmp/fN.out` paths and read one back, making it non-reentrant — a concurrent instance can flip a verdict, and this branch made the test runner concurrent. Now per-run `mktemp -d`. `check-manifests.sh` had no disk-to-marketplace pass, so a plugin directory absent from `marketplace.json` passed every gate while the `validate-plugins` hook globbed it. The "listed" match is restricted to remote-source entry names; matching any entry name let a genuine orphan through on a name coincidence. `run-bats.sh` reported an empty TAP stream as `0 tests, 0 failures`, exit 0 — a total harness failure reading as a pass. The test-side changes are the larger half, because the guards were the real problem. `test-sync-marketplace-mirror.sh` could overwrite the live tracked mirror under an inherited GIT_DIR, which is precisely the git-hook context it runs in. The bash-3.2 scan hand-maintained its file list, omitting the new shared runner, and had no rule for `wait -n` or `nproc` — the two hazards the previous review round found live. It now derives 43 files across three globs with per-glob floors. Several assertions were decoration: the concurrency checks caught the reentrancy defect 0 times in 10, the leak fix was green either way, and two manifest fixtures passed with the code they claimed to cover deleted. Every assertion now has a revert it provably fails against. Refs: #90 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
This commit is contained in:
@@ -209,6 +209,122 @@ else
|
||||
pass "exits non-zero when a .vale.ini no longer names the Kyberforge style"
|
||||
fi
|
||||
|
||||
# --- 9b. Exits 1 when a per-rule override leaves a rule at anything but error ---
|
||||
# The third way to switch a rule off without touching a style file or a glob.
|
||||
# CONTEXT.md's "Vale audit prefilter" entry: "Every rule is `level: error` and
|
||||
# every alert is a FAIL -- no ignorable tier". Vale's exit code keys on `error`
|
||||
# alerts alone, so any such override leaves the glob intact, the styles
|
||||
# byte-identical, and the run at `0 errors`, exit 0, `Passed`.
|
||||
#
|
||||
# Asserted as an ALLOWLIST because that is vale 3.15.2's own semantic, verified
|
||||
# by enumerating the value space: only the exact tokens `YES` and `error` keep a
|
||||
# rule blocking. `warning`/`suggestion` downgrade it (alert printed, exit 0 --
|
||||
# invisible, since pre-commit swallows a passing hook's output); EVERY other
|
||||
# value silences it outright, including `false`, `0`, `off`, an empty value,
|
||||
# `garbage`, and lowercase `yes`. That last one is why a blocklist of
|
||||
# `NO|warning|suggestion` was not enough: `= yes` reads as "enabled" to a human
|
||||
# and disables the rule. Case 10's glob probe backstops none of this -- it keys
|
||||
# on one Kyberforge.VagueWording alert, so DescriptionOpener, PaddingPhrase,
|
||||
# SentenceOpenerThereIs and ProactivePhrase can each be retired underneath it,
|
||||
# which is why the cases below deliberately target rules that probe never sees.
|
||||
#
|
||||
# Two cases below are about comment forms, and they are NOT symmetric in vale:
|
||||
# `error # note` (spaced) is stripped by vale and stays live, while `error# note`
|
||||
# (no space) is not stripped and silences the rule. The gate demands a bare
|
||||
# token, so it flags both -- deliberately stricter than vale for the spaced form,
|
||||
# and the only way to catch the no-space form without reimplementing vale's
|
||||
# comment parsing. `Kyberforge.Vague2` covers rule names carrying a digit: such a
|
||||
# rule is genuinely silenced by `= NO`, and an alpha-only name class in the gate
|
||||
# would not even see the line.
|
||||
echo ""
|
||||
echo "--- exits 1 when a .vale.ini overrides a Kyberforge rule to anything but YES/error ---"
|
||||
while IFS= read -r override; do
|
||||
[[ -n "$override" ]] || continue
|
||||
# `<EMPTY>` stands in for a bare `Rule =` with no value at all, which the
|
||||
# heredoc cannot carry as a trailing space without a linter eating it.
|
||||
override="${override/<EMPTY>/}"
|
||||
FIXTURE_OV="$(make_fixture)"
|
||||
FIXTURES+=("$FIXTURE_OV")
|
||||
echo "$override" >> "$FIXTURE_OV/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
|
||||
if bash "$SCRIPT" "$FIXTURE_OV" > /dev/null 2>&1; then
|
||||
fail "exited 0 with '$override' in skill-audit's .vale.ini -- expected exit 1"
|
||||
else
|
||||
pass "exits non-zero on '$override'"
|
||||
fi
|
||||
done <<'EOF_OVERRIDES'
|
||||
Kyberforge.SentenceOpenerThereIs = NO
|
||||
Kyberforge.VagueWording = warning
|
||||
Kyberforge.SentenceOpenerThereIs = suggestion
|
||||
Kyberforge.SentenceOpenerThereIs = false
|
||||
Kyberforge.DescriptionOpener = 0
|
||||
Kyberforge.PaddingPhrase = off
|
||||
Kyberforge.SentenceOpenerThereIs = yes
|
||||
Kyberforge.DescriptionOpener = garbage
|
||||
Kyberforge.PaddingPhrase =<EMPTY>
|
||||
Kyberforge.SentenceOpenerThereIs = NO # keep quiet
|
||||
Kyberforge.DescriptionOpener = error# silenced, vale strips no comment without a space
|
||||
Kyberforge.PaddingPhrase = error; silenced too, same no-space rule for ';'
|
||||
Kyberforge.DescriptionOpener = error # stripped by vale, still rejected: bare token required
|
||||
Kyberforge.Vague2 = NO
|
||||
Kyberforge.Vague_2 = NO
|
||||
Kyberforge.Vague-2 = NO
|
||||
EOF_OVERRIDES
|
||||
# Same in agent-audit's copy: the check runs over both .vale.ini files, and a
|
||||
# rule retired in only the canonical copy is the likelier direction. `= false`
|
||||
# on ProactivePhrase is the sharpest shape -- one word off the original defect,
|
||||
# on a KyberforgeCopilot rule no glob probe covers.
|
||||
FIXTURE_OV_AGENT="$(make_fixture)"
|
||||
FIXTURES+=("$FIXTURE_OV_AGENT")
|
||||
echo "KyberforgeCopilot.ProactivePhrase = false" \
|
||||
>> "$FIXTURE_OV_AGENT/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini"
|
||||
if bash "$SCRIPT" "$FIXTURE_OV_AGENT" > /dev/null 2>&1; then
|
||||
fail "exited 0 with 'KyberforgeCopilot.ProactivePhrase = false' in agent-audit's .vale.ini -- expected exit 1"
|
||||
else
|
||||
pass "exits non-zero when agent-audit's copy retires a KyberforgeCopilot rule"
|
||||
fi
|
||||
# The two allowlisted values must NOT trip the assertion -- otherwise it would
|
||||
# fire on any legitimate explicit enablement. Kept as a positive case so an
|
||||
# over-broad tightening of the regex shows up here rather than in the repo.
|
||||
FIXTURE_OV_OK="$(make_fixture)"
|
||||
FIXTURES+=("$FIXTURE_OV_OK")
|
||||
{
|
||||
echo "Kyberforge.SentenceOpenerThereIs = YES"
|
||||
echo "Kyberforge.VagueWording = error"
|
||||
} >> "$FIXTURE_OV_OK/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
|
||||
if bash "$SCRIPT" "$FIXTURE_OV_OK" > /dev/null 2>&1; then
|
||||
pass "an explicit '= YES' / '= error' override is not flagged"
|
||||
else
|
||||
fail "flagged an explicit '= YES' / '= error' override -- those are the two values that keep a rule blocking"
|
||||
bash "$SCRIPT" "$FIXTURE_OV_OK" 2>&1 | sed 's/^/ /' || true
|
||||
fi
|
||||
|
||||
# --- 9c. Exits 1 when agent-audit ships KyberforgeCopilot but never loads it ---
|
||||
# Case 9 asserts only that Kyberforge is named, because skill-audit's copy
|
||||
# legitimately has no Copilot style. So dropping just `, KyberforgeCopilot` from
|
||||
# agent-audit's [**/*.agent.md] section unloaded the whole style silently: no
|
||||
# glob broke, the styles/ diff stayed clean (the directory is still shipped,
|
||||
# only never loaded), the two .vale.ini files are deliberately unequal so no
|
||||
# equality check applies, and case 10's probe still passed because it keys on a
|
||||
# Kyberforge alert. Verified dead by probing a `.agent.md` carrying
|
||||
# "Use proactively": 0 alerts under the broken config, KyberforgeCopilot.
|
||||
# ProactivePhrase under the shipped one. CONTEXT.md describes the style as
|
||||
# "scoped only to `.agent.md` files for the Copilot-only 'Use proactively has
|
||||
# no effect' check", so shipping it unloaded is drift.
|
||||
echo ""
|
||||
echo "--- exits 1 when the shipped KyberforgeCopilot style is named by no BasedOnStyles ---"
|
||||
FIXTURE11C="$(make_fixture)"
|
||||
FIXTURES+=("$FIXTURE11C")
|
||||
break_glob "$FIXTURE11C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
|
||||
'BasedOnStyles = Kyberforge, KyberforgeCopilot' 'BasedOnStyles = Kyberforge'
|
||||
if bash "$SCRIPT" "$FIXTURE11C" > /dev/null 2>&1; then
|
||||
fail "exited 0 when KyberforgeCopilot was dropped from BasedOnStyles -- expected exit 1"
|
||||
else
|
||||
pass "exits non-zero when a shipped KyberforgeCopilot style is never loaded"
|
||||
fi
|
||||
# The assertion is conditional on the style being shipped: a copy with no
|
||||
# KyberforgeCopilot directory (skill-audit's, by design) must stay clean --
|
||||
# case 13 below covers the shipped-and-loaded pairing.
|
||||
|
||||
# --- 10. Exits 1 when a glob section stops matching the shape its hook lints ---
|
||||
# One case per glob section, because each covers a file shape the others don't:
|
||||
# agent-audit's [**/*.agent.md] is the only section covering a Copilot agent file
|
||||
@@ -241,6 +357,37 @@ else
|
||||
pass "exits non-zero when agent-audit's *.agent.md glob matches nothing"
|
||||
fi
|
||||
|
||||
# --- 10b. Exits 1 when a glob is narrowed to this repo's own plugins/ layout ---
|
||||
# Every probe path used to start with `plugins/`, so a glob narrowed from a
|
||||
# filename shape to a location (`[**/SKILL.md]` -> `[**/.apm/skills/*/SKILL.md]`)
|
||||
# still matched all of them and the check passed -- while a project-scope
|
||||
# `.claude/skills/foo/SKILL.md` started linting as `0 errors ... in 0 files`,
|
||||
# exit 0, hook `Passed`: the exact failure the script's own header comment says
|
||||
# it exists to catch. CONTEXT.md: "A `SKILL.md` outside `plugins/` (e.g.
|
||||
# project-scope `.claude/skills/foo/SKILL.md`) still matches `[**/SKILL.md]` and
|
||||
# gets linted normally -- the globs constrain filename shape, not location."
|
||||
# These narrowings are still valid glob syntax and break no `plugins/`-shaped
|
||||
# file, so only a non-`plugins/` probe path catches them.
|
||||
echo ""
|
||||
echo "--- exits 1 when a .vale.ini glob is narrowed from a filename shape to a location ---"
|
||||
FIXTURE14B="$(make_fixture)"
|
||||
FIXTURE14C="$(make_fixture)"
|
||||
FIXTURES+=("$FIXTURE14B" "$FIXTURE14C")
|
||||
break_glob "$FIXTURE14B/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \
|
||||
'[**/SKILL.md]' '[**/.apm/skills/*/SKILL.md]'
|
||||
break_glob "$FIXTURE14C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
|
||||
'[**/agents/*.md]' '[**/.apm/agents/*.md]'
|
||||
if bash "$SCRIPT" "$FIXTURE14B" > /dev/null 2>&1; then
|
||||
fail "exited 0 when skill-audit's glob stopped covering a SKILL.md outside plugins/ -- expected exit 1"
|
||||
else
|
||||
pass "exits non-zero when skill-audit's glob stops covering a project-scope SKILL.md"
|
||||
fi
|
||||
if bash "$SCRIPT" "$FIXTURE14C" > /dev/null 2>&1; then
|
||||
fail "exited 0 when agent-audit's glob stopped covering an agents/*.md outside plugins/ -- expected exit 1"
|
||||
else
|
||||
pass "exits non-zero when agent-audit's glob stops covering a project-scope agents/*.md"
|
||||
fi
|
||||
|
||||
# --- 11. Exits 1 when a probe path falls out of every hook's `files:` regex ---
|
||||
# The probe paths are hardcoded, so they can silently stop representing anything
|
||||
# the hooks lint. Rescoping the shipped agent hook away from the `.agent.md`
|
||||
|
||||
Reference in New Issue
Block a user