Files
holocron/tests/test-check-vale-style-sync.sh
Defame1297 413a750819 fix(scripts): close gates that passed while the thing they guard was disabled
Four repo gates reported success in states they exist to reject.

`check-vale-style-sync.sh` passed while a Kyberforge lint rule was silenced. The
check matched a blocklist of severity values, but Vale's semantic is an allowlist:
anything that is not exactly YES/error/warning/suggestion disables the rule. So
`= false`, `= 0`, `= garbage`, an empty value and — worst — a lowercase `= yes` all
killed enforcement while reading as "enabled" to a human. Inverted to an allowlist.
Two sibling holes: dropping `KyberforgeCopilot` from `BasedOnStyles` unloaded the
Copilot-only check silently, and narrowing a section glob to a location made Vale
lint zero files, which is the "0 files, hook Passed" failure the script's own
comment says it exists to catch.

`sync-marketplace-mirror.sh --check` failed open when its source was missing, while
its sibling correctly errored in the same state.

`check-scope-walkup-sync.sh` wrote to hardcoded `/tmp/fN.out` paths and read one
back, making it non-reentrant — a concurrent instance can flip a verdict, and this
branch made the test runner concurrent. Now per-run `mktemp -d`.

`check-manifests.sh` had no disk-to-marketplace pass, so a plugin directory absent
from `marketplace.json` passed every gate while the `validate-plugins` hook globbed
it. The "listed" match is restricted to remote-source entry names; matching any
entry name let a genuine orphan through on a name coincidence.

`run-bats.sh` reported an empty TAP stream as `0 tests, 0 failures`, exit 0 — a
total harness failure reading as a pass.

The test-side changes are the larger half, because the guards were the real problem.
`test-sync-marketplace-mirror.sh` could overwrite the live tracked mirror under an
inherited GIT_DIR, which is precisely the git-hook context it runs in. The bash-3.2
scan hand-maintained its file list, omitting the new shared runner, and had no rule
for `wait -n` or `nproc` — the two hazards the previous review round found live. It
now derives 43 files across three globs with per-glob floors. Several assertions
were decoration: the concurrency checks caught the reentrancy defect 0 times in 10,
the leak fix was green either way, and two manifest fixtures passed with the code
they claimed to cover deleted. Every assertion now has a revert it provably fails
against.

Refs: #90

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
2026-08-14 01:52:56 +00:00

501 lines
24 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$REPO_ROOT/scripts/check-vale-style-sync.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
# One trap over a registry, rather than rebuilding the trap line per fixture:
# the guard is there because bash 3.2 treats "${arr[@]}" on an empty array as
# unbound under `set -u`.
FIXTURES=()
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
trap cleanup EXIT
# Helper: make a fixture repo with skill-audit/agent-audit's Vale copies, in sync by default.
# The wrapper is a stub — the script only diffs it — but the Vale assets and both
# pre-commit manifests are the repo's real ones, because the .vale.ini checks ask
# vale to apply those globs for real and cross-check them against the shipped
# hooks' `files:` regexes. A synthetic style or manifest would prove nothing, and
# copying the real ones keeps agent-audit's intentional KyberforgeCopilot
# divergence in the fixture instead of a sanitized stand-in for it.
make_fixture() {
local dir
dir="$(mktemp -d)"
local skill_audit="$dir/plugins/kyberforge/.apm/skills/skill-audit"
local agent_audit="$dir/plugins/kyberforge/.apm/skills/agent-audit"
mkdir -p "$skill_audit/scripts" "$agent_audit/scripts"
echo '#!/usr/bin/env bash' > "$skill_audit/scripts/vale-wrap.sh"
echo 'echo wrap' >> "$skill_audit/scripts/vale-wrap.sh"
cp "$skill_audit/scripts/vale-wrap.sh" "$agent_audit/scripts/vale-wrap.sh"
cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/assets" "$skill_audit/"
cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/assets" "$agent_audit/"
cp "$REPO_ROOT/.pre-commit-hooks.yaml" "$REPO_ROOT/.pre-commit-config.yaml" "$dir/"
echo "$dir"
}
# Helper: rewrite a glob section header in one copy's .vale.ini, leaving every
# other line — StylesPath, BasedOnStyles — intact. This is the shape of the
# typo the check exists to catch: the hook still matches the file via its
# `files:` regex, vale lints nothing, and pre-commit reports `Passed`.
break_glob() {
local ini="$1" old="$2" new="$3"
python3 - "$ini" "$old" "$new" <<'PYTHON'
import sys
path, old, new = sys.argv[1], sys.argv[2], sys.argv[3]
with open(path, encoding='utf-8') as fh:
content = fh.read()
assert old in content, f"{old} not found in {path}"
with open(path, 'w', encoding='utf-8') as fh:
fh.write(content.replace(old, new))
PYTHON
}
# --- 1. Exits 0 when the two copies are in sync ---
echo ""
echo "--- exits 0 when skill-audit and agent-audit copies are in sync ---"
FIXTURE="$(make_fixture)"
FIXTURES+=("$FIXTURE")
if bash "$SCRIPT" "$FIXTURE" > /dev/null 2>&1; then
pass "exits 0 when copies are in sync"
else
fail "exited non-zero against in-sync copies"
bash "$SCRIPT" "$FIXTURE" 2>&1 | sed 's/^/ /' || true
fi
# --- 2. Exits 1 when vale-wrap.sh differs between the two copies ---
echo ""
echo "--- exits 1 when vale-wrap.sh differs ---"
FIXTURE2="$(make_fixture)"
FIXTURES+=("$FIXTURE2")
echo 'echo different' >> "$FIXTURE2/plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh"
if bash "$SCRIPT" "$FIXTURE2" > /dev/null 2>&1; then
fail "exited 0 when vale-wrap.sh copies differ — expected exit 1"
else
pass "exits non-zero when vale-wrap.sh copies differ"
fi
# --- 3. Exits 1 when a style rule differs between the two copies ---
echo ""
echo "--- exits 1 when a Kyberforge style rule differs ---"
FIXTURE3="$(make_fixture)"
FIXTURES+=("$FIXTURE3")
echo ' - divergent token' >> "$FIXTURE3/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/VagueWording.yml"
if bash "$SCRIPT" "$FIXTURE3" > /dev/null 2>&1; then
fail "exited 0 when a style rule differs — expected exit 1"
else
pass "exits non-zero when a Kyberforge style rule differs between copies"
fi
# --- 4. Exits 1 when a rule file exists in only one copy ---
echo ""
echo "--- exits 1 when a rule file is missing from one copy ---"
FIXTURE4="$(make_fixture)"
FIXTURES+=("$FIXTURE4")
cat > "$FIXTURE4/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/Extra.yml" <<'EOF'
extends: existence
message: "Extra: '%s'"
level: error
tokens:
- divergent token
EOF
if bash "$SCRIPT" "$FIXTURE4" > /dev/null 2>&1; then
fail "exited 0 when a rule file exists in only one copy — expected exit 1"
else
pass "exits non-zero when a rule file is missing from one copy"
fi
# --- 5. Exits 0 (no-op) when kyberforge isn't present in the target repo ---
echo ""
echo "--- exits 0 when kyberforge skills are absent (no-op) ---"
FIXTURE5="$(mktemp -d)"
FIXTURES+=("$FIXTURE5")
if bash "$SCRIPT" "$FIXTURE5" > /dev/null 2>&1; then
pass "exits 0 as a no-op when skill-audit/agent-audit don't exist"
else
fail "exited non-zero when skill-audit/agent-audit are simply absent"
fi
# --- 5b. Exits 1 when REPO_ROOT does not exist ---
# A nonexistent path used to fall through to the "neither copy present" no-op
# (test 5 above) and exit 0 — indistinguishable from a real, verified in-sync
# result. That guard is for a repo legitimately missing kyberforge, not a
# typo'd or stale path.
echo ""
echo "--- exits 1 when REPO_ROOT does not exist ---"
if bash "$SCRIPT" "/nonexistent/path/$(date +%s)-$$" > /dev/null 2>&1; then
fail "exited 0 for a nonexistent REPO_ROOT — expected exit 1"
else
pass "exits non-zero for a nonexistent REPO_ROOT"
fi
# --- 6. Exits 1 when only one of the two copies is present ---
# The no-op guard used `||`, so a single missing copy also exited 0 — a deleted
# or renamed copy passed the sync check silently.
echo ""
echo "--- exits 1 when only one of the two copies is present ---"
FIXTURE6="$(make_fixture)"
FIXTURE7="$(make_fixture)"
FIXTURES+=("$FIXTURE6" "$FIXTURE7")
rm -rf "$FIXTURE6/plugins/kyberforge/.apm/skills/skill-audit"
rm -rf "$FIXTURE7/plugins/kyberforge/.apm/skills/agent-audit"
if bash "$SCRIPT" "$FIXTURE6" > /dev/null 2>&1; then
fail "exited 0 when only agent-audit is present — expected exit 1"
else
pass "exits non-zero when skill-audit's copy is missing but agent-audit's is present"
fi
if bash "$SCRIPT" "$FIXTURE7" > /dev/null 2>&1; then
fail "exited 0 when only skill-audit is present — expected exit 1"
else
pass "exits non-zero when agent-audit's canonical copy is missing but skill-audit's is present"
fi
# --- 7. Exits 1 when a .vale.ini is missing entirely ---
# Without it vale falls back to an upward config search and lints the file with
# whatever config it happens to find, which is not a failure anyone sees.
echo ""
echo "--- exits 1 when a .vale.ini is missing ---"
FIXTURE8="$(make_fixture)"
FIXTURES+=("$FIXTURE8")
rm -f "$FIXTURE8/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
if bash "$SCRIPT" "$FIXTURE8" > /dev/null 2>&1; then
fail "exited 0 when skill-audit's .vale.ini is missing — expected exit 1"
else
pass "exits non-zero when a .vale.ini is missing"
fi
# --- 8. Exits 1 when the shared StylesPath line is dropped from either copy ---
# StylesPath resolves relative to the .vale.ini, which is the only reason the
# bundled styles are found from a consuming repo's clone prefix.
echo ""
echo "--- exits 1 when StylesPath is missing from either .vale.ini ---"
FIXTURE9="$(make_fixture)"
FIXTURE10="$(make_fixture)"
FIXTURES+=("$FIXTURE9" "$FIXTURE10")
break_glob "$FIXTURE9/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \
'StylesPath = styles' 'StylesPath = elsewhere'
break_glob "$FIXTURE10/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'StylesPath = styles' 'StylesPath = elsewhere'
if bash "$SCRIPT" "$FIXTURE9" > /dev/null 2>&1; then
fail "exited 0 when skill-audit's .vale.ini lost StylesPath — expected exit 1"
else
pass "exits non-zero when skill-audit's .vale.ini lost StylesPath"
fi
if bash "$SCRIPT" "$FIXTURE10" > /dev/null 2>&1; then
fail "exited 0 when agent-audit's .vale.ini lost StylesPath — expected exit 1"
else
pass "exits non-zero when agent-audit's .vale.ini lost StylesPath"
fi
# --- 9. Exits 1 when no section's BasedOnStyles names Kyberforge ---
# Every rule the prefilter gates on lives in that style, so a section that keeps
# its glob but loses the style lints the file and reports nothing.
echo ""
echo "--- exits 1 when BasedOnStyles no longer names Kyberforge ---"
FIXTURE11="$(make_fixture)"
FIXTURES+=("$FIXTURE11")
break_glob "$FIXTURE11/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'BasedOnStyles = Kyberforge' 'BasedOnStyles = KyberforgeCopilot'
if bash "$SCRIPT" "$FIXTURE11" > /dev/null 2>&1; then
fail "exited 0 when agent-audit's .vale.ini stopped naming Kyberforge — expected exit 1"
else
pass "exits non-zero when a .vale.ini no longer names the Kyberforge style"
fi
# --- 9b. Exits 1 when a per-rule override leaves a rule at anything but error ---
# The third way to switch a rule off without touching a style file or a glob.
# CONTEXT.md's "Vale audit prefilter" entry: "Every rule is `level: error` and
# every alert is a FAIL -- no ignorable tier". Vale's exit code keys on `error`
# alerts alone, so any such override leaves the glob intact, the styles
# byte-identical, and the run at `0 errors`, exit 0, `Passed`.
#
# Asserted as an ALLOWLIST because that is vale 3.15.2's own semantic, verified
# by enumerating the value space: only the exact tokens `YES` and `error` keep a
# rule blocking. `warning`/`suggestion` downgrade it (alert printed, exit 0 --
# invisible, since pre-commit swallows a passing hook's output); EVERY other
# value silences it outright, including `false`, `0`, `off`, an empty value,
# `garbage`, and lowercase `yes`. That last one is why a blocklist of
# `NO|warning|suggestion` was not enough: `= yes` reads as "enabled" to a human
# and disables the rule. Case 10's glob probe backstops none of this -- it keys
# on one Kyberforge.VagueWording alert, so DescriptionOpener, PaddingPhrase,
# SentenceOpenerThereIs and ProactivePhrase can each be retired underneath it,
# which is why the cases below deliberately target rules that probe never sees.
#
# Two cases below are about comment forms, and they are NOT symmetric in vale:
# `error # note` (spaced) is stripped by vale and stays live, while `error# note`
# (no space) is not stripped and silences the rule. The gate demands a bare
# token, so it flags both -- deliberately stricter than vale for the spaced form,
# and the only way to catch the no-space form without reimplementing vale's
# comment parsing. `Kyberforge.Vague2` covers rule names carrying a digit: such a
# rule is genuinely silenced by `= NO`, and an alpha-only name class in the gate
# would not even see the line.
echo ""
echo "--- exits 1 when a .vale.ini overrides a Kyberforge rule to anything but YES/error ---"
while IFS= read -r override; do
[[ -n "$override" ]] || continue
# `<EMPTY>` stands in for a bare `Rule =` with no value at all, which the
# heredoc cannot carry as a trailing space without a linter eating it.
override="${override/<EMPTY>/}"
FIXTURE_OV="$(make_fixture)"
FIXTURES+=("$FIXTURE_OV")
echo "$override" >> "$FIXTURE_OV/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
if bash "$SCRIPT" "$FIXTURE_OV" > /dev/null 2>&1; then
fail "exited 0 with '$override' in skill-audit's .vale.ini -- expected exit 1"
else
pass "exits non-zero on '$override'"
fi
done <<'EOF_OVERRIDES'
Kyberforge.SentenceOpenerThereIs = NO
Kyberforge.VagueWording = warning
Kyberforge.SentenceOpenerThereIs = suggestion
Kyberforge.SentenceOpenerThereIs = false
Kyberforge.DescriptionOpener = 0
Kyberforge.PaddingPhrase = off
Kyberforge.SentenceOpenerThereIs = yes
Kyberforge.DescriptionOpener = garbage
Kyberforge.PaddingPhrase =<EMPTY>
Kyberforge.SentenceOpenerThereIs = NO # keep quiet
Kyberforge.DescriptionOpener = error# silenced, vale strips no comment without a space
Kyberforge.PaddingPhrase = error; silenced too, same no-space rule for ';'
Kyberforge.DescriptionOpener = error # stripped by vale, still rejected: bare token required
Kyberforge.Vague2 = NO
Kyberforge.Vague_2 = NO
Kyberforge.Vague-2 = NO
EOF_OVERRIDES
# Same in agent-audit's copy: the check runs over both .vale.ini files, and a
# rule retired in only the canonical copy is the likelier direction. `= false`
# on ProactivePhrase is the sharpest shape -- one word off the original defect,
# on a KyberforgeCopilot rule no glob probe covers.
FIXTURE_OV_AGENT="$(make_fixture)"
FIXTURES+=("$FIXTURE_OV_AGENT")
echo "KyberforgeCopilot.ProactivePhrase = false" \
>> "$FIXTURE_OV_AGENT/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini"
if bash "$SCRIPT" "$FIXTURE_OV_AGENT" > /dev/null 2>&1; then
fail "exited 0 with 'KyberforgeCopilot.ProactivePhrase = false' in agent-audit's .vale.ini -- expected exit 1"
else
pass "exits non-zero when agent-audit's copy retires a KyberforgeCopilot rule"
fi
# The two allowlisted values must NOT trip the assertion -- otherwise it would
# fire on any legitimate explicit enablement. Kept as a positive case so an
# over-broad tightening of the regex shows up here rather than in the repo.
FIXTURE_OV_OK="$(make_fixture)"
FIXTURES+=("$FIXTURE_OV_OK")
{
echo "Kyberforge.SentenceOpenerThereIs = YES"
echo "Kyberforge.VagueWording = error"
} >> "$FIXTURE_OV_OK/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
if bash "$SCRIPT" "$FIXTURE_OV_OK" > /dev/null 2>&1; then
pass "an explicit '= YES' / '= error' override is not flagged"
else
fail "flagged an explicit '= YES' / '= error' override -- those are the two values that keep a rule blocking"
bash "$SCRIPT" "$FIXTURE_OV_OK" 2>&1 | sed 's/^/ /' || true
fi
# --- 9c. Exits 1 when agent-audit ships KyberforgeCopilot but never loads it ---
# Case 9 asserts only that Kyberforge is named, because skill-audit's copy
# legitimately has no Copilot style. So dropping just `, KyberforgeCopilot` from
# agent-audit's [**/*.agent.md] section unloaded the whole style silently: no
# glob broke, the styles/ diff stayed clean (the directory is still shipped,
# only never loaded), the two .vale.ini files are deliberately unequal so no
# equality check applies, and case 10's probe still passed because it keys on a
# Kyberforge alert. Verified dead by probing a `.agent.md` carrying
# "Use proactively": 0 alerts under the broken config, KyberforgeCopilot.
# ProactivePhrase under the shipped one. CONTEXT.md describes the style as
# "scoped only to `.agent.md` files for the Copilot-only 'Use proactively has
# no effect' check", so shipping it unloaded is drift.
echo ""
echo "--- exits 1 when the shipped KyberforgeCopilot style is named by no BasedOnStyles ---"
FIXTURE11C="$(make_fixture)"
FIXTURES+=("$FIXTURE11C")
break_glob "$FIXTURE11C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'BasedOnStyles = Kyberforge, KyberforgeCopilot' 'BasedOnStyles = Kyberforge'
if bash "$SCRIPT" "$FIXTURE11C" > /dev/null 2>&1; then
fail "exited 0 when KyberforgeCopilot was dropped from BasedOnStyles -- expected exit 1"
else
pass "exits non-zero when a shipped KyberforgeCopilot style is never loaded"
fi
# The assertion is conditional on the style being shipped: a copy with no
# KyberforgeCopilot directory (skill-audit's, by design) must stay clean --
# case 13 below covers the shipped-and-loaded pairing.
# --- 10. Exits 1 when a glob section stops matching the shape its hook lints ---
# One case per glob section, because each covers a file shape the others don't:
# agent-audit's [**/*.agent.md] is the only section covering a Copilot agent file
# outside an agents/ directory, so breaking it alone is invisible to the others.
echo ""
echo "--- exits 1 when a .vale.ini glob no longer matches its hook's file shape ---"
FIXTURE12="$(make_fixture)"
FIXTURE13="$(make_fixture)"
FIXTURE14="$(make_fixture)"
FIXTURES+=("$FIXTURE12" "$FIXTURE13" "$FIXTURE14")
break_glob "$FIXTURE12/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \
'[**/SKILL.md]' '[**/NOMATCH.md]'
break_glob "$FIXTURE13/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'[**/agents/*.md]' '[**/NOMATCH-agents/*.md]'
break_glob "$FIXTURE14/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'[**/*.agent.md]' '[**/*.NOMATCH.md]'
if bash "$SCRIPT" "$FIXTURE12" > /dev/null 2>&1; then
fail "exited 0 when skill-audit's SKILL.md glob matched nothing — expected exit 1"
else
pass "exits non-zero when skill-audit's SKILL.md glob matches nothing"
fi
if bash "$SCRIPT" "$FIXTURE13" > /dev/null 2>&1; then
fail "exited 0 when agent-audit's agents/*.md glob matched nothing — expected exit 1"
else
pass "exits non-zero when agent-audit's agents/*.md glob matches nothing"
fi
if bash "$SCRIPT" "$FIXTURE14" > /dev/null 2>&1; then
fail "exited 0 when agent-audit's *.agent.md glob matched nothing — expected exit 1"
else
pass "exits non-zero when agent-audit's *.agent.md glob matches nothing"
fi
# --- 10b. Exits 1 when a glob is narrowed to this repo's own plugins/ layout ---
# Every probe path used to start with `plugins/`, so a glob narrowed from a
# filename shape to a location (`[**/SKILL.md]` -> `[**/.apm/skills/*/SKILL.md]`)
# still matched all of them and the check passed -- while a project-scope
# `.claude/skills/foo/SKILL.md` started linting as `0 errors ... in 0 files`,
# exit 0, hook `Passed`: the exact failure the script's own header comment says
# it exists to catch. CONTEXT.md: "A `SKILL.md` outside `plugins/` (e.g.
# project-scope `.claude/skills/foo/SKILL.md`) still matches `[**/SKILL.md]` and
# gets linted normally -- the globs constrain filename shape, not location."
# These narrowings are still valid glob syntax and break no `plugins/`-shaped
# file, so only a non-`plugins/` probe path catches them.
echo ""
echo "--- exits 1 when a .vale.ini glob is narrowed from a filename shape to a location ---"
FIXTURE14B="$(make_fixture)"
FIXTURE14C="$(make_fixture)"
FIXTURES+=("$FIXTURE14B" "$FIXTURE14C")
break_glob "$FIXTURE14B/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \
'[**/SKILL.md]' '[**/.apm/skills/*/SKILL.md]'
break_glob "$FIXTURE14C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'[**/agents/*.md]' '[**/.apm/agents/*.md]'
if bash "$SCRIPT" "$FIXTURE14B" > /dev/null 2>&1; then
fail "exited 0 when skill-audit's glob stopped covering a SKILL.md outside plugins/ -- expected exit 1"
else
pass "exits non-zero when skill-audit's glob stops covering a project-scope SKILL.md"
fi
if bash "$SCRIPT" "$FIXTURE14C" > /dev/null 2>&1; then
fail "exited 0 when agent-audit's glob stopped covering an agents/*.md outside plugins/ -- expected exit 1"
else
pass "exits non-zero when agent-audit's glob stops covering a project-scope agents/*.md"
fi
# --- 11. Exits 1 when a probe path falls out of every hook's `files:` regex ---
# The probe paths are hardcoded, so they can silently stop representing anything
# the hooks lint. Rescoping the shipped agent hook away from the `.agent.md`
# shape has to fail here rather than leave a probe testing a shape no hook
# matches any more.
echo ""
echo "--- exits 1 when a probe path matches no hook's files: regex ---"
FIXTURE16="$(make_fixture)"
FIXTURES+=("$FIXTURE16")
break_glob "$FIXTURE16/.pre-commit-hooks.yaml" \
"files: '(^|/)agents/[^/]+\\.md\$|\\.agent\\.md\$'" "files: '(^|/)agents/[^/]+\\.md\$'"
if bash "$SCRIPT" "$FIXTURE16" > /dev/null 2>&1; then
fail "exited 0 when the agent hook was rescoped away from .agent.md — expected exit 1"
else
pass "exits non-zero when a probe path is in no hook's scope any more"
fi
# --- 11b. Exits 1 when the local config's files: regex narrows out of sync
# with the canonical .pre-commit-hooks.yaml regex ---
# hook_file_regexes() used to union the two manifests' `files:` regexes before
# checking probe coverage, so a probe that matched only the old, looser
# .pre-commit-hooks.yaml pattern still passed as "in scope" even after
# .pre-commit-config.yaml's copy of the same hook was narrowed away from it.
# That is exactly the shape of rescoping this repo's own agent hook went
# through (SKILL/agent `.md` -> `.apm/.../*.agent.md`): the local hook quietly
# stopped linting a shape the shipped, external-facing manifest still claims
# to cover, and nothing caught it. Reproduce it directly: narrow only the
# fixture's local config regex (leave .pre-commit-hooks.yaml as shipped) and
# assert the check now flags the disagreement instead of passing silently.
echo ""
echo "--- exits 1 when .pre-commit-config.yaml's files: regex drifts out of sync with .pre-commit-hooks.yaml's ---"
FIXTURE16B="$(make_fixture)"
FIXTURES+=("$FIXTURE16B")
break_glob "$FIXTURE16B/.pre-commit-config.yaml" \
"files: '^plugins/[^/]+/\\.apm/agents/[^/]+\\.agent\\.md\$'" \
"files: '^plugins/kyberforge/\\.apm/agents/[^/]+\\.agent\\.md\$'"
if bash "$SCRIPT" "$FIXTURE16B" > /dev/null 2>&1; then
fail "exited 0 when the local config regex narrowed out of sync with .pre-commit-hooks.yaml — expected exit 1"
else
pass "exits non-zero when the local config regex narrows out of sync with the canonical .pre-commit-hooks.yaml regex"
fi
# --- 12. The text-level assertions hold on a machine without vale ---
# They are the fallback when the glob probe cannot run. With vale on PATH the
# probe fails on these same mutations, so it would mask them: only masking vale
# proves a clean run here means the text assertions themselves ran.
echo ""
echo "--- the StylesPath / BasedOnStyles assertions still gate with vale masked off PATH ---"
VALE_DIR="$(dirname "$(command -v vale 2>/dev/null || echo /nonexistent/vale)")"
PATH_NO_VALE="$(printf '%s' "$PATH" | tr ':' '\n' | grep -vxF "$VALE_DIR" | paste -sd: -)"
if (PATH="$PATH_NO_VALE"; command -v vale >/dev/null 2>&1); then
fail "could not mask vale off PATH — the vale-absent fallback was not exercised"
else
FIXTURE17="$(make_fixture)"
FIXTURE18="$(make_fixture)"
FIXTURE19="$(make_fixture)"
FIXTURES+=("$FIXTURE17" "$FIXTURE18" "$FIXTURE19")
break_glob "$FIXTURE18/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \
'StylesPath = styles' 'StylesPath = elsewhere'
break_glob "$FIXTURE19/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \
'BasedOnStyles = Kyberforge' 'BasedOnStyles = KyberforgeCopilot'
if PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE17" > /dev/null 2>&1; then
pass "exits 0 on in-sync copies with vale unavailable"
else
fail "exited non-zero on in-sync copies with vale unavailable — the missing binary must warn, not fail"
fi
if PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE18" > /dev/null 2>&1; then
fail "exited 0 on a dropped StylesPath with vale unavailable — expected exit 1"
else
pass "exits non-zero on a dropped StylesPath with vale unavailable"
fi
if PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE19" > /dev/null 2>&1; then
fail "exited 0 on a BasedOnStyles that dropped Kyberforge with vale unavailable — expected exit 1"
else
pass "exits non-zero on a BasedOnStyles that dropped Kyberforge with vale unavailable"
fi
# A clean run without vale must say so — silence would read as verified.
if PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE17" 2>&1 | grep -q "vale is not installed"; then
pass "warns that glob coverage was not verified when vale is unavailable"
else
fail "exited clean without vale and said nothing — an unverified run looks identical to a verified one"
fi
fi
# --- 13. The intentional agent-audit-only divergence is NOT flagged ---
# The two .vale.ini files are deliberately different: agent-audit ships an extra
# [**/*.agent.md] section and the KyberforgeCopilot style. A check that diffed
# them would fail the repo as it stands, so assert the divergence is really in
# the fixture before asserting the check tolerates it — otherwise this case would
# still pass if the fixture had quietly stopped carrying it.
echo ""
echo "--- exits 0 despite agent-audit's KyberforgeCopilot divergence ---"
FIXTURE15="$(make_fixture)"
FIXTURES+=("$FIXTURE15")
AGENT_INI15="$FIXTURE15/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini"
SKILL_INI15="$FIXTURE15/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini"
if ! grep -q "KyberforgeCopilot" "$AGENT_INI15" \
|| grep -q "KyberforgeCopilot" "$SKILL_INI15" \
|| [[ ! -d "$FIXTURE15/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/KyberforgeCopilot" ]]; then
fail "the fixture no longer carries the agent-audit-only KyberforgeCopilot divergence, so tolerating it proves nothing"
elif bash "$SCRIPT" "$FIXTURE15" > /dev/null 2>&1; then
pass "exits 0 with agent-audit's extra KyberforgeCopilot section and style present"
else
fail "flagged the intentional agent-audit-only KyberforgeCopilot divergence — expected exit 0"
bash "$SCRIPT" "$FIXTURE15" 2>&1 | sed 's/^/ /' || true
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]