fix(scripts): close gates that passed while the thing they guard was disabled
Four repo gates reported success in states they exist to reject. `check-vale-style-sync.sh` passed while a Kyberforge lint rule was silenced. The check matched a blocklist of severity values, but Vale's semantic is an allowlist: anything that is not exactly YES/error/warning/suggestion disables the rule. So `= false`, `= 0`, `= garbage`, an empty value and — worst — a lowercase `= yes` all killed enforcement while reading as "enabled" to a human. Inverted to an allowlist. Two sibling holes: dropping `KyberforgeCopilot` from `BasedOnStyles` unloaded the Copilot-only check silently, and narrowing a section glob to a location made Vale lint zero files, which is the "0 files, hook Passed" failure the script's own comment says it exists to catch. `sync-marketplace-mirror.sh --check` failed open when its source was missing, while its sibling correctly errored in the same state. `check-scope-walkup-sync.sh` wrote to hardcoded `/tmp/fN.out` paths and read one back, making it non-reentrant — a concurrent instance can flip a verdict, and this branch made the test runner concurrent. Now per-run `mktemp -d`. `check-manifests.sh` had no disk-to-marketplace pass, so a plugin directory absent from `marketplace.json` passed every gate while the `validate-plugins` hook globbed it. The "listed" match is restricted to remote-source entry names; matching any entry name let a genuine orphan through on a name coincidence. `run-bats.sh` reported an empty TAP stream as `0 tests, 0 failures`, exit 0 — a total harness failure reading as a pass. The test-side changes are the larger half, because the guards were the real problem. `test-sync-marketplace-mirror.sh` could overwrite the live tracked mirror under an inherited GIT_DIR, which is precisely the git-hook context it runs in. The bash-3.2 scan hand-maintained its file list, omitting the new shared runner, and had no rule for `wait -n` or `nproc` — the two hazards the previous review round found live. It now derives 43 files across three globs with per-glob floors. Several assertions were decoration: the concurrency checks caught the reentrancy defect 0 times in 10, the leak fix was green either way, and two manifest fixtures passed with the code they claimed to cover deleted. Every assertion now has a revert it provably fails against. Refs: #90 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
This commit is contained in:
@@ -46,6 +46,18 @@ FIXTURES=()
|
||||
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
# Per-run scratch directory for the per-fixture output captures below. These
|
||||
# used to be fixed paths in the shared system temp directory, which made this
|
||||
# script non-reentrant: tests/run-tests.sh now fans its scripts out
|
||||
# concurrently, and two instances sharing one path clobber each other's
|
||||
# captures. Fixture 6 reads its capture back (`[[ -z "$(cat ...)" ]]`), so a
|
||||
# cross-run write there silently flips a real verdict, and a pre-existing
|
||||
# directory sitting at one of the paths breaks the run outright. Keep these
|
||||
# under a per-run mktemp -d; tests/test-check-scope-walkup-sync.sh asserts it.
|
||||
# Registered in FIXTURES so the single cleanup trap already here removes it.
|
||||
RUN_TMP="$(mktemp -d)"
|
||||
FIXTURES+=("$RUN_TMP")
|
||||
|
||||
# Fill a new-agent.sh-scaffolded pair's FILL IN: placeholders with valid
|
||||
# content, isolating the scope/counterpart-lookup question from unrelated
|
||||
# content-quality FAILs when cross-checking against validate.sh.
|
||||
@@ -78,11 +90,11 @@ else
|
||||
else
|
||||
fill_agent_pair "$F1_HOME/.claude/agents/$NAME1.md" "$NAME1"
|
||||
fill_agent_pair "$F1_HOME/.copilot/agents/$NAME1.agent.md" "$NAME1"
|
||||
if env HOME="$F1_HOME" bash "$VALIDATE" "$F1_HOME/.claude/agents/$NAME1.md" >/tmp/f1.out 2>&1; then
|
||||
if env HOME="$F1_HOME" bash "$VALIDATE" "$F1_HOME/.claude/agents/$NAME1.md" >"$RUN_TMP/f1.out" 2>&1; then
|
||||
ok "validate.sh agrees: user scope, counterpart found under \$HOME/.copilot"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh's user-scope classification at root exactly \$HOME"
|
||||
sed 's/^/ /' /tmp/f1.out
|
||||
sed 's/^/ /' "$RUN_TMP/f1.out"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
@@ -111,22 +123,22 @@ else
|
||||
ok "new-agent.sh: nested marker-less dir under \$HOME scaffolds project scope at the nested dir"
|
||||
fill_agent_pair "$F2_NESTED/.claude/agents/$NAME2.md" "$NAME2"
|
||||
fill_agent_pair "$F2_NESTED/.github/agents/$NAME2.agent.md" "$NAME2"
|
||||
if env HOME="$F2_HOME" bash "$VALIDATE" "$F2_NESTED/.claude/agents/$NAME2.md" >/tmp/f2.out 2>&1; then
|
||||
if env HOME="$F2_HOME" bash "$VALIDATE" "$F2_NESTED/.claude/agents/$NAME2.md" >"$RUN_TMP/f2.out" 2>&1; then
|
||||
ok "validate.sh agrees: project scope, counterpart found at the nested dir (not \$HOME/.copilot)"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh: misclassified the nested marker-less \$HOME subdirectory"
|
||||
sed 's/^/ /' /tmp/f2.out
|
||||
sed 's/^/ /' "$RUN_TMP/f2.out"
|
||||
fi
|
||||
# new-skill.sh has no user/project distinction of its own (no $HOME
|
||||
# awareness at all — see new-skill.sh's find_package_root), but it shares
|
||||
# the same .git/apm.yml walk-up primitive. It must land its standalone
|
||||
# scaffold at the given path too, not get redirected toward $HOME.
|
||||
if env HOME="$F2_HOME" bash "$NEW_SKILL" probe-home-nested-skill "$F2_NESTED" >/tmp/f2skill.out 2>&1 \
|
||||
if env HOME="$F2_HOME" bash "$NEW_SKILL" probe-home-nested-skill "$F2_NESTED" >"$RUN_TMP/f2skill.out" 2>&1 \
|
||||
&& [[ -d "$F2_NESTED/probe-home-nested-skill" ]]; then
|
||||
ok "new-skill.sh agrees: standalone mode scaffolds at the nested dir, not redirected toward \$HOME"
|
||||
else
|
||||
err "new-skill.sh disagreed with new-agent.sh/validate.sh on the nested marker-less \$HOME subdirectory"
|
||||
sed 's/^/ /' /tmp/f2skill.out
|
||||
sed 's/^/ /' "$RUN_TMP/f2skill.out"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
@@ -156,11 +168,11 @@ else
|
||||
else
|
||||
fill_agent_pair "$F3_PROBE/.claude/agents/$NAME3.md" "$NAME3"
|
||||
fill_agent_pair "$F3_PROBE/.github/agents/$NAME3.agent.md" "$NAME3"
|
||||
if env HOME="$F3_HOME" bash "$VALIDATE" "$F3_PROBE/.claude/agents/$NAME3.md" >/tmp/f3.out 2>&1; then
|
||||
if env HOME="$F3_HOME" bash "$VALIDATE" "$F3_PROBE/.claude/agents/$NAME3.md" >"$RUN_TMP/f3.out" 2>&1; then
|
||||
ok "validate.sh agrees: .git boundary keeps this project scope, not promoted to user scope at \$HOME"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh on the .git-boundary-before-\$HOME fixture"
|
||||
sed 's/^/ /' /tmp/f3.out
|
||||
sed 's/^/ /' "$RUN_TMP/f3.out"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
@@ -188,11 +200,11 @@ else
|
||||
else
|
||||
fill_agent_pair "$F3B_PROBE/.claude/agents/$NAME3B.md" "$NAME3B"
|
||||
fill_agent_pair "$F3B_PROBE/.github/agents/$NAME3B.agent.md" "$NAME3B"
|
||||
if bash "$VALIDATE" "$F3B_PROBE/.claude/agents/$NAME3B.md" >/tmp/f3b.out 2>&1; then
|
||||
if bash "$VALIDATE" "$F3B_PROBE/.claude/agents/$NAME3B.md" >"$RUN_TMP/f3b.out" 2>&1; then
|
||||
ok "validate.sh agrees: scope root is <root>, not the .git ancestor above it"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh: resolved scope to the .git ancestor instead of <root>"
|
||||
sed 's/^/ /' /tmp/f3b.out
|
||||
sed 's/^/ /' "$RUN_TMP/f3b.out"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
@@ -214,19 +226,19 @@ elif [[ ! -f "$F4_ROOT/.apm/agents/$NAME4.agent.md" ]]; then
|
||||
err "new-agent.sh did not scaffold plugin scope at the type-bearing apm.yml root"
|
||||
else
|
||||
ok "new-agent.sh: plugin scope at type-bearing apm.yml root"
|
||||
if bash "$NEW_SKILL" probe-plugin-skill "$F4_ROOT" >/tmp/f4skill.out 2>&1 \
|
||||
if bash "$NEW_SKILL" probe-plugin-skill "$F4_ROOT" >"$RUN_TMP/f4skill.out" 2>&1 \
|
||||
&& [[ -d "$F4_ROOT/.apm/skills/probe-plugin-skill" ]]; then
|
||||
ok "new-skill.sh agrees: package mode at the same apm.yml root"
|
||||
else
|
||||
err "new-skill.sh disagreed with new-agent.sh on the type-bearing apm.yml root"
|
||||
sed 's/^/ /' /tmp/f4skill.out
|
||||
sed 's/^/ /' "$RUN_TMP/f4skill.out"
|
||||
fi
|
||||
fill_agent_pair "$F4_ROOT/.apm/agents/$NAME4.agent.md" "$NAME4"
|
||||
if bash "$VALIDATE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4validate.out 2>&1; then
|
||||
if bash "$VALIDATE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >"$RUN_TMP/f4validate.out" 2>&1; then
|
||||
ok "validate.sh agrees: plugin/APM scope, structural checks pass"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh: did not treat the type-bearing apm.yml root as plugin scope"
|
||||
sed 's/^/ /' /tmp/f4validate.out
|
||||
sed 's/^/ /' "$RUN_TMP/f4validate.out"
|
||||
fi
|
||||
# source_keys + a matching sources.md round-trips only if validate-provenance.sh
|
||||
# resolves the SAME plugin root new-agent.sh/new-skill.sh did.
|
||||
@@ -251,11 +263,11 @@ EOF
|
||||
- **Research doc:** (none)
|
||||
- **Status:** \`extracted\`
|
||||
EOF
|
||||
if bash "$VALIDATE_PROVENANCE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4prov.out 2>&1; then
|
||||
if bash "$VALIDATE_PROVENANCE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >"$RUN_TMP/f4prov.out" 2>&1; then
|
||||
ok "validate-provenance.sh agrees: resolves the same plugin root, sources.md round-trips"
|
||||
else
|
||||
err "validate-provenance.sh disagreed on the plugin root for the type-bearing apm.yml fixture"
|
||||
sed 's/^/ /' /tmp/f4prov.out
|
||||
sed 's/^/ /' "$RUN_TMP/f4prov.out"
|
||||
fi
|
||||
fi
|
||||
|
||||
@@ -280,11 +292,11 @@ else
|
||||
else
|
||||
fill_agent_pair "$F5_ROOT/.claude/agents/$NAME5.md" "$NAME5"
|
||||
fill_agent_pair "$F5_ROOT/.github/agents/$NAME5.agent.md" "$NAME5"
|
||||
if env HOME="$F5_UNRELATED_HOME" bash "$VALIDATE" "$F5_ROOT/.claude/agents/$NAME5.md" >/tmp/f5.out 2>&1; then
|
||||
if env HOME="$F5_UNRELATED_HOME" bash "$VALIDATE" "$F5_ROOT/.claude/agents/$NAME5.md" >"$RUN_TMP/f5.out" 2>&1; then
|
||||
ok "validate.sh agrees: filesystem-boundary fallback resolves to project scope"
|
||||
else
|
||||
err "validate.sh disagreed with new-agent.sh on the filesystem-boundary fallback fixture"
|
||||
sed 's/^/ /' /tmp/f5.out
|
||||
sed 's/^/ /' "$RUN_TMP/f5.out"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
@@ -325,12 +337,12 @@ EOF
|
||||
# No sources.md exists anywhere under $F6_HOME or at the ancestor package
|
||||
# root — if find_plugin_root walked past $HOME to the ancestor apm.yml,
|
||||
# this would FAIL on Check 0 (source_keys declared but sources.md absent).
|
||||
if env HOME="$F6_HOME" bash "$VALIDATE_PROVENANCE" "$F6_HOME/.apm/agents/probe-prov.agent.md" >/tmp/f6.out 2>&1 \
|
||||
&& [[ -z "$(cat /tmp/f6.out)" ]]; then
|
||||
if env HOME="$F6_HOME" bash "$VALIDATE_PROVENANCE" "$F6_HOME/.apm/agents/probe-prov.agent.md" >"$RUN_TMP/f6.out" 2>&1 \
|
||||
&& [[ -z "$(cat "$RUN_TMP/f6.out")" ]]; then
|
||||
ok "validate-provenance.sh agrees: \$HOME boundary stops the walk, exits 0 silently (not plugin scope)"
|
||||
else
|
||||
err "validate-provenance.sh walked past \$HOME to the ancestor apm.yml — disagrees with new-agent.sh"
|
||||
sed 's/^/ /' /tmp/f6.out
|
||||
sed 's/^/ /' "$RUN_TMP/f6.out"
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
Reference in New Issue
Block a user