fix(tests): replace pipefail-racy echo | grep -q with here-strings
Why Two suites failed intermittently — tests/test-vale-wrap.sh case 21 and tests/test-check-release-needed.sh cases 4 and 15 — on correct output, and never when run alone. The cause is the `echo "$OUT" | grep -q P` idiom under `set -o pipefail`: grep -q exits as soon as it has an answer, bash's echo can hand a multi-line value to the pipe one line at a time, and a write after the reader is gone kills echo with SIGPIPE. pipefail then reports the writer's death, so output that DID match reads as "no match". Every observed failure had lines after its match; case 15's match is on line 1 of 6, the widest window in that file. Forced with a pause before the writer's last line, the pipe form failed 50 of 50 runs; a here-string, a match on the last line, and the same pipe without pipefail each passed 50 of 50. Unforced the rate is about 1 per 670 suite runs, which is why it read as a flaky gate rather than a bug. The failures at review time are consistent with this, but were not proven to be it: the suite was running while agents edited live config files in place, and a brief change to .vale.ini or .pre-commit-hooks.yaml would produce the same two failures. The race is real and fixed either way. Implementation Notes `grep -q P <<< "$VAR"` has no separate writer process, so there is nothing to race. It is not a retry or a sleep. 121 sites converted across 9 files, three of them scripts rather than tests: new-agent.sh, new-skill.sh and check-executables-allow-sync.sh. None ships via .pre-commit-hooks.yaml, so no external consumer pins them, and all three are single-pipeline checks whose verdict cannot change. Left alone deliberately: 14 sites whose writer is a command, not a shell builtin — they either absorb the writer's status with `|| true` or are python3 and awk, which write once at exit — and one file with no pipefail. `printf '%s'` sites differ from a here-string only by a trailing newline, which no -q verdict on a non-empty pattern depends on. tests/test-no-pipefail-early-exit-grep.sh is a static guard against new occurrences, discovered automatically by run-tests.sh. It only scans files that set pipefail, joins continuation lines, skips comments, and flags only echo/printf writers. Its first case proves the scanner can fail before its second trusts a clean verdict on the tree. A guard covers exactly the spellings its regex models, so the miss surface was measured rather than assumed. Four were found and closed: pipefail declared as `set -o errexit -o pipefail` (where the old pattern required pipefail to follow the FIRST -o, and a file-level miss skips every site in that file); a writer separated from grep by an intermediate stage; a pipeline wrapped on a trailing `|` rather than a backslash; and readers spelled egrep, fgrep, /bin/grep, `command grep` or with an env-var prefix. Segment characters exclude a bare `&` so `echo ok && other | grep -q x`, whose writer is `other`, does not false-fire. Widening surfaced 5 live sites invisible to the original scanner, all in tests/test-apm-current-hook.sh, all `echo "$out" | json_field ... | grep -q`; they are safe today only because json_field is python3, which reads to EOF and writes once. Fixtures go 4 to 12 vulnerable spellings plus near-miss negatives. Two `grep ... | head -1` sites (test-vale-wrap.sh) are the same race with a different early-exiting reader, and are fixed by absorbing the writer. The scanner deliberately does not model `head`, `sed -n 1p` or a bare `read`: most legitimate uses in this tree are already absorbed with `|| true` and the scanner cannot see absorption from pipeline text, so a high false-positive rate would be how this guard gets weakened. Heredoc bodies are scanned as code; none in the tree trips it today. Impact The bug predates the factory-audit merge: every converted site in check-release-needed and case 21 dates to4d018afandaa8cc22(2026-08-09). Test suites go 19 to 20. `run-tests.sh --strict` passes 20/20 with 0 skipped, four consecutive runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
@@ -246,12 +246,12 @@ EOF
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5"' EXIT
|
||||
OUT5=$(run_wrap "$FIXTURE5" --config "$VALE_CONFIG" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md)
|
||||
if echo "$OUT5" | grep -q "VagueWording"; then
|
||||
if grep -q "VagueWording" <<< "$OUT5"; then
|
||||
pass "flags vague wording when the description contains an apostrophe"
|
||||
else
|
||||
fail "silently missed vague wording in a description containing an apostrophe"
|
||||
fi
|
||||
if echo "$OUT5" | grep -qi "yaml:"; then
|
||||
if grep -qi "yaml:" <<< "$OUT5"; then
|
||||
fail "flattened copy with an apostrophe produced a YAML parse error"
|
||||
else
|
||||
pass "flattened copy with an apostrophe is valid YAML (no parse error)"
|
||||
@@ -297,9 +297,9 @@ EOF
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7"' EXIT
|
||||
OUT7=$(run_wrap "$FIXTURE7" --config "$VALE_CONFIG" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md)
|
||||
if echo "$OUT7" | grep -q "Traceback"; then
|
||||
if grep -q "Traceback" <<< "$OUT7"; then
|
||||
fail "crashed while flattening a description with a blank line between paragraphs"
|
||||
elif echo "$OUT7" | grep -q "VagueWording"; then
|
||||
elif grep -q "VagueWording" <<< "$OUT7"; then
|
||||
pass "still flags vague wording in the second paragraph after a blank line"
|
||||
else
|
||||
fail "silently missed vague wording in the second paragraph after a blank line — the bug this test guards against"
|
||||
@@ -337,7 +337,7 @@ REL_CFG8="../../../../.vale.ini"
|
||||
REL_FILE8="SKILL.md"
|
||||
OUT_EQ=$(run_wrap "$SUBDIR8" "--config=$REL_CFG8" "$REL_FILE8")
|
||||
OUT_TWO=$(run_wrap "$SUBDIR8" --config "$REL_CFG8" "$REL_FILE8")
|
||||
if echo "$OUT_EQ" | grep -q "VagueWording" && [[ "$OUT_EQ" == "$OUT_TWO" ]]; then
|
||||
if grep -q "VagueWording" <<< "$OUT_EQ" && [[ "$OUT_EQ" == "$OUT_TWO" ]]; then
|
||||
pass "cwd-relative --config resolves from a subdirectory in both argv forms"
|
||||
else
|
||||
fail "cwd-relative --config did not resolve from a subdirectory (equals form vs two-argv form)"
|
||||
@@ -355,7 +355,7 @@ RC_REL_CFG=$?
|
||||
OUT_ABS_CFG=$(cd "$SUBDIR8" && bash "$SCRIPT" --config "$FIXTURE8/.vale.ini" "$REL_FILE8" 2>&1)
|
||||
RC_ABS_CFG=$?
|
||||
set -e
|
||||
if echo "$OUT_REL_CFG" | grep -qi "does not exist"; then
|
||||
if grep -qi "does not exist" <<< "$OUT_REL_CFG"; then
|
||||
fail "cwd-relative --config hard-errored ('does not exist') — the bug this test guards against"
|
||||
elif [[ "$OUT_REL_CFG" == "$OUT_ABS_CFG" && "$RC_REL_CFG" -eq "$RC_ABS_CFG" ]]; then
|
||||
pass "cwd-relative --config matches the absolute-path invocation (output and exit code)"
|
||||
@@ -373,9 +373,9 @@ echo ""
|
||||
echo "--- flattens a cwd-relative file argument passed from a subdirectory ---"
|
||||
WRAPPED_REL=$(run_wrap "$SUBDIR8" --config "$FIXTURE8/.vale.ini" "$REL_FILE8")
|
||||
BARE_REL=$(cd "$SUBDIR8" && vale --config "$FIXTURE8/.vale.ini" "$REL_FILE8" 2>&1 || true)
|
||||
if ! echo "$WRAPPED_REL" | grep -q "VagueWording"; then
|
||||
if ! grep -q "VagueWording" <<< "$WRAPPED_REL"; then
|
||||
fail "cwd-relative file argument produced no alert — flattening was silently skipped, the bug this test guards against"
|
||||
elif echo "$BARE_REL" | grep -q "VagueWording"; then
|
||||
elif grep -q "VagueWording" <<< "$BARE_REL"; then
|
||||
fail "bare vale already flags this fixture, so the test can't detect a silently-skipped flattening"
|
||||
else
|
||||
pass "cwd-relative file argument is flattened and flagged where bare vale reports nothing"
|
||||
@@ -443,7 +443,7 @@ trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTU
|
||||
REL11="plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
WRAPPED_OUT=$(run_wrap "$FIXTURE11" --config "$VALE_CONFIG" "$REL11")
|
||||
BARE_OUT=$(cd "$FIXTURE11" && vale --config "$VALE_CONFIG" "$REL11" 2>&1 || true)
|
||||
if ! echo "$BARE_OUT" | grep -q "VagueWording"; then
|
||||
if ! grep -q "VagueWording" <<< "$BARE_OUT"; then
|
||||
fail "bare vale reports nothing for a literal (|) block scalar — the 'literal blocks are not broken' premise is wrong"
|
||||
elif [[ "$WRAPPED_OUT" == "$BARE_OUT" ]]; then
|
||||
pass "literal (|) block scalar output matches bare vale exactly — untouched by flattening"
|
||||
@@ -461,7 +461,7 @@ echo "--- defaults --config to the wrapper's own sibling assets/vale/.vale.ini -
|
||||
FIXTURE12="$(make_fixture 2)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7" "$FIXTURE8" "$FIXTURE10" "$FIXTURE11" "$FIXTURE12"' EXIT
|
||||
OUT12=$(run_wrap "$FIXTURE12" plugins/testplugin/skills/zzzskill/SKILL.md)
|
||||
if echo "$OUT12" | grep -q "VagueWording"; then
|
||||
if grep -q "VagueWording" <<< "$OUT12"; then
|
||||
pass "a --config-less invocation uses the wrapper's bundled config"
|
||||
else
|
||||
fail "a --config-less invocation found no config — external pre-commit consumers get E100, the bug this test guards against"
|
||||
@@ -490,9 +490,9 @@ EOF
|
||||
chmod +x "$STUB13/realpath"
|
||||
OUT13=$(cd "$FIXTURE12" && PATH="$STUB13:$PATH" bash "$SCRIPT" --config "$VALE_CONFIG" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md 2>&1 || true)
|
||||
if echo "$OUT13" | grep -q "illegal option"; then
|
||||
if grep -q "illegal option" <<< "$OUT13"; then
|
||||
fail "invoked realpath -m — fails on macOS's BSD realpath, the bug this test guards against"
|
||||
elif echo "$OUT13" | grep -q "VagueWording"; then
|
||||
elif grep -q "VagueWording" <<< "$OUT13"; then
|
||||
pass "flattens and flags with no GNU realpath available"
|
||||
else
|
||||
fail "produced no alert under a BSD-style realpath: $OUT13"
|
||||
@@ -509,9 +509,9 @@ FIXTURE14="$(make_fixture 2)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7" "$FIXTURE8" "$FIXTURE10" "$FIXTURE11" "$FIXTURE12" "$STUB13" "$FIXTURE14"' EXIT
|
||||
WRAPPED_DIR=$(run_wrap "$FIXTURE14" --config "$VALE_CONFIG" plugins)
|
||||
BARE_DIR=$(cd "$FIXTURE14" && vale --config "$VALE_CONFIG" plugins 2>&1 || true)
|
||||
if ! echo "$WRAPPED_DIR" | grep -q "VagueWording"; then
|
||||
if ! grep -q "VagueWording" <<< "$WRAPPED_DIR"; then
|
||||
fail "a directory argument produced no alert — flattening was silently skipped, the bug this test guards against"
|
||||
elif echo "$BARE_DIR" | grep -q "VagueWording"; then
|
||||
elif grep -q "VagueWording" <<< "$BARE_DIR"; then
|
||||
fail "bare vale already flags this fixture, so the test can't detect a silently-skipped flattening"
|
||||
else
|
||||
pass "a directory argument is walked and its files flattened"
|
||||
@@ -525,7 +525,7 @@ mkdir -p "$SPACED15"
|
||||
cp "$FIXTURE14/plugins/testplugin/skills/zzzskill/SKILL.md" "$SPACED15/SKILL.md"
|
||||
rm -rf "$FIXTURE14/plugins/testplugin/skills/zzzskill"
|
||||
OUT15=$(run_wrap "$FIXTURE14" --config "$VALE_CONFIG" plugins/testplugin/skills)
|
||||
if echo "$OUT15" | grep -q "zzz skill" && echo "$OUT15" | grep -q "VagueWording"; then
|
||||
if grep -q "zzz skill" <<< "$OUT15" && grep -q "VagueWording" <<< "$OUT15"; then
|
||||
pass "a file under a directory whose name contains a space is walked and flattened"
|
||||
else
|
||||
fail "a path with a space was dropped from the directory walk"
|
||||
@@ -626,7 +626,7 @@ unguarded_expansions() {
|
||||
done < <(sourced_files "$file")
|
||||
while IFS= read -r hit; do
|
||||
name="$(printf '%s\n' "$hit" \
|
||||
| grep -oE '\$\{[A-Za-z_][A-Za-z0-9_]*\[@\]\}' | head -1 \
|
||||
| { grep -oE '\$\{[A-Za-z_][A-Za-z0-9_]*\[@\]\}' || true; } | head -1 \
|
||||
| sed -E 's/^\$\{//; s/\[@\]\}$//')"
|
||||
# Shell-maintained arrays are never seeded by a `NAME=(...)` line, so the
|
||||
# seeding exemption below can never clear them: without this case they are
|
||||
@@ -653,8 +653,7 @@ unguarded_expansions() {
|
||||
PIPESTATUS|BASH_SOURCE|BASH_LINENO|BASH_VERSINFO|GROUPS|DIRSTACK) continue ;;
|
||||
esac
|
||||
# Same-line emptiness short-circuit: the expansion cannot be reached empty.
|
||||
if printf '%s\n' "$hit" \
|
||||
| grep -qE "\\\$\{#$name\[@\]\}[[:space:]]*-(eq|lt)[[:space:]]*[01][^|]*\|\|"; then
|
||||
if grep -qE "\\\$\{#$name\[@\]\}[[:space:]]*-(eq|lt)[[:space:]]*[01][^|]*\|\|" <<< "$hit"; then
|
||||
continue
|
||||
fi
|
||||
for seed_file in ${seed_files[@]+"${seed_files[@]}"}; do
|
||||
@@ -809,7 +808,7 @@ for ARGS17 in "" "--config $VALE_CONFIG" "--config $VALE_CONFIG emptydir"; do
|
||||
# shellcheck disable=SC2086 # deliberate word splitting of the argv fixture
|
||||
OUT17+="$( (cd "$FIXTURE17" && "$OLD_BASH" "$SCRIPT" $ARGS17 </dev/null 2>&1) || true)"
|
||||
done
|
||||
if echo "$OUT17" | grep -q "unbound variable"; then
|
||||
if grep -q "unbound variable" <<< "$OUT17"; then
|
||||
fail "aborted with 'unbound variable' on bash $OLD_BASH_VER — the bug this test guards against"
|
||||
else
|
||||
pass "degenerate invocations run clean under bash $OLD_BASH_VER ($OLD_BASH)"
|
||||
@@ -828,7 +827,7 @@ SPACED18="$FIXTURE18/plugins/testplugin/skills/zzz skill dir"
|
||||
mkdir -p "$SPACED18"
|
||||
mv "$FIXTURE18/plugins/testplugin/skills/zzzskill/SKILL.md" "$SPACED18/SKILL.md"
|
||||
OUT18=$(run_wrap "$FIXTURE18" --config "$VALE_CONFIG" "plugins/testplugin/skills/zzz skill dir/SKILL.md")
|
||||
if echo "$OUT18" | grep -q "zzz skill dir/SKILL.md" && echo "$OUT18" | grep -q "VagueWording"; then
|
||||
if grep -q "zzz skill dir/SKILL.md" <<< "$OUT18" && grep -q "VagueWording" <<< "$OUT18"; then
|
||||
pass "a path argument with a space is passed to vale as one word"
|
||||
else
|
||||
fail "a path argument with a space was split by the array expansion: $OUT18"
|
||||
@@ -934,7 +933,7 @@ else
|
||||
new_fixture "$DIR19"
|
||||
BARE19="$(cd "$DIR19" && vale --config "$VALE_CONFIG" "$REL_SKILL19" 2>&1 || true)"
|
||||
GOT19="$(alert_text "$(run_wrap "$DIR19" --config "$VALE_CONFIG" "$REL_SKILL19")")"
|
||||
if echo "$BARE19" | grep -q "VagueWording"; then
|
||||
if grep -q "VagueWording" <<< "$BARE19"; then
|
||||
fail "bare vale already flags the $FORM19 form, so this case can't detect a silently-skipped flattening"
|
||||
elif [[ "$GOT19" == "$BASELINE19" ]]; then
|
||||
pass "a $FORM19 multi-line description reports the same alerts as its single-line form"
|
||||
@@ -1044,7 +1043,7 @@ done
|
||||
echo ""
|
||||
echo "--- the |- literal-block fallback lints normally and preserves line numbers ---"
|
||||
OUT20B=$(run_wrap "$FIXTURE20_BLOCK" --config "$VALE_CONFIG" "$REL_SKILL19")
|
||||
if echo "$OUT20B" | grep -q "VagueWording"; then
|
||||
if grep -q "VagueWording" <<< "$OUT20B"; then
|
||||
pass "a description needing quotes with both an apostrophe and a double quote is still linted"
|
||||
else
|
||||
fail "a description needing quotes with both an apostrophe and a double quote produced no alerts"
|
||||
@@ -1053,7 +1052,7 @@ WANT20B_LINE="$(grep -n 'flattening marker phrase' "$FIXTURE20_BLOCK/$REL_SKILL1
|
||||
# `--output line` prints `file:line:col:Rule:message`, so the line number reads
|
||||
# back without any wrapping or colour to strip.
|
||||
GOT20B_LINE="$(run_wrap "$FIXTURE20_BLOCK" --config "$APOS_STYLE/.vale.ini" --output line "$REL_SKILL19" \
|
||||
| grep 'Apostrophe.Body' | head -1 | cut -d: -f2)"
|
||||
| { grep 'Apostrophe.Body' || true; } | head -1 | cut -d: -f2)"
|
||||
if [[ "$GOT20B_LINE" == "$WANT20B_LINE" ]]; then
|
||||
pass "a body line after a |- flattened description keeps its original line number ($WANT20B_LINE)"
|
||||
else
|
||||
@@ -1079,7 +1078,10 @@ if [[ "$BARE21_FILES" != "in 1 file" ]]; then
|
||||
fail "bare vale did not lint the symlinked file ($BARE21_FILES), so this case can't detect the walk dropping it"
|
||||
elif [[ "$WRAPPED21_FILES" != "$BARE21_FILES" ]]; then
|
||||
fail "the directory walk dropped a symlinked file: wrapper saw '$WRAPPED21_FILES', bare vale '$BARE21_FILES'"
|
||||
elif echo "$WRAPPED21" | grep -q "VagueWording"; then
|
||||
# A here-string, not `echo "$WRAPPED21" | grep -q`: the match sits on line 3 of
|
||||
# 8, and under pipefail grep -q exiting early can SIGPIPE echo mid-write and
|
||||
# fail this branch on correct output (see tests/test-check-release-needed.sh).
|
||||
elif grep -q "VagueWording" <<< "$WRAPPED21"; then
|
||||
pass "a symlinked file under a directory argument is mirrored, flattened and flagged"
|
||||
else
|
||||
fail "a symlinked file was mirrored but not flattened — no alert came back"
|
||||
@@ -1117,9 +1119,9 @@ RC23=$?
|
||||
set -e
|
||||
if [[ $RC23 -eq 0 ]]; then
|
||||
fail "a typo'd path exited 0 — indistinguishable from a clean run, the bug this test guards against"
|
||||
elif echo "$OUT23" | grep -q "in stdin"; then
|
||||
elif grep -q "in stdin" <<< "$OUT23"; then
|
||||
fail "a typo'd path fell back to reading stdin and reported 'in stdin' instead of erroring"
|
||||
elif echo "$OUT23" | grep -q "SKILLL.md"; then
|
||||
elif grep -q "SKILLL.md" <<< "$OUT23"; then
|
||||
pass "a typo'd path exits nonzero with a message naming the path"
|
||||
else
|
||||
fail "a typo'd path exited $RC23 but the message does not name it: $OUT23"
|
||||
@@ -1141,9 +1143,9 @@ mkdir -p "$FIXTURE24/line"
|
||||
for FORM24 in "--output line" "--output=line" "--output JSON" "--output=JSON"; do
|
||||
# shellcheck disable=SC2086 # deliberate word splitting of the argv fixture
|
||||
OUT24="$(run_wrap "$FIXTURE24" --config "$VALE_CONFIG" $FORM24 "$REL_SKILL19")"
|
||||
if echo "$OUT24" | grep -q "E100"; then
|
||||
if grep -q "E100" <<< "$OUT24"; then
|
||||
fail "'$FORM24' was rewritten to a cwd path and vale flipped into template mode — the bug this test guards against"
|
||||
elif echo "$OUT24" | grep -q "VagueWording"; then
|
||||
elif grep -q "VagueWording" <<< "$OUT24"; then
|
||||
pass "'$FORM24' is passed through as a built-in style name"
|
||||
else
|
||||
fail "'$FORM24' produced no alert: $OUT24"
|
||||
@@ -1215,12 +1217,12 @@ FOUND26="$(unguarded_expansions "$FIXTURE26/probe.sh")"
|
||||
MISSING26=""
|
||||
LEAKED26=""
|
||||
for ARR26 in $EXEMPT26; do
|
||||
if echo "$FOUND26" | grep -q "{$ARR26\[@\]}"; then
|
||||
if grep -q "{$ARR26\[@\]}" <<< "$FOUND26"; then
|
||||
LEAKED26+="$ARR26 "
|
||||
fi
|
||||
done
|
||||
for ARR26 in $FLAGGED26; do
|
||||
if ! echo "$FOUND26" | grep -q "{$ARR26\[@\]}"; then
|
||||
if ! grep -q "{$ARR26\[@\]}" <<< "$FOUND26"; then
|
||||
MISSING26+="$ARR26 "
|
||||
fi
|
||||
done
|
||||
@@ -1465,7 +1467,7 @@ matches_any_regex28() {
|
||||
[[ -n "$regexes" ]] || return 1
|
||||
while IFS= read -r re; do
|
||||
[[ -n "$re" ]] || continue
|
||||
if printf '%s\n' "$rel" | grep -Eq "$re"; then
|
||||
if grep -Eq "$re" <<< "$rel"; then
|
||||
return 0
|
||||
fi
|
||||
done <<EOF_RE28
|
||||
@@ -1499,7 +1501,7 @@ elif [[ -z "$SECTIONS28" ]]; then
|
||||
else
|
||||
while IFS= read -r SEC28; do
|
||||
[[ -n "$SEC28" ]] || continue
|
||||
if ! printf '%s\n' "$PROBE_TABLE28" | grep -qF "|$SEC28|isolating"; then
|
||||
if ! grep -qF "|$SEC28|isolating" <<< "$PROBE_TABLE28"; then
|
||||
UNPROBED28+="$SEC28 "
|
||||
fi
|
||||
done <<EOF_SEC28
|
||||
@@ -1508,7 +1510,7 @@ EOF_SEC28
|
||||
STALE_SECTIONS28=""
|
||||
while IFS= read -r TSEC28; do
|
||||
[[ -n "$TSEC28" ]] || continue
|
||||
printf '%s\n' "$SECTIONS28" | grep -qF -- "$TSEC28" || STALE_SECTIONS28+="$TSEC28 "
|
||||
grep -qF -- "$TSEC28" <<< "$SECTIONS28" || STALE_SECTIONS28+="$TSEC28 "
|
||||
done <<EOF_TSEC28
|
||||
$TABLE_SECTIONS28
|
||||
EOF_TSEC28
|
||||
@@ -1548,7 +1550,7 @@ probe_coverage28() {
|
||||
echo "FAIL|$rel|vale printed no summary line for $rel, so it is not known whether anything was scanned: ${report:-<empty>}"
|
||||
elif [[ "$count" -eq 0 ]]; then
|
||||
echo "FAIL|$rel|$sec scanned 0 files for $rel — the section's glob covers no path of that shape, so vale exits 0 and every gate reads it as a pass"
|
||||
elif ! printf '%s\n' "$report" | grep -qF "Kyberforge.VagueWording"; then
|
||||
elif ! grep -qF "Kyberforge.VagueWording" <<< "$report"; then
|
||||
echo "FAIL|$rel|$sec scanned $rel but raised no Kyberforge alert — the glob matches but the style is not loaded, which lints the file and reports nothing"
|
||||
else
|
||||
echo "PASS|$rel|$sec scans $rel ($count file) and raises a Kyberforge alert"
|
||||
@@ -1622,7 +1624,7 @@ if [[ "$VALE_READY" == true && "$PART_A_CLEAN28" == true ]]; then
|
||||
[[ -n "$REL28" && "$ISO28" == "isolating" ]] || continue
|
||||
LINE28="$(printf '%s\n' "$RESULTS28" | { grep -F "|$REL28|" || true; } | head -1)"
|
||||
if [[ "$ROWSEC28" == "$MSEC28" ]]; then
|
||||
printf '%s\n' "$LINE28" | grep -qF "but raised no Kyberforge alert" \
|
||||
grep -qF "but raised no Kyberforge alert" <<< "$LINE28" \
|
||||
|| STYLE_MUT_FAILS28+="[$MSEC28 lost Kyberforge but $REL28 did not fail as style-not-loaded: ${LINE28:-<no result>}] "
|
||||
else
|
||||
[[ "$LINE28" == PASS\|* ]] \
|
||||
@@ -1719,7 +1721,7 @@ copilot_scope_failures30() {
|
||||
[[ -n "$rel" ]] || continue
|
||||
report="$(vale_report28 "$cfg" "$TREE28" "$rel")"
|
||||
has=false
|
||||
if printf '%s\n' "$report" | grep -qF "KyberforgeCopilot.ProactivePhrase"; then has=true; fi
|
||||
if grep -qF "KyberforgeCopilot.ProactivePhrase" <<< "$report"; then has=true; fi
|
||||
case "$rel" in
|
||||
*.agent.md)
|
||||
[[ "$has" == true ]] || bad+="[$rel is an agent file but raised no ProactivePhrase alert — the Copilot style is shipped but never loaded for it, so its rules lint nothing] "
|
||||
@@ -1728,7 +1730,7 @@ copilot_scope_failures30() {
|
||||
# Non-vacuity guard: `absent` only means `scoped out` if the file was
|
||||
# scanned at all. Without the Kyberforge half, a glob that stopped
|
||||
# matching this path entirely would read as correct scoping.
|
||||
if ! printf '%s\n' "$report" | grep -qF "Kyberforge.VagueWording"; then
|
||||
if ! grep -qF "Kyberforge.VagueWording" <<< "$report"; then
|
||||
bad+="[$rel raised no Kyberforge alert either, so its missing ProactivePhrase proves nothing about scoping] "
|
||||
elif [[ "$has" == true ]]; then
|
||||
bad+="[$rel is not an .agent.md file but raised a ProactivePhrase alert — the Copilot style has leaked past the scope ADR-0013 fixes it to] "
|
||||
@@ -1789,9 +1791,9 @@ UNLOAD_FAILS30="$(copilot_scope_failures30 "$UNLOAD30/.vale.ini")"
|
||||
LEAK_FAILS30="$(copilot_scope_failures30 "$LEAK30/.vale.ini")"
|
||||
if cmp -s "$VALE_ASSETS28/.vale.ini" "$UNLOAD30/.vale.ini" || cmp -s "$VALE_ASSETS28/.vale.ini" "$LEAK30/.vale.ini"; then
|
||||
fail "a Copilot-scope mutation left the copied config unchanged, so Part B mutated nothing and proves nothing about Part A"
|
||||
elif ! printf '%s' "$UNLOAD_FAILS30" | grep -qF "[copilot/demo.agent.md is an agent file but raised no ProactivePhrase alert"; then
|
||||
elif ! grep -qF "[copilot/demo.agent.md is an agent file but raised no ProactivePhrase alert" <<< "$UNLOAD_FAILS30"; then
|
||||
fail "dropping KyberforgeCopilot from [**/*.agent.md] did not fail Part A, so an unloaded Copilot style would pass silently again: ${UNLOAD_FAILS30:-<no failure>}"
|
||||
elif ! printf '%s' "$LEAK_FAILS30" | grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is not an .agent.md file but raised a ProactivePhrase alert"; then
|
||||
elif ! grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is not an .agent.md file but raised a ProactivePhrase alert" <<< "$LEAK_FAILS30"; then
|
||||
fail "adding KyberforgeCopilot to [**/SKILL.md] did not fail Part A, so the style could leak past ADR-0013's scope unnoticed: ${LEAK_FAILS30:-<no failure>}"
|
||||
else
|
||||
pass "unloading KyberforgeCopilot from .agent.md files and leaking it onto SKILL.md files are each caught by Part A"
|
||||
@@ -1972,7 +1974,7 @@ SILENCED31="$(vale_report28 "$OVERRIDE_DIR31/.vale.ini" "$TREE28" "copilot/demo.
|
||||
SILENCED_COUNT31="$(files_scanned28 "$SILENCED31")"
|
||||
if [[ -z "$SILENCED_COUNT31" || "$SILENCED_COUNT31" -eq 0 ]]; then
|
||||
fail "the override fixture scanned no file at all, so the disappearance of the VagueWording alert proves nothing about overrides"
|
||||
elif printf '%s\n' "$SILENCED31" | grep -qF "Kyberforge.VagueWording"; then
|
||||
elif grep -qF "Kyberforge.VagueWording" <<< "$SILENCED31"; then
|
||||
# Reported as a FAIL, not a pass. It is not a defect in the config, but it
|
||||
# means Parts A and B are guarding a failure mode this vale build no longer
|
||||
# has — and a guard that guards nothing while reporting PASS is the same
|
||||
@@ -2115,13 +2117,13 @@ awk '/^[ \t]*files:/ { sub(/\^/, "^zzz-no-such-path/") } { print }' \
|
||||
"$PC_CONFIG32" > "$MUT32/.pre-commit-config.yaml"
|
||||
MUT_RECORDS32="$(hook_records28 "$MUT32/.pre-commit-config.yaml")"
|
||||
MUT_FAILS32="$(prefilter_scope_failures32 "$MUT32/.pre-commit-config.yaml" "$REPO_FILES32")"
|
||||
if ! printf '%s\n' "$MUT_RECORDS32" | grep -q 'zzz-no-such-path'; then
|
||||
if ! grep -q 'zzz-no-such-path' <<< "$MUT_RECORDS32"; then
|
||||
fail "the narrowed regexes never reached the copied config, so Part B narrowed nothing and proves nothing about Part A"
|
||||
elif [[ "$(printf '%s\n' "$MUT_RECORDS32" | grep -c .)" -ne 2 ]]; then
|
||||
fail "the mutated config did not parse back as two prefilter hooks, so any failure below would come from the parser, not from the narrowing"
|
||||
elif ! printf '%s' "$MUT_FAILS32" | grep -qF "vale-audit-prefilter-skill: 'files: ^zzz-no-such-path/"; then
|
||||
elif ! grep -qF "vale-audit-prefilter-skill: 'files: ^zzz-no-such-path/" <<< "$MUT_FAILS32"; then
|
||||
fail "narrowing the skill hook's regex to match zero files did not fail this check, so Part A cannot detect a prefilter that has been silently switched off for SKILL.md files"
|
||||
elif ! printf '%s' "$MUT_FAILS32" | grep -qF "vale-audit-prefilter-agent: 'files: ^zzz-no-such-path/"; then
|
||||
elif ! grep -qF "vale-audit-prefilter-agent: 'files: ^zzz-no-such-path/" <<< "$MUT_FAILS32"; then
|
||||
fail "narrowing the agent hook's regex to match zero files did not fail this check, so Part A cannot detect a prefilter that has been silently switched off for agent files"
|
||||
else
|
||||
pass "narrowing either hook's 'files:' regex to match zero files is caught by Part A, which is what makes its pass mean something"
|
||||
@@ -2314,15 +2316,15 @@ if ! grep -qF '^plugins/kyberforge/\.apm/skills/' "$MUT33/skill.yaml" \
|
||||
|| ! grep -qF '^plugins/kyberforge/\.apm/agents/' "$MUT33/agent.yaml" \
|
||||
|| ! grep -qF 'vale-audit-prefilter-skill-renamed' "$MUT33/id.yaml"; then
|
||||
fail "a mutation never reached its copied config, so Part B mutated nothing and proves nothing about Part A"
|
||||
elif ! printf '%s' "$SKILL_FAILS33" | grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is in scope of kyberforge-vale-audit-skill"; then
|
||||
elif ! grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is in scope of kyberforge-vale-audit-skill" <<< "$SKILL_FAILS33"; then
|
||||
fail "narrowing the local skill hook to ^plugins/kyberforge/ did not fail Part A, so the prefilter can drop every other plugin's skills with every gate green: ${SKILL_FAILS33:-<no failure>}"
|
||||
elif [[ -z "$AGREE_FAILS33" ]] && printf '%s' "$SKILL_FAILS33" | grep -qF "kyberforge-vale-audit-agent"; then
|
||||
elif [[ -z "$AGREE_FAILS33" ]] && grep -qF "kyberforge-vale-audit-agent" <<< "$SKILL_FAILS33"; then
|
||||
# Only meaningful against a clean base: when Part A already failed, the copy
|
||||
# inherits that defect, and reporting it again here would be one defect twice.
|
||||
fail "narrowing only the skill hook also reported an agent-class defect, so the comparison is not confined to its class: $SKILL_FAILS33"
|
||||
elif ! printf '%s' "$AGENT_FAILS33" | grep -qF "[plugins/demo/.apm/agents/demo.agent.md is in scope of kyberforge-vale-audit-agent"; then
|
||||
elif ! grep -qF "[plugins/demo/.apm/agents/demo.agent.md is in scope of kyberforge-vale-audit-agent" <<< "$AGENT_FAILS33"; then
|
||||
fail "narrowing the local agent hook to ^plugins/kyberforge/ did not fail Part A: ${AGENT_FAILS33:-<no failure>}"
|
||||
elif ! printf '%s' "$ID_FAILS33" | grep -qF "[no hook with id 'vale-audit-prefilter-skill' and a files: regex in id.yaml"; then
|
||||
elif ! grep -qF "[no hook with id 'vale-audit-prefilter-skill' and a files: regex in id.yaml" <<< "$ID_FAILS33"; then
|
||||
fail "renaming the local skill hook's id did not fail Part A by name, so the skill class could fall out of the comparison silently: ${ID_FAILS33:-<no failure>}"
|
||||
else
|
||||
pass "narrowing either local hook to one plugin, or renaming one, is caught by Part A"
|
||||
|
||||
Reference in New Issue
Block a user