Defame1297
384756b343
The repo's gates were not pinned to their wiring. Deleting the check-skill-version-bump block from .pre-commit-config.yaml left the whole suite green; deleting eight blocks at once, run-tests among them, also left it green. Only 4 of 20 hook ids had their wiring pinned anywhere, so a merge conflict resolved badly could stop the suite running at pre-push forever while every test still reported green. test-adr0020-contract now derives the repo-authored hooks from the repo: local entries and pins each one's id, entry and stages against an explicit expected set, both directions, with the same non-vacuity guards the file already applies to its own fixtures. Upstream hooks and their rev: values are untouched, so a rev bump does not churn the test. Mutation-checked: a removed block, a repointed entry and a hook moved off pre-push each go red; a rev bump, a comment edit and reordering stay green. 29 -> 44 assertions. run-bats computed each file's TAP plan and then discarded it, so a process printing "1..10", three ok lines and exit 0 was counted as "3 tests, 0 failures" with seven tests silently gone. That is exactly the wrapper-swallows-the-status case the runner's own comment puts in its threat model, and the plan was the only surviving signal. The plan is now enforced in both directions when a file emits exactly one. Also: test-no-pipefail-early-exit-grep's live-tree floor goes from 20 to 50 against an actual 57, matching test-vale-wrap's per-glob discipline, and test-vale-wrap's header names the real path to vale-wrap.sh. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2
885 lines
42 KiB
Bash
Executable File
885 lines
42 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Regression test for the STRUCTURAL claims the ADR-0020 gate family makes about
|
|
# itself. None of them was pinned anywhere before this file, and each one fails
|
|
# silently — which is the whole reason they need a test rather than a comment:
|
|
#
|
|
# 1. "ONE resolver, and both of its consumers run it." The ADR-0020 boundary
|
|
# resolver has exactly one copy, factory-audit's sourced
|
|
# lib-boundary-resolver.sh. scripts/skill-size-check.sh used to embed a
|
|
# second, byte-identical copy, because it was also exported through a
|
|
# published hook manifest whose consumers could not reach a file inside the
|
|
# plugin; 4de5b6b retired that export (ADR-0014), so the hook now sources
|
|
# the library too. What must not fail silently: the hook growing its own
|
|
# copy back, the library being gutted, or the hook no longer running the
|
|
# library's text at all. So this asserts the library is real content, the
|
|
# hook carries no marker pair, the hook fails closed without the library,
|
|
# and — by a sentinel planted in a copied library — that the text the hook
|
|
# executes IS the library's.
|
|
# 1a. The library is the ONLY authority, and validate.sh sources it in both
|
|
# mode branches — the same authority checks 1b makes for the parser.
|
|
# 1b. The same claim, one directory over, for the Contributing-files parser.
|
|
# That one was worse: it was embedded in both validate-provenance.sh copies,
|
|
# the agent-audit copy's docstring ASSERTED it was kept behaviourally
|
|
# identical to skill-audit's, and the two had drifted (cosmetically, at
|
|
# 484357a, re-unified at 598a7c3 — nothing had caught it). ADR-0025
|
|
# removed the second copy, so there is no longer a pair to hash — but
|
|
# deleting the assertion would restore exactly the condition that let the
|
|
# drift happen, so it is CONVERTED rather than dropped: it now pins that
|
|
# lib-contributing-files.sh is the single authority (one marker pair), that
|
|
# validate-provenance.sh actually SOURCES it, and that nobody has re-inlined
|
|
# the parser into a mode library or anywhere else in the tree.
|
|
# 2. Both interpreter preflights, in both scripts and both of the merged
|
|
# entry point's modes. python3 and PyYAML are declared HARD dependencies
|
|
# precisely so a missing one cannot turn into a vacuous pass, and the two
|
|
# are checked separately so the message names the thing to install rather
|
|
# than the wrong one. factory-audit/scripts/validate.sh detects its mode
|
|
# first and only then calls the preflight, which lives in the mode library
|
|
# it sources (lib-checks-skill.sh / lib-checks-agent.sh) — so each mode is
|
|
# probed with its own target shape, not just one of them.
|
|
# 3. `verbose: true` on the skill-size-check hook. It is the ENTIRE delivery
|
|
# mechanism for the SUGGESTION tier: pre-commit prints nothing at all for a
|
|
# passing hook, and a SUGGESTION deliberately does not fail, so dropping
|
|
# one word from the config silences the tier ADR-0020 depends on while
|
|
# every test and every hook still reports green.
|
|
# 4. The WIRING of every repo-authored hook — its id, its `entry:` and its
|
|
# `stages:`. Same failure shape as 3, one level up: the tests drive these
|
|
# scripts by path, so nothing noticed whether .pre-commit-config.yaml
|
|
# still invoked them. On a scratch copy, deleting eight local hook blocks
|
|
# at once — `run-tests` among them — left every suite green. Upstream
|
|
# hooks and every `rev:` are deliberately out of scope.
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
HOOK="$REPO_ROOT/scripts/skill-size-check.sh"
|
|
FACTORY_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit"
|
|
# ADR-0025: one auto-detecting entry point, and one sourced copy of the resolver
|
|
# behind it, which the root hook sources as well. The entry point is what the
|
|
# preflight assertions run; the library is what assertion 1 inspects.
|
|
FACTORY_VALIDATE="$FACTORY_AUDIT/scripts/validate.sh"
|
|
FACTORY_RESOLVER="$FACTORY_AUDIT/scripts/lib-boundary-resolver.sh"
|
|
PASS=0
|
|
FAIL=0
|
|
|
|
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
|
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
|
|
|
TMPDIR_T="$(mktemp -d)"
|
|
trap 'rm -rf "$TMPDIR_T"' EXIT
|
|
|
|
BEGIN_MARKER='# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER ====='
|
|
END_MARKER='# ===== END ADR-0020 SHARED BOUNDARY RESOLVER ====='
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. The resolver has one copy, and the root hook runs it
|
|
# ---------------------------------------------------------------------------
|
|
# One copy since the 2026-09-16 change: factory-audit's lib-boundary-resolver.sh.
|
|
# scripts/skill-size-check.sh sources it rather than embedding a second copy, so
|
|
# there is no pair left to hash. What replaces the hash is the set of ways a
|
|
# single sourced copy can still fail quietly.
|
|
echo ""
|
|
echo "--- the ADR-0020 resolver has one copy, and skill-size-check.sh runs it ---"
|
|
|
|
# (i) The library carries exactly one well-formed marker pair around real
|
|
# content. An unbalanced pair makes every extraction measure the wrong span, and
|
|
# an emptied-out block would still "load" while enforcing nothing. The resolver
|
|
# is ~1,060 lines; 100 is a floor low enough never to need maintenance and high
|
|
# enough that a gutted block cannot sneak past.
|
|
if [[ ! -f "$FACTORY_RESOLVER" ]]; then
|
|
fail "resolver library not found: ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
|
|
else
|
|
b="$(grep -cFx "$BEGIN_MARKER" "$FACTORY_RESOLVER" || true)"
|
|
e="$(grep -cFx "$END_MARKER" "$FACTORY_RESOLVER" || true)"
|
|
if [[ "$b" == "1" && "$e" == "1" ]]; then
|
|
pass "${FACTORY_RESOLVER#"$REPO_ROOT/"} carries exactly one BEGIN and one END marker"
|
|
span="$(sed -n "/^${BEGIN_MARKER}\$/,/^${END_MARKER}\$/p" "$FACTORY_RESOLVER" | wc -l | tr -d ' ')"
|
|
if [[ "$span" -gt 100 ]]; then
|
|
pass "the resolver span is $span lines — real content, not an empty block"
|
|
else
|
|
fail "the resolver span is only $span lines — a gutted block would load and resolve nothing"
|
|
fi
|
|
else
|
|
fail "${FACTORY_RESOLVER#"$REPO_ROOT/"} has $b BEGIN and $e END markers, expected 1 and 1"
|
|
fi
|
|
fi
|
|
|
|
# (ii) The hook carries no copy of its own. A column-0 marker line in the hook
|
|
# is the shape the old embedded copy had, and the shape a paste-back would have.
|
|
if [[ ! -f "$HOOK" ]]; then
|
|
fail "hook not found: ${HOOK#"$REPO_ROOT/"}"
|
|
else
|
|
hb="$(grep -cFx "$BEGIN_MARKER" "$HOOK" || true)"
|
|
he="$(grep -cFx "$END_MARKER" "$HOOK" || true)"
|
|
if [[ "$hb" == "0" && "$he" == "0" ]]; then
|
|
pass "${HOOK#"$REPO_ROOT/"} carries no resolver marker lines — it has not grown its own copy back"
|
|
else
|
|
fail "${HOOK#"$REPO_ROOT/"} carries $hb BEGIN and $he END marker lines — a second copy of the resolver is back in the hook"
|
|
fi
|
|
fi
|
|
|
|
# (iii) and (iv) run the hook from a scratch tree that mirrors the two paths it
|
|
# depends on, so the real library is never touched. The scratch hook is a copy
|
|
# of the real one; its library is either absent, gutted, or the real library
|
|
# with a sentinel planted inside the resolver block.
|
|
SSC_TREE="$TMPDIR_T/ssc-tree"
|
|
SSC_LIB_DIR="$SSC_TREE/plugins/kyberforge/.apm/skills/factory-audit/scripts"
|
|
mkdir -p "$SSC_TREE/scripts" "$SSC_LIB_DIR" "$TMPDIR_T/ssc-skill/probe-skill"
|
|
cp "$HOOK" "$SSC_TREE/scripts/skill-size-check.sh"
|
|
printf -- '---\nname: probe-skill\ndescription: Use when probing the resolver wiring.\nmetadata:\n version: "0.1.0"\n---\n\n## Step 1\n\nDo the thing.\n' \
|
|
> "$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
|
|
PROBE="$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
|
|
|
|
run_scratch_hook() {
|
|
local rc=0
|
|
SSC_OUT="$(bash "$SSC_TREE/scripts/skill-size-check.sh" "$PROBE" 2>&1)" || rc=$?
|
|
SSC_RC=$rc
|
|
}
|
|
|
|
# (iii) Fail closed: no library, then a library that defines nothing.
|
|
rm -f "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"boundary resolver library was not found"* ]]; then
|
|
pass "with the library missing, the hook exits $SSC_RC and names the missing library — not a vacuous pass"
|
|
else
|
|
fail "with the library missing, the hook exited $SSC_RC without naming it: $SSC_OUT"
|
|
fi
|
|
printf '# gutted\n' > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"did not define the ADR-0020 boundary resolver"* ]]; then
|
|
pass "with a library that defines no resolver, the hook exits $SSC_RC and says so"
|
|
else
|
|
fail "with a gutted library, the hook exited $SSC_RC without saying so: $SSC_OUT"
|
|
fi
|
|
|
|
# (iv) The text the hook executes IS the library's. A sentinel print planted
|
|
# just after the BEGIN marker of a copied library must appear in the hook's
|
|
# output. Without this, a hook that sourced the library but ran some other
|
|
# program would pass (i)-(iii).
|
|
SENTINEL="ADR0020-RESOLVER-SENTINEL-$$"
|
|
if [[ -f "$FACTORY_RESOLVER" ]]; then
|
|
awk -v m="$BEGIN_MARKER" -v s="$SENTINEL" '{ print } $0 == m { print "print(\"" s "\")" }' \
|
|
"$FACTORY_RESOLVER" > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
if [[ "$(grep -cF "$SENTINEL" "$SSC_LIB_DIR/lib-boundary-resolver.sh" || true)" -ne 1 ]]; then
|
|
fail "fixture check: the sentinel was not planted exactly once in the copied library — the case below would prove nothing"
|
|
else
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -eq 0 && "$SSC_OUT" == *"$SENTINEL"* ]]; then
|
|
pass "a sentinel planted in the library's resolver block runs inside the hook — the hook executes the library's text"
|
|
else
|
|
fail "the hook did not run the library's resolver text (rc=$SSC_RC, sentinel absent from output): $SSC_OUT"
|
|
fi
|
|
fi
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1a. The resolver library is the ONLY authority, and validate.sh sources it
|
|
# ---------------------------------------------------------------------------
|
|
# Assertion 1 says nothing about a copy somewhere else in the tree, and nothing
|
|
# about whether validate.sh runs the library. Assertion 1b
|
|
# pins both of those for the Contributing-files parser; the resolver is the same
|
|
# defect class and gets the same two checks:
|
|
#
|
|
# a. validate.sh actually SOURCES lib-boundary-resolver.sh, in BOTH mode
|
|
# branches — asserted inside each arm of `case "$MODE" in`, not by counting
|
|
# source lines file-wide, because a count cannot see a branch. A library
|
|
# that is identical, unique and never sourced is a copy that has quietly
|
|
# been replaced by an inline one — and assertion 1 would stay green over it.
|
|
# b. Nothing has re-inlined it. The BEGIN marker and a def unique to the
|
|
# resolver (`_authoring_root`) appear in exactly one file,
|
|
# lib-boundary-resolver.sh, and nowhere else under the tree. A mode library
|
|
# or a root script that grows a "just this once" copy would otherwise
|
|
# escape assertion 1 entirely, because 1 inspects only the files it names.
|
|
echo ""
|
|
echo "--- the ADR-0020 resolver has exactly one authority, and validate.sh sources it ---"
|
|
|
|
# Deployed and vendored trees are generated copies, not authorities: .claude/ is
|
|
# apm install output, apm_modules/ is resolved dependencies, build/ is release
|
|
# artifacts. This file is excluded because it necessarily quotes what it
|
|
# searches for. Markdown is excluded because an authority is code that runs:
|
|
# ADR-0025 and gates.md quote these needles to document this very check, and a
|
|
# prose mention is not a re-inlined copy.
|
|
SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
|
|
tree_scan() {
|
|
grep -rlF --binary-files=without-match \
|
|
--exclude-dir=.git --exclude-dir=build --exclude-dir=.claude \
|
|
--exclude-dir=apm_modules --exclude-dir=node_modules --exclude='*.md' \
|
|
-- "$1" "$REPO_ROOT" 2>/dev/null | grep -vFx "$SELF" | sort || true
|
|
}
|
|
|
|
# (a) Sourced, once per mode branch — asserted PER BRANCH, not by counting.
|
|
#
|
|
# This used to be a file-wide `grep -c ... >= 2`, and a count cannot see a
|
|
# branch. Proven by mutation: moving the agent arm's source line out and
|
|
# duplicating the skill arm's leaves the count at 2 and the old assertion
|
|
# printed PASS while claiming "in both mode branches" — so a branch that lost
|
|
# its resolver gracefully (a defaulted variable, a `set +u` region, an optional
|
|
# resolver) stayed green and lying. The claim is per-branch, so the check is
|
|
# too: each arm of `case "$MODE" in` must source the resolver inside its own
|
|
# body. The mutation that fooled the old form is run against the new one below,
|
|
# because an assertion about branches that has never been shown to fail on a
|
|
# count-preserving branch edit is exactly the assertion that was here before.
|
|
RESOLVER_SOURCE_RE='^[[:space:]]*(\.|source)[[:space:]]+.*lib-boundary-resolver\.sh'
|
|
|
|
# The body of one arm of the `case "$MODE" in` block: everything between the
|
|
# arm's label and its `;;`. Structural, not positional — the file is edited
|
|
# often and line numbers or a whole-file hash would pin the wrong thing.
|
|
mode_arm_body() {
|
|
awk -v arm="$2" '
|
|
!incase && $0 ~ /^[[:space:]]*case[[:space:]]+"?\$MODE"?[[:space:]]+in[[:space:]]*$/ { incase = 1; next }
|
|
incase && $0 ~ /^[[:space:]]*esac([[:space:]]|$)/ { incase = 0; next }
|
|
incase && !inarm && $0 ~ "^[[:space:]]*\\(?" arm "\\)[[:space:]]*$" { inarm = 1; next }
|
|
inarm && $0 ~ /^[[:space:]]*;;[[:space:]]*$/ { inarm = 0; next }
|
|
inarm { print }
|
|
' "$1"
|
|
}
|
|
|
|
# Every mode arm sources the resolver exactly once, inside its own body.
|
|
# Returns 0/1 and leaves the reason in ARM_DETAIL, so the same function can be
|
|
# run against the real file and against the mutant below.
|
|
ARM_DETAIL=""
|
|
check_resolver_per_arm() {
|
|
local file="$1" arm body n rc=0
|
|
ARM_DETAIL=""
|
|
for arm in skill agent; do
|
|
body="$(mode_arm_body "$file" "$arm")"
|
|
if [[ -z "$body" ]]; then
|
|
ARM_DETAIL+="the $arm) arm of the case \"\$MODE\" block was not found or is empty; "
|
|
rc=1
|
|
continue
|
|
fi
|
|
n="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$body" || true)"
|
|
if [[ "$n" -eq 0 ]]; then
|
|
ARM_DETAIL+="the $arm) arm never sources lib-boundary-resolver.sh, so that mode runs some other resolver or none; "
|
|
rc=1
|
|
elif [[ "$n" -ne 1 ]]; then
|
|
ARM_DETAIL+="the $arm) arm sources lib-boundary-resolver.sh $n times; "
|
|
rc=1
|
|
fi
|
|
done
|
|
return $rc
|
|
}
|
|
|
|
if [[ ! -f "$FACTORY_VALIDATE" ]]; then
|
|
fail "entry point not found: ${FACTORY_VALIDATE#"$REPO_ROOT/"}"
|
|
else
|
|
RESOLVER_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$FACTORY_VALIDATE" || true)"
|
|
if check_resolver_per_arm "$FACTORY_VALIDATE"; then
|
|
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"}: the skill) and agent) arms of its case \"\$MODE\" block EACH source lib-boundary-resolver.sh inside their own body, exactly once"
|
|
else
|
|
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} does not source lib-boundary-resolver.sh once per mode arm: ${ARM_DETAIL%; }"
|
|
fi
|
|
# Secondary, and deliberately not the verdict: with one source per arm proven
|
|
# above, a file-wide total of exactly 2 says there is no third source line
|
|
# sitting outside both arms.
|
|
if [[ "$RESOLVER_SOURCES" -eq 2 ]]; then
|
|
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries exactly 2 resolver source lines file-wide — the two arm sources and nothing else"
|
|
else
|
|
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries $RESOLVER_SOURCES resolver source lines file-wide, expected the 2 that belong to the mode arms"
|
|
fi
|
|
|
|
# Mutation self-test. The mutation is the one the retired count could not
|
|
# see: the agent arm's source line is removed and the skill arm's duplicated,
|
|
# so the FILE-WIDE COUNT IS UNCHANGED. Written into a copy; the real file is
|
|
# never touched.
|
|
MUT_DIR="$(mktemp -d "$TMPDIR_T/resolver-mutant.XXXXXX")"
|
|
MUT="$MUT_DIR/validate.sh"
|
|
cp "$FACTORY_VALIDATE" "$MUT"
|
|
if python3 - "$MUT" <<'PY'
|
|
import re
|
|
import sys
|
|
|
|
path = sys.argv[1]
|
|
with open(path, encoding='utf-8') as fh:
|
|
lines = fh.read().split('\n')
|
|
|
|
case_re = re.compile(r'^\s*case\s+"?\$MODE"?\s+in\s*$')
|
|
esac_re = re.compile(r'^\s*esac(\s|$)')
|
|
term_re = re.compile(r'^\s*;;\s*$')
|
|
src_re = re.compile(r'^\s*(\.|source)\s+.*lib-boundary-resolver\.sh')
|
|
|
|
starts = [i for i, l in enumerate(lines) if case_re.match(l)]
|
|
assert len(starts) == 1, 'expected exactly one `case "$MODE" in`, found %d' % len(starts)
|
|
ci = starts[0]
|
|
ends = [i for i in range(ci + 1, len(lines)) if esac_re.match(lines[i])]
|
|
assert ends, 'the case "$MODE" block has no esac'
|
|
ei = ends[0]
|
|
|
|
|
|
def arm_sources(name):
|
|
for i in range(ci + 1, ei):
|
|
if re.match(r'^\s*\(?%s\)\s*$' % name, lines[i]):
|
|
for j in range(i + 1, ei):
|
|
if term_re.match(lines[j]):
|
|
return [k for k in range(i + 1, j) if src_re.match(lines[k])]
|
|
raise AssertionError('the %s) arm has no ;;' % name)
|
|
raise AssertionError('no %s) arm in the case "$MODE" block' % name)
|
|
|
|
|
|
skill = arm_sources('skill')
|
|
agent = arm_sources('agent')
|
|
assert len(skill) == 1 and len(agent) == 1, \
|
|
'expected one resolver source per arm before mutating, got skill=%d agent=%d' % (len(skill), len(agent))
|
|
|
|
out = list(lines)
|
|
del out[agent[0]] # the agent arm loses its resolver ...
|
|
out.insert(skill[0], lines[skill[0]]) # ... and the skill arm gains a duplicate
|
|
with open(path, 'w', encoding='utf-8') as fh:
|
|
fh.write('\n'.join(out))
|
|
PY
|
|
then
|
|
MUT_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$MUT" || true)"
|
|
MUT_AGENT="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$(mode_arm_body "$MUT" agent)" || true)"
|
|
# Guard the fixture before trusting its verdict: the mutation must have
|
|
# actually emptied the agent arm AND left the file-wide count where it was,
|
|
# or the case below proves nothing about the defect it stands for.
|
|
if [[ "$MUT_SOURCES" -eq "$RESOLVER_SOURCES" && "$MUT_AGENT" -eq 0 ]]; then
|
|
pass "fixture check: the mutant's agent arm sources no resolver while the file-wide count is still $MUT_SOURCES — the retired 'count >= 2' assertion would have passed it"
|
|
else
|
|
fail "the resolver mutation did not land as intended (file-wide $MUT_SOURCES vs $RESOLVER_SOURCES, agent arm $MUT_AGENT) — the case below would prove nothing"
|
|
fi
|
|
if check_resolver_per_arm "$MUT"; then
|
|
fail "the per-arm check PASSED a validate.sh whose agent arm has no resolver source — it is still counting, not reading branches"
|
|
else
|
|
pass "the per-arm check FAILS the count-preserving mutant (${ARM_DETAIL%; }) — it reads the branches, not a total"
|
|
fi
|
|
else
|
|
fail "could not build the resolver mutation fixture — validate.sh's case \"\$MODE\" structure is not the shape this self-test knows, so the per-arm check is unproven"
|
|
fi
|
|
fi
|
|
|
|
# (b) Exactly the one authority, for both spellings of a copy.
|
|
EXPECTED_RESOLVERS="$FACTORY_RESOLVER"
|
|
check_resolver_authorities() {
|
|
local label="$1" needle="$2"
|
|
local found
|
|
found="$(tree_scan "$needle")"
|
|
if [[ "$found" == "$EXPECTED_RESOLVERS" ]]; then
|
|
pass "$label appears in exactly the one resolver authority and nowhere else"
|
|
elif [[ -z "$found" ]]; then
|
|
fail "$label was found in NO file at all — the scan is looking for the wrong text"
|
|
else
|
|
fail "$label appears in an unexpected set of files, so the resolver has been re-inlined or lost: $(echo "$found" | tr '\n' ' ')— expected exactly ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
|
|
fi
|
|
}
|
|
check_resolver_authorities "the resolver's BEGIN marker" "$BEGIN_MARKER"
|
|
check_resolver_authorities "a 'def _authoring_root' definition" "def _authoring_root("
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1b. The Contributing-files parser has exactly ONE authority, and it is sourced
|
|
# ---------------------------------------------------------------------------
|
|
# Same defect class, one directory over. parse_contributing_files() used to be
|
|
# embedded in both validate-provenance.sh copies for the same reason the resolver
|
|
# was once embedded in several scripts, and until this assertion existed the agent-audit copy's
|
|
# docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's
|
|
# copy" — an invariant nothing checked, and the two did drift into different
|
|
# spellings of the bullet loop at 484357a. That drift happened to be
|
|
# behaviour-neutral and was re-unified by hand at 598a7c3; the next one need
|
|
# not be. The parser decides whether checks 4,
|
|
# 5 and 8 run at all, so a one-sided edit disables a check in one script while
|
|
# every other test stays green.
|
|
#
|
|
# ADR-0025 merged the two skills, so there is now ONE copy and nothing left to
|
|
# hash against. That does NOT retire the assertion: a byte-identity check over a
|
|
# single copy is vacuous, and deleting it outright would restore exactly the
|
|
# condition that allowed the original drift — a parser with no pinned authority.
|
|
# So the claim is CONVERTED, not dropped. It is the same claim ("the parser has
|
|
# exactly one authority") stated against the new structure:
|
|
#
|
|
# a. factory-audit/scripts/lib-contributing-files.sh exists and carries exactly
|
|
# one BEGIN/END marker pair, over a span of real content.
|
|
# b. validate-provenance.sh actually SOURCES it. A library nobody sources is a
|
|
# copy that has silently been replaced by an inline one somewhere else.
|
|
# c. Nothing has re-inlined it. No other file in the tree carries the marker
|
|
# pair, and no other file defines parse_contributing_files. This is the part
|
|
# that fails if the merge is ever partially reverted, or if a mode library
|
|
# grows its own "just this once" copy — which is precisely how the drift
|
|
# this assertion was written for got in.
|
|
echo ""
|
|
echo "--- the Contributing-files parser has exactly one authority, and validate-provenance.sh sources it ---"
|
|
|
|
CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ====='
|
|
CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER ====='
|
|
CF_LIB="$FACTORY_AUDIT/scripts/lib-contributing-files.sh"
|
|
FACTORY_PROV="$FACTORY_AUDIT/scripts/validate-provenance.sh"
|
|
|
|
# (a) One library, one well-formed marker pair, over real content.
|
|
CF_MARKERS_OK=true
|
|
if [[ ! -f "$CF_LIB" ]]; then
|
|
fail "the single parser authority is missing: ${CF_LIB#"$REPO_ROOT/"}"
|
|
CF_MARKERS_OK=false
|
|
else
|
|
b="$(grep -cFx "$CF_BEGIN" "$CF_LIB" || true)"
|
|
e="$(grep -cFx "$CF_END" "$CF_LIB" || true)"
|
|
if [[ "$b" == "1" && "$e" == "1" ]]; then
|
|
pass "${CF_LIB#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker"
|
|
else
|
|
fail "${CF_LIB#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1"
|
|
CF_MARKERS_OK=false
|
|
fi
|
|
fi
|
|
|
|
if ! $CF_MARKERS_OK; then
|
|
fail "skipping the parser content check — the marker pair is not well-formed, so any extraction would measure the wrong span"
|
|
else
|
|
# A span gutted down to its docstring would still satisfy every structural
|
|
# check above while enforcing nothing, exactly as for the resolver. The parser
|
|
# block is ~93 lines; 40 is a floor low enough never to need maintenance and
|
|
# high enough that a gutted block cannot sneak past.
|
|
CF_LINECOUNT="$(sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$CF_LIB" | wc -l | tr -d ' ')"
|
|
if [[ "$CF_LINECOUNT" -gt 40 ]]; then
|
|
pass "the extracted parser block is $CF_LINECOUNT lines — a real parser, not an empty or docstring-only span"
|
|
else
|
|
fail "the extracted parser block is only $CF_LINECOUNT lines — a gutted span asserts nothing"
|
|
fi
|
|
fi
|
|
|
|
# (b) The one entry point sources it. Without this, (a) and (c) are satisfied by
|
|
# a library that is present, unique and entirely unused.
|
|
if [[ ! -f "$FACTORY_PROV" ]]; then
|
|
fail "entry point not found: ${FACTORY_PROV#"$REPO_ROOT/"}"
|
|
elif grep -Eq '^[[:space:]]*(\.|source)[[:space:]]+.*lib-contributing-files\.sh' "$FACTORY_PROV"; then
|
|
pass "${FACTORY_PROV#"$REPO_ROOT/"} sources lib-contributing-files.sh — the single copy is the one that actually runs"
|
|
else
|
|
fail "${FACTORY_PROV#"$REPO_ROOT/"} never sources lib-contributing-files.sh — the library is dead code and the parser that runs is some other copy"
|
|
fi
|
|
|
|
# (c) Nobody re-inlined it. Both spellings are scanned: the marker pair (a
|
|
# copy-paste of the block) and a second `def parse_contributing_files` (a
|
|
# re-implementation that skipped the markers). tree_scan (defined in 1a)
|
|
# excludes the same generated trees and this file.
|
|
cf_scan() { tree_scan "$1"; }
|
|
check_sole_authority() {
|
|
local label="$1" needle="$2"
|
|
local found extra
|
|
found="$(cf_scan "$needle")"
|
|
extra="$(printf '%s\n' "$found" | grep -vFx "$CF_LIB" | grep -v '^$' || true)"
|
|
if [[ -z "$found" ]]; then
|
|
fail "$label was found in NO file at all — the parser authority has vanished, or the scan is looking for the wrong text"
|
|
elif [[ -n "$extra" ]]; then
|
|
fail "$label appears outside the single authority, so the parser has been re-inlined: $(echo "$extra" | tr '\n' ' ')— delete the copy and source lib-contributing-files.sh instead"
|
|
else
|
|
pass "$label appears only in ${CF_LIB#"$REPO_ROOT/"} — one authority, no re-inlined copies"
|
|
fi
|
|
}
|
|
check_sole_authority "the parser's BEGIN marker" "$CF_BEGIN"
|
|
check_sole_authority "a 'def parse_contributing_files' definition" "def parse_contributing_files("
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. Both interpreter preflights, in both scripts and both modes
|
|
# ---------------------------------------------------------------------------
|
|
# The two are checked separately on purpose: `python3 -c 'import yaml'` fails
|
|
# identically whether python3 is missing or PyYAML is, and naming the wrong one
|
|
# sends the reader to install the wrong thing.
|
|
#
|
|
# ADR-0025 merged the two validators into one auto-detecting entry point, but
|
|
# the preflight did NOT merge with them: validate.sh detects the mode first and
|
|
# then calls kyberforge_skill_preflight or kyberforge_agent_preflight from the
|
|
# mode library it sources. There are still two preflights, so both are still
|
|
# probed — once with a skill target and once with an agent target. Collapsing
|
|
# these to a single probe would leave one mode's preflight unpinned, and a mode
|
|
# whose preflight is gone reports a vacuous pass on a machine with no PyYAML.
|
|
REAL_PYTHON="$(command -v python3)"
|
|
# Absolute path, deliberately. The no-python3 fixture below replaces PATH
|
|
# wholesale, so a bare `bash` (or `/usr/bin/env bash`) would be resolved against
|
|
# that stripped PATH and die with "No such file or directory" before the script
|
|
# under test ever starts -- a 127 that looks like the preflight firing.
|
|
BASH_BIN="$(command -v bash)"
|
|
|
|
# A PATH that genuinely has no python3 on it. Built by symlinking the handful of
|
|
# binaries the three scripts touch before their own preflight rather than by
|
|
# hiding python3 from a full PATH, because there is no portable way to subtract
|
|
# one entry from a directory. `bash` is invoked by absolute path below so the
|
|
# interpreter itself does not have to be on this PATH.
|
|
NOPY_BIN="$TMPDIR_T/nopython-bin"
|
|
mkdir -p "$NOPY_BIN"
|
|
for b in awk cat cut dirname basename grep sed pwd rm mkdir tr; do
|
|
src="$(command -v "$b" 2>/dev/null || true)"
|
|
[[ -n "$src" ]] && ln -sf "$src" "$NOPY_BIN/$b"
|
|
done
|
|
|
|
# A python3 that runs but cannot import yaml. A shim on PATH re-execs the real
|
|
# interpreter with a PYTHONPATH entry holding a `yaml` module that raises on
|
|
# import; PYTHONPATH precedes site-packages on sys.path, so it shadows a real
|
|
# PyYAML install without touching it.
|
|
SHADOW="$TMPDIR_T/shadow"
|
|
mkdir -p "$SHADOW"
|
|
printf 'raise ImportError("PyYAML deliberately unavailable in this fixture")\n' \
|
|
> "$SHADOW/yaml.py"
|
|
NOYAML_BIN="$TMPDIR_T/noyaml-bin"
|
|
mkdir -p "$NOYAML_BIN"
|
|
cat > "$NOYAML_BIN/python3" <<EOF
|
|
#!/bin/sh
|
|
PYTHONPATH="$SHADOW\${PYTHONPATH:+:\$PYTHONPATH}" exec "$REAL_PYTHON" "\$@"
|
|
EOF
|
|
chmod +x "$NOYAML_BIN/python3"
|
|
|
|
# Sanity-check the two fixtures themselves before trusting any verdict they
|
|
# produce. A shim that silently still imports yaml would make every PyYAML
|
|
# assertion below pass for the wrong reason.
|
|
if PATH="$NOYAML_BIN:$PATH" python3 -c 'import yaml' 2>/dev/null; then
|
|
fail "the no-PyYAML shim does not actually shadow PyYAML — every PyYAML assertion below would be vacuous"
|
|
else
|
|
pass "fixture check: the no-PyYAML shim makes 'import yaml' fail while python3 still runs"
|
|
fi
|
|
if PATH="$NOPY_BIN" command -v python3 > /dev/null 2>&1; then
|
|
fail "the no-python3 PATH still resolves python3 — every python3 assertion below would be vacuous"
|
|
else
|
|
pass "fixture check: the no-python3 PATH resolves no python3"
|
|
fi
|
|
|
|
# A minimal, entirely clean subject for each script. The preflight must fire
|
|
# before any measurement, so the subject's own content is irrelevant — which is
|
|
# exactly what makes a clean one the right choice: nothing else can produce the
|
|
# non-zero exit these cases assert.
|
|
SUBJECT_SKILL_DIR="$TMPDIR_T/subject/my-skill"
|
|
mkdir -p "$SUBJECT_SKILL_DIR"
|
|
cat > "$SUBJECT_SKILL_DIR/SKILL.md" <<'EOF'
|
|
---
|
|
name: my-skill
|
|
description: A short valid description. Do not use for anything else.
|
|
metadata:
|
|
version: "1.0.0"
|
|
---
|
|
|
|
Do the thing.
|
|
EOF
|
|
SUBJECT_AGENT_ROOT="$TMPDIR_T/subject-agent"
|
|
mkdir -p "$SUBJECT_AGENT_ROOT/.apm/agents"
|
|
cat > "$SUBJECT_AGENT_ROOT/apm.yml" <<'EOF'
|
|
name: test-package
|
|
version: 0.1.0
|
|
type: skill
|
|
EOF
|
|
cat > "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" <<'EOF'
|
|
---
|
|
name: my-agent
|
|
description: A short valid description. Do not use for anything else.
|
|
---
|
|
|
|
You are a test agent. When invoked, do the thing.
|
|
EOF
|
|
|
|
# probe_preflight <label> <env-kind: nopython|noyaml> <expect-needle> <cmd...>
|
|
probe_preflight() {
|
|
local label="$1" kind="$2" needle="$3"
|
|
shift 3
|
|
local out status=0
|
|
set +e
|
|
if [[ "$kind" == nopython ]]; then
|
|
out="$(env -i PATH="$NOPY_BIN" HOME="$HOME" "$BASH_BIN" "$@" 2>&1)"
|
|
else
|
|
out="$(env PATH="$NOYAML_BIN:$PATH" "$BASH_BIN" "$@" 2>&1)"
|
|
fi
|
|
status=$?
|
|
set -e
|
|
if [[ $status -eq 0 ]]; then
|
|
fail "$label exited 0 — a missing hard dependency became a vacuous pass (output: ${out:-<empty>})"
|
|
elif [[ "$out" != *"$needle"* ]]; then
|
|
# The needle is the DIAGNOSTIC ("python3 is required"), not the bare word.
|
|
# Deleting the preflight entirely would still produce a non-zero exit and a
|
|
# message mentioning python3 -- bash's own "python3: command not found" --
|
|
# so a bare-word needle would go green on a script with no preflight at all.
|
|
fail "$label exited $status but never produced the '$needle' diagnostic (output: ${out:-<empty>})"
|
|
elif [[ "$kind" == nopython && "$out" == *PyYAML* ]]; then
|
|
fail "$label reported PyYAML when python3 itself is missing — that sends the reader to install the wrong thing (output: $out)"
|
|
else
|
|
pass "$label"
|
|
fi
|
|
}
|
|
|
|
echo ""
|
|
echo "--- a PATH with no python3 is a hard failure in both scripts and both modes, naming python3 ---"
|
|
probe_preflight "scripts/skill-size-check.sh reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
|
|
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
|
|
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
|
|
|
|
echo ""
|
|
echo "--- a python3 that cannot import yaml is a hard failure in both scripts and both modes, naming PyYAML ---"
|
|
probe_preflight "scripts/skill-size-check.sh reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
|
|
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
|
|
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
|
|
|
|
# The control. Without it, "fails when the dependency is missing" is satisfied by
|
|
# a script that fails unconditionally, and the two cases above would be green on
|
|
# a gate that never runs at all.
|
|
echo ""
|
|
echo "--- control: with both dependencies present the same subjects pass ---"
|
|
# The label is carried explicitly because the two validate.sh probes now name the
|
|
# same script and differ only in the mode its target selects.
|
|
for probe in "scripts/skill-size-check.sh|$HOOK|$SUBJECT_SKILL_DIR/SKILL.md" \
|
|
"factory-audit/scripts/validate.sh (skill mode)|$FACTORY_VALIDATE|$SUBJECT_SKILL_DIR" \
|
|
"factory-audit/scripts/validate.sh (agent mode)|$FACTORY_VALIDATE|$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"; do
|
|
IFS='|' read -r label script arg <<< "$probe"
|
|
set +e
|
|
ctl_out="$(bash "$script" "$arg" 2>&1)"
|
|
ctl_rc=$?
|
|
set -e
|
|
if [[ $ctl_rc -eq 0 ]]; then
|
|
pass "$label exits 0 on a clean subject with python3 and PyYAML available"
|
|
else
|
|
fail "$label failed a clean subject (exit $ctl_rc): $ctl_out"
|
|
fi
|
|
done
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. verbose: true on the skill-size-check hook
|
|
# ---------------------------------------------------------------------------
|
|
# pre-commit prints nothing for a passing hook, so dropping the flag silences
|
|
# the SUGGESTION tier without failing anything. The published
|
|
# .pre-commit-hooks.yaml that once carried a second copy of this hook was
|
|
# retired (ADR-0014, 2026-09-16 amendment); if it returns, assert it here too.
|
|
echo ""
|
|
echo "--- the skill-size-check hook declares verbose: true ---"
|
|
VERBOSE_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
|
|
import os
|
|
import sys
|
|
|
|
import yaml
|
|
|
|
root = sys.argv[1]
|
|
|
|
|
|
def emit(status, msg):
|
|
print("%s\t%s" % (status, msg))
|
|
|
|
|
|
# Repo config: nested repos[].hooks[].
|
|
path = os.path.join(root, '.pre-commit-config.yaml')
|
|
try:
|
|
with open(path, encoding='utf-8') as fh:
|
|
cfg = yaml.safe_load(fh) or {}
|
|
except Exception as exc:
|
|
emit('FAIL', '.pre-commit-config.yaml did not parse: %s' % exc)
|
|
cfg = {}
|
|
found = None
|
|
for repo in cfg.get('repos') or []:
|
|
for hook in (repo.get('hooks') or []):
|
|
if hook.get('id') == 'skill-size-check':
|
|
found = hook
|
|
if found is None:
|
|
emit('FAIL', '.pre-commit-config.yaml declares no hook with id skill-size-check')
|
|
elif found.get('verbose') is True:
|
|
emit('PASS', '.pre-commit-config.yaml: skill-size-check is verbose: true')
|
|
else:
|
|
emit('FAIL', '.pre-commit-config.yaml: skill-size-check has verbose=%r — '
|
|
'pre-commit prints nothing for a passing hook, so every '
|
|
'ADR-0020 SUGGESTION is swallowed' % (found.get('verbose'),))
|
|
PY
|
|
)"
|
|
while IFS=$'\t' read -r status msg; do
|
|
[[ -n "$status" ]] || continue
|
|
if [[ "$status" == PASS ]]; then
|
|
pass "$msg"
|
|
else
|
|
fail "$msg"
|
|
fi
|
|
done <<< "$VERBOSE_REPORT"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. Every repo-authored hook is still WIRED, at the stage it claims
|
|
# ---------------------------------------------------------------------------
|
|
# Same failure mode as assertion 3, one level up. Every gate this repo owns is
|
|
# driven in tests by PATH -- tests/test-skill-version-bump.sh runs
|
|
# scripts/check-skill-version-bump.sh directly -- so the script staying correct
|
|
# and the hook still existing are independent facts, and only the first one was
|
|
# pinned. Deleting the `run-tests` block from .pre-commit-config.yaml (a routine
|
|
# merge-conflict casualty) stops the entire suite from running at pre-push
|
|
# forever, and `bash tests/run-tests.sh --strict` still prints all green: the
|
|
# suite cannot notice that nothing invokes it. Verified on a scratch copy --
|
|
# eight repo-authored hook blocks were deleted at once and every suite stayed
|
|
# green.
|
|
#
|
|
# So: id + entry + stages, for the repo-authored hooks only. "Repo-authored"
|
|
# is derived from the config (every hook under a `repo: local` entry), not
|
|
# hardcoded, and the derived set is then compared against the expected list
|
|
# below -- a new local hook that nobody pinned is itself a failure.
|
|
#
|
|
# Deliberately NOT pinned: the stock upstream hooks (gitleaks, check-yaml,
|
|
# pretty-format-json, shellcheck, conventional-pre-commit, the `repo: meta`
|
|
# pair) and every `rev:`. Those are somebody else's contract; a rev bump must
|
|
# not churn this test.
|
|
echo ""
|
|
echo "--- every repo-authored hook is wired, with the entry and stages it claims ---"
|
|
WIRING_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
|
|
import os
|
|
import re
|
|
import sys
|
|
|
|
import yaml
|
|
|
|
root = sys.argv[1]
|
|
|
|
|
|
def emit(status, msg):
|
|
print("%s\t%s" % (status, msg))
|
|
|
|
|
|
# The expected wiring of every hook this repo authors: id -> (entry, stages).
|
|
# ADDING OR REMOVING A REPO-AUTHORED HOOK DELIBERATELY REQUIRES EDITING THIS
|
|
# LIST. That is the point -- the list is the second party to the agreement, so a
|
|
# hook cannot leave .pre-commit-config.yaml without someone saying so here.
|
|
# Upstream hooks are absent on purpose and must stay absent.
|
|
EXPECTED = {
|
|
# pre-push
|
|
'run-tests': (
|
|
'bash tests/run-tests.sh --strict', ['pre-push']),
|
|
'check-executables-allow-sync': (
|
|
'bash scripts/check-executables-allow-sync.sh', ['pre-push']),
|
|
'apm-audit-ci': (
|
|
'bash -c \'for d in . plugins/*/; do (cd "$d" && apm audit --ci) || '
|
|
'{ echo "apm audit --ci failed in $d" >&2; exit 1; }; done\'',
|
|
['pre-push']),
|
|
'check-apm-agents-valid': (
|
|
'bash scripts/check-apm-agents-valid.sh', ['pre-push']),
|
|
'apm-pack-check-clean': (
|
|
'apm pack --check-versions --check-clean --dry-run', ['pre-push']),
|
|
'check-scope-walkup-sync': (
|
|
'bash scripts/check-scope-walkup-sync.sh', ['pre-push']),
|
|
'check-skill-version-bump': (
|
|
'bash scripts/check-skill-version-bump.sh', ['pre-push']),
|
|
'validate-marketplace': (
|
|
'claude plugin validate --strict .claude-plugin/marketplace.json',
|
|
['pre-push']),
|
|
# pre-commit
|
|
'skill-size-check': (
|
|
'scripts/skill-size-check.sh', ['pre-commit']),
|
|
'check-rtk-prefix': (
|
|
'scripts/check-rtk-prefix.sh', ['pre-commit']),
|
|
'vale-audit-prefilter-skill': (
|
|
'plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh',
|
|
['pre-commit']),
|
|
'vale-audit-prefilter-agent': (
|
|
'plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh',
|
|
['pre-commit']),
|
|
}
|
|
|
|
path = os.path.join(root, '.pre-commit-config.yaml')
|
|
try:
|
|
with open(path, encoding='utf-8') as fh:
|
|
cfg = yaml.safe_load(fh) or {}
|
|
except Exception as exc:
|
|
emit('FAIL', '.pre-commit-config.yaml did not parse: %s' % exc)
|
|
sys.exit(0)
|
|
|
|
local = {}
|
|
duplicates = []
|
|
for repo in cfg.get('repos') or []:
|
|
if repo.get('repo') != 'local':
|
|
continue
|
|
for hook in (repo.get('hooks') or []):
|
|
hid = hook.get('id')
|
|
if hid in local:
|
|
duplicates.append(hid)
|
|
local[hid] = hook
|
|
|
|
# Non-vacuity, first. Every assertion below iterates the derived set, so an
|
|
# empty one would report nothing and the section would pass having checked
|
|
# nothing at all -- the same shape of silent hole this whole file exists to
|
|
# close.
|
|
if not EXPECTED:
|
|
emit('FAIL', 'the expected repo-authored hook list is empty — '
|
|
'every wiring assertion below is vacuous')
|
|
sys.exit(0)
|
|
if not local:
|
|
emit('FAIL', '.pre-commit-config.yaml declares no `repo: local` hooks at '
|
|
'all — every gate this repo authors has been unwired, or the '
|
|
'config moved and this assertion is now measuring nothing')
|
|
sys.exit(0)
|
|
emit('PASS', '.pre-commit-config.yaml declares %d repo-authored (`repo: local`) '
|
|
'hooks — the wiring assertions below are not vacuous' % len(local))
|
|
|
|
if duplicates:
|
|
emit('FAIL', '.pre-commit-config.yaml declares duplicate local hook ids '
|
|
'(%s) — pre-commit runs one of them and the other is dead '
|
|
'config' % ', '.join(sorted(set(duplicates))))
|
|
|
|
# The set, before the per-hook detail: a deleted block shows up here as a
|
|
# missing id even if nothing else in the file changed.
|
|
missing = sorted(set(EXPECTED) - set(local))
|
|
extra = sorted(set(local) - set(EXPECTED))
|
|
if missing or extra:
|
|
parts = []
|
|
if missing:
|
|
parts.append('NOT WIRED in .pre-commit-config.yaml: %s' % ', '.join(missing))
|
|
if extra:
|
|
parts.append('wired but not pinned in this test: %s' % ', '.join(extra))
|
|
emit('FAIL', 'the repo-authored hook set has drifted — %s. A hook that '
|
|
'leaves the config stops running at push time while every '
|
|
'test stays green; a hook that arrives unpinned can leave '
|
|
'again unnoticed. Fix the config, or update EXPECTED in '
|
|
'tests/test-adr0020-contract.sh deliberately.' % '; '.join(parts))
|
|
else:
|
|
emit('PASS', '.pre-commit-config.yaml wires exactly the %d expected '
|
|
'repo-authored hooks, no more and no fewer' % len(EXPECTED))
|
|
|
|
# Per hook: the entry that runs and the stage it runs at. Both are single
|
|
# tokens whose loss is invisible -- an entry repointed at a path that no longer
|
|
# exists makes pre-commit fail loudly, but an entry repointed at a DIFFERENT
|
|
# real script does not, and a hook moved off pre-push simply never fires.
|
|
entry_paths = []
|
|
for hid in sorted(EXPECTED):
|
|
hook = local.get(hid)
|
|
if hook is None:
|
|
continue # already reported as missing above
|
|
exp_entry, exp_stages = EXPECTED[hid]
|
|
got_entry = hook.get('entry')
|
|
got_stages = hook.get('stages')
|
|
problems = []
|
|
if got_entry != exp_entry:
|
|
problems.append('entry is %r, expected %r' % (got_entry, exp_entry))
|
|
if list(got_stages or []) != exp_stages:
|
|
problems.append('stages is %r, expected %r — a hook at the wrong stage '
|
|
'(or at none) never fires' % (got_stages, exp_stages))
|
|
if problems:
|
|
emit('FAIL', 'hook %s: %s' % (hid, '; '.join(problems)))
|
|
else:
|
|
emit('PASS', 'hook %s runs `%s` at %s'
|
|
% (hid, exp_entry, ','.join(exp_stages)))
|
|
# Collect the in-repo script/manifest paths the pinned entry names, so the
|
|
# pin cannot agree with a config that points at nothing.
|
|
for token in re.split(r'\s+', exp_entry):
|
|
token = token.strip('\'"')
|
|
if re.fullmatch(r'[A-Za-z0-9_.\-/]+\.(sh|json)', token):
|
|
entry_paths.append((hid, token))
|
|
|
|
# Every pinned entry path exists on disk. Without this, EXPECTED and the config
|
|
# could agree perfectly on a script that was deleted.
|
|
if not entry_paths:
|
|
emit('FAIL', 'no pinned entry named an in-repo script or manifest — the '
|
|
'existence check below iterated zero times and proved nothing')
|
|
else:
|
|
broken = ['%s -> %s' % (hid, token)
|
|
for hid, token in entry_paths
|
|
if not os.path.exists(os.path.join(root, token))]
|
|
if broken:
|
|
emit('FAIL', 'pinned hook entries point at files that do not exist: %s'
|
|
% ', '.join(broken))
|
|
else:
|
|
emit('PASS', 'all %d in-repo files named by a pinned hook entry exist '
|
|
'on disk' % len(entry_paths))
|
|
PY
|
|
)"
|
|
while IFS=$'\t' read -r status msg; do
|
|
[[ -n "$status" ]] || continue
|
|
if [[ "$status" == PASS ]]; then
|
|
pass "$msg"
|
|
else
|
|
fail "$msg"
|
|
fi
|
|
done <<< "$WIRING_REPORT"
|
|
|
|
echo ""
|
|
echo "Results: $PASS passed, $FAIL failed"
|
|
[[ $FAIL -eq 0 ]]
|