Defame1297
fbd030c7ea
plugins/onedev is the first plugin package to declare a real dependency, and that arms a check every previous plugin left vacuous. apm treats any directory holding both apm.yml and apm.lock.yaml as an install root; a package is not one, so there is no green state for it. Without a package lockfile, lockfile-exists fails outright. With one, it passes and thereby arms the other nine checks, where drift then demands the dependency's skills be deployed inside the package and apm lock leaves an apm_modules/ tree behind. scripts/apm-audit-ci.sh replaces the inline bash -c loop and waives that single check for a non-root manifest. It fails closed on three axes: the root is never waived; the failing check must be lockfile-exists and no other, asserted by matching "1 of 1 check(s) failed"; and unrecognised output fails. Dropping --ci for package directories was the smaller change and is wrong. Verified on apm 0.28.0 against a scratch package whose dependency entry carried no git/path/registry field: apm audit --ci exits 1 naming it, while plain apm audit exits 0 and says nothing. Malformed-dependency detection is the reason gates.md gives for auditing packages at all, and a package with dependencies is the only kind that can carry a malformed dependency entry. The waiver matches on apm's stdout, so an apm upgrade rewording either line turns it off. That fails the push rather than hiding a defect. Also records the onedev entry in apm.lock.yaml, which PR #136 could not carry because the plugin was not yet resolvable from the remote's main. ADR: 0026 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2
883 lines
42 KiB
Bash
Executable File
883 lines
42 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Regression test for the STRUCTURAL claims the ADR-0020 gate family makes about
|
|
# itself. None of them was pinned anywhere before this file, and each one fails
|
|
# silently — which is the whole reason they need a test rather than a comment:
|
|
#
|
|
# 1. "ONE resolver, and both of its consumers run it." The ADR-0020 boundary
|
|
# resolver has exactly one copy, factory-audit's sourced
|
|
# lib-boundary-resolver.sh. scripts/skill-size-check.sh used to embed a
|
|
# second, byte-identical copy, because it was also exported through a
|
|
# published hook manifest whose consumers could not reach a file inside the
|
|
# plugin; 4de5b6b retired that export (ADR-0014), so the hook now sources
|
|
# the library too. What must not fail silently: the hook growing its own
|
|
# copy back, the library being gutted, or the hook no longer running the
|
|
# library's text at all. So this asserts the library is real content, the
|
|
# hook carries no marker pair, the hook fails closed without the library,
|
|
# and — by a sentinel planted in a copied library — that the text the hook
|
|
# executes IS the library's.
|
|
# 1a. The library is the ONLY authority, and validate.sh sources it in both
|
|
# mode branches — the same authority checks 1b makes for the parser.
|
|
# 1b. The same claim, one directory over, for the Contributing-files parser.
|
|
# That one was worse: it was embedded in both validate-provenance.sh copies,
|
|
# the agent-audit copy's docstring ASSERTED it was kept behaviourally
|
|
# identical to skill-audit's, and the two had drifted (cosmetically, at
|
|
# 484357a, re-unified at 598a7c3 — nothing had caught it). ADR-0025
|
|
# removed the second copy, so there is no longer a pair to hash — but
|
|
# deleting the assertion would restore exactly the condition that let the
|
|
# drift happen, so it is CONVERTED rather than dropped: it now pins that
|
|
# lib-contributing-files.sh is the single authority (one marker pair), that
|
|
# validate-provenance.sh actually SOURCES it, and that nobody has re-inlined
|
|
# the parser into a mode library or anywhere else in the tree.
|
|
# 2. Both interpreter preflights, in both scripts and both of the merged
|
|
# entry point's modes. python3 and PyYAML are declared HARD dependencies
|
|
# precisely so a missing one cannot turn into a vacuous pass, and the two
|
|
# are checked separately so the message names the thing to install rather
|
|
# than the wrong one. factory-audit/scripts/validate.sh detects its mode
|
|
# first and only then calls the preflight, which lives in the mode library
|
|
# it sources (lib-checks-skill.sh / lib-checks-agent.sh) — so each mode is
|
|
# probed with its own target shape, not just one of them.
|
|
# 3. `verbose: true` on the skill-size-check hook. It is the ENTIRE delivery
|
|
# mechanism for the SUGGESTION tier: pre-commit prints nothing at all for a
|
|
# passing hook, and a SUGGESTION deliberately does not fail, so dropping
|
|
# one word from the config silences the tier ADR-0020 depends on while
|
|
# every test and every hook still reports green.
|
|
# 4. The WIRING of every repo-authored hook — its id, its `entry:` and its
|
|
# `stages:`. Same failure shape as 3, one level up: the tests drive these
|
|
# scripts by path, so nothing noticed whether .pre-commit-config.yaml
|
|
# still invoked them. On a scratch copy, deleting eight local hook blocks
|
|
# at once — `run-tests` among them — left every suite green. Upstream
|
|
# hooks and every `rev:` are deliberately out of scope.
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
HOOK="$REPO_ROOT/scripts/skill-size-check.sh"
|
|
FACTORY_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit"
|
|
# ADR-0025: one auto-detecting entry point, and one sourced copy of the resolver
|
|
# behind it, which the root hook sources as well. The entry point is what the
|
|
# preflight assertions run; the library is what assertion 1 inspects.
|
|
FACTORY_VALIDATE="$FACTORY_AUDIT/scripts/validate.sh"
|
|
FACTORY_RESOLVER="$FACTORY_AUDIT/scripts/lib-boundary-resolver.sh"
|
|
PASS=0
|
|
FAIL=0
|
|
|
|
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
|
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
|
|
|
TMPDIR_T="$(mktemp -d)"
|
|
trap 'rm -rf "$TMPDIR_T"' EXIT
|
|
|
|
BEGIN_MARKER='# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER ====='
|
|
END_MARKER='# ===== END ADR-0020 SHARED BOUNDARY RESOLVER ====='
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. The resolver has one copy, and the root hook runs it
|
|
# ---------------------------------------------------------------------------
|
|
# One copy since the 2026-09-16 change: factory-audit's lib-boundary-resolver.sh.
|
|
# scripts/skill-size-check.sh sources it rather than embedding a second copy, so
|
|
# there is no pair left to hash. What replaces the hash is the set of ways a
|
|
# single sourced copy can still fail quietly.
|
|
echo ""
|
|
echo "--- the ADR-0020 resolver has one copy, and skill-size-check.sh runs it ---"
|
|
|
|
# (i) The library carries exactly one well-formed marker pair around real
|
|
# content. An unbalanced pair makes every extraction measure the wrong span, and
|
|
# an emptied-out block would still "load" while enforcing nothing. The resolver
|
|
# is ~1,060 lines; 100 is a floor low enough never to need maintenance and high
|
|
# enough that a gutted block cannot sneak past.
|
|
if [[ ! -f "$FACTORY_RESOLVER" ]]; then
|
|
fail "resolver library not found: ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
|
|
else
|
|
b="$(grep -cFx "$BEGIN_MARKER" "$FACTORY_RESOLVER" || true)"
|
|
e="$(grep -cFx "$END_MARKER" "$FACTORY_RESOLVER" || true)"
|
|
if [[ "$b" == "1" && "$e" == "1" ]]; then
|
|
pass "${FACTORY_RESOLVER#"$REPO_ROOT/"} carries exactly one BEGIN and one END marker"
|
|
span="$(sed -n "/^${BEGIN_MARKER}\$/,/^${END_MARKER}\$/p" "$FACTORY_RESOLVER" | wc -l | tr -d ' ')"
|
|
if [[ "$span" -gt 100 ]]; then
|
|
pass "the resolver span is $span lines — real content, not an empty block"
|
|
else
|
|
fail "the resolver span is only $span lines — a gutted block would load and resolve nothing"
|
|
fi
|
|
else
|
|
fail "${FACTORY_RESOLVER#"$REPO_ROOT/"} has $b BEGIN and $e END markers, expected 1 and 1"
|
|
fi
|
|
fi
|
|
|
|
# (ii) The hook carries no copy of its own. A column-0 marker line in the hook
|
|
# is the shape the old embedded copy had, and the shape a paste-back would have.
|
|
if [[ ! -f "$HOOK" ]]; then
|
|
fail "hook not found: ${HOOK#"$REPO_ROOT/"}"
|
|
else
|
|
hb="$(grep -cFx "$BEGIN_MARKER" "$HOOK" || true)"
|
|
he="$(grep -cFx "$END_MARKER" "$HOOK" || true)"
|
|
if [[ "$hb" == "0" && "$he" == "0" ]]; then
|
|
pass "${HOOK#"$REPO_ROOT/"} carries no resolver marker lines — it has not grown its own copy back"
|
|
else
|
|
fail "${HOOK#"$REPO_ROOT/"} carries $hb BEGIN and $he END marker lines — a second copy of the resolver is back in the hook"
|
|
fi
|
|
fi
|
|
|
|
# (iii) and (iv) run the hook from a scratch tree that mirrors the two paths it
|
|
# depends on, so the real library is never touched. The scratch hook is a copy
|
|
# of the real one; its library is either absent, gutted, or the real library
|
|
# with a sentinel planted inside the resolver block.
|
|
SSC_TREE="$TMPDIR_T/ssc-tree"
|
|
SSC_LIB_DIR="$SSC_TREE/plugins/kyberforge/.apm/skills/factory-audit/scripts"
|
|
mkdir -p "$SSC_TREE/scripts" "$SSC_LIB_DIR" "$TMPDIR_T/ssc-skill/probe-skill"
|
|
cp "$HOOK" "$SSC_TREE/scripts/skill-size-check.sh"
|
|
printf -- '---\nname: probe-skill\ndescription: Use when probing the resolver wiring.\nmetadata:\n version: "0.1.0"\n---\n\n## Step 1\n\nDo the thing.\n' \
|
|
> "$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
|
|
PROBE="$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
|
|
|
|
run_scratch_hook() {
|
|
local rc=0
|
|
SSC_OUT="$(bash "$SSC_TREE/scripts/skill-size-check.sh" "$PROBE" 2>&1)" || rc=$?
|
|
SSC_RC=$rc
|
|
}
|
|
|
|
# (iii) Fail closed: no library, then a library that defines nothing.
|
|
rm -f "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"boundary resolver library was not found"* ]]; then
|
|
pass "with the library missing, the hook exits $SSC_RC and names the missing library — not a vacuous pass"
|
|
else
|
|
fail "with the library missing, the hook exited $SSC_RC without naming it: $SSC_OUT"
|
|
fi
|
|
printf '# gutted\n' > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"did not define the ADR-0020 boundary resolver"* ]]; then
|
|
pass "with a library that defines no resolver, the hook exits $SSC_RC and says so"
|
|
else
|
|
fail "with a gutted library, the hook exited $SSC_RC without saying so: $SSC_OUT"
|
|
fi
|
|
|
|
# (iv) The text the hook executes IS the library's. A sentinel print planted
|
|
# just after the BEGIN marker of a copied library must appear in the hook's
|
|
# output. Without this, a hook that sourced the library but ran some other
|
|
# program would pass (i)-(iii).
|
|
SENTINEL="ADR0020-RESOLVER-SENTINEL-$$"
|
|
if [[ -f "$FACTORY_RESOLVER" ]]; then
|
|
awk -v m="$BEGIN_MARKER" -v s="$SENTINEL" '{ print } $0 == m { print "print(\"" s "\")" }' \
|
|
"$FACTORY_RESOLVER" > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
|
|
if [[ "$(grep -cF "$SENTINEL" "$SSC_LIB_DIR/lib-boundary-resolver.sh" || true)" -ne 1 ]]; then
|
|
fail "fixture check: the sentinel was not planted exactly once in the copied library — the case below would prove nothing"
|
|
else
|
|
run_scratch_hook
|
|
if [[ "$SSC_RC" -eq 0 && "$SSC_OUT" == *"$SENTINEL"* ]]; then
|
|
pass "a sentinel planted in the library's resolver block runs inside the hook — the hook executes the library's text"
|
|
else
|
|
fail "the hook did not run the library's resolver text (rc=$SSC_RC, sentinel absent from output): $SSC_OUT"
|
|
fi
|
|
fi
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1a. The resolver library is the ONLY authority, and validate.sh sources it
|
|
# ---------------------------------------------------------------------------
|
|
# Assertion 1 says nothing about a copy somewhere else in the tree, and nothing
|
|
# about whether validate.sh runs the library. Assertion 1b
|
|
# pins both of those for the Contributing-files parser; the resolver is the same
|
|
# defect class and gets the same two checks:
|
|
#
|
|
# a. validate.sh actually SOURCES lib-boundary-resolver.sh, in BOTH mode
|
|
# branches — asserted inside each arm of `case "$MODE" in`, not by counting
|
|
# source lines file-wide, because a count cannot see a branch. A library
|
|
# that is identical, unique and never sourced is a copy that has quietly
|
|
# been replaced by an inline one — and assertion 1 would stay green over it.
|
|
# b. Nothing has re-inlined it. The BEGIN marker and a def unique to the
|
|
# resolver (`_authoring_root`) appear in exactly one file,
|
|
# lib-boundary-resolver.sh, and nowhere else under the tree. A mode library
|
|
# or a root script that grows a "just this once" copy would otherwise
|
|
# escape assertion 1 entirely, because 1 inspects only the files it names.
|
|
echo ""
|
|
echo "--- the ADR-0020 resolver has exactly one authority, and validate.sh sources it ---"
|
|
|
|
# Deployed and vendored trees are generated copies, not authorities: .claude/ is
|
|
# apm install output, apm_modules/ is resolved dependencies, build/ is release
|
|
# artifacts. This file is excluded because it necessarily quotes what it
|
|
# searches for. Markdown is excluded because an authority is code that runs:
|
|
# ADR-0025 and gates.md quote these needles to document this very check, and a
|
|
# prose mention is not a re-inlined copy.
|
|
SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
|
|
tree_scan() {
|
|
grep -rlF --binary-files=without-match \
|
|
--exclude-dir=.git --exclude-dir=build --exclude-dir=.claude \
|
|
--exclude-dir=apm_modules --exclude-dir=node_modules --exclude='*.md' \
|
|
-- "$1" "$REPO_ROOT" 2>/dev/null | grep -vFx "$SELF" | sort || true
|
|
}
|
|
|
|
# (a) Sourced, once per mode branch — asserted PER BRANCH, not by counting.
|
|
#
|
|
# This used to be a file-wide `grep -c ... >= 2`, and a count cannot see a
|
|
# branch. Proven by mutation: moving the agent arm's source line out and
|
|
# duplicating the skill arm's leaves the count at 2 and the old assertion
|
|
# printed PASS while claiming "in both mode branches" — so a branch that lost
|
|
# its resolver gracefully (a defaulted variable, a `set +u` region, an optional
|
|
# resolver) stayed green and lying. The claim is per-branch, so the check is
|
|
# too: each arm of `case "$MODE" in` must source the resolver inside its own
|
|
# body. The mutation that fooled the old form is run against the new one below,
|
|
# because an assertion about branches that has never been shown to fail on a
|
|
# count-preserving branch edit is exactly the assertion that was here before.
|
|
RESOLVER_SOURCE_RE='^[[:space:]]*(\.|source)[[:space:]]+.*lib-boundary-resolver\.sh'
|
|
|
|
# The body of one arm of the `case "$MODE" in` block: everything between the
|
|
# arm's label and its `;;`. Structural, not positional — the file is edited
|
|
# often and line numbers or a whole-file hash would pin the wrong thing.
|
|
mode_arm_body() {
|
|
awk -v arm="$2" '
|
|
!incase && $0 ~ /^[[:space:]]*case[[:space:]]+"?\$MODE"?[[:space:]]+in[[:space:]]*$/ { incase = 1; next }
|
|
incase && $0 ~ /^[[:space:]]*esac([[:space:]]|$)/ { incase = 0; next }
|
|
incase && !inarm && $0 ~ "^[[:space:]]*\\(?" arm "\\)[[:space:]]*$" { inarm = 1; next }
|
|
inarm && $0 ~ /^[[:space:]]*;;[[:space:]]*$/ { inarm = 0; next }
|
|
inarm { print }
|
|
' "$1"
|
|
}
|
|
|
|
# Every mode arm sources the resolver exactly once, inside its own body.
|
|
# Returns 0/1 and leaves the reason in ARM_DETAIL, so the same function can be
|
|
# run against the real file and against the mutant below.
|
|
ARM_DETAIL=""
|
|
check_resolver_per_arm() {
|
|
local file="$1" arm body n rc=0
|
|
ARM_DETAIL=""
|
|
for arm in skill agent; do
|
|
body="$(mode_arm_body "$file" "$arm")"
|
|
if [[ -z "$body" ]]; then
|
|
ARM_DETAIL+="the $arm) arm of the case \"\$MODE\" block was not found or is empty; "
|
|
rc=1
|
|
continue
|
|
fi
|
|
n="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$body" || true)"
|
|
if [[ "$n" -eq 0 ]]; then
|
|
ARM_DETAIL+="the $arm) arm never sources lib-boundary-resolver.sh, so that mode runs some other resolver or none; "
|
|
rc=1
|
|
elif [[ "$n" -ne 1 ]]; then
|
|
ARM_DETAIL+="the $arm) arm sources lib-boundary-resolver.sh $n times; "
|
|
rc=1
|
|
fi
|
|
done
|
|
return $rc
|
|
}
|
|
|
|
if [[ ! -f "$FACTORY_VALIDATE" ]]; then
|
|
fail "entry point not found: ${FACTORY_VALIDATE#"$REPO_ROOT/"}"
|
|
else
|
|
RESOLVER_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$FACTORY_VALIDATE" || true)"
|
|
if check_resolver_per_arm "$FACTORY_VALIDATE"; then
|
|
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"}: the skill) and agent) arms of its case \"\$MODE\" block EACH source lib-boundary-resolver.sh inside their own body, exactly once"
|
|
else
|
|
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} does not source lib-boundary-resolver.sh once per mode arm: ${ARM_DETAIL%; }"
|
|
fi
|
|
# Secondary, and deliberately not the verdict: with one source per arm proven
|
|
# above, a file-wide total of exactly 2 says there is no third source line
|
|
# sitting outside both arms.
|
|
if [[ "$RESOLVER_SOURCES" -eq 2 ]]; then
|
|
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries exactly 2 resolver source lines file-wide — the two arm sources and nothing else"
|
|
else
|
|
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries $RESOLVER_SOURCES resolver source lines file-wide, expected the 2 that belong to the mode arms"
|
|
fi
|
|
|
|
# Mutation self-test. The mutation is the one the retired count could not
|
|
# see: the agent arm's source line is removed and the skill arm's duplicated,
|
|
# so the FILE-WIDE COUNT IS UNCHANGED. Written into a copy; the real file is
|
|
# never touched.
|
|
MUT_DIR="$(mktemp -d "$TMPDIR_T/resolver-mutant.XXXXXX")"
|
|
MUT="$MUT_DIR/validate.sh"
|
|
cp "$FACTORY_VALIDATE" "$MUT"
|
|
if python3 - "$MUT" <<'PY'
|
|
import re
|
|
import sys
|
|
|
|
path = sys.argv[1]
|
|
with open(path, encoding='utf-8') as fh:
|
|
lines = fh.read().split('\n')
|
|
|
|
case_re = re.compile(r'^\s*case\s+"?\$MODE"?\s+in\s*$')
|
|
esac_re = re.compile(r'^\s*esac(\s|$)')
|
|
term_re = re.compile(r'^\s*;;\s*$')
|
|
src_re = re.compile(r'^\s*(\.|source)\s+.*lib-boundary-resolver\.sh')
|
|
|
|
starts = [i for i, l in enumerate(lines) if case_re.match(l)]
|
|
assert len(starts) == 1, 'expected exactly one `case "$MODE" in`, found %d' % len(starts)
|
|
ci = starts[0]
|
|
ends = [i for i in range(ci + 1, len(lines)) if esac_re.match(lines[i])]
|
|
assert ends, 'the case "$MODE" block has no esac'
|
|
ei = ends[0]
|
|
|
|
|
|
def arm_sources(name):
|
|
for i in range(ci + 1, ei):
|
|
if re.match(r'^\s*\(?%s\)\s*$' % name, lines[i]):
|
|
for j in range(i + 1, ei):
|
|
if term_re.match(lines[j]):
|
|
return [k for k in range(i + 1, j) if src_re.match(lines[k])]
|
|
raise AssertionError('the %s) arm has no ;;' % name)
|
|
raise AssertionError('no %s) arm in the case "$MODE" block' % name)
|
|
|
|
|
|
skill = arm_sources('skill')
|
|
agent = arm_sources('agent')
|
|
assert len(skill) == 1 and len(agent) == 1, \
|
|
'expected one resolver source per arm before mutating, got skill=%d agent=%d' % (len(skill), len(agent))
|
|
|
|
out = list(lines)
|
|
del out[agent[0]] # the agent arm loses its resolver ...
|
|
out.insert(skill[0], lines[skill[0]]) # ... and the skill arm gains a duplicate
|
|
with open(path, 'w', encoding='utf-8') as fh:
|
|
fh.write('\n'.join(out))
|
|
PY
|
|
then
|
|
MUT_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$MUT" || true)"
|
|
MUT_AGENT="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$(mode_arm_body "$MUT" agent)" || true)"
|
|
# Guard the fixture before trusting its verdict: the mutation must have
|
|
# actually emptied the agent arm AND left the file-wide count where it was,
|
|
# or the case below proves nothing about the defect it stands for.
|
|
if [[ "$MUT_SOURCES" -eq "$RESOLVER_SOURCES" && "$MUT_AGENT" -eq 0 ]]; then
|
|
pass "fixture check: the mutant's agent arm sources no resolver while the file-wide count is still $MUT_SOURCES — the retired 'count >= 2' assertion would have passed it"
|
|
else
|
|
fail "the resolver mutation did not land as intended (file-wide $MUT_SOURCES vs $RESOLVER_SOURCES, agent arm $MUT_AGENT) — the case below would prove nothing"
|
|
fi
|
|
if check_resolver_per_arm "$MUT"; then
|
|
fail "the per-arm check PASSED a validate.sh whose agent arm has no resolver source — it is still counting, not reading branches"
|
|
else
|
|
pass "the per-arm check FAILS the count-preserving mutant (${ARM_DETAIL%; }) — it reads the branches, not a total"
|
|
fi
|
|
else
|
|
fail "could not build the resolver mutation fixture — validate.sh's case \"\$MODE\" structure is not the shape this self-test knows, so the per-arm check is unproven"
|
|
fi
|
|
fi
|
|
|
|
# (b) Exactly the one authority, for both spellings of a copy.
|
|
EXPECTED_RESOLVERS="$FACTORY_RESOLVER"
|
|
check_resolver_authorities() {
|
|
local label="$1" needle="$2"
|
|
local found
|
|
found="$(tree_scan "$needle")"
|
|
if [[ "$found" == "$EXPECTED_RESOLVERS" ]]; then
|
|
pass "$label appears in exactly the one resolver authority and nowhere else"
|
|
elif [[ -z "$found" ]]; then
|
|
fail "$label was found in NO file at all — the scan is looking for the wrong text"
|
|
else
|
|
fail "$label appears in an unexpected set of files, so the resolver has been re-inlined or lost: $(echo "$found" | tr '\n' ' ')— expected exactly ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
|
|
fi
|
|
}
|
|
check_resolver_authorities "the resolver's BEGIN marker" "$BEGIN_MARKER"
|
|
check_resolver_authorities "a 'def _authoring_root' definition" "def _authoring_root("
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1b. The Contributing-files parser has exactly ONE authority, and it is sourced
|
|
# ---------------------------------------------------------------------------
|
|
# Same defect class, one directory over. parse_contributing_files() used to be
|
|
# embedded in both validate-provenance.sh copies for the same reason the resolver
|
|
# was once embedded in several scripts, and until this assertion existed the agent-audit copy's
|
|
# docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's
|
|
# copy" — an invariant nothing checked, and the two did drift into different
|
|
# spellings of the bullet loop at 484357a. That drift happened to be
|
|
# behaviour-neutral and was re-unified by hand at 598a7c3; the next one need
|
|
# not be. The parser decides whether checks 4,
|
|
# 5 and 8 run at all, so a one-sided edit disables a check in one script while
|
|
# every other test stays green.
|
|
#
|
|
# ADR-0025 merged the two skills, so there is now ONE copy and nothing left to
|
|
# hash against. That does NOT retire the assertion: a byte-identity check over a
|
|
# single copy is vacuous, and deleting it outright would restore exactly the
|
|
# condition that allowed the original drift — a parser with no pinned authority.
|
|
# So the claim is CONVERTED, not dropped. It is the same claim ("the parser has
|
|
# exactly one authority") stated against the new structure:
|
|
#
|
|
# a. factory-audit/scripts/lib-contributing-files.sh exists and carries exactly
|
|
# one BEGIN/END marker pair, over a span of real content.
|
|
# b. validate-provenance.sh actually SOURCES it. A library nobody sources is a
|
|
# copy that has silently been replaced by an inline one somewhere else.
|
|
# c. Nothing has re-inlined it. No other file in the tree carries the marker
|
|
# pair, and no other file defines parse_contributing_files. This is the part
|
|
# that fails if the merge is ever partially reverted, or if a mode library
|
|
# grows its own "just this once" copy — which is precisely how the drift
|
|
# this assertion was written for got in.
|
|
echo ""
|
|
echo "--- the Contributing-files parser has exactly one authority, and validate-provenance.sh sources it ---"
|
|
|
|
CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ====='
|
|
CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER ====='
|
|
CF_LIB="$FACTORY_AUDIT/scripts/lib-contributing-files.sh"
|
|
FACTORY_PROV="$FACTORY_AUDIT/scripts/validate-provenance.sh"
|
|
|
|
# (a) One library, one well-formed marker pair, over real content.
|
|
CF_MARKERS_OK=true
|
|
if [[ ! -f "$CF_LIB" ]]; then
|
|
fail "the single parser authority is missing: ${CF_LIB#"$REPO_ROOT/"}"
|
|
CF_MARKERS_OK=false
|
|
else
|
|
b="$(grep -cFx "$CF_BEGIN" "$CF_LIB" || true)"
|
|
e="$(grep -cFx "$CF_END" "$CF_LIB" || true)"
|
|
if [[ "$b" == "1" && "$e" == "1" ]]; then
|
|
pass "${CF_LIB#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker"
|
|
else
|
|
fail "${CF_LIB#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1"
|
|
CF_MARKERS_OK=false
|
|
fi
|
|
fi
|
|
|
|
if ! $CF_MARKERS_OK; then
|
|
fail "skipping the parser content check — the marker pair is not well-formed, so any extraction would measure the wrong span"
|
|
else
|
|
# A span gutted down to its docstring would still satisfy every structural
|
|
# check above while enforcing nothing, exactly as for the resolver. The parser
|
|
# block is ~93 lines; 40 is a floor low enough never to need maintenance and
|
|
# high enough that a gutted block cannot sneak past.
|
|
CF_LINECOUNT="$(sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$CF_LIB" | wc -l | tr -d ' ')"
|
|
if [[ "$CF_LINECOUNT" -gt 40 ]]; then
|
|
pass "the extracted parser block is $CF_LINECOUNT lines — a real parser, not an empty or docstring-only span"
|
|
else
|
|
fail "the extracted parser block is only $CF_LINECOUNT lines — a gutted span asserts nothing"
|
|
fi
|
|
fi
|
|
|
|
# (b) The one entry point sources it. Without this, (a) and (c) are satisfied by
|
|
# a library that is present, unique and entirely unused.
|
|
if [[ ! -f "$FACTORY_PROV" ]]; then
|
|
fail "entry point not found: ${FACTORY_PROV#"$REPO_ROOT/"}"
|
|
elif grep -Eq '^[[:space:]]*(\.|source)[[:space:]]+.*lib-contributing-files\.sh' "$FACTORY_PROV"; then
|
|
pass "${FACTORY_PROV#"$REPO_ROOT/"} sources lib-contributing-files.sh — the single copy is the one that actually runs"
|
|
else
|
|
fail "${FACTORY_PROV#"$REPO_ROOT/"} never sources lib-contributing-files.sh — the library is dead code and the parser that runs is some other copy"
|
|
fi
|
|
|
|
# (c) Nobody re-inlined it. Both spellings are scanned: the marker pair (a
|
|
# copy-paste of the block) and a second `def parse_contributing_files` (a
|
|
# re-implementation that skipped the markers). tree_scan (defined in 1a)
|
|
# excludes the same generated trees and this file.
|
|
cf_scan() { tree_scan "$1"; }
|
|
check_sole_authority() {
|
|
local label="$1" needle="$2"
|
|
local found extra
|
|
found="$(cf_scan "$needle")"
|
|
extra="$(printf '%s\n' "$found" | grep -vFx "$CF_LIB" | grep -v '^$' || true)"
|
|
if [[ -z "$found" ]]; then
|
|
fail "$label was found in NO file at all — the parser authority has vanished, or the scan is looking for the wrong text"
|
|
elif [[ -n "$extra" ]]; then
|
|
fail "$label appears outside the single authority, so the parser has been re-inlined: $(echo "$extra" | tr '\n' ' ')— delete the copy and source lib-contributing-files.sh instead"
|
|
else
|
|
pass "$label appears only in ${CF_LIB#"$REPO_ROOT/"} — one authority, no re-inlined copies"
|
|
fi
|
|
}
|
|
check_sole_authority "the parser's BEGIN marker" "$CF_BEGIN"
|
|
check_sole_authority "a 'def parse_contributing_files' definition" "def parse_contributing_files("
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. Both interpreter preflights, in both scripts and both modes
|
|
# ---------------------------------------------------------------------------
|
|
# The two are checked separately on purpose: `python3 -c 'import yaml'` fails
|
|
# identically whether python3 is missing or PyYAML is, and naming the wrong one
|
|
# sends the reader to install the wrong thing.
|
|
#
|
|
# ADR-0025 merged the two validators into one auto-detecting entry point, but
|
|
# the preflight did NOT merge with them: validate.sh detects the mode first and
|
|
# then calls kyberforge_skill_preflight or kyberforge_agent_preflight from the
|
|
# mode library it sources. There are still two preflights, so both are still
|
|
# probed — once with a skill target and once with an agent target. Collapsing
|
|
# these to a single probe would leave one mode's preflight unpinned, and a mode
|
|
# whose preflight is gone reports a vacuous pass on a machine with no PyYAML.
|
|
REAL_PYTHON="$(command -v python3)"
|
|
# Absolute path, deliberately. The no-python3 fixture below replaces PATH
|
|
# wholesale, so a bare `bash` (or `/usr/bin/env bash`) would be resolved against
|
|
# that stripped PATH and die with "No such file or directory" before the script
|
|
# under test ever starts -- a 127 that looks like the preflight firing.
|
|
BASH_BIN="$(command -v bash)"
|
|
|
|
# A PATH that genuinely has no python3 on it. Built by symlinking the handful of
|
|
# binaries the three scripts touch before their own preflight rather than by
|
|
# hiding python3 from a full PATH, because there is no portable way to subtract
|
|
# one entry from a directory. `bash` is invoked by absolute path below so the
|
|
# interpreter itself does not have to be on this PATH.
|
|
NOPY_BIN="$TMPDIR_T/nopython-bin"
|
|
mkdir -p "$NOPY_BIN"
|
|
for b in awk cat cut dirname basename grep sed pwd rm mkdir tr; do
|
|
src="$(command -v "$b" 2>/dev/null || true)"
|
|
[[ -n "$src" ]] && ln -sf "$src" "$NOPY_BIN/$b"
|
|
done
|
|
|
|
# A python3 that runs but cannot import yaml. A shim on PATH re-execs the real
|
|
# interpreter with a PYTHONPATH entry holding a `yaml` module that raises on
|
|
# import; PYTHONPATH precedes site-packages on sys.path, so it shadows a real
|
|
# PyYAML install without touching it.
|
|
SHADOW="$TMPDIR_T/shadow"
|
|
mkdir -p "$SHADOW"
|
|
printf 'raise ImportError("PyYAML deliberately unavailable in this fixture")\n' \
|
|
> "$SHADOW/yaml.py"
|
|
NOYAML_BIN="$TMPDIR_T/noyaml-bin"
|
|
mkdir -p "$NOYAML_BIN"
|
|
cat > "$NOYAML_BIN/python3" <<EOF
|
|
#!/bin/sh
|
|
PYTHONPATH="$SHADOW\${PYTHONPATH:+:\$PYTHONPATH}" exec "$REAL_PYTHON" "\$@"
|
|
EOF
|
|
chmod +x "$NOYAML_BIN/python3"
|
|
|
|
# Sanity-check the two fixtures themselves before trusting any verdict they
|
|
# produce. A shim that silently still imports yaml would make every PyYAML
|
|
# assertion below pass for the wrong reason.
|
|
if PATH="$NOYAML_BIN:$PATH" python3 -c 'import yaml' 2>/dev/null; then
|
|
fail "the no-PyYAML shim does not actually shadow PyYAML — every PyYAML assertion below would be vacuous"
|
|
else
|
|
pass "fixture check: the no-PyYAML shim makes 'import yaml' fail while python3 still runs"
|
|
fi
|
|
if PATH="$NOPY_BIN" command -v python3 > /dev/null 2>&1; then
|
|
fail "the no-python3 PATH still resolves python3 — every python3 assertion below would be vacuous"
|
|
else
|
|
pass "fixture check: the no-python3 PATH resolves no python3"
|
|
fi
|
|
|
|
# A minimal, entirely clean subject for each script. The preflight must fire
|
|
# before any measurement, so the subject's own content is irrelevant — which is
|
|
# exactly what makes a clean one the right choice: nothing else can produce the
|
|
# non-zero exit these cases assert.
|
|
SUBJECT_SKILL_DIR="$TMPDIR_T/subject/my-skill"
|
|
mkdir -p "$SUBJECT_SKILL_DIR"
|
|
cat > "$SUBJECT_SKILL_DIR/SKILL.md" <<'EOF'
|
|
---
|
|
name: my-skill
|
|
description: A short valid description. Do not use for anything else.
|
|
metadata:
|
|
version: "1.0.0"
|
|
---
|
|
|
|
Do the thing.
|
|
EOF
|
|
SUBJECT_AGENT_ROOT="$TMPDIR_T/subject-agent"
|
|
mkdir -p "$SUBJECT_AGENT_ROOT/.apm/agents"
|
|
cat > "$SUBJECT_AGENT_ROOT/apm.yml" <<'EOF'
|
|
name: test-package
|
|
version: 0.1.0
|
|
type: skill
|
|
EOF
|
|
cat > "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" <<'EOF'
|
|
---
|
|
name: my-agent
|
|
description: A short valid description. Do not use for anything else.
|
|
---
|
|
|
|
You are a test agent. When invoked, do the thing.
|
|
EOF
|
|
|
|
# probe_preflight <label> <env-kind: nopython|noyaml> <expect-needle> <cmd...>
|
|
probe_preflight() {
|
|
local label="$1" kind="$2" needle="$3"
|
|
shift 3
|
|
local out status=0
|
|
set +e
|
|
if [[ "$kind" == nopython ]]; then
|
|
out="$(env -i PATH="$NOPY_BIN" HOME="$HOME" "$BASH_BIN" "$@" 2>&1)"
|
|
else
|
|
out="$(env PATH="$NOYAML_BIN:$PATH" "$BASH_BIN" "$@" 2>&1)"
|
|
fi
|
|
status=$?
|
|
set -e
|
|
if [[ $status -eq 0 ]]; then
|
|
fail "$label exited 0 — a missing hard dependency became a vacuous pass (output: ${out:-<empty>})"
|
|
elif [[ "$out" != *"$needle"* ]]; then
|
|
# The needle is the DIAGNOSTIC ("python3 is required"), not the bare word.
|
|
# Deleting the preflight entirely would still produce a non-zero exit and a
|
|
# message mentioning python3 -- bash's own "python3: command not found" --
|
|
# so a bare-word needle would go green on a script with no preflight at all.
|
|
fail "$label exited $status but never produced the '$needle' diagnostic (output: ${out:-<empty>})"
|
|
elif [[ "$kind" == nopython && "$out" == *PyYAML* ]]; then
|
|
fail "$label reported PyYAML when python3 itself is missing — that sends the reader to install the wrong thing (output: $out)"
|
|
else
|
|
pass "$label"
|
|
fi
|
|
}
|
|
|
|
echo ""
|
|
echo "--- a PATH with no python3 is a hard failure in both scripts and both modes, naming python3 ---"
|
|
probe_preflight "scripts/skill-size-check.sh reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
|
|
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
|
|
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing python3" \
|
|
nopython "python3 is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
|
|
|
|
echo ""
|
|
echo "--- a python3 that cannot import yaml is a hard failure in both scripts and both modes, naming PyYAML ---"
|
|
probe_preflight "scripts/skill-size-check.sh reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
|
|
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
|
|
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing PyYAML" \
|
|
noyaml "PyYAML is required" \
|
|
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
|
|
|
|
# The control. Without it, "fails when the dependency is missing" is satisfied by
|
|
# a script that fails unconditionally, and the two cases above would be green on
|
|
# a gate that never runs at all.
|
|
echo ""
|
|
echo "--- control: with both dependencies present the same subjects pass ---"
|
|
# The label is carried explicitly because the two validate.sh probes now name the
|
|
# same script and differ only in the mode its target selects.
|
|
for probe in "scripts/skill-size-check.sh|$HOOK|$SUBJECT_SKILL_DIR/SKILL.md" \
|
|
"factory-audit/scripts/validate.sh (skill mode)|$FACTORY_VALIDATE|$SUBJECT_SKILL_DIR" \
|
|
"factory-audit/scripts/validate.sh (agent mode)|$FACTORY_VALIDATE|$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"; do
|
|
IFS='|' read -r label script arg <<< "$probe"
|
|
set +e
|
|
ctl_out="$(bash "$script" "$arg" 2>&1)"
|
|
ctl_rc=$?
|
|
set -e
|
|
if [[ $ctl_rc -eq 0 ]]; then
|
|
pass "$label exits 0 on a clean subject with python3 and PyYAML available"
|
|
else
|
|
fail "$label failed a clean subject (exit $ctl_rc): $ctl_out"
|
|
fi
|
|
done
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 3. verbose: true on the skill-size-check hook
|
|
# ---------------------------------------------------------------------------
|
|
# pre-commit prints nothing for a passing hook, so dropping the flag silences
|
|
# the SUGGESTION tier without failing anything. The published
|
|
# .pre-commit-hooks.yaml that once carried a second copy of this hook was
|
|
# retired (ADR-0014, 2026-09-16 amendment); if it returns, assert it here too.
|
|
echo ""
|
|
echo "--- the skill-size-check hook declares verbose: true ---"
|
|
VERBOSE_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
|
|
import os
|
|
import sys
|
|
|
|
import yaml
|
|
|
|
root = sys.argv[1]
|
|
|
|
|
|
def emit(status, msg):
|
|
print("%s\t%s" % (status, msg))
|
|
|
|
|
|
# Repo config: nested repos[].hooks[].
|
|
path = os.path.join(root, '.pre-commit-config.yaml')
|
|
try:
|
|
with open(path, encoding='utf-8') as fh:
|
|
cfg = yaml.safe_load(fh) or {}
|
|
except Exception as exc:
|
|
emit('FAIL', '.pre-commit-config.yaml did not parse: %s' % exc)
|
|
cfg = {}
|
|
found = None
|
|
for repo in cfg.get('repos') or []:
|
|
for hook in (repo.get('hooks') or []):
|
|
if hook.get('id') == 'skill-size-check':
|
|
found = hook
|
|
if found is None:
|
|
emit('FAIL', '.pre-commit-config.yaml declares no hook with id skill-size-check')
|
|
elif found.get('verbose') is True:
|
|
emit('PASS', '.pre-commit-config.yaml: skill-size-check is verbose: true')
|
|
else:
|
|
emit('FAIL', '.pre-commit-config.yaml: skill-size-check has verbose=%r — '
|
|
'pre-commit prints nothing for a passing hook, so every '
|
|
'ADR-0020 SUGGESTION is swallowed' % (found.get('verbose'),))
|
|
PY
|
|
)"
|
|
while IFS=$'\t' read -r status msg; do
|
|
[[ -n "$status" ]] || continue
|
|
if [[ "$status" == PASS ]]; then
|
|
pass "$msg"
|
|
else
|
|
fail "$msg"
|
|
fi
|
|
done <<< "$VERBOSE_REPORT"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 4. Every repo-authored hook is still WIRED, at the stage it claims
|
|
# ---------------------------------------------------------------------------
|
|
# Same failure mode as assertion 3, one level up. Every gate this repo owns is
|
|
# driven in tests by PATH -- tests/test-skill-version-bump.sh runs
|
|
# scripts/check-skill-version-bump.sh directly -- so the script staying correct
|
|
# and the hook still existing are independent facts, and only the first one was
|
|
# pinned. Deleting the `run-tests` block from .pre-commit-config.yaml (a routine
|
|
# merge-conflict casualty) stops the entire suite from running at pre-push
|
|
# forever, and `bash tests/run-tests.sh --strict` still prints all green: the
|
|
# suite cannot notice that nothing invokes it. Verified on a scratch copy --
|
|
# eight repo-authored hook blocks were deleted at once and every suite stayed
|
|
# green.
|
|
#
|
|
# So: id + entry + stages, for the repo-authored hooks only. "Repo-authored"
|
|
# is derived from the config (every hook under a `repo: local` entry), not
|
|
# hardcoded, and the derived set is then compared against the expected list
|
|
# below -- a new local hook that nobody pinned is itself a failure.
|
|
#
|
|
# Deliberately NOT pinned: the stock upstream hooks (gitleaks, check-yaml,
|
|
# pretty-format-json, shellcheck, conventional-pre-commit, the `repo: meta`
|
|
# pair) and every `rev:`. Those are somebody else's contract; a rev bump must
|
|
# not churn this test.
|
|
echo ""
|
|
echo "--- every repo-authored hook is wired, with the entry and stages it claims ---"
|
|
WIRING_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
|
|
import os
|
|
import re
|
|
import sys
|
|
|
|
import yaml
|
|
|
|
root = sys.argv[1]
|
|
|
|
|
|
def emit(status, msg):
|
|
print("%s\t%s" % (status, msg))
|
|
|
|
|
|
# The expected wiring of every hook this repo authors: id -> (entry, stages).
|
|
# ADDING OR REMOVING A REPO-AUTHORED HOOK DELIBERATELY REQUIRES EDITING THIS
|
|
# LIST. That is the point -- the list is the second party to the agreement, so a
|
|
# hook cannot leave .pre-commit-config.yaml without someone saying so here.
|
|
# Upstream hooks are absent on purpose and must stay absent.
|
|
EXPECTED = {
|
|
# pre-push
|
|
'run-tests': (
|
|
'bash tests/run-tests.sh --strict', ['pre-push']),
|
|
'check-executables-allow-sync': (
|
|
'bash scripts/check-executables-allow-sync.sh', ['pre-push']),
|
|
'apm-audit-ci': (
|
|
'scripts/apm-audit-ci.sh', ['pre-push']),
|
|
'check-apm-agents-valid': (
|
|
'bash scripts/check-apm-agents-valid.sh', ['pre-push']),
|
|
'apm-pack-check-clean': (
|
|
'apm pack --check-versions --check-clean --dry-run', ['pre-push']),
|
|
'check-scope-walkup-sync': (
|
|
'bash scripts/check-scope-walkup-sync.sh', ['pre-push']),
|
|
'check-skill-version-bump': (
|
|
'bash scripts/check-skill-version-bump.sh', ['pre-push']),
|
|
'validate-marketplace': (
|
|
'claude plugin validate --strict .claude-plugin/marketplace.json',
|
|
['pre-push']),
|
|
# pre-commit
|
|
'skill-size-check': (
|
|
'scripts/skill-size-check.sh', ['pre-commit']),
|
|
'check-rtk-prefix': (
|
|
'scripts/check-rtk-prefix.sh', ['pre-commit']),
|
|
'vale-audit-prefilter-skill': (
|
|
'plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh',
|
|
['pre-commit']),
|
|
'vale-audit-prefilter-agent': (
|
|
'plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh',
|
|
['pre-commit']),
|
|
}
|
|
|
|
path = os.path.join(root, '.pre-commit-config.yaml')
|
|
try:
|
|
with open(path, encoding='utf-8') as fh:
|
|
cfg = yaml.safe_load(fh) or {}
|
|
except Exception as exc:
|
|
emit('FAIL', '.pre-commit-config.yaml did not parse: %s' % exc)
|
|
sys.exit(0)
|
|
|
|
local = {}
|
|
duplicates = []
|
|
for repo in cfg.get('repos') or []:
|
|
if repo.get('repo') != 'local':
|
|
continue
|
|
for hook in (repo.get('hooks') or []):
|
|
hid = hook.get('id')
|
|
if hid in local:
|
|
duplicates.append(hid)
|
|
local[hid] = hook
|
|
|
|
# Non-vacuity, first. Every assertion below iterates the derived set, so an
|
|
# empty one would report nothing and the section would pass having checked
|
|
# nothing at all -- the same shape of silent hole this whole file exists to
|
|
# close.
|
|
if not EXPECTED:
|
|
emit('FAIL', 'the expected repo-authored hook list is empty — '
|
|
'every wiring assertion below is vacuous')
|
|
sys.exit(0)
|
|
if not local:
|
|
emit('FAIL', '.pre-commit-config.yaml declares no `repo: local` hooks at '
|
|
'all — every gate this repo authors has been unwired, or the '
|
|
'config moved and this assertion is now measuring nothing')
|
|
sys.exit(0)
|
|
emit('PASS', '.pre-commit-config.yaml declares %d repo-authored (`repo: local`) '
|
|
'hooks — the wiring assertions below are not vacuous' % len(local))
|
|
|
|
if duplicates:
|
|
emit('FAIL', '.pre-commit-config.yaml declares duplicate local hook ids '
|
|
'(%s) — pre-commit runs one of them and the other is dead '
|
|
'config' % ', '.join(sorted(set(duplicates))))
|
|
|
|
# The set, before the per-hook detail: a deleted block shows up here as a
|
|
# missing id even if nothing else in the file changed.
|
|
missing = sorted(set(EXPECTED) - set(local))
|
|
extra = sorted(set(local) - set(EXPECTED))
|
|
if missing or extra:
|
|
parts = []
|
|
if missing:
|
|
parts.append('NOT WIRED in .pre-commit-config.yaml: %s' % ', '.join(missing))
|
|
if extra:
|
|
parts.append('wired but not pinned in this test: %s' % ', '.join(extra))
|
|
emit('FAIL', 'the repo-authored hook set has drifted — %s. A hook that '
|
|
'leaves the config stops running at push time while every '
|
|
'test stays green; a hook that arrives unpinned can leave '
|
|
'again unnoticed. Fix the config, or update EXPECTED in '
|
|
'tests/test-adr0020-contract.sh deliberately.' % '; '.join(parts))
|
|
else:
|
|
emit('PASS', '.pre-commit-config.yaml wires exactly the %d expected '
|
|
'repo-authored hooks, no more and no fewer' % len(EXPECTED))
|
|
|
|
# Per hook: the entry that runs and the stage it runs at. Both are single
|
|
# tokens whose loss is invisible -- an entry repointed at a path that no longer
|
|
# exists makes pre-commit fail loudly, but an entry repointed at a DIFFERENT
|
|
# real script does not, and a hook moved off pre-push simply never fires.
|
|
entry_paths = []
|
|
for hid in sorted(EXPECTED):
|
|
hook = local.get(hid)
|
|
if hook is None:
|
|
continue # already reported as missing above
|
|
exp_entry, exp_stages = EXPECTED[hid]
|
|
got_entry = hook.get('entry')
|
|
got_stages = hook.get('stages')
|
|
problems = []
|
|
if got_entry != exp_entry:
|
|
problems.append('entry is %r, expected %r' % (got_entry, exp_entry))
|
|
if list(got_stages or []) != exp_stages:
|
|
problems.append('stages is %r, expected %r — a hook at the wrong stage '
|
|
'(or at none) never fires' % (got_stages, exp_stages))
|
|
if problems:
|
|
emit('FAIL', 'hook %s: %s' % (hid, '; '.join(problems)))
|
|
else:
|
|
emit('PASS', 'hook %s runs `%s` at %s'
|
|
% (hid, exp_entry, ','.join(exp_stages)))
|
|
# Collect the in-repo script/manifest paths the pinned entry names, so the
|
|
# pin cannot agree with a config that points at nothing.
|
|
for token in re.split(r'\s+', exp_entry):
|
|
token = token.strip('\'"')
|
|
if re.fullmatch(r'[A-Za-z0-9_.\-/]+\.(sh|json)', token):
|
|
entry_paths.append((hid, token))
|
|
|
|
# Every pinned entry path exists on disk. Without this, EXPECTED and the config
|
|
# could agree perfectly on a script that was deleted.
|
|
if not entry_paths:
|
|
emit('FAIL', 'no pinned entry named an in-repo script or manifest — the '
|
|
'existence check below iterated zero times and proved nothing')
|
|
else:
|
|
broken = ['%s -> %s' % (hid, token)
|
|
for hid, token in entry_paths
|
|
if not os.path.exists(os.path.join(root, token))]
|
|
if broken:
|
|
emit('FAIL', 'pinned hook entries point at files that do not exist: %s'
|
|
% ', '.join(broken))
|
|
else:
|
|
emit('PASS', 'all %d in-repo files named by a pinned hook entry exist '
|
|
'on disk' % len(entry_paths))
|
|
PY
|
|
)"
|
|
while IFS=$'\t' read -r status msg; do
|
|
[[ -n "$status" ]] || continue
|
|
if [[ "$status" == PASS ]]; then
|
|
pass "$msg"
|
|
else
|
|
fail "$msg"
|
|
fi
|
|
done <<< "$WIRING_REPORT"
|
|
|
|
echo ""
|
|
echo "Results: $PASS passed, $FAIL failed"
|
|
[[ $FAIL -eq 0 ]]
|