Files
holocron/tests/test-adr0020-contract.sh
Defame1297 ef27c9751a refactor(gates): source the boundary resolver into skill-size-check
Why: scripts/skill-size-check.sh embedded a byte-identical 1,061-line copy
of the ADR-0020 boundary resolver only because it was also exported
through .pre-commit-hooks.yaml, whose consumers could not reach a file
inside the plugin. 4de5b6b retired that export, so the hook now runs only
in this repo and can source factory-audit's lib-boundary-resolver.sh like
validate.sh does. One copy removes the edit-one-paste-the-other hazard.

Implementation Notes:
- The hook's Python program is assembled from its own preamble, the
  library's resolver and its own checks, read from quoted here-docs. The
  assembled program matches the old one line for line except one comment,
  and the hook's stdout, stderr and exit code are identical over every
  corpus SKILL.md and the 26 differential-suite fixtures.
- The hook fails closed, naming the library, when it is missing or
  defines no resolver.
- test-adr0020-contract.sh assertion 1 now pins the single copy: one
  marker pair in the library, none in the hook, fail-closed on a missing
  or gutted library, and a sentinel planted in a copied library that must
  appear in the hook's output. 1a expects exactly one authority. 27 -> 29
  passes.
- ADR-0020 and ADR-0025 carry dated amendments; gates.md and the
  library, hook and mode-library comments no longer describe two copies.
- factory-audit is new on this branch, so the version-bump gate exempts
  it; kyberforge is already at 2.0.0 against main's 1.6.2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 13:25:16 +00:00

686 lines
34 KiB
Bash
Executable File

#!/usr/bin/env bash
# Regression test for the STRUCTURAL claims the ADR-0020 gate family makes about
# itself. None of them was pinned anywhere before this file, and each one fails
# silently — which is the whole reason they need a test rather than a comment:
#
# 1. "ONE resolver, and both of its consumers run it." The ADR-0020 boundary
# resolver has exactly one copy, factory-audit's sourced
# lib-boundary-resolver.sh. scripts/skill-size-check.sh used to embed a
# second, byte-identical copy, because it was also exported through a
# published hook manifest whose consumers could not reach a file inside the
# plugin; 4de5b6b retired that export (ADR-0014), so the hook now sources
# the library too. What must not fail silently: the hook growing its own
# copy back, the library being gutted, or the hook no longer running the
# library's text at all. So this asserts the library is real content, the
# hook carries no marker pair, the hook fails closed without the library,
# and — by a sentinel planted in a copied library — that the text the hook
# executes IS the library's.
# 1a. The library is the ONLY authority, and validate.sh sources it in both
# mode branches — the same authority checks 1b makes for the parser.
# 1b. The same claim, one directory over, for the Contributing-files parser.
# That one was worse: it was embedded in both validate-provenance.sh copies,
# the agent-audit copy's docstring ASSERTED it was kept behaviourally
# identical to skill-audit's, and the two had drifted (cosmetically, at
# 484357a, re-unified at 598a7c3 — nothing had caught it). ADR-0025
# removed the second copy, so there is no longer a pair to hash — but
# deleting the assertion would restore exactly the condition that let the
# drift happen, so it is CONVERTED rather than dropped: it now pins that
# lib-contributing-files.sh is the single authority (one marker pair), that
# validate-provenance.sh actually SOURCES it, and that nobody has re-inlined
# the parser into a mode library or anywhere else in the tree.
# 2. Both interpreter preflights, in both scripts and both of the merged
# entry point's modes. python3 and PyYAML are declared HARD dependencies
# precisely so a missing one cannot turn into a vacuous pass, and the two
# are checked separately so the message names the thing to install rather
# than the wrong one. factory-audit/scripts/validate.sh detects its mode
# first and only then calls the preflight, which lives in the mode library
# it sources (lib-checks-skill.sh / lib-checks-agent.sh) — so each mode is
# probed with its own target shape, not just one of them.
# 3. `verbose: true` on the skill-size-check hook. It is the ENTIRE delivery
# mechanism for the SUGGESTION tier: pre-commit prints nothing at all for a
# passing hook, and a SUGGESTION deliberately does not fail, so dropping
# one word from the config silences the tier ADR-0020 depends on while
# every test and every hook still reports green.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
HOOK="$REPO_ROOT/scripts/skill-size-check.sh"
FACTORY_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit"
# ADR-0025: one auto-detecting entry point, and one sourced copy of the resolver
# behind it, which the root hook sources as well. The entry point is what the
# preflight assertions run; the library is what assertion 1 inspects.
FACTORY_VALIDATE="$FACTORY_AUDIT/scripts/validate.sh"
FACTORY_RESOLVER="$FACTORY_AUDIT/scripts/lib-boundary-resolver.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
TMPDIR_T="$(mktemp -d)"
trap 'rm -rf "$TMPDIR_T"' EXIT
BEGIN_MARKER='# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER ====='
END_MARKER='# ===== END ADR-0020 SHARED BOUNDARY RESOLVER ====='
# ---------------------------------------------------------------------------
# 1. The resolver has one copy, and the root hook runs it
# ---------------------------------------------------------------------------
# One copy since the 2026-09-16 change: factory-audit's lib-boundary-resolver.sh.
# scripts/skill-size-check.sh sources it rather than embedding a second copy, so
# there is no pair left to hash. What replaces the hash is the set of ways a
# single sourced copy can still fail quietly.
echo ""
echo "--- the ADR-0020 resolver has one copy, and skill-size-check.sh runs it ---"
# (i) The library carries exactly one well-formed marker pair around real
# content. An unbalanced pair makes every extraction measure the wrong span, and
# an emptied-out block would still "load" while enforcing nothing. The resolver
# is ~1,060 lines; 100 is a floor low enough never to need maintenance and high
# enough that a gutted block cannot sneak past.
if [[ ! -f "$FACTORY_RESOLVER" ]]; then
fail "resolver library not found: ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
else
b="$(grep -cFx "$BEGIN_MARKER" "$FACTORY_RESOLVER" || true)"
e="$(grep -cFx "$END_MARKER" "$FACTORY_RESOLVER" || true)"
if [[ "$b" == "1" && "$e" == "1" ]]; then
pass "${FACTORY_RESOLVER#"$REPO_ROOT/"} carries exactly one BEGIN and one END marker"
span="$(sed -n "/^${BEGIN_MARKER}\$/,/^${END_MARKER}\$/p" "$FACTORY_RESOLVER" | wc -l | tr -d ' ')"
if [[ "$span" -gt 100 ]]; then
pass "the resolver span is $span lines — real content, not an empty block"
else
fail "the resolver span is only $span lines — a gutted block would load and resolve nothing"
fi
else
fail "${FACTORY_RESOLVER#"$REPO_ROOT/"} has $b BEGIN and $e END markers, expected 1 and 1"
fi
fi
# (ii) The hook carries no copy of its own. A column-0 marker line in the hook
# is the shape the old embedded copy had, and the shape a paste-back would have.
if [[ ! -f "$HOOK" ]]; then
fail "hook not found: ${HOOK#"$REPO_ROOT/"}"
else
hb="$(grep -cFx "$BEGIN_MARKER" "$HOOK" || true)"
he="$(grep -cFx "$END_MARKER" "$HOOK" || true)"
if [[ "$hb" == "0" && "$he" == "0" ]]; then
pass "${HOOK#"$REPO_ROOT/"} carries no resolver marker lines — it has not grown its own copy back"
else
fail "${HOOK#"$REPO_ROOT/"} carries $hb BEGIN and $he END marker lines — a second copy of the resolver is back in the hook"
fi
fi
# (iii) and (iv) run the hook from a scratch tree that mirrors the two paths it
# depends on, so the real library is never touched. The scratch hook is a copy
# of the real one; its library is either absent, gutted, or the real library
# with a sentinel planted inside the resolver block.
SSC_TREE="$TMPDIR_T/ssc-tree"
SSC_LIB_DIR="$SSC_TREE/plugins/kyberforge/.apm/skills/factory-audit/scripts"
mkdir -p "$SSC_TREE/scripts" "$SSC_LIB_DIR" "$TMPDIR_T/ssc-skill/probe-skill"
cp "$HOOK" "$SSC_TREE/scripts/skill-size-check.sh"
printf -- '---\nname: probe-skill\ndescription: Use when probing the resolver wiring.\nmetadata:\n version: "0.1.0"\n---\n\n## Step 1\n\nDo the thing.\n' \
> "$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
PROBE="$TMPDIR_T/ssc-skill/probe-skill/SKILL.md"
run_scratch_hook() {
local rc=0
SSC_OUT="$(bash "$SSC_TREE/scripts/skill-size-check.sh" "$PROBE" 2>&1)" || rc=$?
SSC_RC=$rc
}
# (iii) Fail closed: no library, then a library that defines nothing.
rm -f "$SSC_LIB_DIR/lib-boundary-resolver.sh"
run_scratch_hook
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"boundary resolver library was not found"* ]]; then
pass "with the library missing, the hook exits $SSC_RC and names the missing library — not a vacuous pass"
else
fail "with the library missing, the hook exited $SSC_RC without naming it: $SSC_OUT"
fi
printf '# gutted\n' > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
run_scratch_hook
if [[ "$SSC_RC" -ne 0 && "$SSC_OUT" == *"did not define the ADR-0020 boundary resolver"* ]]; then
pass "with a library that defines no resolver, the hook exits $SSC_RC and says so"
else
fail "with a gutted library, the hook exited $SSC_RC without saying so: $SSC_OUT"
fi
# (iv) The text the hook executes IS the library's. A sentinel print planted
# just after the BEGIN marker of a copied library must appear in the hook's
# output. Without this, a hook that sourced the library but ran some other
# program would pass (i)-(iii).
SENTINEL="ADR0020-RESOLVER-SENTINEL-$$"
if [[ -f "$FACTORY_RESOLVER" ]]; then
awk -v m="$BEGIN_MARKER" -v s="$SENTINEL" '{ print } $0 == m { print "print(\"" s "\")" }' \
"$FACTORY_RESOLVER" > "$SSC_LIB_DIR/lib-boundary-resolver.sh"
if [[ "$(grep -cF "$SENTINEL" "$SSC_LIB_DIR/lib-boundary-resolver.sh" || true)" -ne 1 ]]; then
fail "fixture check: the sentinel was not planted exactly once in the copied library — the case below would prove nothing"
else
run_scratch_hook
if [[ "$SSC_RC" -eq 0 && "$SSC_OUT" == *"$SENTINEL"* ]]; then
pass "a sentinel planted in the library's resolver block runs inside the hook — the hook executes the library's text"
else
fail "the hook did not run the library's resolver text (rc=$SSC_RC, sentinel absent from output): $SSC_OUT"
fi
fi
fi
# ---------------------------------------------------------------------------
# 1a. The resolver library is the ONLY authority, and validate.sh sources it
# ---------------------------------------------------------------------------
# Assertion 1 says nothing about a copy somewhere else in the tree, and nothing
# about whether validate.sh runs the library. Assertion 1b
# pins both of those for the Contributing-files parser; the resolver is the same
# defect class and gets the same two checks:
#
# a. validate.sh actually SOURCES lib-boundary-resolver.sh, in BOTH mode
# branches — asserted inside each arm of `case "$MODE" in`, not by counting
# source lines file-wide, because a count cannot see a branch. A library
# that is identical, unique and never sourced is a copy that has quietly
# been replaced by an inline one — and assertion 1 would stay green over it.
# b. Nothing has re-inlined it. The BEGIN marker and a def unique to the
# resolver (`_authoring_root`) appear in exactly one file,
# lib-boundary-resolver.sh, and nowhere else under the tree. A mode library
# or a root script that grows a "just this once" copy would otherwise
# escape assertion 1 entirely, because 1 inspects only the files it names.
echo ""
echo "--- the ADR-0020 resolver has exactly one authority, and validate.sh sources it ---"
# Deployed and vendored trees are generated copies, not authorities: .claude/ is
# apm install output, apm_modules/ is resolved dependencies, build/ is release
# artifacts. This file is excluded because it necessarily quotes what it
# searches for. Markdown is excluded because an authority is code that runs:
# ADR-0025 and gates.md quote these needles to document this very check, and a
# prose mention is not a re-inlined copy.
SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
tree_scan() {
grep -rlF --binary-files=without-match \
--exclude-dir=.git --exclude-dir=build --exclude-dir=.claude \
--exclude-dir=apm_modules --exclude-dir=node_modules --exclude='*.md' \
-- "$1" "$REPO_ROOT" 2>/dev/null | grep -vFx "$SELF" | sort || true
}
# (a) Sourced, once per mode branch — asserted PER BRANCH, not by counting.
#
# This used to be a file-wide `grep -c ... >= 2`, and a count cannot see a
# branch. Proven by mutation: moving the agent arm's source line out and
# duplicating the skill arm's leaves the count at 2 and the old assertion
# printed PASS while claiming "in both mode branches" — so a branch that lost
# its resolver gracefully (a defaulted variable, a `set +u` region, an optional
# resolver) stayed green and lying. The claim is per-branch, so the check is
# too: each arm of `case "$MODE" in` must source the resolver inside its own
# body. The mutation that fooled the old form is run against the new one below,
# because an assertion about branches that has never been shown to fail on a
# count-preserving branch edit is exactly the assertion that was here before.
RESOLVER_SOURCE_RE='^[[:space:]]*(\.|source)[[:space:]]+.*lib-boundary-resolver\.sh'
# The body of one arm of the `case "$MODE" in` block: everything between the
# arm's label and its `;;`. Structural, not positional — the file is edited
# often and line numbers or a whole-file hash would pin the wrong thing.
mode_arm_body() {
awk -v arm="$2" '
!incase && $0 ~ /^[[:space:]]*case[[:space:]]+"?\$MODE"?[[:space:]]+in[[:space:]]*$/ { incase = 1; next }
incase && $0 ~ /^[[:space:]]*esac([[:space:]]|$)/ { incase = 0; next }
incase && !inarm && $0 ~ "^[[:space:]]*\\(?" arm "\\)[[:space:]]*$" { inarm = 1; next }
inarm && $0 ~ /^[[:space:]]*;;[[:space:]]*$/ { inarm = 0; next }
inarm { print }
' "$1"
}
# Every mode arm sources the resolver exactly once, inside its own body.
# Returns 0/1 and leaves the reason in ARM_DETAIL, so the same function can be
# run against the real file and against the mutant below.
ARM_DETAIL=""
check_resolver_per_arm() {
local file="$1" arm body n rc=0
ARM_DETAIL=""
for arm in skill agent; do
body="$(mode_arm_body "$file" "$arm")"
if [[ -z "$body" ]]; then
ARM_DETAIL+="the $arm) arm of the case \"\$MODE\" block was not found or is empty; "
rc=1
continue
fi
n="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$body" || true)"
if [[ "$n" -eq 0 ]]; then
ARM_DETAIL+="the $arm) arm never sources lib-boundary-resolver.sh, so that mode runs some other resolver or none; "
rc=1
elif [[ "$n" -ne 1 ]]; then
ARM_DETAIL+="the $arm) arm sources lib-boundary-resolver.sh $n times; "
rc=1
fi
done
return $rc
}
if [[ ! -f "$FACTORY_VALIDATE" ]]; then
fail "entry point not found: ${FACTORY_VALIDATE#"$REPO_ROOT/"}"
else
RESOLVER_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$FACTORY_VALIDATE" || true)"
if check_resolver_per_arm "$FACTORY_VALIDATE"; then
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"}: the skill) and agent) arms of its case \"\$MODE\" block EACH source lib-boundary-resolver.sh inside their own body, exactly once"
else
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} does not source lib-boundary-resolver.sh once per mode arm: ${ARM_DETAIL%; }"
fi
# Secondary, and deliberately not the verdict: with one source per arm proven
# above, a file-wide total of exactly 2 says there is no third source line
# sitting outside both arms.
if [[ "$RESOLVER_SOURCES" -eq 2 ]]; then
pass "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries exactly 2 resolver source lines file-wide — the two arm sources and nothing else"
else
fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries $RESOLVER_SOURCES resolver source lines file-wide, expected the 2 that belong to the mode arms"
fi
# Mutation self-test. The mutation is the one the retired count could not
# see: the agent arm's source line is removed and the skill arm's duplicated,
# so the FILE-WIDE COUNT IS UNCHANGED. Written into a copy; the real file is
# never touched.
MUT_DIR="$(mktemp -d "$TMPDIR_T/resolver-mutant.XXXXXX")"
MUT="$MUT_DIR/validate.sh"
cp "$FACTORY_VALIDATE" "$MUT"
if python3 - "$MUT" <<'PY'
import re
import sys
path = sys.argv[1]
with open(path, encoding='utf-8') as fh:
lines = fh.read().split('\n')
case_re = re.compile(r'^\s*case\s+"?\$MODE"?\s+in\s*$')
esac_re = re.compile(r'^\s*esac(\s|$)')
term_re = re.compile(r'^\s*;;\s*$')
src_re = re.compile(r'^\s*(\.|source)\s+.*lib-boundary-resolver\.sh')
starts = [i for i, l in enumerate(lines) if case_re.match(l)]
assert len(starts) == 1, 'expected exactly one `case "$MODE" in`, found %d' % len(starts)
ci = starts[0]
ends = [i for i in range(ci + 1, len(lines)) if esac_re.match(lines[i])]
assert ends, 'the case "$MODE" block has no esac'
ei = ends[0]
def arm_sources(name):
for i in range(ci + 1, ei):
if re.match(r'^\s*\(?%s\)\s*$' % name, lines[i]):
for j in range(i + 1, ei):
if term_re.match(lines[j]):
return [k for k in range(i + 1, j) if src_re.match(lines[k])]
raise AssertionError('the %s) arm has no ;;' % name)
raise AssertionError('no %s) arm in the case "$MODE" block' % name)
skill = arm_sources('skill')
agent = arm_sources('agent')
assert len(skill) == 1 and len(agent) == 1, \
'expected one resolver source per arm before mutating, got skill=%d agent=%d' % (len(skill), len(agent))
out = list(lines)
del out[agent[0]] # the agent arm loses its resolver ...
out.insert(skill[0], lines[skill[0]]) # ... and the skill arm gains a duplicate
with open(path, 'w', encoding='utf-8') as fh:
fh.write('\n'.join(out))
PY
then
MUT_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$MUT" || true)"
MUT_AGENT="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$(mode_arm_body "$MUT" agent)" || true)"
# Guard the fixture before trusting its verdict: the mutation must have
# actually emptied the agent arm AND left the file-wide count where it was,
# or the case below proves nothing about the defect it stands for.
if [[ "$MUT_SOURCES" -eq "$RESOLVER_SOURCES" && "$MUT_AGENT" -eq 0 ]]; then
pass "fixture check: the mutant's agent arm sources no resolver while the file-wide count is still $MUT_SOURCES — the retired 'count >= 2' assertion would have passed it"
else
fail "the resolver mutation did not land as intended (file-wide $MUT_SOURCES vs $RESOLVER_SOURCES, agent arm $MUT_AGENT) — the case below would prove nothing"
fi
if check_resolver_per_arm "$MUT"; then
fail "the per-arm check PASSED a validate.sh whose agent arm has no resolver source — it is still counting, not reading branches"
else
pass "the per-arm check FAILS the count-preserving mutant (${ARM_DETAIL%; }) — it reads the branches, not a total"
fi
else
fail "could not build the resolver mutation fixture — validate.sh's case \"\$MODE\" structure is not the shape this self-test knows, so the per-arm check is unproven"
fi
fi
# (b) Exactly the one authority, for both spellings of a copy.
EXPECTED_RESOLVERS="$FACTORY_RESOLVER"
check_resolver_authorities() {
local label="$1" needle="$2"
local found
found="$(tree_scan "$needle")"
if [[ "$found" == "$EXPECTED_RESOLVERS" ]]; then
pass "$label appears in exactly the one resolver authority and nowhere else"
elif [[ -z "$found" ]]; then
fail "$label was found in NO file at all — the scan is looking for the wrong text"
else
fail "$label appears in an unexpected set of files, so the resolver has been re-inlined or lost: $(echo "$found" | tr '\n' ' ')— expected exactly ${FACTORY_RESOLVER#"$REPO_ROOT/"}"
fi
}
check_resolver_authorities "the resolver's BEGIN marker" "$BEGIN_MARKER"
check_resolver_authorities "a 'def _authoring_root' definition" "def _authoring_root("
# ---------------------------------------------------------------------------
# 1b. The Contributing-files parser has exactly ONE authority, and it is sourced
# ---------------------------------------------------------------------------
# Same defect class, one directory over. parse_contributing_files() used to be
# embedded in both validate-provenance.sh copies for the same reason the resolver
# was once embedded in several scripts, and until this assertion existed the agent-audit copy's
# docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's
# copy" — an invariant nothing checked, and the two did drift into different
# spellings of the bullet loop at 484357a. That drift happened to be
# behaviour-neutral and was re-unified by hand at 598a7c3; the next one need
# not be. The parser decides whether checks 4,
# 5 and 8 run at all, so a one-sided edit disables a check in one script while
# every other test stays green.
#
# ADR-0025 merged the two skills, so there is now ONE copy and nothing left to
# hash against. That does NOT retire the assertion: a byte-identity check over a
# single copy is vacuous, and deleting it outright would restore exactly the
# condition that allowed the original drift — a parser with no pinned authority.
# So the claim is CONVERTED, not dropped. It is the same claim ("the parser has
# exactly one authority") stated against the new structure:
#
# a. factory-audit/scripts/lib-contributing-files.sh exists and carries exactly
# one BEGIN/END marker pair, over a span of real content.
# b. validate-provenance.sh actually SOURCES it. A library nobody sources is a
# copy that has silently been replaced by an inline one somewhere else.
# c. Nothing has re-inlined it. No other file in the tree carries the marker
# pair, and no other file defines parse_contributing_files. This is the part
# that fails if the merge is ever partially reverted, or if a mode library
# grows its own "just this once" copy — which is precisely how the drift
# this assertion was written for got in.
echo ""
echo "--- the Contributing-files parser has exactly one authority, and validate-provenance.sh sources it ---"
CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ====='
CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER ====='
CF_LIB="$FACTORY_AUDIT/scripts/lib-contributing-files.sh"
FACTORY_PROV="$FACTORY_AUDIT/scripts/validate-provenance.sh"
# (a) One library, one well-formed marker pair, over real content.
CF_MARKERS_OK=true
if [[ ! -f "$CF_LIB" ]]; then
fail "the single parser authority is missing: ${CF_LIB#"$REPO_ROOT/"}"
CF_MARKERS_OK=false
else
b="$(grep -cFx "$CF_BEGIN" "$CF_LIB" || true)"
e="$(grep -cFx "$CF_END" "$CF_LIB" || true)"
if [[ "$b" == "1" && "$e" == "1" ]]; then
pass "${CF_LIB#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker"
else
fail "${CF_LIB#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1"
CF_MARKERS_OK=false
fi
fi
if ! $CF_MARKERS_OK; then
fail "skipping the parser content check — the marker pair is not well-formed, so any extraction would measure the wrong span"
else
# A span gutted down to its docstring would still satisfy every structural
# check above while enforcing nothing, exactly as for the resolver. The parser
# block is ~93 lines; 40 is a floor low enough never to need maintenance and
# high enough that a gutted block cannot sneak past.
CF_LINECOUNT="$(sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$CF_LIB" | wc -l | tr -d ' ')"
if [[ "$CF_LINECOUNT" -gt 40 ]]; then
pass "the extracted parser block is $CF_LINECOUNT lines — a real parser, not an empty or docstring-only span"
else
fail "the extracted parser block is only $CF_LINECOUNT lines — a gutted span asserts nothing"
fi
fi
# (b) The one entry point sources it. Without this, (a) and (c) are satisfied by
# a library that is present, unique and entirely unused.
if [[ ! -f "$FACTORY_PROV" ]]; then
fail "entry point not found: ${FACTORY_PROV#"$REPO_ROOT/"}"
elif grep -Eq '^[[:space:]]*(\.|source)[[:space:]]+.*lib-contributing-files\.sh' "$FACTORY_PROV"; then
pass "${FACTORY_PROV#"$REPO_ROOT/"} sources lib-contributing-files.sh — the single copy is the one that actually runs"
else
fail "${FACTORY_PROV#"$REPO_ROOT/"} never sources lib-contributing-files.sh — the library is dead code and the parser that runs is some other copy"
fi
# (c) Nobody re-inlined it. Both spellings are scanned: the marker pair (a
# copy-paste of the block) and a second `def parse_contributing_files` (a
# re-implementation that skipped the markers). tree_scan (defined in 1a)
# excludes the same generated trees and this file.
cf_scan() { tree_scan "$1"; }
check_sole_authority() {
local label="$1" needle="$2"
local found extra
found="$(cf_scan "$needle")"
extra="$(printf '%s\n' "$found" | grep -vFx "$CF_LIB" | grep -v '^$' || true)"
if [[ -z "$found" ]]; then
fail "$label was found in NO file at all — the parser authority has vanished, or the scan is looking for the wrong text"
elif [[ -n "$extra" ]]; then
fail "$label appears outside the single authority, so the parser has been re-inlined: $(echo "$extra" | tr '\n' ' ')— delete the copy and source lib-contributing-files.sh instead"
else
pass "$label appears only in ${CF_LIB#"$REPO_ROOT/"} — one authority, no re-inlined copies"
fi
}
check_sole_authority "the parser's BEGIN marker" "$CF_BEGIN"
check_sole_authority "a 'def parse_contributing_files' definition" "def parse_contributing_files("
# ---------------------------------------------------------------------------
# 2. Both interpreter preflights, in both scripts and both modes
# ---------------------------------------------------------------------------
# The two are checked separately on purpose: `python3 -c 'import yaml'` fails
# identically whether python3 is missing or PyYAML is, and naming the wrong one
# sends the reader to install the wrong thing.
#
# ADR-0025 merged the two validators into one auto-detecting entry point, but
# the preflight did NOT merge with them: validate.sh detects the mode first and
# then calls kyberforge_skill_preflight or kyberforge_agent_preflight from the
# mode library it sources. There are still two preflights, so both are still
# probed — once with a skill target and once with an agent target. Collapsing
# these to a single probe would leave one mode's preflight unpinned, and a mode
# whose preflight is gone reports a vacuous pass on a machine with no PyYAML.
REAL_PYTHON="$(command -v python3)"
# Absolute path, deliberately. The no-python3 fixture below replaces PATH
# wholesale, so a bare `bash` (or `/usr/bin/env bash`) would be resolved against
# that stripped PATH and die with "No such file or directory" before the script
# under test ever starts -- a 127 that looks like the preflight firing.
BASH_BIN="$(command -v bash)"
# A PATH that genuinely has no python3 on it. Built by symlinking the handful of
# binaries the three scripts touch before their own preflight rather than by
# hiding python3 from a full PATH, because there is no portable way to subtract
# one entry from a directory. `bash` is invoked by absolute path below so the
# interpreter itself does not have to be on this PATH.
NOPY_BIN="$TMPDIR_T/nopython-bin"
mkdir -p "$NOPY_BIN"
for b in awk cat cut dirname basename grep sed pwd rm mkdir tr; do
src="$(command -v "$b" 2>/dev/null || true)"
[[ -n "$src" ]] && ln -sf "$src" "$NOPY_BIN/$b"
done
# A python3 that runs but cannot import yaml. A shim on PATH re-execs the real
# interpreter with a PYTHONPATH entry holding a `yaml` module that raises on
# import; PYTHONPATH precedes site-packages on sys.path, so it shadows a real
# PyYAML install without touching it.
SHADOW="$TMPDIR_T/shadow"
mkdir -p "$SHADOW"
printf 'raise ImportError("PyYAML deliberately unavailable in this fixture")\n' \
> "$SHADOW/yaml.py"
NOYAML_BIN="$TMPDIR_T/noyaml-bin"
mkdir -p "$NOYAML_BIN"
cat > "$NOYAML_BIN/python3" <<EOF
#!/bin/sh
PYTHONPATH="$SHADOW\${PYTHONPATH:+:\$PYTHONPATH}" exec "$REAL_PYTHON" "\$@"
EOF
chmod +x "$NOYAML_BIN/python3"
# Sanity-check the two fixtures themselves before trusting any verdict they
# produce. A shim that silently still imports yaml would make every PyYAML
# assertion below pass for the wrong reason.
if PATH="$NOYAML_BIN:$PATH" python3 -c 'import yaml' 2>/dev/null; then
fail "the no-PyYAML shim does not actually shadow PyYAML — every PyYAML assertion below would be vacuous"
else
pass "fixture check: the no-PyYAML shim makes 'import yaml' fail while python3 still runs"
fi
if PATH="$NOPY_BIN" command -v python3 > /dev/null 2>&1; then
fail "the no-python3 PATH still resolves python3 — every python3 assertion below would be vacuous"
else
pass "fixture check: the no-python3 PATH resolves no python3"
fi
# A minimal, entirely clean subject for each script. The preflight must fire
# before any measurement, so the subject's own content is irrelevant — which is
# exactly what makes a clean one the right choice: nothing else can produce the
# non-zero exit these cases assert.
SUBJECT_SKILL_DIR="$TMPDIR_T/subject/my-skill"
mkdir -p "$SUBJECT_SKILL_DIR"
cat > "$SUBJECT_SKILL_DIR/SKILL.md" <<'EOF'
---
name: my-skill
description: A short valid description. Do not use for anything else.
metadata:
version: "1.0.0"
---
Do the thing.
EOF
SUBJECT_AGENT_ROOT="$TMPDIR_T/subject-agent"
mkdir -p "$SUBJECT_AGENT_ROOT/.apm/agents"
cat > "$SUBJECT_AGENT_ROOT/apm.yml" <<'EOF'
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" <<'EOF'
---
name: my-agent
description: A short valid description. Do not use for anything else.
---
You are a test agent. When invoked, do the thing.
EOF
# probe_preflight <label> <env-kind: nopython|noyaml> <expect-needle> <cmd...>
probe_preflight() {
local label="$1" kind="$2" needle="$3"
shift 3
local out status=0
set +e
if [[ "$kind" == nopython ]]; then
out="$(env -i PATH="$NOPY_BIN" HOME="$HOME" "$BASH_BIN" "$@" 2>&1)"
else
out="$(env PATH="$NOYAML_BIN:$PATH" "$BASH_BIN" "$@" 2>&1)"
fi
status=$?
set -e
if [[ $status -eq 0 ]]; then
fail "$label exited 0 — a missing hard dependency became a vacuous pass (output: ${out:-<empty>})"
elif [[ "$out" != *"$needle"* ]]; then
# The needle is the DIAGNOSTIC ("python3 is required"), not the bare word.
# Deleting the preflight entirely would still produce a non-zero exit and a
# message mentioning python3 -- bash's own "python3: command not found" --
# so a bare-word needle would go green on a script with no preflight at all.
fail "$label exited $status but never produced the '$needle' diagnostic (output: ${out:-<empty>})"
elif [[ "$kind" == nopython && "$out" == *PyYAML* ]]; then
fail "$label reported PyYAML when python3 itself is missing — that sends the reader to install the wrong thing (output: $out)"
else
pass "$label"
fi
}
echo ""
echo "--- a PATH with no python3 is a hard failure in both scripts and both modes, naming python3 ---"
probe_preflight "scripts/skill-size-check.sh reports missing python3" \
nopython "python3 is required" \
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing python3" \
nopython "python3 is required" \
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing python3" \
nopython "python3 is required" \
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
echo ""
echo "--- a python3 that cannot import yaml is a hard failure in both scripts and both modes, naming PyYAML ---"
probe_preflight "scripts/skill-size-check.sh reports missing PyYAML" \
noyaml "PyYAML is required" \
"$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md"
probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing PyYAML" \
noyaml "PyYAML is required" \
"$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR"
probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing PyYAML" \
noyaml "PyYAML is required" \
"$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"
# The control. Without it, "fails when the dependency is missing" is satisfied by
# a script that fails unconditionally, and the two cases above would be green on
# a gate that never runs at all.
echo ""
echo "--- control: with both dependencies present the same subjects pass ---"
# The label is carried explicitly because the two validate.sh probes now name the
# same script and differ only in the mode its target selects.
for probe in "scripts/skill-size-check.sh|$HOOK|$SUBJECT_SKILL_DIR/SKILL.md" \
"factory-audit/scripts/validate.sh (skill mode)|$FACTORY_VALIDATE|$SUBJECT_SKILL_DIR" \
"factory-audit/scripts/validate.sh (agent mode)|$FACTORY_VALIDATE|$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"; do
IFS='|' read -r label script arg <<< "$probe"
set +e
ctl_out="$(bash "$script" "$arg" 2>&1)"
ctl_rc=$?
set -e
if [[ $ctl_rc -eq 0 ]]; then
pass "$label exits 0 on a clean subject with python3 and PyYAML available"
else
fail "$label failed a clean subject (exit $ctl_rc): $ctl_out"
fi
done
# ---------------------------------------------------------------------------
# 3. verbose: true on the skill-size-check hook
# ---------------------------------------------------------------------------
# pre-commit prints nothing for a passing hook, so dropping the flag silences
# the SUGGESTION tier without failing anything. The published
# .pre-commit-hooks.yaml that once carried a second copy of this hook was
# retired (ADR-0014, 2026-09-16 amendment); if it returns, assert it here too.
echo ""
echo "--- the skill-size-check hook declares verbose: true ---"
VERBOSE_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
import os
import sys
import yaml
root = sys.argv[1]
def emit(status, msg):
print("%s\t%s" % (status, msg))
# Repo config: nested repos[].hooks[].
path = os.path.join(root, '.pre-commit-config.yaml')
try:
with open(path, encoding='utf-8') as fh:
cfg = yaml.safe_load(fh) or {}
except Exception as exc:
emit('FAIL', '.pre-commit-config.yaml did not parse: %s' % exc)
cfg = {}
found = None
for repo in cfg.get('repos') or []:
for hook in (repo.get('hooks') or []):
if hook.get('id') == 'skill-size-check':
found = hook
if found is None:
emit('FAIL', '.pre-commit-config.yaml declares no hook with id skill-size-check')
elif found.get('verbose') is True:
emit('PASS', '.pre-commit-config.yaml: skill-size-check is verbose: true')
else:
emit('FAIL', '.pre-commit-config.yaml: skill-size-check has verbose=%r — '
'pre-commit prints nothing for a passing hook, so every '
'ADR-0020 SUGGESTION is swallowed' % (found.get('verbose'),))
PY
)"
while IFS=$'\t' read -r status msg; do
[[ -n "$status" ]] || continue
if [[ "$status" == PASS ]]; then
pass "$msg"
else
fail "$msg"
fi
done <<< "$VERBOSE_REPORT"
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]