feat(gates): sweep the provenance corpus on pre-push

Nothing ran validate-provenance.sh across the real corpus, so the 36
INFOs it reported for Research doc mismatches were found only by a
manual loop, and a FAIL tier would have been inert. Add
scripts/check-provenance-corpus.sh, which runs the validator over every
plugins/*/.apm/skills/*/ that has references/sources.md.

Exit 1 when any skill FAILs, naming them; INFO lines are printed but do
not fail; exit 2 when the gate cannot run (missing validator, validator
exit 2, or no skills found). Registered as a pre-push hook shaped like
check-scope-walkup-sync, documented in docs/spec/gates.md, and pinned in
test-adr0020-contract.sh's list of repo-authored hooks.

Refs: #121
ADR: 0028
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EGHFJextYtVQseaHPDDhxB
This commit is contained in:
2026-09-21 17:29:20 +00:00
parent 740f631d1d
commit a1f9fa9091
5 changed files with 338 additions and 1 deletions

View File

@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
# Corpus-wide provenance sweep: runs factory-audit's validate-provenance.sh over
# every plugins/*/.apm/skills/*/ directory that has a references/sources.md, and
# fails on any FAIL.
#
# WHY THIS GATE EXISTS (ADR-0028, #121). Nothing else runs the validator over the
# real corpus. check-scope-walkup-sync.sh invokes it, but only against synthetic
# mktemp fixtures, and the factory-audit bats suite does the same. So a
# `Research doc:` that named the wrong file, or a slug absent from its Research
# registry, could only be found by hand-running the validator in a loop -- which
# is how 36 mismatches sat unnoticed while every gate stayed green. ADR-0028
# promotes "the check ran and found a mismatch" from INFO to FAIL; without a
# caller across the corpus that FAIL tier would be inert.
#
# Exit codes, kept distinct on purpose:
# 0 every skill validated (INFO-only findings are printed, never swallowed)
# 1 at least one skill FAILed -- a real finding about the corpus
# 2 the gate itself could not run: validator missing, a validator exit 2
# ("not auditable"), or NO skill with a references/sources.md found. A
# gate that discovers nothing must not read as a pass, and a skill that
# could not be audited must not read as a skill that failed the audit.
#
# The skill set is discovered by glob, not hardcoded, so a new skill is covered
# the moment it grows a references/sources.md. Run from repo root or pass
# REPO_ROOT as arg.
REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"
if [[ ! -d "$REPO_ROOT" ]]; then
echo "Provenance corpus check failed: REPO_ROOT '$REPO_ROOT' is not a directory." >&2
exit 2
fi
REPO_ROOT="$(cd "$REPO_ROOT" && pwd)"
VALIDATOR="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh"
if [[ ! -f "$VALIDATOR" ]]; then
echo "Provenance corpus check failed: $VALIDATOR does not exist, so no skill was audited. If factory-audit's scripts moved, update this path." >&2
exit 2
fi
shopt -s nullglob
sources_files=("$REPO_ROOT"/plugins/*/.apm/skills/*/references/sources.md)
shopt -u nullglob
if [[ ${#sources_files[@]} -eq 0 ]]; then
echo "Provenance corpus check failed: found no plugins/*/.apm/skills/*/references/sources.md under $REPO_ROOT. Discovering zero skills is an error, not a pass -- the glob has gone stale or the corpus moved." >&2
exit 2
fi
failing=()
errored=()
for sources in "${sources_files[@]}"; do
refs_dir="${sources%/*}"
skill_dir="${refs_dir%/*}"
rel="${skill_dir#"$REPO_ROOT"/plugins/}"
label="${rel%%/*}/${skill_dir##*/}"
rc=0
out="$(bash "$VALIDATOR" "$skill_dir" 2>&1)" || rc=$?
case "$rc" in
0)
# Exit 0 with output means INFO-only: a check that could not run,
# announced rather than skipped. Print it so it is not swallowed.
if [[ -n "$out" ]]; then
echo "== $label"
echo "$out"
fi
;;
1)
echo "== $label"
echo "$out"
failing+=("$label")
;;
*)
echo "== $label (validator exit $rc)"
echo "$out"
errored+=("$label")
;;
esac
done
echo ""
echo "Provenance corpus: ${#sources_files[@]} skill(s) checked."
if [[ ${#errored[@]} -gt 0 ]]; then
echo "Provenance corpus check errored (could not audit): ${errored[*]}" >&2
if [[ ${#failing[@]} -gt 0 ]]; then
echo "Failing skills: ${failing[*]}" >&2
fi
exit 2
fi
if [[ ${#failing[@]} -gt 0 ]]; then
echo "Failing skills: ${failing[*]}" >&2
echo "Fix each FAIL above (see ADR-0028 for the Research doc / Basis grammar); INFO lines do not fail the gate." >&2
exit 1
fi
echo "Provenance corpus check passed."