chore(gates): retire the external pre-commit hook contract
Why: .pre-commit-hooks.yaml and its release-tag gate served external consumers that do not exist. No repo on the Gitea instance pins these hooks, and the README names apm as the only supported install path. The mechanism was also already failing: skill-size-check.sh changed after v2.0.1 with no tag cut, and the gate cannot fire through Gitea's merge button. (Simplification audit finding 36.) Implementation Notes: - Delete .pre-commit-hooks.yaml, scripts/check-release-needed.sh, tests/test-check-release-needed.sh and tests/test-vale-hooks-consumer.sh, and remove the check-release-needed pre-push hook. The repo: local skill-size-check and vale-audit-prefilter-* hooks are unchanged. - ADR-0014 is amended, not retired: its runtime decision to bundle Vale inside factory-audit stands. The amendment keeps the entry[0]-only constraint (LESSONS.md:101,105) in case the export returns. ADR-0025 gets a pointer. - test-vale-wrap.sh: drop case 33 (the cross-manifest drift check) and case 28's hook-scope half, which read the published manifest. Case 32 now also requires each hook to select every tracked file of its class, which keeps case 33's one-plugin-narrowing guard, with a mutation test. - test-skill-size-check.sh and test-adr0020-contract.sh now assert the hook contract and verbose: true on .pre-commit-config.yaml only. - gates.md: pre-push count goes from 9 to 8 authored hooks (11 to 10 reported), and the Release table, the External consumers section and the two-manifest scope table are removed. README and script/test comments no longer describe the export as live. The resolver comment is edited identically in both copies. - The v1.0.0/v2.0.0/v2.0.1 tags are left in place; they are inert. ADR: 0014 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -9,10 +9,10 @@
|
||||
# ADR-0025 merged skill-audit and agent-audit, which dropped the count from
|
||||
# three copies to two: factory-audit now holds ONE copy in a sourced
|
||||
# lib-boundary-resolver.sh, and scripts/skill-size-check.sh keeps its
|
||||
# embedded copy because it is a repo-root hook consumed through
|
||||
# .pre-commit-hooks.yaml, where entry[0] is the only token pre-commit
|
||||
# rewrites — it cannot reach a file inside the plugin at a path any consumer
|
||||
# has. Nothing but this file asserts the two copies are still identical, and
|
||||
# embedded copy because it is a repo-root hook kept fit for a published
|
||||
# hook manifest (retired; ADR-0014), where entry[0] is the only token
|
||||
# pre-commit rewrites — it could not reach a file inside the plugin at a
|
||||
# path any consumer has. Nothing but this file asserts the two copies are still identical, and
|
||||
# a one-line edit to a single copy is invisible: every constant-agreement
|
||||
# assertion in tests/test-skill-size-check.sh still passes, because the
|
||||
# CONSTANTS are not what drifted.
|
||||
@@ -590,14 +590,14 @@ for probe in "scripts/skill-size-check.sh|$HOOK|$SUBJECT_SKILL_DIR/SKILL.md" \
|
||||
done
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. verbose: true on the skill-size-check hook, in BOTH manifests
|
||||
# 3. verbose: true on the skill-size-check hook
|
||||
# ---------------------------------------------------------------------------
|
||||
# .pre-commit-config.yaml governs this repo; .pre-commit-hooks.yaml is what a
|
||||
# CONSUMER repo gets when it points at this one. Dropping the flag from either
|
||||
# silences the SUGGESTION tier for that audience alone, which is the hardest
|
||||
# version of the defect to notice.
|
||||
# pre-commit prints nothing for a passing hook, so dropping the flag silences
|
||||
# the SUGGESTION tier without failing anything. The published
|
||||
# .pre-commit-hooks.yaml that once carried a second copy of this hook was
|
||||
# retired (ADR-0014, 2026-09-16 amendment); if it returns, assert it here too.
|
||||
echo ""
|
||||
echo "--- the skill-size-check hook declares verbose: true in both manifests ---"
|
||||
echo "--- the skill-size-check hook declares verbose: true ---"
|
||||
VERBOSE_REPORT="$(python3 - "$REPO_ROOT" <<'PY'
|
||||
import os
|
||||
import sys
|
||||
@@ -632,27 +632,6 @@ else:
|
||||
emit('FAIL', '.pre-commit-config.yaml: skill-size-check has verbose=%r — '
|
||||
'pre-commit prints nothing for a passing hook, so every '
|
||||
'ADR-0020 SUGGESTION is swallowed' % (found.get('verbose'),))
|
||||
|
||||
# Consumer manifest: a flat list of hooks.
|
||||
path = os.path.join(root, '.pre-commit-hooks.yaml')
|
||||
try:
|
||||
with open(path, encoding='utf-8') as fh:
|
||||
hooks = yaml.safe_load(fh) or []
|
||||
except Exception as exc:
|
||||
emit('FAIL', '.pre-commit-hooks.yaml did not parse: %s' % exc)
|
||||
hooks = []
|
||||
found = None
|
||||
for hook in hooks:
|
||||
if isinstance(hook, dict) and hook.get('id') == 'kyberforge-skill-size-check':
|
||||
found = hook
|
||||
if found is None:
|
||||
emit('FAIL', '.pre-commit-hooks.yaml declares no hook with id kyberforge-skill-size-check')
|
||||
elif found.get('verbose') is True:
|
||||
emit('PASS', '.pre-commit-hooks.yaml: kyberforge-skill-size-check is verbose: true')
|
||||
else:
|
||||
emit('FAIL', '.pre-commit-hooks.yaml: kyberforge-skill-size-check has verbose=%r — '
|
||||
'a consumer repo would never see the SUGGESTION tier'
|
||||
% (found.get('verbose'),))
|
||||
PY
|
||||
)"
|
||||
while IFS=$'\t' read -r status msg; do
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
# and agent-audit behind one auto-detecting entry point; every fixture here is a
|
||||
# skill directory, so every invocation below runs the skill flow. The agent flow
|
||||
# has no counterpart hook to differ from — there is no agent-file size gate in
|
||||
# .pre-commit-hooks.yaml — so it is out of this suite's scope, not dropped from it.
|
||||
# .pre-commit-config.yaml — so it is out of this suite's scope, not dropped from it.
|
||||
#
|
||||
# Why this exists as a separate suite. tests/test-skill-size-check.sh already
|
||||
# asserts the two agree on their CONSTANTS, and that assertion is necessary but
|
||||
|
||||
@@ -1,449 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
SCRIPT="$REPO_ROOT/scripts/check-release-needed.sh"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
||||
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
# Output assertions are `grep -q PATTERN <<< "$OUT"`, never `echo "$OUT" | grep -q`.
|
||||
# Under pipefail the pipe form is scheduling-dependent: bash's echo writes a
|
||||
# multi-line value one line at a time, `grep -q` exits on its first match, and a
|
||||
# later line then hits a closed pipe. echo dies of SIGPIPE, pipefail reports the
|
||||
# pipeline as failed, and a correct output reads as a missing match. The
|
||||
# here-string has no writer process to race.
|
||||
|
||||
# Both entry shapes the real .pre-commit-hooks.yaml ships: a bare script with no
|
||||
# bundled data, and a bare script whose sibling assets/ tree it self-locates at
|
||||
# runtime. Neither carries arguments — pre-commit only rewrites entry[0] to the
|
||||
# hook-repo clone path, so an argument path would resolve against the consuming
|
||||
# repo. RELEASE_PATHS is derived from the manifest rather than hand-maintained,
|
||||
# so it has to cope with both.
|
||||
HOOK_DIR="plugins/demo/skills/demo-audit"
|
||||
|
||||
write_manifest() {
|
||||
local dir="$1"
|
||||
cat > "$dir/.pre-commit-hooks.yaml" <<EOF
|
||||
- id: fake-size-check
|
||||
entry: scripts/skill-size-check.sh
|
||||
language: script
|
||||
- id: fake-vale-check
|
||||
entry: $HOOK_DIR/scripts/vale-wrap.sh
|
||||
language: script
|
||||
EOF
|
||||
}
|
||||
|
||||
# Helper: writes the files both manifest entries expose — the two hook scripts
|
||||
# plus the bundled Vale config and style rule the second one self-locates.
|
||||
write_release_paths() {
|
||||
local dir="$1"
|
||||
mkdir -p "$dir/scripts" "$dir/$HOOK_DIR/scripts" "$dir/$HOOK_DIR/assets/vale/styles/Kyberforge"
|
||||
echo "v1" > "$dir/scripts/skill-size-check.sh"
|
||||
echo "v1" > "$dir/$HOOK_DIR/scripts/vale-wrap.sh"
|
||||
echo "cfg" > "$dir/$HOOK_DIR/assets/vale/.vale.ini"
|
||||
echo "rule: v1" > "$dir/$HOOK_DIR/assets/vale/styles/Kyberforge/DemoRule.yml"
|
||||
}
|
||||
|
||||
# Helper: a fixture repo with a manifest and every release-relevant path it
|
||||
# exposes, committed and tagged v1.0.0.
|
||||
make_tagged_fixture() {
|
||||
local dir
|
||||
dir="$(mktemp -d)"
|
||||
(cd "$dir" && git init -q && git config user.email t@t.t && git config user.name t)
|
||||
write_manifest "$dir"
|
||||
write_release_paths "$dir"
|
||||
(cd "$dir" && git add -A && git commit -q -m "initial" && git tag v1.0.0)
|
||||
echo "$dir"
|
||||
}
|
||||
|
||||
# Helper: a fixture whose manifest carries one malformed entry: at the tag *and*
|
||||
# at HEAD, plus a post-tag change to the file that entry was meant to cover.
|
||||
# Committing the bad entry before the tag is what makes the assertion sharp — an
|
||||
# edited manifest is itself release-relevant, so the gate would fail for the
|
||||
# wrong reason and hide a parser that degrades silently.
|
||||
make_malformed_fixture() {
|
||||
local entry="$1" dir
|
||||
dir="$(mktemp -d)"
|
||||
(cd "$dir" && git init -q && git config user.email t@t.t && git config user.name t)
|
||||
write_release_paths "$dir"
|
||||
cat > "$dir/.pre-commit-hooks.yaml" <<EOF
|
||||
- id: fake-size-check
|
||||
entry: $entry
|
||||
language: script
|
||||
EOF
|
||||
(cd "$dir" && git add -A && git commit -q -m "initial" && git tag v1.0.0)
|
||||
echo "v2" > "$dir/scripts/skill-size-check.sh"
|
||||
(cd "$dir" && git add -A && git commit -q -m "change the file the malformed entry should cover")
|
||||
echo "$dir"
|
||||
}
|
||||
|
||||
# $3 is optional: pre-commit's PRE_COMMIT_TO_REF, the local sha being pushed.
|
||||
# Left off entirely, the variable stays unset and the script falls back to HEAD,
|
||||
# exactly as a plain `git push <remote> <current-branch>` behaves.
|
||||
# The fixture repo is the subject under test, so every PRE_COMMIT_* input must
|
||||
# come from this function and nowhere else. Any such variable already in the
|
||||
# environment belongs to the *caller's* repo: run under the pre-push hook this
|
||||
# suite guards, PRE_COMMIT_TO_REF holds a sha of the real repo, which does not
|
||||
# exist in the fixture, and the script resolves against the wrong rev. Clearing
|
||||
# them is what makes a standalone run and a pre-push run the same test — this
|
||||
# suite passed everywhere except under the hook it exists to protect.
|
||||
run_check() {
|
||||
local dir="$1" branch="$2"
|
||||
if [[ $# -ge 3 ]]; then
|
||||
(cd "$dir" && unset PRE_COMMIT_FROM_REF \
|
||||
&& PRE_COMMIT_REMOTE_BRANCH="$branch" PRE_COMMIT_TO_REF="$3" bash "$SCRIPT" 2>&1)
|
||||
else
|
||||
(cd "$dir" && unset PRE_COMMIT_FROM_REF PRE_COMMIT_TO_REF \
|
||||
&& PRE_COMMIT_REMOTE_BRANCH="$branch" bash "$SCRIPT" 2>&1)
|
||||
fi
|
||||
}
|
||||
|
||||
CLEANUP_DIRS=()
|
||||
trap 'rm -rf ${CLEANUP_DIRS[@]+"${CLEANUP_DIRS[@]}"}' EXIT
|
||||
track() { CLEANUP_DIRS+=("$1"); }
|
||||
|
||||
# --- 1. Not targeting main: silent no-op regardless of state ---
|
||||
echo ""
|
||||
echo "--- exits 0 when not pushing to main, even with no tags ---"
|
||||
FIXTURE1="$(mktemp -d)"; track "$FIXTURE1"
|
||||
(cd "$FIXTURE1" && git init -q)
|
||||
if run_check "$FIXTURE1" "refs/heads/feature-branch" > /dev/null; then
|
||||
pass "exits 0 when target branch isn't main"
|
||||
else
|
||||
fail "exited non-zero on a non-main target branch"
|
||||
fi
|
||||
|
||||
# --- 2. Targeting main, no tag exists at all: hard fail ---
|
||||
echo ""
|
||||
echo "--- exits 1 when targeting main and no tag exists ---"
|
||||
FIXTURE2="$(mktemp -d)"; track "$FIXTURE2"
|
||||
(cd "$FIXTURE2" && git init -q && git config user.email t@t.t && git config user.name t)
|
||||
write_manifest "$FIXTURE2"
|
||||
write_release_paths "$FIXTURE2"
|
||||
(cd "$FIXTURE2" && git add -A && git commit -q -m "initial")
|
||||
if run_check "$FIXTURE2" "refs/heads/main" > /dev/null; then
|
||||
fail "exited 0 when targeting main with no tag — expected exit 1"
|
||||
else
|
||||
pass "exits non-zero when targeting main and no tag exists yet"
|
||||
fi
|
||||
|
||||
# --- 3. Targeting main, tag exists, no release-relevant changes since: passes ---
|
||||
echo ""
|
||||
echo "--- exits 0 when targeting main and nothing release-relevant changed since the tag ---"
|
||||
FIXTURE3="$(make_tagged_fixture)"; track "$FIXTURE3"
|
||||
echo "unrelated" > "$FIXTURE3/README.md"
|
||||
(cd "$FIXTURE3" && git add -A && git commit -q -m "unrelated change")
|
||||
if run_check "$FIXTURE3" "refs/heads/main" > /dev/null; then
|
||||
pass "exits 0 when only unrelated files changed since the tag"
|
||||
else
|
||||
fail "exited non-zero despite no release-relevant changes since the tag"
|
||||
fi
|
||||
|
||||
# --- 4. Targeting main, tag exists, a release-relevant file changed since: hard fail ---
|
||||
echo ""
|
||||
echo "--- exits 1 when a release-relevant file changed since the tag ---"
|
||||
FIXTURE4="$(make_tagged_fixture)"; track "$FIXTURE4"
|
||||
echo "v2" > "$FIXTURE4/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE4" && git add -A && git commit -q -m "update release-relevant script")
|
||||
OUT4=$(run_check "$FIXTURE4" "refs/heads/main" || true)
|
||||
if grep -q "skill-size-check.sh" <<< "$OUT4"; then
|
||||
pass "exits non-zero and names the changed file when a release-relevant path changed since the tag"
|
||||
else
|
||||
fail "did not flag the release-relevant file that changed since the tag"
|
||||
fi
|
||||
|
||||
# --- 5. Not targeting main even with release-relevant changes and a tag: still a no-op ---
|
||||
echo ""
|
||||
echo "--- exits 0 on a feature branch even with release-relevant changes since the tag ---"
|
||||
FIXTURE5="$(make_tagged_fixture)"; track "$FIXTURE5"
|
||||
echo "v2" > "$FIXTURE5/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE5" && git add -A && git commit -q -m "update release-relevant script")
|
||||
if run_check "$FIXTURE5" "refs/heads/some-feature" > /dev/null; then
|
||||
pass "exits 0 on a feature branch regardless of un-tagged release-relevant changes"
|
||||
else
|
||||
fail "hard-failed on a feature branch — should only ever fail when targeting main"
|
||||
fi
|
||||
|
||||
# --- 6. A release-relevant path deleted since the tag is still flagged ---
|
||||
echo ""
|
||||
echo "--- exits 1 when a release-relevant path was deleted since the tag, not just modified ---"
|
||||
FIXTURE6="$(make_tagged_fixture)"; track "$FIXTURE6"
|
||||
rm -f "$FIXTURE6/$HOOK_DIR/assets/vale/.vale.ini"
|
||||
(cd "$FIXTURE6" && git add -A && git commit -q -m "delete the bundled vale config")
|
||||
OUT6=$(run_check "$FIXTURE6" "refs/heads/main" || true)
|
||||
if grep -q "assets/vale/.vale.ini" <<< "$OUT6"; then
|
||||
pass "flags a deleted release-relevant path instead of silently dropping it from the diff"
|
||||
else
|
||||
fail "did not flag deletion of a release-relevant path since the tag"
|
||||
fi
|
||||
|
||||
# --- 7. A git diff failure hard-fails instead of reading as a clean pass ---
|
||||
echo ""
|
||||
echo "--- exits 1 (not a silent pass) when the underlying git diff errors out ---"
|
||||
FIXTURE7="$(make_tagged_fixture)"; track "$FIXTURE7"
|
||||
TAG_TREE="$(cd "$FIXTURE7" && git rev-parse 'v1.0.0^{tree}')"
|
||||
echo "v2" > "$FIXTURE7/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE7" && git add -A && git commit -q -m "advance past the tag")
|
||||
rm -f "$FIXTURE7/.git/objects/${TAG_TREE:0:2}/${TAG_TREE:2}"
|
||||
if run_check "$FIXTURE7" "refs/heads/main" > /dev/null; then
|
||||
fail "silently exited 0 when the underlying git diff failed"
|
||||
else
|
||||
pass "hard-fails instead of silently passing when git diff can't be computed"
|
||||
fi
|
||||
|
||||
# --- 8. A non-version tag reachable from HEAD does not become the diff baseline ---
|
||||
echo ""
|
||||
echo "--- ignores a non-vX.Y.Z tag and still flags a change since the real release tag ---"
|
||||
FIXTURE8="$(make_tagged_fixture)"; track "$FIXTURE8"
|
||||
echo "checkpoint" > "$FIXTURE8/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE8" && git add -A && git commit -q -m "checkpoint work" && git tag checkpoint-1)
|
||||
echo "v2" > "$FIXTURE8/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE8" && git add -A && git commit -q -m "real release-relevant change")
|
||||
OUT8=$(run_check "$FIXTURE8" "refs/heads/main" || true)
|
||||
if grep -q "skill-size-check.sh" <<< "$OUT8"; then
|
||||
pass "still flags the release-relevant change since v1.0.0, ignoring the non-version checkpoint tag"
|
||||
else
|
||||
fail "an incidental non-version tag shifted the baseline and hid a real release-relevant change"
|
||||
fi
|
||||
|
||||
# --- 9. A file outside every manifest entry does not trigger a fail ---
|
||||
echo ""
|
||||
echo "--- exits 0 when a changed file sits near, but isn't referenced by, a manifest entry ---"
|
||||
FIXTURE9="$(make_tagged_fixture)"; track "$FIXTURE9"
|
||||
echo "irrelevant" > "$FIXTURE9/scripts/unrelated-helper.sh"
|
||||
(cd "$FIXTURE9" && git add -A && git commit -q -m "add an unrelated script alongside the exposed one")
|
||||
if run_check "$FIXTURE9" "refs/heads/main" > /dev/null; then
|
||||
pass "exits 0 for a file that lives alongside, but isn't referenced by, any manifest entry"
|
||||
else
|
||||
fail "flagged a file that no .pre-commit-hooks.yaml entry actually exposes"
|
||||
fi
|
||||
|
||||
# --- 10. A change confined to a hook's bundled styles/ tree is release-relevant ---
|
||||
# The manifest entry names only the wrapper script; the Vale rules it enforces
|
||||
# live in the sibling assets/ tree it self-locates at runtime. If that tree is
|
||||
# not covered, editing a rule and landing it on main demands no new tag, and a
|
||||
# consumer pinned to the old rev keeps the stale rules forever.
|
||||
echo ""
|
||||
echo "--- exits 1 when only a bundled Vale style rule changed since the tag ---"
|
||||
FIXTURE10="$(make_tagged_fixture)"; track "$FIXTURE10"
|
||||
echo "rule: v2" > "$FIXTURE10/$HOOK_DIR/assets/vale/styles/Kyberforge/DemoRule.yml"
|
||||
(cd "$FIXTURE10" && git add -A && git commit -q -m "tighten a vale rule")
|
||||
OUT10=$(run_check "$FIXTURE10" "refs/heads/main" || true)
|
||||
if grep -q "assets/vale/styles/Kyberforge/DemoRule.yml" <<< "$OUT10"; then
|
||||
pass "flags a change confined to a hook's bundled assets/vale/styles/ tree"
|
||||
else
|
||||
fail "a bundled Vale style rule changed since the tag without demanding a release"
|
||||
fi
|
||||
|
||||
# --- 11. The assets/ derivation must not invent a path for a bundle-less hook ---
|
||||
# scripts/skill-size-check.sh has no sibling assets/ tree, so its derived
|
||||
# candidate normalises to a bare top-level assets/ — a directory this repo does
|
||||
# not ship. Adding it unconditionally would make any unrelated repo-root
|
||||
# assets/ file falsely demand a release.
|
||||
echo ""
|
||||
echo "--- exits 0 when a top-level assets/ file changed but no hook bundles one ---"
|
||||
FIXTURE11="$(make_tagged_fixture)"; track "$FIXTURE11"
|
||||
mkdir -p "$FIXTURE11/assets"
|
||||
echo "unrelated" > "$FIXTURE11/assets/logo.txt"
|
||||
(cd "$FIXTURE11" && git add -A && git commit -q -m "add an unrelated top-level assets file")
|
||||
if run_check "$FIXTURE11" "refs/heads/main" > /dev/null; then
|
||||
pass "exits 0 for a top-level assets/ file that no manifest entry bundles"
|
||||
else
|
||||
fail "invented a bogus assets/ path for a hook script with no bundled tree"
|
||||
fi
|
||||
|
||||
# --- 12. Deleting a hook's entire bundled assets/ tree is release-relevant ---
|
||||
# The worktree-only derivation guarded the assets/ path on the directory still
|
||||
# existing, so wiping the whole tree removed the path from the pathspec instead
|
||||
# of diffing it: the single most consumer-breaking change possible diffed clean.
|
||||
# The path list therefore has to be unioned with what $LAST_TAG exposed.
|
||||
echo ""
|
||||
echo "--- exits 1 when a hook's entire bundled assets/ tree was deleted since the tag ---"
|
||||
FIXTURE12="$(make_tagged_fixture)"; track "$FIXTURE12"
|
||||
rm -rf "${FIXTURE12:?}/$HOOK_DIR/assets"
|
||||
(cd "$FIXTURE12" && git add -A && git commit -q -m "delete the whole bundled assets tree")
|
||||
OUT12=$(run_check "$FIXTURE12" "refs/heads/main" || true)
|
||||
if grep -q "assets/vale/.vale.ini" <<< "$OUT12"; then
|
||||
pass "flags a wholesale deletion of a hook's bundled assets/ tree"
|
||||
else
|
||||
fail "a hook's entire bundled assets/ tree vanished since the tag without demanding a release"
|
||||
fi
|
||||
|
||||
# --- 13. A hook script deleted while its manifest entry survives is flagged ---
|
||||
# Characterisation test, not a bug fix: tokens[0] is added to the pathspec
|
||||
# unconditionally (no existence guard), so this case was already covered. It is
|
||||
# pinned here so the tagged-tree union can't accidentally introduce an existence
|
||||
# guard on tokens[0] and reopen the hole its assets/ sibling had.
|
||||
echo ""
|
||||
echo "--- exits 1 when a hook script was deleted but its manifest entry remains ---"
|
||||
FIXTURE13="$(make_tagged_fixture)"; track "$FIXTURE13"
|
||||
rm -f "$FIXTURE13/$HOOK_DIR/scripts/vale-wrap.sh"
|
||||
(cd "$FIXTURE13" && git add -A && git commit -q -m "delete a hook script, keep its manifest entry")
|
||||
OUT13=$(run_check "$FIXTURE13" "refs/heads/main" || true)
|
||||
if grep -q "vale-wrap.sh" <<< "$OUT13"; then
|
||||
pass "flags a hook script deleted out from under a surviving manifest entry"
|
||||
else
|
||||
fail "a manifest entry's script vanished since the tag without demanding a release"
|
||||
fi
|
||||
|
||||
# --- 14. Retiring a whole hook names what the tag exposed, not just the manifest ---
|
||||
# Removing the entry and everything it shipped changes $HOOKS_MANIFEST, so the
|
||||
# gate fires either way — but a derivation that only reads the current manifest
|
||||
# can no longer name the retired script or its assets, and the failure message
|
||||
# understates the breakage to consumers pinned at the old rev. The tagged
|
||||
# manifest is what makes those paths reportable.
|
||||
echo ""
|
||||
echo "--- names the retired hook's own paths when an entry and its files are removed together ---"
|
||||
FIXTURE14="$(make_tagged_fixture)"; track "$FIXTURE14"
|
||||
cat > "$FIXTURE14/.pre-commit-hooks.yaml" <<'EOF'
|
||||
- id: fake-size-check
|
||||
entry: scripts/skill-size-check.sh
|
||||
language: script
|
||||
EOF
|
||||
rm -rf "${FIXTURE14:?}/$HOOK_DIR"
|
||||
(cd "$FIXTURE14" && git add -A && git commit -q -m "retire the vale hook entirely")
|
||||
OUT14=$(run_check "$FIXTURE14" "refs/heads/main" || true)
|
||||
if grep -q "vale-wrap.sh" <<< "$OUT14" && grep -q "assets/vale/.vale.ini" <<< "$OUT14"; then
|
||||
pass "names the retired hook's script and bundled assets, not just the manifest edit"
|
||||
else
|
||||
fail "reported only the manifest change and hid which shipped paths the retirement removed"
|
||||
fi
|
||||
|
||||
# --- 15. A multi-token entry: is rejected loudly, not silently mis-parsed ---
|
||||
# ADR-0014 binds entries to a bare script path, but nothing enforced it, and the
|
||||
# sibling .pre-commit-config.yaml already ships `entry: bash <script>`. Under the
|
||||
# old parser tokens[0] became "bash": a pathspec matching nothing (which git diff
|
||||
# accepts in silence) and a bundle root of "." (skipped), so the hook's whole
|
||||
# surface dropped out of the gate and the post-tag change below diffed clean.
|
||||
echo ""
|
||||
echo "--- exits 1 naming the hook when an entry: carries more than one token ---"
|
||||
# The entry is quoted back verbatim, not just its first token: that is what makes
|
||||
# the diagnostic point at the argument the author has to remove, and what
|
||||
# distinguishes this from the unresolvable-path rejection test 16 covers.
|
||||
FIXTURE15="$(make_malformed_fixture "bash scripts/skill-size-check.sh")"; track "$FIXTURE15"
|
||||
OUT15=$(run_check "$FIXTURE15" "refs/heads/main" || true)
|
||||
if run_check "$FIXTURE15" "refs/heads/main" > /dev/null; then
|
||||
fail "silently exited 0 on a multi-token entry, dropping that hook's paths from the gate"
|
||||
elif grep -q "fake-size-check" <<< "$OUT15" \
|
||||
&& grep -q "bash scripts/skill-size-check.sh" <<< "$OUT15" \
|
||||
&& grep -q "ADR-0014" <<< "$OUT15"; then
|
||||
pass "rejects a multi-token entry, quoting it back and naming the hook and ADR-0014"
|
||||
else
|
||||
fail "rejected the multi-token entry without naming the hook, the entry, and ADR-0014"
|
||||
fi
|
||||
|
||||
# --- 16. An entry naming no file this repo ships is rejected loudly ---
|
||||
# The token-count guard alone still lets a single bare command name (`entry:
|
||||
# vale`, valid for language: system) through as a pathspec matching nothing.
|
||||
# Existence is checked against the union of the worktree and $LAST_TAG, so this
|
||||
# cannot misfire on the deletion cases tests 12-14 pin.
|
||||
echo ""
|
||||
echo "--- exits 1 naming the hook when an entry: names no file in the worktree or at the tag ---"
|
||||
FIXTURE16="$(make_malformed_fixture "vale")"; track "$FIXTURE16"
|
||||
OUT16=$(run_check "$FIXTURE16" "refs/heads/main" || true)
|
||||
if run_check "$FIXTURE16" "refs/heads/main" > /dev/null; then
|
||||
fail "silently exited 0 on an entry that names no shipped file"
|
||||
elif grep -q "fake-size-check" <<< "$OUT16" && grep -q "ADR-0014" <<< "$OUT16"; then
|
||||
pass "rejects an entry that resolves to no file, naming the hook and the ADR-0014 constraint"
|
||||
else
|
||||
fail "rejected the unresolvable entry without naming the hook and the ADR-0014 constraint"
|
||||
fi
|
||||
|
||||
# --- 17. The pushed ref, not HEAD, is what gets gated ---
|
||||
# pre-commit exports the local sha of each pushed ref as PRE_COMMIT_TO_REF.
|
||||
# `git push <remote> pushed-tip:main` from a checkout sitting on an older commit
|
||||
# is the false-negative direction: HEAD is still at the tag and diffs clean while
|
||||
# the branch actually landing on main carries an untagged, release-relevant
|
||||
# change. HEAD is reset back to the tag so the two genuinely differ.
|
||||
echo ""
|
||||
echo "--- exits 1 on a release-relevant change reachable only from PRE_COMMIT_TO_REF ---"
|
||||
FIXTURE17="$(make_tagged_fixture)"; track "$FIXTURE17"
|
||||
echo "v2" > "$FIXTURE17/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE17" && git add -A && git commit -q -m "release-relevant change" \
|
||||
&& git branch pushed-tip && git reset -q --hard v1.0.0)
|
||||
OUT17=$(run_check "$FIXTURE17" "refs/heads/main" "pushed-tip" || true)
|
||||
if grep -q "skill-size-check.sh" <<< "$OUT17"; then
|
||||
pass "gates the pushed ref's tip, not HEAD, when HEAD is behind it"
|
||||
else
|
||||
fail "diffed HEAD instead of PRE_COMMIT_TO_REF and missed a release-relevant change"
|
||||
fi
|
||||
|
||||
# --- 18. Neither the diff tip nor the tag baseline may come from a newer HEAD ---
|
||||
# The false-positive direction: HEAD has moved past a v2.0.0 that the pushed ref
|
||||
# never saw. Reading either end of the diff off HEAD fails a push that is clean
|
||||
# since its own baseline — diffing v2.0.0..HEAD flags HEAD's untagged commit, and
|
||||
# resolving the tag from HEAD while diffing pushed-tip flags v2.0.0's change.
|
||||
echo ""
|
||||
echo "--- exits 0 when the pushed ref is clean since its own tag but HEAD has moved on ---"
|
||||
FIXTURE18="$(make_tagged_fixture)"; track "$FIXTURE18"
|
||||
(cd "$FIXTURE18" && git branch pushed-tip)
|
||||
echo "v2" > "$FIXTURE18/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE18" && git add -A && git commit -q -m "released change" && git tag v2.0.0)
|
||||
echo "v3" > "$FIXTURE18/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE18" && git add -A && git commit -q -m "unreleased change on HEAD's line")
|
||||
if run_check "$FIXTURE18" "refs/heads/main" "pushed-tip" > /dev/null; then
|
||||
pass "exits 0 for a pushed ref clean since the tag reachable from it, ignoring HEAD's line"
|
||||
else
|
||||
fail "gated HEAD's tag or tip and falsely demanded a release for a clean pushed ref"
|
||||
fi
|
||||
|
||||
# --- 19. A branch deletion is a no-op, not a confusing git failure ---
|
||||
# pre-commit sets PRE_COMMIT_TO_REF to an all-zeros sha when the push deletes a
|
||||
# branch. Nothing is being shipped, and the sha resolves to nothing, so without
|
||||
# an explicit guard the gate reports "could not diff" on an unrelated operation.
|
||||
echo ""
|
||||
echo "--- exits 0 when PRE_COMMIT_TO_REF is the all-zeros branch-deletion sha ---"
|
||||
FIXTURE19="$(make_tagged_fixture)"; track "$FIXTURE19"
|
||||
echo "v2" > "$FIXTURE19/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE19" && git add -A && git commit -q -m "release-relevant change")
|
||||
if run_check "$FIXTURE19" "refs/heads/main" "0000000000000000000000000000000000000000" > /dev/null; then
|
||||
pass "treats an all-zeros PRE_COMMIT_TO_REF as a branch deletion and exits 0"
|
||||
else
|
||||
fail "turned a branch deletion into a failure instead of a no-op"
|
||||
fi
|
||||
|
||||
# --- 20. The repo's own .pre-commit-hooks.yaml satisfies the entry constraints ---
|
||||
# The parser guards above are only safe to ship if the manifest actually in tree
|
||||
# passes them. It is replayed into a fixture (with the paths its entries name
|
||||
# created) rather than run against the real repo, which has no release tag yet.
|
||||
echo ""
|
||||
echo "--- accepts the real .pre-commit-hooks.yaml this repo ships ---"
|
||||
FIXTURE20="$(mktemp -d)"; track "$FIXTURE20"
|
||||
(cd "$FIXTURE20" && git init -q && git config user.email t@t.t && git config user.name t)
|
||||
cp "$REPO_ROOT/.pre-commit-hooks.yaml" "$FIXTURE20/.pre-commit-hooks.yaml"
|
||||
while IFS= read -r real_entry; do
|
||||
mkdir -p "$FIXTURE20/$(dirname "$real_entry")"
|
||||
echo "v1" > "$FIXTURE20/$real_entry"
|
||||
done < <(sed -n 's/^[[:space:]]*entry:[[:space:]]*//p' "$REPO_ROOT/.pre-commit-hooks.yaml")
|
||||
(cd "$FIXTURE20" && git add -A && git commit -q -m "initial" && git tag v1.0.0)
|
||||
OUT20=$(run_check "$FIXTURE20" "refs/heads/main" || true)
|
||||
if [[ -z "$OUT20" ]]; then
|
||||
pass "parses every entry in the repo's real .pre-commit-hooks.yaml without complaint"
|
||||
else
|
||||
fail "the repo's own .pre-commit-hooks.yaml no longer satisfies the entry constraints: $OUT20"
|
||||
fi
|
||||
|
||||
# --- 21. A vX.Y.Z-suffixed checkpoint tag must not satisfy the release gate ---
|
||||
# git describe --match uses shell-glob semantics, not regex: the trailing `*` in
|
||||
# 'v[0-9]*.[0-9]*.[0-9]*' matches any suffix, so a pre-release/checkpoint tag like
|
||||
# v1.0.1-checkpoint also satisfies the glob and can be picked as LAST_TAG instead
|
||||
# of the true last release tag — hiding a real release-relevant change that landed
|
||||
# before the checkpoint tag from the diff.
|
||||
echo ""
|
||||
echo "--- ignores a vX.Y.Z-checkpoint tag and still flags the change since the real release tag ---"
|
||||
FIXTURE21="$(make_tagged_fixture)"; track "$FIXTURE21"
|
||||
echo "v2" > "$FIXTURE21/scripts/skill-size-check.sh"
|
||||
(cd "$FIXTURE21" && git add -A && git commit -q -m "real release-relevant change" && git tag v1.0.1-checkpoint)
|
||||
OUT21=$(run_check "$FIXTURE21" "refs/heads/main" || true)
|
||||
if grep -q "skill-size-check.sh" <<< "$OUT21"; then
|
||||
pass "still flags the release-relevant change since v1.0.0, ignoring the vX.Y.Z-checkpoint tag"
|
||||
else
|
||||
fail "a vX.Y.Z-checkpoint tag satisfied the glob and hid a real release-relevant change"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
@@ -738,8 +738,8 @@ expect_gate "a fixture with no authoring root reports DID NOT RUN and exits 0" \
|
||||
# provider-adapter-author's validate-adapter.sh and the one vale-wrap.sh already
|
||||
# used: {0,1} are verdicts, 2 is "you invoked this wrong".
|
||||
#
|
||||
# SAFE FOR THE HOOK. Both manifests declare pass_filenames: true and neither
|
||||
# sets always_run, and pre-commit skips a filename-passing hook outright when
|
||||
# SAFE FOR THE HOOK. The skill-size-check hook declares pass_filenames: true
|
||||
# and does not set always_run, and pre-commit skips a filename-passing hook outright when
|
||||
# its `files:` pattern matches nothing, so pre-commit never invokes this script
|
||||
# with an empty argument list. That claim is asserted below rather than left in
|
||||
# prose, so a config edit that turns it false fails here.
|
||||
@@ -773,7 +773,7 @@ if [[ $CLEAN_RC -eq 0 && $FINDING_RC -eq 1 && $USAGE_RC -eq 2 ]]; then
|
||||
else
|
||||
fail "exit codes collide — clean=$CLEAN_RC findings=$FINDING_RC usage=$USAGE_RC"
|
||||
fi
|
||||
# The hook contract the usage exit depends on. If either manifest ever stops
|
||||
# The hook contract the usage exit depends on. If the hook ever stops
|
||||
# passing filenames, or starts always_run, pre-commit could invoke the script
|
||||
# with no paths and exit 2 would break the hook rather than diagnose a caller.
|
||||
HOOK_CONTRACT="$(python3 - "$REPO_ROOT" <<'PYHOOK'
|
||||
@@ -805,19 +805,11 @@ for repo in cfg.get('repos') or []:
|
||||
found = hook
|
||||
check('.pre-commit-config.yaml skill-size-check', found)
|
||||
|
||||
with open(os.path.join(root, '.pre-commit-hooks.yaml'), encoding='utf-8') as fh:
|
||||
hooks = yaml.safe_load(fh) or []
|
||||
found = None
|
||||
for hook in hooks:
|
||||
if isinstance(hook, dict) and hook.get('id') == 'kyberforge-skill-size-check':
|
||||
found = hook
|
||||
check('.pre-commit-hooks.yaml kyberforge-skill-size-check', found)
|
||||
|
||||
print('; '.join(problems))
|
||||
PYHOOK
|
||||
)"
|
||||
if [[ -z "$HOOK_CONTRACT" ]]; then
|
||||
pass "both manifests pass filenames and neither is always_run, so pre-commit never invokes the script with no paths"
|
||||
pass "the hook passes filenames and is not always_run, so pre-commit never invokes the script with no paths"
|
||||
else
|
||||
fail "the usage exit would break the hook: $HOOK_CONTRACT"
|
||||
fi
|
||||
|
||||
@@ -18,7 +18,7 @@ if ! python3 -c 'import yaml' 2>/dev/null; then
|
||||
fi
|
||||
|
||||
# Output assertions use here-strings, never `echo | grep -q` (pipefail race;
|
||||
# see tests/test-check-release-needed.sh).
|
||||
# see docs/spec/gates.md, Tests).
|
||||
|
||||
CLEANUP_DIRS=()
|
||||
trap 'rm -rf ${CLEANUP_DIRS[@]+"${CLEANUP_DIRS[@]}"}' EXIT
|
||||
|
||||
@@ -1,276 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Integration test for .pre-commit-hooks.yaml as an EXTERNAL hook repo — the
|
||||
# contract ADR-0014 exists to provide, and the one thing running pre-commit
|
||||
# inside this repo can never exercise: `repo: local` makes pre-commit's clone
|
||||
# prefix equal to the consuming repo's root, so a hook entry that only works
|
||||
# because those two coincide passes here and hard-fails everywhere else.
|
||||
# (It did: every argument after entry[0] resolves against the CONSUMING repo,
|
||||
# so a `--config plugins/.../.vale.ini` argument gave external consumers
|
||||
# `E100 [--config] Runtime error ... does not exist`, exit 2, on both Vale hooks.)
|
||||
#
|
||||
# The hook repo is built from the WORKING TREE, not from HEAD, so an uncommitted
|
||||
# change to the manifest or the wrapper is what gets tested.
|
||||
set -euo pipefail
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
PASS=0
|
||||
FAIL=0
|
||||
|
||||
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
||||
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
for bin in pre-commit vale git; do
|
||||
if ! command -v "$bin" &>/dev/null; then
|
||||
echo "SKIP: $bin is not installed — cannot stand up a consumer repo"
|
||||
exit 77
|
||||
fi
|
||||
done
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
HOOK_REPO="$WORK/hookrepo"
|
||||
CONSUMER="$WORK/consumer"
|
||||
export PRE_COMMIT_HOME="$WORK/pc-home"
|
||||
|
||||
mkdir -p "$HOOK_REPO/plugins/kyberforge/.apm/skills" "$HOOK_REPO/scripts"
|
||||
cp "$REPO_ROOT/.pre-commit-hooks.yaml" "$HOOK_REPO/"
|
||||
cp "$REPO_ROOT/scripts/skill-size-check.sh" "$HOOK_REPO/scripts/"
|
||||
# One skill since ADR-0025 merged skill-audit and agent-audit into factory-audit,
|
||||
# and one vale-wrap.sh with it. Both Vale hook IDs still ship and both are still
|
||||
# registered by the consumer below — they now point at the same entry and differ
|
||||
# only in their `files:` scope, which is exactly what the per-hook attribution in
|
||||
# case 1 exists to prove is still true.
|
||||
skill=factory-audit
|
||||
mkdir -p "$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill"
|
||||
cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/scripts" \
|
||||
"$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/assets" \
|
||||
"$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill/"
|
||||
git -C "$HOOK_REPO" init -q
|
||||
git -C "$HOOK_REPO" add -A
|
||||
git -C "$HOOK_REPO" -c user.email=test@example.invalid -c user.name=test commit -qm "hook repo"
|
||||
HOOK_REV="$(git -C "$HOOK_REPO" rev-parse HEAD)"
|
||||
|
||||
# Every hook scopes by filename, so the consumer needs one file of each shape:
|
||||
# a hook with nothing to match reports `Skipped` and proves nothing. All three
|
||||
# hooks .pre-commit-hooks.yaml ships are registered — an unregistered one would
|
||||
# let a regression (a lost `100755` bit, a bad entry path) reach every external
|
||||
# consumer while this repo's own `repo: local` runs stayed green.
|
||||
mkdir -p "$CONSUMER/skills/demo" "$CONSUMER/agents"
|
||||
git -C "$CONSUMER" init -q
|
||||
cat > "$CONSUMER/.pre-commit-config.yaml" <<EOF
|
||||
repos:
|
||||
- repo: file://$HOOK_REPO
|
||||
rev: $HOOK_REV
|
||||
hooks:
|
||||
- id: kyberforge-vale-audit-skill
|
||||
- id: kyberforge-vale-audit-agent
|
||||
- id: kyberforge-skill-size-check
|
||||
EOF
|
||||
|
||||
# The two fixtures carry DIFFERENT flagged tokens so an alert can never be
|
||||
# credited to the hook that did not raise it. Both bodies land mid-sentence in a
|
||||
# folded block scalar that still spans two physical lines, which is the
|
||||
# flattening the wrapper exists to do.
|
||||
write_fixtures() {
|
||||
local skill_body="$1"
|
||||
local agent_body="${2:-$1}"
|
||||
cat > "$CONSUMER/skills/demo/SKILL.md" <<EOF
|
||||
---
|
||||
name: demo
|
||||
description: >
|
||||
Use when the caller wants a demonstration skill $skill_body across two
|
||||
physical lines of one folded block scalar.
|
||||
metadata:
|
||||
version: "1.0.0"
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
cat > "$CONSUMER/agents/demo.md" <<EOF
|
||||
---
|
||||
name: demo
|
||||
description: >
|
||||
Use when the caller wants a demonstration agent $agent_body across two
|
||||
physical lines of one folded block scalar.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
git -C "$CONSUMER" add -A
|
||||
}
|
||||
|
||||
# Vale prints each linted path as its own header line with that file's alerts
|
||||
# indented beneath it, so an alert belongs to the nearest preceding path line.
|
||||
# Reads a hook log on stdin and prints only the alert lines filed under `$1`.
|
||||
# The `sed` strips vale's ANSI colouring, which it emits into pre-commit's pipe
|
||||
# too, so the header lines compare as plain paths.
|
||||
alerts_for() {
|
||||
sed $'s/\033\\[[0-9;]*m//g' | awk -v want="$1" '
|
||||
/^[^[:space:]].*\.md$/ { cur = $0; next }
|
||||
/^[[:space:]]*[0-9]+:[0-9]+[[:space:]]/ { if (cur == want) print }
|
||||
'
|
||||
}
|
||||
|
||||
# --- 1. Each Vale hook resolves its config and gates its own file shape ---
|
||||
# Asserted per hook, against that hook's own fixture path and its own token. An
|
||||
# aggregate alert count over both hooks' combined output does not prove this:
|
||||
# one fixture description carries every flagged token, so ONE working hook
|
||||
# already clears a `>= 2` threshold. And a hook whose .vale.ini globs match
|
||||
# nothing reaches neither of the guards below — it still MATCHES the file via
|
||||
# its `files:` regex, so pre-commit does not report `Skipped`; vale simply lints
|
||||
# nothing, prints `0 errors ... in 1 file` and exits 0, and the hook shows
|
||||
# `Passed`. Attribution is the only thing that catches it.
|
||||
echo ""
|
||||
echo "--- each Vale hook flags its own fixture in an external consumer repo ---"
|
||||
write_fixtures "that helps with things" "that will utilize things"
|
||||
while IFS='|' read -r HOOK_ID FIXTURE TOKEN; do
|
||||
[[ -n "$HOOK_ID" ]] || continue
|
||||
LOG="$WORK/$HOOK_ID.log"
|
||||
set +e
|
||||
(cd "$CONSUMER" && pre-commit run "$HOOK_ID" --all-files > "$LOG" 2>&1)
|
||||
RC_HOOK=$?
|
||||
set -e
|
||||
if grep -q "does not exist" "$LOG"; then
|
||||
fail "$HOOK_ID hard-errored on a path resolved against the consumer repo (E100) — the bug this test guards against"
|
||||
sed 's/^/ /' "$LOG"
|
||||
elif grep -q "Skipped" "$LOG"; then
|
||||
fail "$HOOK_ID matched no files, so it proved nothing"
|
||||
sed 's/^/ /' "$LOG"
|
||||
elif [[ $RC_HOOK -eq 0 ]]; then
|
||||
fail "$HOOK_ID passed $FIXTURE despite its flagged '$TOKEN' — a .vale.ini glob matching nothing lints zero files and exits 0"
|
||||
sed 's/^/ /' "$LOG"
|
||||
elif alerts_for "$FIXTURE" < "$LOG" | grep -qF "'$TOKEN'"; then
|
||||
pass "$HOOK_ID flattens $FIXTURE and flags its '$TOKEN' in a consumer repo"
|
||||
else
|
||||
fail "$HOOK_ID failed, but no alert quoting '$TOKEN' was filed under $FIXTURE"
|
||||
sed 's/^/ /' "$LOG"
|
||||
fi
|
||||
done <<'EOF'
|
||||
kyberforge-vale-audit-skill|skills/demo/SKILL.md|helps with
|
||||
kyberforge-vale-audit-agent|agents/demo.md|utilize
|
||||
EOF
|
||||
|
||||
# --- 1b. Every rule in the shipped style is asserted to FIRE, not merely to
|
||||
# exist. A Vale rule can be well-formed, load without a diagnostic, and match
|
||||
# nothing at all: `extends: existence` CONCATENATES multiple `raw:` entries
|
||||
# rather than alternating them, so a rule written as a list of alternatives
|
||||
# silently becomes one impossible expression, lints every file clean and exits
|
||||
# 0 — indistinguishable from a corpus with no violations. `Kyberforge.CompositionNote`
|
||||
# was written that way first and passed all 43 skill and agent files before the
|
||||
# defect was found by hand. Each rule gets its own fixture pass, with the token
|
||||
# it must quote attributed to the file that raised it, so one rule's alert can
|
||||
# never stand in for another's.
|
||||
echo ""
|
||||
echo "--- each Kyberforge description rule fires through both shipped hooks ---"
|
||||
write_desc_fixtures() {
|
||||
local skill_desc="$1" agent_desc="$2"
|
||||
cat > "$CONSUMER/skills/demo/SKILL.md" <<EOF
|
||||
---
|
||||
name: demo
|
||||
description: >
|
||||
$skill_desc across two
|
||||
physical lines of one folded block scalar.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
cat > "$CONSUMER/agents/demo.md" <<EOF
|
||||
---
|
||||
name: demo
|
||||
description: >
|
||||
$agent_desc across two
|
||||
physical lines of one folded block scalar.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
git -C "$CONSUMER" add -A
|
||||
}
|
||||
|
||||
run_rule_case() {
|
||||
local label="$1" hook_id="$2" fixture="$3" token="$4"
|
||||
local log="$WORK/rule-$label.log"
|
||||
set +e
|
||||
(cd "$CONSUMER" && pre-commit run "$hook_id" --all-files > "$log" 2>&1)
|
||||
local rc=$?
|
||||
set -e
|
||||
if grep -q "Skipped" "$log"; then
|
||||
fail "$hook_id matched no files for $label, so it proved nothing"
|
||||
sed 's/^/ /' "$log"
|
||||
elif [[ $rc -eq 0 ]]; then
|
||||
fail "$hook_id passed $fixture despite its flagged '$token' — $label matches nothing"
|
||||
sed 's/^/ /' "$log"
|
||||
elif alerts_for "$fixture" < "$log" | grep -qF "'$token'"; then
|
||||
pass "$label fires through $hook_id and quotes '$token' under $fixture"
|
||||
else
|
||||
fail "$hook_id failed, but no $label alert quoting '$token' was filed under $fixture"
|
||||
sed 's/^/ /' "$log"
|
||||
fi
|
||||
}
|
||||
|
||||
# CompositionNote: a distinct banned token per file shape.
|
||||
write_desc_fixtures \
|
||||
"Use when the caller wants a demo skill that composes other skills" \
|
||||
"Use when the caller wants a cross-cutting demo agent"
|
||||
run_rule_case "Kyberforge.CompositionNote" kyberforge-vale-audit-skill skills/demo/SKILL.md "composes"
|
||||
run_rule_case "Kyberforge.CompositionNote" kyberforge-vale-audit-agent agents/demo.md "cross-cutting"
|
||||
|
||||
# DescriptionOpener: the widened pattern catches every non-imperative "This..."
|
||||
# opener, not only the literal "This skill"/"This agent" pair it was anchored to
|
||||
# before. Both fixtures open with "This is", the form two shipped descriptions
|
||||
# used mid-sentence and which the old pattern could not express.
|
||||
write_desc_fixtures \
|
||||
"This is a demo skill for callers who want one" \
|
||||
"This is a demo agent for callers who want one"
|
||||
run_rule_case "Kyberforge.DescriptionOpener" kyberforge-vale-audit-skill skills/demo/SKILL.md "This"
|
||||
run_rule_case "Kyberforge.DescriptionOpener" kyberforge-vale-audit-agent agents/demo.md "This"
|
||||
|
||||
# --- 2. Clean files pass — the hooks gate, they don't just always fail ---
|
||||
echo ""
|
||||
echo "--- all three hooks pass clean files in an external consumer repo ---"
|
||||
write_fixtures "of the packaged hook contract"
|
||||
set +e
|
||||
(cd "$CONSUMER" && pre-commit run --all-files > "$WORK/clean.log" 2>&1)
|
||||
RC_CLEAN=$?
|
||||
set -e
|
||||
if grep -q "Skipped" "$WORK/clean.log"; then
|
||||
fail "a hook matched no files on the clean run, so it proved nothing"
|
||||
sed 's/^/ /' "$WORK/clean.log"
|
||||
elif [[ $RC_CLEAN -eq 0 ]]; then
|
||||
pass "all three hooks exit 0 on clean files"
|
||||
else
|
||||
fail "hooks failed on clean files (rc=$RC_CLEAN)"
|
||||
sed 's/^/ /' "$WORK/clean.log"
|
||||
fi
|
||||
|
||||
# --- 3. The size hook gates too. It ran clean above, which is what proves it
|
||||
# is executable and its entry path resolves; this half proves it still fails a
|
||||
# file that breaks the ceiling rather than passing everything. ---
|
||||
echo ""
|
||||
echo "--- kyberforge-skill-size-check fails an oversized SKILL.md in an external consumer repo ---"
|
||||
mkdir -p "$CONSUMER/skills/oversized"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: oversized"
|
||||
echo "description: Use when the caller wants an oversized fixture."
|
||||
echo "---"
|
||||
for ((i = 1; i <= 600; i++)); do
|
||||
echo "word"
|
||||
done
|
||||
} > "$CONSUMER/skills/oversized/SKILL.md"
|
||||
git -C "$CONSUMER" add -A
|
||||
set +e
|
||||
(cd "$CONSUMER" && pre-commit run kyberforge-skill-size-check --all-files > "$WORK/size.log" 2>&1)
|
||||
RC_SIZE=$?
|
||||
set -e
|
||||
if [[ $RC_SIZE -ne 0 ]] && grep -q "500-line ceiling" "$WORK/size.log"; then
|
||||
pass "kyberforge-skill-size-check exits non-zero and names the ceiling it broke"
|
||||
else
|
||||
fail "kyberforge-skill-size-check did not gate an oversized SKILL.md (rc=$RC_SIZE)"
|
||||
sed 's/^/ /' "$WORK/size.log"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
@@ -21,8 +21,8 @@ fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
# Vale absent skips the Vale-DEPENDENT cases, not the suite. An early `exit 77`
|
||||
# here used to skip everything, including the checks that are plain greps and
|
||||
# awk over the config and the two hook manifests (cases 16, 26-28's static
|
||||
# halves, 31 Parts A/B, 32, 33) -- so a machine without vale reported a skip
|
||||
# awk over the config and the hook config (cases 16, 26-28's static
|
||||
# halves, 31 Parts A/B, 32) -- so a machine without vale reported a skip
|
||||
# while never looking at a manifest it could have read. Those still run; the
|
||||
# suite exits 77 at the end only if they all passed, so run-tests.sh keeps
|
||||
# reporting SKIPPED and `--strict` keeps turning that skip into a failure. A
|
||||
@@ -452,10 +452,11 @@ else
|
||||
fi
|
||||
|
||||
# --- 12. With no --config at all, the wrapper falls back to its own sibling
|
||||
# assets/vale/.vale.ini. `.pre-commit-hooks.yaml` relies on this: pre-commit
|
||||
# prefixes only entry[0] with the hook-repo clone path, so a --config argument
|
||||
# there resolves against the consuming repo and hard-errors (E100) for every
|
||||
# external consumer.
|
||||
# assets/vale/.vale.ini. factory-audit's Step 1 and both prefilter hooks rely
|
||||
# on this: they pass no --config. A published hook manifest would too, were it
|
||||
# restored (ADR-0014): pre-commit prefixes only entry[0] with the hook-repo clone
|
||||
# path, so a --config argument there resolves against the consuming repo and
|
||||
# hard-errors (E100) for every external consumer.
|
||||
echo ""
|
||||
echo "--- defaults --config to the wrapper's own sibling assets/vale/.vale.ini ---"
|
||||
FIXTURE12="$(make_fixture 2)"
|
||||
@@ -464,7 +465,7 @@ OUT12=$(run_wrap "$FIXTURE12" plugins/testplugin/skills/zzzskill/SKILL.md)
|
||||
if grep -q "VagueWording" <<< "$OUT12"; then
|
||||
pass "a --config-less invocation uses the wrapper's bundled config"
|
||||
else
|
||||
fail "a --config-less invocation found no config — external pre-commit consumers get E100, the bug this test guards against"
|
||||
fail "a --config-less invocation found no config — factory-audit's Step 1 and both prefilter hooks pass no --config, so they would get E100"
|
||||
fi
|
||||
|
||||
# --- 13. No GNU-only `realpath -m`. macOS ships the BSD realpath, which has no
|
||||
@@ -1080,7 +1081,7 @@ elif [[ "$WRAPPED21_FILES" != "$BARE21_FILES" ]]; then
|
||||
fail "the directory walk dropped a symlinked file: wrapper saw '$WRAPPED21_FILES', bare vale '$BARE21_FILES'"
|
||||
# A here-string, not `echo "$WRAPPED21" | grep -q`: the match sits on line 3 of
|
||||
# 8, and under pipefail grep -q exiting early can SIGPIPE echo mid-write and
|
||||
# fail this branch on correct output (see tests/test-check-release-needed.sh).
|
||||
# fail this branch on correct output (see docs/spec/gates.md, Tests).
|
||||
elif grep -q "VagueWording" <<< "$WRAPPED21"; then
|
||||
pass "a symlinked file under a directory argument is mirrored, flattened and flagged"
|
||||
else
|
||||
@@ -1341,7 +1342,7 @@ fi
|
||||
# exits 0. Every gate in this repo reads that as a pass.
|
||||
#
|
||||
# ADR-0014's reason for two hook IDs was this same problem, and .pre-commit-
|
||||
# hooks.yaml still carries both IDs after the merge for that reason.
|
||||
# config.yaml still carries both IDs after the merge for that reason.
|
||||
|
||||
# One representative path per file shape the prefilter is supposed to cover,
|
||||
# tagged with the `.vale.ini` section that is supposed to cover it and with
|
||||
@@ -1365,8 +1366,15 @@ fi
|
||||
#
|
||||
# `demo.md` (bare, no `.agent.md` suffix) exercises `[**/agents/*.md]` in
|
||||
# isolation, not because any current `.apm/agents/*` file has that shape -- per
|
||||
# ADR-0016 they are all `*.agent.md`. `.pre-commit-hooks.yaml`'s agent regex
|
||||
# still covers the bare shape, which is what keeps the row honest.
|
||||
# ADR-0016 they are all `*.agent.md`. factory-audit's agent flow can still be
|
||||
# handed that shape in a consuming repo, which is what keeps the row honest.
|
||||
#
|
||||
# These rows describe what factory-audit's own Vale call can be handed at
|
||||
# runtime in any repo, not what this repo's hooks select: most of them sit
|
||||
# outside `.pre-commit-config.yaml`'s `^plugins/`-anchored `files:` regexes on
|
||||
# purpose. The hook-scope half that once held each row to a hook regex read the
|
||||
# published `.pre-commit-hooks.yaml`, which is retired (ADR-0014, 2026-09-16
|
||||
# amendment); case 32 owns the local hooks' scope.
|
||||
PROBE_TABLE28="$(cat <<'EOF_PROBE28'
|
||||
plugins/demo/.apm/skills/demo/SKILL.md|[**/SKILL.md]|isolating
|
||||
.claude/skills/demo/SKILL.md|[**/SKILL.md]|isolating
|
||||
@@ -1428,12 +1436,11 @@ files_scanned28() {
|
||||
| tail -1
|
||||
}
|
||||
|
||||
# Prints `<id>|<files regex>` for every hook in the given pre-commit manifest
|
||||
# whose entry is factory-audit's vale-wrap.sh -- either manifest, since the two
|
||||
# carry the same two hooks in the same shape. Records are delimited by their
|
||||
# Prints `<id>|<files regex>` for every hook in the given pre-commit config
|
||||
# whose entry is factory-audit's vale-wrap.sh. Records are delimited by their
|
||||
# `- id:` line, so this does not depend on `entry:` preceding `files:` within a
|
||||
# record. Case 28 wants the regexes alone and case 32 needs to know which hook
|
||||
# each belongs to, so the id is carried here and dropped by the wrapper below.
|
||||
# record. Case 32 needs to know which hook each regex belongs to, so the id is
|
||||
# carried here.
|
||||
hook_records28() {
|
||||
local manifest="$1" id raw
|
||||
[[ -f "$manifest" ]] || return 0
|
||||
@@ -1456,29 +1463,8 @@ hook_records28() {
|
||||
done
|
||||
}
|
||||
|
||||
# Prints the `files:` regex of every hook in the given manifest whose entry is
|
||||
# factory-audit's vale-wrap.sh.
|
||||
hook_file_regexes28() {
|
||||
hook_records28 "$1" | cut -d'|' -f2-
|
||||
}
|
||||
|
||||
matches_any_regex28() {
|
||||
local rel="$1" regexes="$2" re
|
||||
[[ -n "$regexes" ]] || return 1
|
||||
while IFS= read -r re; do
|
||||
[[ -n "$re" ]] || continue
|
||||
if grep -Eq "$re" <<< "$rel"; then
|
||||
return 0
|
||||
fi
|
||||
done <<EOF_RE28
|
||||
$regexes
|
||||
EOF_RE28
|
||||
return 1
|
||||
}
|
||||
|
||||
TREE28="$(build_probe_tree28)"
|
||||
new_fixture "$TREE28"
|
||||
HOOK_REGEXES28="$(hook_file_regexes28 "$REPO_ROOT/.pre-commit-hooks.yaml")"
|
||||
|
||||
echo ""
|
||||
echo "--- every .vale.ini glob section actually scans a real file shape ---"
|
||||
@@ -1526,24 +1512,13 @@ fi
|
||||
# One `PASS|<rel>|<message>` or `FAIL|<rel>|<message>` line per probe row, for
|
||||
# the config at $1. A function rather than an inline loop so Part B can hold a
|
||||
# mutated copy to this exact logic -- a second, "equivalent" loop for the
|
||||
# fixture would prove nothing about the live check. With $2 = false only the
|
||||
# hook-scope half runs: that half reads .pre-commit-hooks.yaml, not vale, so a
|
||||
# machine without vale still gets it.
|
||||
# fixture would prove nothing about the live check. With $2 = false it prints
|
||||
# nothing: every remaining half of it needs vale.
|
||||
probe_coverage28() {
|
||||
local cfg="$1" with_vale="$2" rel sec report count
|
||||
[[ "$with_vale" == true ]] || return 0
|
||||
while IFS='|' read -r rel sec _; do
|
||||
[[ -n "$rel" ]] || continue
|
||||
if ! matches_any_regex28 "$rel" "$HOOK_REGEXES28"; then
|
||||
# Original wording: the probe path is stale, or the hook was rescoped away
|
||||
# from a shape it still needs to lint. Either way the row below stops
|
||||
# describing anything the push gate actually hands to vale.
|
||||
echo "FAIL|$rel|$rel matches no 'files:' regex of any factory-audit vale hook in .pre-commit-hooks.yaml — the probe path is stale, or the hook was rescoped away from a shape it still needs to lint"
|
||||
continue
|
||||
fi
|
||||
if [[ "$with_vale" != true ]]; then
|
||||
echo "PASS|$rel|$rel is in scope of a published factory-audit vale hook (glob coverage not checked: Vale-dependent half held back)"
|
||||
continue
|
||||
fi
|
||||
report="$(vale_report28 "$cfg" "$TREE28" "$rel")"
|
||||
count="$(files_scanned28 "$report")"
|
||||
if [[ -z "$count" ]]; then
|
||||
@@ -1995,25 +1970,29 @@ fi
|
||||
# which worked only because the two skills gave the two hooks two distinct entry
|
||||
# paths. After the merge both hooks share one `entry:`, so that selector can no
|
||||
# longer tell them apart and a faithful port would have to key on hook `id:`
|
||||
# instead. Case 33 is that port; this case covers the separate question of
|
||||
# whether each local hook selects a live corpus at all. The zero-match half of the hole stands on its own, and nothing
|
||||
# else in the repo covers it: tests/test-vale-hooks-consumer.sh synthesises its
|
||||
# own consumer config out of `.pre-commit-hooks.yaml` and never reads the local
|
||||
# one, and cases 28-30 read `.pre-commit-hooks.yaml` too. This repo's OWN
|
||||
# prefilter regexes -- `.pre-commit-config.yaml`'s vale-audit-prefilter-skill and
|
||||
# vale-audit-prefilter-agent -- are therefore asserted by no test at all. Narrow
|
||||
# either one to match zero files and every gate still passes: pre-commit does not
|
||||
# instead. That port was case 33, deleted with the published
|
||||
# `.pre-commit-hooks.yaml` it compared against (ADR-0014, 2026-09-16
|
||||
# amendment); its one guard that did not depend on the second manifest -- a
|
||||
# local regex narrowed to a single plugin -- is property 3 below. Nothing else
|
||||
# in the repo asserts this repo's OWN prefilter regexes --
|
||||
# `.pre-commit-config.yaml`'s vale-audit-prefilter-skill and
|
||||
# vale-audit-prefilter-agent. Narrow either one to match zero files and every gate still passes: pre-commit does not
|
||||
# error on a hook that matches nothing, it simply never runs it. That is the same
|
||||
# silent-zero failure mode case 28 guards on the vale side of this pipeline, one
|
||||
# layer up -- there the glob scans 0 files and exits 0, here the hook is handed 0
|
||||
# files and never starts.
|
||||
#
|
||||
# Two properties, because matching SOMETHING is not the same as matching the
|
||||
# Three properties, because matching SOMETHING is not the same as matching the
|
||||
# right thing: a regex loosened to `^plugins/` would match hundreds of files and
|
||||
# clear a bare non-emptiness check while handing vale a corpus it has no glob
|
||||
# for. So each hook must also select only its own artifact class -- ADR-0014's
|
||||
# reason for two hook IDs, carried across the merge by ADR-0025's comment in the
|
||||
# config, is precisely that the two scopes stay independently addressable.
|
||||
# And each hook must select ALL of its class's authoring source: a regex
|
||||
# narrowed from `^plugins/[^/]+/...` to `^plugins/kyberforge/...` still matches
|
||||
# tracked files, all of the right class, while silently dropping every other
|
||||
# plugin out of the prefilter. Measured before any case caught it: that exact
|
||||
# narrowing left 6 of 38 skills prefiltered and the whole suite green.
|
||||
|
||||
echo ""
|
||||
echo "--- each .pre-commit-config.yaml vale prefilter hook matches a real, correctly-classed file ---"
|
||||
@@ -2031,7 +2010,7 @@ REPO_FILES32="$(cd "$REPO_ROOT" && git ls-files)"
|
||||
# the live check.
|
||||
prefilter_scope_failures32() {
|
||||
local config="$1" files="$2"
|
||||
local records id re class matched count offenders offending m
|
||||
local records id re class matched count offenders offending m corpus missing nmissing
|
||||
local seen_skill=false seen_agent=false bad=""
|
||||
records="$(hook_records28 "$config")"
|
||||
if [[ -z "$records" ]]; then
|
||||
@@ -2082,6 +2061,19 @@ EOF_MATCHED32
|
||||
if [[ "$offending" -gt 0 ]]; then
|
||||
bad+="[$id: 'files: $re' selects $count file(s), $offending of them outside the $class artifact class, so the two prefilter scopes are no longer independently addressable and vale is handed files no glob in its config covers — first: $offenders] "
|
||||
fi
|
||||
# The class's full authoring-source corpus, per the layout AGENTS.md fixes
|
||||
# (`plugins/<name>/.apm/` is the only authoring source). Every path in it
|
||||
# must be selected, or part of the corpus is silently unprefiltered.
|
||||
if [[ "$class" == skill ]]; then
|
||||
corpus="$(printf '%s\n' "$files" | { grep -E '^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$' || true; })"
|
||||
else
|
||||
corpus="$(printf '%s\n' "$files" | { grep -E '^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$' || true; })"
|
||||
fi
|
||||
missing="$(comm -23 <(printf '%s\n' "$corpus" | grep . | sort) <(printf '%s\n' "$matched" | grep . | sort) || true)"
|
||||
nmissing="$(printf '%s\n' "$missing" | grep -c . || true)"
|
||||
if [[ "$nmissing" -gt 0 ]]; then
|
||||
bad+="[$id: 'files: $re' misses $nmissing tracked $class file(s) under plugins/*/.apm/, so part of the corpus is never prefiltered while every gate still reports a pass — first: $(printf '%s\n' "$missing" | head -3 | tr '\n' ' ')] "
|
||||
fi
|
||||
done <<EOF_RECORDS32
|
||||
$records
|
||||
EOF_RECORDS32
|
||||
@@ -2129,205 +2121,35 @@ else
|
||||
pass "narrowing either hook's 'files:' regex to match zero files is caught by Part A, which is what makes its pass mean something"
|
||||
fi
|
||||
|
||||
# --- 33. The local and published vale hooks agree on every shared file shape --
|
||||
#
|
||||
# The cross-manifest `files:` agreement check of the deleted
|
||||
# scripts/check-vale-style-sync.sh (ADR-0025), ported. Case 32 does not cover
|
||||
# it: a local regex narrowed from `^plugins/[^/]+/...` to
|
||||
# `^plugins/kyberforge/...` still matches tracked files, all of them SKILL.md,
|
||||
# so it clears both of 32's properties while silently dropping every other
|
||||
# plugin's skills out of this repo's prefilter. Measured before this case
|
||||
# existed: that exact narrowing left the whole suite green.
|
||||
#
|
||||
# The original's comment on why the two manifests are compared per hook rather
|
||||
# than unioned, verbatim in substance: "A union here previously let a probe that
|
||||
# matched only the older, looser .pre-commit-hooks.yaml pattern read as 'in
|
||||
# scope' even after .pre-commit-config.yaml's copy of the same hook had been
|
||||
# narrowed away from it -- silently masking exactly the kind of hook-rescoping
|
||||
# drift this script exists to catch."
|
||||
#
|
||||
# What changed in the port is the selector and nothing else. The original found
|
||||
# each manifest's hook by `entry ~ skill "/scripts/vale-wrap.sh"`, which told the
|
||||
# two hooks apart only because two skills gave them two entry paths; after the
|
||||
# merge both hooks share one entry. They are paired by `id:` instead, from an
|
||||
# explicit table. The pairing is explicit rather than inferred from an id suffix
|
||||
# so that renaming either id fails here by name instead of quietly dropping a
|
||||
# class out of the comparison. The probe table and its shared/hooks-only scopes
|
||||
# are the original's rows, unchanged:
|
||||
#
|
||||
# shared -- a shape this repo's own layout has, so both manifests must
|
||||
# agree on it. This is what catches the narrowing above.
|
||||
# hooks-only -- a shape only the layout-agnostic published manifest has to
|
||||
# cover. `.pre-commit-config.yaml` pinning this repo's own
|
||||
# `plugins/*/.apm/` layout is by design, not drift; bare
|
||||
# `agents/demo.md` is hooks-only because per ADR-0016 every
|
||||
# `.apm/agents/` file is `*.agent.md`.
|
||||
#
|
||||
# A probe in scope of neither manifest fails too, exactly as in the original:
|
||||
# the probe path is stale, or both hooks were rescoped away from it.
|
||||
|
||||
HOOK_PAIRS33="$(cat <<'EOF_PAIRS33'
|
||||
skill|kyberforge-vale-audit-skill|vale-audit-prefilter-skill
|
||||
agent|kyberforge-vale-audit-agent|vale-audit-prefilter-agent
|
||||
EOF_PAIRS33
|
||||
)"
|
||||
|
||||
PROBES33="$(cat <<'EOF_PROBES33'
|
||||
skill|plugins/demo/.apm/skills/demo/SKILL.md|shared
|
||||
skill|.claude/skills/demo/SKILL.md|hooks-only
|
||||
agent|plugins/demo/.apm/agents/demo.md|hooks-only
|
||||
agent|plugins/demo/.apm/agents/demo.agent.md|shared
|
||||
agent|.claude/agents/demo.md|hooks-only
|
||||
agent|copilot/demo.agent.md|hooks-only
|
||||
EOF_PROBES33
|
||||
)"
|
||||
|
||||
# Prints the `files:` regex of the hook whose id is exactly $2 in manifest $1,
|
||||
# or nothing. Records are delimited by their `- id:` line, as in
|
||||
# hook_records28, but selected by id alone: after the merge `entry:` no longer
|
||||
# distinguishes them.
|
||||
hook_regex_by_id33() {
|
||||
local manifest="$1" want="$2" raw
|
||||
[[ -f "$manifest" ]] || return 0
|
||||
raw="$(WANT="$want" awk '
|
||||
function flush() {
|
||||
if (id == ENVIRON["WANT"] && files != "") print files
|
||||
id = ""; files = ""
|
||||
}
|
||||
/^[ \t]*-[ \t]*id:/ { flush(); id = $0; sub(/^[ \t]*-[ \t]*id:[ \t]*/, "", id); sub(/[ \t]+$/, "", id) }
|
||||
/^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) }
|
||||
END { flush() }
|
||||
' "$manifest" | head -1)"
|
||||
raw="${raw%\'}"; raw="${raw#\'}"
|
||||
raw="${raw%\"}"; raw="${raw#\"}"
|
||||
printf '%s' "$raw"
|
||||
}
|
||||
|
||||
# Prints one failure token per defect for published manifest $1 against local
|
||||
# manifest $2; empty output means every probe is in scope of at least one of its
|
||||
# class's hooks and every shared probe is in scope of both. Manifest paths are
|
||||
# arguments so Part B runs this exact function against mutated copies.
|
||||
cross_manifest_failures33() {
|
||||
local published="$1" local_cfg="$2"
|
||||
local class pub_id loc_id rel scope pub_re loc_re in_hooks in_config
|
||||
local bad="" checked=0 shared_classes="" missing_classes=""
|
||||
while IFS='|' read -r class pub_id loc_id; do
|
||||
[[ -n "$class" ]] || continue
|
||||
if [[ -z "$(hook_regex_by_id33 "$published" "$pub_id")" ]]; then
|
||||
bad+="[no hook with id '$pub_id' and a files: regex in ${published##*/}, so the $class class is compared against nothing] "
|
||||
missing_classes+="$class "
|
||||
fi
|
||||
if [[ -z "$(hook_regex_by_id33 "$local_cfg" "$loc_id")" ]]; then
|
||||
bad+="[no hook with id '$loc_id' and a files: regex in ${local_cfg##*/}, so the $class class is compared against nothing] "
|
||||
missing_classes+="$class "
|
||||
fi
|
||||
done <<EOF_PAIRCHK33
|
||||
$HOOK_PAIRS33
|
||||
EOF_PAIRCHK33
|
||||
while IFS='|' read -r class rel scope; do
|
||||
[[ -n "$class" ]] || continue
|
||||
pub_id="$(printf '%s\n' "$HOOK_PAIRS33" | awk -F'|' -v c="$class" '$1 == c { print $2 }')"
|
||||
loc_id="$(printf '%s\n' "$HOOK_PAIRS33" | awk -F'|' -v c="$class" '$1 == c { print $3 }')"
|
||||
if [[ -z "$pub_id" || -z "$loc_id" ]]; then
|
||||
bad+="[probe $rel names class '$class', which has no row in the hook pair table] "
|
||||
continue
|
||||
fi
|
||||
pub_re="$(hook_regex_by_id33 "$published" "$pub_id")"
|
||||
loc_re="$(hook_regex_by_id33 "$local_cfg" "$loc_id")"
|
||||
# A missing hook was reported above; comparing against it would add a
|
||||
# second message for the same defect.
|
||||
[[ -n "$pub_re" && -n "$loc_re" ]] || continue
|
||||
checked=$((checked + 1))
|
||||
[[ "$scope" != shared ]] || shared_classes+="$class "
|
||||
in_hooks=false
|
||||
matches_any_regex28 "$rel" "$pub_re" && in_hooks=true
|
||||
in_config=false
|
||||
matches_any_regex28 "$rel" "$loc_re" && in_config=true
|
||||
if [[ "$in_hooks" == false && "$in_config" == false ]]; then
|
||||
bad+="[$rel matches neither $pub_id's nor $loc_id's 'files:' regex — the probe path is stale, or the hook was rescoped away from a shape it still needs to lint] "
|
||||
elif [[ "$scope" == shared && "$in_hooks" != "$in_config" ]]; then
|
||||
bad+="[$rel is in scope of $pub_id in ${published##*/} (hooks=$in_hooks) but not of $loc_id in ${local_cfg##*/} (config=$in_config), or vice versa — the local and published 'files:' regexes have drifted, and the narrower one silently stops prefiltering that shape] "
|
||||
fi
|
||||
done <<EOF_PROBECHK33
|
||||
$PROBES33
|
||||
EOF_PROBECHK33
|
||||
# The original's PROBES_CHECKED floor, plus one it lacked: a class with no
|
||||
# shared probe is never compared across manifests at all.
|
||||
if [[ "$checked" -eq 0 && -z "$missing_classes" ]]; then
|
||||
bad+="[no probe row was checked, so the agreement check verified nothing] "
|
||||
fi
|
||||
while IFS='|' read -r class _ _; do
|
||||
[[ -n "$class" ]] || continue
|
||||
# A class whose hook is missing was already reported, and its probes were
|
||||
# skipped for that reason, not for want of a shared row.
|
||||
[[ " $missing_classes" != *" $class "* ]] || continue
|
||||
[[ " $shared_classes" == *" $class "* ]] \
|
||||
|| bad+="[the $class class has no shared probe row, so its two hooks are never compared] "
|
||||
done <<EOF_CLASSCHK33
|
||||
$HOOK_PAIRS33
|
||||
EOF_CLASSCHK33
|
||||
printf '%s' "$bad"
|
||||
}
|
||||
|
||||
echo ""
|
||||
echo "--- the local and published vale hooks agree on every shared file shape ---"
|
||||
|
||||
PUBLISHED33="$REPO_ROOT/.pre-commit-hooks.yaml"
|
||||
|
||||
# Part A: the live assertion, against the two real manifests.
|
||||
AGREE_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$PC_CONFIG32")"
|
||||
if [[ -n "$AGREE_FAILS33" ]]; then
|
||||
fail ".pre-commit-config.yaml and .pre-commit-hooks.yaml disagree on the vale hooks' scope: $AGREE_FAILS33"
|
||||
else
|
||||
pass "every probe is in scope of its class's hooks, and both manifests agree on every shared shape"
|
||||
fi
|
||||
|
||||
# Part B: proof that Part A can fail. Each mutation lands in a COPY of
|
||||
# .pre-commit-config.yaml, never the real file.
|
||||
#
|
||||
# skill -- the exact narrowing that slipped through before this case existed.
|
||||
# agent -- the same narrowing applied to the agent hook.
|
||||
# id -- the local skill hook renamed, which must fail by name rather than
|
||||
# drop the skill class out of the comparison.
|
||||
#
|
||||
# Replacement is fixed-string, via ENVIRON, so the regex's backslashes reach awk
|
||||
# literally; `awk -v` would process them as escape sequences.
|
||||
narrow_config33() {
|
||||
OLD33="$2" NEW33="$3" awk '
|
||||
$0 ~ ENVIRON["LINE33"] {
|
||||
i = index($0, ENVIRON["OLD33"])
|
||||
if (i) $0 = substr($0, 1, i - 1) ENVIRON["NEW33"] substr($0, i + length(ENVIRON["OLD33"]))
|
||||
# Part C: proof that property 3 can fail. Each hook's regex is narrowed to one
|
||||
# plugin in a COPY of the config -- the exact narrowing that once slipped
|
||||
# through. Replacement is fixed-string, via ENVIRON, so the regex's backslashes
|
||||
# reach awk literally; `awk -v` would process them as escape sequences.
|
||||
narrow_config32() {
|
||||
OLD32="$2" NEW32="$3" awk '
|
||||
/^[ \t]*files:/ {
|
||||
i = index($0, ENVIRON["OLD32"])
|
||||
if (i) $0 = substr($0, 1, i - 1) ENVIRON["NEW32"] substr($0, i + length(ENVIRON["OLD32"]))
|
||||
}
|
||||
{ print }
|
||||
' "$1"
|
||||
}
|
||||
MUT33="$(mktemp -d)"
|
||||
new_fixture "$MUT33"
|
||||
LINE33='^[ \t]*files:' narrow_config33 "$PC_CONFIG32" \
|
||||
'^plugins/[^/]+/\.apm/skills/' '^plugins/kyberforge/\.apm/skills/' > "$MUT33/skill.yaml"
|
||||
LINE33='^[ \t]*files:' narrow_config33 "$PC_CONFIG32" \
|
||||
'^plugins/[^/]+/\.apm/agents/' '^plugins/kyberforge/\.apm/agents/' > "$MUT33/agent.yaml"
|
||||
LINE33='^[ \t]*-[ \t]*id:' narrow_config33 "$PC_CONFIG32" \
|
||||
'vale-audit-prefilter-skill' 'vale-audit-prefilter-skill-renamed' > "$MUT33/id.yaml"
|
||||
SKILL_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/skill.yaml")"
|
||||
AGENT_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/agent.yaml")"
|
||||
ID_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/id.yaml")"
|
||||
if ! grep -qF '^plugins/kyberforge/\.apm/skills/' "$MUT33/skill.yaml" \
|
||||
|| ! grep -qF '^plugins/kyberforge/\.apm/agents/' "$MUT33/agent.yaml" \
|
||||
|| ! grep -qF 'vale-audit-prefilter-skill-renamed' "$MUT33/id.yaml"; then
|
||||
fail "a mutation never reached its copied config, so Part B mutated nothing and proves nothing about Part A"
|
||||
elif ! grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is in scope of kyberforge-vale-audit-skill" <<< "$SKILL_FAILS33"; then
|
||||
fail "narrowing the local skill hook to ^plugins/kyberforge/ did not fail Part A, so the prefilter can drop every other plugin's skills with every gate green: ${SKILL_FAILS33:-<no failure>}"
|
||||
elif [[ -z "$AGREE_FAILS33" ]] && grep -qF "kyberforge-vale-audit-agent" <<< "$SKILL_FAILS33"; then
|
||||
# Only meaningful against a clean base: when Part A already failed, the copy
|
||||
# inherits that defect, and reporting it again here would be one defect twice.
|
||||
fail "narrowing only the skill hook also reported an agent-class defect, so the comparison is not confined to its class: $SKILL_FAILS33"
|
||||
elif ! grep -qF "[plugins/demo/.apm/agents/demo.agent.md is in scope of kyberforge-vale-audit-agent" <<< "$AGENT_FAILS33"; then
|
||||
fail "narrowing the local agent hook to ^plugins/kyberforge/ did not fail Part A: ${AGENT_FAILS33:-<no failure>}"
|
||||
elif ! grep -qF "[no hook with id 'vale-audit-prefilter-skill' and a files: regex in id.yaml" <<< "$ID_FAILS33"; then
|
||||
fail "renaming the local skill hook's id did not fail Part A by name, so the skill class could fall out of the comparison silently: ${ID_FAILS33:-<no failure>}"
|
||||
narrow_config32 "$PC_CONFIG32" \
|
||||
'^plugins/[^/]+/\.apm/skills/' '^plugins/kyberforge/\.apm/skills/' > "$MUT32/skill.yaml"
|
||||
narrow_config32 "$PC_CONFIG32" \
|
||||
'^plugins/[^/]+/\.apm/agents/' '^plugins/kyberforge/\.apm/agents/' > "$MUT32/agent.yaml"
|
||||
NARROW_SKILL32="$(prefilter_scope_failures32 "$MUT32/skill.yaml" "$REPO_FILES32")"
|
||||
NARROW_AGENT32="$(prefilter_scope_failures32 "$MUT32/agent.yaml" "$REPO_FILES32")"
|
||||
if cmp -s "$PC_CONFIG32" "$MUT32/skill.yaml" || cmp -s "$PC_CONFIG32" "$MUT32/agent.yaml"; then
|
||||
fail "a one-plugin narrowing left the copied config unchanged, so Part C narrowed nothing and proves nothing about property 3"
|
||||
elif ! grep -qF "vale-audit-prefilter-skill: 'files: ^plugins/kyberforge/" <<< "$NARROW_SKILL32" \
|
||||
|| ! grep -qF "tracked skill file(s)" <<< "$NARROW_SKILL32"; then
|
||||
fail "narrowing the skill hook to ^plugins/kyberforge/ did not fail Part A as an incomplete corpus: ${NARROW_SKILL32:-<no failure>}"
|
||||
elif ! grep -qF "vale-audit-prefilter-agent: 'files: ^plugins/kyberforge/" <<< "$NARROW_AGENT32" \
|
||||
|| ! grep -qF "tracked agent file(s)" <<< "$NARROW_AGENT32"; then
|
||||
fail "narrowing the agent hook to ^plugins/kyberforge/ did not fail Part A as an incomplete corpus: ${NARROW_AGENT32:-<no failure>}"
|
||||
else
|
||||
pass "narrowing either local hook to one plugin, or renaming one, is caught by Part A"
|
||||
pass "narrowing either hook's 'files:' regex to one plugin is caught by Part A as an incomplete corpus"
|
||||
fi
|
||||
|
||||
# --- 34. Every glob section loads a real style, asserted without vale --------
|
||||
@@ -2342,7 +2164,7 @@ fi
|
||||
# file the section matches with NO rule, prints `0 errors ... in 1 file` and
|
||||
# exits 0, and every gate in this repo reads that as a pass. That is the
|
||||
# silent-pass class ADR-0013 exists to prevent, and `--strict` -- which a
|
||||
# consumer's clone does not run -- is the only thing standing in front of it.
|
||||
# consuming repo's install does not run -- is the only thing standing in front of it.
|
||||
#
|
||||
# So this case asks the same question of the config TEXT, with no dependency on
|
||||
# vale being installed. It is a separate case rather than an addition to either
|
||||
@@ -2354,9 +2176,9 @@ fi
|
||||
# `[[ "$sp" == /* ]] || sp="$dir/$sp"`, which ACCEPTS an absolute StylesPath --
|
||||
# a path that resolves on the machine that wrote it and on no other. Relative
|
||||
# resolution against the config's own directory is the only reason the bundled
|
||||
# styles are found under a CONSUMING repo's clone prefix, so an absolute one
|
||||
# passes every check here and hard-fails every external consumer of
|
||||
# .pre-commit-hooks.yaml. It is asserted here, not in case 0, to keep case 0's
|
||||
# styles are found under a CONSUMING repo's install of factory-audit, so an
|
||||
# absolute one passes every check here and hard-fails every repo that installs
|
||||
# the plugin. It is asserted here, not in case 0, to keep case 0's
|
||||
# scope the one its comment describes.
|
||||
|
||||
VALE_ASSETS34="$FACTORY_AUDIT/assets/vale"
|
||||
@@ -2383,7 +2205,7 @@ section_styles34() {
|
||||
|
||||
# One failure token per defect in the config at $1; empty output means every
|
||||
# section would load at least one real style for a consumer. The section floor
|
||||
# is here for the same reason case 28 Part A and case 33 carry theirs: a config
|
||||
# is here for the same reason case 28 Part A carries its own: a config
|
||||
# whose sections were all deleted lints nothing at all, and without a floor this
|
||||
# function would report it clean.
|
||||
style_load_defects34() {
|
||||
@@ -2402,7 +2224,7 @@ style_load_defects34() {
|
||||
return 0
|
||||
fi
|
||||
if [[ "$sp" == /* ]]; then
|
||||
printf '%s' "[${cfg##*/} sets the ABSOLUTE StylesPath '$sp'; it resolves only on the machine that wrote it, and a consuming repo's clone -- which is the only reason .pre-commit-hooks.yaml ships these styles -- gets 'path does not exist'] "
|
||||
printf '%s' "[${cfg##*/} sets the ABSOLUTE StylesPath '$sp'; it resolves only on the machine that wrote it, and a consuming repo's install of factory-audit -- which is the only reason these styles ship -- gets 'path does not exist'] "
|
||||
return 0
|
||||
fi
|
||||
if [[ ! -d "$dir/$sp" ]]; then
|
||||
@@ -2490,7 +2312,7 @@ EOF_MUT34
|
||||
ABS34="$(mktemp -d)"
|
||||
new_fixture "$ABS34"
|
||||
cp -r "$VALE_ASSETS34/." "$ABS34/"
|
||||
# Written through ENVIRON, as in narrow_config33, so the path reaches awk
|
||||
# Written through ENVIRON, as in narrow_config32, so the path reaches awk
|
||||
# literally rather than through `-v`'s escape processing.
|
||||
ABS_SP34="$ABS34/styles" awk '
|
||||
/^[ \t]*StylesPath[ \t]*=/ { print "StylesPath = " ENVIRON["ABS_SP34"]; next }
|
||||
|
||||
Reference in New Issue
Block a user