Files
holocron/tests/test-check-release-needed.sh
Defame1297 acd2f1d422 fix(lint): harden check-release-needed.sh, script the vale-style sync
A review of PR #85's last two commits (1164f3a, 4d018af) found the new
release-gate script fails open in four separate ways, and the new drift
check for the duplicated Vale styles only ever detects drift after a
human already hand-edited both copies out of sync.

check-release-needed.sh:
- The `-e` existence filter dropped a RELEASE_PATHS entry from the diff
  pathspec once it was deleted from the tree, so deleting a path exposed
  via .pre-commit-hooks.yaml since the last tag passed the gate clean —
  exactly the breakage the gate exists to catch. git diff reports
  deletions fine without an existence check; the filter is gone.
- `git diff ... 2>/dev/null || true` turned any git failure (a shallow
  clone missing the tag's objects, a corrupted ref) into an empty,
  falsely-clean diff. The diff result is no longer swallowed: a failure
  now hard-fails with the underlying git error visible.
- RELEASE_PATHS was a hand-maintained array duplicating
  .pre-commit-hooks.yaml's entry: paths with only a comment holding them
  in sync, and was already over-broad (it swept in validate.sh /
  validate-provenance.sh, which no hook entry references). It's now
  parsed straight from .pre-commit-hooks.yaml's entry: lines at
  runtime, so it can't drift from the manifest and only tracks what a
  hook actually exposes.
- `git describe --tags --abbrev=0` accepted any tag reachable from HEAD
  as the diff baseline, not just release tags. Added
  `--match 'v[0-9]*.[0-9]*.[0-9]*'` so an incidental checkpoint tag
  can't shift the baseline and mask a real release-relevant change.

check-vale-style-sync.sh still only detects drift between skill-audit's
and agent-audit's duplicated vale-wrap.sh/styles/Kyberforge copies
(both copies must exist independently per the plugin's no-cross-skill-
path packaging rule — a symlink would break at install time). Added
scripts/sync-vale-styles.sh to regenerate skill-audit's copy from
agent-audit's canonical one on demand, and pointed the sync check's
failure message at it, so fixing drift is one command instead of a
hand diff across two files.

Also recorded, rather than silently left unfixed: check-release-needed.sh
only fires on a local `git push` through pre-commit's pre-push hook — a
PR merged via Gitea's merge button, or CI invoking
`pre-commit run --hook-stage pre-push` directly, never sets
PRE_COMMIT_REMOTE_BRANCH and skips the gate entirely. Closing that needs
a server-side CI job this repo doesn't have yet; documented as a known
limitation in ADR-0014 rather than papered over.

Separately, LESSONS.md's "a clean check can mean nothing ran" entry was
marked **Graduated** without ever being promoted per the repo's own
graduation rule (3+ instances → a standing doc, marked
`[graduated → target file]`). Actually promoted it into
core/instructions/testing.md and fixed the marker.

tests/test-check-release-needed.sh gained 4 regression tests, one per
check-release-needed.sh fix above, each verified to fail against the
pre-fix script and pass against the current one.

Verification: bash tests/run-tests.sh (11 scripts + 125 bats, all
passing), pre-commit run --all-files, and
pre-commit run --all-files --hook-stage pre-push all clean.

ADR: 0014
2026-08-09 11:20:55 +00:00

172 lines
7.1 KiB
Bash
Executable File

#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$REPO_ROOT/scripts/check-release-needed.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
# Helper: write a minimal .pre-commit-hooks.yaml exposing one script entry and
# one --config-bearing entry, so RELEASE_PATHS (derived from the manifest, not
# hand-maintained) has both shapes to parse.
write_manifest() {
local dir="$1"
mkdir -p "$dir/vale"
cat > "$dir/.pre-commit-hooks.yaml" <<'EOF'
- id: fake-size-check
entry: scripts/skill-size-check.sh
language: script
- id: fake-vale-check
entry: scripts/vale-wrap.sh --config vale/.vale.ini
language: script
EOF
}
# Helper: a fixture repo with a manifest and its two referenced release-relevant
# paths, committed and tagged v1.0.0.
make_tagged_fixture() {
local dir
dir="$(mktemp -d)"
(cd "$dir" && git init -q && git config user.email t@t.t && git config user.name t)
write_manifest "$dir"
mkdir -p "$dir/scripts"
echo "v1" > "$dir/scripts/skill-size-check.sh"
echo "cfg" > "$dir/vale/.vale.ini"
(cd "$dir" && git add -A && git commit -q -m "initial" && git tag v1.0.0)
echo "$dir"
}
run_check() {
local dir="$1" branch="$2"
(cd "$dir" && PRE_COMMIT_REMOTE_BRANCH="$branch" bash "$SCRIPT" 2>&1)
}
CLEANUP_DIRS=()
trap 'rm -rf "${CLEANUP_DIRS[@]}"' EXIT
track() { CLEANUP_DIRS+=("$1"); }
# --- 1. Not targeting main: silent no-op regardless of state ---
echo ""
echo "--- exits 0 when not pushing to main, even with no tags ---"
FIXTURE1="$(mktemp -d)"; track "$FIXTURE1"
(cd "$FIXTURE1" && git init -q)
if run_check "$FIXTURE1" "refs/heads/feature-branch" > /dev/null; then
pass "exits 0 when target branch isn't main"
else
fail "exited non-zero on a non-main target branch"
fi
# --- 2. Targeting main, no tag exists at all: hard fail ---
echo ""
echo "--- exits 1 when targeting main and no tag exists ---"
FIXTURE2="$(mktemp -d)"; track "$FIXTURE2"
(cd "$FIXTURE2" && git init -q && git config user.email t@t.t && git config user.name t)
write_manifest "$FIXTURE2"
mkdir -p "$FIXTURE2/scripts"
echo "v1" > "$FIXTURE2/scripts/skill-size-check.sh"
echo "cfg" > "$FIXTURE2/vale/.vale.ini"
(cd "$FIXTURE2" && git add -A && git commit -q -m "initial")
if run_check "$FIXTURE2" "refs/heads/main" > /dev/null; then
fail "exited 0 when targeting main with no tag — expected exit 1"
else
pass "exits non-zero when targeting main and no tag exists yet"
fi
# --- 3. Targeting main, tag exists, no release-relevant changes since: passes ---
echo ""
echo "--- exits 0 when targeting main and nothing release-relevant changed since the tag ---"
FIXTURE3="$(make_tagged_fixture)"; track "$FIXTURE3"
echo "unrelated" > "$FIXTURE3/README.md"
(cd "$FIXTURE3" && git add -A && git commit -q -m "unrelated change")
if run_check "$FIXTURE3" "refs/heads/main" > /dev/null; then
pass "exits 0 when only unrelated files changed since the tag"
else
fail "exited non-zero despite no release-relevant changes since the tag"
fi
# --- 4. Targeting main, tag exists, a release-relevant file changed since: hard fail ---
echo ""
echo "--- exits 1 when a release-relevant file changed since the tag ---"
FIXTURE4="$(make_tagged_fixture)"; track "$FIXTURE4"
echo "v2" > "$FIXTURE4/scripts/skill-size-check.sh"
(cd "$FIXTURE4" && git add -A && git commit -q -m "update release-relevant script")
OUT4=$(run_check "$FIXTURE4" "refs/heads/main" || true)
if echo "$OUT4" | grep -q "skill-size-check.sh"; then
pass "exits non-zero and names the changed file when a release-relevant path changed since the tag"
else
fail "did not flag the release-relevant file that changed since the tag"
fi
# --- 5. Not targeting main even with release-relevant changes and a tag: still a no-op ---
echo ""
echo "--- exits 0 on a feature branch even with release-relevant changes since the tag ---"
FIXTURE5="$(make_tagged_fixture)"; track "$FIXTURE5"
echo "v2" > "$FIXTURE5/scripts/skill-size-check.sh"
(cd "$FIXTURE5" && git add -A && git commit -q -m "update release-relevant script")
if run_check "$FIXTURE5" "refs/heads/some-feature" > /dev/null; then
pass "exits 0 on a feature branch regardless of un-tagged release-relevant changes"
else
fail "hard-failed on a feature branch — should only ever fail when targeting main"
fi
# --- 6. A release-relevant path deleted since the tag is still flagged ---
echo ""
echo "--- exits 1 when a release-relevant path was deleted since the tag, not just modified ---"
FIXTURE6="$(make_tagged_fixture)"; track "$FIXTURE6"
rm -rf "$FIXTURE6/vale"
(cd "$FIXTURE6" && git add -A && git commit -q -m "delete the vale config dir")
OUT6=$(run_check "$FIXTURE6" "refs/heads/main" || true)
if echo "$OUT6" | grep -q "vale/.vale.ini"; then
pass "flags a deleted release-relevant path instead of silently dropping it from the diff"
else
fail "did not flag deletion of a release-relevant path since the tag"
fi
# --- 7. A git diff failure hard-fails instead of reading as a clean pass ---
echo ""
echo "--- exits 1 (not a silent pass) when the underlying git diff errors out ---"
FIXTURE7="$(make_tagged_fixture)"; track "$FIXTURE7"
TAG_TREE="$(cd "$FIXTURE7" && git rev-parse 'v1.0.0^{tree}')"
echo "v2" > "$FIXTURE7/scripts/skill-size-check.sh"
(cd "$FIXTURE7" && git add -A && git commit -q -m "advance past the tag")
rm -f "$FIXTURE7/.git/objects/${TAG_TREE:0:2}/${TAG_TREE:2}"
if run_check "$FIXTURE7" "refs/heads/main" > /dev/null; then
fail "silently exited 0 when the underlying git diff failed"
else
pass "hard-fails instead of silently passing when git diff can't be computed"
fi
# --- 8. A non-version tag reachable from HEAD does not become the diff baseline ---
echo ""
echo "--- ignores a non-vX.Y.Z tag and still flags a change since the real release tag ---"
FIXTURE8="$(make_tagged_fixture)"; track "$FIXTURE8"
echo "checkpoint" > "$FIXTURE8/scripts/skill-size-check.sh"
(cd "$FIXTURE8" && git add -A && git commit -q -m "checkpoint work" && git tag checkpoint-1)
echo "v2" > "$FIXTURE8/scripts/skill-size-check.sh"
(cd "$FIXTURE8" && git add -A && git commit -q -m "real release-relevant change")
OUT8=$(run_check "$FIXTURE8" "refs/heads/main" || true)
if echo "$OUT8" | grep -q "skill-size-check.sh"; then
pass "still flags the release-relevant change since v1.0.0, ignoring the non-version checkpoint tag"
else
fail "an incidental non-version tag shifted the baseline and hid a real release-relevant change"
fi
# --- 9. A file outside every manifest entry does not trigger a fail ---
echo ""
echo "--- exits 0 when a changed file sits near, but isn't referenced by, a manifest entry ---"
FIXTURE9="$(make_tagged_fixture)"; track "$FIXTURE9"
echo "irrelevant" > "$FIXTURE9/scripts/unrelated-helper.sh"
(cd "$FIXTURE9" && git add -A && git commit -q -m "add an unrelated script alongside the exposed one")
if run_check "$FIXTURE9" "refs/heads/main" > /dev/null; then
pass "exits 0 for a file that lives alongside, but isn't referenced by, any manifest entry"
else
fail "flagged a file that no .pre-commit-hooks.yaml entry actually exposes"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]