LESSONS.md

@@ -140,7 +140,7 @@ During write-skill refactor, an "open thread" note (about a deferred research st
 
 ## 2026-08-08 — A clean linter result can mean "nothing was checked"
 
-Three separate times in one PR (#85), a check reported success because it had silently not run. (1) Vale's `text.frontmatter.description` scope stops matching once the value is a multi-line YAML block scalar — the style most skills here use — so a repo-wide sweep returned 0 alerts across 49 files and was read as a clean repo. (2) Five of six rules were `level: warning`, but Vale's exit code keys on `error` alone and pre-commit hides output from passing hooks, so those rules were invisible and blocked nothing for two review rounds while the ADR described them as "enforcing immediately." (3) `.vale.ini`'s globs matched no file outside `plugins/`, so Vale printed "0 files" and exited 0, which both audit skills read as "no findings" and used to skip their own judgment passes. Each time the green result was worse than no check at all, because it was cited as positive evidence of cleanliness. Fix: for any new check, prove it fails before trusting that it passes — run it against a deliberately-bad fixture, confirm the failure, then run the real corpus. Where a check can scan zero inputs, assert on the input count, not just the exit code. **Graduated** (4th instance below).
+Three separate times in one PR (#85), a check reported success because it had silently not run. (1) Vale's `text.frontmatter.description` scope stops matching once the value is a multi-line YAML block scalar — the style most skills here use — so a repo-wide sweep returned 0 alerts across 49 files and was read as a clean repo. (2) Five of six rules were `level: warning`, but Vale's exit code keys on `error` alone and pre-commit hides output from passing hooks, so those rules were invisible and blocked nothing for two review rounds while the ADR described them as "enforcing immediately." (3) `.vale.ini`'s globs matched no file outside `plugins/`, so Vale printed "0 files" and exited 0, which both audit skills read as "no findings" and used to skip their own judgment passes. Each time the green result was worse than no check at all, because it was cited as positive evidence of cleanliness. Fix: for any new check, prove it fails before trusting that it passes — run it against a deliberately-bad fixture, confirm the failure, then run the real corpus. Where a check can scan zero inputs, assert on the input count, not just the exit code. **[graduated → core/instructions/testing.md]** (4th instance below, kept for audit trail).
 
 **4th instance (2026-08-09, ADR-0014):** splitting the single root `.vale.ini` into two skill-scoped copies (skill-audit: `SKILL.md` only; agent-audit: agent files only) meant a single retargeted pre-commit hook pointed at agent-audit's copy alone would have silently scanned 0 `SKILL.md` files and exited 0 — caught only because the full corpus was dry-run against both the old and new config and the outputs diffed before the old config was deleted, not because any test asserted on file counts. Standing practice going forward: when a Vale (or any linter) config that serves multiple file-glob scopes is split or moved, dry-run the full corpus through both the old and new config and diff the outputs before removing the superseded source — a hook silently scanning 0 files looks identical to a clean pass.
 

core/instructions/testing.md

@@ -4,3 +4,4 @@
 - Automate everything automatable. Manual testing only for nuanced UI/UX or agent interaction behaviour requiring human judgment.
 - Test observable end-state, not implementation internals. Tests must survive refactoring.
 - No test is better than a wrong test. A passing mock that masks a real failure is actively harmful.
+- A clean result can mean nothing ran. Before trusting a new check, prove it fails against a deliberately-bad fixture, then run it against the real target. Where a check can scan zero inputs, assert on the input count, not just the exit code — a zero-file run and a real clean pass look identical otherwise.

docs/adr/0014-vale-prefilter-ships-from-the-plugin.md

@@ -107,3 +107,13 @@ doesn't wonder if it was overlooked.
   `main`, covering the very first release. This is deterministic tooling, not a standing
   instruction to remember — consistent with `check-manifests.sh`/`check-vale-style-sync.sh`
   already using the same pre-push, main-agnostic-elsewhere pattern.
+- **Known limitation, not yet closed:** `check-release-needed.sh` only fires when a human runs
+  `git push` locally with pre-commit's hooks installed — `PRE_COMMIT_REMOTE_BRANCH` is set by
+  pre-commit's client-side `hook-impl` script parsing `git push`'s stdin protocol. A PR merged
+  through Gitea's merge button (server-side, no local push) or a CI runner invoking
+  `pre-commit run --hook-stage pre-push` directly never sets it, so the gate silently doesn't run
+  in either path. This repo has no CI workflow yet (`has_actions` is enabled but unused), so
+  closing this gap needs a server-side job re-running the same script on merge to `main` — deferred
+  as a separate piece of infrastructure, not fixed here. `RELEASE_PATHS` is derived from
+  `.pre-commit-hooks.yaml`'s own `entry:` lines rather than hand-maintained, so at least the set of
+  paths it checks can't drift from the manifest on its own.

scripts/check-release-needed.sh

@@ -8,6 +8,12 @@ set -euo pipefail
 # for pre-push hooks; on every other branch (feature work mid-review) this is a
 # silent no-op — pushing WIP commits there must not be blocked on cutting a
 # premature tag (see ADR-0014's repo: local vs pinned self-reference decision).
+#
+# Known gap: this only fires on a local `git push` through pre-commit's pre-push
+# hook. A PR merged via Gitea's merge button (server-side, no local push) or a
+# CI runner invoking `pre-commit run --hook-stage pre-push` directly does not set
+# PRE_COMMIT_REMOTE_BRANCH and will not trigger this check — closing that
+# requires a server-side CI job, which this repo does not have yet.
 
 TARGET_BRANCH="refs/heads/main"
 
@@ -18,18 +24,34 @@ fi
 REPO_ROOT="$(git rev-parse --show-toplevel)"
 cd "$REPO_ROOT"
 
-# Paths whose content .pre-commit-hooks.yaml exposes to external consumers.
-# Keep in sync with .pre-commit-hooks.yaml's entry: paths.
-RELEASE_PATHS=(
-  .pre-commit-hooks.yaml
-  scripts/skill-size-check.sh
-  plugins/kyberforge/skills/skill-audit/scripts
-  plugins/kyberforge/skills/skill-audit/assets/vale
-  plugins/kyberforge/skills/agent-audit/scripts
-  plugins/kyberforge/skills/agent-audit/assets/vale
-)
+HOOKS_MANIFEST=".pre-commit-hooks.yaml"
 
-LAST_TAG="$(git describe --tags --abbrev=0 2>/dev/null || true)"
+if [[ ! -f "$HOOKS_MANIFEST" ]]; then
+  exit 0
+fi
+
+# Derive release-relevant paths from .pre-commit-hooks.yaml's own entry: lines
+# instead of hand-maintaining a parallel list — the manifest is the single
+# source of truth for what external consumers actually pull at a pinned rev,
+# so a hook added/removed/renamed there can't silently drift out of sync here.
+# Each entry is "<script> [--config <path>] [...]"; the script itself and the
+# directory holding any --config target (vale-wrap.sh needs its .vale.ini's
+# sibling styles/ tree, not just the ini file) are release-relevant.
+RELEASE_PATHS=("$HOOKS_MANIFEST")
+while IFS= read -r entry; do
+  read -ra tokens <<< "$entry"
+  [[ ${#tokens[@]} -eq 0 ]] && continue
+  RELEASE_PATHS+=("${tokens[0]}")
+  for ((i = 1; i < ${#tokens[@]}; i++)); do
+    if [[ "${tokens[i]}" == "--config" && -n "${tokens[i + 1]:-}" ]]; then
+      RELEASE_PATHS+=("$(dirname "${tokens[i + 1]}")")
+    fi
+  done
+done < <(sed -n 's/^[[:space:]]*entry:[[:space:]]*//p' "$HOOKS_MANIFEST")
+
+# Only vX.Y.Z release tags count as a baseline — an incidental checkpoint or
+# experiment tag reachable from HEAD must not shift the diff baseline.
+LAST_TAG="$(git describe --tags --abbrev=0 --match 'v[0-9]*.[0-9]*.[0-9]*' 2>/dev/null || true)"
 
 if [[ -z "$LAST_TAG" ]]; then
   echo "FAIL: no release tag exists yet, but .pre-commit-hooks.yaml already exposes hooks to external consumers." >&2
@@ -37,17 +59,17 @@ if [[ -z "$LAST_TAG" ]]; then
   exit 1
 fi
 
-EXISTING_PATHS=()
-for p in "${RELEASE_PATHS[@]}"; do
-  [[ -e "$p" ]] && EXISTING_PATHS+=("$p")
-done
-
-if [[ ${#EXISTING_PATHS[@]} -eq 0 ]]; then
-  exit 0
+# No -e/existence filtering: a path deleted since $LAST_TAG is exactly the case
+# that must be caught (external consumers pinning the old tag would hit a
+# missing file), and `git diff` reports deletions fine without it existing at
+# HEAD. A git failure (e.g. a shallow clone missing $LAST_TAG's history) must
+# fail closed, not be swallowed into an empty, falsely-clean diff.
+if ! CHANGED="$(git diff --name-only "$LAST_TAG"..HEAD -- "${RELEASE_PATHS[@]}")"; then
+  echo "FAIL: could not diff $LAST_TAG..HEAD to check for release-relevant changes (see git error above)." >&2
+  echo "  Fix: ensure full tag history is available (e.g. git fetch --unshallow) and retry." >&2
+  exit 1
 fi
 
-CHANGED="$(git diff --name-only "$LAST_TAG"..HEAD -- "${EXISTING_PATHS[@]}" 2>/dev/null || true)"
-
 if [[ -n "$CHANGED" ]]; then
   echo "FAIL: files covered by .pre-commit-hooks.yaml changed since $LAST_TAG:" >&2
   echo "$CHANGED" | sed 's/^/    /' >&2

scripts/check-vale-style-sync.sh

@@ -30,6 +30,6 @@ if ! diff -rq "$SKILL_AUDIT/assets/vale/styles/Kyberforge" "$AGENT_AUDIT/assets/
 fi
 
 if [[ $FAIL -gt 0 ]]; then
-  echo "Vale style sync check failed: $FAIL error(s). agent-audit's copy is canonical — sync skill-audit's copy to match." >&2
+  echo "Vale style sync check failed: $FAIL error(s). agent-audit's copy is canonical — run scripts/sync-vale-styles.sh to regenerate skill-audit's copy, then commit both." >&2
   exit 1
 fi

scripts/sync-vale-styles.sh

@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Regenerates skill-audit's Vale copy from agent-audit's canonical copy (see
+# scripts/check-vale-style-sync.sh / ADR-0014). Both copies must exist on disk
+# independently — a plugin's cache-install only copies each skill's own files,
+# so a symlink or shared path would break at install time — but that doesn't
+# mean the copy step has to be manual. Run this after editing agent-audit's
+# vale-wrap.sh or styles/Kyberforge, review the diff, then commit both trees
+# together.
+
+REPO_ROOT="${1:-$(git rev-parse --show-toplevel)}"
+SKILL_AUDIT="$REPO_ROOT/plugins/kyberforge/skills/skill-audit"
+AGENT_AUDIT="$REPO_ROOT/plugins/kyberforge/skills/agent-audit"
+
+cp "$AGENT_AUDIT/scripts/vale-wrap.sh" "$SKILL_AUDIT/scripts/vale-wrap.sh"
+rm -rf "$SKILL_AUDIT/assets/vale/styles/Kyberforge"
+cp -r "$AGENT_AUDIT/assets/vale/styles/Kyberforge" "$SKILL_AUDIT/assets/vale/styles/Kyberforge"
+
+echo "Synced skill-audit's vale-wrap.sh and styles/Kyberforge from agent-audit's canonical copy."
+echo "Review the diff, then commit both directories together."

tests/test-check-release-needed.sh

@@ -9,13 +9,32 @@ FAIL=0
 pass() { echo "  PASS: $1"; PASS=$((PASS + 1)); }
 fail() { echo "  FAIL: $1"; FAIL=$((FAIL + 1)); }
 
-# Helper: a fixture repo with one release-relevant file, committed and tagged.
+# Helper: write a minimal .pre-commit-hooks.yaml exposing one script entry and
+# one --config-bearing entry, so RELEASE_PATHS (derived from the manifest, not
+# hand-maintained) has both shapes to parse.
+write_manifest() {
+  local dir="$1"
+  mkdir -p "$dir/vale"
+  cat > "$dir/.pre-commit-hooks.yaml" <<'EOF'
+-   id: fake-size-check
+    entry: scripts/skill-size-check.sh
+    language: script
+-   id: fake-vale-check
+    entry: scripts/vale-wrap.sh --config vale/.vale.ini
+    language: script
+EOF
+}
+
+# Helper: a fixture repo with a manifest and its two referenced release-relevant
+# paths, committed and tagged v1.0.0.
 make_tagged_fixture() {
   local dir
   dir="$(mktemp -d)"
   (cd "$dir" && git init -q && git config user.email t@t.t && git config user.name t)
+  write_manifest "$dir"
   mkdir -p "$dir/scripts"
   echo "v1" > "$dir/scripts/skill-size-check.sh"
+  echo "cfg" > "$dir/vale/.vale.ini"
   (cd "$dir" && git add -A && git commit -q -m "initial" && git tag v1.0.0)
   echo "$dir"
 }
@@ -25,11 +44,14 @@ run_check() {
   (cd "$dir" && PRE_COMMIT_REMOTE_BRANCH="$branch" bash "$SCRIPT" 2>&1)
 }
 
+CLEANUP_DIRS=()
+trap 'rm -rf "${CLEANUP_DIRS[@]}"' EXIT
+track() { CLEANUP_DIRS+=("$1"); }
+
 # --- 1. Not targeting main: silent no-op regardless of state ---
 echo ""
 echo "--- exits 0 when not pushing to main, even with no tags ---"
-FIXTURE1="$(mktemp -d)"
-trap 'rm -rf "$FIXTURE1"' EXIT
+FIXTURE1="$(mktemp -d)"; track "$FIXTURE1"
 (cd "$FIXTURE1" && git init -q)
 if run_check "$FIXTURE1" "refs/heads/feature-branch" > /dev/null; then
   pass "exits 0 when target branch isn't main"
@@ -40,11 +62,12 @@ fi
 # --- 2. Targeting main, no tag exists at all: hard fail ---
 echo ""
 echo "--- exits 1 when targeting main and no tag exists ---"
-FIXTURE2="$(mktemp -d)"
-trap 'rm -rf "$FIXTURE1" "$FIXTURE2"' EXIT
+FIXTURE2="$(mktemp -d)"; track "$FIXTURE2"
 (cd "$FIXTURE2" && git init -q && git config user.email t@t.t && git config user.name t)
+write_manifest "$FIXTURE2"
 mkdir -p "$FIXTURE2/scripts"
 echo "v1" > "$FIXTURE2/scripts/skill-size-check.sh"
+echo "cfg" > "$FIXTURE2/vale/.vale.ini"
 (cd "$FIXTURE2" && git add -A && git commit -q -m "initial")
 if run_check "$FIXTURE2" "refs/heads/main" > /dev/null; then
   fail "exited 0 when targeting main with no tag — expected exit 1"
@@ -55,8 +78,7 @@ fi
 # --- 3. Targeting main, tag exists, no release-relevant changes since: passes ---
 echo ""
 echo "--- exits 0 when targeting main and nothing release-relevant changed since the tag ---"
-FIXTURE3="$(make_tagged_fixture)"
-trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3"' EXIT
+FIXTURE3="$(make_tagged_fixture)"; track "$FIXTURE3"
 echo "unrelated" > "$FIXTURE3/README.md"
 (cd "$FIXTURE3" && git add -A && git commit -q -m "unrelated change")
 if run_check "$FIXTURE3" "refs/heads/main" > /dev/null; then
@@ -68,8 +90,7 @@ fi
 # --- 4. Targeting main, tag exists, a release-relevant file changed since: hard fail ---
 echo ""
 echo "--- exits 1 when a release-relevant file changed since the tag ---"
-FIXTURE4="$(make_tagged_fixture)"
-trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4"' EXIT
+FIXTURE4="$(make_tagged_fixture)"; track "$FIXTURE4"
 echo "v2" > "$FIXTURE4/scripts/skill-size-check.sh"
 (cd "$FIXTURE4" && git add -A && git commit -q -m "update release-relevant script")
 OUT4=$(run_check "$FIXTURE4" "refs/heads/main" || true)
@@ -82,8 +103,7 @@ fi
 # --- 5. Not targeting main even with release-relevant changes and a tag: still a no-op ---
 echo ""
 echo "--- exits 0 on a feature branch even with release-relevant changes since the tag ---"
-FIXTURE5="$(make_tagged_fixture)"
-trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5"' EXIT
+FIXTURE5="$(make_tagged_fixture)"; track "$FIXTURE5"
 echo "v2" > "$FIXTURE5/scripts/skill-size-check.sh"
 (cd "$FIXTURE5" && git add -A && git commit -q -m "update release-relevant script")
 if run_check "$FIXTURE5" "refs/heads/some-feature" > /dev/null; then
@@ -92,6 +112,60 @@ else
   fail "hard-failed on a feature branch — should only ever fail when targeting main"
 fi
 
+# --- 6. A release-relevant path deleted since the tag is still flagged ---
+echo ""
+echo "--- exits 1 when a release-relevant path was deleted since the tag, not just modified ---"
+FIXTURE6="$(make_tagged_fixture)"; track "$FIXTURE6"
+rm -rf "$FIXTURE6/vale"
+(cd "$FIXTURE6" && git add -A && git commit -q -m "delete the vale config dir")
+OUT6=$(run_check "$FIXTURE6" "refs/heads/main" || true)
+if echo "$OUT6" | grep -q "vale/.vale.ini"; then
+  pass "flags a deleted release-relevant path instead of silently dropping it from the diff"
+else
+  fail "did not flag deletion of a release-relevant path since the tag"
+fi
+
+# --- 7. A git diff failure hard-fails instead of reading as a clean pass ---
+echo ""
+echo "--- exits 1 (not a silent pass) when the underlying git diff errors out ---"
+FIXTURE7="$(make_tagged_fixture)"; track "$FIXTURE7"
+TAG_TREE="$(cd "$FIXTURE7" && git rev-parse 'v1.0.0^{tree}')"
+echo "v2" > "$FIXTURE7/scripts/skill-size-check.sh"
+(cd "$FIXTURE7" && git add -A && git commit -q -m "advance past the tag")
+rm -f "$FIXTURE7/.git/objects/${TAG_TREE:0:2}/${TAG_TREE:2}"
+if run_check "$FIXTURE7" "refs/heads/main" > /dev/null; then
+  fail "silently exited 0 when the underlying git diff failed"
+else
+  pass "hard-fails instead of silently passing when git diff can't be computed"
+fi
+
+# --- 8. A non-version tag reachable from HEAD does not become the diff baseline ---
+echo ""
+echo "--- ignores a non-vX.Y.Z tag and still flags a change since the real release tag ---"
+FIXTURE8="$(make_tagged_fixture)"; track "$FIXTURE8"
+echo "checkpoint" > "$FIXTURE8/scripts/skill-size-check.sh"
+(cd "$FIXTURE8" && git add -A && git commit -q -m "checkpoint work" && git tag checkpoint-1)
+echo "v2" > "$FIXTURE8/scripts/skill-size-check.sh"
+(cd "$FIXTURE8" && git add -A && git commit -q -m "real release-relevant change")
+OUT8=$(run_check "$FIXTURE8" "refs/heads/main" || true)
+if echo "$OUT8" | grep -q "skill-size-check.sh"; then
+  pass "still flags the release-relevant change since v1.0.0, ignoring the non-version checkpoint tag"
+else
+  fail "an incidental non-version tag shifted the baseline and hid a real release-relevant change"
+fi
+
+# --- 9. A file outside every manifest entry does not trigger a fail ---
+echo ""
+echo "--- exits 0 when a changed file sits near, but isn't referenced by, a manifest entry ---"
+FIXTURE9="$(make_tagged_fixture)"; track "$FIXTURE9"
+echo "irrelevant" > "$FIXTURE9/scripts/unrelated-helper.sh"
+(cd "$FIXTURE9" && git add -A && git commit -q -m "add an unrelated script alongside the exposed one")
+if run_check "$FIXTURE9" "refs/heads/main" > /dev/null; then
+  pass "exits 0 for a file that lives alongside, but isn't referenced by, any manifest entry"
+else
+  fail "flagged a file that no .pre-commit-hooks.yaml entry actually exposes"
+fi
+
 echo ""
 echo "Results: $PASS passed, $FAIL failed"
 [[ $FAIL -eq 0 ]]