Files
holocron/tests/test-adr0020-targets.sh
Defame1297 23b843a311 test: drop the gitea-labels pin now the retrofit removed that target
tests/test-adr0020-targets.sh pinned the corpus dangling set as exactly
{gitea-labels, neuledge-context} and tests/test-skill-size-check.sh probed both
individually. The gitea-issues retrofit cut the composition sentence whose
'>'-folded scalar produced 'gitea-labels- milestones', so that target no longer
exists and both suites went red.

EXPECTED_DANGLING is now {neuledge-context} and the gitea-issues probe is
removed rather than skipped, per the rule the probe file states about itself: a
probe whose fixture has been retrofitted is a pin that needs updating, not an
assertion-free pass counted in the totals.

The exact-set assertion stays. An empty expected set is still valid and still
pins that no new dangling target appeared -- which is what it becomes once
research is retrofitted. Both loops carry a shellcheck SC2043 waiver for the
same reason: one entry is the expected steady state, not bad quoting.

Refs #99
2026-08-30 12:34:24 +00:00

614 lines
33 KiB
Bash
Executable File

#!/usr/bin/env bash
# Regression test for the two properties of ADR-0020 boundary-target resolution
# that decide whether the gate can be trusted at all.
#
# 1. MACHINE INDEPENDENCE. The resolution universe is derived by walking up
# FROM THE TARGET FILE to an authoring root, and when one is found the
# deployed .claude/ and .agents/ trees are deliberately NOT consulted. Those
# trees are `apm install` output — gitignored, and present only on a machine
# that has run it. Four cross-plugin targets in this repo resolved through
# .claude/skills/ alone, so the same commit measured 2 dangling targets on a
# developer machine and 6 on a fresh clone. A gate shipping hot with no
# baseline cannot give two answers, so this file asserts the verdict is
# identical with and without a deployed tree — on a synthetic fixture AND on
# the real 39-skill corpus.
#
# Three further ways the universe can be built out of the wrong directory,
# each of which shipped: a `.git` at the CONSUMER root (the fallback is
# truthy in any git repo, which made the deployed-tree branch dead code), a
# `.git` INSIDE a plugin (the walk-up is two passes precisely so this cannot
# capture the root), and glob metacharacters in the checkout path (which
# turned the directory name into a character class matching nothing, and the
# resolver into a no-op that still reported green).
#
# 2. THE BARE-TARGET GRAMMAR RULE. A hyphenated token used as a compound
# MODIFIER ("pre-commit hooks", "pull-request template") is prose, not a
# route; a terminal one is a real target. Getting this wrong in either
# direction is fatal: firing on prose makes the gate untrustworthy and it
# gets turned off, while suppressing too much deletes the only two true
# positives the corpus has. Both live true positives are BARE, which is why
# the rule keys on the FOLLOWER TOKEN rather than on marking, and why they
# are pinned by name below — a future false-positive fix must not be able to
# quietly take them with it.
#
# 3. IN-SENTENCE CORROBORATION. Terminal position alone is not evidence of a
# route: "run `pre-commit` instead", "see `commit-msg`", "use the clean-up
# instead" and "run unit-tests" are all terminal, all prose, and all were
# hard FAILs with no suppression mechanism anywhere in the gate. A
# prose-form target therefore blocks only when its own sentence names
# another target that RESOLVES; otherwise it is reported at SUGGESTION tier
# and the commit proceeds. Route NOTATION (`/name`, `-> name`) is exempt
# and always blocks. Both halves are asserted below: the prose class must
# report-not-block, and the notation and corroborated forms must still
# ERROR, or the fix would have eaten the gate rather than narrowed it.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
HOOK="$REPO_ROOT/scripts/skill-size-check.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
TMPDIR_T="$(mktemp -d)"
trap 'rm -rf "$TMPDIR_T"' EXIT
# write_skill <skill-dir> <name> <desc>
write_skill() {
mkdir -p "$1"
{
echo "---"
echo "name: $2"
echo "description: $3"
echo "---"
echo ""
echo "Do the thing."
} > "$1/SKILL.md"
}
# ---------------------------------------------------------------------------
# 1. Machine independence — synthetic fixture
# ---------------------------------------------------------------------------
# Two trees, identical except that one also carries a deployed .claude/ tree
# holding a skill and an agent that exist NOWHERE in plugins/. The subject routes
# to one name that lives in a sibling plugin (must resolve in both) and one that
# lives only in .claude/ (must DANGLE in both — an authoring root exists, so the
# deployed tree is not part of the universe).
#
# If the deployed tree were consulted, the second target would resolve on the
# machine that has run `apm install` and dangle on a fresh clone. That is the
# 2-vs-6 defect exactly, at fixture scale.
echo ""
echo "--- the same file gets the same verdict with and without a deployed .claude/ tree ---"
build_tree() {
local root="$1"
write_skill "$root/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$root/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use cross-plugin-skill or deployed-only-skill instead."
}
build_tree "$TMPDIR_T/no-claude"
build_tree "$TMPDIR_T/with-claude"
# The deployed tree, present only in the second root. Both a skill and an agent,
# because both are valid routing targets and both would leak.
mkdir -p "$TMPDIR_T/with-claude/.claude/skills/deployed-only-skill" \
"$TMPDIR_T/with-claude/.claude/agents"
: > "$TMPDIR_T/with-claude/.claude/agents/deployed-only-agent.md"
run_subject() {
local root="$1" out
set +e
out="$(bash "$HOOK" "$root/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
set -e
# Normalise the tree root out of the paths so the two runs are comparable.
printf '%s\n' "$out" | sed "s#$root#<ROOT>#g"
}
NO_CLAUDE_OUT="$(run_subject "$TMPDIR_T/no-claude")"
WITH_CLAUDE_OUT="$(run_subject "$TMPDIR_T/with-claude")"
if [[ "$NO_CLAUDE_OUT" == "$WITH_CLAUDE_OUT" ]]; then
pass "identical output with and without a deployed .claude/ tree"
else
fail "the deployed .claude/ tree changed the verdict — without: [$NO_CLAUDE_OUT] with: [$WITH_CLAUDE_OUT]"
fi
# Identical-but-wrong is still possible (both could resolve everything, or
# neither could resolve anything), so the CONTENT is asserted too: the
# sibling-plugin name must resolve and the deployed-only name must not.
if [[ "$WITH_CLAUDE_OUT" == *"routes to 'deployed-only-skill'"* ]]; then
pass "a name that exists only in .claude/ still dangles when an authoring root is present"
else
fail "the deployed-only target did not dangle — the deployed tree is being read into the universe: $WITH_CLAUDE_OUT"
fi
if [[ "$WITH_CLAUDE_OUT" != *"routes to 'cross-plugin-skill'"* ]]; then
pass "a name in a SIBLING PLUGIN resolves, so the comparison above is not 'nothing resolves'"
else
fail "the sibling-plugin target dangled — the monorepo universe is not being built: $WITH_CLAUDE_OUT"
fi
# The other half of the rule: with NO authoring root, deployed trees ARE the
# universe. That is the consumer case, and without this the rule above could be
# implemented as "never read .claude/", which would leave consumers with no
# resolution at all.
echo ""
echo "--- with no authoring root, a deployed .claude/ tree IS the universe ---"
CONSUMER="$TMPDIR_T/consumer"
mkdir -p "$CONSUMER/.claude/skills/deployed-only-skill"
write_skill "$CONSUMER/.claude/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use deployed-only-skill instead."
set +e
CONSUMER_OUT="$(bash "$HOOK" "$CONSUMER/.claude/skills/my-skill/SKILL.md" 2>&1)"
CONSUMER_RC=$?
set -e
if [[ $CONSUMER_RC -eq 0 && "$CONSUMER_OUT" != *"routes to"* && "$CONSUMER_OUT" != *"DID NOT RUN"* ]]; then
pass "a sibling in a deployed .claude/skills/ tree resolves when there is no authoring root"
else
fail "the consumer path did not resolve through the deployed tree (exit $CONSUMER_RC): ${CONSUMER_OUT:-<empty>}"
fi
# ---------------------------------------------------------------------------
# 1a-bis. The consumer case with the one thing every real consumer has: .git
# ---------------------------------------------------------------------------
# The fixture immediately above has no .git, and that is precisely why it could
# never catch this. _authoring_root() falls back to the nearest .git ancestor, so
# it returns truthy in ANY git repo — a consumer checkout included. The branch
# that reads the deployed trees was guarded by `else`, so in every consumer
# checkout the fallback won, _collect_authoring_root() contributed nothing
# (there is no plugins/ directory to collect), and _deployed_roots() was dead
# code in exactly the case it exists for.
#
# The pair below is the whole test: the SAME tree, once with .git and once
# without. Old behaviour was rc=1 with .git and rc=0 without; a test covering
# only the no-.git shape reports green on both.
#
# `deployed-only-agent` lives ONLY in .agents/agents/, so it can be reached
# through no route but _deployed_roots(). `sibling-skill` sits in .claude/skills/
# beside the subject, which the sibling-collection block above reaches on its own
# — it is the corroborator that makes the dangling target BLOCKING rather than a
# SUGGESTION, so the old failure shows up in the exit code and not only in prose.
echo ""
echo "--- a consumer checkout resolves through its deployed trees even though it is a git repo ---"
build_consumer() {
local root="$1"
mkdir -p "$root/.agents/agents"
write_skill "$root/.claude/skills/sibling-skill" sibling-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$root/.claude/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use sibling-skill or deployed-only-agent instead."
: > "$root/.agents/agents/deployed-only-agent.agent.md"
}
build_consumer "$TMPDIR_T/consumer-git"
mkdir -p "$TMPDIR_T/consumer-git/.git"
build_consumer "$TMPDIR_T/consumer-nogit"
# consumer_case <label> <root>
consumer_case() {
local label="$1" root="$2" out status=0
set +e
out="$(bash "$HOOK" "$root/.claude/skills/my-skill/SKILL.md" 2>&1)"
status=$?
set -e
if [[ $status -eq 0 && "$out" != *"routes to"* && "$out" != *"DID NOT RUN"* ]]; then
pass "$label"
else
fail "$label (exit $status, output: ${out:-<empty>})"
fi
}
consumer_case "an agent in .agents/agents/ resolves in a consumer checkout that HAS a .git directory" \
"$TMPDIR_T/consumer-git"
consumer_case "control: the same tree without .git resolves too (the shape that always passed)" \
"$TMPDIR_T/consumer-nogit"
# ---------------------------------------------------------------------------
# 1a-ter. A monorepo with ONE plugin is still a monorepo
# ---------------------------------------------------------------------------
# The first attempt at the fix above conditioned the deployed branch on whether
# the authoring root had CONTRIBUTED a name — `if len(names) == before:`. That
# reads as "the .git fallback collected nothing, so fall through", and it is
# wrong: _collect_authoring_root() re-collects the subject's OWN plugin, whose
# names the sibling and package blocks have already added. With two plugins
# (fixture 1) the cross-plugin name makes the delta non-zero and the guard stays
# shut. With ONE plugin the delta is zero, the guard fires in a genuine
# monorepo, and _deployed_roots() walks up to ten levels — reaching the user's
# global ~/.claude/skills. That is install-dependence again, in the shape
# ADR-0020 lines 118-127 exist to forbid.
#
# So the predicate is which PROBE matched, not how many names arrived. The
# assertion is the same shape as fixture 1 — identical verdict either way — but
# on a single-plugin tree, which fixture 1 cannot express.
echo ""
echo "--- a SINGLE-plugin monorepo does not fall through to the deployed trees ---"
build_single() {
local root="$1"
write_skill "$root/plugins/only-plugin/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use /deployed-only-skill instead."
}
build_single "$TMPDIR_T/single-no-claude"
build_single "$TMPDIR_T/single-with-claude"
write_skill "$TMPDIR_T/single-with-claude/.claude/skills/deployed-only-skill" deployed-only-skill \
"Use when doing the other thing. Do not use for anything else."
run_single() {
local root="$1" out
set +e
out="$(bash "$HOOK" "$root/plugins/only-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
set -e
printf '%s\n' "$out" | sed "s#$root#<ROOT>#g"
}
SINGLE_NO_OUT="$(run_single "$TMPDIR_T/single-no-claude")"
SINGLE_WITH_OUT="$(run_single "$TMPDIR_T/single-with-claude")"
if [[ "$SINGLE_NO_OUT" == "$SINGLE_WITH_OUT" ]]; then
pass "a single-plugin monorepo gets the same verdict with and without a deployed .claude/ tree"
else
fail "the deployed tree changed the verdict in a single-plugin monorepo — without: [$SINGLE_NO_OUT] with: [$SINGLE_WITH_OUT]"
fi
# Identical-but-wrong guard, as in fixture 1: the deployed-only name must DANGLE,
# not resolve. Written as `/deployed-only-skill` so it blocks on its own without
# needing a second target in the sentence to corroborate it.
if [[ "$SINGLE_WITH_OUT" == *"routes to 'deployed-only-skill'"* ]]; then
pass "the deployed-only target dangles in a single-plugin monorepo (~/.claude/skills is not in the universe)"
else
fail "the deployed-only target resolved — the single-plugin tree fell through to _deployed_roots(): $SINGLE_WITH_OUT"
fi
# ---------------------------------------------------------------------------
# 1c. A nested .git inside a plugin must not beat the monorepo root
# ---------------------------------------------------------------------------
# ADR-0020 records the walk-up as TWO passes — plugins/*/.apm/{skills,agents}
# first, .git only afterwards — specifically so a .git inside a plugin (a
# submodule, or a sub-package with its own worktree) cannot capture the root.
# Nothing anywhere placed a .git inside a plugin, so the second pass was
# structural claim only. Collapsing the two probes into one interleaved walk
# passes every other fixture in this repo and fails here.
echo ""
echo "--- a .git INSIDE a plugin does not shadow the monorepo root above it ---"
NESTED="$TMPDIR_T/nested-git"
write_skill "$NESTED/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
# The trap: a git checkout one level BELOW the monorepo root and above the skill.
mkdir -p "$NESTED/plugins/subject-plugin/.git"
set +e
NESTED_OUT="$(bash "$HOOK" "$NESTED/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
NESTED_RC=$?
set -e
# The sibling-plugin name is the discriminator: it is reachable ONLY from the
# monorepo root. If the nested .git won, subject-plugin would be the root, its
# plugins/ glob would collect nothing, and cross-plugin-skill would dangle —
# corroborated by sibling-skill in the same sentence, so it would BLOCK.
if [[ $NESTED_RC -eq 0 && "$NESTED_OUT" != *"routes to"* && "$NESTED_OUT" != *"DID NOT RUN"* ]]; then
pass "a sibling-plugin target still resolves with a .git directory inside the subject's own plugin"
else
fail "the nested .git captured the authoring root (exit $NESTED_RC): ${NESTED_OUT:-<empty>}"
fi
# ---------------------------------------------------------------------------
# 1d. Glob metacharacters in the checkout path
# ---------------------------------------------------------------------------
# The universe is built with glob.glob() against paths that begin with the
# checkout directory. A `[`, `]`, `*` or `?` anywhere in that prefix — a worktree
# named `feature[2]`, a CI workspace named `build[1]` — turned the literal
# directory name into a character class that matched nothing. The resolver then
# found no universe at all and degraded to the "DID NOT RUN" INFO with rc=0:
# every routing target in the tree silently unchecked, on a gate that reports
# green. Same monorepo as above, one directory renamed.
echo ""
echo "--- glob metacharacters in the checkout path do not silently disable the resolver ---"
GLOBDIR="$TMPDIR_T/gl[1]?x/mono"
write_skill "$GLOBDIR/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
"Use when doing the other thing. Do not use for anything else."
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
set +e
GLOB_OUT="$(bash "$HOOK" "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
GLOB_RC=$?
set -e
if [[ $GLOB_RC -eq 0 && "$GLOB_OUT" != *"DID NOT RUN"* && "$GLOB_OUT" != *"routes to"* ]]; then
pass "a monorepo under a directory named 'gl[1]?x' resolves exactly like any other"
else
fail "glob metacharacters in the path changed the verdict (exit $GLOB_RC): ${GLOB_OUT:-<empty>}"
fi
# ---------------------------------------------------------------------------
# 1b. Machine independence — the real corpus
# ---------------------------------------------------------------------------
# The fixture above proves the rule; this proves it at the scale where it broke.
#
# The A/B is built rather than borrowed. plugins/ is copied TWICE: once bare (a
# fresh clone), and once with a synthetic .claude/skills/ tree deployed beside it
# holding a directory for every name the corpus currently reports as dangling. If
# deployed trees leaked back into the universe, the second copy would resolve
# those names and report an empty dangling set while the first reported two —
# 2-vs-6, reproduced deterministically.
#
# Deliberately NOT keyed on whether THIS machine has run `apm install`. Doing that
# would make the suite fail on a fresh clone (where there is no .claude/ to
# contrast against) — a test of machine independence that is itself
# machine-dependent. The live tree is still compared, as a third data point, but
# nothing here requires it to be in either state.
echo ""
echo "--- the real corpus reports the same dangling targets with and without a deployed tree ---"
dangling_set() {
local -a files=()
local f
# Collected with a `while read` loop, not `mapfile`: macOS ships bash 3.2,
# which has no `mapfile`, and tests/test-vale-wrap.sh scans tests/*.sh for
# exactly that hazard. `find` rather than a glob so both roots walk identically.
while IFS= read -r f; do
files+=("$f")
done < <(find "$1" -path '*/.apm/skills/*/SKILL.md' | sort)
if [[ ${#files[@]} -eq 0 ]]; then
echo "NO-FILES-FOUND"
return
fi
set +e
bash "$HOOK" ${files[@]+"${files[@]}"} 2>&1 \
| grep -oE "routes to '[^']+'" \
| sed "s/routes to '//; s/'//" \
| sort -u
set -e
}
LIVE_DANGLING="$(dangling_set "$REPO_ROOT/plugins")"
FRESH_ROOT="$TMPDIR_T/fresh-clone"
mkdir -p "$FRESH_ROOT"
cp -R "$REPO_ROOT/plugins" "$FRESH_ROOT/plugins"
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$FRESH_ROOT/apm.yml"
FRESH_DANGLING="$(dangling_set "$FRESH_ROOT/plugins")"
DEPLOYED_ROOT="$TMPDIR_T/deployed-clone"
mkdir -p "$DEPLOYED_ROOT/.claude/skills" "$DEPLOYED_ROOT/.claude/agents"
cp -R "$REPO_ROOT/plugins" "$DEPLOYED_ROOT/plugins"
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$DEPLOYED_ROOT/apm.yml"
# Deploy exactly the names that currently dangle. That is the strongest possible
# bait: if the deployed tree were consulted, every one of them would resolve and
# the dangling set would collapse to empty.
DEPLOY_COUNT=0
while IFS= read -r name; do
[[ -n "$name" ]] || continue
mkdir -p "$DEPLOYED_ROOT/.claude/skills/$name"
DEPLOY_COUNT=$((DEPLOY_COUNT + 1))
done <<< "$FRESH_DANGLING"
DEPLOYED_DANGLING="$(dangling_set "$DEPLOYED_ROOT/plugins")"
if [[ "$DEPLOY_COUNT" -gt 0 ]]; then
pass "precondition: $DEPLOY_COUNT dangling name(s) deployed into the contrast tree's .claude/skills/, so the A/B has something to distinguish"
else
fail "no dangling names to deploy — the corpus reports none, so this A/B distinguishes nothing. Deploy a known-absent name explicitly instead of deriving one."
fi
if [[ ! -d "$FRESH_ROOT/.claude" && ! -d "$FRESH_ROOT/.agents" ]]; then
pass "precondition: the fresh-clone copy has no deployed tree of its own"
else
fail "the fresh-clone copy picked up a deployed tree — it is not a fresh-clone fixture"
fi
if [[ "$FRESH_DANGLING" == "$DEPLOYED_DANGLING" ]]; then
pass "deploying every dangling name into .claude/skills/ changes nothing: $(echo "$FRESH_DANGLING" | tr '\n' ' ')"
else
fail "the corpus verdict depends on whether apm install has been run — fresh clone: [$(echo "$FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$DEPLOYED_DANGLING" | tr '\n' ' ')]"
fi
# Third data point: whatever state THIS machine happens to be in, the live tree
# must agree with a bare copy of the same plugins/. No precondition on that state
# — see the section header.
if [[ "$LIVE_DANGLING" == "$FRESH_DANGLING" ]]; then
pass "the live tree agrees with a bare copy (this machine $( [[ -d "$REPO_ROOT/.claude/skills" ]] && echo "HAS" || echo "has no" ) deployed .claude/skills/ tree)"
else
fail "the live tree disagrees with a bare copy of the same plugins/ — live: [$(echo "$LIVE_DANGLING" | tr '\n' ' ')] fresh clone: [$(echo "$FRESH_DANGLING" | tr '\n' ' ')]"
fi
# ---------------------------------------------------------------------------
# 1c. The two live true positives, pinned by name
# ---------------------------------------------------------------------------
# ADR-0020 records these as real broken routing targets and splits fixing them
# into Gitea issue #100. Until that lands they are the ONLY evidence the dangling
# check finds anything at all in real prose, so they are asserted as an exact set
# rather than a "contains" — a false-positive fix that suppressed one of them
# would otherwise land green.
#
# `gitea-labels` WAS the subtler of the two: it was never written anywhere as
# `gitea-labels`. gitea-issues' description said "Composes `gitea-labels-\n
# milestones`" in a `>`-folded scalar, and the fold joined the lines into
# "gitea-labels- milestones" — the trailing hyphen is what kept the token
# terminal and therefore danglable. The issue #99 retrofit cut that composition
# sentence and the dangling target went with it, so the set is down to one.
#
# WHEN `research` IS RETROFITTED: drop neuledge-context and leave the set empty.
# Do not delete the assertion — an empty expected set is fine and still pins
# that no NEW dangling target appeared.
echo ""
echo "--- the live dangling targets in the corpus are exactly the ADR-0020 records still open ---"
EXPECTED_DANGLING="$(printf '%s\n' neuledge-context)"
if [[ "$LIVE_DANGLING" == "$EXPECTED_DANGLING" ]]; then
pass "the corpus dangling set is exactly {neuledge-context}"
else
fail "the corpus dangling set changed — expected [$(echo "$EXPECTED_DANGLING" | tr '\n' ' ')], got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. If a retrofit fixed one, update EXPECTED_DANGLING; if a false-positive fix silently deleted one, that is the regression this asserts."
fi
# shellcheck disable=SC2043 # one probe left by design -- the list shrinks as
# each fixture is retrofitted and reaches zero when `research` lands. Keeping the
# loop means removing the last entry is a one-line edit, not a restructure.
for probe in \
"plugins/bin/.apm/skills/research/SKILL.md:neuledge-context"; do
probe_file="$REPO_ROOT/${probe%%:*}"
probe_name="${probe##*:}"
if [[ ! -f "$probe_file" ]]; then
fail "the true-positive fixture ${probe%%:*} no longer exists — this pin has become vacuous"
continue
fi
set +e
probe_out="$(bash "$HOOK" "$probe_file" 2>&1)"
set -e
if [[ "$probe_out" == *"routes to '$probe_name'"* ]]; then
pass "detects the dangling '$probe_name' target in ${probe%%:*}"
else
fail "did not detect the dangling '$probe_name' target in ${probe%%:*} — a false-positive fix has taken a true positive with it: $probe_out"
fi
done
# ---------------------------------------------------------------------------
# 2. The bare-target grammar rule
# ---------------------------------------------------------------------------
# Every fixture is built inside a real plugin tree. In a bare temp directory the
# resolver would decline ("DID NOT RUN") and every must-not-error case would pass
# vacuously, proving nothing about extraction.
echo ""
echo "--- attributive compound modifiers are prose, not routing targets ---"
GRAMMAR_ROOT="$TMPDIR_T/grammar"
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/sibling-skill" sibling-skill \
"Use when doing the other thing. Do not use for anything else."
# grammar_case <slug> <expect: silent|errors> <needle> <description>
grammar_case() {
local slug="$1" mode="$2" needle="$3" desc="$4" out status=0
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/$slug" "$slug" "$desc"
set +e
out="$(bash "$HOOK" "$GRAMMAR_ROOT/plugins/p/.apm/skills/$slug/SKILL.md" 2>&1)"
status=$?
set -e
if [[ "$out" == *"DID NOT RUN"* ]]; then
fail "\"$desc\" — the resolver declined, so this case asserts nothing about extraction: $out"
return
fi
case "$mode" in
silent)
if [[ $status -eq 0 && -z "$out" ]]; then
pass "not a dangling target: \"$desc\""
else
fail "\"$desc\" (exit $status, output: ${out:-<empty>})"
fi
;;
errors)
if [[ $status -ne 0 && "$out" == *"$needle"* ]]; then
pass "still a dangling target: \"$desc\""
else
fail "\"$desc\" should have ERRORed with $needle (exit $status, output: ${out:-<empty>})"
fi
;;
suggests)
# Reported, not blocking. Both halves matter: an ERROR here would be the
# unsuppressable false positive this tier exists to remove, and silence
# would mean the gate stopped noticing the target at all.
if [[ $status -eq 0 && "$out" == *"SUGGESTION"*"$needle"* && "$out" != *"ERROR"* ]]; then
pass "reported but not blocking: \"$desc\""
else
fail "\"$desc\" should have exited 0 with a SUGGESTION naming $needle (exit $status, output: ${out:-<empty>})"
fi
;;
esac
}
# The four phrasings that were hard dangling FAILs with no suppression. All four
# are lifted from real descriptions in this corpus.
grammar_case fp-precommit-hooks silent "" \
"Use when running the linter. Use pre-commit hooks instead of ad-hoc scripts."
grammar_case fp-pull-request silent "" \
"Use when opening changes. Invoke the pull-request template instead of writing one by hand."
grammar_case fp-conventional silent "" \
"Use when writing history. Use conventional-commits formatting rather than free-form messages."
grammar_case fp-prepush-backticked silent "" \
"Use when checking a branch. Do not use for local edits — run the \`pre-push\` hooks instead."
echo ""
echo "--- a lone unresolvable token in terminal position is REPORTED, not blocking ---"
# The class this tier was added for. Every one of these is grammatically
# identical to a real broken route — "route verb + name + terminal" is also how
# prose cites a hook, a linter, a file format or an English compound — and every
# one of them was a hard FAIL with no suppression mechanism anywhere in the gate.
# The skills most exposed are exactly the ones the ADR-0020 retrofit sends
# authors back to rewrite first: pc-run, pc-author, vale-run, vale-config and the
# apm-* family are all ABOUT hyphenated tools.
grammar_case fp-precommit-terminal suggests "routes to 'pre-commit'" \
"Use when running the linter. Do not use for running hooks — run \`pre-commit\` instead."
grammar_case fp-commit-msg suggests "routes to 'commit-msg'" \
"Use when writing history. Do not use for the commit message — see \`commit-msg\`."
grammar_case fp-type-check suggests "routes to 'type-check'" \
"Use when compiling. Do not use for type errors — run \`type-check\` first."
grammar_case fp-semantic-release suggests "routes to 'semantic-release'" \
"Use when tagging a version. Instead, use \`semantic-release\`."
# Single-word tool names are the same defect: `eslint` in terminal position hit
# the marked-target path and hard-FAILed just as `pre-commit` did.
grammar_case fp-eslint suggests "routes to 'eslint'" \
"Use when linting JS. Do not use for style — run \`eslint\` instead."
# Bare English compounds, which the backtick path never sees at all.
grammar_case fp-clean-up suggests "routes to 'clean-up'" \
"Use when doing the thing. Do not use for the old flow — use the clean-up instead."
grammar_case fp-built-in suggests "routes to 'built-in'" \
"Use when doing the thing. Do not use for the custom path — Instead, prefer the built-in."
grammar_case fp-write-up suggests "routes to 'write-up'" \
"Use when doing the thing. Do not use for the summary — see the write-up."
grammar_case fp-front-end suggests "routes to 'front-end'" \
"Use when doing the thing. Do not use for the API layer — use the front-end."
grammar_case fp-unit-tests suggests "routes to 'unit-tests'" \
"Use when testing. Do not run end-to-end, run unit-tests."
echo ""
echo "--- terminal targets that resolve to nothing still ERROR ---"
# The controls. Without them the cases above are satisfied by a check that never
# fires, and the narrowing would have eaten the gate rather than sharpened it.
# Three forms, three code paths:
# * route NOTATION — `/name` and `-> name` — is exempt from corroboration and
# blocks on its own. Nobody writes `/pre-commit` or `-> pre-commit` to mean
# the hook, so there is no ambiguity to resolve, and an author who wants a
# route checked unconditionally has two ways to say so.
# * a PROSE-form target — backticked or bare — blocks when its own sentence
# names another target that resolves. `sibling-skill` is that corroborator
# here; it is the same shape as both live true positives, which sit beside
# `write-docs` and `gitea-labels-milestones` respectively.
grammar_case tp-arrow errors "routes to 'no-such-arrow-target'" \
"Use when doing the thing. Not the other thing → no-such-arrow-target."
grammar_case tp-slash errors "routes to 'no-such-slash-skill'" \
"Use when doing the thing. Do not use for improvements — use /no-such-slash-skill instead."
grammar_case tp-backticked errors "routes to 'no-such-backticked-skill'" \
"Use when doing the thing. Do not use for improvements — use \`sibling-skill\` or \`no-such-backticked-skill\` instead."
grammar_case tp-bare-terminal errors "routes to 'no-such-bare-skill'" \
"Use when doing the thing. Do not use for improvements — use sibling-skill or no-such-bare-skill instead."
echo ""
echo "--- corroboration is scoped to a REAL sentence, not to whatever the splitter says ---"
# Corroboration decides SUGGESTION vs blocking ERROR, so a mis-placed sentence
# boundary moves a target between the two tiers. The naive "period, space,
# capital" rule got this wrong in both directions, and both were live:
#
# OVER-SPLIT. `e.g. "..."` is not a sentence end, but the quote looks like a
# start. The clause was cut in half and the corroborator stranded on the far
# side, so a target that DOES sit beside a resolving sibling silently demoted
# to SUGGESTION — a measurement taken and then discarded.
grammar_case abbrev-split errors "routes to 'no-such-abbrev-skill'" \
"Use when doing the thing. Do not use for improvements — use sibling-skill first, e.g. \"run the audit\", then use no-such-abbrev-skill instead."
#
# UNDER-SPLIT. A sentence opening with a lowercase word or a code span was not
# seen as a start at all, so two sentences merged and a resolving target in the
# FIRST vouched for an unresolvable one in the SECOND that it never stood
# beside — a hard FAIL with no escape hatch, which is the exact failure
# corroboration was added to prevent. The target must still be REPORTED; only
# the power to block is withdrawn.
grammar_case lowercase-start suggests "routes to 'no-such-lower-skill'" \
"Use when doing the thing. Use sibling-skill for the main case. do not use for improvements — use no-such-lower-skill instead."
grammar_case backtick-start suggests "routes to 'no-such-tick-skill'" \
"Use when doing the thing. Use sibling-skill for the main case. \`no-such-tick-skill\` is not for this — do not use it instead."
# And the confirming half of the grammar rule: a compound-modifier target is
# CONFIRM-ONLY, not ignored. When the name does exist it still counts as a route
# — the rule suppresses the ERROR, it does not delete the target.
echo ""
echo "--- an attributive target that DOES resolve is still a route, not a discarded token ---"
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/attributive-subject" attributive-subject \
"Use when doing the thing. Do not use for the other thing — use the sibling-skill helper instead."
set +e
ATTR_OUT="$(bash "$HOOK" "$GRAMMAR_ROOT/plugins/p/.apm/skills/attributive-subject/SKILL.md" 2>&1)"
ATTR_RC=$?
set -e
if [[ $ATTR_RC -eq 0 && -z "$ATTR_OUT" ]]; then
pass "a resolving attributive target neither errors nor is reported"
else
fail "an attributive target naming a REAL skill produced output (exit $ATTR_RC): $ATTR_OUT"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]