Two defects in the routing-target resolver, both latent in the corpus but hot for anything written next. The free-standing `/name` sweep sat inside `if boundary:`, so route notation in a sentence carrying no boundary marker was never extracted at all — not an ERROR, not a SUGGESTION, not an INFO. That contradicted ADR-0020's amendment and gates.md, which both promise `/name` blocks unconditionally. The sweep now runs over every sentence. `-> name` and backticked forms stay gated deliberately: an arrow also writes a process chain and a code span cites tools, files and skills alike, so ungating either fires on ordinary prose. The path guard used `\b`, which still holds after a hyphen, so the engine backtracked to a shorter hyphen-terminated prefix whenever the lookahead rejected the full segment. `/api-docs/v2.md` in a boundary clause raised blocking ERRORs for 'api' and 'api-docs' — names no author wrote, with no corroboration escape. `(?![\w-])` forbids the shortened prefix outright; MARKED_TARGET, which had no trailing guard at all, gained one. Zero arguments now exits 2 rather than 0, so a mis-scoped `files:` pattern is no longer indistinguishable from a clean corpus. Both hook manifests pass filenames and pre-commit skips a filename-passing hook when nothing matches, so the hook never sees an empty argv — that contract is now asserted by a test rather than left in prose. Deleting the sweep entirely used to leave every suite green. It now kills eight assertions. The suite also gains its first slash-path and URL fixtures, in both directions. Refs: #107, #110, #124 ADR: 0020
947 lines
54 KiB
Bash
Executable File
947 lines
54 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Regression test for the two properties of ADR-0020 boundary-target resolution
|
|
# that decide whether the gate can be trusted at all.
|
|
#
|
|
# 1. MACHINE INDEPENDENCE. The resolution universe is derived by walking up
|
|
# FROM THE TARGET FILE to an authoring root, and when one is found the
|
|
# deployed .claude/ and .agents/ trees are deliberately NOT consulted. Those
|
|
# trees are `apm install` output — gitignored, and present only on a machine
|
|
# that has run it. Four cross-plugin targets in this repo resolved through
|
|
# .claude/skills/ alone, so the same commit measured 2 dangling targets on a
|
|
# developer machine and 6 on a fresh clone. A gate shipping hot with no
|
|
# baseline cannot give two answers, so this file asserts the verdict is
|
|
# identical with and without a deployed tree — on a synthetic fixture AND on
|
|
# the real 39-skill corpus.
|
|
#
|
|
# Three further ways the universe can be built out of the wrong directory,
|
|
# each of which shipped: a `.git` at the CONSUMER root (the fallback is
|
|
# truthy in any git repo, which made the deployed-tree branch dead code), a
|
|
# `.git` INSIDE a plugin (the walk-up is two passes precisely so this cannot
|
|
# capture the root), and glob metacharacters in the checkout path (which
|
|
# turned the directory name into a character class matching nothing, and the
|
|
# resolver into a no-op that still reported green).
|
|
#
|
|
# 2. THE BARE-TARGET GRAMMAR RULE. A hyphenated token used as a compound
|
|
# MODIFIER ("pre-commit hooks", "pull-request template") is prose, not a
|
|
# route; a terminal one is a real target. Getting this wrong in either
|
|
# direction is fatal: firing on prose makes the gate untrustworthy and it
|
|
# gets turned off, while suppressing too much deletes the only two true
|
|
# positives the corpus has. Both live true positives are BARE, which is why
|
|
# the rule keys on the FOLLOWER TOKEN rather than on marking, and why they
|
|
# are pinned by name below — a future false-positive fix must not be able to
|
|
# quietly take them with it.
|
|
#
|
|
# 3. IN-SENTENCE CORROBORATION. Terminal position alone is not evidence of a
|
|
# route: "run `pre-commit` instead", "see `commit-msg`", "use the clean-up
|
|
# instead" and "run unit-tests" are all terminal, all prose, and all were
|
|
# hard FAILs with no suppression mechanism anywhere in the gate. A
|
|
# prose-form target therefore blocks only when its own sentence names
|
|
# another target that RESOLVES; otherwise it is reported at SUGGESTION tier
|
|
# and the commit proceeds. Route NOTATION (`/name`, `-> name`) is exempt
|
|
# and always blocks. Both halves are asserted below: the prose class must
|
|
# report-not-block, and the notation and corroborated forms must still
|
|
# ERROR, or the fix would have eaten the gate rather than narrowed it.
|
|
set -euo pipefail
|
|
|
|
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
HOOK="$REPO_ROOT/scripts/skill-size-check.sh"
|
|
PASS=0
|
|
FAIL=0
|
|
|
|
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
|
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
|
|
|
TMPDIR_T="$(mktemp -d)"
|
|
trap 'rm -rf "$TMPDIR_T"' EXIT
|
|
|
|
# write_skill <skill-dir> <name> <desc>
|
|
write_skill() {
|
|
mkdir -p "$1"
|
|
{
|
|
echo "---"
|
|
echo "name: $2"
|
|
echo "description: $3"
|
|
echo "---"
|
|
echo ""
|
|
echo "Do the thing."
|
|
} > "$1/SKILL.md"
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1. Machine independence — synthetic fixture
|
|
# ---------------------------------------------------------------------------
|
|
# Two trees, identical except that one also carries a deployed .claude/ tree
|
|
# holding a skill and an agent that exist NOWHERE in plugins/. The subject routes
|
|
# to one name that lives in a sibling plugin (must resolve in both) and one that
|
|
# lives only in .claude/ (must DANGLE in both — an authoring root exists, so the
|
|
# deployed tree is not part of the universe).
|
|
#
|
|
# If the deployed tree were consulted, the second target would resolve on the
|
|
# machine that has run `apm install` and dangle on a fresh clone. That is the
|
|
# 2-vs-6 defect exactly, at fixture scale.
|
|
echo ""
|
|
echo "--- the same file gets the same verdict with and without a deployed .claude/ tree ---"
|
|
build_tree() {
|
|
local root="$1"
|
|
write_skill "$root/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$root/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use cross-plugin-skill or deployed-only-skill instead."
|
|
}
|
|
build_tree "$TMPDIR_T/no-claude"
|
|
build_tree "$TMPDIR_T/with-claude"
|
|
# The deployed tree, present only in the second root. Both a skill and an agent,
|
|
# because both are valid routing targets and both would leak.
|
|
#
|
|
# The skill gets a real SKILL.md. That is not decoration: a directory under
|
|
# skills/ is a resolvable name only when it HOLDS one, so an empty directory
|
|
# would dangle for the wrong reason and the assertion below would pass without
|
|
# testing the deployed-tree rule at all.
|
|
mkdir -p "$TMPDIR_T/with-claude/.claude/agents"
|
|
write_skill "$TMPDIR_T/with-claude/.claude/skills/deployed-only-skill" deployed-only-skill \
|
|
"Use when doing the deployed thing. Do not use for anything else."
|
|
: > "$TMPDIR_T/with-claude/.claude/agents/deployed-only-agent.md"
|
|
|
|
run_subject() {
|
|
local root="$1" out
|
|
set +e
|
|
out="$(bash "$HOOK" "$root/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
set -e
|
|
# Normalise the tree root out of the paths so the two runs are comparable.
|
|
printf '%s\n' "$out" | sed "s#$root#<ROOT>#g"
|
|
}
|
|
NO_CLAUDE_OUT="$(run_subject "$TMPDIR_T/no-claude")"
|
|
WITH_CLAUDE_OUT="$(run_subject "$TMPDIR_T/with-claude")"
|
|
|
|
if [[ "$NO_CLAUDE_OUT" == "$WITH_CLAUDE_OUT" ]]; then
|
|
pass "identical output with and without a deployed .claude/ tree"
|
|
else
|
|
fail "the deployed .claude/ tree changed the verdict — without: [$NO_CLAUDE_OUT] with: [$WITH_CLAUDE_OUT]"
|
|
fi
|
|
# Identical-but-wrong is still possible (both could resolve everything, or
|
|
# neither could resolve anything), so the CONTENT is asserted too: the
|
|
# sibling-plugin name must resolve and the deployed-only name must not.
|
|
if [[ "$WITH_CLAUDE_OUT" == *"routes to 'deployed-only-skill'"* ]]; then
|
|
pass "a name that exists only in .claude/ still dangles when an authoring root is present"
|
|
else
|
|
fail "the deployed-only target did not dangle — the deployed tree is being read into the universe: $WITH_CLAUDE_OUT"
|
|
fi
|
|
if [[ "$WITH_CLAUDE_OUT" != *"routes to 'cross-plugin-skill'"* ]]; then
|
|
pass "a name in a SIBLING PLUGIN resolves, so the comparison above is not 'nothing resolves'"
|
|
else
|
|
fail "the sibling-plugin target dangled — the monorepo universe is not being built: $WITH_CLAUDE_OUT"
|
|
fi
|
|
|
|
# The other half of the rule: with NO authoring root, deployed trees ARE the
|
|
# universe. That is the consumer case, and without this the rule above could be
|
|
# implemented as "never read .claude/", which would leave consumers with no
|
|
# resolution at all.
|
|
echo ""
|
|
echo "--- with no authoring root, a deployed .claude/ tree IS the universe ---"
|
|
CONSUMER="$TMPDIR_T/consumer"
|
|
write_skill "$CONSUMER/.claude/skills/deployed-only-skill" deployed-only-skill \
|
|
"Use when doing the deployed thing. Do not use for anything else."
|
|
write_skill "$CONSUMER/.claude/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use deployed-only-skill instead."
|
|
set +e
|
|
CONSUMER_OUT="$(bash "$HOOK" "$CONSUMER/.claude/skills/my-skill/SKILL.md" 2>&1)"
|
|
CONSUMER_RC=$?
|
|
set -e
|
|
if [[ $CONSUMER_RC -eq 0 && "$CONSUMER_OUT" != *"routes to"* && "$CONSUMER_OUT" != *"DID NOT RUN"* ]]; then
|
|
pass "a sibling in a deployed .claude/skills/ tree resolves when there is no authoring root"
|
|
else
|
|
fail "the consumer path did not resolve through the deployed tree (exit $CONSUMER_RC): ${CONSUMER_OUT:-<empty>}"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1a-bis. The consumer case with the one thing every real consumer has: .git
|
|
# ---------------------------------------------------------------------------
|
|
# The fixture immediately above has no .git, and that is precisely why it could
|
|
# never catch this. _authoring_root() falls back to the nearest .git ancestor, so
|
|
# it returns truthy in ANY git repo — a consumer checkout included. The branch
|
|
# that reads the deployed trees was guarded by `else`, so in every consumer
|
|
# checkout the fallback won, _collect_authoring_root() contributed nothing
|
|
# (there is no plugins/ directory to collect), and _deployed_roots() was dead
|
|
# code in exactly the case it exists for.
|
|
#
|
|
# The pair below is the whole test: the SAME tree, once with .git and once
|
|
# without. Old behaviour was rc=1 with .git and rc=0 without; a test covering
|
|
# only the no-.git shape reports green on both.
|
|
#
|
|
# `deployed-only-agent` lives ONLY in .agents/agents/, so it can be reached
|
|
# through no route but _deployed_roots(). `sibling-skill` sits in .claude/skills/
|
|
# beside the subject, which the sibling-collection block above reaches on its own
|
|
# — it is the corroborator that makes the dangling target BLOCKING rather than a
|
|
# SUGGESTION, so the old failure shows up in the exit code and not only in prose.
|
|
echo ""
|
|
echo "--- a consumer checkout resolves through its deployed trees even though it is a git repo ---"
|
|
build_consumer() {
|
|
local root="$1"
|
|
mkdir -p "$root/.agents/agents"
|
|
write_skill "$root/.claude/skills/sibling-skill" sibling-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$root/.claude/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use sibling-skill or deployed-only-agent instead."
|
|
: > "$root/.agents/agents/deployed-only-agent.agent.md"
|
|
}
|
|
build_consumer "$TMPDIR_T/consumer-git"
|
|
mkdir -p "$TMPDIR_T/consumer-git/.git"
|
|
build_consumer "$TMPDIR_T/consumer-nogit"
|
|
|
|
# consumer_case <label> <root>
|
|
consumer_case() {
|
|
local label="$1" root="$2" out status=0
|
|
set +e
|
|
out="$(bash "$HOOK" "$root/.claude/skills/my-skill/SKILL.md" 2>&1)"
|
|
status=$?
|
|
set -e
|
|
if [[ $status -eq 0 && "$out" != *"routes to"* && "$out" != *"DID NOT RUN"* ]]; then
|
|
pass "$label"
|
|
else
|
|
fail "$label (exit $status, output: ${out:-<empty>})"
|
|
fi
|
|
}
|
|
consumer_case "an agent in .agents/agents/ resolves in a consumer checkout that HAS a .git directory" \
|
|
"$TMPDIR_T/consumer-git"
|
|
consumer_case "control: the same tree without .git resolves too (the shape that always passed)" \
|
|
"$TMPDIR_T/consumer-nogit"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1a-ter. A monorepo with ONE plugin is still a monorepo
|
|
# ---------------------------------------------------------------------------
|
|
# The first attempt at the fix above conditioned the deployed branch on whether
|
|
# the authoring root had CONTRIBUTED a name — `if len(names) == before:`. That
|
|
# reads as "the .git fallback collected nothing, so fall through", and it is
|
|
# wrong: _collect_authoring_root() re-collects the subject's OWN plugin, whose
|
|
# names the sibling and package blocks have already added. With two plugins
|
|
# (fixture 1) the cross-plugin name makes the delta non-zero and the guard stays
|
|
# shut. With ONE plugin the delta is zero, the guard fires in a genuine
|
|
# monorepo, and _deployed_roots() walks up to ten levels — reaching the user's
|
|
# global ~/.claude/skills. That is install-dependence again, in the shape
|
|
# ADR-0020 lines 118-127 exist to forbid.
|
|
#
|
|
# So the predicate is which PROBE matched, not how many names arrived. The
|
|
# assertion is the same shape as fixture 1 — identical verdict either way — but
|
|
# on a single-plugin tree, which fixture 1 cannot express.
|
|
echo ""
|
|
echo "--- a SINGLE-plugin monorepo does not fall through to the deployed trees ---"
|
|
build_single() {
|
|
local root="$1"
|
|
write_skill "$root/plugins/only-plugin/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use /deployed-only-skill instead."
|
|
}
|
|
build_single "$TMPDIR_T/single-no-claude"
|
|
build_single "$TMPDIR_T/single-with-claude"
|
|
write_skill "$TMPDIR_T/single-with-claude/.claude/skills/deployed-only-skill" deployed-only-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
|
|
run_single() {
|
|
local root="$1" out
|
|
set +e
|
|
out="$(bash "$HOOK" "$root/plugins/only-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
set -e
|
|
printf '%s\n' "$out" | sed "s#$root#<ROOT>#g"
|
|
}
|
|
SINGLE_NO_OUT="$(run_single "$TMPDIR_T/single-no-claude")"
|
|
SINGLE_WITH_OUT="$(run_single "$TMPDIR_T/single-with-claude")"
|
|
|
|
if [[ "$SINGLE_NO_OUT" == "$SINGLE_WITH_OUT" ]]; then
|
|
pass "a single-plugin monorepo gets the same verdict with and without a deployed .claude/ tree"
|
|
else
|
|
fail "the deployed tree changed the verdict in a single-plugin monorepo — without: [$SINGLE_NO_OUT] with: [$SINGLE_WITH_OUT]"
|
|
fi
|
|
# Identical-but-wrong guard, as in fixture 1: the deployed-only name must DANGLE,
|
|
# not resolve. Written as `/deployed-only-skill` so it blocks on its own without
|
|
# needing a second target in the sentence to corroborate it.
|
|
if [[ "$SINGLE_WITH_OUT" == *"routes to 'deployed-only-skill'"* ]]; then
|
|
pass "the deployed-only target dangles in a single-plugin monorepo (~/.claude/skills is not in the universe)"
|
|
else
|
|
fail "the deployed-only target resolved — the single-plugin tree fell through to _deployed_roots(): $SINGLE_WITH_OUT"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1c. A nested .git inside a plugin must not beat the monorepo root
|
|
# ---------------------------------------------------------------------------
|
|
# ADR-0020 records the walk-up as TWO passes — plugins/*/.apm/{skills,agents}
|
|
# first, .git only afterwards — specifically so a .git inside a plugin (a
|
|
# submodule, or a sub-package with its own worktree) cannot capture the root.
|
|
# Nothing anywhere placed a .git inside a plugin, so the second pass was
|
|
# structural claim only. Collapsing the two probes into one interleaved walk
|
|
# passes every other fixture in this repo and fails here.
|
|
echo ""
|
|
echo "--- a .git INSIDE a plugin does not shadow the monorepo root above it ---"
|
|
NESTED="$TMPDIR_T/nested-git"
|
|
write_skill "$NESTED/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
|
|
# The trap: a git checkout one level BELOW the monorepo root and above the skill.
|
|
mkdir -p "$NESTED/plugins/subject-plugin/.git"
|
|
set +e
|
|
NESTED_OUT="$(bash "$HOOK" "$NESTED/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
NESTED_RC=$?
|
|
set -e
|
|
# The sibling-plugin name is the discriminator: it is reachable ONLY from the
|
|
# monorepo root. If the nested .git won, subject-plugin would be the root, its
|
|
# plugins/ glob would collect nothing, and cross-plugin-skill would dangle —
|
|
# corroborated by sibling-skill in the same sentence, so it would BLOCK.
|
|
if [[ $NESTED_RC -eq 0 && "$NESTED_OUT" != *"routes to"* && "$NESTED_OUT" != *"DID NOT RUN"* ]]; then
|
|
pass "a sibling-plugin target still resolves with a .git directory inside the subject's own plugin"
|
|
else
|
|
fail "the nested .git captured the authoring root (exit $NESTED_RC): ${NESTED_OUT:-<empty>}"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1d. Glob metacharacters in the checkout path
|
|
# ---------------------------------------------------------------------------
|
|
# The universe is built with glob.glob() against paths that begin with the
|
|
# checkout directory. A `[`, `]`, `*` or `?` anywhere in that prefix — a worktree
|
|
# named `feature[2]`, a CI workspace named `build[1]` — turned the literal
|
|
# directory name into a character class that matched nothing. The resolver then
|
|
# found no universe at all and degraded to the "DID NOT RUN" INFO with rc=0:
|
|
# every routing target in the tree silently unchecked, on a gate that reports
|
|
# green. Same monorepo as above, one directory renamed.
|
|
echo ""
|
|
echo "--- glob metacharacters in the checkout path do not silently disable the resolver ---"
|
|
GLOBDIR="$TMPDIR_T/gl[1]?x/mono"
|
|
write_skill "$GLOBDIR/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
|
|
set +e
|
|
GLOB_OUT="$(bash "$HOOK" "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
GLOB_RC=$?
|
|
set -e
|
|
if [[ $GLOB_RC -eq 0 && "$GLOB_OUT" != *"DID NOT RUN"* && "$GLOB_OUT" != *"routes to"* ]]; then
|
|
pass "a monorepo under a directory named 'gl[1]?x' resolves exactly like any other"
|
|
else
|
|
fail "glob metacharacters in the path changed the verdict (exit $GLOB_RC): ${GLOB_OUT:-<empty>}"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1b. Machine independence — the real corpus
|
|
# ---------------------------------------------------------------------------
|
|
# The fixture above proves the rule; this proves it at the scale where it broke.
|
|
#
|
|
# The A/B is built rather than borrowed. plugins/ is copied TWICE: once bare (a
|
|
# fresh clone), and once with a synthetic .claude/skills/ tree deployed beside it
|
|
# holding a directory for every name the corpus currently reports as dangling. If
|
|
# deployed trees leaked back into the universe, the second copy would resolve
|
|
# those names and report an empty dangling set while the first reported two —
|
|
# 2-vs-6, reproduced deterministically.
|
|
#
|
|
# Deliberately NOT keyed on whether THIS machine has run `apm install`. Doing that
|
|
# would make the suite fail on a fresh clone (where there is no .claude/ to
|
|
# contrast against) — a test of machine independence that is itself
|
|
# machine-dependent. The live tree is still compared, as a third data point, but
|
|
# nothing here requires it to be in either state.
|
|
echo ""
|
|
echo "--- the real corpus reports the same dangling targets with and without a deployed tree ---"
|
|
dangling_set() {
|
|
local -a files=()
|
|
local f
|
|
# Collected with a `while read` loop, not `mapfile`: macOS ships bash 3.2,
|
|
# which has no `mapfile`, and tests/test-vale-wrap.sh scans tests/*.sh for
|
|
# exactly that hazard. `find` rather than a glob so both roots walk identically.
|
|
while IFS= read -r f; do
|
|
files+=("$f")
|
|
done < <(find "$1" -path '*/.apm/skills/*/SKILL.md' | sort)
|
|
if [[ ${#files[@]} -eq 0 ]]; then
|
|
echo "NO-FILES-FOUND"
|
|
return
|
|
fi
|
|
set +e
|
|
bash "$HOOK" ${files[@]+"${files[@]}"} 2>&1 \
|
|
| grep -oE "routes to '[^']+'" \
|
|
| sed "s/routes to '//; s/'//" \
|
|
| sort -u
|
|
set -e
|
|
}
|
|
LIVE_DANGLING="$(dangling_set "$REPO_ROOT/plugins")"
|
|
|
|
FRESH_ROOT="$TMPDIR_T/fresh-clone"
|
|
mkdir -p "$FRESH_ROOT"
|
|
cp -R "$REPO_ROOT/plugins" "$FRESH_ROOT/plugins"
|
|
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$FRESH_ROOT/apm.yml"
|
|
FRESH_DANGLING="$(dangling_set "$FRESH_ROOT/plugins")"
|
|
|
|
# The bait used to be DERIVED from the corpus: deploy exactly the names that
|
|
# currently dangle. That was the strongest bait available while the corpus had
|
|
# dangling names — and it silently became vacuous the moment issue #99 fixed the
|
|
# last one, because a corpus reporting nothing gives nothing to deploy. A test of
|
|
# "deployed trees do not leak" must not depend on the corpus staying broken.
|
|
#
|
|
# So the bait is now EXPLICIT. Both contrast copies get one synthetic skill whose
|
|
# boundary clause routes to a name guaranteed absent from the monorepo, and only
|
|
# the deployed copy gets that name planted in .claude/skills/. If deployed trees
|
|
# leaked into the resolver's universe, the deployed copy would resolve it and
|
|
# report an empty set while the fresh copy reported one. The A/B now distinguishes
|
|
# something on every run, forever, whatever the corpus does.
|
|
BAIT_NAME="no-such-deployed-only-skill"
|
|
BAIT_FRESH="$TMPDIR_T/bait-fresh"
|
|
BAIT_DEPLOYED="$TMPDIR_T/bait-deployed"
|
|
for bait_root in "$BAIT_FRESH" "$BAIT_DEPLOYED"; do
|
|
mkdir -p "$bait_root"
|
|
cp -R "$REPO_ROOT/plugins" "$bait_root/plugins"
|
|
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$bait_root/apm.yml"
|
|
write_skill "$bait_root/plugins/bin/.apm/skills/deployed-tree-probe" deployed-tree-probe \
|
|
"Use when doing the probe thing. Do not use for the other thing — use $BAIT_NAME instead."
|
|
done
|
|
# Only the deployed copy gets the name planted where `apm install` would put it,
|
|
# as a REAL skill directory holding a SKILL.md — an empty directory is not a
|
|
# resolvable name, so baiting with one would make the A/B pass vacuously.
|
|
mkdir -p "$BAIT_DEPLOYED/.claude/agents"
|
|
write_skill "$BAIT_DEPLOYED/.claude/skills/$BAIT_NAME" "$BAIT_NAME" \
|
|
"Use when doing the bait thing. Do not use for anything else."
|
|
BAIT_FRESH_DANGLING="$(dangling_set "$BAIT_FRESH/plugins")"
|
|
BAIT_DEPLOYED_DANGLING="$(dangling_set "$BAIT_DEPLOYED/plugins")"
|
|
|
|
if [[ "$BAIT_FRESH_DANGLING" == *"$BAIT_NAME"* ]]; then
|
|
pass "precondition: the bait target dangles in the un-deployed copy, so the A/B has something to distinguish"
|
|
else
|
|
fail "the bait target '$BAIT_NAME' does not dangle even without a deployed tree — the fixture is broken, so the contrast below proves nothing. Got: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')]"
|
|
fi
|
|
if [[ ! -d "$BAIT_FRESH/.claude" && ! -d "$BAIT_FRESH/.agents" ]]; then
|
|
pass "precondition: the fresh-clone copy has no deployed tree of its own"
|
|
else
|
|
fail "the fresh-clone copy picked up a deployed tree — it is not a fresh-clone fixture"
|
|
fi
|
|
if [[ "$BAIT_FRESH_DANGLING" == "$BAIT_DEPLOYED_DANGLING" ]]; then
|
|
pass "planting the dangling name in .claude/skills/ changes nothing: $(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')"
|
|
else
|
|
fail "the verdict depends on whether apm install has been run — fresh clone: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')]. A deployed .claude/skills/ tree is leaking into the resolver's universe."
|
|
fi
|
|
# Third data point: whatever state THIS machine happens to be in, the live tree
|
|
# must agree with a bare copy of the same plugins/. No precondition on that state
|
|
# — see the section header.
|
|
if [[ "$LIVE_DANGLING" == "$FRESH_DANGLING" ]]; then
|
|
pass "the live tree agrees with a bare copy (this machine $( [[ -d "$REPO_ROOT/.claude/skills" ]] && echo "HAS" || echo "has no" ) deployed .claude/skills/ tree)"
|
|
else
|
|
fail "the live tree disagrees with a bare copy of the same plugins/ — live: [$(echo "$LIVE_DANGLING" | tr '\n' ' ')] fresh clone: [$(echo "$FRESH_DANGLING" | tr '\n' ' ')]"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 1c. The two live true positives, pinned by name
|
|
# ---------------------------------------------------------------------------
|
|
# ADR-0020 records these as real broken routing targets and splits fixing them
|
|
# into Gitea issue #100. Until that lands they are the ONLY evidence the dangling
|
|
# check finds anything at all in real prose, so they are asserted as an exact set
|
|
# rather than a "contains" — a false-positive fix that suppressed one of them
|
|
# would otherwise land green.
|
|
#
|
|
# `gitea-labels` WAS the subtler of the two: it was never written anywhere as
|
|
# `gitea-labels`. gitea-issues' description said "Composes `gitea-labels-\n
|
|
# milestones`" in a `>`-folded scalar, and the fold joined the lines into
|
|
# "gitea-labels- milestones" — the trailing hyphen is what kept the token
|
|
# terminal and therefore danglable. The issue #99 retrofit cut that composition
|
|
# sentence and the dangling target went with it, so the set is down to one.
|
|
#
|
|
# `neuledge-context` was the last one. The issue #99 wave-3 retrofit deleted that
|
|
# boundary clause outright — commit `6146120` had already deleted the skill it
|
|
# named, and nothing has owned MCP-server installation since — so the corpus
|
|
# dangling set is now EMPTY.
|
|
#
|
|
# The assertion stays, and it is not vacuous now that it expects nothing: it is
|
|
# the only thing standing between a newly-authored boundary clause naming a
|
|
# non-existent target and a green suite. An empty expected set pins "no NEW
|
|
# dangling target appeared", which is the property that actually matters from
|
|
# here on.
|
|
#
|
|
# The per-target probe loop that used to sit below is GONE, not emptied. Its job
|
|
# was to prove the check detects each live dangling target individually, and with
|
|
# no live targets left there is nothing to point it at. A loop over an empty list
|
|
# is an assertion-free result counted in the totals — exactly the vacuous-pass
|
|
# shape the comment above rejects. Detection is still covered, and covered
|
|
# better, by the synthetic fixtures in section 2 below, which build a real plugin
|
|
# tree and assert the resolver fires. Do not reinstate the loop unless a real
|
|
# dangling target reappears in the corpus.
|
|
echo ""
|
|
echo "--- no skill in the corpus routes to a target that does not resolve ---"
|
|
EXPECTED_DANGLING=""
|
|
if [[ "$LIVE_DANGLING" == "$EXPECTED_DANGLING" ]]; then
|
|
pass "the corpus dangling set is empty"
|
|
else
|
|
fail "a dangling routing target appeared in the corpus — expected none, got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. A boundary clause names a skill or agent that does not resolve; fix the clause or the target. This assertion is the corpus-wide backstop, so do not relax it to make a new skill pass."
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2. The bare-target grammar rule
|
|
# ---------------------------------------------------------------------------
|
|
# Every fixture is built inside a real plugin tree. In a bare temp directory the
|
|
# resolver would decline ("DID NOT RUN") and every must-not-error case would pass
|
|
# vacuously, proving nothing about extraction.
|
|
echo ""
|
|
echo "--- attributive compound modifiers are prose, not routing targets ---"
|
|
GRAMMAR_ROOT="$TMPDIR_T/grammar"
|
|
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/sibling-skill" sibling-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
|
|
# grammar_case <slug> <expect: silent|errors> <needle> <description>
|
|
grammar_case() {
|
|
local slug="$1" mode="$2" needle="$3" desc="$4" out status=0
|
|
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/$slug" "$slug" "$desc"
|
|
set +e
|
|
out="$(bash "$HOOK" "$GRAMMAR_ROOT/plugins/p/.apm/skills/$slug/SKILL.md" 2>&1)"
|
|
status=$?
|
|
set -e
|
|
if [[ "$out" == *"DID NOT RUN"* ]]; then
|
|
fail "\"$desc\" — the resolver declined, so this case asserts nothing about extraction: $out"
|
|
return
|
|
fi
|
|
case "$mode" in
|
|
silent)
|
|
if [[ $status -eq 0 && -z "$out" ]]; then
|
|
pass "not a dangling target: \"$desc\""
|
|
else
|
|
fail "\"$desc\" (exit $status, output: ${out:-<empty>})"
|
|
fi
|
|
;;
|
|
errors)
|
|
if [[ $status -ne 0 && "$out" == *"$needle"* ]]; then
|
|
pass "still a dangling target: \"$desc\""
|
|
else
|
|
fail "\"$desc\" should have ERRORed with $needle (exit $status, output: ${out:-<empty>})"
|
|
fi
|
|
;;
|
|
suggests)
|
|
# Reported, not blocking. Both halves matter: an ERROR here would be the
|
|
# unsuppressable false positive this tier exists to remove, and silence
|
|
# would mean the gate stopped noticing the target at all.
|
|
if [[ $status -eq 0 && "$out" == *"SUGGESTION"*"$needle"* && "$out" != *"ERROR"* ]]; then
|
|
pass "reported but not blocking: \"$desc\""
|
|
else
|
|
fail "\"$desc\" should have exited 0 with a SUGGESTION naming $needle (exit $status, output: ${out:-<empty>})"
|
|
fi
|
|
;;
|
|
esac
|
|
}
|
|
|
|
# The four phrasings that were hard dangling FAILs with no suppression. All four
|
|
# are lifted from real descriptions in this corpus.
|
|
#
|
|
# THEY ARE `suggests`, NOT `silent`, AND THE DIFFERENCE IS THE POINT. The
|
|
# follower rule takes away the power to BLOCK a commit on a compound modifier;
|
|
# it does not take away visibility, and it used to. FOLLOWER_OK is a closed
|
|
# whitelist of about eighty words, so a non-terminal verdict means "the next
|
|
# token is outside a list someone maintains by hand", not "this is prose" — and
|
|
# `continue`ing on it made the gate fail OPEN on its own unfamiliarity: any
|
|
# target followed by an unlisted word was neither blocked nor mentioned at any
|
|
# tier. Asserting silence here pinned that hole in place. The assertion that
|
|
# still matters is `!= ERROR`, which `suggests` checks, and which is what keeps
|
|
# a false positive from stopping a commit.
|
|
grammar_case fp-precommit-hooks suggests "routes to 'pre-commit'" \
|
|
"Use when running the linter. Use pre-commit hooks instead of ad-hoc scripts."
|
|
grammar_case fp-pull-request suggests "routes to 'pull-request'" \
|
|
"Use when opening changes. Invoke the pull-request template instead of writing one by hand."
|
|
grammar_case fp-conventional suggests "routes to 'conventional-commits'" \
|
|
"Use when writing history. Use conventional-commits formatting rather than free-form messages."
|
|
grammar_case fp-prepush-backticked suggests "routes to 'pre-push'" \
|
|
"Use when checking a branch. Do not use for local edits — run the \`pre-push\` hooks instead."
|
|
|
|
echo ""
|
|
echo "--- a lone unresolvable token in terminal position is REPORTED, not blocking ---"
|
|
# The class this tier was added for. Every one of these is grammatically
|
|
# identical to a real broken route — "route verb + name + terminal" is also how
|
|
# prose cites a hook, a linter, a file format or an English compound — and every
|
|
# one of them was a hard FAIL with no suppression mechanism anywhere in the gate.
|
|
# The skills most exposed are exactly the ones the ADR-0020 retrofit sends
|
|
# authors back to rewrite first: pc-run, pc-author, vale-run, vale-config and the
|
|
# apm-* family are all ABOUT hyphenated tools.
|
|
grammar_case fp-precommit-terminal suggests "routes to 'pre-commit'" \
|
|
"Use when running the linter. Do not use for running hooks — run \`pre-commit\` instead."
|
|
grammar_case fp-commit-msg suggests "routes to 'commit-msg'" \
|
|
"Use when writing history. Do not use for the commit message — see \`commit-msg\`."
|
|
grammar_case fp-type-check suggests "routes to 'type-check'" \
|
|
"Use when compiling. Do not use for type errors — run \`type-check\` first."
|
|
grammar_case fp-semantic-release suggests "routes to 'semantic-release'" \
|
|
"Use when tagging a version. Instead, use \`semantic-release\`."
|
|
# Single-word tool names are the same defect: `eslint` in terminal position hit
|
|
# the marked-target path and hard-FAILed just as `pre-commit` did.
|
|
grammar_case fp-eslint suggests "routes to 'eslint'" \
|
|
"Use when linting JS. Do not use for style — run \`eslint\` instead."
|
|
# Bare English compounds, which the backtick path never sees at all.
|
|
grammar_case fp-clean-up suggests "routes to 'clean-up'" \
|
|
"Use when doing the thing. Do not use for the old flow — use the clean-up instead."
|
|
grammar_case fp-built-in suggests "routes to 'built-in'" \
|
|
"Use when doing the thing. Do not use for the custom path — Instead, prefer the built-in."
|
|
grammar_case fp-write-up suggests "routes to 'write-up'" \
|
|
"Use when doing the thing. Do not use for the summary — see the write-up."
|
|
grammar_case fp-front-end suggests "routes to 'front-end'" \
|
|
"Use when doing the thing. Do not use for the API layer — use the front-end."
|
|
grammar_case fp-unit-tests suggests "routes to 'unit-tests'" \
|
|
"Use when testing. Do not run end-to-end, run unit-tests."
|
|
|
|
echo ""
|
|
echo "--- terminal targets that resolve to nothing still ERROR ---"
|
|
# The controls. Without them the cases above are satisfied by a check that never
|
|
# fires, and the narrowing would have eaten the gate rather than sharpened it.
|
|
# Three forms, three code paths:
|
|
# * route NOTATION — `/name` and `-> name` — is exempt from corroboration and
|
|
# blocks on its own. Nobody writes `/pre-commit` or `-> pre-commit` to mean
|
|
# the hook, so there is no ambiguity to resolve, and an author who wants a
|
|
# route checked unconditionally has two ways to say so.
|
|
# * a PROSE-form target — backticked or bare — blocks when its own sentence
|
|
# names another target that resolves. `sibling-skill` is that corroborator
|
|
# here; it is the same shape as both live true positives, which sit beside
|
|
# `write-docs` and `gitea-labels-milestones` respectively.
|
|
grammar_case tp-arrow errors "routes to 'no-such-arrow-target'" \
|
|
"Use when doing the thing. Not the other thing → no-such-arrow-target."
|
|
grammar_case tp-slash errors "routes to 'no-such-slash-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use /no-such-slash-skill instead."
|
|
grammar_case tp-backticked errors "routes to 'no-such-backticked-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use \`sibling-skill\` or \`no-such-backticked-skill\` instead."
|
|
grammar_case tp-bare-terminal errors "routes to 'no-such-bare-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use sibling-skill or no-such-bare-skill instead."
|
|
|
|
echo ""
|
|
echo "--- corroboration is scoped to a REAL sentence, not to whatever the splitter says ---"
|
|
# Corroboration decides SUGGESTION vs blocking ERROR, so a mis-placed sentence
|
|
# boundary moves a target between the two tiers. The naive "period, space,
|
|
# capital" rule got this wrong in both directions, and both were live:
|
|
#
|
|
# OVER-SPLIT. `e.g. "..."` is not a sentence end, but the quote looks like a
|
|
# start. The clause was cut in half and the corroborator stranded on the far
|
|
# side, so a target that DOES sit beside a resolving sibling silently demoted
|
|
# to SUGGESTION — a measurement taken and then discarded.
|
|
grammar_case abbrev-split errors "routes to 'no-such-abbrev-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use sibling-skill first, e.g. \"run the audit\", then use no-such-abbrev-skill instead."
|
|
#
|
|
# UNDER-SPLIT. A sentence opening with a lowercase word or a code span was not
|
|
# seen as a start at all, so two sentences merged and a resolving target in the
|
|
# FIRST vouched for an unresolvable one in the SECOND that it never stood
|
|
# beside — a hard FAIL with no escape hatch, which is the exact failure
|
|
# corroboration was added to prevent. The target must still be REPORTED; only
|
|
# the power to block is withdrawn.
|
|
grammar_case lowercase-start suggests "routes to 'no-such-lower-skill'" \
|
|
"Use when doing the thing. Use sibling-skill for the main case. do not use for improvements — use no-such-lower-skill instead."
|
|
grammar_case backtick-start suggests "routes to 'no-such-tick-skill'" \
|
|
"Use when doing the thing. Use sibling-skill for the main case. \`no-such-tick-skill\` is not for this — do not use it instead."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2a. Route NOTATION always blocks, whatever token follows it
|
|
# ---------------------------------------------------------------------------
|
|
# FOLLOWER_OK is a closed whitelist of about eighty words. A target followed by
|
|
# anything outside it was non-terminal, and `/name` reached _add() with
|
|
# strict=None, so it fell to the follower test and lost the power to block —
|
|
# contradicting the header's own promise that route notation "always blocks",
|
|
# for the one form Claude Code actually uses. Combined with the old `continue`
|
|
# in unresolved_targets(), `use /no-such-skill afterwards.` exited 0 with no
|
|
# output at all: the gate failed OPEN on a word nobody had thought to enumerate.
|
|
#
|
|
# "afterwards" is the probe in every case below. It is ordinary English, it is
|
|
# not in FOLLOWER_OK, and it is not going to be added to it.
|
|
echo ""
|
|
echo "--- route notation blocks even when the following token is outside FOLLOWER_OK ---"
|
|
grammar_case notation-slash-unlisted errors "routes to 'no-such-slash-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use /no-such-slash-skill afterwards."
|
|
grammar_case notation-arrow-unlisted errors "routes to 'no-such-arrow-skill'" \
|
|
"Use when doing the thing. Not the other thing -> no-such-arrow-skill afterwards."
|
|
|
|
echo ""
|
|
echo "--- a target the follower rule cannot vouch for is REPORTED, never invisible ---"
|
|
# The other half of the same defect, and the one that cost visibility rather
|
|
# than enforcement: a PROSE-form target with an unlisted follower may not block
|
|
# (that is what the follower rule is for) but it must still be named. Silence
|
|
# here is the vacuous-green shape the whole script forbids itself.
|
|
grammar_case follower-unlisted-bare suggests "routes to 'no-such-modifier-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use no-such-modifier-skill afterwards."
|
|
grammar_case follower-unlisted-backticked suggests "routes to 'no-such-ticked-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use \`no-such-ticked-skill\` afterwards."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2b. Capitalised abbreviations do not over-split a sentence
|
|
# ---------------------------------------------------------------------------
|
|
# SENTENCE_SPLIT was the one pattern in the resolver built without re.I, so its
|
|
# five abbreviation lookbehinds only covered the lowercase spelling. `E.g.` and
|
|
# `I.e.` — the SENTENCE-INITIAL spellings, which is exactly where an
|
|
# abbreviation lands — matched none of them. The clause split at the
|
|
# abbreviation, the corroborating target was stranded on the far side of the
|
|
# cut, and a genuinely dangling target silently demoted from blocking ERROR to
|
|
# SUGGESTION. The lowercase twin of each case below is `abbrev-split` above and
|
|
# already passed, which is precisely why the gap survived.
|
|
echo ""
|
|
echo "--- a CAPITALISED abbreviation does not strand the corroborator ---"
|
|
grammar_case abbrev-split-caps-eg errors "routes to 'no-such-caps-eg-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use sibling-skill first, E.g. \"run the audit\", then use no-such-caps-eg-skill instead."
|
|
grammar_case abbrev-split-caps-ie errors "routes to 'no-such-caps-ie-skill'" \
|
|
"Use when doing the thing. Do not use for improvements — use sibling-skill first, I.e. \"run the audit\", then use no-such-caps-ie-skill instead."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2c. A boundary clause naming a dotted filename (issue #110)
|
|
# ---------------------------------------------------------------------------
|
|
# `[^.;]` cannot cross the `.` in `AGENTS.md` or `.pre-commit-config.yaml`, so a
|
|
# clause naming a dotted file between "Not" and the arrow was invisible to both
|
|
# BOUNDARY_ARROW and ARROW_BOUNDARY. Two different failures came out of that:
|
|
# with a backticked target the clause was merely MISDIAGNOSED as missing, and
|
|
# with a BARE target it was never extracted at all, so the dangling check
|
|
# silently did not run on it. Both directions are pinned.
|
|
echo ""
|
|
echo "--- a boundary clause naming a dotted filename is seen, and its target is checked ---"
|
|
grammar_case dotted-bare-target errors "routes to 'no-such-dotted-skill'" \
|
|
"Use when doing the thing. Not AGENTS.md -> no-such-dotted-skill."
|
|
grammar_case dotted-clause-seen silent "" \
|
|
"Use when doing the thing. Not .pre-commit-config.yaml -> sibling-skill."
|
|
# The guard that makes the fix a fix and not a hole: a REAL sentence end still
|
|
# ends the clause. A `.` followed by whitespace terminates it exactly as before,
|
|
# so "Not applicable here." plus an arrow two sentences later is not a boundary
|
|
# clause and is still reported as one missing.
|
|
grammar_case dotted-sentence-end-guard suggests "has no boundary clause" \
|
|
"Use when doing the thing. Not applicable here. Reproduce -> minimise."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2d. "Present but unparsed" is a different finding from "missing"
|
|
# ---------------------------------------------------------------------------
|
|
# Issue #110's standing request. An arrow clause ALWAYS names a target, so one
|
|
# that yields none is a parse failure and must say so — telling the author the
|
|
# clause is missing sends them to add a second copy of a clause that is already
|
|
# there. The live shape is a single-word target, which is deliberately not
|
|
# matchable bare because `research`, `triage` and `forge` are all skill names
|
|
# AND ordinary English.
|
|
echo ""
|
|
echo "--- an arrow clause that yields no target is reported as unparsed, not as missing ---"
|
|
grammar_case arrow-single-word-target suggests "no target could be read" \
|
|
"Use when doing the thing. Not the other thing -> forge."
|
|
# Control, so the case above is not satisfied by a check that fires on every
|
|
# arrow clause: the same clause with the target written in a shape the extractor
|
|
# can see produces nothing at all.
|
|
grammar_case arrow-single-word-marked silent "" \
|
|
"Use when doing the thing. Not the other thing -> \`sibling-skill\`."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2e. One arrow, one target (issue #107)
|
|
# ---------------------------------------------------------------------------
|
|
# Only the first target after an arrow is resolved: the conjunction continuation
|
|
# is wired to the prose route verbs and never to arrows. So the second name in
|
|
# `Not X -> a or b` was resolved by nothing and reported by nothing, and the
|
|
# audit then printed "1 of 1 boundary target(s) resolve" on a clause naming two.
|
|
# A typo in the second target shipped through a green gate.
|
|
#
|
|
# The fix rejects the shape rather than widening the extractor. The case below
|
|
# is the exact failure: a bare `Not ... ->` sentence carries no BOUNDARY_MARKER,
|
|
# so the backtick sweep does not run and the second target is genuinely
|
|
# invisible to every other rule in the resolver.
|
|
echo ""
|
|
echo "--- an arrow clause naming two targets is rejected, so the unchecked one is visible ---"
|
|
grammar_case multi-arrow-second-target suggests "names more than one target" \
|
|
"Use when doing the thing. Not the other thing -> \`sibling-skill\` or \`no-such-second-target\`."
|
|
grammar_case multi-arrow-comma suggests "names more than one target" \
|
|
"Use when doing the thing. Not the other thing -> \`sibling-skill\`, \`no-such-comma-target\`."
|
|
# Control: one arrow, one target — the convention the SUGGESTION is asking for —
|
|
# stays silent. Without this the case above is satisfied by a check that fires
|
|
# on every arrow clause in the corpus.
|
|
grammar_case multi-arrow-control silent "" \
|
|
"Use when doing the thing. Not the other thing -> \`sibling-skill\`."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2f. A skill directory with no SKILL.md is not a skill
|
|
# ---------------------------------------------------------------------------
|
|
# _collect_package() added a name for every directory matching skills/*/, with
|
|
# no check that anything was in it. A leftover empty directory — a deleted skill
|
|
# whose directory survived, a scaffolding stub, an editor's stray mkdir — is
|
|
# untracked by git, so it exists on the machine that made it and nowhere else.
|
|
# The hook went green locally and red in a fresh clone: the same
|
|
# install-dependence the deployed-tree rule exists to remove, arriving through a
|
|
# different door. Both directions are asserted, because "never resolve" would
|
|
# also satisfy the first half.
|
|
echo ""
|
|
echo "--- an empty skills/<name>/ directory does not make a routing target resolve ---"
|
|
GHOST="$TMPDIR_T/ghost-dir"
|
|
write_skill "$GHOST/plugins/p/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use /ghost-skill instead."
|
|
mkdir -p "$GHOST/plugins/p/.apm/skills/ghost-skill"
|
|
# NOT wrapped in a helper function: command substitution runs the body in a
|
|
# subshell, so an exit status assigned inside one never reaches the caller —
|
|
# under `set -u` the second read of it aborts the suite.
|
|
set +e
|
|
GHOST_OUT="$(bash "$HOOK" "$GHOST/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
GHOST_RC=$?
|
|
set -e
|
|
if [[ $GHOST_RC -ne 0 && "$GHOST_OUT" == *"routes to 'ghost-skill'"* ]]; then
|
|
pass "a directory with no SKILL.md in it is not a resolvable name"
|
|
else
|
|
fail "an empty skills/ghost-skill/ directory resolved a routing target (exit $GHOST_RC): ${GHOST_OUT:-<empty>}"
|
|
fi
|
|
# The confirming half: drop a SKILL.md into the same directory and the identical
|
|
# description resolves. Without this the rule could be implemented as "skills/
|
|
# never contributes anything" and still pass above.
|
|
write_skill "$GHOST/plugins/p/.apm/skills/ghost-skill" ghost-skill \
|
|
"Use when doing the other thing. Do not use for anything else."
|
|
set +e
|
|
GHOST_OUT="$(bash "$HOOK" "$GHOST/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
GHOST_RC=$?
|
|
set -e
|
|
if [[ $GHOST_RC -eq 0 && -z "$GHOST_OUT" ]]; then
|
|
pass "the same directory WITH a SKILL.md resolves, so the rule is 'no SKILL.md' and not 'never'"
|
|
else
|
|
fail "a populated skills/ghost-skill/ directory still did not resolve (exit $GHOST_RC): ${GHOST_OUT:-<empty>}"
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2g. The FREE-STANDING /name sweep, and its reach beyond a boundary sentence
|
|
# ---------------------------------------------------------------------------
|
|
# NOTATION_SLASH's own sweep in _extract_sentence() is what sees `/name` when no
|
|
# route verb and no arrow precedes it. Nothing pinned it: every `/name` fixture
|
|
# in this suite before these cases ALSO carried a route verb ("use
|
|
# /no-such-slash-skill instead"), which ROUTE_ANY/ROUTE_MARKED extract on their
|
|
# own, so deleting the sweep outright left the whole suite green. The eight
|
|
# phrasings below carry no route verb in front of the target, so each of them is
|
|
# invisible without the sweep — which is exactly the silence the sweep exists to
|
|
# repair, and the shape the SUGGESTION tier's own remedy ("write it as `/name`
|
|
# and it will be checked properly") used to teach an author to produce.
|
|
echo ""
|
|
echo "--- /name with no route verb in front of it is still extracted ---"
|
|
grammar_case sweep-dash errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — /no-such-skill instead."
|
|
grammar_case sweep-semicolon errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y; /no-such-skill handles that."
|
|
grammar_case sweep-paren errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y (/no-such-skill covers it)."
|
|
grammar_case sweep-possessive errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — that is /no-such-skill's job."
|
|
grammar_case sweep-defer errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
|
|
grammar_case sweep-terminal errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — /no-such-skill."
|
|
# The arrow twin. `;` ends CLAUSE_BODY, so ARROW_BOUNDARY cannot reach across it
|
|
# from `not`; only NOTATION_ARROW's own sweep sees this one.
|
|
grammar_case sweep-arrow-after-semicolon errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y; -> no-such-skill covers it."
|
|
|
|
# THE SWEEP IS NOT SCOPED TO A BOUNDARY SENTENCE, and this is the case that
|
|
# proves it. Extraction is per-sentence (corroboration is scoped to one
|
|
# sentence), so gating the `/name` sweep on the sentence carrying a
|
|
# BOUNDARY_MARKER meant a route written one sentence AFTER the boundary clause
|
|
# was never looked at: exit 0, no ERROR, no SUGGESTION, not even the name. That
|
|
# contradicts ADR-0020's amendment and docs/spec/gates.md, which both promise
|
|
# `/name` blocks unconditionally, for any name.
|
|
#
|
|
# The first sentence's `/sibling-skill` is deliberate: it resolves, so the
|
|
# fixture is not "the gate fires on any slash it sees" — it fires on the one
|
|
# that dangles, in the sentence that carries no boundary marker at all.
|
|
echo ""
|
|
echo "--- /name is checked in a sentence that carries no boundary marker ---"
|
|
grammar_case sweep-outside-boundary errors "routes to 'no-such-skill'" \
|
|
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /no-such-skill is the right entry point."
|
|
# Control, so the case above is not satisfied by a gate that fires on every
|
|
# unresolvable-looking token in a non-boundary sentence: the same shape with a
|
|
# name that RESOLVES stays silent.
|
|
grammar_case sweep-outside-boundary-control silent "" \
|
|
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /sibling-skill is the right entry point."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2h. A slash PATH is not a route (the trailing guard, and its backtracking)
|
|
# ---------------------------------------------------------------------------
|
|
# There was no path or URL fixture anywhere in this suite, and the guard was
|
|
# defeated by ordinary regex backtracking. `/(NAME_ANY)\b(?!/|\.\S)` looks like
|
|
# it refuses a path, and does not: when the lookahead rejects the FULL segment
|
|
# the engine backtracks to a shorter hyphen-terminated prefix, `\b` still holds
|
|
# after a hyphen, and the phantom is reported as a hard BLOCKING ERROR naming a
|
|
# skill nobody wrote:
|
|
# /opt-tools/bin/thing -> ERROR: routes to 'opt'
|
|
# /api-docs/v2.md -> ERROR: routes to 'api' AND to 'api-docs'
|
|
# /no-such-skill.md -> ERROR: routes to 'no-such'
|
|
# `(?![\w-])` is the guard that actually holds, because it forbids the shortened
|
|
# prefix instead of merely disliking the full one. MARKED_TARGET carries it too:
|
|
# that pattern had NO trailing lookahead at all, which is where the second
|
|
# 'api-docs' error above came from.
|
|
#
|
|
# These are `silent`, not `suggests`. A path is not a routing target at any
|
|
# tier — reporting one would be the same false positive one notch quieter, on
|
|
# the skills most likely to name a path in a boundary clause.
|
|
echo ""
|
|
echo "--- a slash PATH in a boundary sentence is not a routing target ---"
|
|
grammar_case path-absolute silent "" \
|
|
"Use when doing the thing. Do not use for Y; the config lives at /opt-tools/bin/thing."
|
|
grammar_case path-dotted-file silent "" \
|
|
"Use when doing the thing. Do not use for Y — see /api-docs/v2.md for the schema."
|
|
grammar_case path-dotted-backticked silent "" \
|
|
"Use when doing the thing. Do not use for Y — see \`/api-docs/v2.md\` for the schema."
|
|
grammar_case path-md-suffix silent "" \
|
|
"Use when doing the thing. Do not use for Y — the file /no-such-skill.md documents it."
|
|
# The two suppressions that were already working and must keep working: a URL
|
|
# (the `/` is preceded by a word character or by another `/`) and a relative
|
|
# references/ pointer. Asserted explicitly because the guard above is a change to
|
|
# the same lookarounds, and a fix that traded one silence for another would look
|
|
# identical from the corpus.
|
|
grammar_case path-url silent "" \
|
|
"Use when doing the thing. Do not use for Y — see https://example.com/no-such-skill for details."
|
|
grammar_case path-relative silent "" \
|
|
"Use when doing the thing. Do not use for Y — see references/no-such-skill.md for details."
|
|
# The other direction, which is what stops the guard from becoming a hole: a
|
|
# name whose only follower is the SENTENCE-ENDING dot is still a route. A
|
|
# closing `.` is not followed by a non-space, so `(?!\.\S)` does not reject it.
|
|
# Without these, "refuse every /name near a dot or a slash" would pass every
|
|
# case above and silently delete the notation tier.
|
|
echo ""
|
|
echo "--- the path guard does not swallow a /name at a real sentence end ---"
|
|
grammar_case path-guard-sentence-end errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
|
|
grammar_case path-guard-mid-sentence errors "routes to 'no-such-skill'" \
|
|
"Use when doing the thing. Do not use for Y — use /no-such-skill for that instead."
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# 2i. A DIRECTORY named <something>.md is not an agent
|
|
# ---------------------------------------------------------------------------
|
|
# The skills branch of _collect_package() tests for a SKILL.md; the agents
|
|
# branch takes every `*.md` glob hit on trust, and glob does not distinguish a
|
|
# file from a directory. A leftover directory named `ghost-agent.md` — a botched
|
|
# `mkdir`, an editor's stray save, a half-deleted agent — is untracked by git, so
|
|
# it exists on the machine that made it and nowhere else, and it resolved a
|
|
# routing target there and dangled everywhere else. That is exactly the
|
|
# install-dependence fixture 2f pins one directory over, and the isfile() guard
|
|
# closing it had no test at all: deleting it left every suite green.
|
|
echo ""
|
|
echo "--- an agents/<name>.md DIRECTORY does not make a routing target resolve ---"
|
|
GHOST_AGENT="$TMPDIR_T/ghost-agent-dir"
|
|
write_skill "$GHOST_AGENT/plugins/p/.apm/skills/my-skill" my-skill \
|
|
"Use when doing the thing. Do not use for the other thing — use /ghost-agent instead."
|
|
mkdir -p "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
|
set +e
|
|
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
GHOST_AGENT_RC=$?
|
|
set -e
|
|
if [[ $GHOST_AGENT_RC -ne 0 && "$GHOST_AGENT_OUT" == *"routes to 'ghost-agent'"* ]]; then
|
|
pass "a DIRECTORY named ghost-agent.md is not a resolvable agent name"
|
|
else
|
|
fail "a directory named agents/ghost-agent.md resolved a routing target (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
|
|
fi
|
|
# The confirming half, exactly as in 2f: replace the directory with a real file
|
|
# and the identical description resolves. Without it the rule could be
|
|
# implemented as "agents/ never contributes anything" and still pass above.
|
|
rmdir "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
|
: > "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
|
set +e
|
|
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
|
GHOST_AGENT_RC=$?
|
|
set -e
|
|
if [[ $GHOST_AGENT_RC -eq 0 && -z "$GHOST_AGENT_OUT" ]]; then
|
|
pass "the same path as a FILE resolves, so the rule is 'not a file' and not 'never'"
|
|
else
|
|
fail "a real agents/ghost-agent.md file still did not resolve (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
|
|
fi
|
|
|
|
# And the confirming half of the grammar rule: a compound-modifier target is
|
|
# CONFIRM-ONLY, not ignored. When the name does exist it still counts as a route
|
|
# — the rule suppresses the ERROR, it does not delete the target.
|
|
echo ""
|
|
echo "--- an attributive target that DOES resolve is still a route, not a discarded token ---"
|
|
write_skill "$GRAMMAR_ROOT/plugins/p/.apm/skills/attributive-subject" attributive-subject \
|
|
"Use when doing the thing. Do not use for the other thing — use the sibling-skill helper instead."
|
|
set +e
|
|
ATTR_OUT="$(bash "$HOOK" "$GRAMMAR_ROOT/plugins/p/.apm/skills/attributive-subject/SKILL.md" 2>&1)"
|
|
ATTR_RC=$?
|
|
set -e
|
|
if [[ $ATTR_RC -eq 0 && -z "$ATTR_OUT" ]]; then
|
|
pass "a resolving attributive target neither errors nor is reported"
|
|
else
|
|
fail "an attributive target naming a REAL skill produced output (exit $ATTR_RC): $ATTR_OUT"
|
|
fi
|
|
|
|
echo ""
|
|
echo "Results: $PASS passed, $FAIL failed"
|
|
[[ $FAIL -eq 0 ]]
|