refactor(skills): retrofit the corpus to the ADR-0020 context contract (#129)

Retrofits all 39 skills to ADR-0020's description/body context contract, then fixes what six rounds of independent review found in that retrofit — including four ways the hot gate itself failed open.

Closes #99, #107, #108, #110, #111, #114, #115, #120.

## The retrofit (waves 1-5)

| | Start | Now |
|---|---|---|
| Description FAILs (>400 chars) | 26 | **0** |
| Body FAILs (>900 words, body-only) | 9 | **0** |
| Dangling routing targets | 2 | **0** |
| `Kyberforge.CompositionNote` | 10 | **0** |
| Preload tax | 21,005 chars | **~10,500** |

Under the 12,000-char success criterion. Per-wave detail is on #99.

## The review fixes

**The gate failed open four ways, three of them found after the retrofit shipped.** An unrecognised follower token made a dangling target vanish. A skill directory with no `SKILL.md` resolved as a valid target, so a commit could be green locally and red in a fresh clone — three existing fixtures were relying on that, one of which made the install-leak A/B pass vacuously. Then the free-standing `/name` sweep turned out to be gated on the sentence carrying a boundary marker, so route notation in any other sentence was invisible — not an ERROR, not a SUGGESTION, not an INFO — which left the documented "`/name` always blocks" promise false from a second direction. All four fixed and pinned.

**Two checks were silently not running.** `validate-provenance.sh` checks 7-8 were dead across nine skills. Waking them exposed a deeper problem: they assume `Research doc:` names a source index, but 30 of 121 entries point at topic content documents, so every new check-7 INFO was a false positive and check 8 was saved from a false-FAIL flood only by an *unannounced* skip. Checks 7/8 are now scoped to source indexes and every skip announces itself (#121).

**The retrofit's own anti-goal, four times.** ADR-0020 warns that a blunt gate gets satisfied by deleting content rather than relocating it. `diagnose` and `skill-audit` relocated prose and then read it unconditionally; `prototype` and `vale-config` deleted rules outright that survived nowhere. All four addressed.

## Verification

- `bash tests/run-tests.sh --strict` — 24 suites, 0 skipped, 0 failed
- `bash tests/run-bats.sh` — 325 tests, 0 failures
- `pre-commit run --all-files` — 17/17
- `pre-commit run --hook-stage pre-push --all-files` — 16/16, with `apm marketplace check` and `apm pack --check-clean` run against the remote, not skipped
- `scripts/skill-size-check.sh` over all 39 skills — rc 0, 0 ERROR/FAIL, SUGGESTION-only
- Preload tax measured at **10,498 chars**, max description 390 — both inside budget
- Every new test proven non-vacuous by a deliberate mutation of the behaviour it covers

**Per-commit sync, stated accurately:** the ten commits from the latest review round each pass `check-plugin-content-sync` in isolation, verified by checking each out in a detached worktree with a clean between. The earlier gitea window (`dfacf05..bedbd1d`, nine commits) does **not** — its mirror was regenerated in one batch at `bbc7300`. An earlier revision of this description claimed the property held for every commit; it does not, and a bisect through that window lands on a red commit. **Squash-merge** to collapse it, or accept that this range is not bisectable.

## Version bump

Six plugins and the catalog take a **patch**, not a minor. The branch is **89 commits — 40 `fix` / 30 `refactor` / 12 `docs` / 5 `chore` / 2 `test` — zero `feat`, zero `!`, zero `BREAKING CHANGE`** — and adds no skill, agent, command or hook. (Two earlier revisions of this section cited a stale histogram, most recently 78 commits; the figures above are measured at HEAD.) Both rules this repo ships (`forge/references/version-bump.md`, landing in this PR, and `git-commits/references/conventional-commits-spec.md`) make that a patch, and the catalog set is unchanged at 7 entries.

Not settled by that: four published files were removed from the installed tree, three moved, and `caveman` gained `disable-model-invocation`, retiring its old triggers. Under a strict reading those are major-class and currently ship under `refactor:` with no marker. Whether the deployed skill surface is a public contract is written down nowhere — worth deciding, but it outlives this PR.

## Deliberately not in scope

#112 (cherry-pick ownership, now resolved in favour of `git-commits`), #113 (`rtk git` normalisation), #116 (research fan-out), #101 (audit-skill merge), #122 (non-spec skill-root files), #123 (no PRD producer) stay open. #117 is the one worth reading: the contract's remedy is to move prose into `references/`, which is exactly where neither the size gate nor Vale looks — and the blind spot is wider than #117 currently records, since there is no root `.vale.ini` at all, so every ADR, `CONTEXT.md` and `README.md` is unlinted too.

That blind spot let this branch carry two `level: error` `Kyberforge.SentenceOpenerThereIs` violations into `references/` files it created — `provider-adapter-author/references/provider-matrix.md:31` and `agent-audit/references/finding-criteria.md:95`. Both are reworded in `afadaae`, confirmed by routing each file through the audit's own `vale-wrap.sh` (1 error each before, 0 after). Five further occurrences sit in `references/` files already on `main`; those are the pre-existing corpus and stay with #117, which is the real fix.

Also unfixed and not this PR's: `apm install` appends a duplicate `SessionStart` entry to `.claude/settings.json`, so a fresh clone cannot get pre-push green without an edit AGENTS.md warns against. Reproduces identically on `main`.

Co-authored-by: Defame1297 <gitea@rkdr.net>
Reviewed-on: https://git.dev.rkdr.net/Defame1297/holocron/pulls/129
Co-authored-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net>
Co-committed-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net>
This commit was merged in pull request #129.
This commit is contained in:
Claude Code AI - Gitea MCP
2026-09-01 13:47:46 +00:00
committed by Defame1297
parent 0e91a3ae66
commit 598a7c326a
420 changed files with 15303 additions and 4740 deletions

View File

@@ -1,8 +1,7 @@
#!/usr/bin/env bash
# Regression test for the three STRUCTURAL claims the ADR-0020 gate family makes
# about itself. None of them was pinned anywhere before this file, and each one
# fails silently — which is the whole reason they need a test rather than a
# comment:
# Regression test for the STRUCTURAL claims the ADR-0020 gate family makes about
# itself. None of them was pinned anywhere before this file, and each one fails
# silently — which is the whole reason they need a test rather than a comment:
#
# 1. "ONE resolver, embedded VERBATIM in three scripts." The block between the
# BEGIN/END markers is copied, not imported, because a cache-installed
@@ -11,6 +10,10 @@
# one-line edit to a single copy is invisible: every constant-agreement
# assertion in tests/test-skill-size-check.sh still passes, because the
# CONSTANTS are not what drifted.
# 1b. The same claim, one directory over, for the Contributing-files parser
# embedded in both validate-provenance.sh copies. That one was worse: the
# agent-audit copy's docstring ASSERTED it was kept behaviourally identical
# to skill-audit's, and the two had already drifted.
# 2. Both interpreter preflights, in all three scripts. python3 and PyYAML are
# declared HARD dependencies precisely so a missing one cannot turn into a
# vacuous pass, and the two are checked separately so the message names the
@@ -93,6 +96,69 @@ else
fi
fi
# ---------------------------------------------------------------------------
# 1b. The shared Contributing-files parser is byte-identical in both copies
# ---------------------------------------------------------------------------
# Same defect class, one directory over. parse_contributing_files() is embedded
# in both validate-provenance.sh copies for the same reason the resolver is
# embedded three times, and until this assertion existed the agent-audit copy's
# docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's
# copy" — an invariant nothing checked, and the two had already drifted into
# different spellings of the bullet loop. The parser decides whether checks 4,
# 5 and 8 run at all, so a one-sided edit disables a check in one script while
# every other test stays green.
echo ""
echo "--- the shared Contributing-files parser is byte-identical in both validate-provenance.sh copies ---"
CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ====='
CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER ====='
SKILL_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh"
AGENT_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh"
CF_MARKERS_OK=true
for f in "$SKILL_PROV" "$AGENT_PROV"; do
if [[ ! -f "$f" ]]; then
fail "script not found: $f"
CF_MARKERS_OK=false
continue
fi
b="$(grep -cFx "$CF_BEGIN" "$f" || true)"
e="$(grep -cFx "$CF_END" "$f" || true)"
if [[ "$b" == "1" && "$e" == "1" ]]; then
pass "${f#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker"
else
fail "${f#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1"
CF_MARKERS_OK=false
fi
done
if ! $CF_MARKERS_OK; then
fail "skipping the parser byte-identity comparison — the marker pairs are not well-formed, so any extraction would measure the wrong span"
else
CF_HASHES=()
CF_LINECOUNTS=()
for f in "$SKILL_PROV" "$AGENT_PROV"; do
out="$TMPDIR_T/cfblock-$(echo "$f" | md5sum | cut -c1-8).txt"
sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$f" > "$out"
CF_HASHES+=("$(md5sum < "$out" | cut -d' ' -f1)")
CF_LINECOUNTS+=("$(wc -l < "$out" | tr -d ' ')")
done
if [[ "${CF_HASHES[0]}" == "${CF_HASHES[1]}" ]]; then
pass "both copies hash to ${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines) — agreement by construction, not by coincidence"
else
fail "the shared Contributing-files parser has DRIFTED: skill-audit=${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines), agent-audit=${CF_HASHES[1]} (${CF_LINECOUNTS[1]} lines). Edit one copy, then paste it over the other."
fi
# Two identical EMPTY spans would hash equal and assert nothing, exactly as
# for the resolver above. The parser block is ~93 lines; 40 is a floor low
# enough never to need maintenance and high enough that a gutted block — or
# one reduced to its docstring — cannot sneak past.
if [[ "${CF_LINECOUNTS[0]}" -gt 40 ]]; then
pass "the extracted parser block is ${CF_LINECOUNTS[0]} lines — the comparison is over real content, not an empty span"
else
fail "the extracted parser block is only ${CF_LINECOUNTS[0]} lines — two identical empty spans would compare equal and assert nothing"
fi
fi
# ---------------------------------------------------------------------------
# 2. Both interpreter preflights, in all three scripts
# ---------------------------------------------------------------------------

View File

@@ -92,8 +92,14 @@ build_tree "$TMPDIR_T/no-claude"
build_tree "$TMPDIR_T/with-claude"
# The deployed tree, present only in the second root. Both a skill and an agent,
# because both are valid routing targets and both would leak.
mkdir -p "$TMPDIR_T/with-claude/.claude/skills/deployed-only-skill" \
"$TMPDIR_T/with-claude/.claude/agents"
#
# The skill gets a real SKILL.md. That is not decoration: a directory under
# skills/ is a resolvable name only when it HOLDS one, so an empty directory
# would dangle for the wrong reason and the assertion below would pass without
# testing the deployed-tree rule at all.
mkdir -p "$TMPDIR_T/with-claude/.claude/agents"
write_skill "$TMPDIR_T/with-claude/.claude/skills/deployed-only-skill" deployed-only-skill \
"Use when doing the deployed thing. Do not use for anything else."
: > "$TMPDIR_T/with-claude/.claude/agents/deployed-only-agent.md"
run_subject() {
@@ -133,7 +139,8 @@ fi
echo ""
echo "--- with no authoring root, a deployed .claude/ tree IS the universe ---"
CONSUMER="$TMPDIR_T/consumer"
mkdir -p "$CONSUMER/.claude/skills/deployed-only-skill"
write_skill "$CONSUMER/.claude/skills/deployed-only-skill" deployed-only-skill \
"Use when doing the deployed thing. Do not use for anything else."
write_skill "$CONSUMER/.claude/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use deployed-only-skill instead."
set +e
@@ -362,35 +369,51 @@ cp -R "$REPO_ROOT/plugins" "$FRESH_ROOT/plugins"
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$FRESH_ROOT/apm.yml"
FRESH_DANGLING="$(dangling_set "$FRESH_ROOT/plugins")"
DEPLOYED_ROOT="$TMPDIR_T/deployed-clone"
mkdir -p "$DEPLOYED_ROOT/.claude/skills" "$DEPLOYED_ROOT/.claude/agents"
cp -R "$REPO_ROOT/plugins" "$DEPLOYED_ROOT/plugins"
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$DEPLOYED_ROOT/apm.yml"
# Deploy exactly the names that currently dangle. That is the strongest possible
# bait: if the deployed tree were consulted, every one of them would resolve and
# the dangling set would collapse to empty.
DEPLOY_COUNT=0
while IFS= read -r name; do
[[ -n "$name" ]] || continue
mkdir -p "$DEPLOYED_ROOT/.claude/skills/$name"
DEPLOY_COUNT=$((DEPLOY_COUNT + 1))
done <<< "$FRESH_DANGLING"
DEPLOYED_DANGLING="$(dangling_set "$DEPLOYED_ROOT/plugins")"
# The bait used to be DERIVED from the corpus: deploy exactly the names that
# currently dangle. That was the strongest bait available while the corpus had
# dangling names — and it silently became vacuous the moment issue #99 fixed the
# last one, because a corpus reporting nothing gives nothing to deploy. A test of
# "deployed trees do not leak" must not depend on the corpus staying broken.
#
# So the bait is now EXPLICIT. Both contrast copies get one synthetic skill whose
# boundary clause routes to a name guaranteed absent from the monorepo, and only
# the deployed copy gets that name planted in .claude/skills/. If deployed trees
# leaked into the resolver's universe, the deployed copy would resolve it and
# report an empty set while the fresh copy reported one. The A/B now distinguishes
# something on every run, forever, whatever the corpus does.
BAIT_NAME="no-such-deployed-only-skill"
BAIT_FRESH="$TMPDIR_T/bait-fresh"
BAIT_DEPLOYED="$TMPDIR_T/bait-deployed"
for bait_root in "$BAIT_FRESH" "$BAIT_DEPLOYED"; do
mkdir -p "$bait_root"
cp -R "$REPO_ROOT/plugins" "$bait_root/plugins"
[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$bait_root/apm.yml"
write_skill "$bait_root/plugins/bin/.apm/skills/deployed-tree-probe" deployed-tree-probe \
"Use when doing the probe thing. Do not use for the other thing — use $BAIT_NAME instead."
done
# Only the deployed copy gets the name planted where `apm install` would put it,
# as a REAL skill directory holding a SKILL.md — an empty directory is not a
# resolvable name, so baiting with one would make the A/B pass vacuously.
mkdir -p "$BAIT_DEPLOYED/.claude/agents"
write_skill "$BAIT_DEPLOYED/.claude/skills/$BAIT_NAME" "$BAIT_NAME" \
"Use when doing the bait thing. Do not use for anything else."
BAIT_FRESH_DANGLING="$(dangling_set "$BAIT_FRESH/plugins")"
BAIT_DEPLOYED_DANGLING="$(dangling_set "$BAIT_DEPLOYED/plugins")"
if [[ "$DEPLOY_COUNT" -gt 0 ]]; then
pass "precondition: $DEPLOY_COUNT dangling name(s) deployed into the contrast tree's .claude/skills/, so the A/B has something to distinguish"
if [[ "$BAIT_FRESH_DANGLING" == *"$BAIT_NAME"* ]]; then
pass "precondition: the bait target dangles in the un-deployed copy, so the A/B has something to distinguish"
else
fail "no dangling names to deploy — the corpus reports none, so this A/B distinguishes nothing. Deploy a known-absent name explicitly instead of deriving one."
fail "the bait target '$BAIT_NAME' does not dangle even without a deployed tree — the fixture is broken, so the contrast below proves nothing. Got: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')]"
fi
if [[ ! -d "$FRESH_ROOT/.claude" && ! -d "$FRESH_ROOT/.agents" ]]; then
if [[ ! -d "$BAIT_FRESH/.claude" && ! -d "$BAIT_FRESH/.agents" ]]; then
pass "precondition: the fresh-clone copy has no deployed tree of its own"
else
fail "the fresh-clone copy picked up a deployed tree — it is not a fresh-clone fixture"
fi
if [[ "$FRESH_DANGLING" == "$DEPLOYED_DANGLING" ]]; then
pass "deploying every dangling name into .claude/skills/ changes nothing: $(echo "$FRESH_DANGLING" | tr '\n' ' ')"
if [[ "$BAIT_FRESH_DANGLING" == "$BAIT_DEPLOYED_DANGLING" ]]; then
pass "planting the dangling name in .claude/skills/ changes nothing: $(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')"
else
fail "the corpus verdict depends on whether apm install has been run — fresh clone: [$(echo "$FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$DEPLOYED_DANGLING" | tr '\n' ' ')]"
fail "the verdict depends on whether apm install has been run — fresh clone: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')]. A deployed .claude/skills/ tree is leaking into the resolver's universe."
fi
# Third data point: whatever state THIS machine happens to be in, the live tree
# must agree with a bare copy of the same plugins/. No precondition on that state
@@ -410,41 +433,40 @@ fi
# rather than a "contains" — a false-positive fix that suppressed one of them
# would otherwise land green.
#
# `gitea-labels` is the subtler of the two and is worth keeping: it is not
# written anywhere as `gitea-labels`. gitea-issues' description says "Composes
# `gitea-labels-\n milestones`" in a `>`-folded scalar, and the fold joins the
# lines into "gitea-labels- milestones" — the trailing hyphen is what keeps the
# token terminal and therefore danglable.
# `gitea-labels` WAS the subtler of the two: it was never written anywhere as
# `gitea-labels`. gitea-issues' description said "Composes `gitea-labels-\n
# milestones`" in a `>`-folded scalar, and the fold joined the lines into
# "gitea-labels- milestones" — the trailing hyphen is what kept the token
# terminal and therefore danglable. The issue #99 retrofit cut that composition
# sentence and the dangling target went with it, so the set is down to one.
#
# WHEN ISSUE #100 IS FIXED: update EXPECTED_DANGLING to match. Do not delete the
# assertion — an empty expected set is fine and still pins that no NEW dangling
# target appeared.
# `neuledge-context` was the last one. The issue #99 wave-3 retrofit deleted that
# boundary clause outright — commit `6146120` had already deleted the skill it
# named, and nothing has owned MCP-server installation since — so the corpus
# dangling set is now EMPTY.
#
# The assertion stays, and it is not vacuous now that it expects nothing: it is
# the only thing standing between a newly-authored boundary clause naming a
# non-existent target and a green suite. An empty expected set pins "no NEW
# dangling target appeared", which is the property that actually matters from
# here on.
#
# The per-target probe loop that used to sit below is GONE, not emptied. Its job
# was to prove the check detects each live dangling target individually, and with
# no live targets left there is nothing to point it at. A loop over an empty list
# is an assertion-free result counted in the totals — exactly the vacuous-pass
# shape the comment above rejects. Detection is still covered, and covered
# better, by the synthetic fixtures in section 2 below, which build a real plugin
# tree and assert the resolver fires. Do not reinstate the loop unless a real
# dangling target reappears in the corpus.
echo ""
echo "--- the two live dangling targets in the corpus are exactly the two ADR-0020 records ---"
EXPECTED_DANGLING="$(printf '%s\n' gitea-labels neuledge-context)"
echo "--- no skill in the corpus routes to a target that does not resolve ---"
EXPECTED_DANGLING=""
if [[ "$LIVE_DANGLING" == "$EXPECTED_DANGLING" ]]; then
pass "the corpus dangling set is exactly {gitea-labels, neuledge-context}"
pass "the corpus dangling set is empty"
else
fail "the corpus dangling set changed — expected [$(echo "$EXPECTED_DANGLING" | tr '\n' ' ')], got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. If a retrofit fixed one, update EXPECTED_DANGLING; if a false-positive fix silently deleted one, that is the regression this asserts."
fail "a dangling routing target appeared in the corpus — expected none, got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. A boundary clause names a skill or agent that does not resolve; fix the clause or the target. This assertion is the corpus-wide backstop, so do not relax it to make a new skill pass."
fi
for probe in \
"plugins/bin/.apm/skills/research/SKILL.md:neuledge-context" \
"plugins/gitea/.apm/skills/gitea-issues/SKILL.md:gitea-labels"; do
probe_file="$REPO_ROOT/${probe%%:*}"
probe_name="${probe##*:}"
if [[ ! -f "$probe_file" ]]; then
fail "the true-positive fixture ${probe%%:*} no longer exists — this pin has become vacuous"
continue
fi
set +e
probe_out="$(bash "$HOOK" "$probe_file" 2>&1)"
set -e
if [[ "$probe_out" == *"routes to '$probe_name'"* ]]; then
pass "detects the dangling '$probe_name' target in ${probe%%:*}"
else
fail "did not detect the dangling '$probe_name' target in ${probe%%:*} — a false-positive fix has taken a true positive with it: $probe_out"
fi
done
# ---------------------------------------------------------------------------
# 2. The bare-target grammar rule
@@ -500,13 +522,24 @@ grammar_case() {
# The four phrasings that were hard dangling FAILs with no suppression. All four
# are lifted from real descriptions in this corpus.
grammar_case fp-precommit-hooks silent "" \
#
# THEY ARE `suggests`, NOT `silent`, AND THE DIFFERENCE IS THE POINT. The
# follower rule takes away the power to BLOCK a commit on a compound modifier;
# it does not take away visibility, and it used to. FOLLOWER_OK is a closed
# whitelist of about eighty words, so a non-terminal verdict means "the next
# token is outside a list someone maintains by hand", not "this is prose" — and
# `continue`ing on it made the gate fail OPEN on its own unfamiliarity: any
# target followed by an unlisted word was neither blocked nor mentioned at any
# tier. Asserting silence here pinned that hole in place. The assertion that
# still matters is `!= ERROR`, which `suggests` checks, and which is what keeps
# a false positive from stopping a commit.
grammar_case fp-precommit-hooks suggests "routes to 'pre-commit'" \
"Use when running the linter. Use pre-commit hooks instead of ad-hoc scripts."
grammar_case fp-pull-request silent "" \
grammar_case fp-pull-request suggests "routes to 'pull-request'" \
"Use when opening changes. Invoke the pull-request template instead of writing one by hand."
grammar_case fp-conventional silent "" \
grammar_case fp-conventional suggests "routes to 'conventional-commits'" \
"Use when writing history. Use conventional-commits formatting rather than free-form messages."
grammar_case fp-prepush-backticked silent "" \
grammar_case fp-prepush-backticked suggests "routes to 'pre-push'" \
"Use when checking a branch. Do not use for local edits — run the \`pre-push\` hooks instead."
echo ""
@@ -588,6 +621,309 @@ grammar_case lowercase-start suggests "routes to 'no-such-lower-skill'" \
grammar_case backtick-start suggests "routes to 'no-such-tick-skill'" \
"Use when doing the thing. Use sibling-skill for the main case. \`no-such-tick-skill\` is not for this — do not use it instead."
# ---------------------------------------------------------------------------
# 2a. Route NOTATION always blocks, whatever token follows it
# ---------------------------------------------------------------------------
# FOLLOWER_OK is a closed whitelist of about eighty words. A target followed by
# anything outside it was non-terminal, and `/name` reached _add() with
# strict=None, so it fell to the follower test and lost the power to block —
# contradicting the header's own promise that route notation "always blocks",
# for the one form Claude Code actually uses. Combined with the old `continue`
# in unresolved_targets(), `use /no-such-skill afterwards.` exited 0 with no
# output at all: the gate failed OPEN on a word nobody had thought to enumerate.
#
# "afterwards" is the probe in every case below. It is ordinary English, it is
# not in FOLLOWER_OK, and it is not going to be added to it.
echo ""
echo "--- route notation blocks even when the following token is outside FOLLOWER_OK ---"
grammar_case notation-slash-unlisted errors "routes to 'no-such-slash-skill'" \
"Use when doing the thing. Do not use for improvements — use /no-such-slash-skill afterwards."
grammar_case notation-arrow-unlisted errors "routes to 'no-such-arrow-skill'" \
"Use when doing the thing. Not the other thing -> no-such-arrow-skill afterwards."
echo ""
echo "--- a target the follower rule cannot vouch for is REPORTED, never invisible ---"
# The other half of the same defect, and the one that cost visibility rather
# than enforcement: a PROSE-form target with an unlisted follower may not block
# (that is what the follower rule is for) but it must still be named. Silence
# here is the vacuous-green shape the whole script forbids itself.
grammar_case follower-unlisted-bare suggests "routes to 'no-such-modifier-skill'" \
"Use when doing the thing. Do not use for improvements — use no-such-modifier-skill afterwards."
grammar_case follower-unlisted-backticked suggests "routes to 'no-such-ticked-skill'" \
"Use when doing the thing. Do not use for improvements — use \`no-such-ticked-skill\` afterwards."
# ---------------------------------------------------------------------------
# 2b. Capitalised abbreviations do not over-split a sentence
# ---------------------------------------------------------------------------
# SENTENCE_SPLIT was the one pattern in the resolver built without re.I, so its
# five abbreviation lookbehinds only covered the lowercase spelling. `E.g.` and
# `I.e.` — the SENTENCE-INITIAL spellings, which is exactly where an
# abbreviation lands — matched none of them. The clause split at the
# abbreviation, the corroborating target was stranded on the far side of the
# cut, and a genuinely dangling target silently demoted from blocking ERROR to
# SUGGESTION. The lowercase twin of each case below is `abbrev-split` above and
# already passed, which is precisely why the gap survived.
echo ""
echo "--- a CAPITALISED abbreviation does not strand the corroborator ---"
grammar_case abbrev-split-caps-eg errors "routes to 'no-such-caps-eg-skill'" \
"Use when doing the thing. Do not use for improvements — use sibling-skill first, E.g. \"run the audit\", then use no-such-caps-eg-skill instead."
grammar_case abbrev-split-caps-ie errors "routes to 'no-such-caps-ie-skill'" \
"Use when doing the thing. Do not use for improvements — use sibling-skill first, I.e. \"run the audit\", then use no-such-caps-ie-skill instead."
# ---------------------------------------------------------------------------
# 2c. A boundary clause naming a dotted filename (issue #110)
# ---------------------------------------------------------------------------
# `[^.;]` cannot cross the `.` in `AGENTS.md` or `.pre-commit-config.yaml`, so a
# clause naming a dotted file between "Not" and the arrow was invisible to both
# BOUNDARY_ARROW and ARROW_BOUNDARY. Two different failures came out of that:
# with a backticked target the clause was merely MISDIAGNOSED as missing, and
# with a BARE target it was never extracted at all, so the dangling check
# silently did not run on it. Both directions are pinned.
echo ""
echo "--- a boundary clause naming a dotted filename is seen, and its target is checked ---"
grammar_case dotted-bare-target errors "routes to 'no-such-dotted-skill'" \
"Use when doing the thing. Not AGENTS.md -> no-such-dotted-skill."
grammar_case dotted-clause-seen silent "" \
"Use when doing the thing. Not .pre-commit-config.yaml -> sibling-skill."
# The guard that makes the fix a fix and not a hole: a REAL sentence end still
# ends the clause. A `.` followed by whitespace terminates it exactly as before,
# so "Not applicable here." plus an arrow two sentences later is not a boundary
# clause and is still reported as one missing.
grammar_case dotted-sentence-end-guard suggests "has no boundary clause" \
"Use when doing the thing. Not applicable here. Reproduce -> minimise."
# ---------------------------------------------------------------------------
# 2d. "Present but unparsed" is a different finding from "missing"
# ---------------------------------------------------------------------------
# Issue #110's standing request. An arrow clause ALWAYS names a target, so one
# that yields none is a parse failure and must say so — telling the author the
# clause is missing sends them to add a second copy of a clause that is already
# there. The live shape is a single-word target, which is deliberately not
# matchable bare because `research`, `triage` and `forge` are all skill names
# AND ordinary English.
echo ""
echo "--- an arrow clause that yields no target is reported as unparsed, not as missing ---"
grammar_case arrow-single-word-target suggests "no target could be read" \
"Use when doing the thing. Not the other thing -> forge."
# Control, so the case above is not satisfied by a check that fires on every
# arrow clause: the same clause with the target written in a shape the extractor
# can see produces nothing at all.
grammar_case arrow-single-word-marked silent "" \
"Use when doing the thing. Not the other thing -> \`sibling-skill\`."
# ---------------------------------------------------------------------------
# 2e. One arrow, one target (issue #107)
# ---------------------------------------------------------------------------
# Only the first target after an arrow is resolved: the conjunction continuation
# is wired to the prose route verbs and never to arrows. So the second name in
# `Not X -> a or b` was resolved by nothing and reported by nothing, and the
# audit then printed "1 of 1 boundary target(s) resolve" on a clause naming two.
# A typo in the second target shipped through a green gate.
#
# The fix rejects the shape rather than widening the extractor. The case below
# is the exact failure: a bare `Not ... ->` sentence carries no BOUNDARY_MARKER,
# so the backtick sweep does not run and the second target is genuinely
# invisible to every other rule in the resolver.
echo ""
echo "--- an arrow clause naming two targets is rejected, so the unchecked one is visible ---"
grammar_case multi-arrow-second-target suggests "names more than one target" \
"Use when doing the thing. Not the other thing -> \`sibling-skill\` or \`no-such-second-target\`."
grammar_case multi-arrow-comma suggests "names more than one target" \
"Use when doing the thing. Not the other thing -> \`sibling-skill\`, \`no-such-comma-target\`."
# Control: one arrow, one target — the convention the SUGGESTION is asking for —
# stays silent. Without this the case above is satisfied by a check that fires
# on every arrow clause in the corpus.
grammar_case multi-arrow-control silent "" \
"Use when doing the thing. Not the other thing -> \`sibling-skill\`."
# ---------------------------------------------------------------------------
# 2f. A skill directory with no SKILL.md is not a skill
# ---------------------------------------------------------------------------
# _collect_package() added a name for every directory matching skills/*/, with
# no check that anything was in it. A leftover empty directory — a deleted skill
# whose directory survived, a scaffolding stub, an editor's stray mkdir — is
# untracked by git, so it exists on the machine that made it and nowhere else.
# The hook went green locally and red in a fresh clone: the same
# install-dependence the deployed-tree rule exists to remove, arriving through a
# different door. Both directions are asserted, because "never resolve" would
# also satisfy the first half.
echo ""
echo "--- an empty skills/<name>/ directory does not make a routing target resolve ---"
GHOST="$TMPDIR_T/ghost-dir"
write_skill "$GHOST/plugins/p/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use /ghost-skill instead."
mkdir -p "$GHOST/plugins/p/.apm/skills/ghost-skill"
# NOT wrapped in a helper function: command substitution runs the body in a
# subshell, so an exit status assigned inside one never reaches the caller —
# under `set -u` the second read of it aborts the suite.
set +e
GHOST_OUT="$(bash "$HOOK" "$GHOST/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
GHOST_RC=$?
set -e
if [[ $GHOST_RC -ne 0 && "$GHOST_OUT" == *"routes to 'ghost-skill'"* ]]; then
pass "a directory with no SKILL.md in it is not a resolvable name"
else
fail "an empty skills/ghost-skill/ directory resolved a routing target (exit $GHOST_RC): ${GHOST_OUT:-<empty>}"
fi
# The confirming half: drop a SKILL.md into the same directory and the identical
# description resolves. Without this the rule could be implemented as "skills/
# never contributes anything" and still pass above.
write_skill "$GHOST/plugins/p/.apm/skills/ghost-skill" ghost-skill \
"Use when doing the other thing. Do not use for anything else."
set +e
GHOST_OUT="$(bash "$HOOK" "$GHOST/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
GHOST_RC=$?
set -e
if [[ $GHOST_RC -eq 0 && -z "$GHOST_OUT" ]]; then
pass "the same directory WITH a SKILL.md resolves, so the rule is 'no SKILL.md' and not 'never'"
else
fail "a populated skills/ghost-skill/ directory still did not resolve (exit $GHOST_RC): ${GHOST_OUT:-<empty>}"
fi
# ---------------------------------------------------------------------------
# 2g. The FREE-STANDING /name sweep, and its reach beyond a boundary sentence
# ---------------------------------------------------------------------------
# NOTATION_SLASH's own sweep in _extract_sentence() is what sees `/name` when no
# route verb and no arrow precedes it. Nothing pinned it: every `/name` fixture
# in this suite before these cases ALSO carried a route verb ("use
# /no-such-slash-skill instead"), which ROUTE_ANY/ROUTE_MARKED extract on their
# own, so deleting the sweep outright left the whole suite green. The eight
# phrasings below carry no route verb in front of the target, so each of them is
# invisible without the sweep — which is exactly the silence the sweep exists to
# repair, and the shape the SUGGESTION tier's own remedy ("write it as `/name`
# and it will be checked properly") used to teach an author to produce.
echo ""
echo "--- /name with no route verb in front of it is still extracted ---"
grammar_case sweep-dash errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — /no-such-skill instead."
grammar_case sweep-semicolon errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y; /no-such-skill handles that."
grammar_case sweep-paren errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y (/no-such-skill covers it)."
grammar_case sweep-possessive errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — that is /no-such-skill's job."
grammar_case sweep-defer errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
grammar_case sweep-terminal errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — /no-such-skill."
# The arrow twin. `;` ends CLAUSE_BODY, so ARROW_BOUNDARY cannot reach across it
# from `not`; only NOTATION_ARROW's own sweep sees this one.
grammar_case sweep-arrow-after-semicolon errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y; -> no-such-skill covers it."
# THE SWEEP IS NOT SCOPED TO A BOUNDARY SENTENCE, and this is the case that
# proves it. Extraction is per-sentence (corroboration is scoped to one
# sentence), so gating the `/name` sweep on the sentence carrying a
# BOUNDARY_MARKER meant a route written one sentence AFTER the boundary clause
# was never looked at: exit 0, no ERROR, no SUGGESTION, not even the name. That
# contradicts ADR-0020's amendment and docs/spec/gates.md, which both promise
# `/name` blocks unconditionally, for any name.
#
# The first sentence's `/sibling-skill` is deliberate: it resolves, so the
# fixture is not "the gate fires on any slash it sees" — it fires on the one
# that dangles, in the sentence that carries no boundary marker at all.
echo ""
echo "--- /name is checked in a sentence that carries no boundary marker ---"
grammar_case sweep-outside-boundary errors "routes to 'no-such-skill'" \
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /no-such-skill is the right entry point."
# Control, so the case above is not satisfied by a gate that fires on every
# unresolvable-looking token in a non-boundary sentence: the same shape with a
# name that RESOLVES stays silent.
grammar_case sweep-outside-boundary-control silent "" \
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /sibling-skill is the right entry point."
# ---------------------------------------------------------------------------
# 2h. A slash PATH is not a route (the trailing guard, and its backtracking)
# ---------------------------------------------------------------------------
# There was no path or URL fixture anywhere in this suite, and the guard was
# defeated by ordinary regex backtracking. `/(NAME_ANY)\b(?!/|\.\S)` looks like
# it refuses a path, and does not: when the lookahead rejects the FULL segment
# the engine backtracks to a shorter hyphen-terminated prefix, `\b` still holds
# after a hyphen, and the phantom is reported as a hard BLOCKING ERROR naming a
# skill nobody wrote:
# /opt-tools/bin/thing -> ERROR: routes to 'opt'
# /api-docs/v2.md -> ERROR: routes to 'api' AND to 'api-docs'
# /no-such-skill.md -> ERROR: routes to 'no-such'
# `(?![\w-])` is the guard that actually holds, because it forbids the shortened
# prefix instead of merely disliking the full one. MARKED_TARGET carries it too:
# that pattern had NO trailing lookahead at all, which is where the second
# 'api-docs' error above came from.
#
# These are `silent`, not `suggests`. A path is not a routing target at any
# tier — reporting one would be the same false positive one notch quieter, on
# the skills most likely to name a path in a boundary clause.
echo ""
echo "--- a slash PATH in a boundary sentence is not a routing target ---"
grammar_case path-absolute silent "" \
"Use when doing the thing. Do not use for Y; the config lives at /opt-tools/bin/thing."
grammar_case path-dotted-file silent "" \
"Use when doing the thing. Do not use for Y — see /api-docs/v2.md for the schema."
grammar_case path-dotted-backticked silent "" \
"Use when doing the thing. Do not use for Y — see \`/api-docs/v2.md\` for the schema."
grammar_case path-md-suffix silent "" \
"Use when doing the thing. Do not use for Y — the file /no-such-skill.md documents it."
# The two suppressions that were already working and must keep working: a URL
# (the `/` is preceded by a word character or by another `/`) and a relative
# references/ pointer. Asserted explicitly because the guard above is a change to
# the same lookarounds, and a fix that traded one silence for another would look
# identical from the corpus.
grammar_case path-url silent "" \
"Use when doing the thing. Do not use for Y — see https://example.com/no-such-skill for details."
grammar_case path-relative silent "" \
"Use when doing the thing. Do not use for Y — see references/no-such-skill.md for details."
# The other direction, which is what stops the guard from becoming a hole: a
# name whose only follower is the SENTENCE-ENDING dot is still a route. A
# closing `.` is not followed by a non-space, so `(?!\.\S)` does not reject it.
# Without these, "refuse every /name near a dot or a slash" would pass every
# case above and silently delete the notation tier.
echo ""
echo "--- the path guard does not swallow a /name at a real sentence end ---"
grammar_case path-guard-sentence-end errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
grammar_case path-guard-mid-sentence errors "routes to 'no-such-skill'" \
"Use when doing the thing. Do not use for Y — use /no-such-skill for that instead."
# ---------------------------------------------------------------------------
# 2i. A DIRECTORY named <something>.md is not an agent
# ---------------------------------------------------------------------------
# The skills branch of _collect_package() tests for a SKILL.md; the agents
# branch takes every `*.md` glob hit on trust, and glob does not distinguish a
# file from a directory. A leftover directory named `ghost-agent.md` — a botched
# `mkdir`, an editor's stray save, a half-deleted agent — is untracked by git, so
# it exists on the machine that made it and nowhere else, and it resolved a
# routing target there and dangled everywhere else. That is exactly the
# install-dependence fixture 2f pins one directory over, and the isfile() guard
# closing it had no test at all: deleting it left every suite green.
echo ""
echo "--- an agents/<name>.md DIRECTORY does not make a routing target resolve ---"
GHOST_AGENT="$TMPDIR_T/ghost-agent-dir"
write_skill "$GHOST_AGENT/plugins/p/.apm/skills/my-skill" my-skill \
"Use when doing the thing. Do not use for the other thing — use /ghost-agent instead."
mkdir -p "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
set +e
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
GHOST_AGENT_RC=$?
set -e
if [[ $GHOST_AGENT_RC -ne 0 && "$GHOST_AGENT_OUT" == *"routes to 'ghost-agent'"* ]]; then
pass "a DIRECTORY named ghost-agent.md is not a resolvable agent name"
else
fail "a directory named agents/ghost-agent.md resolved a routing target (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
fi
# The confirming half, exactly as in 2f: replace the directory with a real file
# and the identical description resolves. Without it the rule could be
# implemented as "agents/ never contributes anything" and still pass above.
rmdir "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
: > "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
set +e
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
GHOST_AGENT_RC=$?
set -e
if [[ $GHOST_AGENT_RC -eq 0 && -z "$GHOST_AGENT_OUT" ]]; then
pass "the same path as a FILE resolves, so the rule is 'not a file' and not 'never'"
else
fail "a real agents/ghost-agent.md file still did not resolve (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
fi
# And the confirming half of the grammar rule: a compound-modifier target is
# CONFIRM-ONLY, not ignored. When the name does exist it still counts as a route
# — the rule suppresses the ERROR, it does not delete the target.

View File

@@ -331,13 +331,32 @@ PY
#
# The sibling plugin is what makes "every plugin in the monorepo contributes its
# names" testable; without it a cross-plugin target and a typo are the same.
#
# Both sibling skill directories get a real SKILL.md, and that is load-bearing
# rather than tidiness: a skill directory is a resolvable name only if it HOLDS
# a SKILL.md. An empty leftover directory is untracked by git, so counting one
# made a target resolve on the machine that made it and dangle in a fresh clone
# — the same install-dependence the deployed-tree rule exists to remove. This
# fixture used to `mkdir` the two siblings and write nothing into them, so it
# was itself relying on the behaviour the resolver no longer has.
make_tree_fixture() {
local label="$1" desc="$2" body_words="$3" root apm
local label="$1" desc="$2" body_words="$3" root apm sib
root="$TMPDIR/tree-$label"
apm="$root/plugins/subject-plugin/.apm"
mkdir -p "$apm/skills/$label" "$apm/skills/sibling-skill" "$apm/agents" \
"$root/plugins/other-plugin/.apm/skills/cross-plugin-skill"
: > "$apm/agents/sibling-agent.agent.md"
for sib in "$apm/skills/sibling-skill" \
"$root/plugins/other-plugin/.apm/skills/cross-plugin-skill"; do
{
echo "---"
echo "name: $(basename "$sib")"
echo "description: Use when doing the other thing. Do not use for anything else."
echo "---"
echo ""
echo "Do the thing."
} > "$sib/SKILL.md"
done
{
echo "---"
echo "name: $label"
@@ -364,8 +383,17 @@ expect_gate() {
fail "$label (exit $status, output: ${out:-<empty>})"
fi
;;
# The tier and the needle are matched ADJACENTLY — `*"SUGGESTION"*"$needle"*`
# — not as two independent substring tests. Independently, any output
# carrying a SUGGESTION anywhere and the needle anywhere satisfied the
# assertion, so a needle emitted at the WRONG TIER still passed: a finding
# that moved from SUGGESTION to a blocking ERROR line would be caught only
# by the exit-status test, and one that moved from SUGGESTION to INFO would
# not be caught at all. grammar_case's `suggests` branch in
# tests/test-adr0020-targets.sh has always matched them adjacently; this is
# the same rule.
suggest)
if [[ $status -eq 0 && "$out" == *"SUGGESTION"* && "$out" == *"$needle"* ]]; then
if [[ $status -eq 0 && "$out" == *"SUGGESTION"*"$needle"* ]]; then
pass "$label"
else
fail "$label (exit $status, output: ${out:-<empty>})"
@@ -450,9 +478,14 @@ expect_gate "body at $((BODY_MAX_WORDS + 1)) words fails" \
echo ""
echo "--- the body gate and the whole-file gate are independent measurements ---"
BODY_ONLY_DESC="$(python3 -c "print(' '.join(['w'] * 100))")"
# The needle pins the COUNT, not the bare word "words". "words" appears in the
# whole-file ceiling message, in the body ceiling message and in the body target
# message alike, so it was satisfied by any of the three — including the one
# this case exists to prove does NOT fire. Naming the number is what makes the
# assertion about the body-only measurement.
expect_gate "frontmatter words do not count toward the $BODY_MAX_WORDS-word body ceiling" \
suggest "$(make_budget_fixture body-independent "$BODY_ONLY_DESC" "$((BODY_MAX_WORDS - 5))")" \
"words"
"body is $((BODY_MAX_WORDS - 5)) words"
BIG_BODY="$(make_budget_fixture body-over-not-whole-file "$CLEAN_DESC" "$((BODY_MAX_WORDS + 1))")"
BIG_BODY_WORDS="$(wc -w < "$BIG_BODY")"
if [[ "$BIG_BODY_WORDS" -le "$MAX_WORDS" ]]; then
@@ -461,6 +494,96 @@ else
fail "the body-gate fixture is $BIG_BODY_WORDS whole-file words, which also trips MAX_WORDS=$MAX_WORDS — the test no longer isolates the body gate"
fi
# ---------------------------------------------------------------------------
# ADR-0020's hand-invocation carve-out (issue #108)
# ---------------------------------------------------------------------------
# A skill carrying `disable-model-invocation: true` is removed from the
# model-visible listing entirely — it is not preloaded, and the Skill tool
# refuses to call it — so its description is never matched against user intent.
# ADR-0020, skill-author Step 2 and skill-audit's own Step 0 all give it ONE
# plain human-facing sentence: no trigger list, no boundary clause. No validator
# knew the field existed, so the boundary-clause SUGGESTION fired on exactly the
# shape the contract mandates, and its remedy — "so the router knows where NOT
# to send this skill" — named a router that cannot see the skill at all.
#
# The carve-out is NARROW and the half it does not cover is the half worth
# testing: the body is still loaded on invocation, so the body budget stands,
# and the 400-character ceiling stands because it is an outlier stop rather than
# a routing-quality target. Every case below asserts one of those two halves.
make_hand_invoked_fixture() {
local name="$1" desc="$2" body_words="$3" file
file="$TMPDIR/$name.md"
{
echo "---"
echo "name: $name"
echo "description: $desc"
echo "disable-model-invocation: true"
echo "---"
echo ""
python3 -c "print(' '.join(['word'] * $body_words))"
} > "$file"
echo "$file"
}
# A description over the 250-character target, carrying no boundary clause and
# no routing target — the exact shape `zoom-out` and `caveman` ship. Built with
# no hyphens so nothing in it reads as a target.
HAND_DESC="$(python3 -c "
prefix = 'Tell the agent to zoom out and give broader context. '
print(prefix + 'x' * (300 - len(prefix)))")"
echo ""
echo "--- a hand-invoked skill is exempt from the routing rules, and only those ---"
expect_gate "a hand-invoked skill with a 300-char description and no boundary clause is silent" \
pass "$(make_hand_invoked_fixture hand-quiet "$HAND_DESC" 10)"
# The control that makes the case above mean something. Same description, same
# body, only the frontmatter flag removed: both findings must appear, or the
# exemption is being credited for silence it did not cause.
expect_gate "control: the SAME description without the flag is over the 250-char target" \
suggest "$(make_budget_fixture hand-control "$HAND_DESC" 10)" \
"description is 300 characters"
expect_gate "control: the SAME description without the flag has no boundary clause" \
suggest "$(make_budget_fixture hand-control "$HAND_DESC" 10)" \
"has no boundary clause"
echo ""
echo "--- the carve-out lifts the routing rules ONLY: both size gates still bite ---"
# The description ceiling is not a routing budget: a hand-invoked description is
# still the one line a human reads in the `/` menu, and 400 characters is the
# outlier stop either way.
HAND_OVER_MAX="$(python3 -c "
prefix = 'Tell the agent to zoom out and give broader context. '
print(prefix + 'x' * (401 - len(prefix)))")"
expect_gate "a hand-invoked description over $DESC_MAX_CHARS chars still FAILS" \
fail "$(make_hand_invoked_fixture hand-over-max "$HAND_OVER_MAX" 10)" \
"$DESC_MAX_CHARS-character ceiling"
# The body is loaded on invocation like any other body and competes with the
# caller's live conversation exactly the same way, so neither body tier moves.
expect_gate "a hand-invoked body over $BODY_MAX_WORDS words still FAILS" \
fail "$(make_hand_invoked_fixture hand-over-body "$HAND_DESC" "$((BODY_MAX_WORDS + 1))")" \
"$BODY_MAX_WORDS-word ceiling"
expect_gate "a hand-invoked body over $BODY_SUGGEST_WORDS words is still suggested" \
suggest "$(make_hand_invoked_fixture hand-over-body-suggest "$HAND_DESC" "$((BODY_SUGGEST_WORDS + 1))")" \
"body is $((BODY_SUGGEST_WORDS + 1)) words"
echo ""
echo "--- the flag is read as a BOOLEAN, not as any mention of the key ---"
# `disable-model-invocation: false` is the model-invoked case written out
# longhand. Reading the key's presence instead of its value would hand every
# routing exemption to anyone who typed the field at all.
HAND_FALSE="$TMPDIR/hand-false.md"
{
echo "---"
echo "name: hand-false"
echo "description: $HAND_DESC"
echo "disable-model-invocation: false"
echo "---"
echo ""
echo "Do the thing."
} > "$HAND_FALSE"
expect_gate "disable-model-invocation: false is NOT the carve-out" \
suggest "$HAND_FALSE" "has no boundary clause"
echo ""
echo "--- resolvable boundary targets ---"
# Resolution is against the AUTHORING SOURCE (plugins/*/.apm/skills/ and
@@ -534,10 +657,12 @@ expect_gate "a fixture with no authoring root reports DID NOT RUN and exits 0" \
"Use when doing the thing. Do not use for improvements — use some-other-skill instead." 10)" \
"Unchecked target(s): some-other-skill"
echo ""
echo "--- the live dangling routing targets are caught (issue #100) ---"
# NOTE: this section prints no header and runs no assertions any more — see why
# below. The commentary is kept because it records why probes are removed rather
# than skipped, which is the rule the next person to touch this file needs.
#
# ADR-0020 records the broken routing targets and splits fixing them into its own
# issue. This asserts the gate actually sees them rather than the check being
# issue. This asserted the gate actually sees them rather than the check being
# vacuous in the corpus it was written against.
#
# There used to be a third probe here, for `skill-improve` in skill-audit's
@@ -546,34 +671,205 @@ echo "--- the live dangling routing targets are caught (issue #100) ---"
# in the totals, which is worse than no probe at all because it makes the suite
# look one test stronger than it is. It also contradicted
# tests/test-adr0020-targets.sh, which pins the live dangling set as EXACTLY
# {gitea-labels, neuledge-context}; that file is the authority on the set, this
# one only checks the two are individually detected.
# {neuledge-context}; that file is the authority on the set, this one only
# checks each member is individually detected.
#
# Both SKIP branches are gone with it, for the same reason. A probe whose fixture
# has been retrofitted is not "still passing" — it is a pin that needs updating,
# here and in the exact-set assertion in test-adr0020-targets.sh, and it should
# say so out loud rather than quietly agreeing with whatever it finds.
for probe in \
"plugins/bin/.apm/skills/research/SKILL.md:neuledge-context" \
"plugins/gitea/.apm/skills/gitea-issues/SKILL.md:gitea-labels"; do
probe_file="$REPO_ROOT/${probe%%:*}"
probe_name="${probe##*:}"
if [[ ! -f "$probe_file" ]]; then
fail "the probe fixture ${probe%%:*} no longer exists — this pin has become vacuous; update it and EXPECTED_DANGLING in tests/test-adr0020-targets.sh together"
continue
fi
# Captured, not piped: the script exits non-zero on these files and
# `set -o pipefail` would make the whole pipeline non-zero regardless of what
# grep found.
# The gitea-labels probe was dropped when the issue #99 retrofit cut the
# composition sentence whose YAML fold produced that target. Per the rule above
# it is removed, not skipped.
#
# The `neuledge-context` probe — the last one — went the same way in wave 3 of
# that retrofit, which deleted the boundary clause naming it. **The corpus now
# has zero dangling targets**, so this loop is removed entirely rather than left
# to iterate over an empty list.
#
# That is deliberate and follows the rule stated above. A loop over no probes
# produces no assertion while still returning success, which is the vacuous-pass
# shape this comment block exists to reject — it would make the suite look one
# test stronger than it is, exactly the complaint levelled at the old
# `skill-improve` SKIP branch.
#
# Nothing is lost. This file only ever checked that each member of the live
# dangling set is individually detected; tests/test-adr0020-targets.sh remains
# the authority on the set itself, and now pins it as EMPTY, which is what
# catches a newly-authored clause naming a target that does not resolve. That
# file also carries synthetic fixtures built inside a real plugin tree, which
# exercise the detection path without depending on the corpus staying broken.
#
# If a real dangling target ever reappears, add its probe back here.
# ---------------------------------------------------------------------------
# Usage tier: zero arguments is exit 2, not a clean run
# ---------------------------------------------------------------------------
# The script used to print nothing and exit 0 when handed no paths, which made
# a mis-scoped `files:` pattern indistinguishable from a corpus with no
# findings — the whole ADR-0020 gate family silently disabled while every hook
# reported green. Exit 2 (not 1) is the same split a8cd5e8 made in
# provider-adapter-author's validate-adapter.sh and the one vale-wrap.sh already
# used: {0,1} are verdicts, 2 is "you invoked this wrong".
#
# SAFE FOR THE HOOK. Both manifests declare pass_filenames: true and neither
# sets always_run, and pre-commit skips a filename-passing hook outright when
# its `files:` pattern matches nothing, so pre-commit never invokes this script
# with an empty argument list. That claim is asserted below rather than left in
# prose, so a config edit that turns it false fails here.
echo ""
echo "--- zero arguments is a usage error (exit 2), not a silent clean run ---"
set +e
USAGE_OUT="$("$SCRIPT" 2>&1)"
USAGE_RC=$?
set -e
if [[ $USAGE_RC -eq 2 ]]; then
pass "no arguments exits 2"
else
fail "no arguments exited $USAGE_RC, expected 2 (output: ${USAGE_OUT:-<empty>})"
fi
if [[ "$USAGE_OUT" == *usage* ]]; then
pass "no arguments prints a usage message"
else
fail "no arguments produced no usage message (output: ${USAGE_OUT:-<empty>})"
fi
# The exit code must be DISTINCT from both verdicts, or the split buys nothing.
# $SMALL is the clean fixture built at the top of this file; $MANY_LINES is over
# the line ceiling.
set +e
"$SCRIPT" "$SMALL" > /dev/null 2>&1
CLEAN_RC=$?
"$SCRIPT" "$MANY_LINES" > /dev/null 2>&1
FINDING_RC=$?
set -e
if [[ $CLEAN_RC -eq 0 && $FINDING_RC -eq 1 && $USAGE_RC -eq 2 ]]; then
pass "the three exit codes are distinct: clean=0, findings=1, usage=2"
else
fail "exit codes collide — clean=$CLEAN_RC findings=$FINDING_RC usage=$USAGE_RC"
fi
# The hook contract the usage exit depends on. If either manifest ever stops
# passing filenames, or starts always_run, pre-commit could invoke the script
# with no paths and exit 2 would break the hook rather than diagnose a caller.
HOOK_CONTRACT="$(python3 - "$REPO_ROOT" <<'PYHOOK'
import os
import sys
import yaml
root = sys.argv[1]
problems = []
def check(label, hook):
if hook is None:
problems.append('%s declares no such hook' % label)
return
if hook.get('pass_filenames') is False:
problems.append('%s sets pass_filenames: false' % label)
if hook.get('always_run'):
problems.append('%s sets always_run: true' % label)
with open(os.path.join(root, '.pre-commit-config.yaml'), encoding='utf-8') as fh:
cfg = yaml.safe_load(fh) or {}
found = None
for repo in cfg.get('repos') or []:
for hook in (repo.get('hooks') or []):
if hook.get('id') == 'skill-size-check':
found = hook
check('.pre-commit-config.yaml skill-size-check', found)
with open(os.path.join(root, '.pre-commit-hooks.yaml'), encoding='utf-8') as fh:
hooks = yaml.safe_load(fh) or []
found = None
for hook in hooks:
if isinstance(hook, dict) and hook.get('id') == 'kyberforge-skill-size-check':
found = hook
check('.pre-commit-hooks.yaml kyberforge-skill-size-check', found)
print('; '.join(problems))
PYHOOK
)"
if [[ -z "$HOOK_CONTRACT" ]]; then
pass "both manifests pass filenames and neither is always_run, so pre-commit never invokes the script with no paths"
else
fail "the usage exit would break the hook: $HOOK_CONTRACT"
fi
# ---------------------------------------------------------------------------
# An unreadable path is diagnosed ONCE
# ---------------------------------------------------------------------------
# The stat dance lived twice — a bash pre-loop and the Python per-file loop —
# and both printed the same sentence, so one broken file produced two ERROR
# lines with two different "so ... could not be measured" clauses. Duplicated
# output on a blocking gate reads as two problems and sends the author hunting
# for a second one. The check must still FIRE (silence is the failure this
# script forbids itself); it must fire exactly once.
echo ""
echo "--- an unreadable path produces exactly one ERROR line, not two ---"
UNREADABLE_DIR="$TMPDIR/unreadable"
mkdir -p "$UNREADABLE_DIR/a-directory.md"
ln -sf "$TMPDIR/definitely-not-here.md" "$UNREADABLE_DIR/broken-link.md"
# unreadable_case <label> <path>
unreadable_case() {
local label="$1" path="$2" out status=0 count
set +e
probe_out="$("$SCRIPT" "$probe_file" 2>&1)"
out="$("$SCRIPT" "$path" 2>&1)"
status=$?
set -e
if [[ "$probe_out" == *"routes to '$probe_name'"* ]]; then
pass "detects the dangling '$probe_name' target in ${probe%%:*}"
count="$(printf '%s\n' "$out" | grep -cF "ERROR: $path" || true)"
if [[ $status -eq 0 ]]; then
fail "$label: exited 0 — an unmeasurable path passed in silence (output: ${out:-<empty>})"
elif [[ "$count" != "1" ]]; then
fail "$label: $count ERROR lines name the path, expected exactly 1 (output: $out)"
else
fail "did not detect the dangling '$probe_name' target in ${probe%%:*}. If issue #100 retrofitted it, drop this probe and update EXPECTED_DANGLING in tests/test-adr0020-targets.sh; if a false-positive fix took a true positive with it, that is the regression this asserts."
pass "$label"
fi
done
}
unreadable_case "a path that does not exist is reported once" \
"$TMPDIR/no-such-file.md"
unreadable_case "a DIRECTORY named *.md is reported once" \
"$UNREADABLE_DIR/a-directory.md"
unreadable_case "a broken symlink is reported once" \
"$UNREADABLE_DIR/broken-link.md"
# ---------------------------------------------------------------------------
# Encoding, write side: under LC_ALL=C the report must still print
# ---------------------------------------------------------------------------
# read_text() in the shared ADR-0020 resolver block pins the READS to UTF-8.
# That moved the crash to the WRITE: this script's own message text carries em
# dashes (the boundary SUGGESTION is one), so under LC_ALL=C the streams' ASCII
# default raised UnicodeEncodeError while PRINTING -- after every check had
# already run, losing the whole report at the last step and turning a
# SUGGESTION-only exit 0 into a traceback and an exit 1.
echo ""
echo "--- under LC_ALL=C the SUGGESTION is printed, not lost to a UnicodeEncodeError ---"
LOCALE_SKILL="$TMPDIR/locale-skill"
mkdir -p "$LOCALE_SKILL"
cat > "$LOCALE_SKILL/SKILL.md" <<'LOCALEEOF'
---
name: locale-skill
description: A valid skill description that is well within the limit.
---
## Step 1
Do the thing.
LOCALEEOF
set +e
LOCALE_OUT="$(env LC_ALL=C PYTHONUTF8=0 "$SCRIPT" "$LOCALE_SKILL/SKILL.md" 2>&1)"
LOCALE_STATUS=$?
set -e
if [[ $LOCALE_STATUS -ne 0 ]]; then
fail "a SUGGESTION-only subject exited $LOCALE_STATUS under LC_ALL=C (output: ${LOCALE_OUT:-<empty>})"
elif [[ "$LOCALE_OUT" == *UnicodeEncodeError* || "$LOCALE_OUT" == *Traceback* ]]; then
fail "the report died encoding its own message text under LC_ALL=C (output: $LOCALE_OUT)"
elif [[ "$LOCALE_OUT" != *"description has no boundary clause"* ]]; then
fail "the SUGGESTION never reached stdout under LC_ALL=C (output: ${LOCALE_OUT:-<empty>})"
else
pass "the SUGGESTION survives LC_ALL=C, streams pinned to UTF-8"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"