test: pin the nine ADR-0020 gate defects that shipped untested
Every defect fixed in f7cc279 was reachable because nothing asserted against it.
The gate had 43 assertions and none of them covered a consumer repo, a non-string
description, an unclosed fence, or the two spec ceilings. Each case below fails
against the pre-fix code and passes against the current one; every one was proved
non-vacuous by mutating a scratch copy of the script and watching the test go red,
independently twice.
The two that mattered most had no fixture anywhere. A consumer repo WITH .git is
the shape the resolver exists to serve, and only the no-.git case had ever been
tested, which is exactly why the blocker was invisible. And ADR-0020 says the
walk-up runs in two passes specifically so a nested .git cannot beat a plugins/
root further up — no fixture had ever placed a .git inside a plugin.
test-adr0020-differential.sh loses _non_adr_hook_error(). It excluded MAX_LINES and
MAX_WORDS from the cross-script comparison on the untested assumption that awk and
splitlines() agree. They do not, and the divergence stayed invisible for exactly as
long as the exclusion stood. The ceilings are now compared like any other rule.
Two existing assertions were repairs, not additions. The skill-improve probe had
been fixed by this very branch, so its iteration permanently took an
assertion-free SKIP that still counted as a pass; both branches now fail loudly and
each names the other file's pin so the two stay in step. And the yaml-none fixture
emitted `---/---`, which never matched the frontmatter pattern at all — it passed on
the bare word "frontmatter", present in both messages, while never reaching the
branch it was named for. Needles throughout that file now name their branch.
Refs: #99
ADR: 0020
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015W3iwF9ncfRZddGBxsMCYi
This commit is contained in:
@@ -13,6 +13,14 @@
|
||||
# identical with and without a deployed tree — on a synthetic fixture AND on
|
||||
# the real 39-skill corpus.
|
||||
#
|
||||
# Three further ways the universe can be built out of the wrong directory,
|
||||
# each of which shipped: a `.git` at the CONSUMER root (the fallback is
|
||||
# truthy in any git repo, which made the deployed-tree branch dead code), a
|
||||
# `.git` INSIDE a plugin (the walk-up is two passes precisely so this cannot
|
||||
# capture the root), and glob metacharacters in the checkout path (which
|
||||
# turned the directory name into a character class matching nothing, and the
|
||||
# resolver into a no-op that still reported green).
|
||||
#
|
||||
# 2. THE BARE-TARGET GRAMMAR RULE. A hyphenated token used as a compound
|
||||
# MODIFIER ("pre-commit hooks", "pull-request template") is prose, not a
|
||||
# route; a terminal one is a real target. Getting this wrong in either
|
||||
@@ -138,6 +146,175 @@ else
|
||||
fail "the consumer path did not resolve through the deployed tree (exit $CONSUMER_RC): ${CONSUMER_OUT:-<empty>}"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1a-bis. The consumer case with the one thing every real consumer has: .git
|
||||
# ---------------------------------------------------------------------------
|
||||
# The fixture immediately above has no .git, and that is precisely why it could
|
||||
# never catch this. _authoring_root() falls back to the nearest .git ancestor, so
|
||||
# it returns truthy in ANY git repo — a consumer checkout included. The branch
|
||||
# that reads the deployed trees was guarded by `else`, so in every consumer
|
||||
# checkout the fallback won, _collect_authoring_root() contributed nothing
|
||||
# (there is no plugins/ directory to collect), and _deployed_roots() was dead
|
||||
# code in exactly the case it exists for.
|
||||
#
|
||||
# The pair below is the whole test: the SAME tree, once with .git and once
|
||||
# without. Old behaviour was rc=1 with .git and rc=0 without; a test covering
|
||||
# only the no-.git shape reports green on both.
|
||||
#
|
||||
# `deployed-only-agent` lives ONLY in .agents/agents/, so it can be reached
|
||||
# through no route but _deployed_roots(). `sibling-skill` sits in .claude/skills/
|
||||
# beside the subject, which the sibling-collection block above reaches on its own
|
||||
# — it is the corroborator that makes the dangling target BLOCKING rather than a
|
||||
# SUGGESTION, so the old failure shows up in the exit code and not only in prose.
|
||||
echo ""
|
||||
echo "--- a consumer checkout resolves through its deployed trees even though it is a git repo ---"
|
||||
build_consumer() {
|
||||
local root="$1"
|
||||
mkdir -p "$root/.agents/agents"
|
||||
write_skill "$root/.claude/skills/sibling-skill" sibling-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
write_skill "$root/.claude/skills/my-skill" my-skill \
|
||||
"Use when doing the thing. Do not use for the other thing — use sibling-skill or deployed-only-agent instead."
|
||||
: > "$root/.agents/agents/deployed-only-agent.agent.md"
|
||||
}
|
||||
build_consumer "$TMPDIR_T/consumer-git"
|
||||
mkdir -p "$TMPDIR_T/consumer-git/.git"
|
||||
build_consumer "$TMPDIR_T/consumer-nogit"
|
||||
|
||||
# consumer_case <label> <root>
|
||||
consumer_case() {
|
||||
local label="$1" root="$2" out status=0
|
||||
set +e
|
||||
out="$(bash "$HOOK" "$root/.claude/skills/my-skill/SKILL.md" 2>&1)"
|
||||
status=$?
|
||||
set -e
|
||||
if [[ $status -eq 0 && "$out" != *"routes to"* && "$out" != *"DID NOT RUN"* ]]; then
|
||||
pass "$label"
|
||||
else
|
||||
fail "$label (exit $status, output: ${out:-<empty>})"
|
||||
fi
|
||||
}
|
||||
consumer_case "an agent in .agents/agents/ resolves in a consumer checkout that HAS a .git directory" \
|
||||
"$TMPDIR_T/consumer-git"
|
||||
consumer_case "control: the same tree without .git resolves too (the shape that always passed)" \
|
||||
"$TMPDIR_T/consumer-nogit"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1a-ter. A monorepo with ONE plugin is still a monorepo
|
||||
# ---------------------------------------------------------------------------
|
||||
# The first attempt at the fix above conditioned the deployed branch on whether
|
||||
# the authoring root had CONTRIBUTED a name — `if len(names) == before:`. That
|
||||
# reads as "the .git fallback collected nothing, so fall through", and it is
|
||||
# wrong: _collect_authoring_root() re-collects the subject's OWN plugin, whose
|
||||
# names the sibling and package blocks have already added. With two plugins
|
||||
# (fixture 1) the cross-plugin name makes the delta non-zero and the guard stays
|
||||
# shut. With ONE plugin the delta is zero, the guard fires in a genuine
|
||||
# monorepo, and _deployed_roots() walks up to ten levels — reaching the user's
|
||||
# global ~/.claude/skills. That is install-dependence again, in the shape
|
||||
# ADR-0020 lines 118-127 exist to forbid.
|
||||
#
|
||||
# So the predicate is which PROBE matched, not how many names arrived. The
|
||||
# assertion is the same shape as fixture 1 — identical verdict either way — but
|
||||
# on a single-plugin tree, which fixture 1 cannot express.
|
||||
echo ""
|
||||
echo "--- a SINGLE-plugin monorepo does not fall through to the deployed trees ---"
|
||||
build_single() {
|
||||
local root="$1"
|
||||
write_skill "$root/plugins/only-plugin/.apm/skills/my-skill" my-skill \
|
||||
"Use when doing the thing. Do not use for the other thing — use /deployed-only-skill instead."
|
||||
}
|
||||
build_single "$TMPDIR_T/single-no-claude"
|
||||
build_single "$TMPDIR_T/single-with-claude"
|
||||
write_skill "$TMPDIR_T/single-with-claude/.claude/skills/deployed-only-skill" deployed-only-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
|
||||
run_single() {
|
||||
local root="$1" out
|
||||
set +e
|
||||
out="$(bash "$HOOK" "$root/plugins/only-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
||||
set -e
|
||||
printf '%s\n' "$out" | sed "s#$root#<ROOT>#g"
|
||||
}
|
||||
SINGLE_NO_OUT="$(run_single "$TMPDIR_T/single-no-claude")"
|
||||
SINGLE_WITH_OUT="$(run_single "$TMPDIR_T/single-with-claude")"
|
||||
|
||||
if [[ "$SINGLE_NO_OUT" == "$SINGLE_WITH_OUT" ]]; then
|
||||
pass "a single-plugin monorepo gets the same verdict with and without a deployed .claude/ tree"
|
||||
else
|
||||
fail "the deployed tree changed the verdict in a single-plugin monorepo — without: [$SINGLE_NO_OUT] with: [$SINGLE_WITH_OUT]"
|
||||
fi
|
||||
# Identical-but-wrong guard, as in fixture 1: the deployed-only name must DANGLE,
|
||||
# not resolve. Written as `/deployed-only-skill` so it blocks on its own without
|
||||
# needing a second target in the sentence to corroborate it.
|
||||
if [[ "$SINGLE_WITH_OUT" == *"routes to 'deployed-only-skill'"* ]]; then
|
||||
pass "the deployed-only target dangles in a single-plugin monorepo (~/.claude/skills is not in the universe)"
|
||||
else
|
||||
fail "the deployed-only target resolved — the single-plugin tree fell through to _deployed_roots(): $SINGLE_WITH_OUT"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1c. A nested .git inside a plugin must not beat the monorepo root
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0020 records the walk-up as TWO passes — plugins/*/.apm/{skills,agents}
|
||||
# first, .git only afterwards — specifically so a .git inside a plugin (a
|
||||
# submodule, or a sub-package with its own worktree) cannot capture the root.
|
||||
# Nothing anywhere placed a .git inside a plugin, so the second pass was
|
||||
# structural claim only. Collapsing the two probes into one interleaved walk
|
||||
# passes every other fixture in this repo and fails here.
|
||||
echo ""
|
||||
echo "--- a .git INSIDE a plugin does not shadow the monorepo root above it ---"
|
||||
NESTED="$TMPDIR_T/nested-git"
|
||||
write_skill "$NESTED/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
write_skill "$NESTED/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
|
||||
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
|
||||
# The trap: a git checkout one level BELOW the monorepo root and above the skill.
|
||||
mkdir -p "$NESTED/plugins/subject-plugin/.git"
|
||||
set +e
|
||||
NESTED_OUT="$(bash "$HOOK" "$NESTED/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
||||
NESTED_RC=$?
|
||||
set -e
|
||||
# The sibling-plugin name is the discriminator: it is reachable ONLY from the
|
||||
# monorepo root. If the nested .git won, subject-plugin would be the root, its
|
||||
# plugins/ glob would collect nothing, and cross-plugin-skill would dangle —
|
||||
# corroborated by sibling-skill in the same sentence, so it would BLOCK.
|
||||
if [[ $NESTED_RC -eq 0 && "$NESTED_OUT" != *"routes to"* && "$NESTED_OUT" != *"DID NOT RUN"* ]]; then
|
||||
pass "a sibling-plugin target still resolves with a .git directory inside the subject's own plugin"
|
||||
else
|
||||
fail "the nested .git captured the authoring root (exit $NESTED_RC): ${NESTED_OUT:-<empty>}"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1d. Glob metacharacters in the checkout path
|
||||
# ---------------------------------------------------------------------------
|
||||
# The universe is built with glob.glob() against paths that begin with the
|
||||
# checkout directory. A `[`, `]`, `*` or `?` anywhere in that prefix — a worktree
|
||||
# named `feature[2]`, a CI workspace named `build[1]` — turned the literal
|
||||
# directory name into a character class that matched nothing. The resolver then
|
||||
# found no universe at all and degraded to the "DID NOT RUN" INFO with rc=0:
|
||||
# every routing target in the tree silently unchecked, on a gate that reports
|
||||
# green. Same monorepo as above, one directory renamed.
|
||||
echo ""
|
||||
echo "--- glob metacharacters in the checkout path do not silently disable the resolver ---"
|
||||
GLOBDIR="$TMPDIR_T/gl[1]?x/mono"
|
||||
write_skill "$GLOBDIR/plugins/other-plugin/.apm/skills/cross-plugin-skill" cross-plugin-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/sibling-skill" sibling-skill \
|
||||
"Use when doing the other thing. Do not use for anything else."
|
||||
write_skill "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill" my-skill \
|
||||
"Use when doing the thing. Do not use for the other thing — use sibling-skill or cross-plugin-skill instead."
|
||||
set +e
|
||||
GLOB_OUT="$(bash "$HOOK" "$GLOBDIR/plugins/subject-plugin/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
||||
GLOB_RC=$?
|
||||
set -e
|
||||
if [[ $GLOB_RC -eq 0 && "$GLOB_OUT" != *"DID NOT RUN"* && "$GLOB_OUT" != *"routes to"* ]]; then
|
||||
pass "a monorepo under a directory named 'gl[1]?x' resolves exactly like any other"
|
||||
else
|
||||
fail "glob metacharacters in the path changed the verdict (exit $GLOB_RC): ${GLOB_OUT:-<empty>}"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1b. Machine independence — the real corpus
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user