test: pin the nine ADR-0020 gate defects that shipped untested

Every defect fixed in f7cc279 was reachable because nothing asserted against it.
The gate had 43 assertions and none of them covered a consumer repo, a non-string
description, an unclosed fence, or the two spec ceilings. Each case below fails
against the pre-fix code and passes against the current one; every one was proved
non-vacuous by mutating a scratch copy of the script and watching the test go red,
independently twice.

The two that mattered most had no fixture anywhere. A consumer repo WITH .git is
the shape the resolver exists to serve, and only the no-.git case had ever been
tested, which is exactly why the blocker was invisible. And ADR-0020 says the
walk-up runs in two passes specifically so a nested .git cannot beat a plugins/
root further up — no fixture had ever placed a .git inside a plugin.

test-adr0020-differential.sh loses _non_adr_hook_error(). It excluded MAX_LINES and
MAX_WORDS from the cross-script comparison on the untested assumption that awk and
splitlines() agree. They do not, and the divergence stayed invisible for exactly as
long as the exclusion stood. The ceilings are now compared like any other rule.

Two existing assertions were repairs, not additions. The skill-improve probe had
been fixed by this very branch, so its iteration permanently took an
assertion-free SKIP that still counted as a pass; both branches now fail loudly and
each names the other file's pin so the two stay in step. And the yaml-none fixture
emitted `---/---`, which never matched the frontmatter pattern at all — it passed on
the bare word "frontmatter", present in both messages, while never reaching the
branch it was named for. Needles throughout that file now name their branch.

Refs: #99
ADR: 0020

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015W3iwF9ncfRZddGBxsMCYi
This commit is contained in:
2026-08-16 19:49:15 +00:00
parent f7cc27908c
commit b0d6d08239
6 changed files with 673 additions and 42 deletions

View File

@@ -810,3 +810,136 @@ EOF
assert_success
refute_output --partial "hooks"
}
# ---------------------------------------------------------------------------
# tools: — both YAML spellings
# ---------------------------------------------------------------------------
# The subagent-unavailable-tool SUGGESTION is read off the `tools` field, and
# `tools` has two legal spellings: an inline scalar and a block sequence. The
# field used to be pulled out with a line regex whose capture is newline-bounded
# on purpose, so a block sequence captured NOTHING and the check silently
# stopped firing — on the shape Copilot agent files actually use, which is to say
# on the files it was written for. Both spellings are pinned, and they are pinned
# together: the inline case alone was green throughout.
# make_pair <root> <tools-frontmatter> — a project-scope CC + Copilot pair
# carrying the same `tools` value in both files. `tools` is on neither the
# claude-code-only nor the copilot-only list, so it is legal in both and the pair
# stays otherwise clean; the description carries a boundary clause so the only
# SUGGESTION that can fire is the one under test.
make_tools_pair() {
local root="$1" tools="$2"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
local f
for f in "$root/.claude/agents/my-agent.md" "$root/.github/agents/my-agent.agent.md"; do
{
echo "---"
echo "name: my-agent"
echo "description: A valid agent description. Do not use for anything else."
echo "$tools"
echo "---"
echo ""
echo "You are a test agent. When invoked, do the thing."
} > "$f"
done
}
@test "a subagent-unavailable tool in an INLINE tools scalar raises a SUGGESTION" {
make_tools_pair "$TMPDIR/inline" "tools: Read ExitPlanMode"
run bash "$SCRIPT" "$TMPDIR/inline/.claude/agents/my-agent.md"
assert_success
assert_output --partial "'ExitPlanMode' is listed in tools but is never available to subagents"
}
@test "a subagent-unavailable tool in a BLOCK SEQUENCE tools field raises the same SUGGESTION" {
make_tools_pair "$TMPDIR/block" "$(printf 'tools:\n - Read\n - ExitPlanMode')"
run bash "$SCRIPT" "$TMPDIR/block/.claude/agents/my-agent.md"
assert_success
assert_output --partial "'ExitPlanMode' is listed in tools but is never available to subagents"
}
@test "a tools list with no subagent-unavailable tool stays silent in both spellings" {
# The control. Without it both cases above are satisfied by a check that
# fires on every tools field it can see, which would be the opposite defect.
make_tools_pair "$TMPDIR/inline-clean" "tools: Read Edit"
run bash "$SCRIPT" "$TMPDIR/inline-clean/.claude/agents/my-agent.md"
assert_success
refute_output --partial "never available to subagents"
make_tools_pair "$TMPDIR/block-clean" "$(printf 'tools:\n - Read\n - Edit')"
run bash "$SCRIPT" "$TMPDIR/block-clean/.claude/agents/my-agent.md"
assert_success
refute_output --partial "never available to subagents"
}
# ---------------------------------------------------------------------------
# A file that cannot be read
# ---------------------------------------------------------------------------
# scripts/check-apm-agents-valid.sh derives its expected agent-file set from
# `git ls-files`, so it hands this script paths that are tracked but absent from
# the worktree — a real and expected state, not a corner case. That used to exit
# 1 with a bare FileNotFoundError traceback and no FAIL line at all: non-zero, so
# the gate blocked, but with an interpreter stack instead of a diagnostic naming
# the file. Both scope paths are covered because they are separate call sites
# (check_apm_agent_file and check_file) and each needed its own handler.
#
# `is-a-dir.agent.md` is a DIRECTORY rather than a chmod 000 file on purpose:
# these tests run as root in CI, where mode bits do not deny anything and a
# permissions fixture would be silently readable and prove nothing.
@test "a nonexistent plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
run bash "$SCRIPT" "$root/.apm/agents/absent.agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "absent.agent.md"
refute_output --partial "Traceback"
refute_output --partial "FileNotFoundError"
}
@test "an unreadable plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
local root="$TMPDIR/pkg-dir"
mkdir -p "$root/.apm/agents/is-a-dir.agent.md"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
run bash "$SCRIPT" "$root/.apm/agents/is-a-dir.agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "is-a-dir.agent.md"
refute_output --partial "Traceback"
refute_output --partial "IsADirectoryError"
}
@test "a nonexistent project-scope agent file gets a FAIL naming the path, not a traceback" {
# The counterpart is pre-checked before either file is opened, so this
# exercises the OTHER call site: the counterpart exists, the named file does
# not, and check_file is what has to report it.
local root="$TMPDIR/proj-missing"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description. Do not use for anything else.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "my-agent.md"
refute_output --partial "Traceback"
refute_output --partial "FileNotFoundError"
}