fix(gates): close the /name fail-open and stop the path guard inventing targets
Two defects in the routing-target resolver, both latent in the corpus but hot for anything written next. The free-standing `/name` sweep sat inside `if boundary:`, so route notation in a sentence carrying no boundary marker was never extracted at all — not an ERROR, not a SUGGESTION, not an INFO. That contradicted ADR-0020's amendment and gates.md, which both promise `/name` blocks unconditionally. The sweep now runs over every sentence. `-> name` and backticked forms stay gated deliberately: an arrow also writes a process chain and a code span cites tools, files and skills alike, so ungating either fires on ordinary prose. The path guard used `\b`, which still holds after a hyphen, so the engine backtracked to a shorter hyphen-terminated prefix whenever the lookahead rejected the full segment. `/api-docs/v2.md` in a boundary clause raised blocking ERRORs for 'api' and 'api-docs' — names no author wrote, with no corroboration escape. `(?![\w-])` forbids the shortened prefix outright; MARKED_TARGET, which had no trailing guard at all, gained one. Zero arguments now exits 2 rather than 0, so a mis-scoped `files:` pattern is no longer indistinguishable from a clean corpus. Both hook manifests pass filenames and pre-commit skips a filename-passing hook when nothing matches, so the hook never sees an empty argv — that contract is now asserted by a test rather than left in prose. Deleting the sweep entirely used to leave every suite green. It now kills eight assertions. The suite also gains its first slash-path and URL fixtures, in both directions. Refs: #107, #110, #124 ADR: 0020
This commit is contained in:
@@ -780,6 +780,150 @@ else
|
||||
fail "a populated skills/ghost-skill/ directory still did not resolve (exit $GHOST_RC): ${GHOST_OUT:-<empty>}"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2g. The FREE-STANDING /name sweep, and its reach beyond a boundary sentence
|
||||
# ---------------------------------------------------------------------------
|
||||
# NOTATION_SLASH's own sweep in _extract_sentence() is what sees `/name` when no
|
||||
# route verb and no arrow precedes it. Nothing pinned it: every `/name` fixture
|
||||
# in this suite before these cases ALSO carried a route verb ("use
|
||||
# /no-such-slash-skill instead"), which ROUTE_ANY/ROUTE_MARKED extract on their
|
||||
# own, so deleting the sweep outright left the whole suite green. The eight
|
||||
# phrasings below carry no route verb in front of the target, so each of them is
|
||||
# invisible without the sweep — which is exactly the silence the sweep exists to
|
||||
# repair, and the shape the SUGGESTION tier's own remedy ("write it as `/name`
|
||||
# and it will be checked properly") used to teach an author to produce.
|
||||
echo ""
|
||||
echo "--- /name with no route verb in front of it is still extracted ---"
|
||||
grammar_case sweep-dash errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — /no-such-skill instead."
|
||||
grammar_case sweep-semicolon errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y; /no-such-skill handles that."
|
||||
grammar_case sweep-paren errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y (/no-such-skill covers it)."
|
||||
grammar_case sweep-possessive errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — that is /no-such-skill's job."
|
||||
grammar_case sweep-defer errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
|
||||
grammar_case sweep-terminal errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — /no-such-skill."
|
||||
# The arrow twin. `;` ends CLAUSE_BODY, so ARROW_BOUNDARY cannot reach across it
|
||||
# from `not`; only NOTATION_ARROW's own sweep sees this one.
|
||||
grammar_case sweep-arrow-after-semicolon errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y; -> no-such-skill covers it."
|
||||
|
||||
# THE SWEEP IS NOT SCOPED TO A BOUNDARY SENTENCE, and this is the case that
|
||||
# proves it. Extraction is per-sentence (corroboration is scoped to one
|
||||
# sentence), so gating the `/name` sweep on the sentence carrying a
|
||||
# BOUNDARY_MARKER meant a route written one sentence AFTER the boundary clause
|
||||
# was never looked at: exit 0, no ERROR, no SUGGESTION, not even the name. That
|
||||
# contradicts ADR-0020's amendment and docs/spec/gates.md, which both promise
|
||||
# `/name` blocks unconditionally, for any name.
|
||||
#
|
||||
# The first sentence's `/sibling-skill` is deliberate: it resolves, so the
|
||||
# fixture is not "the gate fires on any slash it sees" — it fires on the one
|
||||
# that dangles, in the sentence that carries no boundary marker at all.
|
||||
echo ""
|
||||
echo "--- /name is checked in a sentence that carries no boundary marker ---"
|
||||
grammar_case sweep-outside-boundary errors "routes to 'no-such-skill'" \
|
||||
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /no-such-skill is the right entry point."
|
||||
# Control, so the case above is not satisfied by a gate that fires on every
|
||||
# unresolvable-looking token in a non-boundary sentence: the same shape with a
|
||||
# name that RESOLVES stays silent.
|
||||
grammar_case sweep-outside-boundary-control silent "" \
|
||||
"Use for X. Do not use for Z — use /sibling-skill instead. For W, /sibling-skill is the right entry point."
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2h. A slash PATH is not a route (the trailing guard, and its backtracking)
|
||||
# ---------------------------------------------------------------------------
|
||||
# There was no path or URL fixture anywhere in this suite, and the guard was
|
||||
# defeated by ordinary regex backtracking. `/(NAME_ANY)\b(?!/|\.\S)` looks like
|
||||
# it refuses a path, and does not: when the lookahead rejects the FULL segment
|
||||
# the engine backtracks to a shorter hyphen-terminated prefix, `\b` still holds
|
||||
# after a hyphen, and the phantom is reported as a hard BLOCKING ERROR naming a
|
||||
# skill nobody wrote:
|
||||
# /opt-tools/bin/thing -> ERROR: routes to 'opt'
|
||||
# /api-docs/v2.md -> ERROR: routes to 'api' AND to 'api-docs'
|
||||
# /no-such-skill.md -> ERROR: routes to 'no-such'
|
||||
# `(?![\w-])` is the guard that actually holds, because it forbids the shortened
|
||||
# prefix instead of merely disliking the full one. MARKED_TARGET carries it too:
|
||||
# that pattern had NO trailing lookahead at all, which is where the second
|
||||
# 'api-docs' error above came from.
|
||||
#
|
||||
# These are `silent`, not `suggests`. A path is not a routing target at any
|
||||
# tier — reporting one would be the same false positive one notch quieter, on
|
||||
# the skills most likely to name a path in a boundary clause.
|
||||
echo ""
|
||||
echo "--- a slash PATH in a boundary sentence is not a routing target ---"
|
||||
grammar_case path-absolute silent "" \
|
||||
"Use when doing the thing. Do not use for Y; the config lives at /opt-tools/bin/thing."
|
||||
grammar_case path-dotted-file silent "" \
|
||||
"Use when doing the thing. Do not use for Y — see /api-docs/v2.md for the schema."
|
||||
grammar_case path-dotted-backticked silent "" \
|
||||
"Use when doing the thing. Do not use for Y — see \`/api-docs/v2.md\` for the schema."
|
||||
grammar_case path-md-suffix silent "" \
|
||||
"Use when doing the thing. Do not use for Y — the file /no-such-skill.md documents it."
|
||||
# The two suppressions that were already working and must keep working: a URL
|
||||
# (the `/` is preceded by a word character or by another `/`) and a relative
|
||||
# references/ pointer. Asserted explicitly because the guard above is a change to
|
||||
# the same lookarounds, and a fix that traded one silence for another would look
|
||||
# identical from the corpus.
|
||||
grammar_case path-url silent "" \
|
||||
"Use when doing the thing. Do not use for Y — see https://example.com/no-such-skill for details."
|
||||
grammar_case path-relative silent "" \
|
||||
"Use when doing the thing. Do not use for Y — see references/no-such-skill.md for details."
|
||||
# The other direction, which is what stops the guard from becoming a hole: a
|
||||
# name whose only follower is the SENTENCE-ENDING dot is still a route. A
|
||||
# closing `.` is not followed by a non-space, so `(?!\.\S)` does not reject it.
|
||||
# Without these, "refuse every /name near a dot or a slash" would pass every
|
||||
# case above and silently delete the notation tier.
|
||||
echo ""
|
||||
echo "--- the path guard does not swallow a /name at a real sentence end ---"
|
||||
grammar_case path-guard-sentence-end errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — defer to /no-such-skill."
|
||||
grammar_case path-guard-mid-sentence errors "routes to 'no-such-skill'" \
|
||||
"Use when doing the thing. Do not use for Y — use /no-such-skill for that instead."
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2i. A DIRECTORY named <something>.md is not an agent
|
||||
# ---------------------------------------------------------------------------
|
||||
# The skills branch of _collect_package() tests for a SKILL.md; the agents
|
||||
# branch takes every `*.md` glob hit on trust, and glob does not distinguish a
|
||||
# file from a directory. A leftover directory named `ghost-agent.md` — a botched
|
||||
# `mkdir`, an editor's stray save, a half-deleted agent — is untracked by git, so
|
||||
# it exists on the machine that made it and nowhere else, and it resolved a
|
||||
# routing target there and dangled everywhere else. That is exactly the
|
||||
# install-dependence fixture 2f pins one directory over, and the isfile() guard
|
||||
# closing it had no test at all: deleting it left every suite green.
|
||||
echo ""
|
||||
echo "--- an agents/<name>.md DIRECTORY does not make a routing target resolve ---"
|
||||
GHOST_AGENT="$TMPDIR_T/ghost-agent-dir"
|
||||
write_skill "$GHOST_AGENT/plugins/p/.apm/skills/my-skill" my-skill \
|
||||
"Use when doing the thing. Do not use for the other thing — use /ghost-agent instead."
|
||||
mkdir -p "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
||||
set +e
|
||||
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
||||
GHOST_AGENT_RC=$?
|
||||
set -e
|
||||
if [[ $GHOST_AGENT_RC -ne 0 && "$GHOST_AGENT_OUT" == *"routes to 'ghost-agent'"* ]]; then
|
||||
pass "a DIRECTORY named ghost-agent.md is not a resolvable agent name"
|
||||
else
|
||||
fail "a directory named agents/ghost-agent.md resolved a routing target (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
|
||||
fi
|
||||
# The confirming half, exactly as in 2f: replace the directory with a real file
|
||||
# and the identical description resolves. Without it the rule could be
|
||||
# implemented as "agents/ never contributes anything" and still pass above.
|
||||
rmdir "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
||||
: > "$GHOST_AGENT/plugins/p/.apm/agents/ghost-agent.md"
|
||||
set +e
|
||||
GHOST_AGENT_OUT="$(bash "$HOOK" "$GHOST_AGENT/plugins/p/.apm/skills/my-skill/SKILL.md" 2>&1)"
|
||||
GHOST_AGENT_RC=$?
|
||||
set -e
|
||||
if [[ $GHOST_AGENT_RC -eq 0 && -z "$GHOST_AGENT_OUT" ]]; then
|
||||
pass "the same path as a FILE resolves, so the rule is 'not a file' and not 'never'"
|
||||
else
|
||||
fail "a real agents/ghost-agent.md file still did not resolve (exit $GHOST_AGENT_RC): ${GHOST_AGENT_OUT:-<empty>}"
|
||||
fi
|
||||
|
||||
# And the confirming half of the grammar rule: a compound-modifier target is
|
||||
# CONFIRM-ONLY, not ignored. When the name does exist it still counts as a route
|
||||
# — the rule suppresses the ERROR, it does not delete the target.
|
||||
|
||||
@@ -702,6 +702,175 @@ expect_gate "a fixture with no authoring root reports DID NOT RUN and exits 0" \
|
||||
#
|
||||
# If a real dangling target ever reappears, add its probe back here.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Usage tier: zero arguments is exit 2, not a clean run
|
||||
# ---------------------------------------------------------------------------
|
||||
# The script used to print nothing and exit 0 when handed no paths, which made
|
||||
# a mis-scoped `files:` pattern indistinguishable from a corpus with no
|
||||
# findings — the whole ADR-0020 gate family silently disabled while every hook
|
||||
# reported green. Exit 2 (not 1) is the same split a8cd5e8 made in
|
||||
# provider-adapter-author's validate-adapter.sh and the one vale-wrap.sh already
|
||||
# used: {0,1} are verdicts, 2 is "you invoked this wrong".
|
||||
#
|
||||
# SAFE FOR THE HOOK. Both manifests declare pass_filenames: true and neither
|
||||
# sets always_run, and pre-commit skips a filename-passing hook outright when
|
||||
# its `files:` pattern matches nothing, so pre-commit never invokes this script
|
||||
# with an empty argument list. That claim is asserted below rather than left in
|
||||
# prose, so a config edit that turns it false fails here.
|
||||
echo ""
|
||||
echo "--- zero arguments is a usage error (exit 2), not a silent clean run ---"
|
||||
set +e
|
||||
USAGE_OUT="$("$SCRIPT" 2>&1)"
|
||||
USAGE_RC=$?
|
||||
set -e
|
||||
if [[ $USAGE_RC -eq 2 ]]; then
|
||||
pass "no arguments exits 2"
|
||||
else
|
||||
fail "no arguments exited $USAGE_RC, expected 2 (output: ${USAGE_OUT:-<empty>})"
|
||||
fi
|
||||
if [[ "$USAGE_OUT" == *usage* ]]; then
|
||||
pass "no arguments prints a usage message"
|
||||
else
|
||||
fail "no arguments produced no usage message (output: ${USAGE_OUT:-<empty>})"
|
||||
fi
|
||||
# The exit code must be DISTINCT from both verdicts, or the split buys nothing.
|
||||
# $SMALL is the clean fixture built at the top of this file; $MANY_LINES is over
|
||||
# the line ceiling.
|
||||
set +e
|
||||
"$SCRIPT" "$SMALL" > /dev/null 2>&1
|
||||
CLEAN_RC=$?
|
||||
"$SCRIPT" "$MANY_LINES" > /dev/null 2>&1
|
||||
FINDING_RC=$?
|
||||
set -e
|
||||
if [[ $CLEAN_RC -eq 0 && $FINDING_RC -eq 1 && $USAGE_RC -eq 2 ]]; then
|
||||
pass "the three exit codes are distinct: clean=0, findings=1, usage=2"
|
||||
else
|
||||
fail "exit codes collide — clean=$CLEAN_RC findings=$FINDING_RC usage=$USAGE_RC"
|
||||
fi
|
||||
# The hook contract the usage exit depends on. If either manifest ever stops
|
||||
# passing filenames, or starts always_run, pre-commit could invoke the script
|
||||
# with no paths and exit 2 would break the hook rather than diagnose a caller.
|
||||
HOOK_CONTRACT="$(python3 - "$REPO_ROOT" <<'PYHOOK'
|
||||
import os
|
||||
import sys
|
||||
|
||||
import yaml
|
||||
|
||||
root = sys.argv[1]
|
||||
problems = []
|
||||
|
||||
|
||||
def check(label, hook):
|
||||
if hook is None:
|
||||
problems.append('%s declares no such hook' % label)
|
||||
return
|
||||
if hook.get('pass_filenames') is False:
|
||||
problems.append('%s sets pass_filenames: false' % label)
|
||||
if hook.get('always_run'):
|
||||
problems.append('%s sets always_run: true' % label)
|
||||
|
||||
|
||||
with open(os.path.join(root, '.pre-commit-config.yaml'), encoding='utf-8') as fh:
|
||||
cfg = yaml.safe_load(fh) or {}
|
||||
found = None
|
||||
for repo in cfg.get('repos') or []:
|
||||
for hook in (repo.get('hooks') or []):
|
||||
if hook.get('id') == 'skill-size-check':
|
||||
found = hook
|
||||
check('.pre-commit-config.yaml skill-size-check', found)
|
||||
|
||||
with open(os.path.join(root, '.pre-commit-hooks.yaml'), encoding='utf-8') as fh:
|
||||
hooks = yaml.safe_load(fh) or []
|
||||
found = None
|
||||
for hook in hooks:
|
||||
if isinstance(hook, dict) and hook.get('id') == 'kyberforge-skill-size-check':
|
||||
found = hook
|
||||
check('.pre-commit-hooks.yaml kyberforge-skill-size-check', found)
|
||||
|
||||
print('; '.join(problems))
|
||||
PYHOOK
|
||||
)"
|
||||
if [[ -z "$HOOK_CONTRACT" ]]; then
|
||||
pass "both manifests pass filenames and neither is always_run, so pre-commit never invokes the script with no paths"
|
||||
else
|
||||
fail "the usage exit would break the hook: $HOOK_CONTRACT"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# An unreadable path is diagnosed ONCE
|
||||
# ---------------------------------------------------------------------------
|
||||
# The stat dance lived twice — a bash pre-loop and the Python per-file loop —
|
||||
# and both printed the same sentence, so one broken file produced two ERROR
|
||||
# lines with two different "so ... could not be measured" clauses. Duplicated
|
||||
# output on a blocking gate reads as two problems and sends the author hunting
|
||||
# for a second one. The check must still FIRE (silence is the failure this
|
||||
# script forbids itself); it must fire exactly once.
|
||||
echo ""
|
||||
echo "--- an unreadable path produces exactly one ERROR line, not two ---"
|
||||
UNREADABLE_DIR="$TMPDIR/unreadable"
|
||||
mkdir -p "$UNREADABLE_DIR/a-directory.md"
|
||||
ln -sf "$TMPDIR/definitely-not-here.md" "$UNREADABLE_DIR/broken-link.md"
|
||||
|
||||
# unreadable_case <label> <path>
|
||||
unreadable_case() {
|
||||
local label="$1" path="$2" out status=0 count
|
||||
set +e
|
||||
out="$("$SCRIPT" "$path" 2>&1)"
|
||||
status=$?
|
||||
set -e
|
||||
count="$(printf '%s\n' "$out" | grep -cF "ERROR: $path" || true)"
|
||||
if [[ $status -eq 0 ]]; then
|
||||
fail "$label: exited 0 — an unmeasurable path passed in silence (output: ${out:-<empty>})"
|
||||
elif [[ "$count" != "1" ]]; then
|
||||
fail "$label: $count ERROR lines name the path, expected exactly 1 (output: $out)"
|
||||
else
|
||||
pass "$label"
|
||||
fi
|
||||
}
|
||||
unreadable_case "a path that does not exist is reported once" \
|
||||
"$TMPDIR/no-such-file.md"
|
||||
unreadable_case "a DIRECTORY named *.md is reported once" \
|
||||
"$UNREADABLE_DIR/a-directory.md"
|
||||
unreadable_case "a broken symlink is reported once" \
|
||||
"$UNREADABLE_DIR/broken-link.md"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Encoding, write side: under LC_ALL=C the report must still print
|
||||
# ---------------------------------------------------------------------------
|
||||
# read_text() in the shared ADR-0020 resolver block pins the READS to UTF-8.
|
||||
# That moved the crash to the WRITE: this script's own message text carries em
|
||||
# dashes (the boundary SUGGESTION is one), so under LC_ALL=C the streams' ASCII
|
||||
# default raised UnicodeEncodeError while PRINTING -- after every check had
|
||||
# already run, losing the whole report at the last step and turning a
|
||||
# SUGGESTION-only exit 0 into a traceback and an exit 1.
|
||||
echo ""
|
||||
echo "--- under LC_ALL=C the SUGGESTION is printed, not lost to a UnicodeEncodeError ---"
|
||||
LOCALE_SKILL="$TMPDIR/locale-skill"
|
||||
mkdir -p "$LOCALE_SKILL"
|
||||
cat > "$LOCALE_SKILL/SKILL.md" <<'LOCALEEOF'
|
||||
---
|
||||
name: locale-skill
|
||||
description: A valid skill description that is well within the limit.
|
||||
---
|
||||
|
||||
## Step 1
|
||||
|
||||
Do the thing.
|
||||
LOCALEEOF
|
||||
set +e
|
||||
LOCALE_OUT="$(env LC_ALL=C PYTHONUTF8=0 "$SCRIPT" "$LOCALE_SKILL/SKILL.md" 2>&1)"
|
||||
LOCALE_STATUS=$?
|
||||
set -e
|
||||
if [[ $LOCALE_STATUS -ne 0 ]]; then
|
||||
fail "a SUGGESTION-only subject exited $LOCALE_STATUS under LC_ALL=C (output: ${LOCALE_OUT:-<empty>})"
|
||||
elif [[ "$LOCALE_OUT" == *UnicodeEncodeError* || "$LOCALE_OUT" == *Traceback* ]]; then
|
||||
fail "the report died encoding its own message text under LC_ALL=C (output: $LOCALE_OUT)"
|
||||
elif [[ "$LOCALE_OUT" != *"description has no boundary clause"* ]]; then
|
||||
fail "the SUGGESTION never reached stdout under LC_ALL=C (output: ${LOCALE_OUT:-<empty>})"
|
||||
else
|
||||
pass "the SUGGESTION survives LC_ALL=C, streams pinned to UTF-8"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
Reference in New Issue
Block a user