From 59f27dbd94b46aa5f177d6f79ee93f395511de07 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Tue, 1 Sep 2026 12:38:04 +0000 Subject: [PATCH] fix(core): close five ways validate-adapter.sh graded an adapter it had not read MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The adapter check shipped green on files it should have failed, and failed files it should have passed. Each defect is a residual of the fix that closes #115. A fenced, indented or HTML-commented `@AGENTS.md` counted as an import, though Claude Code resolves none of them — the adapter deferred to nothing and the gate said so approvingly. Import matching now runs against a character mask that marks fenced blocks and HTML comments inert, and applies CommonMark's four-space rule. The mask is deliberately not applied to prose pointers, where four-space indentation is ordinary list continuation. The encoding fix reached only BOM-carrying UTF-16/32. BOM-less UTF-16LE/BE and UTF-32LE are valid UTF-8, so they still produced the exact false diagnosis the fix was written to remove: "no reference to AGENTS.md" on a file whose first line is `@AGENTS.md`. A NUL-byte check is the complete signal. BOM stripping is no longer positional, which also drops the mirror-image false FAILs on a doubled or mid-file BOM. `@NOTAGENTS.md`, `@zzzAGENTS.md` and `@docs/does/not/exist/AGENTS.md` all passed: the pattern had no path-segment boundary and the target was never resolved on disk. Both now hold, and a zero-byte or blank target is reported rather than credited. Unresolved candidates print as `Near miss:` lines so the author sees why a line was not counted. `--no-import-syntax` still used substring matching, so `Do NOT read AGENTS.md; it is obsolete.` passed as a pointer. That is #115's own defect surviving in the flag's other mode. A mention must now carry a deference cue and must not be negated. An unreadable file passed `isfile()`, raised, and exited 1 with a traceback and no FAIL line — the one exit code no document covered, while Step 3 says to re-run until it exits 0. It now exits 3 with a diagnostic, and the README states all four codes and what to do about each instead of "exits non-zero on any failure". Tests 18 to 42. Two of the new cases initially survived their own mutation and were strengthened: `@NOTAGENTS.md` was being rejected by the disk check before the token boundary ran, and stripping that boundary leaves the fragment `NOT`, which the negation cue then rejects for an unrelated reason. Refs: #115 --- .../skills/provider-adapter-author/SKILL.md | 6 +- .../provider-adapter-author/scripts/README.md | 23 +- .../scripts/validate-adapter.sh | 384 ++++++++++++++++-- .../tests/validate-adapter.bats | 305 ++++++++++++++ .../skills/provider-adapter-author/SKILL.md | 6 +- .../provider-adapter-author/scripts/README.md | 23 +- .../scripts/validate-adapter.sh | 384 ++++++++++++++++-- 7 files changed, 1041 insertions(+), 90 deletions(-) diff --git a/plugins/core/.apm/skills/provider-adapter-author/SKILL.md b/plugins/core/.apm/skills/provider-adapter-author/SKILL.md index 97f7e16..254196e 100644 --- a/plugins/core/.apm/skills/provider-adapter-author/SKILL.md +++ b/plugins/core/.apm/skills/provider-adapter-author/SKILL.md @@ -30,8 +30,8 @@ Then confirm `AGENTS.md` exists at the repo root. If it does not, stop and tell Read the provider file and `AGENTS.md` side by side. Separate the provider file's content into two buckets: lines that restate what `AGENTS.md` already owns (universal rules, conventions, project overview) versus lines that are genuinely provider-specific (tool syntax, IDE behavior, model-specific instructions). Rewrite the provider file: -- **Providers with import syntax** (Claude Code): replace the redundant bucket with an `@AGENTS.md` (or correct relative path) import on a line of its own, keep the provider-specific bucket below it. An import folded into a sentence is not the thin-adapter shape and `scripts/validate-adapter.sh` will not credit it. -- **Providers without import syntax** (Cursor, Copilot, etc.): replace the redundant bucket with a short pointer sentence mentioning `AGENTS.md`, keep the provider-specific bucket. +- **Providers with import syntax** (Claude Code): replace the redundant bucket with an `@AGENTS.md` (or correct relative path) import on a line of its own, keep the provider-specific bucket below it. An import folded into a sentence is not the thin-adapter shape and `scripts/validate-adapter.sh` will not credit it — nor one inside a code fence, an indented block, or an HTML comment, nor one whose path does not resolve to a real, non-empty file on disk. +- **Providers without import syntax** (Cursor, Copilot, etc.): replace the redundant bucket with a short sentence pointing at `AGENTS.md` ("See AGENTS.md at the repo root for ..."), keep the provider-specific bucket. A bare or negated mention is not a pointer and will not be credited. The provider file is the only file this skill ever writes. Never create or edit `AGENTS.md` — not in this step, not in any step, whatever the payoff looks like. @@ -45,7 +45,7 @@ Run the bundled check before finishing — this is the skill's own closeout gate bash scripts/validate-adapter.sh [--no-import-syntax] [--max-lines N] ``` -Fix any `FAIL` by editing the provider file, and re-run until it exits `0`. Exit `2` is not a `FAIL`: it means the invocation or the input is wrong — a bad or missing argument, or a file that is not UTF-8 — so fix that, not the adapter. +Fix any `FAIL` by editing the provider file, and re-run until it exits `0`. Exits `2` and `3` are not `FAIL`s and nothing was graded under either, so neither is a reason to touch the adapter: `2` means the invocation or the input is wrong (a bad, missing, or extra argument, an unknown option, or a file that is not UTF-8), and `3` means a named file exists but could not be read. ## Step 4 — Report diff --git a/plugins/core/.apm/skills/provider-adapter-author/scripts/README.md b/plugins/core/.apm/skills/provider-adapter-author/scripts/README.md index d137d2b..49c4242 100644 --- a/plugins/core/.apm/skills/provider-adapter-author/scripts/README.md +++ b/plugins/core/.apm/skills/provider-adapter-author/scripts/README.md @@ -4,6 +4,25 @@ Deterministic self-check this skill shells out to instead of relying on LLM judg | File | Purpose | |------|---------| -| `validate-adapter.sh` | Checks a rewritten provider file (CLAUDE.md, etc.) has a reference to AGENTS.md, doesn't duplicate its content, and stays under a thin-file line threshold | +| `validate-adapter.sh` | Checks a rewritten provider file (CLAUDE.md, etc.) has a working reference to AGENTS.md, doesn't duplicate its content, and stays under a thin-file line threshold | -Takes ` `, with optional `--no-import-syntax` and `--max-lines N` flags. Prints `FAIL` findings to stdout and exits non-zero on any failure. +Takes exactly ` `, with optional `--no-import-syntax` and `--max-lines N` flags (also accepted as `--max-lines=N`). A third positional argument or an unknown option is an error, not something quietly ignored. + +## What counts as a reference to AGENTS.md + +Both modes require the named path to be a real path segment ending in `AGENTS.md` — `AGENTS.md` or `…/AGENTS.md`, not `NOTAGENTS.md` — that resolves on disk, relative to the adapter file, to a non-empty file. An adapter deferring to a path that is not there defers to nothing, so the check has to touch the disk rather than pattern-match the line. + +A reference only counts where something would actually resolve it. A line inside a fenced code block, an indented code block, or an HTML comment is not credited in either mode: Claude Code resolves an import in none of those, so a fenced `@AGENTS.md` is the silent-drop failure this gate exists to catch, not a pass. + +Default mode wants a real import: `@AGENTS.md` alone on its own line, indented no more than three spaces. `--no-import-syntax` wants a prose pointer that reads as one — the sentence naming `AGENTS.md` must carry a deference cue (see, read, refer to, documented in, conventions, …) and must not be negated. `Do NOT read AGENTS.md; it is obsolete.` and `We deleted AGENTS.md last year.` name the file while pointing the reader away from it, and neither is a pointer. + +## Exit codes + +The distinction matters because the skill's closeout tells the agent to fix any non-zero exit by editing the provider file. That is right for exactly one of these. + +| Code | Meaning | What to do | +|------|---------|------------| +| `0` | Passes every check | Nothing | +| `1` | One or more `FAIL` findings printed to stdout — empty adapter, no working reference to AGENTS.md, excessive duplication, or not thin | Edit the provider file | +| `2` | Usage or input error: a bad, missing, or extra argument, an unknown option, a path that is not a file, or a file that is not UTF-8. Nothing was graded, so there is no `FAIL` line | Fix the invocation or the file's encoding — do not edit the adapter | +| `3` | A named input file exists but could not be read (permissions, I/O error). Nothing was graded and the adapter's contents are unknown | Fix the file's readability — do not edit the adapter | diff --git a/plugins/core/.apm/skills/provider-adapter-author/scripts/validate-adapter.sh b/plugins/core/.apm/skills/provider-adapter-author/scripts/validate-adapter.sh index 66ba72c..9648442 100755 --- a/plugins/core/.apm/skills/provider-adapter-author/scripts/validate-adapter.sh +++ b/plugins/core/.apm/skills/provider-adapter-author/scripts/validate-adapter.sh @@ -14,6 +14,10 @@ Arguments: adapter-file Path to the provider-specific file to check. agents-md-file Path to the AGENTS.md file it should defer to. + Exactly two positional arguments are accepted. Extra ones are rejected + rather than ignored: a third path silently graded nothing but the first + two, so a typo'd invocation passed against the wrong file. + Options: --no-import-syntax The target provider has no native cross-file import mechanism. Require a plain-text pointer line naming @@ -26,26 +30,69 @@ Options: it's considered no longer "thin". Must be a non-negative integer. Default: 60. --help, -h Show this help and exit 0. + -- End of options; every later argument is positional. + +Both flags also accept the --flag=value form (--max-lines=40). An unknown +option is reported as an unknown option, not as a missing file. + +What counts as a reference: + + In both modes the named path must be a real path segment ending in + AGENTS.md ("AGENTS.md" or ".../AGENTS.md" — not NOTAGENTS.md), and it must + resolve on disk, relative to the adapter file, to a non-empty file. An + adapter deferring to a path that is not there defers to nothing. + + A mention inside a fenced code block, an indented code block, or an HTML + comment is not credited in either mode. Nothing resolves those, so an + adapter whose only "import" is fenced silently defers to nothing. + + With --no-import-syntax the pointer must read as a pointer: the sentence + naming AGENTS.md has to carry a deference cue (see, read, refer to, + documented in, conventions, ...) and must not be a negation ("do not read + AGENTS.md", "we deleted AGENTS.md"). A bare mention is not a pointer. Exit codes: 0 Adapter file passes all checks 1 One or more checks failed (empty file, no reference to AGENTS.md, excessive duplication, or file too long) - 2 Usage or input error — a bad or missing argument, a path that is not a - file, or a file that is not UTF-8. Nothing was graded, so there is no - FAIL line and no adapter edit to make: fix the invocation or the file's - encoding and re-run. Kept distinct from 1 because the skill's own - closeout tells the agent to fix every non-zero exit by editing the - provider file, which for a mistyped flag edits the wrong file forever. + 2 Usage or input error — a bad, missing, or extra argument, an unknown + option, a path that is not a file, or a file that is not UTF-8. Nothing + was graded, so there is no FAIL line and no adapter edit to make: fix + the invocation or the file's encoding and re-run. Kept distinct from 1 + because the skill's own closeout tells the agent to fix every non-zero + exit by editing the provider file, which for a mistyped flag edits the + wrong file forever. + 3 A named input file exists but could not be read (permissions, a + directory swapped in mid-run, I/O error). Also not a FAIL: nothing was + graded and the adapter's contents are unknown, so editing it is + guesswork. Fix the file's readability and re-run. EOF } NO_IMPORT_SYNTAX=0 MAX_LINES=60 ARGS=() +END_OF_OPTS=0 + +require_int() { + # $1 = the value to validate + if [[ ! "$1" =~ ^[0-9]+$ ]]; then + echo "Error: --max-lines expects a non-negative integer, got '$1'." >&2 + exit 2 + fi +} while [[ $# -gt 0 ]]; do + if [[ $END_OF_OPTS -eq 1 ]]; then + ARGS+=("$1") + shift + continue + fi case "$1" in + --) + END_OF_OPTS=1 + shift + ;; --help|-h) usage exit 0 @@ -54,17 +101,37 @@ while [[ $# -gt 0 ]]; do NO_IMPORT_SYNTAX=1 shift ;; + --no-import-syntax=*) + echo "Error: --no-import-syntax is a flag and takes no value (got '$1')." >&2 + exit 2 + ;; --max-lines) if [[ $# -lt 2 ]]; then echo "Error: --max-lines requires a value (a non-negative integer)." >&2 exit 2 fi MAX_LINES="$2" - if [[ ! "$MAX_LINES" =~ ^[0-9]+$ ]]; then - echo "Error: --max-lines expects a non-negative integer, got '$MAX_LINES'." >&2 + require_int "$MAX_LINES" + shift 2 + ;; + --max-lines=*) + MAX_LINES="${1#--max-lines=}" + if [[ -z "$MAX_LINES" ]]; then + echo "Error: --max-lines requires a value (a non-negative integer)." >&2 exit 2 fi - shift 2 + require_int "$MAX_LINES" + shift + ;; + -*) + # Reported as an unknown option rather than falling through to the + # positional bucket, where it used to surface as "'--bogus' is not a + # file" — the right exit code attached to a diagnostic that sends the + # reader looking for a path they never typed. + echo "Error: unknown option '$1'." >&2 + echo "" >&2 + usage >&2 + exit 2 ;; *) ARGS+=("$1") @@ -80,6 +147,13 @@ if [[ ${#ARGS[@]} -lt 2 ]]; then exit 2 fi +if [[ ${#ARGS[@]} -gt 2 ]]; then + echo "Error: expected exactly 2 positional arguments (adapter-file and agents-md-file), got ${#ARGS[@]}: ${ARGS[*]}." >&2 + echo "" >&2 + usage >&2 + exit 2 +fi + python3 -u - "${ARGS[0]}" "${ARGS[1]}" "$NO_IMPORT_SYNTAX" "$MAX_LINES" <<'PYTHON' import sys import os @@ -89,45 +163,80 @@ adapter_path, agents_md_path, no_import_syntax, max_lines = sys.argv[1:5] no_import_syntax = no_import_syntax == "1" max_lines = int(max_lines) +EXIT_FAIL = 1 +EXIT_USAGE = 2 +EXIT_UNREADABLE = 3 + if not os.path.isfile(adapter_path): print(f"Error: '{adapter_path}' is not a file.", file=sys.stderr) - sys.exit(2) + sys.exit(EXIT_USAGE) if not os.path.isfile(agents_md_path): print(f"Error: '{agents_md_path}' is not a file.", file=sys.stderr) - sys.exit(2) + sys.exit(EXIT_USAGE) def read_text(path): - r"""File contents as text, UTF-8, BOM stripped. + r"""File contents as text, UTF-8, every BOM stripped. - The BOM strip is not cosmetic. IMPORT_RE anchors on `^\s*@`, and a BOM is - not `\s` in Python, so a CLAUDE.md saved by an editor that emits one had - its first line — the `@AGENTS.md` import, which is the whole adapter — - silently treated as prose. The check then said "no reference to AGENTS.md" - told the author to add the line already sitting in front of them. Same - class of silent BOM miss recorded in scripts/skill-size-check.sh; strip it - at the reader so no later check has to know about it. + The BOM strip is not cosmetic. IMPORT_RE anchors on `^ {0,3}@`, and a BOM + is not whitespace in Python, so a CLAUDE.md saved by an editor that emits + one had its first line — the `@AGENTS.md` import, which is the whole + adapter — silently treated as prose. The check then said "no reference to + AGENTS.md" and told the author to add the line already sitting in front of + them. Same class of silent BOM miss recorded in scripts/skill-size-check.sh; + strip it at the reader so no later check has to know about it. + + Every U+FEFF goes, not just one at offset 0. Stripping exactly the first + one left the mirror-image false FAIL for a doubled BOM (two concatenated + files, or a tool that re-adds one) and for a BOM mid-file at the head of + the import line. U+FEFF has no meaning as a character in a markdown + instruction file, so removing all of them cannot lose signal. Decoding is strict, not errors="replace". Replacement mangles the file and the checks then grade the mangling: a UTF-16 adapter whose first line is `@AGENTS.md` decoded to interleaved NULs and failed as "no reference", which is a true FAIL for a false reason and points the fix at the wrong - thing. A file this gate cannot read gets an encoding diagnostic and exit 2, + thing. But strict UTF-8 alone does not catch it — BOM-less UTF-16LE/BE and + UTF-32LE are *valid* UTF-8, because NUL is a legal code point, so they + decoded clean and produced exactly that false diagnosis anyway. The NUL + byte is the complete signal and is checked first: no plausible markdown + adapter contains one, and every UTF-16/32 encoding of ASCII is full of + them. A file this gate cannot read gets an encoding diagnostic and exit 2, the same policy the ADR-0020 validators' read_text() uses. + + A file that exists but cannot be read at all is neither a pass nor a FAIL — + nothing was graded — so it exits 3 rather than 1. Exit 1 sends the skill's + closeout into "fix the FAIL by editing the provider file", which for a file + it cannot open is an instruction to edit blind. """ try: - with open(path, encoding="utf-8") as fh: - text = fh.read() + with open(path, "rb") as fh: + raw = fh.read() + except OSError as exc: + print(f"Error: '{path}' exists but could not be read ({exc.strerror}). " + "Nothing was checked — fix whatever is blocking the read " + "(permissions, ownership, the underlying device) and re-run; do " + "not edit the adapter on the strength of this.", file=sys.stderr) + sys.exit(EXIT_UNREADABLE) + if b"\x00" in raw: + print(f"Error: '{path}' is not valid UTF-8 — it contains NUL bytes, so " + "it is almost certainly UTF-16 or UTF-32 (with or without a BOM). " + "Re-save it as UTF-8; this check does not guess at other " + "encodings.", file=sys.stderr) + sys.exit(EXIT_USAGE) + try: + text = raw.decode("utf-8") except UnicodeDecodeError as exc: print(f"Error: '{path}' is not valid UTF-8 ({exc.reason} at byte " f"{exc.start}) — re-save it as UTF-8; this check does not guess " "at other encodings.", file=sys.stderr) - sys.exit(2) - return text[1:] if text.startswith("\ufeff") else text + sys.exit(EXIT_USAGE) + return text.replace("\ufeff", "") adapter_content = read_text(adapter_path) agents_md_content = read_text(agents_md_path) +adapter_dir = os.path.dirname(os.path.abspath(adapter_path)) has_fail = False @@ -136,34 +245,229 @@ if not adapter_content.strip(): print(" Why: An empty adapter carries no reference to AGENTS.md and no provider-specific content.") print(" Fix: Add at least an import (or text pointer) to AGENTS.md.") print() - sys.exit(1) + sys.exit(EXIT_FAIL) -IMPORT_RE = re.compile(r'(?m)^\s*@\S*AGENTS\.md\s*$') -lines = adapter_content.splitlines() -import_lines = [ln for ln in lines if IMPORT_RE.match(ln)] -# A prose pointer is any line naming AGENTS.md that is not itself an import -# line — an inert `@AGENTS.md` in a provider that resolves no imports points -# a reader at nothing. -pointer_lines = [ln for ln in lines if not IMPORT_RE.match(ln) and "AGENTS.md" in ln] + +# --- Inert regions ----------------------------------------------------------- +# +# A reference only counts where something would actually resolve it. Fenced +# code blocks, indented code blocks and HTML comments are shown to the reader +# (or hidden from them) as literal text; Claude Code resolves an @import in +# none of them. Without this, a ```-fenced `@AGENTS.md` — the exact +# copy-the-example-into-the-file mistake this gate exists to catch — exited 0 +# with the adapter deferring to nothing. +# +# Indented code blocks are handled by IMPORT_RE's `^ {0,3}` instead of by the +# mask: four leading spaces is what opens an indented code block in CommonMark, +# so an import has to sit within three. The mask deliberately does not apply +# that rule to prose pointers, where four-space indentation is ordinary list +# continuation rather than code. +FENCE_RE = re.compile(r'^( {0,3})(`{3,}|~{3,})(.*)$') +COMMENT_RE = re.compile(r'|\Z)', re.DOTALL) + + +def line_offsets(text): + """[(char offset, line without its terminator)] over `text`.""" + out = [] + off = 0 + for raw in text.splitlines(keepends=True): + out.append((off, raw.rstrip("\r\n"))) + off += len(raw) + return out + + +def build_inert_mask(text, offsets): + """Per-character flags: 1 where a reference would never be resolved.""" + mask = bytearray(len(text)) + fence = None # (fence char, opening run length) + for start, line in offsets: + m = FENCE_RE.match(line) + if fence is None: + if m: + fence = (m.group(2)[0], len(m.group(2))) + for i in range(start, start + len(line)): + mask[i] = 1 + continue + for i in range(start, start + len(line)): + mask[i] = 1 + if (m and m.group(2)[0] == fence[0] + and len(m.group(2)) >= fence[1] + and not m.group(3).strip()): + fence = None + for m in COMMENT_RE.finditer(text): + if m.start() < len(mask) and mask[m.start()]: + continue # a literal " +EOF + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "an @AGENTS.md indented up to three spaces is still credited" { + ADAPTER="$TMPDIR/CLAUDE.md" + printf ' @AGENTS.md\n' > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_success +} + +# --- Q2: encodings that decode as valid UTF-8 but are not UTF-8 --------------- + +@test "a BOM-less UTF-16LE adapter is an encoding error, not a missing reference" { + ADAPTER="$TMPDIR/CLAUDE.md" + python3 -c "import sys; open(sys.argv[1], 'wb').write('@AGENTS.md\n'.encode('utf-16-le'))" "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 2 + assert_output --partial "not valid UTF-8" + refute_output --partial "no reference" +} + +@test "a BOM-less UTF-32LE adapter is an encoding error, not a missing reference" { + ADAPTER="$TMPDIR/CLAUDE.md" + python3 -c "import sys; open(sys.argv[1], 'wb').write('@AGENTS.md\n'.encode('utf-32-le'))" "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 2 + assert_output --partial "not valid UTF-8" + refute_output --partial "no reference" +} + +@test "a doubled UTF-8 BOM does not hide the @import line" { + ADAPTER="$TMPDIR/CLAUDE.md" + python3 -c "import sys; open(sys.argv[1], 'wb').write(('@AGENTS.md\n').encode('utf-8'))" "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_success +} + +@test "a BOM in front of a mid-file @import line does not hide it" { + ADAPTER="$TMPDIR/CLAUDE.md" + python3 -c "import sys; open(sys.argv[1], 'wb').write(('# Claude notes\n\n@AGENTS.md\n').encode('utf-8'))" "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_success +} + +# --- Q3: a file that exists but cannot be read is not a FAIL ----------------- + +@test "an adapter that exists but cannot be read exits 3 with a diagnostic and no FAIL" { + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@AGENTS.md" > "$ADAPTER" + chmod 000 "$ADAPTER" + # chmod is not enough under a uid that bypasses it (root in CI containers). + # /proc/self/mem is a regular file whose read returns EIO for every uid, so + # it exercises the same branch where chmod cannot. + if cat "$ADAPTER" >/dev/null 2>&1; then + if [ -e /proc/self/mem ]; then + ADAPTER=/proc/self/mem + else + chmod 644 "$TMPDIR/CLAUDE.md" + skip "no way to make a readable-by-stat, unreadable-by-open file here" + fi + fi + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + chmod 644 "$TMPDIR/CLAUDE.md" + assert_failure 3 + assert_output --partial "could not be read" + refute_output --partial "FAIL" +} + +# --- Q4: the reference has to name, and resolve to, a real AGENTS.md --------- + +@test "an @import naming a path that does not exist is not credited" { + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@docs/does/not/exist/AGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" + assert_output --partial "does not exist" +} + +@test "@NOTAGENTS.md and @zzzAGENTS.md are not imports of AGENTS.md" { + # The decoys are real files, so the on-disk resolution check cannot be what + # rejects them. Only the path-segment boundary can — without the fixtures + # this test passes against a substring match and proves nothing. + cp "$AGENTS_MD" "$TMPDIR/NOTAGENTS.md" + cp "$AGENTS_MD" "$TMPDIR/zzzAGENTS.md" + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@NOTAGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" + + echo "@zzzAGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "an @import resolving to a zero-byte AGENTS.md is not credited" { + SUB="$TMPDIR/empty" + mkdir -p "$SUB" + : > "$SUB/AGENTS.md" + ADAPTER="$SUB/CLAUDE.md" + echo "@AGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" + assert_output --partial "empty" +} + +@test "an @import naming a real relative path to AGENTS.md is credited" { + mkdir -p "$TMPDIR/docs" + cp "$AGENTS_MD" "$TMPDIR/docs/AGENTS.md" + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@docs/AGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" + assert_success +} + +# --- Q5: --no-import-syntax needs a pointer, not a mention ------------------- + +@test "with --no-import-syntax, a negated mention of AGENTS.md is not a pointer" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +Do NOT read AGENTS.md; it is obsolete. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "with --no-import-syntax, a past-tense mention of a deleted AGENTS.md is not a pointer" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +We deleted AGENTS.md last year. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "with --no-import-syntax, a pointer inside a code fence is not credited" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +Example of what to write: + +``` +See AGENTS.md at the repo root for shared conventions. +``` +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "with --no-import-syntax, a pointer inside an HTML comment is not credited" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +# Copilot instructions + + +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "with --no-import-syntax, a name merely ending in AGENTS.md is not a pointer to it" { + # Real decoy files, so the on-disk resolution check cannot be what rejects + # these — only the token boundary in the mention pattern can. zzzAGENTS.md + # is the load-bearing case: dropping the boundary from NOTAGENTS.md leaves + # the fragment "NOT" behind, which the negation cue then rejects for an + # unrelated reason, so that case alone would prove nothing. + cp "$AGENTS_MD" "$TMPDIR/zzzAGENTS.md" + cp "$AGENTS_MD" "$TMPDIR/NOTAGENTS.md" + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +See zzzAGENTS.md at the repo root for shared conventions. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" + + cat > "$ADAPTER" <<'EOF' +See NOTAGENTS.md at the repo root for shared conventions. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} + +@test "with --no-import-syntax, a pointer naming a path that does not exist is not credited" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +See docs/does/not/exist/AGENTS.md for shared conventions. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" + assert_output --partial "does not exist" +} + +# --- argument handling ------------------------------------------------------- + +@test "a third positional argument is rejected instead of silently ignored" { + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@AGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" "$ADAPTER" "$AGENTS_MD" "$TMPDIR/also-not-graded.md" + assert_failure 2 + assert_output --partial "exactly 2 positional arguments" + # The usage text this prints mentions the word FAIL, so refute the shape of + # a real finding line rather than the bare word. + refute_output --partial "FAIL Adapter" +} + +@test "an unknown option is reported as an unknown option, not as a missing file" { + ADAPTER="$TMPDIR/CLAUDE.md" + echo "@AGENTS.md" > "$ADAPTER" + run bash "$SCRIPT" --bogus "$ADAPTER" "$AGENTS_MD" + assert_failure 2 + assert_output --partial "unknown option '--bogus'" + refute_output --partial "'--bogus' is not a file" + refute_output --partial "FAIL Adapter" +} + +@test "--max-lines=N is accepted in the equals form" { + ADAPTER="$TMPDIR/CLAUDE.md" + { + echo "@AGENTS.md" + for i in $(seq 1 10); do echo "Provider-specific line $i unrelated to AGENTS.md content."; done + } > "$ADAPTER" + run bash "$SCRIPT" --max-lines=5 "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "thin" + + run bash "$SCRIPT" --max-lines=40 "$ADAPTER" "$AGENTS_MD" + assert_success +} + +@test "with --no-import-syntax, a bare mention with no deference cue is not a pointer" { + ADAPTER="$TMPDIR/copilot-instructions.md" + cat > "$ADAPTER" <<'EOF' +# Copilot instructions + +This repo also has an AGENTS.md. + +Prefer inline suggestions over chat for one-line edits. +EOF + run bash "$SCRIPT" --no-import-syntax "$ADAPTER" "$AGENTS_MD" + assert_failure 1 + assert_output --partial "no reference" +} diff --git a/plugins/core/skills/provider-adapter-author/SKILL.md b/plugins/core/skills/provider-adapter-author/SKILL.md index 97f7e16..254196e 100644 --- a/plugins/core/skills/provider-adapter-author/SKILL.md +++ b/plugins/core/skills/provider-adapter-author/SKILL.md @@ -30,8 +30,8 @@ Then confirm `AGENTS.md` exists at the repo root. If it does not, stop and tell Read the provider file and `AGENTS.md` side by side. Separate the provider file's content into two buckets: lines that restate what `AGENTS.md` already owns (universal rules, conventions, project overview) versus lines that are genuinely provider-specific (tool syntax, IDE behavior, model-specific instructions). Rewrite the provider file: -- **Providers with import syntax** (Claude Code): replace the redundant bucket with an `@AGENTS.md` (or correct relative path) import on a line of its own, keep the provider-specific bucket below it. An import folded into a sentence is not the thin-adapter shape and `scripts/validate-adapter.sh` will not credit it. -- **Providers without import syntax** (Cursor, Copilot, etc.): replace the redundant bucket with a short pointer sentence mentioning `AGENTS.md`, keep the provider-specific bucket. +- **Providers with import syntax** (Claude Code): replace the redundant bucket with an `@AGENTS.md` (or correct relative path) import on a line of its own, keep the provider-specific bucket below it. An import folded into a sentence is not the thin-adapter shape and `scripts/validate-adapter.sh` will not credit it — nor one inside a code fence, an indented block, or an HTML comment, nor one whose path does not resolve to a real, non-empty file on disk. +- **Providers without import syntax** (Cursor, Copilot, etc.): replace the redundant bucket with a short sentence pointing at `AGENTS.md` ("See AGENTS.md at the repo root for ..."), keep the provider-specific bucket. A bare or negated mention is not a pointer and will not be credited. The provider file is the only file this skill ever writes. Never create or edit `AGENTS.md` — not in this step, not in any step, whatever the payoff looks like. @@ -45,7 +45,7 @@ Run the bundled check before finishing — this is the skill's own closeout gate bash scripts/validate-adapter.sh [--no-import-syntax] [--max-lines N] ``` -Fix any `FAIL` by editing the provider file, and re-run until it exits `0`. Exit `2` is not a `FAIL`: it means the invocation or the input is wrong — a bad or missing argument, or a file that is not UTF-8 — so fix that, not the adapter. +Fix any `FAIL` by editing the provider file, and re-run until it exits `0`. Exits `2` and `3` are not `FAIL`s and nothing was graded under either, so neither is a reason to touch the adapter: `2` means the invocation or the input is wrong (a bad, missing, or extra argument, an unknown option, or a file that is not UTF-8), and `3` means a named file exists but could not be read. ## Step 4 — Report diff --git a/plugins/core/skills/provider-adapter-author/scripts/README.md b/plugins/core/skills/provider-adapter-author/scripts/README.md index d137d2b..49c4242 100644 --- a/plugins/core/skills/provider-adapter-author/scripts/README.md +++ b/plugins/core/skills/provider-adapter-author/scripts/README.md @@ -4,6 +4,25 @@ Deterministic self-check this skill shells out to instead of relying on LLM judg | File | Purpose | |------|---------| -| `validate-adapter.sh` | Checks a rewritten provider file (CLAUDE.md, etc.) has a reference to AGENTS.md, doesn't duplicate its content, and stays under a thin-file line threshold | +| `validate-adapter.sh` | Checks a rewritten provider file (CLAUDE.md, etc.) has a working reference to AGENTS.md, doesn't duplicate its content, and stays under a thin-file line threshold | -Takes ` `, with optional `--no-import-syntax` and `--max-lines N` flags. Prints `FAIL` findings to stdout and exits non-zero on any failure. +Takes exactly ` `, with optional `--no-import-syntax` and `--max-lines N` flags (also accepted as `--max-lines=N`). A third positional argument or an unknown option is an error, not something quietly ignored. + +## What counts as a reference to AGENTS.md + +Both modes require the named path to be a real path segment ending in `AGENTS.md` — `AGENTS.md` or `…/AGENTS.md`, not `NOTAGENTS.md` — that resolves on disk, relative to the adapter file, to a non-empty file. An adapter deferring to a path that is not there defers to nothing, so the check has to touch the disk rather than pattern-match the line. + +A reference only counts where something would actually resolve it. A line inside a fenced code block, an indented code block, or an HTML comment is not credited in either mode: Claude Code resolves an import in none of those, so a fenced `@AGENTS.md` is the silent-drop failure this gate exists to catch, not a pass. + +Default mode wants a real import: `@AGENTS.md` alone on its own line, indented no more than three spaces. `--no-import-syntax` wants a prose pointer that reads as one — the sentence naming `AGENTS.md` must carry a deference cue (see, read, refer to, documented in, conventions, …) and must not be negated. `Do NOT read AGENTS.md; it is obsolete.` and `We deleted AGENTS.md last year.` name the file while pointing the reader away from it, and neither is a pointer. + +## Exit codes + +The distinction matters because the skill's closeout tells the agent to fix any non-zero exit by editing the provider file. That is right for exactly one of these. + +| Code | Meaning | What to do | +|------|---------|------------| +| `0` | Passes every check | Nothing | +| `1` | One or more `FAIL` findings printed to stdout — empty adapter, no working reference to AGENTS.md, excessive duplication, or not thin | Edit the provider file | +| `2` | Usage or input error: a bad, missing, or extra argument, an unknown option, a path that is not a file, or a file that is not UTF-8. Nothing was graded, so there is no `FAIL` line | Fix the invocation or the file's encoding — do not edit the adapter | +| `3` | A named input file exists but could not be read (permissions, I/O error). Nothing was graded and the adapter's contents are unknown | Fix the file's readability — do not edit the adapter | diff --git a/plugins/core/skills/provider-adapter-author/scripts/validate-adapter.sh b/plugins/core/skills/provider-adapter-author/scripts/validate-adapter.sh index 66ba72c..9648442 100755 --- a/plugins/core/skills/provider-adapter-author/scripts/validate-adapter.sh +++ b/plugins/core/skills/provider-adapter-author/scripts/validate-adapter.sh @@ -14,6 +14,10 @@ Arguments: adapter-file Path to the provider-specific file to check. agents-md-file Path to the AGENTS.md file it should defer to. + Exactly two positional arguments are accepted. Extra ones are rejected + rather than ignored: a third path silently graded nothing but the first + two, so a typo'd invocation passed against the wrong file. + Options: --no-import-syntax The target provider has no native cross-file import mechanism. Require a plain-text pointer line naming @@ -26,26 +30,69 @@ Options: it's considered no longer "thin". Must be a non-negative integer. Default: 60. --help, -h Show this help and exit 0. + -- End of options; every later argument is positional. + +Both flags also accept the --flag=value form (--max-lines=40). An unknown +option is reported as an unknown option, not as a missing file. + +What counts as a reference: + + In both modes the named path must be a real path segment ending in + AGENTS.md ("AGENTS.md" or ".../AGENTS.md" — not NOTAGENTS.md), and it must + resolve on disk, relative to the adapter file, to a non-empty file. An + adapter deferring to a path that is not there defers to nothing. + + A mention inside a fenced code block, an indented code block, or an HTML + comment is not credited in either mode. Nothing resolves those, so an + adapter whose only "import" is fenced silently defers to nothing. + + With --no-import-syntax the pointer must read as a pointer: the sentence + naming AGENTS.md has to carry a deference cue (see, read, refer to, + documented in, conventions, ...) and must not be a negation ("do not read + AGENTS.md", "we deleted AGENTS.md"). A bare mention is not a pointer. Exit codes: 0 Adapter file passes all checks 1 One or more checks failed (empty file, no reference to AGENTS.md, excessive duplication, or file too long) - 2 Usage or input error — a bad or missing argument, a path that is not a - file, or a file that is not UTF-8. Nothing was graded, so there is no - FAIL line and no adapter edit to make: fix the invocation or the file's - encoding and re-run. Kept distinct from 1 because the skill's own - closeout tells the agent to fix every non-zero exit by editing the - provider file, which for a mistyped flag edits the wrong file forever. + 2 Usage or input error — a bad, missing, or extra argument, an unknown + option, a path that is not a file, or a file that is not UTF-8. Nothing + was graded, so there is no FAIL line and no adapter edit to make: fix + the invocation or the file's encoding and re-run. Kept distinct from 1 + because the skill's own closeout tells the agent to fix every non-zero + exit by editing the provider file, which for a mistyped flag edits the + wrong file forever. + 3 A named input file exists but could not be read (permissions, a + directory swapped in mid-run, I/O error). Also not a FAIL: nothing was + graded and the adapter's contents are unknown, so editing it is + guesswork. Fix the file's readability and re-run. EOF } NO_IMPORT_SYNTAX=0 MAX_LINES=60 ARGS=() +END_OF_OPTS=0 + +require_int() { + # $1 = the value to validate + if [[ ! "$1" =~ ^[0-9]+$ ]]; then + echo "Error: --max-lines expects a non-negative integer, got '$1'." >&2 + exit 2 + fi +} while [[ $# -gt 0 ]]; do + if [[ $END_OF_OPTS -eq 1 ]]; then + ARGS+=("$1") + shift + continue + fi case "$1" in + --) + END_OF_OPTS=1 + shift + ;; --help|-h) usage exit 0 @@ -54,17 +101,37 @@ while [[ $# -gt 0 ]]; do NO_IMPORT_SYNTAX=1 shift ;; + --no-import-syntax=*) + echo "Error: --no-import-syntax is a flag and takes no value (got '$1')." >&2 + exit 2 + ;; --max-lines) if [[ $# -lt 2 ]]; then echo "Error: --max-lines requires a value (a non-negative integer)." >&2 exit 2 fi MAX_LINES="$2" - if [[ ! "$MAX_LINES" =~ ^[0-9]+$ ]]; then - echo "Error: --max-lines expects a non-negative integer, got '$MAX_LINES'." >&2 + require_int "$MAX_LINES" + shift 2 + ;; + --max-lines=*) + MAX_LINES="${1#--max-lines=}" + if [[ -z "$MAX_LINES" ]]; then + echo "Error: --max-lines requires a value (a non-negative integer)." >&2 exit 2 fi - shift 2 + require_int "$MAX_LINES" + shift + ;; + -*) + # Reported as an unknown option rather than falling through to the + # positional bucket, where it used to surface as "'--bogus' is not a + # file" — the right exit code attached to a diagnostic that sends the + # reader looking for a path they never typed. + echo "Error: unknown option '$1'." >&2 + echo "" >&2 + usage >&2 + exit 2 ;; *) ARGS+=("$1") @@ -80,6 +147,13 @@ if [[ ${#ARGS[@]} -lt 2 ]]; then exit 2 fi +if [[ ${#ARGS[@]} -gt 2 ]]; then + echo "Error: expected exactly 2 positional arguments (adapter-file and agents-md-file), got ${#ARGS[@]}: ${ARGS[*]}." >&2 + echo "" >&2 + usage >&2 + exit 2 +fi + python3 -u - "${ARGS[0]}" "${ARGS[1]}" "$NO_IMPORT_SYNTAX" "$MAX_LINES" <<'PYTHON' import sys import os @@ -89,45 +163,80 @@ adapter_path, agents_md_path, no_import_syntax, max_lines = sys.argv[1:5] no_import_syntax = no_import_syntax == "1" max_lines = int(max_lines) +EXIT_FAIL = 1 +EXIT_USAGE = 2 +EXIT_UNREADABLE = 3 + if not os.path.isfile(adapter_path): print(f"Error: '{adapter_path}' is not a file.", file=sys.stderr) - sys.exit(2) + sys.exit(EXIT_USAGE) if not os.path.isfile(agents_md_path): print(f"Error: '{agents_md_path}' is not a file.", file=sys.stderr) - sys.exit(2) + sys.exit(EXIT_USAGE) def read_text(path): - r"""File contents as text, UTF-8, BOM stripped. + r"""File contents as text, UTF-8, every BOM stripped. - The BOM strip is not cosmetic. IMPORT_RE anchors on `^\s*@`, and a BOM is - not `\s` in Python, so a CLAUDE.md saved by an editor that emits one had - its first line — the `@AGENTS.md` import, which is the whole adapter — - silently treated as prose. The check then said "no reference to AGENTS.md" - told the author to add the line already sitting in front of them. Same - class of silent BOM miss recorded in scripts/skill-size-check.sh; strip it - at the reader so no later check has to know about it. + The BOM strip is not cosmetic. IMPORT_RE anchors on `^ {0,3}@`, and a BOM + is not whitespace in Python, so a CLAUDE.md saved by an editor that emits + one had its first line — the `@AGENTS.md` import, which is the whole + adapter — silently treated as prose. The check then said "no reference to + AGENTS.md" and told the author to add the line already sitting in front of + them. Same class of silent BOM miss recorded in scripts/skill-size-check.sh; + strip it at the reader so no later check has to know about it. + + Every U+FEFF goes, not just one at offset 0. Stripping exactly the first + one left the mirror-image false FAIL for a doubled BOM (two concatenated + files, or a tool that re-adds one) and for a BOM mid-file at the head of + the import line. U+FEFF has no meaning as a character in a markdown + instruction file, so removing all of them cannot lose signal. Decoding is strict, not errors="replace". Replacement mangles the file and the checks then grade the mangling: a UTF-16 adapter whose first line is `@AGENTS.md` decoded to interleaved NULs and failed as "no reference", which is a true FAIL for a false reason and points the fix at the wrong - thing. A file this gate cannot read gets an encoding diagnostic and exit 2, + thing. But strict UTF-8 alone does not catch it — BOM-less UTF-16LE/BE and + UTF-32LE are *valid* UTF-8, because NUL is a legal code point, so they + decoded clean and produced exactly that false diagnosis anyway. The NUL + byte is the complete signal and is checked first: no plausible markdown + adapter contains one, and every UTF-16/32 encoding of ASCII is full of + them. A file this gate cannot read gets an encoding diagnostic and exit 2, the same policy the ADR-0020 validators' read_text() uses. + + A file that exists but cannot be read at all is neither a pass nor a FAIL — + nothing was graded — so it exits 3 rather than 1. Exit 1 sends the skill's + closeout into "fix the FAIL by editing the provider file", which for a file + it cannot open is an instruction to edit blind. """ try: - with open(path, encoding="utf-8") as fh: - text = fh.read() + with open(path, "rb") as fh: + raw = fh.read() + except OSError as exc: + print(f"Error: '{path}' exists but could not be read ({exc.strerror}). " + "Nothing was checked — fix whatever is blocking the read " + "(permissions, ownership, the underlying device) and re-run; do " + "not edit the adapter on the strength of this.", file=sys.stderr) + sys.exit(EXIT_UNREADABLE) + if b"\x00" in raw: + print(f"Error: '{path}' is not valid UTF-8 — it contains NUL bytes, so " + "it is almost certainly UTF-16 or UTF-32 (with or without a BOM). " + "Re-save it as UTF-8; this check does not guess at other " + "encodings.", file=sys.stderr) + sys.exit(EXIT_USAGE) + try: + text = raw.decode("utf-8") except UnicodeDecodeError as exc: print(f"Error: '{path}' is not valid UTF-8 ({exc.reason} at byte " f"{exc.start}) — re-save it as UTF-8; this check does not guess " "at other encodings.", file=sys.stderr) - sys.exit(2) - return text[1:] if text.startswith("\ufeff") else text + sys.exit(EXIT_USAGE) + return text.replace("\ufeff", "") adapter_content = read_text(adapter_path) agents_md_content = read_text(agents_md_path) +adapter_dir = os.path.dirname(os.path.abspath(adapter_path)) has_fail = False @@ -136,34 +245,229 @@ if not adapter_content.strip(): print(" Why: An empty adapter carries no reference to AGENTS.md and no provider-specific content.") print(" Fix: Add at least an import (or text pointer) to AGENTS.md.") print() - sys.exit(1) + sys.exit(EXIT_FAIL) -IMPORT_RE = re.compile(r'(?m)^\s*@\S*AGENTS\.md\s*$') -lines = adapter_content.splitlines() -import_lines = [ln for ln in lines if IMPORT_RE.match(ln)] -# A prose pointer is any line naming AGENTS.md that is not itself an import -# line — an inert `@AGENTS.md` in a provider that resolves no imports points -# a reader at nothing. -pointer_lines = [ln for ln in lines if not IMPORT_RE.match(ln) and "AGENTS.md" in ln] + +# --- Inert regions ----------------------------------------------------------- +# +# A reference only counts where something would actually resolve it. Fenced +# code blocks, indented code blocks and HTML comments are shown to the reader +# (or hidden from them) as literal text; Claude Code resolves an @import in +# none of them. Without this, a ```-fenced `@AGENTS.md` — the exact +# copy-the-example-into-the-file mistake this gate exists to catch — exited 0 +# with the adapter deferring to nothing. +# +# Indented code blocks are handled by IMPORT_RE's `^ {0,3}` instead of by the +# mask: four leading spaces is what opens an indented code block in CommonMark, +# so an import has to sit within three. The mask deliberately does not apply +# that rule to prose pointers, where four-space indentation is ordinary list +# continuation rather than code. +FENCE_RE = re.compile(r'^( {0,3})(`{3,}|~{3,})(.*)$') +COMMENT_RE = re.compile(r'|\Z)', re.DOTALL) + + +def line_offsets(text): + """[(char offset, line without its terminator)] over `text`.""" + out = [] + off = 0 + for raw in text.splitlines(keepends=True): + out.append((off, raw.rstrip("\r\n"))) + off += len(raw) + return out + + +def build_inert_mask(text, offsets): + """Per-character flags: 1 where a reference would never be resolved.""" + mask = bytearray(len(text)) + fence = None # (fence char, opening run length) + for start, line in offsets: + m = FENCE_RE.match(line) + if fence is None: + if m: + fence = (m.group(2)[0], len(m.group(2))) + for i in range(start, start + len(line)): + mask[i] = 1 + continue + for i in range(start, start + len(line)): + mask[i] = 1 + if (m and m.group(2)[0] == fence[0] + and len(m.group(2)) >= fence[1] + and not m.group(3).strip()): + fence = None + for m in COMMENT_RE.finditer(text): + if m.start() < len(mask) and mask[m.start()]: + continue # a literal "