diff --git a/plugins/bin/.apm/skills/research/META.md b/plugins/bin/.apm/skills/research/META.md deleted file mode 100644 index 824f4c3..0000000 --- a/plugins/bin/.apm/skills/research/META.md +++ /dev/null @@ -1,15 +0,0 @@ -```yaml -version: "1.1" -updated: 2026-06-21 - -when: >- - Invoked when the user wants to gather structured reference documentation for a - tool, library, or API from MCP documentation indexes or web sources. Typically - run before writing a new skill that wraps an external tool, or any time - reference files are needed for a topic. Triggered explicitly - ("/research ") or implicitly when the user asks to look up, - gather, or pull docs for a topic before implementing something. - -references: - - .agents/skills/context7-mcp/SKILL.md # context7-mcp — MCP source channel integrated at step 2 -``` diff --git a/plugins/bin/.apm/skills/research/SKILL.md b/plugins/bin/.apm/skills/research/SKILL.md index 1453ddb..2ab6459 100644 --- a/plugins/bin/.apm/skills/research/SKILL.md +++ b/plugins/bin/.apm/skills/research/SKILL.md @@ -1,14 +1,10 @@ --- name: research description: >- - Use when the user wants to research a topic and generate structured reference - markdown files. Handles: finding canonical docs for a tool/library/API via - Context7 MCP or web sources, reading and deepening into linked pages, - organizing extracted content into topic files (overview, installation, - configuration, cli-reference, api-reference, examples, troubleshooting). Do - NOT use when the user wants to write documentation from existing code or specs - (use write-docs), install or manage the neuledge-context MCP server (use - neuledge-context), or research a bug/incident (use diagnose). + Use when the user wants a tool, library, or API researched from canonical + documentation into structured per-topic reference markdown files. Not + documentation written from existing code or specs -> `write-docs`. Not a bug + or incident -> `diagnose`. metadata: category: research allowed-tools: @@ -21,77 +17,48 @@ allowed-tools: model: sonnet --- - +## Gotchas -## Required inputs +- Never infer the output path. A run writes a directory's worth of files, and a guessed destination scatters them through someone's source tree. If the user named no path, stop and ask. +- Write nothing outside the given output path. A file placed beside the agreed directory is one the user never asked for and will not think to look for. +- Never write an empty topic file. A stub `troubleshooting.md` reads downstream as researched and closed. +- A Context7 response that is a "no results" message, a redirect notice, or header-only boilerplate is not coverage. A topic area counts as covered only when the response carries at least one substantive paragraph. -- **Topic** — the subject to research (tool, library, API, concept); inferred from user description if clear, ask if ambiguous -- **Output path** — directory where reference files will be written; must be provided explicitly — do not infer or default -- **Starting URLs** — optional; if provided, skip discovery websearch and read these first +## Step 1 — Scope against the working directory -## Constraints +Search for existing use of the topic — imports, config files, version pins, reference files already written — and narrow the research to what is missing: the version actually in use, the topics not yet documented. -- Never write files outside the explicitly provided output path -- Skip any default topic file if no relevant content is found for it — do not create empty files -- Create additional topic files beyond the default list when content warrants it (e.g. `webhooks.md`, `rate-limits.md`) -- Subagents handle parallel source reading and link deepening — the orchestrator writes all files; subagents return summaries only, never write directly -- Context7 MCP calls (`resolve-library-id`, `query-docs`) are made only by the orchestrator at step 2 — subagents must not call them -- `sources.md` is always written, even if only one source was read -- Each topic file must have frontmatter with `topic` and `source_keys`; body is prose only — no inline URLs -- Source keys in `sources.md` must be kebab-case slugs: derived from the source domain or page title for web sources; for Context7 sources use `context7-` (e.g. `context7-vercel-next-js`) -- Default topic list and file format spec live in `references/` sub-files — read them at step 1 +Read `references/topics.md` before narrowing, for the default topic list. - +## Step 2 — Resolve against Context7 - +If the topic is a library, framework, or API and the user gave no starting URLs, call `resolve-library-id` with the topic name and the user's full question — match quality depends on the question, not the bare name — then `query-docs` once per default topic area. Record each response as a source with slug `context7-`, and mark which topic areas it covered — those skip the web reads at step 4. -## Process +If the library does not resolve, or the user gave starting URLs, go to step 3. Explicit URLs are a source choice; do not second-guess them with a resolution attempt. -1. **Scan codebase.** Search the working directory for existing usage of the topic — imports, config files, version pins, existing reference files. Use findings to narrow research scope (e.g. target the version already in use, skip topics already documented). Read `references/topics.md` for the default topic list and `references/file-format.md` for the output file format spec. +## Step 3 — Discover sources -2. **Try Context7.** If the topic is a library, framework, or API and no starting URLs were provided, call `resolve-library-id` with the topic name and the user's question. If a match resolves, call `query-docs` once per default topic area (see `references/topics.md`). Treat each response as a source summary with slug `context7-` (e.g. `context7-vercel-next-js`). A topic area has sufficient content when the Context7 response contains at least one substantive paragraph — not a "no results" message, redirect notice, or header-only boilerplate. Mark covered topic areas — skip their subagent web reads in step 4. If the library does not resolve, or starting URLs were provided (explicit source choice by the user), skip this step entirely. +If the user gave starting URLs, skip discovery: those URLs are the source list and go straight to step 4. -3. **Discover sources.** For topics not covered by Context7 (or when no starting URLs were provided and Context7 did not resolve), websearch for canonical documentation (prefer `llms.txt`, developer docs, official API references over tutorials or blog posts). Collect 3–5 candidate URLs before reading any. +Otherwise, for every topic area Context7 did not cover, websearch for canonical documentation — `llms.txt`, official developer docs, and API references ahead of tutorials or blog posts. Collect three to five candidate URLs before reading any of them. -4. **Read sources in parallel.** Spawn one subagent per source URL. Each subagent fetches the page, extracts relevant content, identifies links worth deepening, and returns a structured summary (content by topic area + links to follow). Subagents do not write files. +If nothing usable comes back, stop and report what was searched, then ask for starting URLs rather than settling for tutorials. -5. **Deepen.** For each subagent that returned links worth following, spawn child subagents per branch. Continue until content becomes repetitive or out of scope. Cap at ~10 additional pages total across all branches. +## Step 4 — Read the sources -6. **Consolidate.** Merge all subagent summaries (Context7 and web) by topic area. Identify which default topics have sufficient content and which custom topics emerged. +`WebFetch` each URL in turn. No subagent tool is granted here, so the reads are serial and every fetched page lands in this context: reduce each page to notes by topic area, plus the links worth deepening, before fetching the next one. -7. **Write topic files.** For each topic with content, write `/.md` using the format in `references/file-format.md`. Orchestrator writes all files — never delegate file writing to a subagent. +## Step 5 — Deepen -8. **Write `sources.md`.** Write `/sources.md` mapping each source slug to its URL (use `context7:` as the URL for Context7 sources), description, and list of topic files it contributed to. Include sources that yielded no content, marked `no content extracted`. +`WebFetch` the links worth following, still one at a time and still reducing each page to notes. Stop a branch once its content turns repetitive or leaves the topic, and cap the whole step at roughly ten additional pages — serial reads make that cap a real budget, not a formality. -## Output format +## Step 6 — Write -- `/.md` per topic with content — formatted per `references/file-format.md` -- `/sources.md` — always produced; maps slug → URL, description, contributing files +Merge every set of notes, Context7 and web alike, by topic area. Read `references/file-format.md`, then write, in the output path: - +- `.md` for each topic area that has content, default or custom +- `sources.md`, always, one section per source in the schema that file gives — URL, description, contributing files, and status — including sources that yielded nothing, marked `no content extracted` - +Spell the `sources.md` field names exactly as `references/file-format.md` gives them. The downstream provenance validator matches them literally; prose in their place parses as nothing, and the check passes having verified nothing. -## Failure handling - -- Output path not provided — stop and ask; do not infer or default -- No sources found after websearch — report what was searched, ask user to provide starting URLs -- Subagent returns no usable content — skip that source, log in `sources.md` as `no content extracted` -- All topic files would be empty — stop, report what was searched, do not write any files - -## Self-check - -- [ ] Codebase scanned before any websearch was performed -- [ ] Output path was explicitly provided — not inferred -- [ ] `references/topics.md` and `references/file-format.md` read at step 1 -- [ ] Context7 resolution attempted before websearch when topic is a library/framework/API -- [ ] Context7 calls made only at orchestrator step 2 — no subagent called `resolve-library-id` or `query-docs` -- [ ] Context7 sources recorded in `sources.md` with `context7:` as URL -- [ ] No topic file written without content -- [ ] `sources.md` written with all sources read (including those with no content extracted) -- [ ] All file writes performed by the orchestrator, not subagents -- [ ] Each topic file has `topic` and `source_keys` frontmatter fields -- [ ] All source keys in topic files have a matching entry in `sources.md` -- [ ] No files written outside the provided output path - - +If no topic area has content, write nothing at all, `sources.md` included, and report what was searched. A directory of empty files is worse than an honest miss. diff --git a/plugins/bin/evals/research/research/eval.yaml b/plugins/bin/evals/research/research/eval.yaml index 1f402f1..39ad9c6 100644 --- a/plugins/bin/evals/research/research/eval.yaml +++ b/plugins/bin/evals/research/research/eval.yaml @@ -26,11 +26,6 @@ trigger_tests: query: "Research why these integration tests are failing" should_trigger: false - - id: negative-neuledge - name: "Negative — MCP server setup goes to neuledge-context" - query: "Install the neuledge context server and set it up" - should_trigger: false - - id: negative-context7-direct-question name: "Negative — direct doc question goes to context7-mcp, not research" query: "What are the Next.js middleware options?" diff --git a/plugins/bin/skills/research/META.md b/plugins/bin/skills/research/META.md deleted file mode 100644 index 824f4c3..0000000 --- a/plugins/bin/skills/research/META.md +++ /dev/null @@ -1,15 +0,0 @@ -```yaml -version: "1.1" -updated: 2026-06-21 - -when: >- - Invoked when the user wants to gather structured reference documentation for a - tool, library, or API from MCP documentation indexes or web sources. Typically - run before writing a new skill that wraps an external tool, or any time - reference files are needed for a topic. Triggered explicitly - ("/research ") or implicitly when the user asks to look up, - gather, or pull docs for a topic before implementing something. - -references: - - .agents/skills/context7-mcp/SKILL.md # context7-mcp — MCP source channel integrated at step 2 -``` diff --git a/plugins/bin/skills/research/SKILL.md b/plugins/bin/skills/research/SKILL.md index 1453ddb..2ab6459 100644 --- a/plugins/bin/skills/research/SKILL.md +++ b/plugins/bin/skills/research/SKILL.md @@ -1,14 +1,10 @@ --- name: research description: >- - Use when the user wants to research a topic and generate structured reference - markdown files. Handles: finding canonical docs for a tool/library/API via - Context7 MCP or web sources, reading and deepening into linked pages, - organizing extracted content into topic files (overview, installation, - configuration, cli-reference, api-reference, examples, troubleshooting). Do - NOT use when the user wants to write documentation from existing code or specs - (use write-docs), install or manage the neuledge-context MCP server (use - neuledge-context), or research a bug/incident (use diagnose). + Use when the user wants a tool, library, or API researched from canonical + documentation into structured per-topic reference markdown files. Not + documentation written from existing code or specs -> `write-docs`. Not a bug + or incident -> `diagnose`. metadata: category: research allowed-tools: @@ -21,77 +17,48 @@ allowed-tools: model: sonnet --- - +## Gotchas -## Required inputs +- Never infer the output path. A run writes a directory's worth of files, and a guessed destination scatters them through someone's source tree. If the user named no path, stop and ask. +- Write nothing outside the given output path. A file placed beside the agreed directory is one the user never asked for and will not think to look for. +- Never write an empty topic file. A stub `troubleshooting.md` reads downstream as researched and closed. +- A Context7 response that is a "no results" message, a redirect notice, or header-only boilerplate is not coverage. A topic area counts as covered only when the response carries at least one substantive paragraph. -- **Topic** — the subject to research (tool, library, API, concept); inferred from user description if clear, ask if ambiguous -- **Output path** — directory where reference files will be written; must be provided explicitly — do not infer or default -- **Starting URLs** — optional; if provided, skip discovery websearch and read these first +## Step 1 — Scope against the working directory -## Constraints +Search for existing use of the topic — imports, config files, version pins, reference files already written — and narrow the research to what is missing: the version actually in use, the topics not yet documented. -- Never write files outside the explicitly provided output path -- Skip any default topic file if no relevant content is found for it — do not create empty files -- Create additional topic files beyond the default list when content warrants it (e.g. `webhooks.md`, `rate-limits.md`) -- Subagents handle parallel source reading and link deepening — the orchestrator writes all files; subagents return summaries only, never write directly -- Context7 MCP calls (`resolve-library-id`, `query-docs`) are made only by the orchestrator at step 2 — subagents must not call them -- `sources.md` is always written, even if only one source was read -- Each topic file must have frontmatter with `topic` and `source_keys`; body is prose only — no inline URLs -- Source keys in `sources.md` must be kebab-case slugs: derived from the source domain or page title for web sources; for Context7 sources use `context7-` (e.g. `context7-vercel-next-js`) -- Default topic list and file format spec live in `references/` sub-files — read them at step 1 +Read `references/topics.md` before narrowing, for the default topic list. - +## Step 2 — Resolve against Context7 - +If the topic is a library, framework, or API and the user gave no starting URLs, call `resolve-library-id` with the topic name and the user's full question — match quality depends on the question, not the bare name — then `query-docs` once per default topic area. Record each response as a source with slug `context7-`, and mark which topic areas it covered — those skip the web reads at step 4. -## Process +If the library does not resolve, or the user gave starting URLs, go to step 3. Explicit URLs are a source choice; do not second-guess them with a resolution attempt. -1. **Scan codebase.** Search the working directory for existing usage of the topic — imports, config files, version pins, existing reference files. Use findings to narrow research scope (e.g. target the version already in use, skip topics already documented). Read `references/topics.md` for the default topic list and `references/file-format.md` for the output file format spec. +## Step 3 — Discover sources -2. **Try Context7.** If the topic is a library, framework, or API and no starting URLs were provided, call `resolve-library-id` with the topic name and the user's question. If a match resolves, call `query-docs` once per default topic area (see `references/topics.md`). Treat each response as a source summary with slug `context7-` (e.g. `context7-vercel-next-js`). A topic area has sufficient content when the Context7 response contains at least one substantive paragraph — not a "no results" message, redirect notice, or header-only boilerplate. Mark covered topic areas — skip their subagent web reads in step 4. If the library does not resolve, or starting URLs were provided (explicit source choice by the user), skip this step entirely. +If the user gave starting URLs, skip discovery: those URLs are the source list and go straight to step 4. -3. **Discover sources.** For topics not covered by Context7 (or when no starting URLs were provided and Context7 did not resolve), websearch for canonical documentation (prefer `llms.txt`, developer docs, official API references over tutorials or blog posts). Collect 3–5 candidate URLs before reading any. +Otherwise, for every topic area Context7 did not cover, websearch for canonical documentation — `llms.txt`, official developer docs, and API references ahead of tutorials or blog posts. Collect three to five candidate URLs before reading any of them. -4. **Read sources in parallel.** Spawn one subagent per source URL. Each subagent fetches the page, extracts relevant content, identifies links worth deepening, and returns a structured summary (content by topic area + links to follow). Subagents do not write files. +If nothing usable comes back, stop and report what was searched, then ask for starting URLs rather than settling for tutorials. -5. **Deepen.** For each subagent that returned links worth following, spawn child subagents per branch. Continue until content becomes repetitive or out of scope. Cap at ~10 additional pages total across all branches. +## Step 4 — Read the sources -6. **Consolidate.** Merge all subagent summaries (Context7 and web) by topic area. Identify which default topics have sufficient content and which custom topics emerged. +`WebFetch` each URL in turn. No subagent tool is granted here, so the reads are serial and every fetched page lands in this context: reduce each page to notes by topic area, plus the links worth deepening, before fetching the next one. -7. **Write topic files.** For each topic with content, write `/.md` using the format in `references/file-format.md`. Orchestrator writes all files — never delegate file writing to a subagent. +## Step 5 — Deepen -8. **Write `sources.md`.** Write `/sources.md` mapping each source slug to its URL (use `context7:` as the URL for Context7 sources), description, and list of topic files it contributed to. Include sources that yielded no content, marked `no content extracted`. +`WebFetch` the links worth following, still one at a time and still reducing each page to notes. Stop a branch once its content turns repetitive or leaves the topic, and cap the whole step at roughly ten additional pages — serial reads make that cap a real budget, not a formality. -## Output format +## Step 6 — Write -- `/.md` per topic with content — formatted per `references/file-format.md` -- `/sources.md` — always produced; maps slug → URL, description, contributing files +Merge every set of notes, Context7 and web alike, by topic area. Read `references/file-format.md`, then write, in the output path: - +- `.md` for each topic area that has content, default or custom +- `sources.md`, always, one section per source in the schema that file gives — URL, description, contributing files, and status — including sources that yielded nothing, marked `no content extracted` - +Spell the `sources.md` field names exactly as `references/file-format.md` gives them. The downstream provenance validator matches them literally; prose in their place parses as nothing, and the check passes having verified nothing. -## Failure handling - -- Output path not provided — stop and ask; do not infer or default -- No sources found after websearch — report what was searched, ask user to provide starting URLs -- Subagent returns no usable content — skip that source, log in `sources.md` as `no content extracted` -- All topic files would be empty — stop, report what was searched, do not write any files - -## Self-check - -- [ ] Codebase scanned before any websearch was performed -- [ ] Output path was explicitly provided — not inferred -- [ ] `references/topics.md` and `references/file-format.md` read at step 1 -- [ ] Context7 resolution attempted before websearch when topic is a library/framework/API -- [ ] Context7 calls made only at orchestrator step 2 — no subagent called `resolve-library-id` or `query-docs` -- [ ] Context7 sources recorded in `sources.md` with `context7:` as URL -- [ ] No topic file written without content -- [ ] `sources.md` written with all sources read (including those with no content extracted) -- [ ] All file writes performed by the orchestrator, not subagents -- [ ] Each topic file has `topic` and `source_keys` frontmatter fields -- [ ] All source keys in topic files have a matching entry in `sources.md` -- [ ] No files written outside the provided output path - - +If no topic area has content, write nothing at all, `sources.md` included, and report what was searched. A directory of empty files is worse than an honest miss. diff --git a/tests/test-adr0020-targets.sh b/tests/test-adr0020-targets.sh index f4bf7af..8fd7d69 100755 --- a/tests/test-adr0020-targets.sh +++ b/tests/test-adr0020-targets.sh @@ -362,35 +362,47 @@ cp -R "$REPO_ROOT/plugins" "$FRESH_ROOT/plugins" [[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$FRESH_ROOT/apm.yml" FRESH_DANGLING="$(dangling_set "$FRESH_ROOT/plugins")" -DEPLOYED_ROOT="$TMPDIR_T/deployed-clone" -mkdir -p "$DEPLOYED_ROOT/.claude/skills" "$DEPLOYED_ROOT/.claude/agents" -cp -R "$REPO_ROOT/plugins" "$DEPLOYED_ROOT/plugins" -[[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$DEPLOYED_ROOT/apm.yml" -# Deploy exactly the names that currently dangle. That is the strongest possible -# bait: if the deployed tree were consulted, every one of them would resolve and -# the dangling set would collapse to empty. -DEPLOY_COUNT=0 -while IFS= read -r name; do - [[ -n "$name" ]] || continue - mkdir -p "$DEPLOYED_ROOT/.claude/skills/$name" - DEPLOY_COUNT=$((DEPLOY_COUNT + 1)) -done <<< "$FRESH_DANGLING" -DEPLOYED_DANGLING="$(dangling_set "$DEPLOYED_ROOT/plugins")" +# The bait used to be DERIVED from the corpus: deploy exactly the names that +# currently dangle. That was the strongest bait available while the corpus had +# dangling names — and it silently became vacuous the moment issue #99 fixed the +# last one, because a corpus reporting nothing gives nothing to deploy. A test of +# "deployed trees do not leak" must not depend on the corpus staying broken. +# +# So the bait is now EXPLICIT. Both contrast copies get one synthetic skill whose +# boundary clause routes to a name guaranteed absent from the monorepo, and only +# the deployed copy gets that name planted in .claude/skills/. If deployed trees +# leaked into the resolver's universe, the deployed copy would resolve it and +# report an empty set while the fresh copy reported one. The A/B now distinguishes +# something on every run, forever, whatever the corpus does. +BAIT_NAME="no-such-deployed-only-skill" +BAIT_FRESH="$TMPDIR_T/bait-fresh" +BAIT_DEPLOYED="$TMPDIR_T/bait-deployed" +for bait_root in "$BAIT_FRESH" "$BAIT_DEPLOYED"; do + mkdir -p "$bait_root" + cp -R "$REPO_ROOT/plugins" "$bait_root/plugins" + [[ -f "$REPO_ROOT/apm.yml" ]] && cp "$REPO_ROOT/apm.yml" "$bait_root/apm.yml" + write_skill "$bait_root/plugins/bin/.apm/skills/deployed-tree-probe" deployed-tree-probe \ + "Use when doing the probe thing. Do not use for the other thing — use $BAIT_NAME instead." +done +# Only the deployed copy gets the name planted where `apm install` would put it. +mkdir -p "$BAIT_DEPLOYED/.claude/skills/$BAIT_NAME" "$BAIT_DEPLOYED/.claude/agents" +BAIT_FRESH_DANGLING="$(dangling_set "$BAIT_FRESH/plugins")" +BAIT_DEPLOYED_DANGLING="$(dangling_set "$BAIT_DEPLOYED/plugins")" -if [[ "$DEPLOY_COUNT" -gt 0 ]]; then - pass "precondition: $DEPLOY_COUNT dangling name(s) deployed into the contrast tree's .claude/skills/, so the A/B has something to distinguish" +if [[ "$BAIT_FRESH_DANGLING" == *"$BAIT_NAME"* ]]; then + pass "precondition: the bait target dangles in the un-deployed copy, so the A/B has something to distinguish" else - fail "no dangling names to deploy — the corpus reports none, so this A/B distinguishes nothing. Deploy a known-absent name explicitly instead of deriving one." + fail "the bait target '$BAIT_NAME' does not dangle even without a deployed tree — the fixture is broken, so the contrast below proves nothing. Got: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')]" fi -if [[ ! -d "$FRESH_ROOT/.claude" && ! -d "$FRESH_ROOT/.agents" ]]; then +if [[ ! -d "$BAIT_FRESH/.claude" && ! -d "$BAIT_FRESH/.agents" ]]; then pass "precondition: the fresh-clone copy has no deployed tree of its own" else fail "the fresh-clone copy picked up a deployed tree — it is not a fresh-clone fixture" fi -if [[ "$FRESH_DANGLING" == "$DEPLOYED_DANGLING" ]]; then - pass "deploying every dangling name into .claude/skills/ changes nothing: $(echo "$FRESH_DANGLING" | tr '\n' ' ')" +if [[ "$BAIT_FRESH_DANGLING" == "$BAIT_DEPLOYED_DANGLING" ]]; then + pass "planting the dangling name in .claude/skills/ changes nothing: $(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')" else - fail "the corpus verdict depends on whether apm install has been run — fresh clone: [$(echo "$FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$DEPLOYED_DANGLING" | tr '\n' ' ')]" + fail "the verdict depends on whether apm install has been run — fresh clone: [$(echo "$BAIT_FRESH_DANGLING" | tr '\n' ' ')] with a deployed tree: [$(echo "$BAIT_DEPLOYED_DANGLING" | tr '\n' ' ')]. A deployed .claude/skills/ tree is leaking into the resolver's universe." fi # Third data point: whatever state THIS machine happens to be in, the live tree # must agree with a bare copy of the same plugins/. No precondition on that state @@ -417,37 +429,33 @@ fi # terminal and therefore danglable. The issue #99 retrofit cut that composition # sentence and the dangling target went with it, so the set is down to one. # -# WHEN `research` IS RETROFITTED: drop neuledge-context and leave the set empty. -# Do not delete the assertion — an empty expected set is fine and still pins -# that no NEW dangling target appeared. +# `neuledge-context` was the last one. The issue #99 wave-3 retrofit deleted that +# boundary clause outright — commit `6146120` had already deleted the skill it +# named, and nothing has owned MCP-server installation since — so the corpus +# dangling set is now EMPTY. +# +# The assertion stays, and it is not vacuous now that it expects nothing: it is +# the only thing standing between a newly-authored boundary clause naming a +# non-existent target and a green suite. An empty expected set pins "no NEW +# dangling target appeared", which is the property that actually matters from +# here on. +# +# The per-target probe loop that used to sit below is GONE, not emptied. Its job +# was to prove the check detects each live dangling target individually, and with +# no live targets left there is nothing to point it at. A loop over an empty list +# is an assertion-free result counted in the totals — exactly the vacuous-pass +# shape the comment above rejects. Detection is still covered, and covered +# better, by the synthetic fixtures in section 2 below, which build a real plugin +# tree and assert the resolver fires. Do not reinstate the loop unless a real +# dangling target reappears in the corpus. echo "" -echo "--- the live dangling targets in the corpus are exactly the ADR-0020 records still open ---" -EXPECTED_DANGLING="$(printf '%s\n' neuledge-context)" +echo "--- no skill in the corpus routes to a target that does not resolve ---" +EXPECTED_DANGLING="" if [[ "$LIVE_DANGLING" == "$EXPECTED_DANGLING" ]]; then - pass "the corpus dangling set is exactly {neuledge-context}" + pass "the corpus dangling set is empty" else - fail "the corpus dangling set changed — expected [$(echo "$EXPECTED_DANGLING" | tr '\n' ' ')], got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. If a retrofit fixed one, update EXPECTED_DANGLING; if a false-positive fix silently deleted one, that is the regression this asserts." + fail "a dangling routing target appeared in the corpus — expected none, got [$(echo "$LIVE_DANGLING" | tr '\n' ' ')]. A boundary clause names a skill or agent that does not resolve; fix the clause or the target. This assertion is the corpus-wide backstop, so do not relax it to make a new skill pass." fi -# shellcheck disable=SC2043 # one probe left by design -- the list shrinks as -# each fixture is retrofitted and reaches zero when `research` lands. Keeping the -# loop means removing the last entry is a one-line edit, not a restructure. -for probe in \ - "plugins/bin/.apm/skills/research/SKILL.md:neuledge-context"; do - probe_file="$REPO_ROOT/${probe%%:*}" - probe_name="${probe##*:}" - if [[ ! -f "$probe_file" ]]; then - fail "the true-positive fixture ${probe%%:*} no longer exists — this pin has become vacuous" - continue - fi - set +e - probe_out="$(bash "$HOOK" "$probe_file" 2>&1)" - set -e - if [[ "$probe_out" == *"routes to '$probe_name'"* ]]; then - pass "detects the dangling '$probe_name' target in ${probe%%:*}" - else - fail "did not detect the dangling '$probe_name' target in ${probe%%:*} — a false-positive fix has taken a true positive with it: $probe_out" - fi -done # --------------------------------------------------------------------------- # 2. The bare-target grammar rule diff --git a/tests/test-skill-size-check.sh b/tests/test-skill-size-check.sh index 6356551..3423850 100755 --- a/tests/test-skill-size-check.sh +++ b/tests/test-skill-size-check.sh @@ -534,10 +534,12 @@ expect_gate "a fixture with no authoring root reports DID NOT RUN and exits 0" \ "Use when doing the thing. Do not use for improvements — use some-other-skill instead." 10)" \ "Unchecked target(s): some-other-skill" -echo "" -echo "--- the live dangling routing targets are caught (issue #100) ---" +# NOTE: this section prints no header and runs no assertions any more — see why +# below. The commentary is kept because it records why probes are removed rather +# than skipped, which is the rule the next person to touch this file needs. +# # ADR-0020 records the broken routing targets and splits fixing them into its own -# issue. This asserts the gate actually sees them rather than the check being +# issue. This asserted the gate actually sees them rather than the check being # vacuous in the corpus it was written against. # # There used to be a third probe here, for `skill-improve` in skill-audit's @@ -556,29 +558,26 @@ echo "--- the live dangling routing targets are caught (issue #100) ---" # The gitea-labels probe was dropped when the issue #99 retrofit cut the # composition sentence whose YAML fold produced that target. Per the rule above # it is removed, not skipped. -# shellcheck disable=SC2043 # one probe left by design -- the list shrinks as -# each fixture is retrofitted and reaches zero when `research` lands. Keeping the -# loop means removing the last entry is a one-line edit, not a restructure. -for probe in \ - "plugins/bin/.apm/skills/research/SKILL.md:neuledge-context"; do - probe_file="$REPO_ROOT/${probe%%:*}" - probe_name="${probe##*:}" - if [[ ! -f "$probe_file" ]]; then - fail "the probe fixture ${probe%%:*} no longer exists — this pin has become vacuous; update it and EXPECTED_DANGLING in tests/test-adr0020-targets.sh together" - continue - fi - # Captured, not piped: the script exits non-zero on these files and - # `set -o pipefail` would make the whole pipeline non-zero regardless of what - # grep found. - set +e - probe_out="$("$SCRIPT" "$probe_file" 2>&1)" - set -e - if [[ "$probe_out" == *"routes to '$probe_name'"* ]]; then - pass "detects the dangling '$probe_name' target in ${probe%%:*}" - else - fail "did not detect the dangling '$probe_name' target in ${probe%%:*}. If issue #100 retrofitted it, drop this probe and update EXPECTED_DANGLING in tests/test-adr0020-targets.sh; if a false-positive fix took a true positive with it, that is the regression this asserts." - fi -done +# +# The `neuledge-context` probe — the last one — went the same way in wave 3 of +# that retrofit, which deleted the boundary clause naming it. **The corpus now +# has zero dangling targets**, so this loop is removed entirely rather than left +# to iterate over an empty list. +# +# That is deliberate and follows the rule stated above. A loop over no probes +# produces no assertion while still returning success, which is the vacuous-pass +# shape this comment block exists to reject — it would make the suite look one +# test stronger than it is, exactly the complaint levelled at the old +# `skill-improve` SKIP branch. +# +# Nothing is lost. This file only ever checked that each member of the live +# dangling set is individually detected; tests/test-adr0020-targets.sh remains +# the authority on the set itself, and now pins it as EMPTY, which is what +# catches a newly-authored clause naming a target that does not resolve. That +# file also carries synthetic fixtures built inside a real plugin tree, which +# exercise the detection path without depending on the corpus staying broken. +# +# If a real dangling target ever reappears, add its probe back here. echo "" echo "Results: $PASS passed, $FAIL failed"