diff --git a/CONTEXT.md b/CONTEXT.md index 5bbbd08..4a220bb 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -81,6 +81,13 @@ topic docs and a `sources.md`; the author skill records which sources informed w and internally consistent. _Avoid_: sources, citations, attribution +**Research registry**: +A plugin's research `sources.md` (e.g. `plugins/git/docs/research/docs/git/sources.md`), whose `## H2` +headings are the source slugs. A skill's `Research doc:` field names exactly one, and +`factory-audit` resolves each entry's slug against it. An entry with no registry declares +`Research doc: none` and names what it was actually drawn from in `Basis:`. +_Avoid_: research doc, sources file, topic doc (a topic doc is a digest of sources, not the registry) + ### Governance **HITL** (human-in-the-loop): diff --git a/docs/adr/0028-research-doc-names-the-research-registry.md b/docs/adr/0028-research-doc-names-the-research-registry.md new file mode 100644 index 0000000..70557ea --- /dev/null +++ b/docs/adr/0028-research-doc-names-the-research-registry.md @@ -0,0 +1,136 @@ +# `Research doc:` names one Research registry; entries without one declare `none` and a `Basis:` + +**Status: accepted (2026-09-21).** Resolves #121. Extends ADR-0004's INFO level: it keeps INFO for +the case where a check cannot run and promotes the case where it ran and found a mismatch. + +Each entry in a skill's `references/sources.md` carries a `Research doc:` field. The spec +(`skill-author/references/create.md`) says it names the plugin's research `sources.md`, the file +whose `## H2` headings are the source slugs. The corpus did something else: 29 of 30 mismatched +entries pointed at a research topic doc annotated `(whole-document reference)`, and 6 values were not +a single path (comma-separated lists and shell brace expansion, plus a semicolon pair in +`gitea-releases`). Checks 7 and 8 of `validate-provenance.sh` look the slug up as an H2 in the named +file, so 36 entries reported INFO and nothing failed. Measured by running the script over all 39 skill +directories, since nothing else runs it over the corpus. + +We decided that `Research doc:` names exactly one **Research registry** (the term is in +`CONTEXT.md`), as the spec always said. Slug-to-H2 lookup in the registry is the only provenance link +that can be verified deterministically; a topic doc has no per-source H2 to check against. A link to +the topic doc that digested a source stays as free-text annotation and is not checked. + +## Considered options + +**Q1 — what `Research doc:` refers to.** + +- **(a) The Research registry (chosen).** Check 7 stays as designed (check 8 is retired, see Q6); the + 29 entries repoint mechanically. +- **(b) The topic docs a source fed into (rejected).** Matches what the authors wrote, and is arguably + the more useful pointer for a reader. Rejected because it changes the spec and the checker, and the + slug check has nothing to run against. +- **(c) Both, as two fields (rejected).** Doubles the schema for a link nobody gates on. + +**Q2 — how an entry with no registry declares that honestly.** + +- **(a) `Research doc: none` plus a `Basis:` field (chosen).** `Basis:` takes repeated bullets of + repo paths (ADRs, `core/instructions/*.md`, a live example) and is checked for existence only. + `research_doc_is_none` already parses `none`, and `git-workflow` already writes it. Same shape as + #111: there was no honest way to declare the truthful thing. +- **(b) A non-corpus path stays legal in `Research doc:` (rejected).** Leaves one field meaning two + things depending on its value, and the INFO it produces can never be cleared. +- **(c) Move non-corpus entries out of `sources.md` (rejected).** A larger restructure than the + issue warrants. + +Lists are not needed under Q1(a): the four `pc-author` and `pc-run` brace expansions are one +registry, and the `gitea-releases` pair collapses to one registry. Brace expansion and semicolon +pairs are rejected outright, since nothing expands them in a markdown field. + +**Q3 — tier once the grammar is settled.** + +- **(b) FAIL when the path resolves and check 7 finds a mismatch; INFO when the path does not + resolve (chosen).** Check 8 is not part of this: see Q6. A topic doc in `Research doc:` is now + simply wrong and is a FAIL. An + unresolvable path stays INFO because `skill-file-structure.md` treats `sources.md` pointers as + development-time, and a deployed copy of a skill outside this repo will not have the research docs. + This repo's own corpus is audited from the authoring source, where every path resolves. +- **(a) Everything stays INFO (rejected).** Under ADR-0004 INFO implies no action, which is how 36 + mismatches went unnoticed. +- **(c) Everything FAIL (rejected).** Fails a correctly-provenanced skill audited from a deployed + copy, which the file-structure exemption exists to prevent. + +**Q6 — what happens to check 8.** Found unsatisfiable during the migration, after Q3 was decided. +Check 8 requires every `extracted` slug in the research doc to appear in the skill's `sources.md`. +That worked while entries pointed at topic docs, and was dormant. Under Q1(a) the named file is a +registry shared by many skills (`git/sources.md` backs seven), and nothing ties a registry slug to one +skill, so every skill would fail permanently. The direction that matters, that each slug a skill lists +exists in the registry, is already check 7. + +- **(a) Retire check 8 (chosen).** Check 7 is the FAIL. Under registry semantics check 8 has no + satisfiable meaning. +- **(b) Keep it as an INFO (rejected).** Recreates the noise ADR-0004 warns about: an observation with + no action that every skill emits forever. +- **(c) Redefine it as a registry-side coverage report (rejected for now).** "Registry slugs that no + skill uses" is a coherent check, but it is a report across all skills and separate work from this + issue. + +**Q7 — `Basis:` paths that no longer exist.** Found in the same migration: `git-commits` and +`git-workflow` cite `core/instructions/git.md` and `commits.md`, deleted in `5deed07`. An existence +check on every `Basis:` bullet would fail them. + +- **(a) A bullet annotated `(removed in )` skips the existence check (chosen).** The check stays + for live paths, which is what catches a renamed ADR, and deletion becomes an explicit, auditable + annotation. Weakness: the annotation can be written on any bullet to avoid the check. Verifying the + sha with `git cat-file -e` would close that, and was left out as over-engineering for three bullets. +- **(b) `Basis:` becomes free prose with no existence check (rejected).** Gives up the one check that + catches a renamed or moved ADR. +- **(c) Drop those `Basis:` lines and keep `none` with a prose reason (rejected).** Loses the + machine-readable record of what the entry was drawn from. + +Form: one path per bullet, `- **Basis:** ` repeated, not a header with sub-bullets. + +**Q8 — the `lint` entry with no verifiable basis.** `house-vale-3-15-2-repro` in `vale-config` and +`vale-run` said `none` and claimed six behaviours were "established by running it against purpose-built +fixtures in this repo". No such fixture or test exists in the tree or in history: the entry was added +in `d1afdbe` with no test files, and the only vale test ever deleted (`4de5b6b`) guards an unrelated +`E100`. Under Q2 it FAILed for a missing `Basis:`. + +- **(e) Remove the entry and its `source_keys` citations (chosen).** The stated basis was false, so + there is nothing honest to declare. The behavioural rules stay in the skills; only the provenance + claim goes. The gate needs no allowlist. +- **(a) `Basis: tests/test-vale-wrap.sh` (rejected).** Backs about one of six claims and overstates the + rest. +- **(b) Commit reproduction fixtures (rejected for now).** The right fix if the behaviours matter, but + separate work from this issue. +- **(c) Allow `none` without `Basis:` for "house-verified" entries (rejected).** Reopens Q2 and gives + an escape hatch for unverified claims. +- **(d) Keep the entry and allowlist the two skills in the gate (rejected).** Keeps a false claim in + place and adds a list that can rot. + +`configuration-reference.md` still says its rows were "reproduced against Vale 3.15.2"; that wording +now has no provenance entry behind it and is left for a separate decision. + +**Q4 — enforcement.** A corpus-wide sweep gate lands in the same change: a test or pre-push hook that +runs `validate-provenance.sh` over every `plugins/*/.apm/skills/*/` and fails on any FAIL. Deferring it +was rejected because without a caller the FAIL tier is inert; nothing but `check-scope-walkup-sync.sh` +(on fixtures) invokes the validator today. + +**Q5 — parser parity.** `parse_status` and `parse_research_doc` accept the bullet spelling +(`- **Status:**`) as `parse_contributing_files` already does, with a regression test. Included because +it is the same failure shape as #111 and #118 (a parser returns "nothing found", the caller reads it as +"nothing declared"), sits in the same file, and `gitea-releases` already writes the unhyphenated form. + +## Consequences + +- About 40 `references/sources.md` entries migrate: roughly 30 repoint from a topic doc to the registry, + about 4 move to `Research doc: none` with a `Basis:` list (`provider-adapter-author`, + `git-commits` `org-commit-conventions`, `agentsmd-audit` `governance-secrets-hard-prohibition`, + `git-workflow`), and the `gitea-releases` pair collapses to one path. +- `Basis:` is a new field: `create.md` step 6, `skill-file-structure.md` and the validator's usage text + must state it, and the validator must check that each listed path exists, except a bullet annotated + `(removed in )`. Each `Basis:` path is one bullet. +- Check 7 gains a FAIL tier for resolved-path mismatches. INFO remains for a path that does not + resolve. A topic doc named in `Research doc:` is no longer legal: it is a FAIL, since a topic doc has + no per-source `## H2` to check the slug against. +- Check 8 is retired: remove it from `lib-provenance-skill.sh`, its usage text and the tests, and drop + its mention from `skill-file-structure.md` and `create.md` where present. +- The corpus-wide sweep is a new gate: register it in `docs/spec/gates.md` and + `.pre-commit-config.yaml`. The corpus must be migrated in the same change or the suite goes red. +- Reversing this means re-migrating the same entries, which is why it is recorded.