diff --git a/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md index 99bb81d..52da19d 100644 --- a/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md @@ -50,11 +50,15 @@ on-disk check. Flag any other spelling of a cross-skill reference. Two directories are exempt, and the exemptions are structural rather than discretionary: -- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers, - not runtime references. They are expected to be unresolvable after install, so - `validate-provenance.sh` does not treat an absent path as a FAIL — it emits an INFO naming the - slug and stating that checks 7 and 8 did not run for it. Flagging them as broken references - would make every correctly-provenanced skill fail. +- **`references/sources.md`.** Its `Research doc:` and `Basis:` fields are development-time + provenance pointers, not runtime references. A `Research doc:` path that does not resolve after + install is expected, so `validate-provenance.sh` does not treat an absent path as a FAIL — it + emits an INFO naming the slug and stating that check 7 did not run for it. Flagging them as + broken references would make every correctly-provenanced skill fail. Where the path DOES + resolve, it is checked: `Research doc:` names exactly one Research registry (a `sources.md` + whose H2 headings are the source slugs), and a slug missing from it, a topic document in its + place, or a list of paths is a FAIL. An entry with no registry writes `Research doc: none` plus + `Basis:` repo paths, which are existence-checked unless annotated `(removed in )`. - **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared `tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/` exists and `tests/README.md` is absent or does not document it, that is a FAIL. diff --git a/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh index f9649e3..160e34d 100755 --- a/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh @@ -77,12 +77,13 @@ Checks performed: 4 Contributing files back-reference the parent slug in their source_keys 5 Research doc field present and not placeholder -Agent mode has no counterpart to skill mode's checks 6, 7 and 8 (Research -doc field / upstream forward / upstream reverse are numbered 6, 7, 8 there and -5 here): an agent at plugin scope is a single file with a plugin-root -sources.md, so there is no references/ tree to walk and no upstream research -source index to cross-check. parse_status() and the sources.md-basename gate -that those checks need exist only in lib-provenance-skill.sh. +Agent mode has no counterpart to skill mode's checks 6 and 7 (Research doc +field / slug in the Research registry are numbered 6 and 7 there, and the field +check is 5 here): an agent at plugin scope is a single file with a plugin-root +sources.md, so there is no references/ tree to walk and no Research registry to +cross-check. The sources.md-basename gate and the Basis: check that those checks +need exist only in lib-provenance-skill.sh. Skill mode's check 8 is retired +(ADR-0028). EOF } diff --git a/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh index f9fc8a1..25bb0b3 100755 --- a/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh @@ -63,12 +63,25 @@ Checks performed: read is reported as an INFO saying checks 4 and 5 did not run, never skipped silently. 5 Contributing files back-reference the parent slug in their source_keys - 6 Research doc field present and not placeholder - 7 Slug in sources.md present in upstream research doc (INFO only). A section + 6 Research doc field present and not a placeholder, and exactly ONE path — the Research registry, a plugin's + research sources.md whose H2 headings are the source slugs. A brace + expansion, a comma-separated list, a semicolon-separated pair and a + repeated '- **Research doc:**' line are each a FAIL. An entry with no + registry writes 'Research doc: none' (a trailing annotation after an em + dash is fine) and names what it was drawn from in '- **Basis:**', one + repo path per bullet; a missing Basis, or a Basis path that does not + exist, is a FAIL. A Basis bullet annotated '(removed in )' skips + the existence check. + 7 Slug in sources.md present in the Research registry (FAIL). A section annotation ('§ ...', '→ ...', '(...)') is stripped before the path is - resolved; a path that still does not resolve is reported as an INFO saying - checks 7 and 8 did not run, never skipped silently. - 8 Extracted non-(none) slug in research doc present in sources.md + resolved. A path that does not resolve, or no repo root above the skill + directory, is reported as an INFO saying check 7 did not run, never + skipped silently. A Research doc that resolves to a file NOT named + sources.md (a topic document) is a FAIL. + 8 (retired — #121) The reverse check, "every extracted slug in the research + doc appears in this skill's sources.md", could not be satisfied when one + registry serves many skills. The number is left vacant so check 9 keeps + the name the rest of the repo cites. 9 Description or Contributing files text changed since --base-ref (INFO only — a bash script cannot verify the claim is still TRUE, only that it changed; the auditor reads the named files to check that). Wrapped values @@ -79,11 +92,10 @@ Checks performed: or references/sources.md is not tracked under this path at that ref, this is announced as ONE INFO for the whole check, never a silent skip. - Checks 7 and 8 apply ONLY when the Research doc value names a research SOURCE - INDEX — a file whose basename is sources.md, whose H2 headings ARE source - slugs. A Research doc pointing at a topic document is reported as an INFO - saying the two checks are not applicable, and every other reason they do not - run is announced the same way. + Check 7 applies to a Research doc that names a Research registry — a file + whose basename is sources.md, whose H2 headings ARE source slugs. A topic + document is a FAIL, not a value the check skips, and every other reason it + does not run is announced as an INFO. EOF } @@ -344,25 +356,73 @@ KYBERFORGE_PROV_SKILL_PREAMBLE_PY="${KYBERFORGE_PROV_SKILL_PREAMBLE_PY%$'\n'}" IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_BODY_PY <<'KYBERFORGE_PROV_SKILL_BODY' || true -def parse_research_docs(content, slug): - """Every Research doc value under a given slug H2, in document order. - - The caller uses the first and reports the rest. Returning only the first — - what this did before — meant a second '- **Research doc:**' line in one - entry was silently ignored, so an author who added a doc rather than - replacing one got checks 7 and 8 run against the old path and no hint that - the new one was never looked at. - """ +def _entry_block(content, slug): + """The text under a '## slug' heading, or None when there is no such entry.""" pattern = re.compile( r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', re.MULTILINE | re.DOTALL ) m = pattern.search(content) - if not m: + return m.group(1) if m else None + +def parse_field_values(content, slug, label): + """Every value of a '**label:**' field under a slug H2, in document order. + + The SPELLING of a field must not decide whether it is read. Three + spellings are in the corpus and all three are accepted here: + + - **Label:** value (the documented form) + **Label:** value (no leading hyphen — gitea-releases writes Status so) + **Label:** (a header, then '- value' bullets) + - value + + A field parsed by a regex that knew only the first form returned "nothing + found" for the other two, and every caller read that as "nothing declared" + (#121, second comment; the same failure shape as #111 and #118). A header's + bullets stop at the first line that is neither blank nor a bullet, and a + '- **Other:**' bullet is the NEXT field, not a value of this one. + """ + block = _entry_block(content, slug) + if block is None: return [] - block = m.group(1) - return [v.strip() for v in - re.findall(r'^\- \*\*Research doc:\*\* (.+)$', block, re.MULTILINE)] + values = [] + lines = block.splitlines() + label_re = re.compile(r'^(?:- )?\*\*' + re.escape(label) + r':\*\*[ \t]*(.*)$') + i = 0 + while i < len(lines): + m = label_re.match(lines[i]) + i += 1 + if not m: + continue + inline = m.group(1).strip() + if inline: + values.append(inline) + continue + while i < len(lines): + line = lines[i].strip() + if not line: + i += 1 + continue + if not line.startswith('- ') or line.startswith('- **'): + break + values.append(line[2:].strip()) + i += 1 + return values + +def parse_research_docs(content, slug): + """Every Research doc value under a given slug H2, in document order. + + Research doc takes exactly ONE path, so the caller FAILs on a second value + rather than using the first and announcing the rest — an author who added a + doc rather than replacing one otherwise got check 7 run against the + old path and a verdict that looked complete. + """ + return parse_field_values(content, slug, 'Research doc') + +def parse_basis(content, slug): + """Every Basis value under a slug H2 — the repo paths an entry with no + Research registry was actually drawn from, one per bullet.""" + return parse_field_values(content, slug, 'Basis') # A Research doc value is a path, and very often a path PLUS an annotation # naming the section the slug came from: @@ -371,7 +431,7 @@ def parse_research_docs(content, slug): # plugins/git/docs/research/docs/git/remotes.md → `## Pushing (`git push`)` # .../pre-commit/hooks-reference.md § "pre-commit-hooks (official collection)" # -# os.path.isfile() is false for every one of those strings, and checks 7 and 8 +# os.path.isfile() is false for every one of those strings, and check 7 # used to skip SILENTLY whenever the path did not resolve. The effect was that # both checks were dead on eight of the nine git skills — git-history, the one # skill writing a bare path, was the only place they ran, which is why it was @@ -394,57 +454,24 @@ def research_doc_is_none(value): """ return re.match(r'\(?none\b', value.strip(), re.IGNORECASE) is not None -# The Status value is what gates check 8, so every spelling this parser fails -# to read is a check that does not run. Two were unreadable: -# -# - **Status:** `extracted` — partial fetch (a trailing note) -# **Status:** (the bullet form, the same -# - `extracted` shape parse_contributing_files -# already accepts) -# -# Both used to parse to a string that compared unequal to "`extracted`", and -# check 8 skipped on that inequality without a word. Returning the BACKTICKED -# TOKEN — not the whole line — is what makes the trailing note harmless, and it -# lets the caller name the actual status when it announces a skip. -STATUS_TOKEN_RE = re.compile(r'^`([^`]*)`') +# A Research doc or Basis value names ONE path. The three list spellings seen +# in the corpus — a brace expansion, a comma-separated list and a +# semicolon-separated pair — are humans writing "several documents" into a +# single-path field. Nothing expands a brace in a markdown field, and the +# annotation strip above discards everything after the first '(' or section +# marker, so a second path parked after one was NEVER resolved and no check +# said so. Detected on the raw value, with commas and semicolons INSIDE the +# annotation left alone: those are prose ('cross-cutting; no dedicated +# section'), and only a second path-shaped token after a ';' is a list. +SECOND_PATH_AFTER_SEMICOLON_RE = re.compile(r';\s*[\w.\-]+/[\w./\-]*\.[A-Za-z]+') +BASIS_REMOVED_RE = re.compile(r'\(removed in [0-9a-fA-F]{7,40}\b[^)]*\)') -def parse_status(content, slug): - """Find the Status value for a given slug H2 in content. - - Returns the status with its backticks stripped ('extracted', 'referenced', - 'no content extracted'), or None when the entry has no Status line. - """ - pattern = re.compile( - r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', - re.MULTILINE | re.DOTALL - ) - m = pattern.search(content) - if not m: - return None - block = m.group(1) - - raw = None - st_m = re.search(r'^\- \*\*Status:\*\* (.+)$', block, re.MULTILINE) - if st_m: - raw = st_m.group(1).strip() - else: - st_m = re.search(r'^\*\*Status:\*\*\s*$', block, re.MULTILINE) - if not st_m: - return None - for line in block[st_m.end():].splitlines(): - line = line.strip() - if not line: - continue - if not line.startswith("- "): - break - raw = line[2:].strip() - break - if raw is None: - return None - - token = STATUS_TOKEN_RE.match(raw) - return token.group(1).strip() if token else raw +def names_more_than_one_path(value): + path_part = strip_research_doc_annotation(value) + if '{' in path_part or '}' in path_part or ',' in path_part or ';' in path_part: + return True + return SECOND_PATH_AFTER_SEMICOLON_RE.search(value) is not None def find_repo_root(start_dir): """Walk up from start_dir until we find a directory containing .git.""" @@ -460,7 +487,7 @@ def find_repo_root(start_dir): # --- Check 9 helpers --------------------------------------------------- # Check 9 needs a raw field VALUE (as text, to diff against an earlier # version), not the parsed structure parse_contributing_files() and -# parse_status() return. The ONE normalization applied is whitespace +# parse_field_values() return. The ONE normalization applied is whitespace # collapsing, which is what makes a re-wrap or a re-indent invisible; nothing # else is normalized away. # @@ -532,7 +559,7 @@ def parse_field_raw(content, slug, field_name): """Raw text of a '**:**' field under a slug H2, wrapping joined. Mirrors the two authored shapes parse_contributing_files() and - parse_status() already handle (inline value on the same line, or a + parse_field_values() already handle (inline value on the same line, or a bare heading followed by '- ' bullets), but returns text rather than a parsed structure, because check 9 diffs wording, not semantics. @@ -788,11 +815,8 @@ if os.path.isdir(refs_dir): repo_root = find_repo_root(skill_dir) -# Collect all research doc paths we'll check (for Check 8) -research_docs_seen = {} # abs_path → (rel_path, slugs referencing it, content) - # Every per-slug parser below — parse_contributing_files, parse_research_docs, -# parse_status — locates its block with pattern.search(), so a slug written +# parse_basis — locates its block with pattern.search(), so a slug written # twice resolves to the FIRST block every time. Iterating the raw heading list # therefore checked the first block's fields twice and the second block's # never: a duplicated slug is half-validated, and looked fully validated. The @@ -810,7 +834,7 @@ for _slug in all_slugs: f"references/sources.md (## {_slug})", f"'## {_slug}' appears {_count} times. Every field parser here takes the first match, so the " f"second and later blocks' Contributing files, Research doc and Status are never validated — " - f"checks 4, 5, 6, 7 and 8 did not run for them. " + f"checks 4, 5, 6 and 7 did not run for them. " f"Merge the blocks into one entry, or give each a distinct slug and reference it from source_keys." ) @@ -866,13 +890,13 @@ for slug in unique_slugs: # Check 6: Research doc field required rd_values = parse_research_docs(sources_content, slug) if len(rd_values) > 1: - emit_info( - f"Multiple '- **Research doc:**' lines for '{slug}' — only the first is used", + emit_fail( + f"Multiple '- **Research doc:**' lines for '{slug}' — Research doc takes exactly one path", f"references/sources.md (## {slug})", - f"The '## {slug}' entry has {len(rd_values)} Research doc lines; checks 7 and 8 ran against the first " - f"('{rd_values[0]}') and never looked at the rest. " - f"Keep one Research doc line per entry — if a slug genuinely came from two documents, split it into two slugs, " - f"or name the extra document inside the first value's annotation where it is at least visible." + f"The '## {slug}' entry has {len(rd_values)} Research doc lines. Research doc names one Research registry, " + f"so a second line is a list, and a list is not a grammar this field has.", + f"Keep one Research doc line, pointing at the plugin's research sources.md. If the entry has no registry, " + f"write '- **Research doc:** none' and name what it was drawn from in '- **Basis:**', one repo path per bullet." ) rd_value = rd_values[0] if rd_values else None if rd_value is None: @@ -889,7 +913,67 @@ for slug in unique_slugs: f"The '## {slug}' entry has an unfilled Research doc value.", f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable." ) - elif not research_doc_is_none(rd_value): + elif research_doc_is_none(rd_value): + # An entry with no Research registry must still say what it WAS drawn + # from. Basis names repo paths, one per bullet, and each is checked to + # exist — the honest way to record an org convention, an ADR or a + # house-verified reproduction, none of which has a registry entry. + basis_values = parse_basis(sources_content, slug) + if not basis_values: + emit_fail( + f"Basis missing for '{slug}' — Research doc is 'none'", + f"references/sources.md (## {slug})", + f"The '## {slug}' entry declares no Research registry ('{rd_value}') and no '- **Basis:**' line, " + f"so nothing records what the entry was drawn from.", + f"Add '- **Basis:** ' to the '## {slug}' entry, one line per path, naming the ADR, " + f"convention file or reproduction the entry rests on." + ) + for basis in basis_values: + basis_path = strip_research_doc_annotation(basis) + if PLACEHOLDER_RE.search(basis) or not basis_path: + emit_fail( + f"Basis is empty or placeholder for '{slug}'", + f"references/sources.md (## {slug})", + f"The '## {slug}' entry has an unfilled Basis value '{basis}'.", + f"Set '- **Basis:**' to one repo path." + ) + elif names_more_than_one_path(basis): + emit_fail( + f"Basis value names more than one path for '{slug}'", + f"references/sources.md (## {slug})", + f"The Basis value '{basis}' is a brace expansion or a comma- or semicolon-separated list.", + f"Write one '- **Basis:** ' line per path." + ) + elif not repo_root: + emit_info( + f"Basis check skipped for '{slug}' — no repo root above the skill directory", + f"references/sources.md (## {slug})", + f"'{basis}' is a path relative to the repo root, but no ancestor of the skill directory contains a .git entry, " + f"so it cannot be resolved. Run this script against a skill inside a checkout." + ) + elif BASIS_REMOVED_RE.search(basis): + # A path the entry HISTORICALLY rested on, annotated + # '(removed in )', is a declaration that it is gone on + # purpose. The sha is not resolved: the annotation is the + # author saying "deleted, and here is where to look". + continue + elif not os.path.exists(os.path.join(repo_root, basis_path)): + emit_fail( + f"Basis path '{basis_path}' does not exist", + f"references/sources.md (## {slug})", + f"'{basis}' resolves to '{basis_path}' relative to the repo root and nothing is there.", + f"Correct the path, or remove the Basis line if the entry no longer rests on it." + ) + elif names_more_than_one_path(rd_value): + emit_fail( + f"Research doc names more than one path for '{slug}'", + f"references/sources.md (## {slug})", + f"The Research doc value '{rd_value}' is a brace expansion or a comma- or semicolon-separated list. " + f"Research doc names exactly one Research registry.", + f"Point Research doc at the plugin's research sources.md. If the entry has no registry, write " + f"'- **Research doc:** none' and name what it was drawn from in '- **Basis:**', one repo path per bullet." + ) + else: # Check 7: Upstream forward — slug should appear in research doc. # Every path out of here that does NOT run the check says so out loud. rd_path = strip_research_doc_annotation(rd_value) @@ -898,7 +982,7 @@ for slug in unique_slugs: f"Upstream checks skipped for '{slug}' — no repo root above the skill directory", f"references/sources.md (## {slug})", f"'{rd_value}' is a path relative to the repo root, but no ancestor of the skill directory contains a .git entry, " - f"so it cannot be resolved. Checks 7 and 8 did not run for this slug. " + f"so it cannot be resolved. Check 7 did not run for this slug. " f"Run this script against a skill inside a checkout." ) elif not rd_path: @@ -906,8 +990,8 @@ for slug in unique_slugs: f"Upstream checks skipped for '{slug}' — Research doc value names no path", f"references/sources.md (## {slug})", f"The Research doc value '{rd_value}' is entirely annotation — stripping the section marker leaves no path. " - f"Checks 7 and 8 did not run for this slug. " - f"Give the value a file path relative to the repo root, or record '(none)' if no research doc backs this entry." + f"Check 7 did not run for this slug. " + f"Give the value a file path relative to the repo root, or record 'none' plus a '- **Basis:**' if no registry backs this entry." ) else: rd_abs = os.path.join(repo_root, rd_path) @@ -916,33 +1000,26 @@ for slug in unique_slugs: f"Upstream checks skipped for '{slug}' — research doc '{rd_path}' does not exist", f"references/sources.md (## {slug})", f"'{rd_value}' resolves to '{rd_path}' relative to the repo root and no file is there. " - f"Checks 7 and 8 did not run for this slug, so nothing verified that the research doc still backs it. " - f"Point the value at one existing file — a brace expansion, a comma-separated list of paths, or a bare section title does not resolve — " - f"or record '(none)' if no research doc backs this entry." + f"Check 7 did not run for this slug, so nothing verified that the research doc still backs it. " + f"Point the value at the one existing Research registry (the plugin's research sources.md), " + f"or record 'none' plus a '- **Basis:**' if no registry backs this entry." ) elif os.path.basename(rd_path) != "sources.md": - # Checks 7 and 8 both assume the Research doc is a research - # SOURCE INDEX — a sources.md whose H2 headings ARE source - # slugs. 30 of the 121 corpus entries point instead at a TOPIC - # DOCUMENT (remotes.md, gitflow.md, api-reference.md), whose - # H2s are headings like '## Core Philosophy'. A slug can never - # match one, so check 7 reported all 30 as "slug not found" — - # every one a false positive — and check 8, aimed at documents - # that carry no '- **Status:**' line at all, was saved from a - # matching flood of false FAILs only by an UNANNOUNCED skip on - # that missing status. The premise, not the corpus, was wrong. - # - # A topic-document reference is a legitimate, useful value; it - # just is not something these two checks can verify. Say that - # once, out loud, instead of failing 30 entries for it. - emit_info( - f"Upstream checks not applicable for '{slug}' — research doc '{rd_path}' is a topic document, not a source index", + # Check 7 matches slugs against the H2 headings of a + # Research registry — a sources.md whose H2s ARE source slugs. + # A topic document (remotes.md, gitflow.md) has section headings + # for H2s, so no slug can ever match one. Research doc names the + # registry (#121), so a topic document there is the wrong file, + # not a value these checks cannot verify. A pointer to the topic + # document that digested the source belongs in the free-text + # annotation after the path, where it is not checked. + emit_fail( + f"Research doc '{rd_path}' for '{slug}' is a topic document, not a Research registry", f"references/sources.md (## {slug})", - f"Checks 7 and 8 match slugs against the H2 headings of a research source index — a file named 'sources.md', " - f"where each H2 IS a source slug. '{os.path.basename(rd_path)}' is a topic document, so its H2s are section " - f"headings and no slug will ever match one. Checks 7 and 8 did not run for this slug. " - f"This needs no fix: point the value at the research corpus's own sources.md only if you want the " - f"provenance link machine-verified." + f"'{os.path.basename(rd_path)}' is not a sources.md, so its H2s are section headings and no slug can match one. " + f"Research doc names the plugin's Research registry — the sources.md whose H2s are source slugs.", + f"Repoint '{slug}' at the sibling sources.md in '{os.path.dirname(rd_path)}/', and keep the topic document in the " + f"annotation, e.g. ' (digested in {os.path.basename(rd_path)})'." ) else: try: @@ -951,63 +1028,19 @@ for slug in unique_slugs: emit_info( f"Upstream checks skipped for '{slug}' — research doc '{rd_path}' is {exc}", f"references/sources.md (## {slug})", - f"'{rd_path}' could not be decoded, so checks 7 and 8 did not run for this slug. " + f"'{rd_path}' could not be decoded, so check 7 did not run for this slug. " f"Re-save the research doc as UTF-8." ) continue rd_slugs = set(parse_h2_slugs(rd_content)) if slug not in rd_slugs: - emit_info( + emit_fail( f"Slug '{slug}' not found as H2 in research doc '{rd_path}'", f"references/sources.md (## {slug})", - f"The research doc '{rd_path}' does not have a '## {slug}' heading. " - f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading." + f"The Research registry '{rd_path}' does not have a '## {slug}' heading, so the entry's provenance " + f"link resolves to nothing.", + f"Rename the slug to match a '## ' heading in '{rd_path}', or repoint Research doc at the registry that has it." ) - # Track for Check 8. The content is carried with the entry so - # check 8 reuses this read rather than decoding the file a - # second time, with a second chance to fail differently. - if rd_abs not in research_docs_seen: - research_docs_seen[rd_abs] = (rd_path, set(), rd_content) - research_docs_seen[rd_abs][1].add(slug) - -# --- Check 8: Upstream reverse --- -for rd_abs, (rd_rel, known_slugs, rd_content) in research_docs_seen.items(): - for rd_slug in parse_h2_slugs(rd_content): - # Parse this slug's Contributing files and Status in the research doc - rd_cf = parse_contributing_files(rd_content, rd_slug) - rd_status = parse_status(rd_content, rd_slug) - # Skip if the research doc explicitly records no contributing files - if rd_cf == []: - continue - # Skip if status is not `extracted` — and say so when the skip is what - # kept the slug out of the FAIL below. A status of `referenced` or - # `no content extracted` is a real reason not to demand the slug, but - # it was applied in silence, so an entry that should have been in - # sources.md and a status line nobody had updated produced the same - # output: nothing. Only a MATERIAL skip is announced; when the slug is - # already in sources.md the check passes either way and there is no - # fail-open to disclose. - if rd_status != "extracted": - if rd_slug not in sources_slugs: - shown = f"`{rd_status}`" if rd_status else "absent" - emit_info( - f"Check 8 skipped for research-doc slug '{rd_slug}' — its Status is {shown}, not `extracted`", - f"{rd_rel} (## {rd_slug})", - f"'{rd_rel}' has '## {rd_slug}' with contributing files but Status {shown}, and this skill's " - f"sources.md has no '## {rd_slug}' entry. Check 8 only demands an entry for an `extracted` slug, " - f"so it did not run here. If that status is stale — the content was extracted and the line was never " - f"updated — this skill is missing a source entry; if it is accurate, nothing needs doing." - ) - continue - # This slug should be in sources.md - if rd_slug not in sources_slugs: - emit_fail( - f"Research doc slug '{rd_slug}' missing from skill sources.md", - f"references/sources.md", - f"The research doc '{rd_rel}' has '## {rd_slug}' with status `extracted` and contributing files, " - f"but this skill's sources.md has no '## {rd_slug}' entry.", - f"Add '## {rd_slug}' to references/sources.md or mark it as '(none)' in the research doc's Contributing files." - ) # --- Check 9: Description / Contributing files changed since --base-ref --- # A structural fact — the field's TEXT differs from an earlier revision — is diff --git a/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats index 03d8fe3..c7f4a02 100644 --- a/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats @@ -53,6 +53,10 @@ EOF local slug="${2:-my-source}" local contrib="${3:-SKILL.md}" local research="${4:-(none)}" + # A 'none' Research doc must name its Basis (#121); SKILL.md exists in + # every fixture skill and, once commit_as_base has run, is a repo path. + local basis_line="" + if [[ "$research" == *none* ]]; then basis_line="- **Basis:** SKILL.md"; fi mkdir -p "$dir/references" cat > "$dir/references/sources.md" < "$skill/SKILL.md" < "$skill/SKILL.md" < "$fake_repo/docs/research/sources.md" < "$skill/references/sources.md" < "$skill/SKILL.md" < "$fake_repo/docs/research/sources.md" < "$skill/references/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/remotes.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$fake_repo/docs/research/sources.md" < "$skill/references/other.md" @@ -1886,6 +1518,7 @@ EOF - references/other.md (the rubric) - **Research doc:** (none) +- **Basis:** SKILL.md - **Status:** `extracted` EOF printf -- '---\nsource_keys:\n - my-source\n---\n\nnotes\n' > "$skill/references/other.md" @@ -1924,6 +1557,7 @@ EOF mkdir -p "$skill" make_skill_with_source_keys "$skill" make_sources_md "$skill" + sed -i 's#^- \*\*Basis:\*\* SKILL.md#- **Basis:** tracked-skill/SKILL.md#' "$skill/references/sources.md" commit_as_base "$repo" # A copy of the same skill at a path git does not know — the everyday case @@ -2060,3 +1694,257 @@ EOF refute_output "" assert_output --partial "$dir/notes.txt" } + +# --------------------------------------------------------------------------- +# Cycle 28 — #121: `Research doc:` names exactly one Research registry; +# an entry with no registry writes `none` plus `Basis:` +# +# Helper: a fake repo holding one skill whose single entry is written verbatim +# from the fields passed in — make_upstream_skill only varies the Research doc +# value, and the Basis cases need a second field. The registry lives at +# docs/research/sources.md and a basis file at docs/basis.md. +# --------------------------------------------------------------------------- + +make_entry_skill() { + local repo="$1" + local fields="$2" + local skill="$repo/my-skill" + mkdir -p "$skill/references" "$repo/docs/research" + cat > "$skill/SKILL.md" < "$repo/docs/basis.md" + printf '%s\n' "# Other" > "$repo/docs/other-basis.md" + { + printf '# Sources\n\n## my-source\n\n' + printf '%s\n' '- **URL:** https://example.com/my-source' + printf '%s\n' '- **Description:** A test source.' + printf '%s\n' '- **Contributing files:** SKILL.md' + printf '%s\n' "$fields" + printf '%s\n' '- **Status:** `extracted`' + } > "$skill/references/sources.md" + cat > "$repo/docs/research/sources.md" <)' Basis skips the existence check" { + local repo="$TMPDIR/fakerepo" + make_entry_skill "$repo" "$(printf '%s\n%s\n%s' '- **Research doc:** none' '- **Basis:** docs/basis.md' '- **Basis:** docs/deleted-adr.md (removed in 5b80f30)')" + run bash "$SCRIPT" "$repo/my-skill" + assert_success + assert_output "" +} + +@test "#121 FAIL: a Basis annotated with something other than '(removed in )' is still existence-checked" { + local repo="$TMPDIR/fakerepo" + make_entry_skill "$repo" "$(printf '%s\n%s' '- **Research doc:** none' '- **Basis:** docs/deleted-adr.md (removed later)')" + run bash "$SCRIPT" "$repo/my-skill" + assert_failure + assert_output --partial "Basis path 'docs/deleted-adr.md' does not exist" +} + +@test "#121 pass: a Basis with a trailing annotation resolves its path" { + local repo="$TMPDIR/fakerepo" + make_entry_skill "$repo" "$(printf '%s\n%s' '- **Research doc:** none' '- **Basis:** docs/basis.md (org convention file, not a corpus entry)')" + run bash "$SCRIPT" "$repo/my-skill" + assert_success + assert_output "" +} + +@test "#121 FAIL: check 7 — a slug missing from a resolved registry is a FAIL, not an INFO" { + local repo="$TMPDIR/fakerepo" + make_entry_skill "$repo" '- **Research doc:** docs/research/sources.md' + cat > "$repo/docs/research/sources.md" < "$repo/docs/research/remotes.md" + run bash "$SCRIPT" "$repo/my-skill" + assert_failure + assert_output --partial "is a topic document, not a Research registry" + refute_output --partial "not applicable" +} + +@test "#121 INFO: an unresolvable Research doc path stays INFO and exits 0" { + local repo="$TMPDIR/fakerepo" + make_entry_skill "$repo" '- **Research doc:** docs/research/gone/sources.md' + run bash "$SCRIPT" "$repo/my-skill" + assert_success + assert_output --partial "INFO" + assert_output --partial "does not exist" +} + +@test "#121 INFO: a Basis cannot be existence-checked with no repo root, and says so" { + local skill="$TMPDIR/my-skill" + make_skill_with_source_keys "$skill" + mkdir -p "$skill/references" + cat > "$skill/references/sources.md" <)` once the file is deleted). diff --git a/plugins/kyberforge/.apm/skills/skill-author/references/create.md b/plugins/kyberforge/.apm/skills/skill-author/references/create.md index 1a00348..2db4506 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/references/create.md +++ b/plugins/kyberforge/.apm/skills/skill-author/references/create.md @@ -171,11 +171,20 @@ If a research `sources.md` is present in the conversation context: 2. For each entry, determine which skill files it contributed to (SKILL.md and any files in `references/` that drew from it). Update `Contributing files` accordingly — list skill files, not research topic files. -3. Write the updated content to `references/sources.md`. For each entry, include - `- **Research doc:** ` where `` is the relative path from the repo root to the - plugin-level research sources file this entry was drawn from (e.g. - `plugins/myplugin/docs/research/docs//sources.md`). This field is required on every - entry — it makes the provenance chain explicit and is validated by `/factory-audit`. +3. Write the updated content to `references/sources.md`. Every entry carries exactly one + `- **Research doc:** ` line. `` is the relative path from the repo root to the + **Research registry** — the plugin-level research `sources.md` whose `## H2` headings are the + source slugs (e.g. `plugins/myplugin/docs/research/docs//sources.md`) — never a topic + document, and never a list: no brace expansion, no comma- or semicolon-separated paths, no + second `Research doc:` line. A pointer to the topic document that digested the source goes in + an annotation after the path, e.g. ` (digested in remotes.md)`, where it is not + checked. `/factory-audit` fails a slug missing from the registry it names. + + If the entry has no Research registry — an org convention, an ADR, a house-verified + reproduction — write `- **Research doc:** none` and name what it was drawn from with one + `- **Basis:** ` line per path. Each Basis path is checked to exist; annotate one + that has since been deleted `(removed in )` and the check is skipped. `none` with no Basis + is a FAIL. 4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of sources that informed it. 5. For each file in `references/` that was informed by research sources, add `source_keys`