fix(kyberforge): stop the provenance checker skipping check 8 on unparsed input

484357a taught the Contributing parser the bullet form, but a block it still
could not parse returned the same empty result as an explicit "(none)", so the
checker read "no contributing files" and skipped check 8 rather than reporting
that it could not tell. Checks 7 and 8 were consequently dead across the whole
git plugin without anything failing.

The parser now distinguishes "declared none" from "could not parse", which
wakes both checks. Because the parser is duplicated between the skill-audit and
agent-audit copies, it is fenced with BEGIN/END markers and a test hashes the
two regions so the copies cannot drift apart again silently.

Addresses #111.
This commit is contained in:
2026-08-31 08:01:18 +00:00
parent db5a426416
commit 27a76692b0
7 changed files with 462 additions and 62 deletions

View File

@@ -1,3 +1,7 @@
---
source_keys: []
---
# Orchestrator request contract # Orchestrator request contract
`git-orchestrate` and other calling agents send this shape. The result shape they parse back is in `git-orchestrate` and other calling agents send this shape. The result shape they parse back is in

View File

@@ -1,3 +1,7 @@
---
source_keys: []
---
# Orchestrator request contract # Orchestrator request contract
`git-orchestrate` and other calling agents send this shape. The result shape they parse back is in `git-orchestrate` and other calling agents send this shape. The result shape they parse back is in

View File

@@ -130,15 +130,55 @@ def parse_source_keys(fm):
def parse_h2_slugs(content): def parse_h2_slugs(content):
return re.findall(r'^## (.+)$', content, re.MULTILINE) return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
def parse_contributing_files(content, slug): def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content. """Find the Contributing files for a given slug H2 in content.
Accepts the inline form and the bullet form; recognising only the Both authored forms are accepted, because both are in use across the
inline one silently skips the contributing-file checks on every corpus and only recognising the first silently skipped the contributing-
sources.md written the other way. Returns a list of paths with any file checks on every sources.md written the other way:
trailing parenthetical note stripped; "(none)" returns an empty list
and a slug with no entry returns None. Kept behaviourally identical to - **Contributing files:** SKILL.md, references/a.md
skill-audit's copy, which is where the bug was found.
**Contributing files:**
- SKILL.md (what this source contributed)
- references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped.
Note the bullet form's notes may themselves contain commas, so the list
is built per bullet rather than by splitting the joined value.
The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
""" """
pattern = re.compile( pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
@@ -150,15 +190,19 @@ def parse_contributing_files(content, slug):
block = m.group(1) block = m.group(1)
def strip_note(entry): def strip_note(entry):
# "references/a.md (why)" -> "references/a.md"
return re.sub(r'\s*\(.*$', '', entry).strip() return re.sub(r'\s*\(.*$', '', entry).strip()
# Inline form: value on the same line, comma-separated, no notes.
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE) cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if cf_m: if cf_m:
value = cf_m.group(1).strip() value = cf_m.group(1).strip()
if value.startswith("(none"): if value.startswith("(none"):
return [] return []
return [p for p in (strip_note(x) for x in value.split(",")) if p] return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m: if not cf_m:
return None return None
@@ -177,7 +221,8 @@ def parse_contributing_files(content, slug):
entry = strip_note(entry) entry = strip_note(entry)
if entry: if entry:
files.append(entry) files.append(entry)
return files return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
def parse_research_doc(content, slug): def parse_research_doc(content, slug):
pattern = re.compile( pattern = re.compile(

View File

@@ -18,11 +18,16 @@ Checks performed:
0 source_keys present but references/sources.md absent 0 source_keys present but references/sources.md absent
1 FILL IN: placeholders in sources.md 1 FILL IN: placeholders in sources.md
2 source_keys in SKILL.md → slug exists in sources.md 2 source_keys in SKILL.md → slug exists in sources.md
3 source_keys in references/*.md → slug exists in sources.md (INFO if no source_keys) 3 source_keys in references/*.md → slug exists in sources.md (INFO if no
source_keys; an explicit 'source_keys: []' declares the file house-authored
and passes silently)
4 Contributing files listed in sources.md exist on disk 4 Contributing files listed in sources.md exist on disk
5 Contributing files back-reference the parent slug in their source_keys 5 Contributing files back-reference the parent slug in their source_keys
6 Research doc field present and not placeholder 6 Research doc field present and not placeholder
7 Slug in sources.md present in upstream research doc (INFO only) 7 Slug in sources.md present in upstream research doc (INFO only). A section
annotation ('§ ...', '→ ...', '(...)') is stripped before the path is
resolved; a path that still does not resolve is reported as an INFO saying
checks 7 and 8 did not run, never skipped silently.
8 Extracted non-(none) slug in research doc present in sources.md 8 Extracted non-(none) slug in research doc present in sources.md
EOF EOF
} }
@@ -89,16 +94,53 @@ def parse_source_keys(fm):
in_metadata = False in_metadata = False
return keys return keys
# An explicit `source_keys: []` — top-level or under metadata: — is a
# DECLARATION that the file is house-authored and has no external source.
# parse_source_keys() returns [] both for that and for a file with no
# source_keys key at all, so the two are indistinguishable downstream and
# check 3 emitted the same INFO for each (#111). That left no honest way to
# record "this file has no external source": the only ways to silence the INFO
# were to invent a slug or borrow an unrelated one, both false provenance
# claims that then have to be maintained in sources.md as well. A bare
# `source_keys:` with nothing after it is NOT accepted here — that reads as a
# truncated or half-written entry, not a decision.
EMPTY_SOURCE_KEYS_RE = re.compile(r'^\s*source_keys:\s*\[\s*\]\s*$')
def declares_empty_source_keys(fm):
"""True when frontmatter carries an explicit, empty `source_keys: []`."""
if fm is None:
return False
return any(EMPTY_SOURCE_KEYS_RE.match(line) for line in fm.splitlines())
def parse_h2_slugs(content): def parse_h2_slugs(content):
"""Return list of H2 heading values from a markdown file.""" """Return list of H2 heading values from a markdown file."""
return re.findall(r'^## (.+)$', content, re.MULTILINE) return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
def parse_contributing_files(content, slug): def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content. """Find the Contributing files for a given slug H2 in content.
Both authored forms are accepted, because both are in use across the Both authored forms are accepted, because both are in use across the
corpus and only recognising the first silently skipped checks 4 and 5 corpus and only recognising the first silently skipped the contributing-
on every skill using the second: file checks on every sources.md written the other way:
- **Contributing files:** SKILL.md, references/a.md - **Contributing files:** SKILL.md, references/a.md
@@ -107,10 +149,22 @@ def parse_contributing_files(content, slug):
- references/a.md (what this source contributed) - references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped. Returns a list of paths with any trailing parenthetical note stripped.
A "(none)" value returns an empty list; a slug with no Contributing Note the bullet form's notes may themselves contain commas, so the list
files entry at all returns None. Note the bullet form's notes may is built per bullet rather than by splitting the joined value.
themselves contain commas, so the list is built per bullet rather than
by splitting the joined value. The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
""" """
pattern = re.compile( pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
@@ -131,15 +185,15 @@ def parse_contributing_files(content, slug):
value = cf_m.group(1).strip() value = cf_m.group(1).strip()
if value.startswith("(none"): if value.startswith("(none"):
return [] return []
return [p for p in (strip_note(x) for x in value.split(",")) if p] return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet. # Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m: if not cf_m:
return None return None
rest = block[cf_m.end():]
files = [] files = []
for line in rest.splitlines(): for line in block[cf_m.end():].splitlines():
line = line.strip() line = line.strip()
if not line: if not line:
if files: if files:
@@ -153,7 +207,8 @@ def parse_contributing_files(content, slug):
entry = strip_note(entry) entry = strip_note(entry)
if entry: if entry:
files.append(entry) files.append(entry)
return files return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
def parse_research_doc(content, slug): def parse_research_doc(content, slug):
"""Find the Research doc value for a given slug H2 in content.""" """Find the Research doc value for a given slug H2 in content."""
@@ -170,6 +225,36 @@ def parse_research_doc(content, slug):
return None return None
return rd_m.group(1).strip() return rd_m.group(1).strip()
# A Research doc value is a path, and very often a path PLUS an annotation
# naming the section the slug came from:
#
# plugins/git/docs/research/docs/git/gitflow.md (whole-document reference)
# plugins/git/docs/research/docs/git/remotes.md → `## Pushing (`git push`)`
# .../pre-commit/hooks-reference.md § "pre-commit-hooks (official collection)"
#
# os.path.isfile() is false for every one of those strings, and checks 7 and 8
# used to skip SILENTLY whenever the path did not resolve. The effect was that
# both checks were dead on eight of the nine git skills — git-history, the one
# skill writing a bare path, was the only place they ran, which is why it was
# the only skill ever reporting a check-7 INFO. Strip the annotation before
# resolving, and report when the result still does not resolve: a check that
# quietly does not run is worse than one that fails.
RESEARCH_DOC_ANNOTATION_RE = re.compile(r'[§→(]')
def strip_research_doc_annotation(value):
"""Path part of a Research doc value, with any section annotation removed."""
return RESEARCH_DOC_ANNOTATION_RE.split(value, maxsplit=1)[0].strip()
def research_doc_is_none(value):
"""True when a Research doc value declares that no research doc backs the slug.
Both '(none)' and the bare 'none — org convention, ...' spelling are in
use; recognising only the parenthesised one would report the other as an
unresolvable path. Checked BEFORE the annotation strip, because '(none)'
is itself a parenthesis and would strip to the empty string.
"""
return re.match(r'\(?none\b', value.strip(), re.IGNORECASE) is not None
def parse_status(content, slug): def parse_status(content, slug):
"""Find the Status value for a given slug H2 in content.""" """Find the Status value for a given slug H2 in content."""
pattern = re.compile( pattern = re.compile(
@@ -321,11 +406,17 @@ if os.path.isdir(refs_dir):
ref_fm, _ = parse_frontmatter(ref_content) ref_fm, _ = parse_frontmatter(ref_content)
ref_keys = parse_source_keys(ref_fm) ref_keys = parse_source_keys(ref_fm)
if not ref_keys: if not ref_keys:
# An explicit `source_keys: []` is a deliberate declaration that
# the file is house-authored, and passes silently. The INFO is for
# files that never said either way.
if declares_empty_source_keys(ref_fm):
continue
emit_info( emit_info(
f"No source_keys frontmatter", f"No source_keys frontmatter",
rel, rel,
"This references file has no source_keys — provenance cannot be verified. " "This references file has no source_keys — provenance cannot be verified. "
"Add source_keys frontmatter listing the slugs from references/sources.md that informed this file." "Add source_keys frontmatter listing the slugs from references/sources.md that informed this file, "
"or declare an explicit 'source_keys: []' if the file is house-authored and has no external source."
) )
else: else:
for slug in ref_keys: for slug in ref_keys:
@@ -390,24 +481,51 @@ for slug in parse_h2_slugs(sources_content):
f"The '## {slug}' entry has an unfilled Research doc value.", f"The '## {slug}' entry has an unfilled Research doc value.",
f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable." f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable."
) )
elif not research_doc_is_none(rd_value):
# Check 7: Upstream forward — slug should appear in research doc.
# Every path out of here that does NOT run the check says so out loud.
rd_path = strip_research_doc_annotation(rd_value)
if not repo_root:
emit_info(
f"Upstream checks skipped for '{slug}' — no repo root above the skill directory",
f"references/sources.md (## {slug})",
f"'{rd_value}' is a path relative to the repo root, but no ancestor of the skill directory contains a .git entry, "
f"so it cannot be resolved. Checks 7 and 8 did not run for this slug. "
f"Run this script against a skill inside a checkout."
)
elif not rd_path:
emit_info(
f"Upstream checks skipped for '{slug}' — Research doc value names no path",
f"references/sources.md (## {slug})",
f"The Research doc value '{rd_value}' is entirely annotation — stripping the section marker leaves no path. "
f"Checks 7 and 8 did not run for this slug. "
f"Give the value a file path relative to the repo root, or record '(none)' if no research doc backs this entry."
)
else:
rd_abs = os.path.join(repo_root, rd_path)
if not os.path.isfile(rd_abs):
emit_info(
f"Upstream checks skipped for '{slug}' — research doc '{rd_path}' does not exist",
f"references/sources.md (## {slug})",
f"'{rd_value}' resolves to '{rd_path}' relative to the repo root and no file is there. "
f"Checks 7 and 8 did not run for this slug, so nothing verified that the research doc still backs it. "
f"Point the value at one existing file — a brace expansion, a comma-separated list of paths, or a bare section title does not resolve — "
f"or record '(none)' if no research doc backs this entry."
)
else: else:
# Check 7: Upstream forward — slug should appear in research doc
if repo_root and not rd_value.startswith("(none"):
rd_abs = os.path.join(repo_root, rd_value)
if os.path.isfile(rd_abs):
with open(rd_abs) as f: with open(rd_abs) as f:
rd_content = f.read() rd_content = f.read()
rd_slugs = set(parse_h2_slugs(rd_content)) rd_slugs = set(parse_h2_slugs(rd_content))
if slug not in rd_slugs: if slug not in rd_slugs:
emit_info( emit_info(
f"Slug '{slug}' not found as H2 in research doc '{rd_value}'", f"Slug '{slug}' not found as H2 in research doc '{rd_path}'",
f"references/sources.md (## {slug})", f"references/sources.md (## {slug})",
f"The research doc '{rd_value}' does not have a '## {slug}' heading. " f"The research doc '{rd_path}' does not have a '## {slug}' heading. "
f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading." f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading."
) )
# Track for Check 8 # Track for Check 8
if rd_abs not in research_docs_seen: if rd_abs not in research_docs_seen:
research_docs_seen[rd_abs] = (rd_value, set()) research_docs_seen[rd_abs] = (rd_path, set())
research_docs_seen[rd_abs][1].add(slug) research_docs_seen[rd_abs][1].add(slug)
# --- Check 8: Upstream reverse --- # --- Check 8: Upstream reverse ---

View File

@@ -130,15 +130,55 @@ def parse_source_keys(fm):
def parse_h2_slugs(content): def parse_h2_slugs(content):
return re.findall(r'^## (.+)$', content, re.MULTILINE) return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
def parse_contributing_files(content, slug): def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content. """Find the Contributing files for a given slug H2 in content.
Accepts the inline form and the bullet form; recognising only the Both authored forms are accepted, because both are in use across the
inline one silently skips the contributing-file checks on every corpus and only recognising the first silently skipped the contributing-
sources.md written the other way. Returns a list of paths with any file checks on every sources.md written the other way:
trailing parenthetical note stripped; "(none)" returns an empty list
and a slug with no entry returns None. Kept behaviourally identical to - **Contributing files:** SKILL.md, references/a.md
skill-audit's copy, which is where the bug was found.
**Contributing files:**
- SKILL.md (what this source contributed)
- references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped.
Note the bullet form's notes may themselves contain commas, so the list
is built per bullet rather than by splitting the joined value.
The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
""" """
pattern = re.compile( pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
@@ -150,15 +190,19 @@ def parse_contributing_files(content, slug):
block = m.group(1) block = m.group(1)
def strip_note(entry): def strip_note(entry):
# "references/a.md (why)" -> "references/a.md"
return re.sub(r'\s*\(.*$', '', entry).strip() return re.sub(r'\s*\(.*$', '', entry).strip()
# Inline form: value on the same line, comma-separated, no notes.
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE) cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if cf_m: if cf_m:
value = cf_m.group(1).strip() value = cf_m.group(1).strip()
if value.startswith("(none"): if value.startswith("(none"):
return [] return []
return [p for p in (strip_note(x) for x in value.split(",")) if p] return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m: if not cf_m:
return None return None
@@ -177,7 +221,8 @@ def parse_contributing_files(content, slug):
entry = strip_note(entry) entry = strip_note(entry)
if entry: if entry:
files.append(entry) files.append(entry)
return files return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
def parse_research_doc(content, slug): def parse_research_doc(content, slug):
pattern = re.compile( pattern = re.compile(

View File

@@ -18,11 +18,16 @@ Checks performed:
0 source_keys present but references/sources.md absent 0 source_keys present but references/sources.md absent
1 FILL IN: placeholders in sources.md 1 FILL IN: placeholders in sources.md
2 source_keys in SKILL.md → slug exists in sources.md 2 source_keys in SKILL.md → slug exists in sources.md
3 source_keys in references/*.md → slug exists in sources.md (INFO if no source_keys) 3 source_keys in references/*.md → slug exists in sources.md (INFO if no
source_keys; an explicit 'source_keys: []' declares the file house-authored
and passes silently)
4 Contributing files listed in sources.md exist on disk 4 Contributing files listed in sources.md exist on disk
5 Contributing files back-reference the parent slug in their source_keys 5 Contributing files back-reference the parent slug in their source_keys
6 Research doc field present and not placeholder 6 Research doc field present and not placeholder
7 Slug in sources.md present in upstream research doc (INFO only) 7 Slug in sources.md present in upstream research doc (INFO only). A section
annotation ('§ ...', '→ ...', '(...)') is stripped before the path is
resolved; a path that still does not resolve is reported as an INFO saying
checks 7 and 8 did not run, never skipped silently.
8 Extracted non-(none) slug in research doc present in sources.md 8 Extracted non-(none) slug in research doc present in sources.md
EOF EOF
} }
@@ -89,16 +94,53 @@ def parse_source_keys(fm):
in_metadata = False in_metadata = False
return keys return keys
# An explicit `source_keys: []` — top-level or under metadata: — is a
# DECLARATION that the file is house-authored and has no external source.
# parse_source_keys() returns [] both for that and for a file with no
# source_keys key at all, so the two are indistinguishable downstream and
# check 3 emitted the same INFO for each (#111). That left no honest way to
# record "this file has no external source": the only ways to silence the INFO
# were to invent a slug or borrow an unrelated one, both false provenance
# claims that then have to be maintained in sources.md as well. A bare
# `source_keys:` with nothing after it is NOT accepted here — that reads as a
# truncated or half-written entry, not a decision.
EMPTY_SOURCE_KEYS_RE = re.compile(r'^\s*source_keys:\s*\[\s*\]\s*$')
def declares_empty_source_keys(fm):
"""True when frontmatter carries an explicit, empty `source_keys: []`."""
if fm is None:
return False
return any(EMPTY_SOURCE_KEYS_RE.match(line) for line in fm.splitlines())
def parse_h2_slugs(content): def parse_h2_slugs(content):
"""Return list of H2 heading values from a markdown file.""" """Return list of H2 heading values from a markdown file."""
return re.findall(r'^## (.+)$', content, re.MULTILINE) return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
def parse_contributing_files(content, slug): def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content. """Find the Contributing files for a given slug H2 in content.
Both authored forms are accepted, because both are in use across the Both authored forms are accepted, because both are in use across the
corpus and only recognising the first silently skipped checks 4 and 5 corpus and only recognising the first silently skipped the contributing-
on every skill using the second: file checks on every sources.md written the other way:
- **Contributing files:** SKILL.md, references/a.md - **Contributing files:** SKILL.md, references/a.md
@@ -107,10 +149,22 @@ def parse_contributing_files(content, slug):
- references/a.md (what this source contributed) - references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped. Returns a list of paths with any trailing parenthetical note stripped.
A "(none)" value returns an empty list; a slug with no Contributing Note the bullet form's notes may themselves contain commas, so the list
files entry at all returns None. Note the bullet form's notes may is built per bullet rather than by splitting the joined value.
themselves contain commas, so the list is built per bullet rather than
by splitting the joined value. The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
""" """
pattern = re.compile( pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
@@ -131,15 +185,15 @@ def parse_contributing_files(content, slug):
value = cf_m.group(1).strip() value = cf_m.group(1).strip()
if value.startswith("(none"): if value.startswith("(none"):
return [] return []
return [p for p in (strip_note(x) for x in value.split(",")) if p] return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet. # Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m: if not cf_m:
return None return None
rest = block[cf_m.end():]
files = [] files = []
for line in rest.splitlines(): for line in block[cf_m.end():].splitlines():
line = line.strip() line = line.strip()
if not line: if not line:
if files: if files:
@@ -153,7 +207,8 @@ def parse_contributing_files(content, slug):
entry = strip_note(entry) entry = strip_note(entry)
if entry: if entry:
files.append(entry) files.append(entry)
return files return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
def parse_research_doc(content, slug): def parse_research_doc(content, slug):
"""Find the Research doc value for a given slug H2 in content.""" """Find the Research doc value for a given slug H2 in content."""
@@ -170,6 +225,36 @@ def parse_research_doc(content, slug):
return None return None
return rd_m.group(1).strip() return rd_m.group(1).strip()
# A Research doc value is a path, and very often a path PLUS an annotation
# naming the section the slug came from:
#
# plugins/git/docs/research/docs/git/gitflow.md (whole-document reference)
# plugins/git/docs/research/docs/git/remotes.md → `## Pushing (`git push`)`
# .../pre-commit/hooks-reference.md § "pre-commit-hooks (official collection)"
#
# os.path.isfile() is false for every one of those strings, and checks 7 and 8
# used to skip SILENTLY whenever the path did not resolve. The effect was that
# both checks were dead on eight of the nine git skills — git-history, the one
# skill writing a bare path, was the only place they ran, which is why it was
# the only skill ever reporting a check-7 INFO. Strip the annotation before
# resolving, and report when the result still does not resolve: a check that
# quietly does not run is worse than one that fails.
RESEARCH_DOC_ANNOTATION_RE = re.compile(r'[§→(]')
def strip_research_doc_annotation(value):
"""Path part of a Research doc value, with any section annotation removed."""
return RESEARCH_DOC_ANNOTATION_RE.split(value, maxsplit=1)[0].strip()
def research_doc_is_none(value):
"""True when a Research doc value declares that no research doc backs the slug.
Both '(none)' and the bare 'none — org convention, ...' spelling are in
use; recognising only the parenthesised one would report the other as an
unresolvable path. Checked BEFORE the annotation strip, because '(none)'
is itself a parenthesis and would strip to the empty string.
"""
return re.match(r'\(?none\b', value.strip(), re.IGNORECASE) is not None
def parse_status(content, slug): def parse_status(content, slug):
"""Find the Status value for a given slug H2 in content.""" """Find the Status value for a given slug H2 in content."""
pattern = re.compile( pattern = re.compile(
@@ -321,11 +406,17 @@ if os.path.isdir(refs_dir):
ref_fm, _ = parse_frontmatter(ref_content) ref_fm, _ = parse_frontmatter(ref_content)
ref_keys = parse_source_keys(ref_fm) ref_keys = parse_source_keys(ref_fm)
if not ref_keys: if not ref_keys:
# An explicit `source_keys: []` is a deliberate declaration that
# the file is house-authored, and passes silently. The INFO is for
# files that never said either way.
if declares_empty_source_keys(ref_fm):
continue
emit_info( emit_info(
f"No source_keys frontmatter", f"No source_keys frontmatter",
rel, rel,
"This references file has no source_keys — provenance cannot be verified. " "This references file has no source_keys — provenance cannot be verified. "
"Add source_keys frontmatter listing the slugs from references/sources.md that informed this file." "Add source_keys frontmatter listing the slugs from references/sources.md that informed this file, "
"or declare an explicit 'source_keys: []' if the file is house-authored and has no external source."
) )
else: else:
for slug in ref_keys: for slug in ref_keys:
@@ -390,24 +481,51 @@ for slug in parse_h2_slugs(sources_content):
f"The '## {slug}' entry has an unfilled Research doc value.", f"The '## {slug}' entry has an unfilled Research doc value.",
f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable." f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable."
) )
elif not research_doc_is_none(rd_value):
# Check 7: Upstream forward — slug should appear in research doc.
# Every path out of here that does NOT run the check says so out loud.
rd_path = strip_research_doc_annotation(rd_value)
if not repo_root:
emit_info(
f"Upstream checks skipped for '{slug}' — no repo root above the skill directory",
f"references/sources.md (## {slug})",
f"'{rd_value}' is a path relative to the repo root, but no ancestor of the skill directory contains a .git entry, "
f"so it cannot be resolved. Checks 7 and 8 did not run for this slug. "
f"Run this script against a skill inside a checkout."
)
elif not rd_path:
emit_info(
f"Upstream checks skipped for '{slug}' — Research doc value names no path",
f"references/sources.md (## {slug})",
f"The Research doc value '{rd_value}' is entirely annotation — stripping the section marker leaves no path. "
f"Checks 7 and 8 did not run for this slug. "
f"Give the value a file path relative to the repo root, or record '(none)' if no research doc backs this entry."
)
else:
rd_abs = os.path.join(repo_root, rd_path)
if not os.path.isfile(rd_abs):
emit_info(
f"Upstream checks skipped for '{slug}' — research doc '{rd_path}' does not exist",
f"references/sources.md (## {slug})",
f"'{rd_value}' resolves to '{rd_path}' relative to the repo root and no file is there. "
f"Checks 7 and 8 did not run for this slug, so nothing verified that the research doc still backs it. "
f"Point the value at one existing file — a brace expansion, a comma-separated list of paths, or a bare section title does not resolve — "
f"or record '(none)' if no research doc backs this entry."
)
else: else:
# Check 7: Upstream forward — slug should appear in research doc
if repo_root and not rd_value.startswith("(none"):
rd_abs = os.path.join(repo_root, rd_value)
if os.path.isfile(rd_abs):
with open(rd_abs) as f: with open(rd_abs) as f:
rd_content = f.read() rd_content = f.read()
rd_slugs = set(parse_h2_slugs(rd_content)) rd_slugs = set(parse_h2_slugs(rd_content))
if slug not in rd_slugs: if slug not in rd_slugs:
emit_info( emit_info(
f"Slug '{slug}' not found as H2 in research doc '{rd_value}'", f"Slug '{slug}' not found as H2 in research doc '{rd_path}'",
f"references/sources.md (## {slug})", f"references/sources.md (## {slug})",
f"The research doc '{rd_value}' does not have a '## {slug}' heading. " f"The research doc '{rd_path}' does not have a '## {slug}' heading. "
f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading." f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading."
) )
# Track for Check 8 # Track for Check 8
if rd_abs not in research_docs_seen: if rd_abs not in research_docs_seen:
research_docs_seen[rd_abs] = (rd_value, set()) research_docs_seen[rd_abs] = (rd_path, set())
research_docs_seen[rd_abs][1].add(slug) research_docs_seen[rd_abs][1].add(slug)
# --- Check 8: Upstream reverse --- # --- Check 8: Upstream reverse ---

View File

@@ -1,8 +1,7 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Regression test for the three STRUCTURAL claims the ADR-0020 gate family makes # Regression test for the STRUCTURAL claims the ADR-0020 gate family makes about
# about itself. None of them was pinned anywhere before this file, and each one # itself. None of them was pinned anywhere before this file, and each one fails
# fails silently — which is the whole reason they need a test rather than a # silently — which is the whole reason they need a test rather than a comment:
# comment:
# #
# 1. "ONE resolver, embedded VERBATIM in three scripts." The block between the # 1. "ONE resolver, embedded VERBATIM in three scripts." The block between the
# BEGIN/END markers is copied, not imported, because a cache-installed # BEGIN/END markers is copied, not imported, because a cache-installed
@@ -11,6 +10,10 @@
# one-line edit to a single copy is invisible: every constant-agreement # one-line edit to a single copy is invisible: every constant-agreement
# assertion in tests/test-skill-size-check.sh still passes, because the # assertion in tests/test-skill-size-check.sh still passes, because the
# CONSTANTS are not what drifted. # CONSTANTS are not what drifted.
# 1b. The same claim, one directory over, for the Contributing-files parser
# embedded in both validate-provenance.sh copies. That one was worse: the
# agent-audit copy's docstring ASSERTED it was kept behaviourally identical
# to skill-audit's, and the two had already drifted.
# 2. Both interpreter preflights, in all three scripts. python3 and PyYAML are # 2. Both interpreter preflights, in all three scripts. python3 and PyYAML are
# declared HARD dependencies precisely so a missing one cannot turn into a # declared HARD dependencies precisely so a missing one cannot turn into a
# vacuous pass, and the two are checked separately so the message names the # vacuous pass, and the two are checked separately so the message names the
@@ -93,6 +96,69 @@ else
fi fi
fi fi
# ---------------------------------------------------------------------------
# 1b. The shared Contributing-files parser is byte-identical in both copies
# ---------------------------------------------------------------------------
# Same defect class, one directory over. parse_contributing_files() is embedded
# in both validate-provenance.sh copies for the same reason the resolver is
# embedded three times, and until this assertion existed the agent-audit copy's
# docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's
# copy" — an invariant nothing checked, and the two had already drifted into
# different spellings of the bullet loop. The parser decides whether checks 4,
# 5 and 8 run at all, so a one-sided edit disables a check in one script while
# every other test stays green.
echo ""
echo "--- the shared Contributing-files parser is byte-identical in both validate-provenance.sh copies ---"
CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ====='
CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER ====='
SKILL_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh"
AGENT_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh"
CF_MARKERS_OK=true
for f in "$SKILL_PROV" "$AGENT_PROV"; do
if [[ ! -f "$f" ]]; then
fail "script not found: $f"
CF_MARKERS_OK=false
continue
fi
b="$(grep -cFx "$CF_BEGIN" "$f" || true)"
e="$(grep -cFx "$CF_END" "$f" || true)"
if [[ "$b" == "1" && "$e" == "1" ]]; then
pass "${f#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker"
else
fail "${f#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1"
CF_MARKERS_OK=false
fi
done
if ! $CF_MARKERS_OK; then
fail "skipping the parser byte-identity comparison — the marker pairs are not well-formed, so any extraction would measure the wrong span"
else
CF_HASHES=()
CF_LINECOUNTS=()
for f in "$SKILL_PROV" "$AGENT_PROV"; do
out="$TMPDIR_T/cfblock-$(echo "$f" | md5sum | cut -c1-8).txt"
sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$f" > "$out"
CF_HASHES+=("$(md5sum < "$out" | cut -d' ' -f1)")
CF_LINECOUNTS+=("$(wc -l < "$out" | tr -d ' ')")
done
if [[ "${CF_HASHES[0]}" == "${CF_HASHES[1]}" ]]; then
pass "both copies hash to ${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines) — agreement by construction, not by coincidence"
else
fail "the shared Contributing-files parser has DRIFTED: skill-audit=${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines), agent-audit=${CF_HASHES[1]} (${CF_LINECOUNTS[1]} lines). Edit one copy, then paste it over the other."
fi
# Two identical EMPTY spans would hash equal and assert nothing, exactly as
# for the resolver above. The parser block is ~93 lines; 40 is a floor low
# enough never to need maintenance and high enough that a gutted block — or
# one reduced to its docstring — cannot sneak past.
if [[ "${CF_LINECOUNTS[0]}" -gt 40 ]]; then
pass "the extracted parser block is ${CF_LINECOUNTS[0]} lines — the comparison is over real content, not an empty span"
else
fail "the extracted parser block is only ${CF_LINECOUNTS[0]} lines — two identical empty spans would compare equal and assert nothing"
fi
fi
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 2. Both interpreter preflights, in all three scripts # 2. Both interpreter preflights, in all three scripts
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------