The clean provenance bill was an artifact. Checks 7 and 8 assume `Research doc:`
names a source index whose H2s are slugs, but 30 of 121 corpus entries point at
topic content documents whose H2s are topics. Those 30 produced every new check-7
INFO — all false positives. Check 8 aimed at the same documents, which carry no
`Status:` line at all, would have emitted a large false-FAIL flood; the only thing
preventing it was an unannounced `rd_status != extracted` skip. So "0 new FAILs"
rested on exactly the fail-open class this branch exists to remove, and naively
fixing the skip would have turned the branch red.
Checks 7/8 now run only when the research doc's basename is `sources.md`, and every
other case emits a visible INFO naming the slug. The dangling-path INFO stays ahead
of the basename gate, because a path that does not resolve is rot whatever it is
named. `parse_status` accepts the bullet form and a trailing note after the
backticked value, so a status it cannot read no longer reads as "nothing to check".
Corpus: 36 INFOs of which 30 were false, to 56 of which none are. FAIL stays 0, and
no Status line flipped to `extracted` under the new parser, so no FAIL was
suppressed by luck.
Also closed, each a silent pass: a nonexistent directory, a directory with no
SKILL.md, and extra arguments now exit 2; a UTF-8 BOM no longer defeats frontmatter
parsing; the bare `except Exception: return False` that turned an unreadable file
into a clean pass is gone, with all reads pinned to UTF-8; check 3 walks nested
`references/` subdirectories; `FILL IN:` at end of line no longer escapes checks 1
and 6; duplicate `## slug` blocks and repeated `Research doc:` lines are announced
rather than half-read.
The agent-audit copy carried all of the above unfixed and is now ported, minus the
four fixes that are genuinely N/A at agent scope — it reads a plugin-root
`sources.md` and has no checks 7/8 and no `references/` tree. Its silent exit 0 for
a file outside plugin scope is preserved deliberately: that is a verdict about a
valid file, not a skip, and `check-scope-walkup-sync.sh` pins it. Every exit-2 gate
therefore decides from the argument alone, before the walk-up runs.
`validation-scripts.md` said flatly that silence from the validator is a pass, not
a skip. That sentence is what made a typo'd path dangerous, and both copies are
corrected here. The matching SKILL.md exit-code guidance lands with the audit
rubric change, which touches the same files.
Tests: skill-audit 45 to 65, agent-audit 24 to 43, every new case proven by mutation.
Refs: #111, #118, #121
parse_contributing_files documented that callers depend on None vs [], because a
parse failure returning [] would silently disable the check. Only check 8
honoured it; checks 4/5 (skill-audit) and 3/4 (agent-audit) used a truthiness
test, so an unreadable block disabled them without a word.
Two live corpus entries were skipping this way. A sweep of all 32 sources.md
found 134 entries, exactly 2 parsing to None, both in gitea-files: one heading
carried an inline parenthetical that defeated both regexes, and one (none) was
written without its leading bullet. Also pins EMPTY_SOURCE_KEYS_RE to the two
indents parse_source_keys actually reads.
agent-audit had no INFO tier at all, so it gains one rather than reporting a
check that could not run as a FAIL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EJJrm5YmacbwMdzZpXcoti
484357a taught the Contributing parser the bullet form, but a block it still
could not parse returned the same empty result as an explicit "(none)", so the
checker read "no contributing files" and skipped check 8 rather than reporting
that it could not tell. Checks 7 and 8 were consequently dead across the whole
git plugin without anything failing.
The parser now distinguishes "declared none" from "could not parse", which
wakes both checks. Because the parser is duplicated between the skill-audit and
agent-audit copies, it is fenced with BEGIN/END markers and a test hashes the
two regions so the copies cannot drift apart again silently.
Addresses #111.
validate-provenance.sh matched Contributing files only as a single inline
line beginning "- **Contributing files:**". Seven skills write it as a
bare "**Contributing files:**" heading above a bullet list, so
parse_contributing_files returned None and checks 4 (contributing file
exists) and 5 (bidirectional source_keys) silently verified nothing on
git-branches, git-remotes, git-submodules, git-workflow, git-worktrees,
gitea-files and gitea-releases.
Those are among the skills this branch changed most — git-branches alone
gained five reference files — and the retrofit's mandatory sources.md
collateral went in unchecked. Demonstrated rather than argued: planting a
nonexistent contributing path in git-remotes yields 0 findings under the
old parser and 1 FAIL under the new one.
Both forms are now accepted. The bullet form is parsed per bullet rather
than by splitting a joined value, because its per-file notes contain
commas that would otherwise be read as path separators. The return type
becomes a list of note-stripped paths, with "(none)" as an empty list and
an absent entry as None, so the two callers no longer re-split a string.
Applied to agent-audit's copy as well. No agent ships a sources.md today,
so it is latent there, but it is the same defect.
This is a third gate blind spot alongside #117 and #118, and was unfiled.
One real defect surfaced immediately and is fixed separately.
Refs #99
validate.sh's detect_scope() and validate-provenance.sh's
find_plugin_root() disagreed with new-agent.sh's already-correct,
documented walk-up semantics on three points, each causing validate.sh
to false-FAIL a legitimately-scaffolded project-scope agent pair:
- a marker-less directory walked up into $HOME (no .git/apm.yml of its
own) was classified as user scope instead of project scope
- the .git-boundary branch returned the walked-to .git location instead
of the conventional scope root, breaking any <root> that is a
subdirectory of a larger git-tracked tree (monorepo package dirs)
- the new conventional-root arithmetic introduced to fix the above two
cases had no guard against non-conventional/hand-placed file paths,
which could point it at the wrong ancestor
Also adds scripts/check-scope-walkup-sync.sh, a behavioral drift-guard
(per ADR-0014's no-cross-skill-path precedent) that cross-checks the
four independently hand-ported walk-up implementations (validate.sh,
validate-provenance.sh, new-agent.sh, new-skill.sh) against real
fixture scaffolds, wired into .pre-commit-config.yaml at pre-push so
future drift between the ports is caught automatically.
Verified via bash tests/run-tests.sh (13/13) and targeted before/after
reproduction of each bug this closes.
Post-implementation review of PR #93 (issue #89's apm.yml-native retargeting
of skill-author/skill-audit/agent-author/agent-audit) found four confirmed
defects across the four scripts' apm.yml `type:` walk-up logic:
- field-inventory.md's apm-agent-allowlist was missing `source_keys`,
contradicting agent-author/SKILL.md's own instruction (Step 5 checklist)
to allow it at plugin/APM scope — a correctly-authored file with
source_keys failed validate.sh.
- validate.sh's APM_TYPE_RE and validate-provenance.sh's TYPE_RE disagreed:
the former tolerated a quoted `type: "skill"` value, the latter didn't,
despite agent-audit/SKILL.md explicitly documenting that
validate-provenance.sh walks up "the same way validate.sh does". Both
also used `\b` word-boundary matching, which false-matches a malformed
value like `type: prompts-only` on the `prompts` prefix. Unified both
regexes to be quote-tolerant and require an exact value.
- All four scripts' `.git` project-boundary check used isdir()/[[ -d ]],
which misses git worktrees where `.git` is a regular file (`gitdir: ...`)
rather than a directory. Switched to exists()/[[ -e ]].
- new-agent.sh and new-skill.sh had the same quote-intolerance as above via
inline `grep -qE` calls (new-skill.sh's also had the `\b` false-match
bug); replaced both with a shared-shape `is_apm_package_manifest` bash
helper matching the Python regex's semantics.
Four other findings from the same review turned out not to be bugs: a
bare `plugin.json` no longer signaling plugin scope is documented,
intentional behavior (agent-audit/SKILL.md:30, agent-author/SKILL.md:87),
deferred to issue #90's real plugin.json-to-apm.yml conversion — not
something this fix should reverse.
Verified via direct reproduction of each defect plus the full test suite:
147/147 bats tests, 39/39 shell-script tests, 12/12 summary categories.
Refs: #89
Validates the new single-file .apm/agents/<name>.agent.md shape agent-author
now produces at plugin/APM scope: frontmatter allowlist (name/description/
model only, from a new apm-agent-allowlist entry in field-inventory.md),
no counterpart derivation, and Pair Consistency dropped from that scope's
report entirely (nothing to pair by design). Adds a plugin/APM-scope-only
SUGGESTION when an agent's description/body implies a tool restriction or
Claude-only behavior the vendor-neutral frontmatter can no longer express
(ADR-0016).
Scope detection in both validate.sh and validate-provenance.sh switches
from a flat plugin.json/.claude-plugin/plugin.json check to a walk-up for
the nearest ancestor apm.yml with a top-level type: field, skipping
type:-less marketplace-only manifests — full switch, no dual-mode fallback
to the old plugin.json signal. validate-provenance.sh's walk-up was fixed
to match validate.sh's (it still used the old plugin.json check, and its
counterpart-merge logic was rewritten to read a single file's source_keys
instead of merging a CC+Copilot pair, since plugin/APM scope has no
counterpart). Project/user scope validation is unchanged in both scripts.
Refs: #89
skill-audit/agent-audit now proactively trigger after a skill/agent
file is hand-edited outside skill-author/agent-author, not just on
explicit request — closing a gap from this session where a fork's
direct edits to agent-author/agent-audit shipped without their own
inline audit until forge was invoked to check afterward.
Also: plugin-author gains a gotcha on claude plugin validate --strict
auto-discovering every .md under agents/ regardless of manifest
declarations (ADR-0010); agent-audit's dimension count and
validate-provenance.sh's --help now match actual behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`claude plugin validate --strict` auto-discovers every .md under a
plugin's agents/ directory as an agent requiring frontmatter, so the
provenance file there always needs fake agent frontmatter to pass
validation. Confirmed empirically that an explicit `agents` manifest
array can't suppress this discovery. Move the file to <plugin-root>/
sources.md instead, and update agent-author/agent-audit accordingly.
Adds ADR-0010, partially superseding ADR-0005's `agents/sources.md`
convention. Fixes#63.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
## Why
Two issues were found in `validate-provenance.sh` by `/skill-audit`:
1. `find_plugin_root` only walked up looking for bare `plugin.json`, missing
the `.claude-plugin/plugin.json` layout used by kyberforge plugins — matching
the logic already present in `validate.sh`.
2. Check numbers in `--help` and inline comments had a gap (0,1,2,4,5,6) from
a previously removed check, making the numbering confusing to readers.
## Implementation Notes
Check numbers renumbered sequentially 0–5 in both the `--help` output block
and the inline `# --- Check N:` comments. The `find_plugin_root` condition now
mirrors the `detect_scope` function in `validate.sh`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
## Why
agent-author produces agents/sources.md at plugin scope to record which
research sources informed which agent files. agent-audit had no way to
validate this chain, leaving stale or missing provenance undetected.
## Implementation Notes
Validation is per-pair (the given agent file + its counterpart) rather
than plugin-wide, keeping the scope consistent with validate.sh. The
script exits 0 silently for non-plugin-scope agents.
source_keys is top-level in both CC .md and Copilot .agent.md files
(not under metadata:) to avoid conflict with Copilot's own metadata
field semantics. Checks 0, 1, 2, 4, 5, 6 mirror the skill provenance
set; upstream research-doc cross-reference checks (7, 8) are deferred.
agent-author Steps 2, 3, and 4 updated to formally specify the
agents/sources.md format and instruct authors to add source_keys to
both files when research sources are in context.
Refs: #60
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
