Files

Defame1297 88866b81a3 fix(kyberforge): announce every provenance skip and scope checks 7-8 to source indexes

The clean provenance bill was an artifact. Checks 7 and 8 assume `Research doc:`
names a source index whose H2s are slugs, but 30 of 121 corpus entries point at
topic content documents whose H2s are topics. Those 30 produced every new check-7
INFO — all false positives. Check 8 aimed at the same documents, which carry no
`Status:` line at all, would have emitted a large false-FAIL flood; the only thing
preventing it was an unannounced `rd_status != extracted` skip. So "0 new FAILs"
rested on exactly the fail-open class this branch exists to remove, and naively
fixing the skip would have turned the branch red.

Checks 7/8 now run only when the research doc's basename is `sources.md`, and every
other case emits a visible INFO naming the slug. The dangling-path INFO stays ahead
of the basename gate, because a path that does not resolve is rot whatever it is
named. `parse_status` accepts the bullet form and a trailing note after the
backticked value, so a status it cannot read no longer reads as "nothing to check".

Corpus: 36 INFOs of which 30 were false, to 56 of which none are. FAIL stays 0, and
no Status line flipped to `extracted` under the new parser, so no FAIL was
suppressed by luck.

Also closed, each a silent pass: a nonexistent directory, a directory with no
SKILL.md, and extra arguments now exit 2; a UTF-8 BOM no longer defeats frontmatter
parsing; the bare `except Exception: return False` that turned an unreadable file
into a clean pass is gone, with all reads pinned to UTF-8; check 3 walks nested
`references/` subdirectories; `FILL IN:` at end of line no longer escapes checks 1
and 6; duplicate `## slug` blocks and repeated `Research doc:` lines are announced
rather than half-read.

The agent-audit copy carried all of the above unfixed and is now ported, minus the
four fixes that are genuinely N/A at agent scope — it reads a plugin-root
`sources.md` and has no checks 7/8 and no `references/` tree. Its silent exit 0 for
a file outside plugin scope is preserved deliberately: that is a verdict about a
valid file, not a skip, and `check-scope-walkup-sync.sh` pins it. Every exit-2 gate
therefore decides from the argument alone, before the walk-up runs.

`validation-scripts.md` said flatly that silence from the validator is a pass, not
a skip. That sentence is what made a typo'd path dangerous, and both copies are
corrected here. The matching SKILL.md exit-code guidance lands with the audit
rubric change, which touches the same files.

Tests: skill-audit 45 to 65, agent-audit 24 to 43, every new case proven by mutation.

Refs: #111, #118, #121

2026-09-01 12:37:50 +00:00

4.1 KiB

Raw Blame History

source_keys

claude-code-plugins-docs

claude-code-subagents-docs

github-custom-agents-configuration

Validation Scripts Reference

Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting. Nothing here is needed on a clean run.

Report the gap, do not guess

If a script cannot run at all — Bash denied, python3 unavailable, vale not installed — say so as an INFO finding naming the script and the missing dependency, then fall back to the manual checks below. An INFO never changes PASS/FAIL. Silently omitting the dimension a script would have covered reports a clean audit that checked less than it claims to have checked.

How the scripts detect scope

validate.sh and validate-provenance.sh walk up from the agent file's directory and stop at the first of these:

An apm.yml carrying a top-level type: instructions|skill|hybrid|prompts line — plugin/APM scope, and that directory is the package root. An apm.yml with no type: is a marketplace-only manifest: skip it and keep walking.
$HOME — user scope, checked before .git so a dotfiles-managed home directory that is its own repo cannot shadow it.
A .git directory or file — project scope.
The filesystem root — project scope.

plugin.json and .claude-plugin/plugin.json are not scope signals. A directory holding only a plugin.json and no apm.yml falls through to project or user scope.

validate-provenance.sh exits 0 silently when that walk does not land on a package root, and again when the package has no provenance data. Check the exit code before you believe the silence:

0 — a pass, not a skip you need to investigate. Both silent cases above land here.
1 — real findings, on stdout with Why and Fix.
2 — the check never ran. A missing, doubled, non-file or wrongly-named argument, an undecodable apm.yml, or an absent python3, each with a diagnostic on stderr and no findings at all. Report the ### Provenance dimension as unverified and quote the reason. An exit 2 is never a clean pass: empty stdout there means nothing was checked, not that nothing was wrong.

Manual fallback

Every scope: required fields present (name, description, non-empty body); name is kebab-case; no FILL IN: placeholders in the description or body; the description at or under 400 characters measured on the folded YAML value.

Plugin/APM scope: name matches the filename stem; no HTML comments left in the frontmatter; no frontmatter key outside the apm-agent-allowlist section of references/field-inventory.md — open that file, do not work from memory.

Project/user scope: the counterpart file exists; name matches the filename stem in the Copilot .agent.md only (Claude Code files are exempt); no key from claude-code-only-fields in the Copilot file and none from copilot-only-fields in the CC file, both read from references/field-inventory.md.

Script-specific failures

Error: field-inventory.md not found (exit 2). validate.sh reads its field lists from references/field-inventory.md at load time and refuses to run without it, rather than falling back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not work around it.
vale reports 0 files. Treat the pass as NOT RUN, not as clean, and fall back to full Step 3 judgment for the dimensions it would have covered. The Kyberforge style is scoped to **/agents/*.md and **/*.agent.md, and KyberforgeCopilot to **/*.agent.md alone — a file outside those globs is silently not linted.
E100 Runtime error ... does not exist (exit 2) from vale-wrap.sh. An explicit relative --config was passed. Pass none: the wrapper locates its own assets/vale/.vale.ini from its own path. Do not read this exit code as vale being unavailable.
A path argument that does not exist is a hard error in vale-wrap.sh, deliberately: bare vale would fall back to reading stdin and print a clean-looking 0 errors ... in stdin, which the 0 files guard above does not catch.

4.1 KiB Raw Blame History