Files

Defame1297 620f20b0fd refactor(kyberforge)!: merge skill-audit and agent-audit into factory-audit

Why

The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary
resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser
(93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line
test suite. Sync-by-gate had already failed once: at 484357a the two parser copies drifted into
different spellings of the bullet loop while a docstring asserted they were identical. That drift
was behaviour-neutral and was re-unified by hand at 598a7c3, so the copies were identical at merge
time — but nothing had caught it, and the next drift need not be neutral.

Implementation Notes

Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching
across skill directories, which is why two separate skills needed embedded copies; two files inside
ONE skill may source a third. That is the whole reason the merge removes duplication rather than
relocating it.

The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those
words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path
before any validation, and its table mirrors validate.sh's detection exactly: a directory holding
SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory
(agent); anything else stops without running a validator. Steps 1-3 live in
references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in
that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the
other artifact type, the body restarts at Step 0.

Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that
matches a live sibling reads as ownership rather than membership.

The description carries one arrow per boundary target, because ADR-0020 resolves only the first
target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated
"audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16%
of the body: no size SUGGESTIONs.

The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin
cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml
where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both.
tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources
the resolver and that no third copy exists.

The entry scripts classify the target from its resolved parent directory, so a bare agent filename
typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is
missing, rather than dying with exit 1, the tier the flows relay as real findings.

The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and
return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's
status (`f || RC=$?`) disables errexit for its entire body, and no subshell or
`set -e` inside can re-arm it once the call sits in a condition context
(measured, both spellings). Their error paths use `exit`, which is unaffected
either way; this keeps errexit armed for anything added later.

Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is
access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's
dev environment is root -- so the guard could never fire where it exists to fire.
A read attempt is also the stricter question, catching EIO. This is the reasoning
scripts/check-vale-style-sync.sh carried before this commit deleted it; the
hazard did not go with it.

All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared,
the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise
print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two
remaining bare cd calls take absolute paths, which CDPATH is never consulted for.

Impact

BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to
2.0.0 (catalog 0.4.7).

Check logic is unchanged: differential runs of the old and new validators across every skill and
agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads
differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename.
One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a
target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change,
deliberately (full table in ADR-0025):
- a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is
  by path shape, so a shape-matching path that is simply absent used to reach the validator and come
  back as a FAIL against a file that never existed;
- an unshaped target exits 2 under the generic "matches neither" message, and a directory with no
  SKILL.md under a third, distinct one -- three exit-2 messages, not one;
- a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding
  about the artifact rather than a usage error;
- a SKILL.md file path is audited as its skill directory instead of refused;
- a .md agent outside an agents/ directory is refused rather than audited;
- a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2.
  validate-provenance.sh already exited 2 for the last two; validate.sh now matches it.

.pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both
files: regexes are unchanged; only entry: and description: moved.

scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21),
tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The
checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into
tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the
cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both
hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation
self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite.

Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8.

ADR: 0025
BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by
  factory-audit, which auto-detects whether it was handed a skill directory or an agent file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD

2026-09-16 09:13:57 +00:00

18 KiB

Raw Blame History

A plugin's published description states its domain boundary and never enumerates its skills

Three of this repo's six plugins publish a description that lists the skills they ship. That style has now failed three times in four days, the third time inside the correction for the second. It is enforced by nothing, it obliges a marketplace release on every skill addition, and it was never applied to the other three plugins. This ADR retires it: a published description says what the plugin is for, and the inventory lives where an inventory can be read off the tree.

Status: accepted (2026-08-17).

Amended by ADR-0024 (2026-09-14). The decision stands unchanged — a published description states a boundary and never enumerates skills — but three of the four compile targets named below no longer exist. 718c79a deleted every per-plugin .claude-plugin/plugin.json and .github/plugin/plugin.json; 0dffff3 deleted the repo-wide .github/plugin/marketplace.json mirror; and .agents/plugins/marketplace.json with the codex profile producing it was removed 2026-09-13. A description is still authored twice — plugins/<name>/apm.yml and root apm.yml's marketplace.packages[] — but now compiles into one file, .claude-plugin/marketplace.json. Read the "four generated files" in Context and the "eight generated files" in Consequences as historical counts, true when written. The blast radius shrank; the staleness hazard that motivated this ADR did not.

Context

A plugin's published description is one string authored twice — in plugins/<name>/apm.yml and in the matching marketplace.packages[] entry of the root apm.yml — and compiled into four generated files per plugin edit: the plugin's .claude-plugin/plugin.json and .github/plugin/plugin.json, plus the repo-wide .claude-plugin/marketplace.json and its .github/plugin/marketplace.json mirror. (.agents/plugins/marketplace.json, apm's codex profile, carried no per-package description or version at all and was unaffected — that file and the profile producing it were removed 2026-09-13; see the amendment above.) It is the only text a consumer sees in a marketplace listing before installing. It is not a SKILL.md description: it is never preloaded into an agent's context and routes nothing at runtime. ADR-0020 governs that other artifact; this one governs this one. The overlap is a finding, not a scope: ADR-0020 established that capability enumeration in a description is "a correctness hazard, not only a token cost". The hazard at this layer is different — staleness in published metadata rather than an agent shortcutting the body — but the enumeration is the same construct and it fails the same way.

Measured at de84d1b, the branch tip before this change. Each figure is reproducible from the tree: skill counts are ls plugins/<name>/.apm/skills/ | wc -l, description text is plugins/<name>/apm.yml.

Plugin	Style	Skills	Items enumerated	Skills named	Unnamed
`bin`	enumeration	11	8	9	`caveman`, `zoom-out`
`git`	enumeration	9	8	8	`git-workflow`
`gitea`	enumeration	7	7	6	`gitea-workflow`
`core`	boundary	3	—	—	—
`kyberforge`	boundary	7	—	—	—
`lint`	boundary	2	—	—	—

Three failures, in order.

bb9158d (2026-08-14) — core's description described bin. The text it deleted read "Cross-cutting utility skills for everyday AI-assisted coding — triage, diagnosis, architecture review, and session navigation." All four items are real skills and not one of them is core's: they are bin's triage, diagnose, improve-codebase-architecture and zoom-out. core ships agentsmd-author, agentsmd-audit and provider-adapter-author, and the published description named none of them.

This is the failure the whole style was later adopted against, and it is worth being exact about what it was, because the record has been read the other way twice since. It was wrong content, not an incomplete list. The description was a syntactically perfect, complete, four-item enumeration of a real skill set; it just belonged to a different plugin. Enumerating harder could not have caught it, and a gate that asked "does every enumerated item exist as a skill?" would have passed it — all four did exist. bb9158d's own fix went the other direction: it replaced the enumeration with a domain boundary, and core has needed no correction since. The precedent set by that commit was therefore boundary, and the two commits below cite it while doing the opposite.

65bac15 (2026-08-17) — git advertised gitea's domain, gitea advertised a skill that does not exist. git read "conventional commits, branch management, pull requests, and feature flow"; pull requests reach the forge over HTTP and are gitea's, which is the exact boundary docs/spec/architecture.md draws between the two plugins. gitea read "issues, pull requests, milestones, releases, and wikis"; grep -ri wiki plugins/gitea/.apm/ returns nothing and no wiki skill has ever existed. Both were repaired by re-enumerating.

de84d1b (2026-08-17) — the re-enumeration was itself incomplete. bin's "A place for things to be binned" was replaced with an eight-item list over eleven skills; caveman and zoom-out are absent. zoom-out is the same skill bb9158d had called "session navigation" three days earlier while deleting it from the wrong plugin's description — named when it was in the wrong place, unnamed once it was in the right one. And the miss is not confined to bin: git-workflow is unnamed in git's corrected description, though 65bac15's own commit message states it was added ("omitting pc-author/pc-run, git-submodules and git-workflow"), and gitea-workflow is unnamed in gitea's. Across the three plugins, 23 of 27 skills are named at the third attempt.

Nothing checks any of this. scripts/check-manifests.sh does not contain the string description. The three ADR-0020 validators (scripts/skill-size-check.sh and skill-audit's and agent-audit's validate.sh — two since ADR-0025 merged the audit pair into factory-audit, whose single auto-detecting validate.sh carries both) gate on SKILL.md and agent frontmatter; they do open apm.yml, but only to read dependencies.apm when resolving the boundary-target universe — none of them reads the description: key, and their hook globs match SKILL.md and *.agent.md only. apm audit --ci, apm pack --check-clean and scripts/sync-plugin-content.sh --check --all all compare compiled output against apm.yml, so their entire job is to propagate whatever the description says into those four files byte-for-byte and confirm they match. The wiki claim passed every one of the fourteen pre-push hooks, every day it was published.

Correction (2026-09-14): that gate list is down to one, and it was never two. scripts/sync-plugin-content.sh --check --all does not exist — 718c79a deleted the script and its check-plugin-content-sync hook with the flat mirror (ADR-0024). Of the two names left, apm audit --ci was never a drift gate at all: against this repo it checks only that each apm.yml parses and that a manifest declaring dependencies has a consistent apm.lock.yaml, and it reads no description. So the sole surviving gate that compares compiled output against apm.yml is apm pack --check-versions --check-clean --dry-run, run by the apm-pack-check-clean pre-push hook — and with the per-plugin manifests gone it propagates a description into exactly one file, .claude-plugin/marketplace.json, not four. This narrows the mechanism and changes nothing about the finding: propagation is still not verification, and nothing anywhere reads the description key for sense.

And the obligation is unbounded. Under enumeration, adding one skill to bin, git or gitea means editing two copies of a prose string on top of the version bumps and regeneration any skill addition already owes under this repo's release policy (plugins/kyberforge/.apm/skills/apm-workflow/references/marketplace.md). The bumps are not the marginal cost — the prose edit is, and it is the half nothing checks. A skill rename triggers the same, for a string no consumer can tell went stale. 27 of the repo's 39 skills sat behind a description carrying that obligation; the other 12 did not, and their three plugins have generated no defect of this class.

Scope

This decision covers the six plugins this repo authors. The root marketplace also lists mattpocock-skills, a third-party package whose description is not this repo's to write; its entry is out of scope and is left as published upstream.

(Note, 2026-09-13: mattpocock-skills has since been removed from the root marketplace. This section's scope statement is retained as the reasoning behind the boundary; the entry it describes no longer exists.)

Decision

A plugin's published description states the plugin's domain boundary. It does not enumerate the skills the plugin ships, by name or by paraphrase.

The boundary answers "what kind of work belongs to this plugin, and where is its edge against its nearest sibling" — the question a consumer deciding whether to install is actually asking. It is stable under skill addition, rename and removal, which is the entire point: an artifact that does not change when the tree changes cannot go stale against it.
The boundary must cover everything the plugin actually ships. A boundary drawn narrower than the contents is the same defect as an incomplete enumeration, one level up, and it is the specific risk in this change. git carries pc-author and pc-run, which are not git operations at all; "Skills for working with Git" silently drops them, so the boundary names the pre-commit hooks explicitly rather than trusting a reader to file them under Git.
The two copies — package apm.yml and the root marketplace.packages[] entry — stay identical. This is already the rule in practice and both prior corrections state why: the root entry is what reaches the compiled marketplace, so fixing only the package manifest leaves it half-propagated.

The three descriptions, rewritten here, with core/kyberforge/lint shown for register:

Plugin	Published description	Chars
`bin`	Skills for everyday AI-assisted development work that is not tied to a single tool, forge or language, and has not yet been split into a focused plugin.	152
`git`	Skills and agents for working with a local Git clone over the git wire protocol, and for authoring and running the pre-commit hooks that guard it.	146
`gitea`	Skills and agents for working with a Gitea forge through its HTTP API — the forge's own objects, as distinct from the local git clone.	134
`core`	(unchanged) Skills for authoring and auditing a repo's AGENTS.md and the provider adapter files that defer to it.	101
`kyberforge`	(unchanged) Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace.	105
`lint`	(unchanged) Skills and agents for configuring and running linters.	54

No gate is added. This is a deliberate omission and the reasoning is below, not an item left for later.

Why no gate

The check enumeration would need — "every skill directory appears in the description" — was writable in principle and was never written, including by the two commits that corrected an enumeration by enumerating again and had every reason to. It is also only half a check: it catches a skill missing from the list, and it cannot catch wiki, because "this noun does not name any skill" requires a vocabulary of permissible non-skill nouns that no one is going to maintain. Under a boundary there is no correspondence left to check, which is the property being bought.

What survives un-gated is bb9158d's actual failure: a boundary that is simply wrong about its plugin. That was never machine-checkable in either style — the text was a well-formed description of a real plugin — and it is caught by the same review that has to happen when a published, consumer-facing string is edited at all. A gate that would catch it needs a declared per-plugin skill-to-boundary mapping for the description to be checked against, which is a second artifact requiring exactly the per-skill maintenance this ADR exists to delete, relocated one file over.

Two cheap partial gates were considered and rejected in the same breath. Forbidding a comma-separated run of three or more noun phrases is a prose heuristic that fires on lint's perfectly good "configuring and running linters" class of sentence. Forbidding any string matching a skill directory name under plugins/<name>/.apm/skills/ bans legitimate boundary vocabulary — git-branches exists, and a git boundary has every right to say "branches". Both would be believed, and both would be wrong, which ADR-0020 already records as worse than no gate.

Considered options

Keep enumeration and gate it. The only option that makes the current style safe. Rejected on the three grounds above: the check is one-directional, it cannot see an invented capability, and it makes a marketplace release the consequence of adding a directory. It also hard-couples published consumer copy to internal directory names, so a skill rename becomes a version bump on the plugin and on the marketplace.

Enumerate consistently across all six plugins, on the grounds that the real defect is the split style. Rejected: it takes an obligation that has produced three failures on three plugins and applies it to six. The measured outcome of the most recent attempt to enumerate carefully, with the defect fresh and two prior commits as precedent, is four skills unnamed.

Cap the description length, mirroring ADR-0020's 250/400-character tiers, on the theory that a short description has no room to enumerate. Rejected because length does not measure correspondence: gitea's failing description was 96 characters and asserted a skill that has never existed, while bin's 176-character enumeration is under the same cap. All six descriptions here, before and after, sit inside ADR-0020's tiers; the tier would have been silent through all three failures.

Delete the description to a bare name. Rejected: apm's Claude marketplace mapper emits description into marketplace.json, and it is the only prose a consumer sees before installing.

Point the description at the plugin's README.md. Rejected: a marketplace listing renders a string, not a link — and the README's own plugin list carries the same enumeration with the same staleness, so this relocates the defect rather than fixing it.

Consequences

Three descriptions are rewritten and the compiled output regenerated. Eight generated files change: plugins/{bin,git,gitea}/.claude-plugin/plugin.json, plugins/{bin,git,gitea}/.github/plugin/plugin.json, .claude-plugin/marketplace.json and its byte-identical .github/plugin/marketplace.json mirror. .agents/plugins/marketplace.json (the codex profile) is unchanged and correctly so — it carries no per-package description or version field at all, only name, source, policy and category.

Version bumps, all PATCH under the per_package strategy: bin 1.1.4 → 1.1.5, git 1.3.4 → 1.3.5, gitea 1.3.5 → 1.3.6, marketplace.version 0.4.4 → 0.4.5.

The root apm.yml top-level version: is restored to lockstep with marketplace.version, 0.4.2 → 0.4.5. These two fields have moved together in every commit that has ever touched root apm.yml — 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.4.2 in both — until 65bac15 and de84d1b on this branch bumped marketplace.version to 0.4.3 and then 0.4.4 while leaving the top-level field at 0.4.2. Lockstep is not folklore: it is stated at plugins/kyberforge/.apm/skills/apm-workflow/references/marketplace.md. This is a defect, not a style: apm.yml's comment inside the marketplace block records that the top-level version: is not inherited into the compiled output "despite being used elsewhere (e.g. by apm audit)", so the field is live and was silently two releases behind what the marketplace published. Closed here rather than tracked, because the correction is one line and the drift is three days old.

docs/spec/architecture.md's plugin table is unchanged and stays a routing table. It answers "where does a new skill go" for someone working inside this repo; the published description answers "should I install this" for someone outside it. The two now read similarly, and that is not duplication to collapse — they have different readers and different lifecycles, and the table already says so in its own preamble ("These are routing boundaries, not inventories"). One caveat for whoever next edits that page: its closing sentence sends a reader to the published description "for what a consumer actually gets", which was true against an enumeration and is now a pointer to a second boundary statement. Neither artifact carries an inventory after this change, so that sentence was rewritten in the same branch to point at plugins/<name>/.apm/skills/ and README.md instead.

README.md's plugin bullet list becomes the only place an inventory lives, and it still enumerates. That is deliberate, but it makes the list load-bearing in a way it was not before, so its bin, git and gitea bullets were completed in the same branch to name every skill those plugins ship. This ADR does not otherwise extend to it: a README is a hand-read document where a list of what you get is the useful thing, it is not compiled into four files, and a stale line in it costs a reader a moment rather than misrepresenting a published package. The tradeoff that makes enumeration wrong in a marketplace manifest is precisely the one that makes it fine there.

Nothing in the ADR-0020 gate set changes. Its character and word tiers, its Vale rules and its three validators all read SKILL.md and *.agent.md frontmatter; none of them opens an apm.yml. The two contracts are adjacent and independent, and a future author retrofitting a skill under issue #99 is not touched by this ADR.

The failure mode this leaves open is a wrong boundary, and it is un-gated by design. If a fourth failure of this class occurs it will be a description that describes the wrong plugin — bb9158d's shape, the one enumeration never addressed. That is the trigger to revisit, and the thing to build then is a declared skill-to-boundary mapping, not a return to enumeration.

18 KiB Raw Blame History