Why
The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary
resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser
(93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line
test suite. Sync-by-gate had already failed once: at 484357a the two parser copies drifted into
different spellings of the bullet loop while a docstring asserted they were identical. That drift
was behaviour-neutral and was re-unified by hand at 598a7c3, so the copies were identical at merge
time — but nothing had caught it, and the next drift need not be neutral.
Implementation Notes
Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching
across skill directories, which is why two separate skills needed embedded copies; two files inside
ONE skill may source a third. That is the whole reason the merge removes duplication rather than
relocating it.
The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those
words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path
before any validation, and its table mirrors validate.sh's detection exactly: a directory holding
SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory
(agent); anything else stops without running a validator. Steps 1-3 live in
references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in
that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the
other artifact type, the body restarts at Step 0.
Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that
matches a live sibling reads as ownership rather than membership.
The description carries one arrow per boundary target, because ADR-0020 resolves only the first
target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated
"audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16%
of the body: no size SUGGESTIONs.
The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin
cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml
where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both.
tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources
the resolver and that no third copy exists.
The entry scripts classify the target from its resolved parent directory, so a bare agent filename
typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is
missing, rather than dying with exit 1, the tier the flows relay as real findings.
The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and
return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's
status (`f || RC=$?`) disables errexit for its entire body, and no subshell or
`set -e` inside can re-arm it once the call sits in a condition context
(measured, both spellings). Their error paths use `exit`, which is unaffected
either way; this keeps errexit armed for anything added later.
Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is
access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's
dev environment is root -- so the guard could never fire where it exists to fire.
A read attempt is also the stricter question, catching EIO. This is the reasoning
scripts/check-vale-style-sync.sh carried before this commit deleted it; the
hazard did not go with it.
All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared,
the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise
print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two
remaining bare cd calls take absolute paths, which CDPATH is never consulted for.
Impact
BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to
2.0.0 (catalog 0.4.7).
Check logic is unchanged: differential runs of the old and new validators across every skill and
agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads
differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename.
One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a
target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change,
deliberately (full table in ADR-0025):
- a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is
by path shape, so a shape-matching path that is simply absent used to reach the validator and come
back as a FAIL against a file that never existed;
- an unshaped target exits 2 under the generic "matches neither" message, and a directory with no
SKILL.md under a third, distinct one -- three exit-2 messages, not one;
- a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding
about the artifact rather than a usage error;
- a SKILL.md file path is audited as its skill directory instead of refused;
- a .md agent outside an agents/ directory is refused rather than audited;
- a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2.
validate-provenance.sh already exited 2 for the last two; validate.sh now matches it.
.pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both
files: regexes are unchanged; only entry: and description: moved.
scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21),
tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The
checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into
tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the
cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both
hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation
self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite.
Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8.
ADR: 0025
BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by
factory-audit, which auto-detects whether it was handed a skill directory or an agent file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
A five-agent review of 718c79a and d2480b8 found no skill, agent or hook
regressions (39 skills before and after) and confirmed both hook removals
are genuinely moot -- verified against the tree, not taken on the commit's
word. It did find one functional regression (fixed separately) and this
documentation drift.
Counting errors, all from a git pathspec `*` crossing `/`:
- 17 .bats files shipped to consumers is really 10; 17 counted tracked
paths merely containing /tests/, one of them a template asset
- "roughly 88s off every push" is ~92.4s; 88 omitted validate-plugins
- "roughly 70% of each plugin remains live" holds only for kyberforge;
the real spread is 44.3% (bin) to 70.6%, now a table
- the pre-push enforcement row was half-corrected: 33 entries stood
unstruck (now 27) and 14 -> 11 switched counting basis mid-sentence
- the root .claude-plugin/plugin.json was described as "kept"; it has
never been tracked
gates.md said "Ten hooks" above a nine-row table (11 was decremented for
one removal, not two), and "both need the claude CLI" for one remaining
validator. Its pretty-format-json exclude rationale claimed six
alternations expanding to sixteen files in a passage headed "Mind which
number you are quoting" -- four alternations, two live files; the two
dead ones are dropped from the pattern. check-useless-excludes could not
catch this: it only flags an exclude matching nothing at all.
ADR-0024 cited ADR-0006 for a patch-bump rule it does not contain and
which ADR-0015 explicitly retired; stated apm's marketplace probe order
backwards (.claude-plugin/ is the last candidate, not the first, so the
earlier .github/plugin/ deletion only demoted resolution); undercounted
apm's skill-deploying targets as seven when there are fifteen; and never
recorded that validate-plugins was removed. The symlink hedge is resolved:
apm_cli/security/gate.py's ignore_non_content() drops symlinks silently on
deploy while apm_modules/ materialization dereferences them, so content
survives that far and vanishes at install. Accepted with no replacement
guard, per decision -- kyberforge/docs/hooks.md previously asserted a
guard that had been deleted with its script.
Four plugin READMEs still advertised `claude plugin install`; ADRs 0001,
0006, 0013, 0014, 0015 and 0019 described deleted machinery in the present
tense, 0019 most consequentially as the live justification for the
SessionStart hook's .apm/ path. CONTEXT.md's "apm package" entry forbade
"plugin" while using it in its own body, and "Output profile" lost the
antecedent for "one catalogue serves both".
run-tests.sh gains the .claude/skills/ exclusion run-bats.sh already had.
Latent today -- no test-*.sh lives under any .apm/skills/*/tests/ -- but
apm now deploys those directories, so one would be discovered twice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
Documentation asserted things the code no longer did, which is how several of this
round's defects survived three review passes.
- `docs/spec/architecture.md` still described the pre-APM model — "each plugin has a
`plugin.json` manifest" — when no such file exists. AGENTS.md routes agents here
for the current structure, so it was live drift, not archive. Also cited ADR-0012
where ADR-0003 is meant.
- AGENTS.md never said `.apm/` is the sole hand-edited source, while the generated
mirror is byte-identical with no marker — an agent reading only AGENTS.md would
edit the mirror and find out at pre-push. It also omitted that `apm` and `jq` are
now required to push, unlike the `vale` bullet that exists for exactly that reason,
and understated the pre-push gate. `apm marketplace check` hard-fails offline and
`--offline` does not help, so the `SKIP=` escape hatch is documented rather than
new machinery built.
- ADR-0017 carried hardcoded line numbers that drifted twice within one PR, and said
`mcpServers` reinjection was real-syncs-only where the script's own header says
both modes. Line numbers dropped rather than corrected.
- ADR-0015 asserted issue #90 was closed. It is open. Reworded to defer to the
issue's own state rather than depend on someone remembering to close it.
- ADR-0014 said both audit skills pass `--config`; both SKILL.md files say pass none,
and passing one fails with E100. The ADR was wrong.
- ADR-0007 cross-referenced an archived ADR-0011 whose number was later reused.
Disambiguated with a note — renumbering a published ADR rewrites an immutable
record and breaks every existing citation.
- CONTEXT.md claimed `lint-runner` is report-only via a missing `Edit` tool.
Plugin-scope APM agents cannot express `tools:` (ADR-0016), so it is report-only by
instruction now; ADR-0016 accepted that regression but CONTEXT.md was not updated.
- Recorded two undocumented consequences of the conversion: `displayName` dropped
from all six compiled manifests and `owner.email` from the marketplace, and
`mattpocock-skills` silently version-pinned and manually maintained.
- Wrote the version-bump policy issue #90 specified but never delivered. Its previous
carrier, `marketplace-author`, was deleted in this PR, leaving the per-package rule
documented nowhere.
- README fixes: kyberforge listed skills belonging to the `git` plugin and a deleted
hooks path, `bin` advertised an agents directory it never had, and the docs index
listed files that do not exist.
Refs: #90
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
The apm conversion (5e23250) moved skills/agents to plugins/<name>/.apm/
and deleted plugin-author/marketplace-author, but a review against issue
#90's Definition of Done found several stale pre-conversion references
left behind by straight git-mv's, plus one real content gap:
- ADR-0014 still documented the old flat vale-prefilter paths in 6
places, despite ADR-0015 claiming it had been updated.
- ADR-0015 also overclaimed: it said ADR-0014 had both a skills/ and an
agents/ path regex updated, but ADR-0014 never had an agents/ path
regex to begin with (its one "agents" mention is a Vale glob-section
identifier, not a filesystem path). Corrected the wording.
- CONTEXT.md's Skill glossary entry, its Vale-prefilter section, and its
plugin-author/marketplace-author forward-pointer all still described
the pre-conversion layout or a "pending issue #90" state that has since
landed.
- LESSONS.md and two skill test READMEs pointed at skill-author/
skill-audit/agent-author paths without the .apm/ segment.
- apm-workflow/references/marketplace.md cited plugin-author/SKILL.md's
Gotchas for the Claude Code reserved plugin-name-prefix list, but that
list was never actually carried into apm-workflow during the
conversion despite ADR-0015 claiming it was. Recovered the list from
git history and inlined it directly rather than leaving a dangling
citation.
- agent-author/references/deployment-modes.md had an example
contradicting its own stated .apm/agents/ convention two lines above.
Refs: #90
The flattener's last-resort branch rewrote ASCII ' to U+2019, justified as the
one combination no YAML scalar can carry verbatim. That claim was false: a |-
literal block with a single indented content line carries ', ", \ and ": "
verbatim and keeps text.frontmatter.description matching — as the wrapper's own
docstring already said of literal blocks. The rewrite fired on 12 of 54
in-scope files, silently disabling every rule whose token contains an
apostrophe. Case 20 pinned only that the scope stayed alive, so it passed
either way.
The emission site now splits the emitted scalar on its first newline so a
carried-over trailing comment stays on the "description: |-" header line rather
than becoming part of the value, and pads by span_lines - 1 - newlines. The pad
stays non-negative because the branch is only reachable when the original span
is at least two lines. Verified across all 73 in-scope files: no line-count
changes, and exactly the 12 expected files take the new branch.
One reported position moves: an alert on a description that is itself flagged
shifts from the key line to the block's content line, both inside the original
span. YAML cannot put a literal block's content on the key's own line, so this
is unavoidable; no line at or after the end of any description span moves.
Also: --output no longer absolutises the built-in style names line, JSON and
CLI, which a same-named file or directory in cwd turned into a template path
(exit 2, E100 Runtime error). And case 19's empty-baseline guard no longer
lets five dependent comparisons print vacuous passes — while fixing it the
guard turned out to be unreachable, since under pipefail an alert-free report
aborted the script at the assignment.
Refs: #85
ADR: 0014
Four claims in shipped agent-facing docs did not match verified behaviour.
These are read as ground truth by agents in other repos, so each was
reproduced against vale 3.15.2 before rewriting:
- CONTEXT.md and `vale-config/SKILL.md` said both `>` and `|` block scalars
break the description scope. `|` does not — it lints normally and fires every
alert, while `>` yields zero. An agent following the old text would rewrite a
working `|` description into a plain multi-line scalar, which genuinely does
break, inverting the intended remediation. Both now name the forms that do
break and state that `|` does not.
- CONTEXT.md and ADR-0013 described the size hook as failing only above 500
lines, omitting the 2900-word gate it also enforces. Both now describe the
pair and state that `validate.sh` checks the same two.
- ADR-0014 recorded an accepted residual — a wholesale `assets/` deletion going
unflagged — that commit 14c2c91 closed. Left as the point-in-time record and
amended with an update describing the union-with-tag-manifest mechanism,
following the amendment precedent in ADR-0005.
- `vale-config/SKILL.md` asserted a fresh `.vale.ini` fails until `vale sync`
runs, contradicting its own note that built-in styles need no download. The
claim is now scoped to package styles; this repo's two configs declare no
packages and lint clean with zero syncs.
Also repoints AGENTS.md at the seven `gitea:*` skills — the `bin:gitea` route
it named no longer exists.
Refs: #85
cli-reference.md said vale exits non-zero for any alert at or above
MinAlertLevel. The exit code keys on error-level alerts alone;
MinAlertLevel filters display only. LESSONS.md records this exact
misconception as costing two review rounds, and this research doc is
the cited provenance source for the skills that state it correctly.
CONTEXT.md claimed a SKILL.md outside plugins/ matches no glob section.
[**/SKILL.md] matches any path ending in SKILL.md — the sentence is a
stale leftover from the path-scoped globs at cbc33d9, and contradicted
its own paragraph two sentences earlier. The NOT-RUN 0-files guard it
justifies is correct and is unchanged; only the rationale was wrong.
CONTEXT.md also cited the local files: regex as the scoping mechanism,
where the shipped manifest deliberately stays layout-agnostic.
ADR-0014 records the entry[0]-only prefixing constraint as the reason
the self-locating design is required, and that no entry may grow a
repo-internal path argument.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
A review of PR #85's last two commits (1164f3a, 4d018af) found the new
release-gate script fails open in four separate ways, and the new drift
check for the duplicated Vale styles only ever detects drift after a
human already hand-edited both copies out of sync.
check-release-needed.sh:
- The `-e` existence filter dropped a RELEASE_PATHS entry from the diff
pathspec once it was deleted from the tree, so deleting a path exposed
via .pre-commit-hooks.yaml since the last tag passed the gate clean —
exactly the breakage the gate exists to catch. git diff reports
deletions fine without an existence check; the filter is gone.
- `git diff ... 2>/dev/null || true` turned any git failure (a shallow
clone missing the tag's objects, a corrupted ref) into an empty,
falsely-clean diff. The diff result is no longer swallowed: a failure
now hard-fails with the underlying git error visible.
- RELEASE_PATHS was a hand-maintained array duplicating
.pre-commit-hooks.yaml's entry: paths with only a comment holding them
in sync, and was already over-broad (it swept in validate.sh /
validate-provenance.sh, which no hook entry references). It's now
parsed straight from .pre-commit-hooks.yaml's entry: lines at
runtime, so it can't drift from the manifest and only tracks what a
hook actually exposes.
- `git describe --tags --abbrev=0` accepted any tag reachable from HEAD
as the diff baseline, not just release tags. Added
`--match 'v[0-9]*.[0-9]*.[0-9]*'` so an incidental checkpoint tag
can't shift the baseline and mask a real release-relevant change.
check-vale-style-sync.sh still only detects drift between skill-audit's
and agent-audit's duplicated vale-wrap.sh/styles/Kyberforge copies
(both copies must exist independently per the plugin's no-cross-skill-
path packaging rule — a symlink would break at install time). Added
scripts/sync-vale-styles.sh to regenerate skill-audit's copy from
agent-audit's canonical one on demand, and pointed the sync check's
failure message at it, so fixing drift is one command instead of a
hand diff across two files.
Also recorded, rather than silently left unfixed: check-release-needed.sh
only fires on a local `git push` through pre-commit's pre-push hook — a
PR merged via Gitea's merge button, or CI invoking
`pre-commit run --hook-stage pre-push` directly, never sets
PRE_COMMIT_REMOTE_BRANCH and skips the gate entirely. Closing that needs
a server-side CI job this repo doesn't have yet; documented as a known
limitation in ADR-0014 rather than papered over.
Separately, LESSONS.md's "a clean check can mean nothing ran" entry was
marked **Graduated** without ever being promoted per the repo's own
graduation rule (3+ instances → a standing doc, marked
`[graduated → target file]`). Actually promoted it into
core/instructions/testing.md and fixed the marker.
tests/test-check-release-needed.sh gained 4 regression tests, one per
check-release-needed.sh fix above, each verified to fail against the
pre-fix script and pass against the current one.
Verification: bash tests/run-tests.sh (11 scripts + 125 bats, all
passing), pre-commit run --all-files, and
pre-commit run --all-files --hook-stage pre-push all clean.
ADR: 0014
.pre-commit-hooks.yaml now exposes hooks to external consumers pinning
rev: <tag>, but nothing enforced that a tag actually gets cut when the
files it references change — relying on memory is exactly what this
repo's governance rules say to avoid for a repeatable, deterministic
check.
scripts/check-release-needed.sh hard-fails at pre-push, but only when
PRE_COMMIT_REMOTE_BRANCH (set by pre-commit's hook-impl) is
refs/heads/main: it diffs .pre-commit-hooks.yaml's referenced paths
against the last tag reachable from HEAD, and fails if either no tag
exists yet or something changed since. It's a silent no-op on every
other branch — hard-failing on feature-branch pushes mid-review would
force a premature tag on a commit that might not survive a
squash-merge, the exact risk the repo: local (vs. pinned self-
reference) decision in ADR-0014 already avoids for this repo's own
dev-time gate.
Verified against the real git pre-push hook path (not just the script
in isolation): simulated stdin matching git's pre-push protocol through
.git/hooks/pre-push, confirmed it correctly fires and fails when
targeting main with no tag, and is silent otherwise.
ADR: 0014
Refs: #87
skill-audit/agent-audit's Step 1 resolved vale-wrap.sh/.vale.ini via
`git rev-parse --show-toplevel`, which returns whichever repo the skill
happens to run in. Inside ai-development that works; in any external repo
that installs kyberforge@holocron as a plugin, it resolves to that repo's
own root, which has no .vale.ini — the prefilter silently fell back to
full LLM judgment. ADR-0013 named this as a deliberately deferred gap.
Vale's config/styles/wrapper now ship inside the plugin itself: a
canonical copy in agent-audit/assets/vale/ (Kyberforge + KyberforgeCopilot,
the superset agent-audit needs) and a smaller duplicate in
skill-audit/assets/vale/ (Kyberforge only) — per the no-cross-skill-path
rule already established for plugin cache-installs. Both skills resolve
these relative to their own directory, same as scripts/validate.sh
already does.
A new root .pre-commit-hooks.yaml exposes both copies plus
skill-size-check so any external repo can enforce the same rules via
`repo: <this-repo-url>, rev: <tag>` in its own pre-commit config,
independent of Claude Code entirely — the same mechanism covers CI. This
repo's own pre-commit hook now consumes the identical plugin-bundled
copies via repo: local (not a third root copy, and not a pinned
self-reference, which would lint working-tree edits against the last
tagged release instead of the change being made). Split into
vale-audit-prefilter-skill/-agent hooks after confirming, by diffing the
full corpus against both old and new config before deleting the old
files, that one combined hook pointed at only one copy silently 0-file-
skips the other file type.
scripts/check-vale-style-sync.sh guards the two copies against drift,
wired at pre-push alongside check-manifests.
ADR: 0014
