Three ways the gates could report green having measured nothing. All three were
invisible to a passing test suite, because pre-commit prints nothing at all for a
hook that exits 0 — a gate that declines to check and a gate that checked and
passed produce the identical signal.
- A UTF-8 BOM, a leading blank line, a trailing space after a `---` marker or
CRLF line endings defeated the `^---\n` frontmatter matcher. Every ADR-0020
check was then skipped and the file passed: measured at the time, a
550-character description with a 1,000-word body exited 0 behind a BOM.
All four shapes are now tolerated, and frontmatter that genuinely cannot be
parsed is a hard ERROR rather than a silent skip.
- An agent file with a valueless `description:` followed by another key let a
line regex capture the *next* key, which looked non-empty, so the
missing-or-empty branch never fired and every gate below it early-returned on
the empty folded value — zero output, exit 0, on a blocking gate. The one
field this contract is entirely about was the one field a gate could fail to
notice was absent. Presence is now decided on the YAML-folded value and
nowhere else, and a missing or empty description is a hard FAIL in all three
validators.
- The hand-rolled frontmatter fallback disagreed with PyYAML across the FAIL
boundary on folded scalars, so which reader happened to be available decided
the verdict. A fallback that mis-parses a scalar shape reports a vacuous pass,
which is worse than not running, so it is deleted: python3 and PyYAML are hard
requirements that fail loudly with an install pointer.
Boundary-target resolution no longer derives its universe from its own location.
A `${BASH_SOURCE}`-relative repo root leaked this repo's 39-skill universe into
every consumer repo running the hook through pre-commit, so a consumer skill
routing to `skill-audit` resolved against a plugin it had never installed. The
interim form resolved through `.claude/` and `.agents/`, which are gitignored
`apm install` output — the same commit reported 2 dangling targets on a machine
that had run the install and 6 on a fresh clone. Resolution now walks up from the
file being checked to an authoring root (nearest ancestor holding
`plugins/*/.apm/{skills,agents}`, else the nearest `.git`, in two passes so a
nested `.git` cannot outrank a real monorepo root); the universe is every skill
and agent under `<root>/plugins/*/` plus the file's own apm package and that
package's declared `dependencies.apm`. Deployed trees are consulted only when no
authoring root exists at all — the consumer case. One commit now gets one verdict,
which a gate shipping hot with no baseline file has to.
Narrowed in the same pass: a routing target inferred from the prose boundary form
and corroborated by nothing else reports at SUGGESTION instead of blocking. A
blocking check with no escape hatch is the wrong trade when the inference from
prose is the weak part of it.
New deterministic checks, all previously untested or absent: every
`references/<file>.md` a body names must exist (ERROR — a broken pointer is not a
style opinion); a description with no boundary clause at all, a Gotchas section
over five entries, and a Gotchas section over 25% of the body are SUGGESTIONs.
Where no universe can be determined the target check prints `INFO ... DID NOT
RUN` rather than passing quietly. Each prose-scanning check needed its own
false-positive fix — a fenced example of a Gotchas section was being read as the
section itself — and those fixes are pinned rather than assumed.
The resolver is one block copied verbatim into all three scripts between
BEGIN/END markers, because a cache-installed plugin's scripts cannot read outside
their own plugin directory. Nothing asserted the copies were still identical; a
one-line edit to a single copy passed every constant-agreement assertion, since
constants are not what drifts.
Tests land here rather than in a later commit. The existing suites assert the old
behaviour and go red against these scripts, so splitting them would leave a commit
whose own `run-tests` pre-push gate fails in isolation.
Refs: ADR-0020
Skill name+description pairs are preloaded into every session, costing
~6,200 tokens across 39 skills before any skill is invoked. The authoring
rules mandated that growth: skill-author:104 and description-quality.md:21
both required padding, while skill-author:102 (the deflating rule) had no
FAIL condition behind it.
Gates (blocking, no baseline file):
- description 250 chars SUGGESTION / 400 FAIL, measured on the folded
YAML value
- body-only 600 words SUGGESTION / 900 FAIL, independent of the unchanged
whole-file 2770-word / 500-line spec backstop
- every boundary-clause routing target must resolve to a real skill or
agent; catches skill-improve, neuledge-context and gitea-labels
- agents take the description gates but deliberately no body gate; a test
pins that absence
Vale: DescriptionOpener widened to ^This\b, new CompositionNote rule
banning architecture notes from descriptions. 10 hits, 0 false positives.
Kyberforge's own four skills retrofitted: descriptions 3,364 -> 938 chars
(-72%), bodies 8,306 -> 2,487 words (-70%), all via the apm-workflow
dispatch pattern. Fixes the skill-improve dangling route and the
agent-author misroute to manual review.
Also fixes a pre-existing false positive where any line-initial 'read '
was flagged as interactive input, which had already caused two scripts to
be rewritten around it.
Refs: ADR-0020
Two efficiency findings from a code-review pass:
- The separated (--config X) and joined (--config=X) argument branches
duplicated ~20 lines of absolutize-if-relative path logic. Extracted
into abs_config_value(), used by both branches; the redundant
--config=/* special case falls out since the helper already passes
absolute paths through unchanged.
- The common single-file path spawned python3 twice per file (once for
abspath resolution, once for flatten()). flatten() now optionally
takes a tmpdir arg and does both in one process. The per-file loop
under a directory argument is unchanged — that path wasn't flagged.
agent-audit's copy is canonical; skill-audit's copy was regenerated via
scripts/sync-vale-styles.sh, not hand-edited, to guarantee byte parity.
No hardening (bash 3.2 compat, surrogateescape, symlink guards) touched.
Verified: tests/test-vale-wrap.sh 39/39, check-vale-style-sync.sh clean,
full suite 12/12.
MAX_WORDS=2900 was calibrated to the corpus median density and carried no
margin: at the densest observed 7.22 chars/word (~1.81 tokens/word) it permits
~5,240 tokens against the 5,000 it proxies for. 2770 holds the worst observed
density under the ceiling. The largest SKILL.md is 2,489 words, so the change
costs nothing today — 281 words of margin — and the header comment now argues
the new calibration rather than swapping the digits.
Both enforcement points move together, and a new test asserts they agree, since
a SKILL.md passing its own audit while the commit hook blocks it is the
disagreement this pair exists to prevent.
CONTEXT.md is deliberately left ungated: it is 2,816 words, and gating it would
block the build. Recorded here so the omission reads as a decision rather than
an oversight.
skill-audit's manual-fallback path listed only the line ceiling, so an agent
taking that path passed an oversized SKILL.md the hook then rejected. The word
ceiling is now named alongside it. agent-audit is deliberately unchanged: the
size hook scopes to SKILL.md only and agent-audit's validate.sh has no word
gate, so claiming it there would be false.
The Vale research doc still showed the MDX {/* vale off */} form under a
Markdown heading, contradicting CONTEXT.md and vale-run's troubleshooting
reference — that form suppresses nothing in plain .md. Fixed in both places it
appeared.
tests/run-tests.sh used mapfile (bash 4.0+) with unguarded array expansion,
though AGENTS.md tells contributors to run it and macOS ships bash 3.2. It now
collects via a while-read loop over process substitution and guards every
expansion. The newline-delimited find|sort pipeline is kept rather than -print0
with sort -z, whose BSD portability is the weaker link, and which matches
mapfile -t's previous behaviour exactly.
Refs: #85
ADR: 0013
The flattener's last-resort branch rewrote ASCII ' to U+2019, justified as the
one combination no YAML scalar can carry verbatim. That claim was false: a |-
literal block with a single indented content line carries ', ", \ and ": "
verbatim and keeps text.frontmatter.description matching — as the wrapper's own
docstring already said of literal blocks. The rewrite fired on 12 of 54
in-scope files, silently disabling every rule whose token contains an
apostrophe. Case 20 pinned only that the scope stayed alive, so it passed
either way.
The emission site now splits the emitted scalar on its first newline so a
carried-over trailing comment stays on the "description: |-" header line rather
than becoming part of the value, and pads by span_lines - 1 - newlines. The pad
stays non-negative because the branch is only reachable when the original span
is at least two lines. Verified across all 73 in-scope files: no line-count
changes, and exactly the 12 expected files take the new branch.
One reported position moves: an alert on a description that is itself flagged
shifts from the key line to the block's content line, both inside the original
span. YAML cannot put a literal block's content on the key's own line, so this
is unavoidable; no line at or after the end of any description span moves.
Also: --output no longer absolutises the built-in style names line, JSON and
CLI, which a same-named file or directory in cwd turned into a template path
(exit 2, E100 Runtime error). And case 19's empty-baseline guard no longer
lets five dependent comparisons print vacuous passes — while fixing it the
guard turned out to be unreachable, since under pipefail an alert-free report
aborted the script at the assignment.
Refs: #85
ADR: 0014
Three divergences between what the audit skills claim and what the hooks
enforce, each of which fails silently rather than loudly:
- `skill-size-check.sh` blocked at 2900 words while `validate.sh` checked only
the 500-line ceiling, so `/skill-audit` could report a skill ready to ship
that the commit hook then rejected. `validate.sh` now checks the same pair on
the same inclusive terms; the constants are duplicated with a comment naming
the other file, because a plugin skill's scripts cannot read outside the
plugin directory once installed to the cache.
- Both audit skills' Step 1 passed `--config assets/vale/.vale.ini`, which is
redundant (the wrapper self-locates its sibling config) and fragile: an agent
that resolves the script path against the skill directory but not the config
path gets E100, exit 2, which the surrounding fallback clause misreads as
"vale unavailable" and downgrades to full LLM judgment with no signal.
- The external-consumer test registered only the two Vale hooks, never the
third shipped hook, so a lost executable bit would have broken every consumer
while the local suite stayed green. Verified by mutation: `chmod 644` on the
copied script now turns three passes into two failures.
Also corrects the size hook's calibration comment, which claimed ~5.7-6.5
characters per word against a corpus whose measured median is 6.79 — the stated
upper bound sat below the median, so the "calibrated with margin" claim was
inverted for prose-dense files. MAX_WORDS is unchanged pending a decision; the
comment is now explicit that the gate holds under 5,000 tokens for typical
prose density, not for any file.
Refs: #85
Vale locates a frontmatter description by matching the parsed YAML value back
against the source text, so any scalar whose value is not spelled out verbatim
loses the `text.frontmatter.description` scope entirely. The wrapper only
flattened `>` folded scalars, so plain, double-quoted and single-quoted
multi-line descriptions silently reported zero alerts and exit 0 — a clean pass
indistinguishable from a real one, in a prefilter whose callers are instructed
not to re-derive its verdict by judgment.
Implementation notes:
- Classify the scalar kind after `^description:[ \t]*` and reuse one shared
continuation-line generator for every form; `|` literal blocks keep their
line breaks, stay verbatim-matchable, and are still left untouched.
- Emit the flattened value in whichever scalar form needs no escape at all
(plain, then single-quoted, then double-quoted), because any escape breaks
the verbatim match. The old blanket `'` -> U+2019 substitution silently made
apostrophe-bearing rule tokens unmatchable across 63% of the corpus; it now
survives only for the one combination no YAML scalar can carry verbatim.
- Terminate continuations at a line flush with the key, not only on a shallower
indent — a `description:` followed by a flush-left line previously swallowed
the rest of the frontmatter.
- Route vale's value-taking flags explicitly instead of inferring targets by
file existence, and absolutize relative `--output`/`--path` values the way
`--config` already was, since the run `cd`s into the scratch mirror.
- Fail loudly on a nonexistent path instead of inheriting bare vale's fallback
to stdin, which rendered a typo'd path as `0 errors ... in stdin`, exit 0 —
a form the callers' `0 files` NOT-RUN guard cannot match.
- Follow symlinks when walking a directory argument, matching bare vale.
Refs: #85
Under set -u, "${arr[@]}" on an empty array aborts on bash before 4.4,
which is what macOS ships as /bin/bash. Three expansion sites now use
${arr[@]+"${arr[@]}"} consistently.
The hazard is not currently reachable: verified on a bash 3.2.57 built
from source that all seven invocation shapes succeed against the
previous code, including zero args, flags-only and an empty directory.
vale_args is provably non-empty at every site because the default
--config branch always appends first. The guard is kept because that
invariant is non-local and untested, so an edit to the default-config
branch would reintroduce a macOS-only crash silently.
Test fidelity is deliberately mixed. Case 16 is static and is the only
one that fails against the previous code, since no bash 5 host can
reproduce the abort at runtime. Case 17 runs the emptiest invocations
under the oldest bash it can find and names that shell in its output
so it cannot overclaim. Case 18 guards against the tempting wrong fix
of dropping the quotes, which also silences the abort but word-splits
a path containing a space.
No other bash 4.x construct is present; swept for mapfile, declare -A,
case modification, negative indices, globstar, wait -n and namerefs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
pre-commit prefixes only entry[0] with the hook-repo clone path
(cmd = (prefix.path(cmd[0]), *cmd[1:])), so the --config argument in
.pre-commit-hooks.yaml resolved against the *consuming* repo's root
and hard-failed every external run with E100. Two of the three hooks
ADR-0014 promises were unusable.
vale-wrap.sh now self-locates its config from BASH_SOURCE when no
--config is supplied; an explicit --config still wins in all three
argv forms and stays cwd-relative. Both manifests drop the argument
and are kept byte-identical: the local repo: local config resolved
--config correctly only because the consuming repo *was* this repo,
and that divergence is why three review rounds missed the defect.
Also in the wrapper:
- replace GNU-only `realpath -m` with a portable abspath helper; -m is
load-bearing (dest does not exist yet), so BSD realpath aborted the
script under set -e on macOS
- walk directory arguments instead of passing them through unflattened,
which reported a clean 0-error run for files that fail when named
explicitly
- read/write with errors='surrogateescape' so one non-UTF-8 .md under a
directory argument cannot abort the hook
New test-vale-hooks-consumer.sh builds the hook repo from the working
tree and points a file:// consumer at it, covering the manifest as a
hook repo for the first time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
skill-audit/agent-audit's Step 1 resolved vale-wrap.sh/.vale.ini via
`git rev-parse --show-toplevel`, which returns whichever repo the skill
happens to run in. Inside ai-development that works; in any external repo
that installs kyberforge@holocron as a plugin, it resolves to that repo's
own root, which has no .vale.ini — the prefilter silently fell back to
full LLM judgment. ADR-0013 named this as a deliberately deferred gap.
Vale's config/styles/wrapper now ship inside the plugin itself: a
canonical copy in agent-audit/assets/vale/ (Kyberforge + KyberforgeCopilot,
the superset agent-audit needs) and a smaller duplicate in
skill-audit/assets/vale/ (Kyberforge only) — per the no-cross-skill-path
rule already established for plugin cache-installs. Both skills resolve
these relative to their own directory, same as scripts/validate.sh
already does.
A new root .pre-commit-hooks.yaml exposes both copies plus
skill-size-check so any external repo can enforce the same rules via
`repo: <this-repo-url>, rev: <tag>` in its own pre-commit config,
independent of Claude Code entirely — the same mechanism covers CI. This
repo's own pre-commit hook now consumes the identical plugin-bundled
copies via repo: local (not a third root copy, and not a pinned
self-reference, which would lint working-tree edits against the last
tagged release instead of the change being made). Split into
vale-audit-prefilter-skill/-agent hooks after confirming, by diffing the
full corpus against both old and new config before deleting the old
files, that one combined hook pointed at only one copy silently 0-file-
skips the other file type.
scripts/check-vale-style-sync.sh guards the two copies against drift,
wired at pre-push alongside check-manifests.
ADR: 0014
Add validate-provenance.sh and validate-provenance.bats to enforce the
sources provenance chain introduced by skill-author. Eight checks cover
slug cross-references, Contributing files existence, bidirectional
source_keys linkage, Research doc: field presence, and upstream research
doc alignment (forward INFO, reverse FAIL). Adds a new Provenance report
dimension and INFO finding level (observational, exit-0, counted
separately as · P info in the result block).
Closes#8
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- validate.sh: move FAIL lines and failure summary to stdout; stderr
reserved for fatal script errors only (missing SKILL.md, bad args)
- skill-audit SKILL.md: replace concrete plugins/kyberforge/skills/...
example with abstract placeholder to fix meta-circularity
- skill-write SKILL.md: rephrase placeholder section-heading instruction
to remove embedded FILL IN: from a code span, clearing validator false positive
- skill-improve SKILL.md: wrap Step 2 root-cause example in a text fence
- skill-write assets/templates/README.md: update Files table to individual-
file rows so skill-audit can verify per-file coverage
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Move test infrastructure (validate.bats, new-skill.bats) from scripts/ to
tests/ — the spec defines scripts/ as executable code agents can run, so
test files don't belong there. Add tests/README.md placeholders with
bats-support dependency declaration.
Update skill-audit to permit tests/ and flag other unlisted directories,
add scripts/ purpose check, and add /skill-improve near-miss exclusion.
Update skill-improve and skill-write to cover tests/ in directory lists,
scaffold template, and authoring guidance.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace verbose three-pass output (punch list + priority table + fix
proposals) with a compact findings-only report: coverage line, findings
grouped by dimension with Why+Fix per entry, and a result block with
/skill-improve handoff. Suppress PASS lines — absence confirms pass.
Fix validate.bats executable bit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bats files moved up to scripts/ directly; tests/ subdirectory was non-spec
and created a directory structure not defined by agentskills.io.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reorder skill-audit description to lead with 'Use when...' trigger (P3)
- Add concrete example to 'control calibration' body discipline check (P4)
- Add bats test files to README file tables for both skills
- Fix REPO_ROOT and SCRIPT paths in bats files after tests/ subdirectory removed
- Add three lessons: plugin cache isolation, spec-grounded rubrics, test file placement
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move validate.sh ownership to skill-audit/scripts/ — it is the canonical
structural validator; skill-write now delegates Step 5 to /skill-audit
- Add skill-write/references/scripts.md and deployment-modes.md for progressive
disclosure of package runner patterns and plugin cache isolation rules
- Fix skill-audit Step 1 cross-skill path reference (was repo-absolute, now
skill-relative); add manual fallback for sandboxed/Bash-denied contexts
- Scope Step 2 "read every file" to exclude binaries and unreferenced files
- Fix new-skill.sh next-steps output to reference /skill-audit instead of
the removed validate.sh
- Remove stale Dependencies section from skill-audit README; flip dependency
arrow — skill-write depends on skill-audit, not vice versa
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
