The external-consumer test asserted a combined alert count (>=2) across both
shipped Vale hooks, but the SKILL.md fixture alone raises two alerts — so one
working hook satisfied the threshold. Retargeting agent-audit's glob to match
nothing left the suite reporting "3 passed" under the message "both hooks
flatten and flag". The Skipped guard does not catch this: the hook still
matches the file, Vale lints nothing, reports 0 errors in 1 file and exits 0,
which pre-commit renders as Passed. An assertion aggregating over N subjects
proves nothing about any individual subject.
Each hook now runs individually and its alerts are attributed to the nearest
preceding path header, so an alert is checked by path rather than by presence
in the combined blob. The two fixtures carry distinct VagueWording tokens, so
one hook's alert cannot be credited to another.
Nothing in the repo read either .vale.ini — the sync check diffed only
vale-wrap.sh and styles/Kyberforge, so a one-line glob typo silently disabled
the prefilter for a whole file type. That was the enabling half of the same
defect. The check now asserts the shared lines both copies must carry
(StylesPath, a section naming Kyberforge as a whole word) without flagging
their intentional divergence, and probes each glob section by asking Vale
itself to lint a representative path. Regex-to-glob comparison was rejected as
it means reimplementing doublestar semantics in bash; a file-count dry-run was
rejected because a section whose glob matches but whose BasedOnStyles lost
Kyberforge reports "1 file" with no alerts and would pass it.
Every new assertion is bound to a failing case in both directions: breaking the
artifact fails the suite, and neutering the assertion fails exactly one case.
That reverse sweep exposed two assertions bound to no failing case at all, one
masked by a stronger check running first.
Refs: #85
Three divergences between what the audit skills claim and what the hooks
enforce, each of which fails silently rather than loudly:
- `skill-size-check.sh` blocked at 2900 words while `validate.sh` checked only
the 500-line ceiling, so `/skill-audit` could report a skill ready to ship
that the commit hook then rejected. `validate.sh` now checks the same pair on
the same inclusive terms; the constants are duplicated with a comment naming
the other file, because a plugin skill's scripts cannot read outside the
plugin directory once installed to the cache.
- Both audit skills' Step 1 passed `--config assets/vale/.vale.ini`, which is
redundant (the wrapper self-locates its sibling config) and fragile: an agent
that resolves the script path against the skill directory but not the config
path gets E100, exit 2, which the surrounding fallback clause misreads as
"vale unavailable" and downgrades to full LLM judgment with no signal.
- The external-consumer test registered only the two Vale hooks, never the
third shipped hook, so a lost executable bit would have broken every consumer
while the local suite stayed green. Verified by mutation: `chmod 644` on the
copied script now turns three passes into two failures.
Also corrects the size hook's calibration comment, which claimed ~5.7-6.5
characters per word against a corpus whose measured median is 6.79 — the stated
upper bound sat below the median, so the "calibrated with margin" claim was
inverted for prose-dense files. MAX_WORDS is unchanged pending a decision; the
comment is now explicit that the gate holds under 5,000 tokens for typical
prose density, not for any file.
Refs: #85
pre-commit prefixes only entry[0] with the hook-repo clone path
(cmd = (prefix.path(cmd[0]), *cmd[1:])), so the --config argument in
.pre-commit-hooks.yaml resolved against the *consuming* repo's root
and hard-failed every external run with E100. Two of the three hooks
ADR-0014 promises were unusable.
vale-wrap.sh now self-locates its config from BASH_SOURCE when no
--config is supplied; an explicit --config still wins in all three
argv forms and stays cwd-relative. Both manifests drop the argument
and are kept byte-identical: the local repo: local config resolved
--config correctly only because the consuming repo *was* this repo,
and that divergence is why three review rounds missed the defect.
Also in the wrapper:
- replace GNU-only `realpath -m` with a portable abspath helper; -m is
load-bearing (dest does not exist yet), so BSD realpath aborted the
script under set -e on macOS
- walk directory arguments instead of passing them through unflattened,
which reported a clean 0-error run for files that fail when named
explicitly
- read/write with errors='surrogateescape' so one non-UTF-8 .md under a
directory argument cannot abort the hook
New test-vale-hooks-consumer.sh builds the hook repo from the working
tree and points a file:// consumer at it, covering the manifest as a
hook repo for the first time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
