fix(kyberforge): align the audit word ceiling and drop the fragile --config

Three divergences between what the audit skills claim and what the hooks
enforce, each of which fails silently rather than loudly:

- `skill-size-check.sh` blocked at 2900 words while `validate.sh` checked only
  the 500-line ceiling, so `/skill-audit` could report a skill ready to ship
  that the commit hook then rejected. `validate.sh` now checks the same pair on
  the same inclusive terms; the constants are duplicated with a comment naming
  the other file, because a plugin skill's scripts cannot read outside the
  plugin directory once installed to the cache.
- Both audit skills' Step 1 passed `--config assets/vale/.vale.ini`, which is
  redundant (the wrapper self-locates its sibling config) and fragile: an agent
  that resolves the script path against the skill directory but not the config
  path gets E100, exit 2, which the surrounding fallback clause misreads as
  "vale unavailable" and downgrades to full LLM judgment with no signal.
- The external-consumer test registered only the two Vale hooks, never the
  third shipped hook, so a lost executable bit would have broken every consumer
  while the local suite stayed green. Verified by mutation: `chmod 644` on the
  copied script now turns three passes into two failures.

Also corrects the size hook's calibration comment, which claimed ~5.7-6.5
characters per word against a corpus whose measured median is 6.79 — the stated
upper bound sat below the median, so the "calibrated with margin" claim was
inverted for prose-dense files. MAX_WORDS is unchanged pending a decision; the
comment is now explicit that the gate holds under 5,000 tokens for typical
prose density, not for any file.

Refs: #85
This commit is contained in:
2026-08-09 15:44:16 +00:00
parent aa8cc22695
commit 4ae2429840
6 changed files with 85 additions and 22 deletions

View File

@@ -15,12 +15,23 @@ set -euo pipefail
# audit and still be blocked by the commit hook.
#
# Token counts aren't computed exactly here — word count (`wc -w`) is used as
# a proxy. This repo's own SKILL.md corpus measures ~5.7-6.5 characters per
# word, which at the standard ~4-characters-per-token English approximation
# works out to roughly 1.6-1.7 tokens per word. MAX_WORDS below is calibrated
# from that measured ratio against the 5,000-token ceiling, with margin — it's
# still a proxy, not exact BPE tokenization, but now grounded in actual repo
# content rather than an unverified "conservative" assumption.
# a proxy. Measured over this repo's 39 in-scope SKILL.md files, characters per
# word runs min 5.97 / median 6.79 / mean 6.77 / max 7.22. At the standard
# ~4-characters-per-token English approximation that is 1.49 / 1.70 / 1.69 /
# 1.81 tokens per word.
#
# MAX_WORDS=2900 is therefore calibrated to the corpus MEDIAN, not to its worst
# case, and carries no margin: 2900 x 1.70 = ~4,900 tokens for a median-density
# file, but 2900 x 1.81 = ~5,240 tokens for the densest file in the corpus. So
# what this gate actually guarantees is "under 5,000 tokens for a file of
# typical prose density"; a prose-dense SKILL.md can sit at exactly MAX_WORDS
# and still be a few hundred tokens over the agentskills.io ceiling. Holding
# the worst observed ratio under 5,000 tokens would need MAX_WORDS ~2770.
#
# It is a one-sided proxy in the useful direction — nothing under the word
# ceiling is wildly over the token ceiling — but it is not exact BPE
# tokenization and does not replace one. Re-measure the corpus before treating
# any of these numbers as still current.
MAX_LINES=500
MAX_WORDS=2900