fix(lint): tighten the SKILL.md word ceiling to 2770

MAX_WORDS=2900 was calibrated to the corpus median density and carried no
margin: at the densest observed 7.22 chars/word (~1.81 tokens/word) it permits
~5,240 tokens against the 5,000 it proxies for. 2770 holds the worst observed
density under the ceiling. The largest SKILL.md is 2,489 words, so the change
costs nothing today — 281 words of margin — and the header comment now argues
the new calibration rather than swapping the digits.

Both enforcement points move together, and a new test asserts they agree, since
a SKILL.md passing its own audit while the commit hook blocks it is the
disagreement this pair exists to prevent.

CONTEXT.md is deliberately left ungated: it is 2,816 words, and gating it would
block the build. Recorded here so the omission reads as a decision rather than
an oversight.

skill-audit's manual-fallback path listed only the line ceiling, so an agent
taking that path passed an oversized SKILL.md the hook then rejected. The word
ceiling is now named alongside it. agent-audit is deliberately unchanged: the
size hook scopes to SKILL.md only and agent-audit's validate.sh has no word
gate, so claiming it there would be false.

The Vale research doc still showed the MDX {/* vale off */} form under a
Markdown heading, contradicting CONTEXT.md and vale-run's troubleshooting
reference — that form suppresses nothing in plain .md. Fixed in both places it
appeared.

tests/run-tests.sh used mapfile (bash 4.0+) with unguarded array expansion,
though AGENTS.md tells contributors to run it and macOS ships bash 3.2. It now
collects via a while-read loop over process substitution and guards every
expansion. The newline-delimited find|sort pipeline is kept rather than -print0
with sort -z, whose BSD portability is the weaker link, and which matches
mapfile -t's previous behaviour exactly.

Refs: #85
ADR: 0013
This commit is contained in:
2026-08-09 17:23:51 +00:00
parent f6eb0d295e
commit 302f6d0c19
8 changed files with 80 additions and 23 deletions

View File

@@ -1,12 +1,14 @@
#!/usr/bin/env bash
# Regression test for scripts/skill-size-check.sh: enforces agentskills.io's
# 500-line/5,000-token SKILL.md size ceiling. The token half is enforced via a
# word-count proxy (MAX_WORDS, currently 2900) — 5,000 is the token ceiling,
# 2,900 is the word budget the script derives from it.
# word-count proxy (MAX_WORDS, currently 2770) — 5,000 is the token ceiling,
# 2,770 is the word budget the script derives from it at the corpus's densest
# measured prose.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$REPO_ROOT/scripts/skill-size-check.sh"
VALIDATE="$REPO_ROOT/plugins/kyberforge/skills/skill-audit/scripts/validate.sh"
PASS=0
FAIL=0
@@ -67,6 +69,29 @@ fi
MAX_WORDS="$(grep -oE '^MAX_WORDS=[0-9]+' "$SCRIPT" | cut -d= -f2)"
MAX_LINES="$(grep -oE '^MAX_LINES=[0-9]+' "$SCRIPT" | cut -d= -f2)"
# The audit (skill-audit/scripts/validate.sh) duplicates both ceilings, because
# a cache-installed plugin's scripts cannot read files outside the plugin
# directory. Nothing but this assertion stops the copies drifting, and drift
# means a SKILL.md passes its own audit and is then rejected by the commit hook.
echo ""
echo "--- the hook and skill-audit's validate.sh agree on both ceilings ---"
if [[ ! -f "$VALIDATE" ]]; then
fail "skill-audit validate.sh not found at $VALIDATE"
else
V_MAX_WORDS="$(grep -oE '^MAX_WORDS = [0-9]+' "$VALIDATE" | grep -oE '[0-9]+')"
V_MAX_LINES="$(grep -oE '^MAX_LINES = [0-9]+' "$VALIDATE" | grep -oE '[0-9]+')"
if [[ "$V_MAX_WORDS" == "$MAX_WORDS" ]]; then
pass "both enforce MAX_WORDS=$MAX_WORDS"
else
fail "MAX_WORDS drift: hook says $MAX_WORDS, validate.sh says ${V_MAX_WORDS:-<unset>}"
fi
if [[ "$V_MAX_LINES" == "$MAX_LINES" ]]; then
pass "both enforce MAX_LINES=$MAX_LINES"
else
fail "MAX_LINES drift: hook says $MAX_LINES, validate.sh says ${V_MAX_LINES:-<unset>}"
fi
fi
# make_line_fixture builds a file with an exact total line count (frontmatter
# included), independent of word count, for the line-boundary tests.
make_line_fixture() {