fix(lint): resolve round-1 and round-2 review findings on the Vale prefilter
Addresses PR #85's outstanding review items after grilling the open
questions against ADR-0013/CONTEXT.md/ADR-0010:
Blocking fixes:
- vale-wrap.sh: replace json.dumps() escaping (which silently defeated
Vale's frontmatter scope on any description containing a quote,
backslash, or non-ASCII char — ~58% of the corpus) with a single-quoted
YAML scalar, substituting a Unicode right single quote for embedded
apostrophes rather than '' doubling (Vale's frontmatter scanner isn't a
full YAML parser and silently truncates on '' too).
- vale-wrap.sh: fix a blank-line-inside-a-folded-description truncation
bug via indentation-based, blank-line-tolerant body capture; narrow
flattening to `>`-style scalars only (`|` already works unflattened).
- skill-audit/agent-audit Step 1: make the vale-wrap.sh invocation
cwd-independent via git rev-parse --show-toplevel, fixing a bug where
no single cwd satisfied all three Step 1 commands.
- styles/Kyberforge/VagueQualifier.yml: prune 17 tokens verified
false-positive-dominated on this repo's own voice via a real corpus
sweep (obvious, clearly, usually, several, simple, easy, completely,
simply, tiny, etc.), keep 13 with real or unattested noise. Revert the
28 prose "fixes" those tokens drove across 14 skill files back to their
original, correct wording, including a functional regression to
caveman/SKILL.md's own filler-word list (a mention, not a use) — now
guarded with vale-off comments against recurrence.
Gaps:
- --minAlertLevel=warning on the pre-commit hook and Step 1 invocation
so warning-level rules actually surface, without collapsing the
FAIL/SUGGESTION severity mapping skill-audit/agent-audit rely on.
- vale-wrap.sh: fix --config=<path> equals-form, absolute-path silent
no-op, and a zero-file-argument stdin hang.
- Route vale-run and lint-runner through a documented wrapper script
when a target repo has one, instead of unconditionally recommending
bare `vale`.
- Wire Kyberforge.VagueQualifier/SentenceOpenerThereIs into skill-audit/
agent-audit's dimension-mapping prose (Body discipline).
- Add plugins/lint/sources.md provenance for lint-runner (ADR-0010).
- Sync both marketplace.json lint-entry descriptions with plugin.json.
- Retune skill-size-check.sh's MAX_WORDS 5000->2900 (measured ~1.6-1.7
tokens/word on this repo's corpus, the old value gated at ~8,500
tokens against a stated 5,000 ceiling); fix the >/>= line-count
boundary and wc -l undercount on files with no trailing newline.
- Document the vale binary as a Setup prerequisite in AGENTS.md.
- Fix SentenceOpenerThereIs's dead regex alternative and add a real
sentence-start anchor/scope.
- Fix a stale docs/research/docs/vale/ index pointer in kyberforge's
docs README (moved to plugins/lint/ in e1a5403).
- Rewrite ADR-0013's Consequences section past-tense to describe what
actually landed, and record the styles-portability limitation
(repo-root placement stays intentional; deferred to a separate
session per this PR's review).
Test coverage: 9 new vale-wrap.sh fixtures (quotes, backslash/unicode,
blank-line paragraphs, --config= form, zero-arg/absolute-path handling,
literal-block no-regression) and boundary-pair tests for
skill-size-check.sh's line/word ceilings.
bash tests/run-tests.sh: 9 scripts + 125 bats assertions, all passing.
scripts/check-manifests.sh and claude plugin validate --strict: clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
This commit is contained in:
@@ -60,6 +60,102 @@ else
|
||||
pass "file over the 5,000-word ceiling exits non-zero"
|
||||
fi
|
||||
|
||||
# Boundary-pair tests below read the script's current MAX_WORDS rather than
|
||||
# hardcoding it, so they don't silently drift if the threshold changes again.
|
||||
MAX_WORDS="$(grep -oE '^MAX_WORDS=[0-9]+' "$SCRIPT" | cut -d= -f2)"
|
||||
MAX_LINES="$(grep -oE '^MAX_LINES=[0-9]+' "$SCRIPT" | cut -d= -f2)"
|
||||
|
||||
# make_line_fixture builds a file with an exact total line count (frontmatter
|
||||
# included), independent of word count, for the line-boundary tests.
|
||||
make_line_fixture() {
|
||||
local name="$1" total_lines="$2" file body_lines
|
||||
file="$TMPDIR/$name.md"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: $name"
|
||||
echo "description: Test fixture."
|
||||
echo "---"
|
||||
} > "$file"
|
||||
body_lines=$((total_lines - 4))
|
||||
for ((i = 1; i <= body_lines; i++)); do
|
||||
echo "word"
|
||||
done >> "$file"
|
||||
echo "$file"
|
||||
}
|
||||
|
||||
# The line ceiling is exclusive of the limit itself ("stay under $MAX_LINES
|
||||
# lines", per skill-authoring.md), enforced via `>=` — so $((MAX_LINES - 1))
|
||||
# must pass and $MAX_LINES itself must already fail.
|
||||
echo ""
|
||||
echo "--- passes a file at $((MAX_LINES - 1)) lines, just under the $MAX_LINES-line boundary ---"
|
||||
AT_LINES="$(make_line_fixture at-line-limit "$((MAX_LINES - 1))")"
|
||||
ACTUAL_LINES=$(awk 'END{print NR}' "$AT_LINES")
|
||||
if [[ "$ACTUAL_LINES" -ne "$((MAX_LINES - 1))" ]]; then
|
||||
fail "fixture has $ACTUAL_LINES lines, expected exactly $((MAX_LINES - 1))"
|
||||
elif "$SCRIPT" "$AT_LINES"; then
|
||||
pass "file at $((MAX_LINES - 1)) lines exits 0"
|
||||
else
|
||||
fail "file at $((MAX_LINES - 1)) lines should have exited 0"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--- fails a file at exactly the $MAX_LINES-line boundary ---"
|
||||
OVER_LINES="$(make_line_fixture over-line-limit "$MAX_LINES")"
|
||||
ACTUAL_OVER_LINES=$(awk 'END{print NR}' "$OVER_LINES")
|
||||
if [[ "$ACTUAL_OVER_LINES" -ne "$MAX_LINES" ]]; then
|
||||
fail "fixture has $ACTUAL_OVER_LINES lines, expected exactly $MAX_LINES"
|
||||
elif "$SCRIPT" "$OVER_LINES" 2>/dev/null; then
|
||||
fail "file at exactly $MAX_LINES lines should have exited non-zero (>= ceiling, the boundary bug this test guards against)"
|
||||
else
|
||||
pass "file at exactly $MAX_LINES lines exits non-zero"
|
||||
fi
|
||||
|
||||
# make_word_fixture builds a file with an exact total word count (frontmatter
|
||||
# words included, since the script's `wc -w` counts the whole file) by padding
|
||||
# a body line with just enough "word" tokens to close the gap to the target.
|
||||
make_word_fixture() {
|
||||
local name="$1" target="$2" file cur remaining body
|
||||
file="$TMPDIR/$name.md"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: $name"
|
||||
echo "description: Test fixture."
|
||||
echo "---"
|
||||
} > "$file"
|
||||
cur=$(wc -w < "$file")
|
||||
remaining=$((target - cur))
|
||||
body=""
|
||||
for ((i = 1; i <= remaining; i++)); do
|
||||
body="$body word"
|
||||
done
|
||||
echo "$body" >> "$file"
|
||||
echo "$file"
|
||||
}
|
||||
|
||||
echo ""
|
||||
echo "--- passes a file at exactly the $MAX_WORDS-word boundary ---"
|
||||
AT_WORDS="$(make_word_fixture at-word-limit "$MAX_WORDS")"
|
||||
ACTUAL_WORDS=$(wc -w < "$AT_WORDS")
|
||||
if [[ "$ACTUAL_WORDS" -ne "$MAX_WORDS" ]]; then
|
||||
fail "fixture has $ACTUAL_WORDS words, expected exactly $MAX_WORDS"
|
||||
elif "$SCRIPT" "$AT_WORDS"; then
|
||||
pass "file at exactly $MAX_WORDS words exits 0"
|
||||
else
|
||||
fail "file at exactly $MAX_WORDS words should have exited 0"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--- fails a file one word over the $MAX_WORDS-word boundary ---"
|
||||
OVER_WORDS="$(make_word_fixture over-word-limit "$((MAX_WORDS + 1))")"
|
||||
ACTUAL_OVER_WORDS=$(wc -w < "$OVER_WORDS")
|
||||
if [[ "$ACTUAL_OVER_WORDS" -ne "$((MAX_WORDS + 1))" ]]; then
|
||||
fail "fixture has $ACTUAL_OVER_WORDS words, expected exactly $((MAX_WORDS + 1))"
|
||||
elif "$SCRIPT" "$OVER_WORDS" 2>/dev/null; then
|
||||
fail "file at $((MAX_WORDS + 1)) words should have exited non-zero"
|
||||
else
|
||||
pass "file at $((MAX_WORDS + 1)) words exits non-zero"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
@@ -78,6 +78,210 @@ else
|
||||
fail "reported line number ($MAX_LINE) exceeds original file line count ($ORIG_LINES)"
|
||||
fi
|
||||
|
||||
# make_raw_fixture writes stdin verbatim to a fresh fixture's SKILL.md, for
|
||||
# cases where the exact description body needs to be hand-crafted rather than
|
||||
# generated from the desc_lines loop above.
|
||||
make_raw_fixture() {
|
||||
local dir
|
||||
dir="$(mktemp -d)"
|
||||
(cd "$dir" && git init -q)
|
||||
mkdir -p "$dir/plugins/testplugin/skills/zzzskill"
|
||||
cat > "$dir/plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
echo "$dir"
|
||||
}
|
||||
|
||||
# --- 4. A folded description containing a double quote is still caught ---
|
||||
# This is the exact case that silently passed (zero alerts) before switching
|
||||
# from json.dumps (double-quoted, backslash-escaped) to a single-quoted scalar.
|
||||
echo ""
|
||||
echo "--- catches vague wording when the folded description contains a double quote ---"
|
||||
FIXTURE4="$(make_raw_fixture <<'EOF'
|
||||
---
|
||||
name: zzzskill
|
||||
description: >
|
||||
Use when the user says "audit this skill" and helps with and utilize things.
|
||||
Second line continues the same folded scalar for flattening.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4"' EXIT
|
||||
if (cd "$FIXTURE4" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md) | grep -q "VagueWording"; then
|
||||
pass "flags vague wording when the description contains a double quote"
|
||||
else
|
||||
fail "silently missed vague wording in a description containing a double quote — the bug this test guards against"
|
||||
fi
|
||||
|
||||
# --- 5. A folded description containing an apostrophe fires and stays valid YAML ---
|
||||
echo ""
|
||||
echo "--- catches vague wording when the folded description contains an apostrophe ---"
|
||||
FIXTURE5="$(make_raw_fixture <<'EOF'
|
||||
---
|
||||
name: zzzskill
|
||||
description: >
|
||||
Use when the user's task helps with and utilize things across two lines.
|
||||
Second continuation line for the fold.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5"' EXIT
|
||||
OUT5=$(cd "$FIXTURE5" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md 2>&1)
|
||||
if echo "$OUT5" | grep -q "VagueWording"; then
|
||||
pass "flags vague wording when the description contains an apostrophe"
|
||||
else
|
||||
fail "silently missed vague wording in a description containing an apostrophe"
|
||||
fi
|
||||
if echo "$OUT5" | grep -qi "yaml:"; then
|
||||
fail "flattened copy with an apostrophe produced a YAML parse error"
|
||||
else
|
||||
pass "flattened copy with an apostrophe is valid YAML (no parse error)"
|
||||
fi
|
||||
|
||||
# --- 6. A folded description with a backslash and a non-ASCII character ---
|
||||
echo ""
|
||||
echo "--- catches vague wording when the folded description has a backslash and non-ASCII text ---"
|
||||
FIXTURE6="$(make_raw_fixture <<'EOF'
|
||||
---
|
||||
name: zzzskill
|
||||
description: >
|
||||
Use when the café résumé naïve thing helps with and utilize things here.
|
||||
Path is C:\Users\test and this is the second continuation line.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6"' EXIT
|
||||
if (cd "$FIXTURE6" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md) | grep -q "VagueWording"; then
|
||||
pass "flags vague wording when the description has a backslash and non-ASCII text"
|
||||
else
|
||||
fail "silently missed vague wording in a description with a backslash and non-ASCII text"
|
||||
fi
|
||||
|
||||
# --- 7. A folded description with a blank line between two paragraphs ---
|
||||
echo ""
|
||||
echo "--- handles a blank line inside a folded description without crashing ---"
|
||||
FIXTURE7="$(make_raw_fixture <<'EOF'
|
||||
---
|
||||
name: zzzskill
|
||||
description: >
|
||||
Use when the user needs a general helper.
|
||||
|
||||
Do not use when this helps with and utilize things instead.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7"' EXIT
|
||||
OUT7=$(cd "$FIXTURE7" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" \
|
||||
plugins/testplugin/skills/zzzskill/SKILL.md 2>&1)
|
||||
if echo "$OUT7" | grep -q "Traceback"; then
|
||||
fail "crashed while flattening a description with a blank line between paragraphs"
|
||||
elif echo "$OUT7" | grep -q "VagueWording"; then
|
||||
pass "still flags vague wording in the second paragraph after a blank line"
|
||||
else
|
||||
fail "silently missed vague wording in the second paragraph after a blank line — the bug this test guards against"
|
||||
fi
|
||||
|
||||
# --- 8. --config=<path> (equals form) resolves the same as the two-argv form ---
|
||||
echo ""
|
||||
echo "--- --config=<path> equals form resolves from a subdirectory like the two-argv form ---"
|
||||
FIXTURE8="$(mktemp -d)"
|
||||
(cd "$FIXTURE8" && git init -q)
|
||||
cp "$REPO_ROOT/.vale.ini" "$FIXTURE8/.vale.ini"
|
||||
cp -r "$REPO_ROOT/styles" "$FIXTURE8/styles"
|
||||
mkdir -p "$FIXTURE8/plugins/testplugin/skills/zzzskill"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: zzzskill"
|
||||
echo "description: >"
|
||||
echo " Line one mentions helps with and utilize, plus a colon: like this."
|
||||
echo " Line two continues the same folded scalar for flattening."
|
||||
echo "---"
|
||||
echo ""
|
||||
echo "Body."
|
||||
} > "$FIXTURE8/plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7" "$FIXTURE8"' EXIT
|
||||
SUBDIR8="$FIXTURE8/plugins/testplugin/skills/zzzskill"
|
||||
REL8="plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
OUT_EQ=$(cd "$SUBDIR8" && bash "$SCRIPT" --config=.vale.ini "$REL8" 2>&1)
|
||||
OUT_TWO=$(cd "$SUBDIR8" && bash "$SCRIPT" --config .vale.ini "$REL8" 2>&1)
|
||||
if echo "$OUT_EQ" | grep -q "VagueWording" && [[ "$OUT_EQ" == "$OUT_TWO" ]]; then
|
||||
pass "--config=<path> from a subdirectory resolves and matches the two-argv form"
|
||||
else
|
||||
fail "--config=<path> equals form did not resolve the same as the two-argv form"
|
||||
fi
|
||||
|
||||
# --- 9. Zero file args (or a file list that filters to nothing) exits promptly ---
|
||||
echo ""
|
||||
echo "--- exits promptly instead of hanging on stdin when no files are passed ---"
|
||||
if timeout 5 bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" < <(sleep 100) >/dev/null 2>&1; then
|
||||
pass "exits promptly with zero file args"
|
||||
else
|
||||
RC=$?
|
||||
if [[ $RC -eq 124 ]]; then
|
||||
fail "hung waiting on stdin with zero file args — the bug this test guards against"
|
||||
else
|
||||
pass "exits promptly (nonzero exit) with zero file args"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--- exits promptly when a file list filters down to nothing ---"
|
||||
if timeout 5 bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" --no-such-flag < <(sleep 100) >/dev/null 2>&1; then
|
||||
pass "exits promptly when no file-shaped args remain"
|
||||
else
|
||||
RC=$?
|
||||
if [[ $RC -eq 124 ]]; then
|
||||
fail "hung waiting on stdin when the file list filtered to nothing — the bug this test guards against"
|
||||
else
|
||||
pass "exits promptly (nonzero exit) when the file list filters to nothing"
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 10. An absolute path to the fixture SKILL.md is still linted, not skipped ---
|
||||
echo ""
|
||||
echo "--- lints an absolute path to a skill file instead of silently skipping it ---"
|
||||
FIXTURE10="$(make_fixture 2)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7" "$FIXTURE8" "$FIXTURE10"' EXIT
|
||||
ABS_FILE10="$FIXTURE10/plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
if (cd "$FIXTURE10" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" "$ABS_FILE10") | grep -q "VagueWording"; then
|
||||
pass "an absolute path under repo_root is linted, not silently skipped"
|
||||
else
|
||||
fail "an absolute path under repo_root was silently skipped — the bug this test guards against"
|
||||
fi
|
||||
|
||||
# --- 11. A literal (|) block scalar passes through unflattened (no regression) ---
|
||||
echo ""
|
||||
echo "--- leaves a literal (|) block scalar untouched (narrowed >-only scope) ---"
|
||||
FIXTURE11="$(make_raw_fixture <<'EOF'
|
||||
---
|
||||
name: zzzskill
|
||||
description: |
|
||||
Line one mentions helps with and utilize things here.
|
||||
Line two continues the literal block scalar for this test.
|
||||
---
|
||||
|
||||
Body.
|
||||
EOF
|
||||
)"
|
||||
trap 'rm -rf "$FIXTURE1" "$FIXTURE2" "$FIXTURE3" "$FIXTURE4" "$FIXTURE5" "$FIXTURE6" "$FIXTURE7" "$FIXTURE8" "$FIXTURE10" "$FIXTURE11"' EXIT
|
||||
REL11="plugins/testplugin/skills/zzzskill/SKILL.md"
|
||||
WRAPPED_OUT=$(cd "$FIXTURE11" && bash "$SCRIPT" --config "$REPO_ROOT/.vale.ini" "$REL11" 2>&1 || true)
|
||||
BARE_OUT=$(cd "$FIXTURE11" && vale --config "$REPO_ROOT/.vale.ini" "$REL11" 2>&1 || true)
|
||||
if [[ "$WRAPPED_OUT" == "$BARE_OUT" ]]; then
|
||||
pass "literal (|) block scalar output matches bare vale exactly — untouched by flattening"
|
||||
else
|
||||
fail "wrapper altered output for a literal (|) block scalar description — should be left untouched"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
Reference in New Issue
Block a user