test(lint): back the Vale 3.15.2 behaviour claims with a committed test
The `house-vale-3-15-2-repro` provenance entry claimed behaviours were reproduced against purpose-built fixtures, but no fixtures existed, so the earlier commit in this PR removed it. Commit the fixtures. tests/test-vale-3-15-2-behaviours.sh builds its fixtures in a temp dir and runs the real Vale. It exits 77 (skipped) when vale is missing or is not 3.15.2. It asserts the six vale-config behaviours and the vale-run ones (unmapped .mdx, `vale off` variants, the spelling ignore file, and the ls-* commands never naming a rule). Restore the entry in both sources.md files as `Research doc: none` with `Basis:` naming the test, and re-add its source_keys. Two behaviours are not asserted: the native-MDX suppression column (needs mdx2vast) and the `vale sync` row that adds to Packages (needs the network). The wording in configuration-reference.md and troubleshooting.md now says so. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EGHFJextYtVQseaHPDDhxB
This commit is contained in:
@@ -11,6 +11,7 @@ metadata:
|
||||
version: "0.1.4"
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
topic: configuration-reference
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Core Settings
|
||||
@@ -81,7 +82,7 @@ Only *package* styles need fetching. A style whose YAML rule files are already c
|
||||
- `Vale.Avoid` — enforces the project's rejected vocabulary terms.
|
||||
- `Vale.Repetition` — flags repeated words (e.g. "the the").
|
||||
|
||||
`Packages` (top-level, what `vale sync` downloads) and `BasedOnStyles` (per-glob, what activates) are separate keys: a style lints a file only once it is in both. Every row below reproduced against Vale 3.15.2:
|
||||
`Packages` (top-level, what `vale sync` downloads) and `BasedOnStyles` (per-glob, what activates) are separate keys: a style lints a file only once it is in both. Every row below is asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`) except the `vale sync` row that adds the name to `Packages`, which needs the network and is not covered:
|
||||
|
||||
| Configuration | Result |
|
||||
|---|---|
|
||||
@@ -97,7 +98,7 @@ Only *package* styles need fetching. A style whose YAML rule files are already c
|
||||
|
||||
## Frontmatter Scopes
|
||||
|
||||
House-verified behaviour, not documented on vale.sh — reproduced locally against Vale 3.15.2.
|
||||
House-verified behaviour, not documented on vale.sh — asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`).
|
||||
|
||||
A rule scoped to `text.frontmatter.<key>` (e.g. `text.frontmatter.description`) matches reliably when that field's value is a single physical line, and breaks on most — not all — multi-line forms. Multi-line forms spanning 2+ lines:
|
||||
|
||||
|
||||
@@ -7,3 +7,12 @@
|
||||
- **Research doc:** plugins/lint/docs/research/docs/vale/sources.md
|
||||
- **Contributing files:** SKILL.md, references/configuration-reference.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## house-vale-3-15-2-repro
|
||||
|
||||
- **URL:** (house-verified — reproduced against the `vale` binary by a committed test, not an external source)
|
||||
- **Description:** Behaviour of Vale 3.15.2 asserted by the committed test (purpose-built fixtures, real `vale` run), where vale.sh documents nothing: the `E100 [loadStyles]` / exit-2 failure for a `BasedOnStyles` name absent from `StylesPath`, `vale sync` reporting `Synced 0 package(s)` for a name not declared in `Packages`, the `E201` / exit-2 failure when the `StylesPath` directory does not exist, the exit-0 no-op of an empty style directory, the `E201` / exit-2 failure when a core option is written below a `[glob]` header (with `Packages` as the silent exception), the `E100 [lintMDX]` failure of an unmapped `.mdx` without `mdx2vast`, and the `text.frontmatter.<key>` scope matrix across multi-line YAML forms.
|
||||
- **Research doc:** none
|
||||
- **Basis:** tests/test-vale-3-15-2-behaviours.sh
|
||||
- **Contributing files:** SKILL.md, references/configuration-reference.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -10,6 +10,7 @@ metadata:
|
||||
category: lint
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -7,3 +7,12 @@
|
||||
- **Research doc:** plugins/lint/docs/research/docs/vale/sources.md
|
||||
- **Contributing files:** SKILL.md, references/troubleshooting.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## house-vale-3-15-2-repro
|
||||
|
||||
- **URL:** (house-verified — reproduced against the `vale` binary by a committed test, not an external source)
|
||||
- **Description:** Behaviour of Vale 3.15.2 asserted by the committed test (purpose-built fixtures, real `vale` run), where vale.sh documents nothing or documents it wrongly: an unmapped `.mdx` needs an external `mdx2vast` binary (absent, the whole invocation exits 2 with `E100 [lintMDX]`), under `[formats] mdx = md` the HTML-comment suppression form works and the JSX-comment form does not, the `spelling` check's `ignore` paths resolve against `StylesPath` or the working directory but never against the rule file's own directory and fail silently when they resolve nowhere, `ls-config` and the other `ls-*` subcommands report styles and paths but never rules, and the `text.frontmatter.<key>` scope matrix across multi-line YAML forms. Not asserted: the native-MDX column of the suppression table, which needs `mdx2vast` installed.
|
||||
- **Research doc:** none
|
||||
- **Basis:** tests/test-vale-3-15-2-behaviours.sh
|
||||
- **Contributing files:** SKILL.md, references/troubleshooting.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
# Vale troubleshooting reference
|
||||
@@ -50,8 +51,7 @@ suppression syntax:
|
||||
| `[formats]` maps `mdx = md` (what `vale-config` recommends) | none | Markdown | `<!-- vale off -->` |
|
||||
| no `mdx` mapping (native MDX) | `npm install -g mdx2vast` | MDX | `{/* vale off */}` |
|
||||
|
||||
Key the markup to that config row, never to the file extension. Verified against Vale 3.15.2, same
|
||||
three fixtures under each config:
|
||||
Key the markup to that config row, never to the file extension. Asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`) for the mapped column; the native-MDX column was observed with `mdx2vast` installed and is not covered by that test (it needs the binary):
|
||||
|
||||
| File | Mapped `mdx = md` | Native MDX (`mdx2vast` installed) |
|
||||
|---|---|---|
|
||||
@@ -119,7 +119,7 @@ ignore:
|
||||
**Where the file goes, and why a wrong answer is invisible.** Each entry resolves against the
|
||||
`StylesPath` root, or against the working directory `vale` is invoked from. It does **not** resolve
|
||||
against the rule file's own directory — which is the natural reading of the YAML above, since the
|
||||
path sits inside the rule, and it is wrong. Verified against Vale 3.15.2 across four fresh trees,
|
||||
path sits inside the rule, and it is wrong. Asserted against Vale 3.15.2 by the same test across four fresh trees,
|
||||
each with the same rule and the same unknown word:
|
||||
|
||||
| Where `ignore1.txt` was placed | Result |
|
||||
|
||||
220
tests/test-vale-3-15-2-behaviours.sh
Executable file
220
tests/test-vale-3-15-2-behaviours.sh
Executable file
@@ -0,0 +1,220 @@
|
||||
#!/usr/bin/env bash
|
||||
# Reproduction fixtures for the Vale 3.15.2 behaviours the lint plugin's vale-config and vale-run
|
||||
# skills state as house-verified (provenance key house-vale-3-15-2-repro). Each case builds a
|
||||
# purpose-built fixture in a temp dir, runs the real `vale` binary and asserts exit code plus
|
||||
# output. A behaviour that changes in a later Vale release fails here, which is the point: the
|
||||
# skill text is only backed while this test passes.
|
||||
#
|
||||
# Not reproducible here (mdx2vast is not installed in CI): the native-MDX halves of the mdx
|
||||
# suppression table. Only the mdx2vast-absent E100 and the `[formats] mdx = md` column are asserted.
|
||||
set -uo pipefail
|
||||
|
||||
if ! command -v vale &>/dev/null; then
|
||||
echo "SKIP: vale is not installed"
|
||||
exit 77
|
||||
fi
|
||||
EXPECTED="3.15.2"
|
||||
GOT="$(vale --version | awk '{print $3}')"
|
||||
if [[ "$GOT" != "$EXPECTED" ]]; then
|
||||
echo "SKIP: behaviours are pinned to Vale $EXPECTED, found $GOT"
|
||||
exit 77
|
||||
fi
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
||||
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
OUT="" RC=0
|
||||
# run <dir> <vale args...>: run vale from <dir>, capture combined output and exit code.
|
||||
run() {
|
||||
local dir="$1"; shift
|
||||
OUT="$(cd "$dir" && vale --no-wrap "$@" 2>&1)"; RC=$?
|
||||
OUT="$(sed $'s/\x1b\\[[0-9;]*m//g' <<<"$OUT")"
|
||||
}
|
||||
# expect <label> <rc> <grep-fixed-pattern-or-empty>
|
||||
expect() {
|
||||
local label="$1" rc="$2" pat="${3:-}"
|
||||
if [[ "$RC" -ne "$rc" ]]; then
|
||||
fail "$label: exit $RC, want $rc"; echo "$OUT" | sed 's/^/ /'; return
|
||||
fi
|
||||
if [[ -n "$pat" ]] && ! grep -qF -- "$pat" <<<"$OUT"; then
|
||||
fail "$label: output lacks '$pat'"; echo "$OUT" | sed 's/^/ /'; return
|
||||
fi
|
||||
pass "$label"
|
||||
}
|
||||
|
||||
# tree <name>: fresh dir with styles/ and a one-line doc containing a repeated word.
|
||||
tree() {
|
||||
local d="$WORK/$1"; mkdir -p "$d/styles"
|
||||
printf 'This is the the sample.\n' >"$d/doc.md"
|
||||
echo "$d"
|
||||
}
|
||||
rule() { # rule <dir> <style>: a one-rule custom style flagging the word "foo"
|
||||
mkdir -p "$1/styles/$2"
|
||||
cat >"$1/styles/$2/Foo.yml" <<'Y'
|
||||
extends: existence
|
||||
message: "found '%s'"
|
||||
level: error
|
||||
tokens:
|
||||
- foo
|
||||
Y
|
||||
}
|
||||
|
||||
echo "1. BasedOnStyles names a style absent from StylesPath"
|
||||
d="$(tree c1)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Nope\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E100 loadStyles, exit 2" 2 "E100 [loadStyles]"
|
||||
grep -qF "style 'Nope' does not exist on StylesPath" <<<"$OUT" && pass "message names the style" || fail "message names the style"
|
||||
|
||||
echo "2. vale sync for a name in BasedOnStyles but not Packages"
|
||||
d="$(tree c2)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Nope\n' >"$d/.vale.ini"
|
||||
run "$d" sync
|
||||
expect "Synced 0 package(s), exit 0" 0 "Synced 0 package(s)"
|
||||
[[ -z "$(ls -A "$d/styles")" ]] && pass "nothing downloaded" || fail "nothing downloaded"
|
||||
run "$d" doc.md
|
||||
expect "next lint repeats E100" 2 "E100 [loadStyles]"
|
||||
|
||||
echo "3. StylesPath directory absent, only built-in Vale active"
|
||||
d="$WORK/c3"; mkdir -p "$d"; printf 'x\n' >"$d/doc.md"
|
||||
printf 'StylesPath = nostyles\n[*.md]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E201, exit 2" 2 "E201 Invalid value"
|
||||
grep -q "does not exist" <<<"$OUT" && pass "path-does-not-exist message" || fail "path-does-not-exist message"
|
||||
|
||||
echo "4. Empty style directory loads and lints nothing"
|
||||
d="$(tree c4)"; mkdir "$d/styles/Empty"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Empty\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "0 findings, exit 0" 0 "0 errors, 0 warnings and 0 suggestions"
|
||||
|
||||
echo "4b. Built-in Vale and committed YAML lint with no Packages entry"
|
||||
d="$(tree c4b)"; rule "$d" Mine; printf 'a foo b\n' >>"$d/doc.md"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale, Mine\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "built-in Vale.Repetition fires, exit 1" 1 "Vale.Repetition"
|
||||
grep -qF "Mine.Foo" <<<"$OUT" && pass "committed style fires" || fail "committed style fires"
|
||||
|
||||
echo "4c. Style in Packages-only (not BasedOnStyles) lints nothing"
|
||||
d="$(tree c4c)"; rule "$d" Mine; printf 'a foo b\n' >>"$d/doc.md"
|
||||
printf 'StylesPath = styles\nPackages = Mine\n[*.md]\nBasedOnStyles =\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "0 findings, exit 0" 0 "0 errors, 0 warnings and 0 suggestions"
|
||||
|
||||
echo "5. Core option below a [glob] header"
|
||||
d="$(tree c5)"
|
||||
printf '[*.md]\nBasedOnStyles = Vale\nStylesPath = styles\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E201 core option, exit 2" 2 "is a core option"
|
||||
d="$(tree c5b)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale\nMinAlertLevel = error\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "MinAlertLevel below glob also E201" 2 "is a core option"
|
||||
d="$(tree c5c)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale\nPackages = Foo\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "Packages below glob: no error (exit 1 from the repetition finding)" 1 "Vale.Repetition"
|
||||
run "$d" ls-config
|
||||
grep -q '"Packages": false' <<<"$OUT" && pass "Packages parsed as per-glob rule toggle" || fail "Packages parsed as per-glob rule toggle"
|
||||
run "$d" sync
|
||||
expect "sync reports Synced 0 package(s)" 0 "Synced 0 package(s)"
|
||||
|
||||
echo "6. text.frontmatter.<key> scope across YAML forms"
|
||||
d="$WORK/c6"; mkdir -p "$d/styles/FM"
|
||||
cat >"$d/styles/FM/Foo.yml" <<'Y'
|
||||
extends: existence
|
||||
message: "found '%s'"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
tokens:
|
||||
- foo
|
||||
Y
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = FM\n' >"$d/.vale.ini"
|
||||
fm() { # fm <name> <frontmatter lines...>
|
||||
local n="$1"; shift
|
||||
{ echo '---'; printf '%s\n' "$@"; echo '---'; echo; echo 'Body.'; } >"$d/$n.md"
|
||||
}
|
||||
fm single 'description: has foo here'
|
||||
fm literal 'description: |' ' line one' ' has foo here'
|
||||
fm folded 'description: >' ' line one' ' has foo here'
|
||||
fm plain 'description: line one' ' has foo here'
|
||||
fm squote "description: 'line one" " has foo here'"
|
||||
fm dquote 'description: "line one' ' has foo here"'
|
||||
run "$d" single.md; expect "single line lints" 1 "Foo"
|
||||
run "$d" literal.md; expect "| literal lints" 1 "Foo"
|
||||
run "$d" folded.md; expect "> folded silent" 0 "0 errors"
|
||||
run "$d" plain.md; expect "plain continuation silent" 0 "0 errors"
|
||||
run "$d" squote.md; expect "single-quoted multi-line silent" 0 "0 errors"
|
||||
run "$d" dquote.md; expect "double-quoted multi-line silent" 0 "0 errors"
|
||||
|
||||
echo "7. .mdx without mapping and without mdx2vast"
|
||||
d="$WORK/c7"; mkdir -p "$d/styles"
|
||||
printf 'This is the the sample.\n' >"$d/doc.md"; cp "$d/doc.md" "$d/doc.mdx"
|
||||
printf 'StylesPath = styles\n[*.{md,mdx}]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
if command -v mdx2vast &>/dev/null; then
|
||||
echo " SKIP: mdx2vast is installed; the absent-binary case cannot run"
|
||||
else
|
||||
run "$d" .
|
||||
expect "whole invocation dies with E100 lintMDX, exit 2" 2 "E100 [lintMDX]"
|
||||
grep -qF "mdx2vast not found" <<<"$OUT" && pass "mdx2vast not found" || fail "mdx2vast not found"
|
||||
grep -q "doc.md" <<<"$OUT" && fail ".md alongside produced no output" || pass ".md alongside produced no output"
|
||||
fi
|
||||
|
||||
echo "8. mdx mapped onto md: suppression form"
|
||||
d="$WORK/c8"; mkdir -p "$d/styles"
|
||||
printf 'StylesPath = styles\n[formats]\nmdx = md\n[*.{md,mdx}]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
printf 'This is the the sample.\n' >"$d/ctl.mdx"
|
||||
printf '<!-- vale off -->\nThis is the the sample.\n<!-- vale on -->\n' >"$d/html.mdx"
|
||||
printf '{/* vale off */}\nThis is the the sample.\n{/* vale on */}\n' >"$d/jsx.mdx"
|
||||
run "$d" ctl.mdx; expect "control alerts" 1 "Vale.Repetition"
|
||||
run "$d" html.mdx; expect "HTML comment suppresses" 0 "0 errors"
|
||||
run "$d" jsx.mdx; expect "JSX comment does not suppress" 1 "Vale.Repetition"
|
||||
|
||||
echo "9. spelling ignore path resolution"
|
||||
mk_spell() { # mk_spell <name> ; leaves rule with ignore1.txt, no ignore file placed
|
||||
local d; d="$WORK/$1"; mkdir -p "$d/styles/MyStyle" "$d/proj"
|
||||
cat >"$d/styles/MyStyle/Spell.yml" <<'Y'
|
||||
extends: spelling
|
||||
message: "Did you really mean '%s'?"
|
||||
level: error
|
||||
ignore:
|
||||
- ignore1.txt
|
||||
Y
|
||||
printf 'The zzqwidget is here.\n' >"$d/proj/doc.md"
|
||||
printf 'StylesPath = ../styles\n[*.md]\nBasedOnStyles = MyStyle\n' >"$d/proj/.vale.ini"
|
||||
echo "$d"
|
||||
}
|
||||
d="$(mk_spell s1)"; printf 'zzqwidget\n' >"$d/styles/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file at StylesPath root works" 0 "0 errors"
|
||||
d="$(mk_spell s2)"; printf 'zzqwidget\n' >"$d/proj/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file in working directory works" 0 "0 errors"
|
||||
run "$d" --config=proj/.vale.ini proj/doc.md; expect "working-dir copy fails from another directory" 1 "zzqwidget"
|
||||
d="$(mk_spell s3)"; printf 'zzqwidget\n' >"$d/styles/MyStyle/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file beside the rule is not read" 1 "zzqwidget"
|
||||
d="$(mk_spell s4)"
|
||||
run "$d/proj" doc.md; expect "absent ignore file fails silently" 1 "zzqwidget"
|
||||
grep -qi "ignore1" <<<"$OUT" && fail "no diagnostic emitted for missing ignore file" || pass "no diagnostic emitted for missing ignore file"
|
||||
|
||||
echo "10. ls-config reports styles and paths, never rules"
|
||||
d="$(tree c10)"; rule "$d" MyStyle
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = MyStyle\n' >"$d/.vale.ini"
|
||||
printf 'a foo b\n' >"$d/doc.md"
|
||||
run "$d" doc.md; expect "rule fires" 1 "MyStyle.Foo"
|
||||
run "$d" ls-config
|
||||
grep -qF '"MyStyle"' <<<"$OUT" && pass "ls-config names the style" || fail "ls-config names the style"
|
||||
grep -qF 'Foo' <<<"$OUT" && fail "ls-config must not name the rule" || pass "ls-config does not name the rule"
|
||||
for sub in ls-dirs ls-vars ls-metrics; do
|
||||
run "$d" "$sub"
|
||||
grep -qF 'Foo' <<<"$OUT" && fail "$sub must not name the rule" || pass "$sub does not name the rule"
|
||||
done
|
||||
run "$d" ls-config
|
||||
grep -qF '"Checks": null' <<<"$OUT" && pass "Checks: null" || fail "Checks: null"
|
||||
|
||||
echo
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ "$FAIL" -eq 0 ]]
|
||||
Reference in New Issue
Block a user