Compare commits
4 Commits
2bde9a6a82
...
45d8f19e56
| Author | SHA1 | Date | |
|---|---|---|---|
| 45d8f19e56 | |||
| 58a3f402a6 | |||
| c008da1876 | |||
| 2c4b6d2615 |
@@ -86,7 +86,7 @@ A plugin's research `sources.md` (e.g. `plugins/git/docs/research/docs/git/sourc
|
||||
headings are the source slugs. A skill's `Research doc:` field names exactly one, and
|
||||
`factory-audit` resolves each entry's slug against it. An entry with no registry declares
|
||||
`Research doc: none` and names what it was actually drawn from in `Basis:`.
|
||||
_Avoid_: research doc, sources file, topic doc (a topic doc is a digest of sources, not the registry)
|
||||
_Avoid_: bare "research doc" (the noun; `Research doc:` is the field name), sources file, topic doc (a topic doc is a digest of sources, not the registry)
|
||||
|
||||
### Governance
|
||||
|
||||
|
||||
@@ -5,6 +5,9 @@ merged into `factory-audit`, which dispatches to a skill flow and an agent flow
|
||||
`skill-audit` below as `factory-audit`'s skill flow. The decision itself is unchanged — ADR-0025
|
||||
carried every audit criterion, tier and finding level across as-is.
|
||||
|
||||
**Amended by ADR-0028 (2026-09-21).** INFO stays for a check that cannot run. A check that ran and
|
||||
found a mismatch in `Research doc:` is now a FAIL, so INFO no longer covers it.
|
||||
|
||||
`skill-audit` shipped with two finding levels: FAIL (blocks shipping) and
|
||||
SUGGESTION (optional improvement). Provenance validation introduced observations
|
||||
that are worth surfacing but not actionable: a `references/*.md` file with no
|
||||
|
||||
@@ -9,8 +9,8 @@ whose `## H2` headings are the source slugs. The corpus did something else: 29 o
|
||||
entries pointed at a research topic doc annotated `(whole-document reference)`, and 6 values were not
|
||||
a single path (comma-separated lists and shell brace expansion, plus a semicolon pair in
|
||||
`gitea-releases`). Checks 7 and 8 of `validate-provenance.sh` look the slug up as an H2 in the named
|
||||
file, so 36 entries reported INFO and nothing failed. Measured by running the script over all 39 skill
|
||||
directories, since nothing else runs it over the corpus.
|
||||
file, so 36 entries reported INFO and nothing failed. Measured by running the script over all 38 skill
|
||||
directories (27 with a `references/sources.md`, 11 without), since nothing else runs it over the corpus.
|
||||
|
||||
We decided that `Research doc:` names exactly one **Research registry** (the term is in
|
||||
`CONTEXT.md`), as the spec always said. Slug-to-H2 lookup in the registry is the only provenance link
|
||||
@@ -56,6 +56,17 @@ pairs are rejected outright, since nothing expands them in a markdown field.
|
||||
- **(c) Everything FAIL (rejected).** Fails a correctly-provenanced skill audited from a deployed
|
||||
copy, which the file-structure exemption exists to prevent.
|
||||
|
||||
**Q4 — enforcement.** A corpus-wide sweep gate lands in the same change: a test or pre-push hook that
|
||||
runs `validate-provenance.sh` over every `plugins/*/.apm/skills/*/` and fails on any FAIL. Deferring it
|
||||
was rejected because without a caller the FAIL tier is inert; nothing but `check-scope-walkup-sync.sh`
|
||||
(on fixtures) invokes the validator today.
|
||||
|
||||
**Q5 — parser parity.** `parse_research_doc` accepts the bullet spelling (`- **Research doc:**`) as
|
||||
`parse_contributing_files` already does, with a regression test. `parse_status` was removed from the
|
||||
validator during this change, so it gets no test. Included because it is the same failure shape as
|
||||
#111 and #118 (a parser returns "nothing found", the caller reads it as "nothing declared"), sits in
|
||||
the same file, and `gitea-releases` already writes the unhyphenated form.
|
||||
|
||||
**Q6 — what happens to check 8.** Found unsatisfiable during the migration, after Q3 was decided.
|
||||
Check 8 requires every `extracted` slug in the research doc to appear in the skill's `sources.md`.
|
||||
That worked while entries pointed at topic docs, and was dormant. Under Q1(a) the named file is a
|
||||
@@ -77,8 +88,10 @@ check on every `Basis:` bullet would fail them.
|
||||
|
||||
- **(a) A bullet annotated `(removed in <sha>)` skips the existence check (chosen).** The check stays
|
||||
for live paths, which is what catches a renamed ADR, and deletion becomes an explicit, auditable
|
||||
annotation. Weakness: the annotation can be written on any bullet to avoid the check. Verifying the
|
||||
sha with `git cat-file -e` would close that, and was left out as over-engineering for three bullets.
|
||||
annotation. The annotation is anchored at the end of the value and the sha is 7-40 hex characters.
|
||||
Weakness: the annotation can be written on any bullet to avoid the check. Verifying the sha with
|
||||
`git cat-file -e` would close that; the user decided against it as over-engineering for three
|
||||
bullets, so the sha is format-checked only, not verified.
|
||||
- **(b) `Basis:` becomes free prose with no existence check (rejected).** Gives up the one check that
|
||||
catches a renamed or moved ADR.
|
||||
- **(c) Drop those `Basis:` lines and keep `none` with a prose reason (rejected).** Loses the
|
||||
@@ -92,13 +105,15 @@ fixtures in this repo". No such fixture or test exists in the tree or in history
|
||||
in `d1afdbe` with no test files, and the only vale test ever deleted (`4de5b6b`) guards an unrelated
|
||||
`E100`. Under Q2 it FAILed for a missing `Basis:`.
|
||||
|
||||
- **(e) Remove the entry and its `source_keys` citations (chosen).** The stated basis was false, so
|
||||
there is nothing honest to declare. The behavioural rules stay in the skills; only the provenance
|
||||
claim goes. The gate needs no allowlist.
|
||||
- **(e) Remove the entry and its `source_keys` citations (chosen, as the interim state).** The stated
|
||||
basis was false, so there is nothing honest to declare. The behavioural rules stay in the skills;
|
||||
only the provenance claim goes. The gate needs no allowlist.
|
||||
- **(a) `Basis: tests/test-vale-wrap.sh` (rejected).** Backs about one of six claims and overstates the
|
||||
rest.
|
||||
- **(b) Commit reproduction fixtures (rejected for now).** The right fix if the behaviours matter, but
|
||||
separate work from this issue.
|
||||
- **(b) Commit reproduction fixtures (chosen, supersedes the interim removal).** The user decided to
|
||||
commit real Vale reproduction fixtures under `plugins/lint` rather than soften the wording. The
|
||||
`house-vale-3-15-2-repro` claim is restored only once it is backed by committed fixtures, and it
|
||||
names them via `Basis:` (with `Research doc: none`). Until they land, the claim stays absent.
|
||||
- **(c) Allow `none` without `Basis:` for "house-verified" entries (rejected).** Reopens Q2 and gives
|
||||
an escape hatch for unverified claims.
|
||||
- **(d) Keep the entry and allowlist the two skills in the gate (rejected).** Keeps a false claim in
|
||||
@@ -107,16 +122,6 @@ in `d1afdbe` with no test files, and the only vale test ever deleted (`4de5b6b`)
|
||||
`configuration-reference.md` still says its rows were "reproduced against Vale 3.15.2"; that wording
|
||||
now has no provenance entry behind it and is left for a separate decision.
|
||||
|
||||
**Q4 — enforcement.** A corpus-wide sweep gate lands in the same change: a test or pre-push hook that
|
||||
runs `validate-provenance.sh` over every `plugins/*/.apm/skills/*/` and fails on any FAIL. Deferring it
|
||||
was rejected because without a caller the FAIL tier is inert; nothing but `check-scope-walkup-sync.sh`
|
||||
(on fixtures) invokes the validator today.
|
||||
|
||||
**Q5 — parser parity.** `parse_status` and `parse_research_doc` accept the bullet spelling
|
||||
(`- **Status:**`) as `parse_contributing_files` already does, with a regression test. Included because
|
||||
it is the same failure shape as #111 and #118 (a parser returns "nothing found", the caller reads it as
|
||||
"nothing declared"), sits in the same file, and `gitea-releases` already writes the unhyphenated form.
|
||||
|
||||
## Consequences
|
||||
|
||||
- About 40 `references/sources.md` entries migrate: roughly 30 repoint from a topic doc to the registry,
|
||||
@@ -133,4 +138,6 @@ it is the same failure shape as #111 and #118 (a parser returns "nothing found",
|
||||
its mention from `skill-file-structure.md` and `create.md` where present.
|
||||
- The corpus-wide sweep is a new gate: register it in `docs/spec/gates.md` and
|
||||
`.pre-commit-config.yaml`. The corpus must be migrated in the same change or the suite goes red.
|
||||
- The validator rejects an absolute path or one that escapes the repo with `..` in `Research doc:` and
|
||||
`Basis:`, and rejects a `Research doc:` value with internal whitespace, backticks, or a comma list.
|
||||
- Reversing this means re-migrating the same entries, which is why it is recorded.
|
||||
|
||||
@@ -21,12 +21,12 @@ Install hooks via `pc-run`, wiring **all three stages**. This repo's `.pre-commi
|
||||
`default_install_hook_types`, so a plain install silently skips `commit-msg` (Conventional Commits)
|
||||
and `pre-push` (everything below).
|
||||
|
||||
The pre-push command reports **10** hooks, not 8. The extra two are pre-commit's own `meta` hooks,
|
||||
The pre-push command reports **11** hooks, not 9. The extra two are pre-commit's own `meta` hooks,
|
||||
`check-hooks-apply` and `check-useless-excludes`: they declare no `stages:`, so they run at every
|
||||
stage including this one. Both are declared in this repo's `.pre-commit-config.yaml` like everything
|
||||
else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Eight
|
||||
else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Nine
|
||||
is the count of hooks this repo authors itself, and `--hook-stage pre-push --all-files` is a full
|
||||
rehearsal of all eight. A PR merged through Gitea's merge button runs none of them: no local push
|
||||
rehearsal of all nine. A PR merged through Gitea's merge button runs none of them: no local push
|
||||
happens at all.
|
||||
|
||||
A real push has a gap of its own. When one `git push` carries several refs
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
- **URL:** https://gitea.com/gitea/gitea-mcp
|
||||
- **Description:** Official gitea-mcp repository; operation/*.go source files documenting the MCP tools, their parameters, and CLI flags. Originally extracted at v1.3.0; the input parameter schemas in `references/call-signatures.md` were re-verified live via `ToolSearch` against the deployed server, **last verified at v1.7.0** as reported by `get_gitea_mcp_server_version`.
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digested in gitea/api-reference.md, Releases and Tags section, and gitea/troubleshooting.md, `delete_release` numeric-id gotcha and `per_page` defaults)
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digest: plugins/gitea/docs/research/docs/gitea/api-reference.md, Releases and Tags section; also plugins/gitea/docs/research/docs/gitea/troubleshooting.md, `delete_release` numeric-id gotcha and `per_page` defaults)
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (Dispatch table, Gotchas)
|
||||
@@ -16,7 +16,7 @@
|
||||
|
||||
- **URL:** https://gitea.com/gitea/gitea-mcp/raw/branch/main/operation/repo/slim.go
|
||||
- **Description:** Slim response shape structs from gitea-mcp source; defines exactly which fields the MCP server returns for tags and releases.
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digested in gitea/api-reference.md, Releases and Tags response shapes)
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digest: plugins/gitea/docs/research/docs/gitea/api-reference.md, Releases and Tags response shapes)
|
||||
|
||||
**Contributing files:**
|
||||
- references/call-signatures.md (release/tag object shapes)
|
||||
@@ -27,7 +27,7 @@
|
||||
|
||||
- **URL:** context7:/websites/gitea
|
||||
- **Description:** Official Gitea docs mirror on Context7 (docs.gitea.com content) — release and tag semantics, draft/prerelease behavior.
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digested in gitea/workflow-conventions.md, Release and tag conventions section)
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digest: plugins/gitea/docs/research/docs/gitea/workflow-conventions.md, Release and tag conventions section)
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (Gotchas — draft/prerelease as explicit flags)
|
||||
@@ -39,7 +39,7 @@
|
||||
|
||||
- **URL:** context7:/git_gitea_com/gitea_tea
|
||||
- **Description:** Official `tea` CLI (reference Gitea client) docs on Context7 — practitioner release/tag command patterns, semver tag conventions, draft/prerelease flags, release-notes-from-file conventions.
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digested in gitea/workflow-conventions.md, Release and tag conventions section)
|
||||
- **Research doc:** plugins/gitea/docs/research/docs/gitea/sources.md (digest: plugins/gitea/docs/research/docs/gitea/workflow-conventions.md, Release and tag conventions section)
|
||||
|
||||
**Contributing files:**
|
||||
- references/conventions.md (semver tag naming, release-notes sourcing)
|
||||
|
||||
@@ -380,14 +380,18 @@ def parse_field_values(content, slug, label):
|
||||
found" for the other two, and every caller read that as "nothing declared"
|
||||
(#121, second comment; the same failure shape as #111 and #118). A header's
|
||||
bullets stop at the first line that is neither blank nor a bullet, and a
|
||||
'- **Other:**' bullet is the NEXT field, not a value of this one.
|
||||
'- **Other:**' bullet is the NEXT field, not a value of this one ('* '
|
||||
bullets count too, and a bold bullet with no colon is a value).
|
||||
"""
|
||||
block = _entry_block(content, slug)
|
||||
if block is None:
|
||||
return []
|
||||
values = []
|
||||
lines = block.splitlines()
|
||||
label_re = re.compile(r'^(?:- )?\*\*' + re.escape(label) + r':\*\*[ \t]*(.*)$')
|
||||
label_re = re.compile(r'^(?:[-*] )?\*\*' + re.escape(label) + r':\*\*[ \t]*(.*)$')
|
||||
# A bullet that opens with a bold '**Other:**' label is the NEXT field. A
|
||||
# bold bullet WITHOUT the colon ('- **docs/x.md**') is just a value.
|
||||
next_field_re = re.compile(r'^[-*] \*\*[^*]*:\*\*')
|
||||
i = 0
|
||||
while i < len(lines):
|
||||
m = label_re.match(lines[i])
|
||||
@@ -398,15 +402,21 @@ def parse_field_values(content, slug, label):
|
||||
if inline:
|
||||
values.append(inline)
|
||||
continue
|
||||
found = False
|
||||
while i < len(lines):
|
||||
line = lines[i].strip()
|
||||
if not line:
|
||||
i += 1
|
||||
continue
|
||||
if not line.startswith('- ') or line.startswith('- **'):
|
||||
if not (line.startswith('- ') or line.startswith('* ')) or next_field_re.match(line):
|
||||
break
|
||||
values.append(line[2:].strip())
|
||||
found = True
|
||||
i += 1
|
||||
if not found:
|
||||
# The field is DECLARED but carries nothing: report an empty value,
|
||||
# not an absent field, so callers say 'empty' rather than 'missing'.
|
||||
values.append('')
|
||||
return values
|
||||
|
||||
def parse_research_docs(content, slug):
|
||||
@@ -441,8 +451,10 @@ def parse_basis(content, slug):
|
||||
RESEARCH_DOC_ANNOTATION_RE = re.compile(r'[§→(]')
|
||||
|
||||
def strip_research_doc_annotation(value):
|
||||
"""Path part of a Research doc value, with any section annotation removed."""
|
||||
return RESEARCH_DOC_ANNOTATION_RE.split(value, maxsplit=1)[0].strip()
|
||||
"""Path part of a Research doc value, with any section annotation removed
|
||||
and surrounding backticks unwrapped ('`a/b.md`' resolves as 'a/b.md')."""
|
||||
head = RESEARCH_DOC_ANNOTATION_RE.split(value, maxsplit=1)[0].strip()
|
||||
return head.strip('`').strip()
|
||||
|
||||
def research_doc_is_none(value):
|
||||
"""True when a Research doc value declares that no research doc backs the slug.
|
||||
@@ -452,7 +464,9 @@ def research_doc_is_none(value):
|
||||
unresolvable path. Checked BEFORE the annotation strip, because '(none)'
|
||||
is itself a parenthesis and would strip to the empty string.
|
||||
"""
|
||||
return re.match(r'\(?none\b', value.strip(), re.IGNORECASE) is not None
|
||||
# 'none/foo.md' and 'none-of-these.md' are PATHS: after 'none' only the end,
|
||||
# whitespace or an em/en dash may follow (or the parenthesised '(none)').
|
||||
return re.match(r'(?:\(none\)|none(?=$|\s|[\u2014\u2013]))', value.strip(), re.IGNORECASE) is not None
|
||||
|
||||
# A Research doc or Basis value names ONE path. The three list spellings seen
|
||||
# in the corpus — a brace expansion, a comma-separated list and a
|
||||
@@ -463,15 +477,48 @@ def research_doc_is_none(value):
|
||||
# said so. Detected on the raw value, with commas and semicolons INSIDE the
|
||||
# annotation left alone: those are prose ('cross-cutting; no dedicated
|
||||
# section'), and only a second path-shaped token after a ';' is a list.
|
||||
SECOND_PATH_AFTER_SEMICOLON_RE = re.compile(r';\s*[\w.\-]+/[\w./\-]*\.[A-Za-z]+')
|
||||
SECOND_PATH_AFTER_SEMICOLON_RE = re.compile(r'[;,]\s*[\w.\-]+/[\w./\-]*\.[A-Za-z]+')
|
||||
|
||||
BASIS_REMOVED_RE = re.compile(r'\(removed in [0-9a-fA-F]{7,40}\b[^)]*\)')
|
||||
# Only the LAST character class matters for the removal annotation: it must end
|
||||
# the value, so '(removed in <sha>) but still here' is not the annotation.
|
||||
BASIS_REMOVED_RE = re.compile(r'\(removed in [0-9a-f]{7,40}\)\s*$')
|
||||
|
||||
PAREN_GROUP_RE = re.compile(r'\([^()]*\)')
|
||||
|
||||
def names_more_than_one_path(value):
|
||||
path_part = strip_research_doc_annotation(value)
|
||||
if '{' in path_part or '}' in path_part or ',' in path_part or ';' in path_part:
|
||||
"""True when a Research doc / Basis value is a list rather than one path.
|
||||
|
||||
Three places to look, none of which is prose:
|
||||
- the leading path token: whitespace inside it ('a.md b.md'), or any of
|
||||
, ; { } or a stray backtick, is a list;
|
||||
- the text after it, once balanced '(...)' annotations are removed (a
|
||||
comma or semicolon INSIDE parentheses is prose): a bare , ; { } there
|
||||
is a second path parked after the first ('a.md (x), b.md');
|
||||
- after a section marker (§, →) prose may hold commas, so only a
|
||||
second path-SHAPED token after ',' or ';' counts.
|
||||
"""
|
||||
head = strip_research_doc_annotation(value)
|
||||
if re.search(r'[\s,;{}`]', head):
|
||||
return True
|
||||
return SECOND_PATH_AFTER_SEMICOLON_RE.search(value) is not None
|
||||
rest = value[len(RESEARCH_DOC_ANNOTATION_RE.split(value, maxsplit=1)[0]):]
|
||||
while True:
|
||||
stripped = PAREN_GROUP_RE.sub('', rest)
|
||||
if stripped == rest:
|
||||
break
|
||||
rest = stripped
|
||||
if rest.lstrip().startswith(('§', '→')):
|
||||
return SECOND_PATH_AFTER_SEMICOLON_RE.search(rest) is not None
|
||||
return re.search(r'[,;{}]', rest) is not None
|
||||
|
||||
def path_escapes_repo(repo_root, rel_path):
|
||||
"""True when rel_path is absolute or resolves (symlinks followed) outside
|
||||
repo_root. Research doc and Basis are repo-relative, so anything else is
|
||||
either a mistake or a way to make the checker read a file elsewhere."""
|
||||
if os.path.isabs(rel_path):
|
||||
return True
|
||||
root = os.path.realpath(repo_root)
|
||||
real = os.path.realpath(os.path.join(root, rel_path))
|
||||
return not (real == root or real.startswith(root + os.sep))
|
||||
|
||||
def find_repo_root(start_dir):
|
||||
"""Walk up from start_dir until we find a directory containing .git."""
|
||||
@@ -904,14 +951,16 @@ for slug in unique_slugs:
|
||||
f"Research doc field missing",
|
||||
f"references/sources.md (## {slug})",
|
||||
f"The '## {slug}' entry in sources.md has no '- **Research doc:**' line.",
|
||||
f"Add '- **Research doc:** <path-or-(none)>' to the '## {slug}' entry in references/sources.md."
|
||||
f"Add '- **Research doc:** <path to the plugin's research sources.md>' to the '## {slug}' entry in references/sources.md, "
|
||||
f"or '- **Research doc:** none' plus a '- **Basis:** <repo path>' line if no registry backs it."
|
||||
)
|
||||
elif rd_value == "" or PLACEHOLDER_RE.search(rd_value):
|
||||
emit_fail(
|
||||
f"Research doc field is empty or placeholder",
|
||||
f"references/sources.md (## {slug})",
|
||||
f"The '## {slug}' entry has an unfilled Research doc value.",
|
||||
f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable."
|
||||
f"Set '- **Research doc:**' to the plugin's research sources.md (a path relative to the repo root), or to 'none' "
|
||||
f"with a '- **Basis:** <repo path>' line if no registry backs this entry."
|
||||
)
|
||||
elif research_doc_is_none(rd_value):
|
||||
# An entry with no Research registry must still say what it WAS drawn
|
||||
@@ -944,6 +993,14 @@ for slug in unique_slugs:
|
||||
f"The Basis value '{basis}' is a brace expansion or a comma- or semicolon-separated list.",
|
||||
f"Write one '- **Basis:** <repo path>' line per path."
|
||||
)
|
||||
elif BASIS_REMOVED_RE.search(basis):
|
||||
# A path the entry HISTORICALLY rested on, annotated
|
||||
# '(removed in <sha>)' at the end of the value, is a declaration
|
||||
# that it is gone on purpose. The sha is not resolved
|
||||
# (git cat-file was judged over-engineering, ADR-0028 Q7), and
|
||||
# with no repo root there is nothing to check either way, so
|
||||
# this skips silently in both cases.
|
||||
continue
|
||||
elif not repo_root:
|
||||
emit_info(
|
||||
f"Basis check skipped for '{slug}' — no repo root above the skill directory",
|
||||
@@ -951,12 +1008,13 @@ for slug in unique_slugs:
|
||||
f"'{basis}' is a path relative to the repo root, but no ancestor of the skill directory contains a .git entry, "
|
||||
f"so it cannot be resolved. Run this script against a skill inside a checkout."
|
||||
)
|
||||
elif BASIS_REMOVED_RE.search(basis):
|
||||
# A path the entry HISTORICALLY rested on, annotated
|
||||
# '(removed in <sha>)', is a declaration that it is gone on
|
||||
# purpose. The sha is not resolved: the annotation is the
|
||||
# author saying "deleted, and here is where to look".
|
||||
continue
|
||||
elif path_escapes_repo(repo_root, basis_path):
|
||||
emit_fail(
|
||||
f"Basis path '{basis_path}' is outside the repository for '{slug}'",
|
||||
f"references/sources.md (## {slug})",
|
||||
f"'{basis_path}' is absolute or resolves outside the repo root. Basis names repo paths.",
|
||||
f"Use a path relative to the repo root that stays inside it."
|
||||
)
|
||||
elif not os.path.exists(os.path.join(repo_root, basis_path)):
|
||||
emit_fail(
|
||||
f"Basis path '{basis_path}' does not exist",
|
||||
@@ -993,6 +1051,13 @@ for slug in unique_slugs:
|
||||
f"Check 7 did not run for this slug. "
|
||||
f"Give the value a file path relative to the repo root, or record 'none' plus a '- **Basis:**' if no registry backs this entry."
|
||||
)
|
||||
elif path_escapes_repo(repo_root, rd_path):
|
||||
emit_fail(
|
||||
f"Research doc '{rd_path}' for '{slug}' is outside the repository",
|
||||
f"references/sources.md (## {slug})",
|
||||
f"'{rd_path}' is absolute or resolves outside the repo root. Research doc names a file in this repo.",
|
||||
f"Point Research doc at the plugin's research sources.md, as a path relative to the repo root."
|
||||
)
|
||||
else:
|
||||
rd_abs = os.path.join(repo_root, rd_path)
|
||||
if not os.path.isfile(rd_abs):
|
||||
|
||||
@@ -1948,3 +1948,278 @@ EOF
|
||||
refute_output --partial "Research doc field missing"
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
# --- #121 review round: list detection, repo confinement, parser edge cases --
|
||||
|
||||
# Helper: one-line Research doc / Basis fixtures over make_entry_skill.
|
||||
rd_fixture() { make_entry_skill "$TMPDIR/fakerepo" "$1"; }
|
||||
basis_fixture() { make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s' '- **Research doc:** none' "$1")"; }
|
||||
|
||||
@test "#121 FAIL: a brace-only Research doc (no comma) names more than one path" {
|
||||
rd_fixture '- **Research doc:** docs/research/{sources}.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a bare 'a.md; b.md' Research doc names more than one path" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md; docs/other-basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an annotated first path followed by ', second-path' is a list" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md (x), docs/research/topic.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a space-separated pair of Research docs is a list" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md docs/other-basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a space-separated pair of backticked Research docs is a list" {
|
||||
rd_fixture '- **Research doc:** `docs/research/sources.md` `docs/other-basis.md`'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 pass: a single backticked Research doc path is unwrapped before resolving" {
|
||||
rd_fixture '- **Research doc:** `docs/research/sources.md`'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_success
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
@test "#121 pass: a ';' inside an annotation that holds a path is prose (path part only is checked)" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md (digested; docs/other-basis.md)'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_success
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a bare 'a.md; b.md' Basis names more than one path" {
|
||||
basis_fixture '- **Basis:** docs/basis.md; docs/other-basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis value names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a brace Basis names more than one path" {
|
||||
basis_fixture '- **Basis:** docs/{basis,other-basis}.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis value names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a brace-only Basis names more than one path" {
|
||||
basis_fixture '- **Basis:** docs/{basis}.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis value names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a space-separated Basis pair names more than one path" {
|
||||
basis_fixture '- **Basis:** docs/basis.md docs/other-basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis value names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 pass: a backticked Basis path is unwrapped before resolving" {
|
||||
basis_fixture '- **Basis:** `docs/basis.md`'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_success
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an absolute Research doc path is outside the repo" {
|
||||
rd_fixture '- **Research doc:** /etc/passwd'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a '..' Research doc escape is outside the repo" {
|
||||
rd_fixture '- **Research doc:** ../outside/sources.md'
|
||||
mkdir -p "$TMPDIR/outside"
|
||||
printf '# R\n\n## my-source\n' > "$TMPDIR/outside/sources.md"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an absolute Basis path is outside the repo" {
|
||||
basis_fixture '- **Basis:** /etc/passwd'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a '..' Basis escape is outside the repo even though the file exists" {
|
||||
basis_fixture '- **Basis:** ../outside.md'
|
||||
printf 'x\n' > "$TMPDIR/outside.md"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: '(removed in abc)' is too short a sha to skip the check" {
|
||||
basis_fixture '- **Basis:** docs/deleted-adr.md (removed in abc)'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis path 'docs/deleted-adr.md' does not exist"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: '(removed in <sha>)' followed by more text is not the annotation" {
|
||||
basis_fixture '- **Basis:** docs/deleted-adr.md (removed in 5b80f30) but really still here'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis path 'docs/deleted-adr.md' does not exist"
|
||||
}
|
||||
|
||||
@test "#121 pass: '(removed in <sha>)' Basis with no repo root is skipped silently" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_skill_with_source_keys "$skill"
|
||||
mkdir -p "$skill/references"
|
||||
cat > "$skill/references/sources.md" <<EOF
|
||||
# Sources
|
||||
|
||||
## my-source
|
||||
|
||||
- **URL:** https://example.com/my-source
|
||||
- **Description:** A test source.
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Research doc:** none
|
||||
- **Basis:** docs/gone.md (removed in 5b80f30)
|
||||
- **Status:** \`extracted\`
|
||||
EOF
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
refute_output --partial "Basis check skipped"
|
||||
}
|
||||
|
||||
@test "#121 parity: a '- **X**' bullet under a Basis header is a value, not the next field" {
|
||||
make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s\n%s' '- **Research doc:** none' '**Basis:**' '- **docs/gone.md**')"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "does not exist"
|
||||
refute_output --partial "Basis missing"
|
||||
}
|
||||
|
||||
@test "#121 parity: '* ' bullets under a Basis header are read" {
|
||||
make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s\n%s\n%s' '- **Research doc:** none' '**Basis:**' '* docs/basis.md' '* docs/gone.md')"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis path 'docs/gone.md' does not exist"
|
||||
}
|
||||
|
||||
@test "#121 'none/foo.md' is a path, not a 'none' declaration" {
|
||||
rd_fixture '- **Research doc:** none/foo.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
refute_output --partial "Basis missing"
|
||||
}
|
||||
|
||||
@test "#121 'none-of-these.md' is a path, not a 'none' declaration" {
|
||||
rd_fixture '- **Research doc:** none-of-these.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
refute_output --partial "Basis missing"
|
||||
}
|
||||
|
||||
@test "#121 pass: 'None' and 'NONE' are recognised case-insensitively" {
|
||||
local v
|
||||
for v in None NONE; do
|
||||
make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s' "- **Research doc:** $v" '- **Basis:** docs/basis.md')"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_success
|
||||
assert_output ""
|
||||
done
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an empty Basis value is empty, not missing" {
|
||||
basis_fixture '- **Basis:**'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis is empty or placeholder"
|
||||
refute_output --partial "Basis missing"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a 'FILL IN:' Basis is a placeholder" {
|
||||
basis_fixture '- **Basis:** FILL IN: repo path'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis is empty or placeholder"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an empty inline Research doc says empty, not missing" {
|
||||
rd_fixture '- **Research doc:**'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc field is empty or placeholder"
|
||||
refute_output --partial "Research doc field missing"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: a missing Research doc advises the new grammar, not '<path-or-(none)>'" {
|
||||
rd_fixture ''
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc field missing"
|
||||
refute_output --partial "path-or-(none)"
|
||||
assert_output --partial "Basis"
|
||||
}
|
||||
|
||||
@test "#121 parity: an inline Basis with no leading hyphen is read" {
|
||||
basis_fixture '**Basis:** docs/gone.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis path 'docs/gone.md' does not exist"
|
||||
refute_output --partial "Basis missing"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: 'a.md;b.md' with no space is a list, for Research doc" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md;docs/basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Research doc names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: 'a.md;b.md' with no space is a list, for Basis" {
|
||||
basis_fixture '- **Basis:** docs/basis.md;docs/other-basis.md'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis value names more than one path"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an absolute Basis path is outside the repo even when it points inside the checkout" {
|
||||
make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s' '- **Research doc:** none' "- **Basis:** $TMPDIR/fakerepo/docs/basis.md")"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 FAIL: an absolute Research doc path is outside the repo even when it points inside the checkout" {
|
||||
rd_fixture "- **Research doc:** $TMPDIR/fakerepo/docs/research/sources.md"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "outside the repository"
|
||||
}
|
||||
|
||||
@test "#121 parity: a '* **Basis:**' bullet spelling is read" {
|
||||
make_entry_skill "$TMPDIR/fakerepo" "$(printf '%s\n%s' '- **Research doc:** none' '* **Basis:** docs/gone.md')"
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_failure
|
||||
assert_output --partial "Basis path 'docs/gone.md' does not exist"
|
||||
}
|
||||
|
||||
@test "#121 pass: a comma inside a section-marker annotation is prose, not a list" {
|
||||
rd_fixture '- **Research doc:** docs/research/sources.md § "Foo, bar and baz"'
|
||||
run bash "$SCRIPT" "$TMPDIR/fakerepo/my-skill"
|
||||
assert_success
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
@@ -177,11 +177,11 @@ If a research `sources.md` is present in the conversation context:
|
||||
source slugs (e.g. `plugins/myplugin/docs/research/docs/<topic>/sources.md`) — never a topic
|
||||
document, and never a list: no brace expansion, no comma- or semicolon-separated paths, no
|
||||
second `Research doc:` line. A pointer to the topic document that digested the source goes in
|
||||
an annotation after the path, e.g. `<registry path> (digested in remotes.md)`, where it is not
|
||||
an annotation after the path, e.g. `<registry path> (digest: <full plugins/... path of the topic doc>)`, where it is not
|
||||
checked. `/factory-audit` fails a slug missing from the registry it names.
|
||||
|
||||
If the entry has no Research registry — an org convention, an ADR, a house-verified
|
||||
reproduction — write `- **Research doc:** none` and name what it was drawn from with one
|
||||
If the entry has no Research registry — an org convention, an ADR, a reproduction
|
||||
backed by committed fixtures or tests named in `Basis:` — write `- **Research doc:** none` and name what it was drawn from with one
|
||||
`- **Basis:** <repo path>` line per path. Each Basis path is checked to exist; annotate one
|
||||
that has since been deleted `(removed in <sha>)` and the check is skipped. `none` with no Basis
|
||||
is a FAIL.
|
||||
|
||||
@@ -11,6 +11,7 @@ metadata:
|
||||
version: "0.1.4"
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
topic: configuration-reference
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Core Settings
|
||||
@@ -81,7 +82,7 @@ Only *package* styles need fetching. A style whose YAML rule files are already c
|
||||
- `Vale.Avoid` — enforces the project's rejected vocabulary terms.
|
||||
- `Vale.Repetition` — flags repeated words (e.g. "the the").
|
||||
|
||||
`Packages` (top-level, what `vale sync` downloads) and `BasedOnStyles` (per-glob, what activates) are separate keys: a style lints a file only once it is in both. Every row below reproduced against Vale 3.15.2:
|
||||
`Packages` (top-level, what `vale sync` downloads) and `BasedOnStyles` (per-glob, what activates) are separate keys: a style lints a file only once it is in both. Every row below is asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`) except the `vale sync` row that adds the name to `Packages`, which needs the network and is not covered:
|
||||
|
||||
| Configuration | Result |
|
||||
|---|---|
|
||||
@@ -97,7 +98,7 @@ Only *package* styles need fetching. A style whose YAML rule files are already c
|
||||
|
||||
## Frontmatter Scopes
|
||||
|
||||
House-verified behaviour, not documented on vale.sh — reproduced locally against Vale 3.15.2.
|
||||
House-verified behaviour, not documented on vale.sh — asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`).
|
||||
|
||||
A rule scoped to `text.frontmatter.<key>` (e.g. `text.frontmatter.description`) matches reliably when that field's value is a single physical line, and breaks on most — not all — multi-line forms. Multi-line forms spanning 2+ lines:
|
||||
|
||||
|
||||
@@ -7,3 +7,12 @@
|
||||
- **Research doc:** plugins/lint/docs/research/docs/vale/sources.md
|
||||
- **Contributing files:** SKILL.md, references/configuration-reference.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## house-vale-3-15-2-repro
|
||||
|
||||
- **URL:** (house-verified — reproduced against the `vale` binary by a committed test, not an external source)
|
||||
- **Description:** Behaviour of Vale 3.15.2 asserted by the committed test (purpose-built fixtures, real `vale` run), where vale.sh documents nothing: the `E100 [loadStyles]` / exit-2 failure for a `BasedOnStyles` name absent from `StylesPath`, `vale sync` reporting `Synced 0 package(s)` for a name not declared in `Packages`, the `E201` / exit-2 failure when the `StylesPath` directory does not exist, the exit-0 no-op of an empty style directory, the `E201` / exit-2 failure when a core option is written below a `[glob]` header (with `Packages` as the silent exception), the `E100 [lintMDX]` failure of an unmapped `.mdx` without `mdx2vast`, and the `text.frontmatter.<key>` scope matrix across multi-line YAML forms.
|
||||
- **Research doc:** none
|
||||
- **Basis:** tests/test-vale-3-15-2-behaviours.sh
|
||||
- **Contributing files:** SKILL.md, references/configuration-reference.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -10,6 +10,7 @@ metadata:
|
||||
category: lint
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
@@ -7,3 +7,12 @@
|
||||
- **Research doc:** plugins/lint/docs/research/docs/vale/sources.md
|
||||
- **Contributing files:** SKILL.md, references/troubleshooting.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## house-vale-3-15-2-repro
|
||||
|
||||
- **URL:** (house-verified — reproduced against the `vale` binary by a committed test, not an external source)
|
||||
- **Description:** Behaviour of Vale 3.15.2 asserted by the committed test (purpose-built fixtures, real `vale` run), where vale.sh documents nothing or documents it wrongly: an unmapped `.mdx` needs an external `mdx2vast` binary (absent, the whole invocation exits 2 with `E100 [lintMDX]`), under `[formats] mdx = md` the HTML-comment suppression form works and the JSX-comment form does not, the `spelling` check's `ignore` paths resolve against `StylesPath` or the working directory but never against the rule file's own directory and fail silently when they resolve nowhere, `ls-config` and the other `ls-*` subcommands report styles and paths but never rules, and the `text.frontmatter.<key>` scope matrix across multi-line YAML forms. Not asserted: the native-MDX column of the suppression table, which needs `mdx2vast` installed.
|
||||
- **Research doc:** none
|
||||
- **Basis:** tests/test-vale-3-15-2-behaviours.sh
|
||||
- **Contributing files:** SKILL.md, references/troubleshooting.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-vale-sh
|
||||
- house-vale-3-15-2-repro
|
||||
---
|
||||
|
||||
# Vale troubleshooting reference
|
||||
@@ -50,8 +51,7 @@ suppression syntax:
|
||||
| `[formats]` maps `mdx = md` (what `vale-config` recommends) | none | Markdown | `<!-- vale off -->` |
|
||||
| no `mdx` mapping (native MDX) | `npm install -g mdx2vast` | MDX | `{/* vale off */}` |
|
||||
|
||||
Key the markup to that config row, never to the file extension. Verified against Vale 3.15.2, same
|
||||
three fixtures under each config:
|
||||
Key the markup to that config row, never to the file extension. Asserted against Vale 3.15.2 by `tests/test-vale-3-15-2-behaviours.sh` (slug `house-vale-3-15-2-repro`) for the mapped column; the native-MDX column was observed with `mdx2vast` installed and is not covered by that test (it needs the binary):
|
||||
|
||||
| File | Mapped `mdx = md` | Native MDX (`mdx2vast` installed) |
|
||||
|---|---|---|
|
||||
@@ -119,7 +119,7 @@ ignore:
|
||||
**Where the file goes, and why a wrong answer is invisible.** Each entry resolves against the
|
||||
`StylesPath` root, or against the working directory `vale` is invoked from. It does **not** resolve
|
||||
against the rule file's own directory — which is the natural reading of the YAML above, since the
|
||||
path sits inside the rule, and it is wrong. Verified against Vale 3.15.2 across four fresh trees,
|
||||
path sits inside the rule, and it is wrong. Asserted against Vale 3.15.2 by the same test across four fresh trees,
|
||||
each with the same rule and the same unknown word:
|
||||
|
||||
| Where `ignore1.txt` was placed | Result |
|
||||
|
||||
@@ -23,10 +23,10 @@ set -euo pipefail
|
||||
# could not be audited must not read as a skill that failed the audit.
|
||||
#
|
||||
# The skill set is discovered by glob, not hardcoded, so a new skill is covered
|
||||
# the moment it grows a references/sources.md. Run from repo root or pass
|
||||
# the moment it grows a references/sources.md. Runs from any cwd: REPO_ROOT defaults to the parent of this script's directory, or pass
|
||||
# REPO_ROOT as arg.
|
||||
|
||||
REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"
|
||||
REPO_ROOT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
|
||||
if [[ ! -d "$REPO_ROOT" ]]; then
|
||||
echo "Provenance corpus check failed: REPO_ROOT '$REPO_ROOT' is not a directory." >&2
|
||||
exit 2
|
||||
|
||||
@@ -175,10 +175,85 @@ set +e
|
||||
bash "$SCRIPT" "$REPO_ROOT" > "$RUN_TMP/real.out" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ $rc -eq 0 || $rc -eq 1 ]]; then
|
||||
pass "gate runs to a verdict (0 or 1) against the real corpus (exit $rc)"
|
||||
if [[ $rc -eq 0 ]]; then
|
||||
pass "real corpus is clean (exit 0)"
|
||||
else
|
||||
fail "gate errored (exit $rc) against the real corpus: $(cat "$RUN_TMP/real.out")"
|
||||
fail "real corpus did not validate clean (exit $rc): $(cat "$RUN_TMP/real.out")"
|
||||
fi
|
||||
|
||||
# --- 8. Runs by absolute path from another cwd, with no argument ---
|
||||
echo ""
|
||||
echo "--- other cwd, no argument ---"
|
||||
set +e
|
||||
(cd "$RUN_TMP" && bash "$SCRIPT" > "$RUN_TMP/cwd.out" 2>&1)
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ $rc -eq 0 ]]; then
|
||||
pass "derives REPO_ROOT from the script location, not the cwd"
|
||||
else
|
||||
fail "expected exit 0 from a foreign cwd, got $rc: $(cat "$RUN_TMP/cwd.out")"
|
||||
fi
|
||||
|
||||
# --- 9. A skill dir without references/sources.md is skipped, not an error ---
|
||||
echo ""
|
||||
echo "--- skill without sources.md ---"
|
||||
R="$(make_repo)"
|
||||
make_skill "$R" good known-slug "$REGISTRY"
|
||||
mkdir -p "$R/plugins/p/.apm/skills/nosources"
|
||||
printf -- '---\nname: nosources\ndescription: x\n---\n' > "$R/plugins/p/.apm/skills/nosources/SKILL.md"
|
||||
set +e
|
||||
bash "$SCRIPT" "$R" > "$RUN_TMP/skip.out" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ $rc -eq 0 ]] && grep -q "1 skill(s) checked" "$RUN_TMP/skip.out" && ! grep -q "nosources" "$RUN_TMP/skip.out"; then
|
||||
pass "skill without sources.md is skipped silently and not counted"
|
||||
else
|
||||
fail "expected exit 0, 1 skill checked, no mention (got $rc): $(cat "$RUN_TMP/skip.out")"
|
||||
fi
|
||||
|
||||
# --- 10. Multiple failing skills are all reported ---
|
||||
echo ""
|
||||
echo "--- multiple failing skills ---"
|
||||
R="$(make_repo)"
|
||||
make_skill "$R" good known-slug "$REGISTRY"
|
||||
make_skill "$R" bad1 missing-one "$REGISTRY"
|
||||
make_skill "$R" bad2 missing-two "$REGISTRY"
|
||||
set +e
|
||||
bash "$SCRIPT" "$R" > "$RUN_TMP/multi.out" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ $rc -eq 1 ]] && grep -qE "Failing skills:.*bad1" "$RUN_TMP/multi.out" \
|
||||
&& grep -qE "Failing skills:.*bad2" "$RUN_TMP/multi.out" \
|
||||
&& ! grep -qE "Failing skills:.*good" "$RUN_TMP/multi.out"; then
|
||||
pass "exits 1 and names every failing skill"
|
||||
else
|
||||
fail "expected exit 1 naming bad1 and bad2 (got $rc): $(cat "$RUN_TMP/multi.out")"
|
||||
fi
|
||||
|
||||
# --- 11. An errored skill alongside a failing one: exit 2 wins, both named ---
|
||||
echo ""
|
||||
echo "--- errored + failing precedence ---"
|
||||
R="$(make_repo)"
|
||||
make_skill "$R" failing known-slug "$REGISTRY"
|
||||
make_skill "$R" broken known-slug "$REGISTRY"
|
||||
# Stub validator: FAIL for 'failing', "not auditable" for 'broken'.
|
||||
cat > "$R/$VALIDATOR_DIR/validate-provenance.sh" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
case "$1" in
|
||||
*/failing) echo "FAIL: stub"; exit 1 ;;
|
||||
*/broken) echo "stub: not auditable" >&2; exit 2 ;;
|
||||
esac
|
||||
exit 0
|
||||
EOF
|
||||
set +e
|
||||
bash "$SCRIPT" "$R" > "$RUN_TMP/prec.out" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
if [[ $rc -eq 2 ]] && grep -q "errored (could not audit): .*broken" "$RUN_TMP/prec.out" \
|
||||
&& grep -q "Failing skills: .*failing" "$RUN_TMP/prec.out"; then
|
||||
pass "exit 2 takes precedence over exit 1, and both are reported"
|
||||
else
|
||||
fail "expected exit 2 naming both (got $rc): $(cat "$RUN_TMP/prec.out")"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
220
tests/test-vale-3-15-2-behaviours.sh
Executable file
220
tests/test-vale-3-15-2-behaviours.sh
Executable file
@@ -0,0 +1,220 @@
|
||||
#!/usr/bin/env bash
|
||||
# Reproduction fixtures for the Vale 3.15.2 behaviours the lint plugin's vale-config and vale-run
|
||||
# skills state as house-verified (provenance key house-vale-3-15-2-repro). Each case builds a
|
||||
# purpose-built fixture in a temp dir, runs the real `vale` binary and asserts exit code plus
|
||||
# output. A behaviour that changes in a later Vale release fails here, which is the point: the
|
||||
# skill text is only backed while this test passes.
|
||||
#
|
||||
# Not reproducible here (mdx2vast is not installed in CI): the native-MDX halves of the mdx
|
||||
# suppression table. Only the mdx2vast-absent E100 and the `[formats] mdx = md` column are asserted.
|
||||
set -uo pipefail
|
||||
|
||||
if ! command -v vale &>/dev/null; then
|
||||
echo "SKIP: vale is not installed"
|
||||
exit 77
|
||||
fi
|
||||
EXPECTED="3.15.2"
|
||||
GOT="$(vale --version | awk '{print $3}')"
|
||||
if [[ "$GOT" != "$EXPECTED" ]]; then
|
||||
echo "SKIP: behaviours are pinned to Vale $EXPECTED, found $GOT"
|
||||
exit 77
|
||||
fi
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
|
||||
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
|
||||
|
||||
WORK="$(mktemp -d)"
|
||||
trap 'rm -rf "$WORK"' EXIT
|
||||
|
||||
OUT="" RC=0
|
||||
# run <dir> <vale args...>: run vale from <dir>, capture combined output and exit code.
|
||||
run() {
|
||||
local dir="$1"; shift
|
||||
OUT="$(cd "$dir" && vale --no-wrap "$@" 2>&1)"; RC=$?
|
||||
OUT="$(sed $'s/\x1b\\[[0-9;]*m//g' <<<"$OUT")"
|
||||
}
|
||||
# expect <label> <rc> <grep-fixed-pattern-or-empty>
|
||||
expect() {
|
||||
local label="$1" rc="$2" pat="${3:-}"
|
||||
if [[ "$RC" -ne "$rc" ]]; then
|
||||
fail "$label: exit $RC, want $rc"; echo "$OUT" | sed 's/^/ /'; return
|
||||
fi
|
||||
if [[ -n "$pat" ]] && ! grep -qF -- "$pat" <<<"$OUT"; then
|
||||
fail "$label: output lacks '$pat'"; echo "$OUT" | sed 's/^/ /'; return
|
||||
fi
|
||||
pass "$label"
|
||||
}
|
||||
|
||||
# tree <name>: fresh dir with styles/ and a one-line doc containing a repeated word.
|
||||
tree() {
|
||||
local d="$WORK/$1"; mkdir -p "$d/styles"
|
||||
printf 'This is the the sample.\n' >"$d/doc.md"
|
||||
echo "$d"
|
||||
}
|
||||
rule() { # rule <dir> <style>: a one-rule custom style flagging the word "foo"
|
||||
mkdir -p "$1/styles/$2"
|
||||
cat >"$1/styles/$2/Foo.yml" <<'Y'
|
||||
extends: existence
|
||||
message: "found '%s'"
|
||||
level: error
|
||||
tokens:
|
||||
- foo
|
||||
Y
|
||||
}
|
||||
|
||||
echo "1. BasedOnStyles names a style absent from StylesPath"
|
||||
d="$(tree c1)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Nope\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E100 loadStyles, exit 2" 2 "E100 [loadStyles]"
|
||||
grep -qF "style 'Nope' does not exist on StylesPath" <<<"$OUT" && pass "message names the style" || fail "message names the style"
|
||||
|
||||
echo "2. vale sync for a name in BasedOnStyles but not Packages"
|
||||
d="$(tree c2)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Nope\n' >"$d/.vale.ini"
|
||||
run "$d" sync
|
||||
expect "Synced 0 package(s), exit 0" 0 "Synced 0 package(s)"
|
||||
[[ -z "$(ls -A "$d/styles")" ]] && pass "nothing downloaded" || fail "nothing downloaded"
|
||||
run "$d" doc.md
|
||||
expect "next lint repeats E100" 2 "E100 [loadStyles]"
|
||||
|
||||
echo "3. StylesPath directory absent, only built-in Vale active"
|
||||
d="$WORK/c3"; mkdir -p "$d"; printf 'x\n' >"$d/doc.md"
|
||||
printf 'StylesPath = nostyles\n[*.md]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E201, exit 2" 2 "E201 Invalid value"
|
||||
grep -q "does not exist" <<<"$OUT" && pass "path-does-not-exist message" || fail "path-does-not-exist message"
|
||||
|
||||
echo "4. Empty style directory loads and lints nothing"
|
||||
d="$(tree c4)"; mkdir "$d/styles/Empty"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Empty\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "0 findings, exit 0" 0 "0 errors, 0 warnings and 0 suggestions"
|
||||
|
||||
echo "4b. Built-in Vale and committed YAML lint with no Packages entry"
|
||||
d="$(tree c4b)"; rule "$d" Mine; printf 'a foo b\n' >>"$d/doc.md"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale, Mine\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "built-in Vale.Repetition fires, exit 1" 1 "Vale.Repetition"
|
||||
grep -qF "Mine.Foo" <<<"$OUT" && pass "committed style fires" || fail "committed style fires"
|
||||
|
||||
echo "4c. Style in Packages-only (not BasedOnStyles) lints nothing"
|
||||
d="$(tree c4c)"; rule "$d" Mine; printf 'a foo b\n' >>"$d/doc.md"
|
||||
printf 'StylesPath = styles\nPackages = Mine\n[*.md]\nBasedOnStyles =\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "0 findings, exit 0" 0 "0 errors, 0 warnings and 0 suggestions"
|
||||
|
||||
echo "5. Core option below a [glob] header"
|
||||
d="$(tree c5)"
|
||||
printf '[*.md]\nBasedOnStyles = Vale\nStylesPath = styles\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "E201 core option, exit 2" 2 "is a core option"
|
||||
d="$(tree c5b)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale\nMinAlertLevel = error\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "MinAlertLevel below glob also E201" 2 "is a core option"
|
||||
d="$(tree c5c)"
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = Vale\nPackages = Foo\n' >"$d/.vale.ini"
|
||||
run "$d" doc.md
|
||||
expect "Packages below glob: no error (exit 1 from the repetition finding)" 1 "Vale.Repetition"
|
||||
run "$d" ls-config
|
||||
grep -q '"Packages": false' <<<"$OUT" && pass "Packages parsed as per-glob rule toggle" || fail "Packages parsed as per-glob rule toggle"
|
||||
run "$d" sync
|
||||
expect "sync reports Synced 0 package(s)" 0 "Synced 0 package(s)"
|
||||
|
||||
echo "6. text.frontmatter.<key> scope across YAML forms"
|
||||
d="$WORK/c6"; mkdir -p "$d/styles/FM"
|
||||
cat >"$d/styles/FM/Foo.yml" <<'Y'
|
||||
extends: existence
|
||||
message: "found '%s'"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
tokens:
|
||||
- foo
|
||||
Y
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = FM\n' >"$d/.vale.ini"
|
||||
fm() { # fm <name> <frontmatter lines...>
|
||||
local n="$1"; shift
|
||||
{ echo '---'; printf '%s\n' "$@"; echo '---'; echo; echo 'Body.'; } >"$d/$n.md"
|
||||
}
|
||||
fm single 'description: has foo here'
|
||||
fm literal 'description: |' ' line one' ' has foo here'
|
||||
fm folded 'description: >' ' line one' ' has foo here'
|
||||
fm plain 'description: line one' ' has foo here'
|
||||
fm squote "description: 'line one" " has foo here'"
|
||||
fm dquote 'description: "line one' ' has foo here"'
|
||||
run "$d" single.md; expect "single line lints" 1 "Foo"
|
||||
run "$d" literal.md; expect "| literal lints" 1 "Foo"
|
||||
run "$d" folded.md; expect "> folded silent" 0 "0 errors"
|
||||
run "$d" plain.md; expect "plain continuation silent" 0 "0 errors"
|
||||
run "$d" squote.md; expect "single-quoted multi-line silent" 0 "0 errors"
|
||||
run "$d" dquote.md; expect "double-quoted multi-line silent" 0 "0 errors"
|
||||
|
||||
echo "7. .mdx without mapping and without mdx2vast"
|
||||
d="$WORK/c7"; mkdir -p "$d/styles"
|
||||
printf 'This is the the sample.\n' >"$d/doc.md"; cp "$d/doc.md" "$d/doc.mdx"
|
||||
printf 'StylesPath = styles\n[*.{md,mdx}]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
if command -v mdx2vast &>/dev/null; then
|
||||
echo " SKIP: mdx2vast is installed; the absent-binary case cannot run"
|
||||
else
|
||||
run "$d" .
|
||||
expect "whole invocation dies with E100 lintMDX, exit 2" 2 "E100 [lintMDX]"
|
||||
grep -qF "mdx2vast not found" <<<"$OUT" && pass "mdx2vast not found" || fail "mdx2vast not found"
|
||||
grep -q "doc.md" <<<"$OUT" && fail ".md alongside produced no output" || pass ".md alongside produced no output"
|
||||
fi
|
||||
|
||||
echo "8. mdx mapped onto md: suppression form"
|
||||
d="$WORK/c8"; mkdir -p "$d/styles"
|
||||
printf 'StylesPath = styles\n[formats]\nmdx = md\n[*.{md,mdx}]\nBasedOnStyles = Vale\n' >"$d/.vale.ini"
|
||||
printf 'This is the the sample.\n' >"$d/ctl.mdx"
|
||||
printf '<!-- vale off -->\nThis is the the sample.\n<!-- vale on -->\n' >"$d/html.mdx"
|
||||
printf '{/* vale off */}\nThis is the the sample.\n{/* vale on */}\n' >"$d/jsx.mdx"
|
||||
run "$d" ctl.mdx; expect "control alerts" 1 "Vale.Repetition"
|
||||
run "$d" html.mdx; expect "HTML comment suppresses" 0 "0 errors"
|
||||
run "$d" jsx.mdx; expect "JSX comment does not suppress" 1 "Vale.Repetition"
|
||||
|
||||
echo "9. spelling ignore path resolution"
|
||||
mk_spell() { # mk_spell <name> ; leaves rule with ignore1.txt, no ignore file placed
|
||||
local d; d="$WORK/$1"; mkdir -p "$d/styles/MyStyle" "$d/proj"
|
||||
cat >"$d/styles/MyStyle/Spell.yml" <<'Y'
|
||||
extends: spelling
|
||||
message: "Did you really mean '%s'?"
|
||||
level: error
|
||||
ignore:
|
||||
- ignore1.txt
|
||||
Y
|
||||
printf 'The zzqwidget is here.\n' >"$d/proj/doc.md"
|
||||
printf 'StylesPath = ../styles\n[*.md]\nBasedOnStyles = MyStyle\n' >"$d/proj/.vale.ini"
|
||||
echo "$d"
|
||||
}
|
||||
d="$(mk_spell s1)"; printf 'zzqwidget\n' >"$d/styles/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file at StylesPath root works" 0 "0 errors"
|
||||
d="$(mk_spell s2)"; printf 'zzqwidget\n' >"$d/proj/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file in working directory works" 0 "0 errors"
|
||||
run "$d" --config=proj/.vale.ini proj/doc.md; expect "working-dir copy fails from another directory" 1 "zzqwidget"
|
||||
d="$(mk_spell s3)"; printf 'zzqwidget\n' >"$d/styles/MyStyle/ignore1.txt"
|
||||
run "$d/proj" doc.md; expect "ignore file beside the rule is not read" 1 "zzqwidget"
|
||||
d="$(mk_spell s4)"
|
||||
run "$d/proj" doc.md; expect "absent ignore file fails silently" 1 "zzqwidget"
|
||||
grep -qi "ignore1" <<<"$OUT" && fail "no diagnostic emitted for missing ignore file" || pass "no diagnostic emitted for missing ignore file"
|
||||
|
||||
echo "10. ls-config reports styles and paths, never rules"
|
||||
d="$(tree c10)"; rule "$d" MyStyle
|
||||
printf 'StylesPath = styles\n[*.md]\nBasedOnStyles = MyStyle\n' >"$d/.vale.ini"
|
||||
printf 'a foo b\n' >"$d/doc.md"
|
||||
run "$d" doc.md; expect "rule fires" 1 "MyStyle.Foo"
|
||||
run "$d" ls-config
|
||||
grep -qF '"MyStyle"' <<<"$OUT" && pass "ls-config names the style" || fail "ls-config names the style"
|
||||
grep -qF 'Foo' <<<"$OUT" && fail "ls-config must not name the rule" || pass "ls-config does not name the rule"
|
||||
for sub in ls-dirs ls-vars ls-metrics; do
|
||||
run "$d" "$sub"
|
||||
grep -qF 'Foo' <<<"$OUT" && fail "$sub must not name the rule" || pass "$sub does not name the rule"
|
||||
done
|
||||
run "$d" ls-config
|
||||
grep -qF '"Checks": null' <<<"$OUT" && pass "Checks: null" || fail "Checks: null"
|
||||
|
||||
echo
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ "$FAIL" -eq 0 ]]
|
||||
Reference in New Issue
Block a user