Vale locates a frontmatter description by matching the parsed YAML value back
against the source text, so any scalar whose value is not spelled out verbatim
loses the `text.frontmatter.description` scope entirely. The wrapper only
flattened `>` folded scalars, so plain, double-quoted and single-quoted
multi-line descriptions silently reported zero alerts and exit 0 — a clean pass
indistinguishable from a real one, in a prefilter whose callers are instructed
not to re-derive its verdict by judgment.
Implementation notes:
- Classify the scalar kind after `^description:[ \t]*` and reuse one shared
continuation-line generator for every form; `|` literal blocks keep their
line breaks, stay verbatim-matchable, and are still left untouched.
- Emit the flattened value in whichever scalar form needs no escape at all
(plain, then single-quoted, then double-quoted), because any escape breaks
the verbatim match. The old blanket `'` -> U+2019 substitution silently made
apostrophe-bearing rule tokens unmatchable across 63% of the corpus; it now
survives only for the one combination no YAML scalar can carry verbatim.
- Terminate continuations at a line flush with the key, not only on a shallower
indent — a `description:` followed by a flush-left line previously swallowed
the rest of the frontmatter.
- Route vale's value-taking flags explicitly instead of inferring targets by
file existence, and absolutize relative `--output`/`--path` values the way
`--config` already was, since the run `cd`s into the scratch mirror.
- Fail loudly on a nonexistent path instead of inheriting bare vale's fallback
to stdin, which rendered a typo'd path as `0 errors ... in stdin`, exit 0 —
a form the callers' `0 files` NOT-RUN guard cannot match.
- Follow symlinks when walking a directory argument, matching bare vale.
Refs: #85
Under set -u, "${arr[@]}" on an empty array aborts on bash before 4.4,
which is what macOS ships as /bin/bash. Three expansion sites now use
${arr[@]+"${arr[@]}"} consistently.
The hazard is not currently reachable: verified on a bash 3.2.57 built
from source that all seven invocation shapes succeed against the
previous code, including zero args, flags-only and an empty directory.
vale_args is provably non-empty at every site because the default
--config branch always appends first. The guard is kept because that
invariant is non-local and untested, so an edit to the default-config
branch would reintroduce a macOS-only crash silently.
Test fidelity is deliberately mixed. Case 16 is static and is the only
one that fails against the previous code, since no bash 5 host can
reproduce the abort at runtime. Case 17 runs the emptiest invocations
under the oldest bash it can find and names that shell in its output
so it cannot overclaim. Case 18 guards against the tempting wrong fix
of dropping the quotes, which also silences the abort but word-splits
a path containing a space.
No other bash 4.x construct is present; swept for mapfile, declare -A,
case modification, negative indices, globstar, wait -n and namerefs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
pre-commit prefixes only entry[0] with the hook-repo clone path
(cmd = (prefix.path(cmd[0]), *cmd[1:])), so the --config argument in
.pre-commit-hooks.yaml resolved against the *consuming* repo's root
and hard-failed every external run with E100. Two of the three hooks
ADR-0014 promises were unusable.
vale-wrap.sh now self-locates its config from BASH_SOURCE when no
--config is supplied; an explicit --config still wins in all three
argv forms and stays cwd-relative. Both manifests drop the argument
and are kept byte-identical: the local repo: local config resolved
--config correctly only because the consuming repo *was* this repo,
and that divergence is why three review rounds missed the defect.
Also in the wrapper:
- replace GNU-only `realpath -m` with a portable abspath helper; -m is
load-bearing (dest does not exist yet), so BSD realpath aborted the
script under set -e on macOS
- walk directory arguments instead of passing them through unflattened,
which reported a clean 0-error run for files that fail when named
explicitly
- read/write with errors='surrogateescape' so one non-UTF-8 .md under a
directory argument cannot abort the hook
New test-vale-hooks-consumer.sh builds the hook repo from the working
tree and points a file:// consumer at it, covering the manifest as a
hook repo for the first time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
skill-audit/agent-audit's Step 1 resolved vale-wrap.sh/.vale.ini via
`git rev-parse --show-toplevel`, which returns whichever repo the skill
happens to run in. Inside ai-development that works; in any external repo
that installs kyberforge@holocron as a plugin, it resolves to that repo's
own root, which has no .vale.ini — the prefilter silently fell back to
full LLM judgment. ADR-0013 named this as a deliberately deferred gap.
Vale's config/styles/wrapper now ship inside the plugin itself: a
canonical copy in agent-audit/assets/vale/ (Kyberforge + KyberforgeCopilot,
the superset agent-audit needs) and a smaller duplicate in
skill-audit/assets/vale/ (Kyberforge only) — per the no-cross-skill-path
rule already established for plugin cache-installs. Both skills resolve
these relative to their own directory, same as scripts/validate.sh
already does.
A new root .pre-commit-hooks.yaml exposes both copies plus
skill-size-check so any external repo can enforce the same rules via
`repo: <this-repo-url>, rev: <tag>` in its own pre-commit config,
independent of Claude Code entirely — the same mechanism covers CI. This
repo's own pre-commit hook now consumes the identical plugin-bundled
copies via repo: local (not a third root copy, and not a pinned
self-reference, which would lint working-tree edits against the last
tagged release instead of the change being made). Split into
vale-audit-prefilter-skill/-agent hooks after confirming, by diffing the
full corpus against both old and new config before deleting the old
files, that one combined hook pointed at only one copy silently 0-file-
skips the other file type.
scripts/check-vale-style-sync.sh guards the two copies against drift,
wired at pre-push alongside check-manifests.
ADR: 0014
Round-3 review of PR #85 found the "enforcing" pre-commit hook enforced
nothing. Vale's exit code keys on error-level alerts alone: five of the
six rules were level: warning, so they exited 0, and pre-commit hides
output from a passing hook — the alerts were invisible and blocked
nothing. ADR-0013 rejected a report-only trial tier and then shipped one
by accident.
Flatten every rule to level: error. Vale's own exit code is then correct,
so the hook entry drops to a bare vale-wrap.sh call and the graded
error->FAIL / warning->SUGGESTION mapping disappears from both audit
skills: every alert is a FAIL, in the gate and the audit alike. No
ignorable tier, matching shellcheck, the test suite and
conventional-pre-commit.
Delete Kyberforge.VagueQualifier. Measured against the 41 skill/agent
files as they stood before the rule ever ran: 2 hits. One marginal
("very different" -> "fundamentally different"), one an unfixable false
positive — caveman/SKILL.md quotes "of course" as an example of filler,
a mention not a use — which forced the only Vale suppression comments in
the repo. Those four lines go with it; two of them were dead anyway,
suppressing a frontmatter-scoped rule on a body line. Held-out prose (273
files) fired 15 times, 9 inside out-of-scope research examples and the
rest one word in two idioms in a single doc. SentenceOpenerThereIs
survives: 22 held-out hits, both in-corpus hits clean rewrites, zero
suppressions.
Widen .vale.ini's globs to [**/SKILL.md], [**/agents/*.md] and
[**/*.agent.md]. The plugins/*/-prefixed globs scoped nothing — Vale's *
crosses /, so they already matched docs/research/examples/**/agents/*.md
and assets/templates/SKILL.md, the two paths CONTEXT.md claimed they
excluded. Scoping is and was the hook's files: regex. The old globs also
hid a silent false negative: a skill outside plugins/ matched no section,
so Vale reported 0 files and exited 0, which both audits read as clean.
They now treat a 0-file run as NOT RUN and fall back to full judgment.
Also:
- vale-wrap.sh resolves relative --config values and file arguments
against the caller's cwd, as vale does, instead of the repo root, which
hard-errored from a subdirectory and silently skipped flattening for
file args that did not resolve from the root. Absolute paths inside the
cwd are relativized so reports cite resolvable paths, not scratch ones.
- vale-run's exit-code model was documented backwards ("exits non-zero
whenever it finds an alert at or above MinAlertLevel") and would have
led anyone following it to build a gate that passes everything. Its
Markdown suppression syntax was MDX-only and does not suppress in .md;
corrected in the skill and its troubleshooting reference, with
backtick/fence exemption documented as the first resort.
- skill-size-check.sh fails only above 500 lines, agreeing with
skill-audit's validate.sh <= 500 pass.
- ADR-0013 and CONTEXT.md amended to match, recording why graded
severities cannot gate.
Verified: 9 test scripts / 15 vale-wrap cases pass; vale-audit-prefilter,
skill-size-check and shellcheck pass --all-files; check-manifests and
claude plugin validate --strict clean. New tests fail against the old
script (3 of them) and pass against the new one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
Addresses PR #85's outstanding review items after grilling the open
questions against ADR-0013/CONTEXT.md/ADR-0010:
Blocking fixes:
- vale-wrap.sh: replace json.dumps() escaping (which silently defeated
Vale's frontmatter scope on any description containing a quote,
backslash, or non-ASCII char — ~58% of the corpus) with a single-quoted
YAML scalar, substituting a Unicode right single quote for embedded
apostrophes rather than '' doubling (Vale's frontmatter scanner isn't a
full YAML parser and silently truncates on '' too).
- vale-wrap.sh: fix a blank-line-inside-a-folded-description truncation
bug via indentation-based, blank-line-tolerant body capture; narrow
flattening to `>`-style scalars only (`|` already works unflattened).
- skill-audit/agent-audit Step 1: make the vale-wrap.sh invocation
cwd-independent via git rev-parse --show-toplevel, fixing a bug where
no single cwd satisfied all three Step 1 commands.
- styles/Kyberforge/VagueQualifier.yml: prune 17 tokens verified
false-positive-dominated on this repo's own voice via a real corpus
sweep (obvious, clearly, usually, several, simple, easy, completely,
simply, tiny, etc.), keep 13 with real or unattested noise. Revert the
28 prose "fixes" those tokens drove across 14 skill files back to their
original, correct wording, including a functional regression to
caveman/SKILL.md's own filler-word list (a mention, not a use) — now
guarded with vale-off comments against recurrence.
Gaps:
- --minAlertLevel=warning on the pre-commit hook and Step 1 invocation
so warning-level rules actually surface, without collapsing the
FAIL/SUGGESTION severity mapping skill-audit/agent-audit rely on.
- vale-wrap.sh: fix --config=<path> equals-form, absolute-path silent
no-op, and a zero-file-argument stdin hang.
- Route vale-run and lint-runner through a documented wrapper script
when a target repo has one, instead of unconditionally recommending
bare `vale`.
- Wire Kyberforge.VagueQualifier/SentenceOpenerThereIs into skill-audit/
agent-audit's dimension-mapping prose (Body discipline).
- Add plugins/lint/sources.md provenance for lint-runner (ADR-0010).
- Sync both marketplace.json lint-entry descriptions with plugin.json.
- Retune skill-size-check.sh's MAX_WORDS 5000->2900 (measured ~1.6-1.7
tokens/word on this repo's corpus, the old value gated at ~8,500
tokens against a stated 5,000 ceiling); fix the >/>= line-count
boundary and wc -l undercount on files with no trailing newline.
- Document the vale binary as a Setup prerequisite in AGENTS.md.
- Fix SentenceOpenerThereIs's dead regex alternative and add a real
sentence-start anchor/scope.
- Fix a stale docs/research/docs/vale/ index pointer in kyberforge's
docs README (moved to plugins/lint/ in e1a5403).
- Rewrite ADR-0013's Consequences section past-tense to describe what
actually landed, and record the styles-portability limitation
(repo-root placement stays intentional; deferred to a separate
session per this PR's review).
Test coverage: 9 new vale-wrap.sh fixtures (quotes, backslash/unicode,
blank-line paragraphs, --config= form, zero-arg/absolute-path handling,
literal-block no-regression) and boundary-pair tests for
skill-size-check.sh's line/word ceilings.
bash tests/run-tests.sh: 9 scripts + 125 bats assertions, all passing.
scripts/check-manifests.sh and claude plugin validate --strict: clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
Vale's text.frontmatter.description scope silently stops matching once
the description is a YAML block scalar spanning 2+ physical lines —
the style used by most skills/agents in this repo. scripts/vale-wrap.sh
flattens the description to one line in a scratch copy (preserving the
repo-relative path and total line count) before invoking real vale, and
both audit skills plus the pre-commit hook now call it instead of vale
directly. Also tightens the pre-commit hook's file glob to single path
segments so it can't cross into docs/research examples or asset
templates the way the audit skills' scoped invocations already avoid.
Addresses PR #85 review feedback.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxG5T8EJDgkABXxuneuFfn
Adds repo-root .vale.ini plus a custom Kyberforge style (description-opener,
vague-wording, and generic reference-pointer padding rules) and a
KyberforgeCopilot style scoped to .agent.md files (Use proactively check).
skill-audit and agent-audit Step 1 now run vale against the specific file(s)
being audited and defer the corresponding Description/Patterns/Body checks
to its output instead of re-deriving them by LLM judgment, per the split
proposed in issue #84.
Closes#84
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxG5T8EJDgkABXxuneuFfn
skill-audit/agent-audit now proactively trigger after a skill/agent
file is hand-edited outside skill-author/agent-author, not just on
explicit request — closing a gap from this session where a fork's
direct edits to agent-author/agent-audit shipped without their own
inline audit until forge was invoked to check afterward.
Also: plugin-author gains a gotcha on claude plugin validate --strict
auto-discovering every .md under agents/ regardless of manifest
declarations (ADR-0010); agent-audit's dimension count and
validate-provenance.sh's --help now match actual behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Independent skill-audit clean-check surfaced that the dimension-mapping
list and manual-fallback checklist both skipped the counterpart-missing
case, which validate.sh already treats as a hard FAIL.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
`claude plugin validate --strict` auto-discovers every .md under a
plugin's agents/ directory as an agent requiring frontmatter, so the
provenance file there always needs fake agent frontmatter to pass
validation. Confirmed empirically that an explicit `agents` manifest
array can't suppress this discovery. Move the file to <plugin-root>/
sources.md instead, and update agent-author/agent-audit accordingly.
Adds ADR-0010, partially superseding ADR-0005's `agents/sources.md`
convention. Fixes#63.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- agent-author: convert template comments from YAML (#) to HTML (<!-- -->)
- Easier to spot and distinguish from functional comments
- Add explicit "Delete template comments before shipping" reminders
- Update SKILL.md Steps 2-3 with removal instruction
- agent-audit: add comment-discipline check
- Flag excessive frontmatter documentation comments as padding
- Mirrors skill-audit's body-discipline principle
- Update coverage line to include comment-discipline dimension
This ensures agents follow the same comment-cleanup discipline as skills,
preventing template documentation from shipping with agent definitions.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
## Why
Two issues were found in `validate-provenance.sh` by `/skill-audit`:
1. `find_plugin_root` only walked up looking for bare `plugin.json`, missing
the `.claude-plugin/plugin.json` layout used by kyberforge plugins — matching
the logic already present in `validate.sh`.
2. Check numbers in `--help` and inline comments had a gap (0,1,2,4,5,6) from
a previously removed check, making the numbering confusing to readers.
## Implementation Notes
Check numbers renumbered sequentially 0–5 in both the `--help` output block
and the inline `# --- Check N:` comments. The `find_plugin_root` condition now
mirrors the `detect_scope` function in `validate.sh`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
Remove phantom cross-file name-match claim from SKILL.md manual fallback
list (validate.sh has no such check). Fix broken bats stem-mismatch test
to overwrite the Copilot file instead of the CC file, exercising the
actual Copilot stem check.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
- Fix validate.sh: remove incorrect name==stem check for CC files (CC docs say filename need not match name field); keep check for Copilot CLI only
- Fix validate.sh: plugin scope detection now checks both plugin.json and .claude-plugin/plugin.json
- Fix validate.sh: Copilot cloud/IDE agents (.github/copilot/agents/) have name as optional; path-based guard added
- Add validate.sh checks: Copilot body length >30,000 chars (SUGGESTION), Copilot-only fields in CC files (FAIL), subagent-unavailable tools in tools field (SUGGESTION)
- Add references/description-quality.md as conditional escape hatch for borderline description findings
- SKILL.md: name five audit dimensions in description; sharpen indirect-trigger phrasing
- SKILL.md: label pair-mandate as kyberforge project convention, not platform requirement
- SKILL.md: scope redundant name-match/body-empty checks to manual fallback only
- SKILL.md: add conditional reference to description-quality.md; update provider-safety description for new check categories; fix plugin scope gotcha to mention .claude-plugin/plugin.json
- SKILL.md: add INFO tier to result block template
- Add source_keys frontmatter to references/README.md; update sources.md to add description-quality.md to contributing files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
## Why
agent-author produces agents/sources.md at plugin scope to record which
research sources informed which agent files. agent-audit had no way to
validate this chain, leaving stale or missing provenance undetected.
## Implementation Notes
Validation is per-pair (the given agent file + its counterpart) rather
than plugin-wide, keeping the scope consistent with validate.sh. The
script exits 0 silently for non-plugin-scope agents.
source_keys is top-level in both CC .md and Copilot .agent.md files
(not under metadata:) to avoid conflict with Copilot's own metadata
field semantics. Checks 0, 1, 2, 4, 5, 6 mirror the skill provenance
set; upstream research-doc cross-reference checks (7, 8) are deferred.
agent-author Steps 2, 3, and 4 updated to formally specify the
agents/sources.md format and instruct authors to add source_keys to
both files when research sources are in context.
Refs: #60
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
## Why
agent-author produces paired agent definition files (Claude Code .md +
Copilot .agent.md) but had no companion audit skill to validate them.
agent-audit fills that gap, giving the same structured PASS/FAIL report
that skill-audit provides for SKILL.md files.
## Implementation Notes
- validate.sh uses scope detection (walk up for plugin.json / .git) to
locate the counterpart file and determine whether plugin-silently-ignored
fields (hooks, mcpServers, permissionMode) should be flagged
- CC-only and silently-ignored field lists are read from
references/field-inventory.md at runtime rather than hardcoded —
provenance back to the research corpus; see ADR-0019
- Single-file invocation (pass either file, counterpart derived) chosen
over directory or name+root — see ADR-0018
- 12 bats tests cover provider detection, scope detection, all FAIL paths,
and clean-pair pass
## Impact
- kyberforge bumped to v1.1.2
- agent-author close step should be updated to reference agent-audit (#11)
- Provenance/sources chain check deferred to #60
ADR: docs/adr/0018-agent-audit-single-file-invocation.md
ADR: docs/adr/0019-agent-audit-field-inventory-reference.md
Refs: #11
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0147vXtL5sP6vorDdqXGJJU9
