Files
holocron/plugins/kyberforge/skills/agent-audit/references/validation-scripts.md
Defame1297 4a5c3c0cff feat(kyberforge): enforce the ADR-0020 context contract for skills and agents
Skill name+description pairs are preloaded into every session, costing
~6,200 tokens across 39 skills before any skill is invoked. The authoring
rules mandated that growth: skill-author:104 and description-quality.md:21
both required padding, while skill-author:102 (the deflating rule) had no
FAIL condition behind it.

Gates (blocking, no baseline file):
- description 250 chars SUGGESTION / 400 FAIL, measured on the folded
  YAML value
- body-only 600 words SUGGESTION / 900 FAIL, independent of the unchanged
  whole-file 2770-word / 500-line spec backstop
- every boundary-clause routing target must resolve to a real skill or
  agent; catches skill-improve, neuledge-context and gitea-labels
- agents take the description gates but deliberately no body gate; a test
  pins that absence

Vale: DescriptionOpener widened to ^This\b, new CompositionNote rule
banning architecture notes from descriptions. 10 hits, 0 false positives.

Kyberforge's own four skills retrofitted: descriptions 3,364 -> 938 chars
(-72%), bodies 8,306 -> 2,487 words (-70%), all via the apm-workflow
dispatch pattern. Fixes the skill-improve dangling route and the
agent-author misroute to manual review.

Also fixes a pre-existing false positive where any line-initial 'read '
was flagged as interactive input, which had already caused two scripts to
be rewritten around it.

Refs: ADR-0020
2026-08-14 21:13:13 +00:00

3.6 KiB

source_keys
source_keys
claude-code-plugins-docs
claude-code-subagents-docs
github-custom-agents-configuration

Validation Scripts Reference

Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting. Nothing here is needed on a clean run.

Report the gap, do not guess

If a script cannot run at all — Bash denied, python3 unavailable, vale not installed — say so as an INFO finding naming the script and the missing dependency, then fall back to the manual checks below. An INFO never changes PASS/FAIL. Silently omitting the dimension a script would have covered reports a clean audit that checked less than it claims to have checked.

How the scripts detect scope

validate.sh and validate-provenance.sh walk up from the agent file's directory and stop at the first of these:

  1. An apm.yml carrying a top-level type: instructions|skill|hybrid|prompts line — plugin/APM scope, and that directory is the package root. An apm.yml with no type: is a marketplace-only manifest: skip it and keep walking.
  2. $HOME — user scope, checked before .git so a dotfiles-managed home directory that is its own repo cannot shadow it.
  3. A .git directory or file — project scope.
  4. The filesystem root — project scope.

plugin.json and .claude-plugin/plugin.json are not scope signals. A directory holding only a plugin.json and no apm.yml falls through to project or user scope.

validate-provenance.sh exits 0 silently when that walk does not land on a package root, and again when the package has no provenance data. Silence from it is a pass, not a skip you need to investigate.

Manual fallback

Every scope: required fields present (name, description, non-empty body); name is kebab-case; no FILL IN: placeholders in the description or body; the description at or under 400 characters measured on the folded YAML value.

Plugin/APM scope: name matches the filename stem; no HTML comments left in the frontmatter; no frontmatter key outside the apm-agent-allowlist section of references/field-inventory.md — open that file, do not work from memory.

Project/user scope: the counterpart file exists; name matches the filename stem in the Copilot .agent.md only (Claude Code files are exempt); no key from claude-code-only-fields in the Copilot file and none from copilot-only-fields in the CC file, both read from references/field-inventory.md.

Script-specific failures

  • Error: field-inventory.md not found (exit 2). validate.sh reads its field lists from references/field-inventory.md at load time and refuses to run without it, rather than falling back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not work around it.
  • vale reports 0 files. Treat the pass as NOT RUN, not as clean, and fall back to full Step 3 judgment for the dimensions it would have covered. The Kyberforge style is scoped to **/agents/*.md and **/*.agent.md, and KyberforgeCopilot to **/*.agent.md alone — a file outside those globs is silently not linted.
  • E100 Runtime error ... does not exist (exit 2) from vale-wrap.sh. An explicit relative --config was passed. Pass none: the wrapper locates its own assets/vale/.vale.ini from its own path. Do not read this exit code as vale being unavailable.
  • A path argument that does not exist is a hard error in vale-wrap.sh, deliberately: bare vale would fall back to reading stdin and print a clean-looking 0 errors ... in stdin, which the 0 files guard above does not catch.