Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

fix(kyberforge): bridge apm content to Claude Code's flat plugin discovery

Browse Source 
Claude Code's (and Copilot's) native plugin installer has zero awareness of
.apm/ nesting -- it convention-scans only flat skills/, agents/, commands/,
hooks.json at each plugin's root. Confirmed via strings on the installed
claude binary and live installs of git@holocron/gitea@holocron/kyberforge@
holocron, all reporting Skills(0) Agents(0) Hooks(0) post ADR-0015's apm
conversion. Root cause (apm_cli/core/plugin_manifest.py): apm's plugin.json
compiler deliberately strips skills/agents/commands keys, assuming the host
already auto-discovers those convention directories -- it has no model of
.apm/ being host-visible at all. Separately, apm's own bundle exporter
(apm_cli/bundle/plugin_exporter.py, behind `apm pack --format plugin`)
implements the correct .apm/ -> flat mapping, but only ever targeted
build/<name>-<version>/, a path nothing in marketplace.json's source: points
at.

scripts/sync-plugin-content.sh wraps that bundle exporter and copies its
agents/, skills/, commands/, instructions/, extensions/, and merged
hooks.json back into each plugin's own root as a second tracked
compiled-output category -- same governance status as
.claude-plugin/plugin.json: generated from .apm/, never hand-edited. tests/
subdirectories are excluded from the mirror (dev fixtures, not host-visible
runtime content; several hardcode a relative repo-root walk-up sized for the
.apm/-nested depth, which breaks when duplicated one level shallower).
Applied for real across all 6 plugins and verified two ways: `claude plugin
validate --strict` passes on every real plugin directory, and a live
`claude --plugin-dir <path> -p "list skills/agents"` behavioral test
confirms content is now actually discovered.

Also, from the same issue #90 review round:
- scripts/check-manifests.sh pointed at each plugin's root-level plugin.json
  (checking skills/hooks/mcpServers/agents pointer fields) -- that file was a
  stale near-duplicate of .claude-plugin/plugin.json nothing else read or
  wrote, now deleted across all 6 plugins. check-manifests.sh is rewritten to
  validate .claude-plugin/plugin.json instead, and drops the pointer-field
  checks entirely (nothing to check -- those fields are correctly absent by
  design). Content-presence drift is now check-plugin-content-sync's job, a
  new pre-push hook wired in .pre-commit-config.yaml.

docs/adr/0017 records the root cause and decision in full, including two
rejected alternatives (patching plugin.json's path fields directly -- apm's
compiler strips them on every run; pointing marketplace.json at apm pack's
build/ output -- a version-suffixed non-source directory nothing can install
from without an extra build step). ADR-0015 and CONTEXT.md are updated to
point at it.

Refs: #90
This commit is contained in:
Defame1297
2026-08-13 16:59:03 +00:00
parent 7910b8b12c
commit 38f1ba4e03
217 changed files with 14455 additions and 175 deletions
Download Patch File Download Diff File Expand all files Collapse all files

526
plugins/kyberforge/skills/skill-audit/scripts/vale-wrap.sh Executable file
View File

@@ -0,0 +1,526 @@
#!/usr/bin/env bash
set -euo pipefail
# Works around a Vale limitation: the `text.frontmatter.description` NLP scope
# silently stops matching once the `description:` value spans 2+ physical lines
# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block
# scalar (the style used by most skills/agents in this repo), a plain scalar
# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped
# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed
# value keeps exactly the line breaks the source has, and vale matches it fine
# (verified against vale 3.15.2), so literal blocks are deliberately left alone.
# This script flattens an affected description to a one-line scalar in a scratch
# copy — or, for the rare value no inline scalar can spell out verbatim, to a
# `|-` literal block with a single content line, which vale matches just as well
# (padding with blank lines so every other line number is unchanged), then
# runs the real `vale` binary against the copies. Drop-in replacement for calling
# `vale` directly: same args, same exit code, bar the two documented divergences
# below.
#
# "Same args" means relative paths — path arguments and the values of the
# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against
# the caller's current directory, exactly as bare `vale` resolves them. The flag
# values are rewritten to absolute form because the run ends up `cd`'d into the
# scratch mirror, where a relative one would no longer resolve. (An earlier
# version resolved path arguments against the repo root, an invented convention
# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse,
# silently dropped file arguments that didn't happen to resolve from the repo
# root — skipping the flattening this script exists for.)
#
# Divergence 1: with no `--config` at all, this script's own sibling
# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit
# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in
# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and
# hard-fail (E100) for every external consumer. The manifest therefore passes the
# script alone, and an explicit `--config` from any other caller still wins.
#
# Divergence 2: a path-shaped argument that does not exist is a hard error
# (exit 2). Bare vale drops it, falls back to reading stdin, and prints
# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable
# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather
# than clean, and `in stdin` does not match that guard, so the silent form would
# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way
# to keep that guard honest. Linting prose piped on stdin is therefore
# unsupported here — it already was, since the no-path handoff closes stdin so
# vale can't block on a pipe that will never carry content.
#
# Vale prints each path exactly as it was handed to it, so the scratch tree
# mirrors the caller's absolute cwd: a relative path argument is passed through
# verbatim and resolves to its flattened copy, keeping the report byte-identical
# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that
# property. Only an absolute path outside the cwd is rewritten to its scratch
# copy and so reports a scratch path — unavoidable, since a file can only be
# read from where it actually is.
cwd="$(pwd -P)"
# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 —
# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"`
# on an empty array as an unbound variable under `set -u`. No expansion site is
# reachable while empty on today's control flow, so this is insurance against a
# later edit breaking that invariant, not a live fix.
vale_args=()
path_args=()
pending_flag=""
config_given=false
# `--output` takes either one of vale's built-in style names or a template file
# path. Only the file form needs absolutizing, and the built-in names have to be
# excluded by name *before* the existence test below: a file or directory
# literally called `line` in the caller's cwd would otherwise rewrite the
# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template]
# Runtime error`) where bare vale just uses the built-in. `--path` has no such
# names — it is always a path — so the check is keyed on the flag too.
is_builtin_output() {
case "$2" in
line|JSON|CLI) [[ "$1" == "--output" ]] ;;
*) false ;;
esac
}
# Absolutizes a `--config` value against the caller's cwd. Shared by both
# argument forms below — separated (`--config X`) and joined (`--config=X`)
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
# one place instead of being duplicated per form.
abs_config_value() {
if [[ "$1" == /* ]]; then
printf '%s' "$1"
else
printf '%s' "$cwd/$1"
fi
}
for arg in "$@"; do
if [[ -n "$pending_flag" ]]; then
# Value of a separated two-argv flag. It is never a lint target, however
# file-like it looks. The run ends up `cd`'d into the scratch mirror, so a
# value naming a file has to be absolutized here or it stops resolving.
case "$pending_flag" in
--config)
# Always a path, and required to exist.
vale_args+=("$(abs_config_value "$arg")")
;;
--output|--path)
# See `is_builtin_output` above for why the built-in `--output` names
# are excluded first. Anything that names nothing is passed through and
# left for vale to interpret.
if is_builtin_output "$pending_flag" "$arg"; then
vale_args+=("$arg")
elif [[ "$arg" != /* && -e "$arg" ]]; then
vale_args+=("$cwd/$arg")
else
vale_args+=("$arg")
fi
;;
*)
vale_args+=("$arg")
;;
esac
pending_flag=""
continue
fi
case "$arg" in
--config)
vale_args+=("$arg")
pending_flag="$arg"
config_given=true
continue
;;
--config=*)
vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
config_given=true
continue
;;
# Same cwd-relative resolution for the `--flag=value` spelling of the two
# other path-valued flags.
--output=*|--path=*)
flag_val="${arg#*=}"
if is_builtin_output "${arg%%=*}" "$flag_val"; then
vale_args+=("$arg")
elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then
vale_args+=("${arg%%=*}=$cwd/$flag_val")
else
vale_args+=("$arg")
fi
continue
;;
# Vale's remaining value-taking flags, per `vale --help` (3.x). In the
# separated two-argv form the value must not be classified as a lint target
# — `--output tmpl.tmpl` names a real template file, and treating it as
# input both lints the template and reorders argv so vale sees
# `--output --no-wrap`. The `--flag=value` form needs no entry here: it
# starts with `-` and falls through to vale untouched. A value flag added by
# some future vale release is simply absent from this list and lands back on
# today's behaviour, so this list going stale is never worse than not having
# it.
--ext|--filter|--glob|--minAlertLevel|--output|--path)
vale_args+=("$arg")
pending_flag="$arg"
continue
;;
# Vale's subcommands are bare words that name no file, so they would trip
# the not-found error below. A lint target literally named `sync` (no
# extension, no slash) is misread as the subcommand — accepted, because the
# alternative is failing every `vale-wrap.sh ls-config`.
ls-config|ls-dirs|ls-metrics|ls-vars|sync)
vale_args+=("$arg")
continue
;;
esac
if [[ "$arg" == -* ]]; then
vale_args+=("$arg")
continue
fi
# Everything left is a lint target: `vale [options] [input...]` has no third
# kind of argument. See divergence 2 above for why a missing one is fatal here.
if [[ ! -e "$arg" ]]; then
echo "vale-wrap.sh: no such file or directory: $arg" >&2
exit 2
fi
# An absolute path inside the caller's cwd is relativized so the report cites
# a path that resolves against the real tree. Left absolute, it would be
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
# path to a file that is deleted on exit, which reads as a bug in any report
# quoting it. Absolute paths outside the cwd have no relative form and keep
# the scratch-path behaviour documented above.
if [[ "$arg" == "$cwd"/* ]]; then
path_args+=("${arg#"$cwd"/}")
else
path_args+=("$arg")
fi
done
if [[ "$config_given" == false ]]; then
vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini")
fi
if [[ ${#path_args[@]} -eq 0 ]]; then
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
# block waiting on a pipe that will never carry content.
exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null
fi
# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing)
# is a GNU extension the BSD realpath on macOS doesn't have — and every dest
# below is a path that doesn't exist yet. python3 is already a hard dependency.
abspath() {
python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1"
}
flatten() {
# Two call shapes: `flatten src dest` (dest already resolved and inside the
# scratch tree — the per-markdown-file calls in the directory branch below)
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
# branch further down) additionally resolves `raw_dest` the way a separate
# `abspath` call used to, applies the same sandbox-escape guard, and prints
# the resolved path — folding two python3 spawns per file into one.
python3 - "$@" <<'PYTHON'
import os
import re
import sys
src, dest_input = sys.argv[1], sys.argv[2]
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
if tmpdir is None:
dest = dest_input
else:
dest = os.path.abspath(dest_input)
if not dest.startswith(tmpdir + os.sep):
print(
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
f"land outside {tmpdir}",
file=sys.stderr,
)
sys.exit(2)
os.makedirs(os.path.dirname(dest), exist_ok=True)
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
# a byte-for-byte round trip instead of aborting the whole run on a decode error.
with open(src, encoding='utf-8', errors='surrogateescape') as fh:
content = fh.read()
# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\<newline>` is handled
# separately in unescape_double because it also swallows the next indentation.
DQ_ESCAPES = {
'0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n',
'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"',
'/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028',
'P': '\u2029',
}
# First characters that make a plain (unquoted) scalar mean something other than
# text: YAML's c-indicator set.
PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`'
def unescape_double(text):
"""Decode a double-quoted YAML scalar's body to the string YAML parses."""
out = []
i = 0
while i < len(text):
char = text[i]
if char != '\\':
out.append(char)
i += 1
continue
i += 1
if i >= len(text):
break
esc = text[i]
if esc == '\n':
i += 1
while i < len(text) and text[i] in ' \t':
i += 1
continue
if esc in 'xuU':
width = {'x': 2, 'u': 4, 'U': 8}[esc]
digits = text[i + 1:i + 1 + width]
if len(digits) == width:
try:
out.append(chr(int(digits, 16)))
except ValueError:
pass
else:
i += 1 + width
continue
out.append(DQ_ESCAPES.get(esc, esc))
i += 1
return ''.join(out)
def close_quote(text, quote):
"""Index of the closing `quote` in `text`, which starts just past the
opening one. None while the scalar is still unterminated."""
i = 0
while i < len(text):
char = text[i]
if quote == '"' and char == '\\':
i += 2
continue
if char == quote:
if quote == "'" and text[i + 1:i + 2] == "'":
i += 2
continue
return i
i += 1
return None
def continuation_lines(rest):
"""Yield the physical lines of `rest` that continue the value started on the
`description:` line. Indentation-based and blank-line-tolerant, per YAML:
a blank line (any amount of whitespace) always stays inside; the indent is
set by the first content line; the value ends at the first line indented
less than that, at any line flush with the key (that is the next mapping
key, not a continuation), or at EOF."""
indent = None
for line in rest.splitlines(keepends=True):
text = line.rstrip('\n')
if text.strip() == '':
yield line
continue
line_indent = len(text) - len(text.lstrip(' \t'))
if line_indent == 0:
return
if indent is None:
indent = line_indent
elif line_indent < indent:
return
yield line
def emit(value):
"""Render `value` as a YAML scalar whose source text spells the value out
verbatim. Vale locates the description by matching the parsed value back
against the source, so a scalar carrying any escape — `''` in a
single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the
whole `text.frontmatter.description` scope vanish, the same failure this
script exists to work around. Verbatim forms only, therefore, tried in
descending order of fidelity. The first three occupy one physical line; the
`|-` fallback occupies two, which the caller accounts for when padding."""
if (value
and value[0] not in PLAIN_UNSAFE_FIRST
and ': ' not in value
and not value.endswith(':')
and ' #' not in value):
return value # plain: nothing needs escaping at all
if "'" not in value:
return "'" + value + "'" # single-quoted: only `'` would escape
if '"' not in value and '\\' not in value:
return '"' + value + '"' # double-quoted: only `"`/`\` would
# Last resort: the value needs quoting AND holds an apostrophe AND a double
# quote or backslash, so no *inline* scalar can carry it verbatim. A `|-`
# literal block can — a block scalar's body has no escape syntax at all, so
# `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches
# the description scope against it (the header above says the same of the
# `|` blocks this script deliberately leaves alone; verified against vale
# 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed
# value is exactly `value` with no trailing newline.
return '|-\n ' + value
fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL)
if fm_match:
fm = fm_match.group(2)
header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE)
else:
header_m = None
if header_m:
head_start = header_m.start()
value_start = header_m.end()
header_end = fm.find('\n', value_start)
header_end = len(fm) if header_end == -1 else header_end
first = fm[value_start:header_end]
body_start = header_end + 1
indicator = first.rstrip()
block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator)
if block_m and block_m.group(1) == '|':
kind = None # literal blocks keep their line breaks; vale is fine
elif block_m:
kind = 'block' # folded (`>`): the value starts on the next line
elif indicator == '':
kind = 'block' # bare `description:`: a plain scalar on later lines
elif first[:1] == '"':
kind = 'double'
elif first[:1] == "'":
kind = 'single'
elif first[:1] in '#&*!':
kind = None # comment, anchor, alias or tag — not a plain scalar
else:
kind = 'plain'
text = ''
value_end = value_start
value_lines = 0
if kind in ('block', 'plain'):
body = ''.join(continuation_lines(fm[body_start:]))
value_end = body_start + len(body)
if kind == 'block':
text = body
value_lines = body.count('\n')
else:
text = fm[value_start:value_end]
value_lines = 1 + body.count('\n')
if ' #' in text or text.lstrip().startswith('#'):
# A `#` opens a comment inside a plain scalar. Folding it in
# would lint text YAML never treats as part of the value, so
# leave the file alone rather than lint the wrong string.
kind = None
elif kind in ('double', 'single'):
quote = '"' if kind == 'double' else "'"
inner_start = value_start + 1
acc = fm[inner_start:body_start]
idx = close_quote(acc, quote)
lines = continuation_lines(fm[body_start:])
while idx is None:
try:
acc += next(lines)
except StopIteration:
break
idx = close_quote(acc, quote)
if idx is None:
kind = None # unterminated quote: invalid YAML, leave it to vale
else:
inner = acc[:idx]
value_end = inner_start + idx + 1
text = unescape_double(inner) if quote == '"' else inner.replace("''", "'")
value_lines = 1 + inner.count('\n')
flat = re.sub(r'\s+', ' ', text).strip()
if kind and flat and value_lines >= 2:
# `value_end` can land mid-line, just past a closing quote, so extend to
# the end of that physical line and carry whatever follows (a trailing
# comment) across unchanged.
if value_end > 0 and fm[value_end - 1] == '\n':
span_end = value_end
trailer = ''
else:
newline = fm.find('\n', value_end)
span_end = len(fm) if newline == -1 else newline + 1
trailer = fm[value_end:span_end].rstrip('\n')
scalar = emit(flat)
# A trailing comment carried across from the original line stays on the
# `description:` line itself: after a block scalar's `|-` header it is
# still a comment, but inside the block body it would become part of the
# value.
head, newline_sep, block_body = scalar.partition('\n')
# The replacement displaces the whole span, so the blank-line pad makes
# up the difference between the lines it displaced and the lines it
# occupies — every later line number is unchanged. That is one line for
# the three inline forms and two for the `|-` block; the span itself is
# at least two lines here (`value_lines >= 2` is a precondition), so the
# pad count never goes negative.
pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n'))
new_fm = (fm[:head_start] + 'description: ' + head + trailer
+ newline_sep + block_body + '\n' + pad + fm[span_end:])
content = (fm_match.group(1) + new_fm + fm_match.group(3)
+ content[fm_match.end():])
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
fh.write(content)
if tmpdir is not None:
print(dest)
PYTHON
}
tmpdir="$(cd "$(mktemp -d)" && pwd -P)"
trap 'rm -rf "$tmpdir"' EXIT
# Mirror of the caller's cwd inside the scratch tree; relative path arguments
# are resolved from here.
mirror="$tmpdir$cwd"
mkdir -p "$mirror"
argv_paths=()
for arg in ${path_args[@]+"${path_args[@]}"}; do
if [[ "$arg" == /* ]]; then
raw_dest="$tmpdir$arg"
else
raw_dest="$mirror/$arg"
fi
if [[ -d "$arg" ]]; then
dest="$(abspath "$raw_dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
# A directory is mirrored whole — vale applies its own format filtering to
# the tree, so any file dropped here would be silently unlinted — and then
# every markdown file in the copy is flattened in place. `.git` is pruned:
# vale never lints it and copying it can dwarf the rest of the tree.
# `find -L` follows symlinks because vale does: it lints both a symlinked
# file and a file under a symlinked directory, and a bare `-type f` walk
# would report "0 files" where bare vale reports one. (A symlink loop makes
# `find` warn on stderr and carry on, which is also what vale does.) The
# second walk needs no `-L`: the mirror is all real files by construction.
mkdir -p "$dest"
while IFS= read -r -d '' rel; do
mkdir -p "$dest/$(dirname "$rel")"
cp "$arg/$rel" "$dest/$rel"
done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0)
while IFS= read -r -d '' md; do
flatten "$md" "$md"
done < <(find "$dest" -type f -name '*.md' -print0)
else
# `abspath` + `flatten` folded into one python3 process — see the comment
# atop `flatten` above.
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
fi
if [[ "$arg" == /* ]]; then
argv_paths+=("$dest")
else
argv_paths+=("$arg")
fi
done
cd "$mirror"
vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"}

395
plugins/kyberforge/skills/skill-audit/scripts/validate-provenance.sh Executable file
View File

@@ -0,0 +1,395 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate-provenance.sh <skill-dir>
Validate that a skill's sources provenance chain is complete and internally consistent.
Arguments:
skill-dir Path to the skill directory to validate.
Exit codes:
0 All checks passed (or nothing to validate)
1 One or more checks failed
Checks performed:
0 source_keys present but references/sources.md absent
1 FILL IN: placeholders in sources.md
2 source_keys in SKILL.md → slug exists in sources.md
3 source_keys in references/*.md → slug exists in sources.md (INFO if no source_keys)
4 Contributing files listed in sources.md exist on disk
5 Contributing files back-reference the parent slug in their source_keys
6 Research doc field present and not placeholder
7 Slug in sources.md present in upstream research doc (INFO only)
8 Extracted non-(none) slug in research doc present in sources.md
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: skill-dir is required." >&2
echo "" >&2
usage >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
skill_dir = os.path.abspath(sys.argv[1])
sources_md_path = os.path.join(skill_dir, "references", "sources.md")
refs_dir = os.path.join(skill_dir, "references")
# --- Helpers ---
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
def parse_frontmatter(content):
"""Return (frontmatter_str, body_str) or (None, content) if no frontmatter."""
m = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
if not m:
return None, content
return m.group(1), content[m.end():]
def parse_source_keys(fm):
"""Extract list of source_keys from frontmatter string (handles metadata.source_keys and top-level)."""
if fm is None:
return []
keys = []
# Match either:
# metadata:\n source_keys:\n - slug
# or:
# source_keys:\n - slug
in_source_keys = False
in_metadata = False
for line in fm.splitlines():
if re.match(r'^metadata:', line):
in_metadata = True
continue
if in_metadata and re.match(r'^ source_keys:', line):
in_source_keys = True
continue
if not in_metadata and re.match(r'^source_keys:', line):
in_source_keys = True
continue
if in_source_keys:
m = re.match(r'^[ \t]+-\s+(\S+)', line)
if m:
keys.append(m.group(1).strip())
elif line and not line[0].isspace():
in_source_keys = False
in_metadata = False
return keys
def parse_h2_slugs(content):
"""Return list of H2 heading values from a markdown file."""
return re.findall(r'^## (.+)$', content, re.MULTILINE)
def parse_contributing_files(content, slug):
"""Find the Contributing files value for a given slug H2 in content."""
# Find the H2 block for slug, then look for Contributing files line
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if not cf_m:
return None
return cf_m.group(1).strip()
def parse_research_doc(content, slug):
"""Find the Research doc value for a given slug H2 in content."""
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
rd_m = re.search(r'^\- \*\*Research doc:\*\* (.+)$', block, re.MULTILINE)
if not rd_m:
return None
return rd_m.group(1).strip()
def parse_status(content, slug):
"""Find the Status value for a given slug H2 in content."""
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
st_m = re.search(r'^\- \*\*Status:\*\* (.+)$', block, re.MULTILINE)
if not st_m:
return None
return st_m.group(1).strip()
def find_repo_root(start_dir):
"""Walk up from start_dir until we find a directory containing .git."""
current = start_dir
while True:
if os.path.exists(os.path.join(current, ".git")):
return current
parent = os.path.dirname(current)
if parent == current:
return None
current = parent
findings = []
has_fail = False
def emit_fail(desc, fpath, why, fix):
global has_fail
has_fail = True
findings.append(("FAIL", desc, fpath, why, fix, None))
def emit_info(desc, fpath, note):
findings.append(("INFO", desc, fpath, None, None, note))
def print_findings():
for entry in findings:
kind = entry[0]
desc = entry[1]
fpath = entry[2]
why = entry[3]
fix = entry[4]
note = entry[5]
if kind == "FAIL":
print(f"FAIL {desc} — {fpath}")
print(f" Why: {why}")
print(f" Fix: {fix}")
print()
else:
print(f"INFO {desc} — {fpath}")
print(f" Note: {note}")
print()
# --- Scan for any file with source_keys ---
def file_has_source_keys(fpath):
try:
with open(fpath) as f:
content = f.read()
except Exception:
return False
fm, _ = parse_frontmatter(content)
if fm is None:
return False
return bool(parse_source_keys(fm))
def find_files_with_source_keys():
"""Return list of (relative_path, abs_path) for all skill files with source_keys."""
results = []
for root, dirs, files in os.walk(skill_dir):
# Skip hidden dirs
dirs[:] = [d for d in dirs if not d.startswith('.')]
for fname in files:
if fname.endswith('.md'):
abs_path = os.path.join(root, fname)
if file_has_source_keys(abs_path):
rel = os.path.relpath(abs_path, skill_dir)
results.append((rel, abs_path))
return results
sources_md_exists = os.path.isfile(sources_md_path)
files_with_source_keys = find_files_with_source_keys()
# Early exit: nothing to validate
if not sources_md_exists and not files_with_source_keys:
sys.exit(0)
# Load sources.md if it exists
sources_content = None
if sources_md_exists:
with open(sources_md_path) as f:
sources_content = f.read()
sources_slugs = set(parse_h2_slugs(sources_content))
else:
sources_slugs = set()
# --- Check 0: source_keys without sources.md ---
if not sources_md_exists:
for rel, abs_path in files_with_source_keys:
emit_fail(
f"source_keys declared but references/sources.md is absent",
rel,
"source_keys references research provenance that has no sources index to validate against.",
"Create references/sources.md with an H2 entry for each slug referenced by source_keys."
)
print_findings()
sys.exit(1)
# --- Check 1: FILL IN: placeholders in sources.md ---
for line in sources_content.splitlines():
if PLACEHOLDER_RE.search(line):
emit_fail(
"Unfilled FILL IN: placeholder",
"references/sources.md",
"sources.md contains an unfilled placeholder, meaning provenance is incomplete.",
"Replace all 'FILL IN:' values in references/sources.md with real content."
)
break
# --- Check 2: source_keys in SKILL.md → slug exists in sources.md ---
skill_md_path = os.path.join(skill_dir, "SKILL.md")
if os.path.isfile(skill_md_path):
with open(skill_md_path) as f:
skill_content = f.read()
skill_fm, _ = parse_frontmatter(skill_content)
skill_source_keys = parse_source_keys(skill_fm)
for slug in skill_source_keys:
if slug not in sources_slugs:
emit_fail(
f"source_keys slug '{slug}' not found in sources.md",
"SKILL.md",
f"SKILL.md declares '{slug}' as a source but there is no '## {slug}' heading in references/sources.md.",
f"Add '## {slug}' entry to references/sources.md or remove '{slug}' from SKILL.md source_keys."
)
# --- Check 3: source_keys in references/*.md → slug exists in sources.md (INFO if no source_keys) ---
if os.path.isdir(refs_dir):
for fname in sorted(os.listdir(refs_dir)):
if not fname.endswith('.md'):
continue
if fname == "sources.md":
continue
fpath = os.path.join(refs_dir, fname)
rel = os.path.relpath(fpath, skill_dir)
with open(fpath) as f:
ref_content = f.read()
ref_fm, _ = parse_frontmatter(ref_content)
ref_keys = parse_source_keys(ref_fm)
if not ref_keys:
emit_info(
f"No source_keys frontmatter",
rel,
"This references file has no source_keys — provenance cannot be verified. "
"Add source_keys frontmatter listing the slugs from references/sources.md that informed this file."
)
else:
for slug in ref_keys:
if slug not in sources_slugs:
emit_fail(
f"source_keys slug '{slug}' not found in sources.md",
rel,
f"'{rel}' declares '{slug}' as a source but there is no '## {slug}' heading in references/sources.md.",
f"Add '## {slug}' entry to references/sources.md or remove '{slug}' from {rel} source_keys."
)
# --- Checks 4, 5, 6, 7, 8: Per-slug checks in sources.md ---
repo_root = find_repo_root(skill_dir)
# Collect all research doc paths we'll check (for Check 8)
research_docs_seen = {} # abs_path → set of slugs in sources.md that reference it
for slug in parse_h2_slugs(sources_content):
# Check 4: Contributing files exist
cf_value = parse_contributing_files(sources_content, slug)
if cf_value and not cf_value.startswith("(none"):
# Split by comma
cf_files = [p.strip() for p in cf_value.split(",") if p.strip()]
for cf_rel in cf_files:
cf_abs = os.path.join(skill_dir, cf_rel)
if not os.path.isfile(cf_abs):
emit_fail(
f"Contributing file '{cf_rel}' does not exist",
f"references/sources.md (## {slug})",
f"sources.md claims '{cf_rel}' was contributed to by slug '{slug}' but the file does not exist.",
f"Create '{cf_rel}' relative to the skill directory, or correct the path in sources.md."
)
else:
# Check 5: Bidirectional — file should list slug in its source_keys
# Skip sources.md itself
if cf_rel == "references/sources.md":
continue
with open(cf_abs) as f:
cf_content = f.read()
cf_fm, _ = parse_frontmatter(cf_content)
cf_keys = parse_source_keys(cf_fm)
if slug not in cf_keys:
emit_fail(
f"Contributing file '{cf_rel}' does not list '{slug}' in its source_keys",
f"references/sources.md (## {slug})",
f"sources.md says '{cf_rel}' was informed by '{slug}', but '{cf_rel}' does not declare '{slug}' in its source_keys frontmatter.",
f"Add '{slug}' to the source_keys frontmatter of '{cf_rel}'."
)
# Check 6: Research doc field required
rd_value = parse_research_doc(sources_content, slug)
if rd_value is None:
emit_fail(
f"Research doc field missing",
f"references/sources.md (## {slug})",
f"The '## {slug}' entry in sources.md has no '- **Research doc:**' line.",
f"Add '- **Research doc:** <path-or-(none)>' to the '## {slug}' entry in references/sources.md."
)
elif rd_value == "" or PLACEHOLDER_RE.search(rd_value):
emit_fail(
f"Research doc field is empty or placeholder",
f"references/sources.md (## {slug})",
f"The '## {slug}' entry has an unfilled Research doc value.",
f"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable."
)
else:
# Check 7: Upstream forward — slug should appear in research doc
if repo_root and not rd_value.startswith("(none"):
rd_abs = os.path.join(repo_root, rd_value)
if os.path.isfile(rd_abs):
with open(rd_abs) as f:
rd_content = f.read()
rd_slugs = set(parse_h2_slugs(rd_content))
if slug not in rd_slugs:
emit_info(
f"Slug '{slug}' not found as H2 in research doc '{rd_value}'",
f"references/sources.md (## {slug})",
f"The research doc '{rd_value}' does not have a '## {slug}' heading. "
f"The provenance link may be imprecise — the slug name in sources.md may differ from the research doc's heading."
)
# Track for Check 8
if rd_abs not in research_docs_seen:
research_docs_seen[rd_abs] = (rd_value, set())
research_docs_seen[rd_abs][1].add(slug)
# --- Check 8: Upstream reverse ---
for rd_abs, (rd_rel, known_slugs) in research_docs_seen.items():
with open(rd_abs) as f:
rd_content = f.read()
for rd_slug in parse_h2_slugs(rd_content):
# Parse this slug's Contributing files and Status in the research doc
rd_cf = parse_contributing_files(rd_content, rd_slug)
rd_status = parse_status(rd_content, rd_slug)
# Skip if contributing files start with (none
if rd_cf and rd_cf.startswith("(none"):
continue
# Skip if status is not `extracted`
if rd_status != "`extracted`":
continue
# This slug should be in sources.md
if rd_slug not in sources_slugs:
emit_fail(
f"Research doc slug '{rd_slug}' missing from skill sources.md",
f"references/sources.md",
f"The research doc '{rd_rel}' has '## {rd_slug}' with status `extracted` and contributing files, "
f"but this skill's sources.md has no '## {rd_slug}' entry.",
f"Add '## {rd_slug}' to references/sources.md or mark it as '(none)' in the research doc's Contributing files."
)
print_findings()
sys.exit(1 if has_fail else 0)
PYTHON

199
plugins/kyberforge/skills/skill-audit/scripts/validate.sh Executable file
View File

@@ -0,0 +1,199 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate.sh <skill-dir>
Validate a skill directory against the agentskills.io specification.
Arguments:
skill-dir Path to the skill directory containing SKILL.md.
Exit codes:
0 All checks passed
1 One or more checks failed
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: skill-dir is required." >&2
echo "" >&2
usage >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
skill_dir = os.path.abspath(sys.argv[1])
skill_md = os.path.join(skill_dir, "SKILL.md")
if not os.path.isfile(skill_md):
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
sys.exit(1)
with open(skill_md) as f:
content = f.read()
failed = False
def ok(msg):
print(f"PASS {msg}")
def fail(msg):
global failed
print(f"FAIL {msg}")
failed = True
# --- Parse frontmatter ---
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
if not fm_match:
fail("No valid YAML frontmatter block found (expected ---...---)")
sys.exit(1)
fm = fm_match.group(1)
body_start = fm_match.end()
# Extract name
name_m = re.search(r'^name:\s*(\S+)', fm, re.MULTILINE)
name = name_m.group(1).strip('"\'') if name_m else ""
# Extract description — inline or block scalar (> or |)
desc = ""
desc_m = re.search(r'^description:\s*([>|])\n((?:[ \t]+.+\n?)+)', fm, re.MULTILINE)
if desc_m:
raw = desc_m.group(2)
desc = re.sub(r'\s+', ' ', raw).strip()
else:
desc_inline = re.search(r'^description:\s*(.+)', fm, re.MULTILINE)
if desc_inline:
desc = desc_inline.group(1).strip()
dir_name = os.path.basename(skill_dir)
# --- Checks ---
# name present
if name:
ok(f"name present: '{name}'")
else:
fail("name field is missing or empty")
# name matches directory
if name and dir_name:
if name == dir_name:
ok(f"name '{name}' matches directory '{dir_name}'")
else:
fail(f"name '{name}' does not match directory '{dir_name}'")
# name length
if name:
if len(name) <= 64:
ok(f"name length {len(name)} chars (limit: 64)")
else:
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
# name format
if name:
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
ok(f"name format valid (kebab-case)")
else:
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
# description present
if desc:
ok(f"description present")
else:
fail("description field is missing or empty")
# description length
if desc:
dlen = len(desc)
if dlen <= 1024:
ok(f"description length {dlen} chars (limit: 1024)")
else:
fail(f"description length {dlen} chars — exceeds 1024-character limit")
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
# but not backtick-quoted references like `FILL IN:` used in instructions.
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
# description contains unfilled placeholder
if desc and PLACEHOLDER_RE.search(desc):
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
else:
if desc:
ok("description has no unfilled placeholders")
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
# outside the plugin directory once the plugin is cache-installed, so there is
# no single source to share. Keep the two in sync by hand: if they drift, this
# audit will report a skill ready to ship that the commit hook then rejects.
MAX_LINES = 500
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
# for the full measurement.
MAX_WORDS = 2770
line_count = len(content.splitlines())
if line_count <= MAX_LINES:
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
else:
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
# str.split() with no argument splits on runs of whitespace, matching the
# `wc -w` the hook uses, and counts the whole file including frontmatter.
word_count = len(content.split())
if word_count <= MAX_WORDS:
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
else:
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
# Body unfilled placeholders
body = content[body_start:]
fill_matches = PLACEHOLDER_RE.findall(body)
if fill_matches:
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
else:
ok("SKILL.md body has no unfilled placeholders")
# Scripts checks
scripts_dir = os.path.join(skill_dir, "scripts")
if os.path.isdir(scripts_dir):
scripts = [f for f in os.listdir(scripts_dir)
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
for fname in scripts:
fpath = os.path.join(scripts_dir, fname)
with open(fpath) as f:
sc = f.read()
# Interactive prompt heuristic
if re.search(r'^\s*(read\s|input\()', sc, re.MULTILINE):
fail(f"scripts/{fname}: may use interactive input (read/input detected)")
else:
ok(f"scripts/{fname}: no interactive prompts detected")
# Executable bit
if os.access(fpath, os.X_OK):
ok(f"scripts/{fname}: is executable")
else:
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
# Summary
print()
if not failed:
print("All checks passed.")
sys.exit(0)
else:
print("One or more checks failed.")
sys.exit(1)
PYTHON