Defame1297
b6e68e9a2b
Three ways the gates could report green having measured nothing. All three were
invisible to a passing test suite, because pre-commit prints nothing at all for a
hook that exits 0 — a gate that declines to check and a gate that checked and
passed produce the identical signal.
- A UTF-8 BOM, a leading blank line, a trailing space after a `---` marker or
CRLF line endings defeated the `^---\n` frontmatter matcher. Every ADR-0020
check was then skipped and the file passed: measured at the time, a
550-character description with a 1,000-word body exited 0 behind a BOM.
All four shapes are now tolerated, and frontmatter that genuinely cannot be
parsed is a hard ERROR rather than a silent skip.
- An agent file with a valueless `description:` followed by another key let a
line regex capture the *next* key, which looked non-empty, so the
missing-or-empty branch never fired and every gate below it early-returned on
the empty folded value — zero output, exit 0, on a blocking gate. The one
field this contract is entirely about was the one field a gate could fail to
notice was absent. Presence is now decided on the YAML-folded value and
nowhere else, and a missing or empty description is a hard FAIL in all three
validators.
- The hand-rolled frontmatter fallback disagreed with PyYAML across the FAIL
boundary on folded scalars, so which reader happened to be available decided
the verdict. A fallback that mis-parses a scalar shape reports a vacuous pass,
which is worse than not running, so it is deleted: python3 and PyYAML are hard
requirements that fail loudly with an install pointer.
Boundary-target resolution no longer derives its universe from its own location.
A `${BASH_SOURCE}`-relative repo root leaked this repo's 39-skill universe into
every consumer repo running the hook through pre-commit, so a consumer skill
routing to `skill-audit` resolved against a plugin it had never installed. The
interim form resolved through `.claude/` and `.agents/`, which are gitignored
`apm install` output — the same commit reported 2 dangling targets on a machine
that had run the install and 6 on a fresh clone. Resolution now walks up from the
file being checked to an authoring root (nearest ancestor holding
`plugins/*/.apm/{skills,agents}`, else the nearest `.git`, in two passes so a
nested `.git` cannot outrank a real monorepo root); the universe is every skill
and agent under `<root>/plugins/*/` plus the file's own apm package and that
package's declared `dependencies.apm`. Deployed trees are consulted only when no
authoring root exists at all — the consumer case. One commit now gets one verdict,
which a gate shipping hot with no baseline file has to.
Narrowed in the same pass: a routing target inferred from the prose boundary form
and corroborated by nothing else reports at SUGGESTION instead of blocking. A
blocking check with no escape hatch is the wrong trade when the inference from
prose is the weak part of it.
New deterministic checks, all previously untested or absent: every
`references/<file>.md` a body names must exist (ERROR — a broken pointer is not a
style opinion); a description with no boundary clause at all, a Gotchas section
over five entries, and a Gotchas section over 25% of the body are SUGGESTIONs.
Where no universe can be determined the target check prints `INFO ... DID NOT
RUN` rather than passing quietly. Each prose-scanning check needed its own
false-positive fix — a fenced example of a Gotchas section was being read as the
section itself — and those fixes are pinned rather than assumed.
The resolver is one block copied verbatim into all three scripts between
BEGIN/END markers, because a cache-installed plugin's scripts cannot read outside
their own plugin directory. Nothing asserted the copies were still identical; a
one-line edit to a single copy passed every constant-agreement assertion, since
constants are not what drifts.
Tests land here rather than in a later commit. The existing suites assert the old
behaviour and go red against these scripts, so splitting them would leave a commit
whose own `run-tests` pre-push gate fails in isolation.
Refs: ADR-0020
1124 lines
51 KiB
Bash
Executable File
1124 lines
51 KiB
Bash
Executable File
#!/usr/bin/env bash
|
||
set -euo pipefail
|
||
|
||
usage() {
|
||
cat <<EOF
|
||
Usage: validate.sh <skill-dir>
|
||
|
||
Validate a skill directory against the agentskills.io specification.
|
||
|
||
Arguments:
|
||
skill-dir Path to the skill directory containing SKILL.md.
|
||
|
||
Exit codes:
|
||
0 All checks passed (may include SUGGESTIONs)
|
||
1 One or more checks failed
|
||
EOF
|
||
}
|
||
|
||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||
usage
|
||
exit 0
|
||
fi
|
||
|
||
if [[ $# -lt 1 ]]; then
|
||
echo "Error: skill-dir is required." >&2
|
||
echo "" >&2
|
||
usage >&2
|
||
exit 1
|
||
fi
|
||
|
||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||
# boundary — same description, two verdicts, depending on which reader ran.
|
||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||
# dependencies are declared the same way.
|
||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||
# dependency sends the reader to install the wrong thing.
|
||
if ! command -v python3 > /dev/null 2>&1; then
|
||
echo "Error: python3 is required but was not found on PATH." >&2
|
||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||
exit 1
|
||
fi
|
||
|
||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||
exit 1
|
||
fi
|
||
|
||
python3 -u - "$1" <<'PYTHON'
|
||
import sys
|
||
import os
|
||
import re
|
||
import glob
|
||
|
||
import yaml
|
||
|
||
skill_dir = os.path.abspath(sys.argv[1])
|
||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||
|
||
if not os.path.isfile(skill_md):
|
||
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
|
||
sys.exit(1)
|
||
|
||
failed = False
|
||
suggestions = []
|
||
|
||
def ok(msg):
|
||
print(f"PASS {msg}")
|
||
|
||
def fail(msg):
|
||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
|
||
# Both repo callers capture `2>&1`, so nothing a human reads moves.
|
||
global failed
|
||
print(f"FAIL {msg}", file=sys.stderr)
|
||
failed = True
|
||
|
||
def suggest(msg):
|
||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
|
||
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||
# rather than another silently-ignored warning (ADR-0013).
|
||
suggestions.append(msg)
|
||
|
||
def info(msg):
|
||
# A check that DECLINED to run says so out loud, rather than passing
|
||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||
print(f"INFO {msg}")
|
||
|
||
|
||
# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER =====
|
||
# ONE resolver, embedded VERBATIM in three scripts:
|
||
# scripts/skill-size-check.sh
|
||
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh
|
||
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh
|
||
# The block between these markers must stay byte-identical in all three. It is
|
||
# copied rather than imported because a cache-installed plugin's scripts cannot
|
||
# read files outside their own plugin directory, so there is no single file all
|
||
# three can share (same constraint that forces the ADR-0020 constants to be
|
||
# duplicated). Edit one copy, then paste it over the other two.
|
||
#
|
||
# Requires: glob, os, re, yaml (imported by the host script; PyYAML is a hard
|
||
# dependency, preflighted in bash before the interpreter starts).
|
||
|
||
# --- Input ----------------------------------------------------------------
|
||
# Every file this resolver's callers read goes through read_text(), which pins
|
||
# UTF-8 explicitly instead of inheriting locale.getpreferredencoding(). Under
|
||
# LC_ALL=C that inherited encoding is ASCII, so a perfectly ordinary em dash in
|
||
# a SKILL.md aborted the run with a bare UnicodeDecodeError traceback — loud,
|
||
# but pointing at the interpreter rather than at the file or the fix. A file
|
||
# that genuinely is not UTF-8 still fails; it just says so.
|
||
|
||
|
||
class EncodingError(Exception):
|
||
pass
|
||
|
||
|
||
def read_text(path):
|
||
"""File contents as text, UTF-8, with a diagnostic instead of a traceback."""
|
||
try:
|
||
with open(path, encoding='utf-8') as fh:
|
||
return fh.read()
|
||
except UnicodeDecodeError as exc:
|
||
raise EncodingError(
|
||
"not valid UTF-8 (%s at byte %d) — re-save the file as UTF-8; "
|
||
"this gate does not guess at other encodings"
|
||
% (exc.reason, exc.start))
|
||
|
||
|
||
# --- Universe ------------------------------------------------------------
|
||
# The set of names a boundary clause may resolve against is derived from an
|
||
# AUTHORING ROOT found by walking up FROM THE TARGET FILE. It is NEVER derived
|
||
# from this script's own location: deriving it from ${BASH_SOURCE} leaked
|
||
# holocron's 39-skill universe into every consumer repo that ran this hook
|
||
# through pre-commit, so a consumer skill routing to `skill-audit` resolved
|
||
# against a plugin it had never installed.
|
||
#
|
||
# An authoring root is the nearest ancestor holding plugins/*/.apm/skills/ or
|
||
# plugins/*/.apm/agents/ (a plugin monorepo), falling back to the nearest
|
||
# ancestor holding .git. When one is found the universe is:
|
||
# 1. every skill and agent under <root>/plugins/*/ — sibling plugins resolve,
|
||
# which is what a monorepo means,
|
||
# 2. the target's own apm package,
|
||
# 3. the packages that package DECLARES in apm.yml dependencies.apm.
|
||
# Deployed .claude/ and .agents/ trees are deliberately NOT consulted in that
|
||
# case. They are `apm install` output, gitignored, and present only on a machine
|
||
# that has run it: four cross-plugin targets in this repo (gitea-branches ->
|
||
# git-branches, gitea-branches -> git-history, gitea-issues -> git-branches,
|
||
# gitea-workflow -> git-workflow) resolved through .claude/skills/ alone, so the
|
||
# same commit measured 2 dangling targets on a developer machine and 6 on a
|
||
# fresh clone. A gate shipping hot with no baseline cannot give two answers.
|
||
#
|
||
# Deployed trees are used only when NO authoring root exists — the consumer
|
||
# case, where the file being checked lives in or beside a deployed tree and
|
||
# there is no monorepo to read.
|
||
|
||
|
||
def _is_fs_root(path):
|
||
return os.path.dirname(path) == path
|
||
|
||
|
||
def _collect_package(pkg_dir, names):
|
||
"""Add every skill/agent name a package directory exposes, any layout."""
|
||
for sub in ('.apm/skills/*/', 'skills/*/'):
|
||
for path in glob.glob(os.path.join(pkg_dir, sub)):
|
||
names.add(os.path.basename(path.rstrip('/')).lower())
|
||
for sub in ('.apm/agents/*.md', 'agents/*.md'):
|
||
for path in glob.glob(os.path.join(pkg_dir, sub)):
|
||
base = os.path.basename(path)
|
||
if base.endswith('.agent.md'):
|
||
base = base[:-len('.agent.md')]
|
||
else:
|
||
base = base[:-len('.md')]
|
||
names.add(base.lower())
|
||
|
||
|
||
def _apm_package_root(start_dir):
|
||
"""Nearest ancestor that is an apm package root (apm.yml or .apm/).
|
||
|
||
The filesystem root is never a candidate: a stray /.apm/skills/ — a
|
||
scaffolding test's leftover, say, and one really does exist on at least one
|
||
machine here — would otherwise become the package root of every path on it.
|
||
Capped at ten levels so a pathological path can't become a filesystem
|
||
crawl; that covers every real layout by a wide margin.
|
||
"""
|
||
current = os.path.abspath(start_dir)
|
||
for _ in range(10):
|
||
if _is_fs_root(current):
|
||
return None
|
||
if (os.path.isfile(os.path.join(current, 'apm.yml'))
|
||
or os.path.isdir(os.path.join(current, '.apm'))):
|
||
return current
|
||
current = os.path.dirname(current)
|
||
return None
|
||
|
||
|
||
def _authoring_root(start_dir):
|
||
"""Nearest ancestor that is a plugin monorepo, else the nearest .git tree.
|
||
|
||
Two passes, not one interleaved walk: a nested .git (a submodule, a
|
||
worktree of a sub-package) must not win over a real plugins/ root further
|
||
up. Both passes stop before the filesystem root for the same reason
|
||
_apm_package_root does.
|
||
"""
|
||
for probe in (
|
||
lambda d: bool(glob.glob(os.path.join(d, 'plugins', '*', '.apm', 'skills'))
|
||
or glob.glob(os.path.join(d, 'plugins', '*', '.apm', 'agents'))),
|
||
lambda d: os.path.exists(os.path.join(d, '.git'))):
|
||
current = os.path.abspath(start_dir)
|
||
for _ in range(12):
|
||
if _is_fs_root(current):
|
||
break
|
||
if probe(current):
|
||
return current
|
||
current = os.path.dirname(current)
|
||
return None
|
||
|
||
|
||
def _collect_authoring_root(root, names):
|
||
"""Every plugin in the monorepo contributes its names."""
|
||
for pkg in glob.glob(os.path.join(root, 'plugins', '*')):
|
||
if os.path.isdir(pkg):
|
||
_collect_package(pkg, names)
|
||
|
||
|
||
def _declared_dependency_dirs(pkg_dir):
|
||
"""Directories of the apm packages pkg_dir's manifest DECLARES.
|
||
|
||
Reads dependencies.apm and resolves each entry to a directory on disk:
|
||
a monorepo-relative `path:` (against the package root and the nearest
|
||
ancestor manifest, which is the monorepo root) or an installed
|
||
apm_modules/<name>/. Entries that resolve to nothing are skipped — an
|
||
undeployed dependency contributes no names rather than an error.
|
||
"""
|
||
manifest = os.path.join(pkg_dir, 'apm.yml')
|
||
if not os.path.isfile(manifest):
|
||
return []
|
||
try:
|
||
data = yaml.safe_load(read_text(manifest)) or {}
|
||
except Exception:
|
||
return []
|
||
if not isinstance(data, dict):
|
||
return []
|
||
deps = data.get('dependencies')
|
||
deps = deps.get('apm') if isinstance(deps, dict) else None
|
||
if not isinstance(deps, list):
|
||
return []
|
||
|
||
roots = [pkg_dir]
|
||
ancestor = os.path.dirname(os.path.abspath(pkg_dir))
|
||
for _ in range(10):
|
||
if _is_fs_root(ancestor):
|
||
break
|
||
if os.path.isfile(os.path.join(ancestor, 'apm.yml')):
|
||
roots.append(ancestor)
|
||
break
|
||
ancestor = os.path.dirname(ancestor)
|
||
|
||
found = []
|
||
for entry in deps:
|
||
candidates = []
|
||
if isinstance(entry, dict):
|
||
rel = entry.get('path')
|
||
name = entry.get('name')
|
||
if not name and rel:
|
||
name = os.path.basename(str(rel).rstrip('/'))
|
||
if rel:
|
||
candidates.extend(os.path.join(r, str(rel)) for r in roots)
|
||
if name:
|
||
candidates.append(os.path.join(pkg_dir, 'apm_modules', str(name)))
|
||
elif isinstance(entry, str):
|
||
name = re.split(r'[#@]', entry)[0].strip().rstrip('/').split('/')[-1]
|
||
if name:
|
||
candidates.append(os.path.join(pkg_dir, 'apm_modules', name))
|
||
candidates.extend(os.path.join(r, 'plugins', name) for r in roots)
|
||
for candidate in candidates:
|
||
if os.path.isdir(candidate):
|
||
found.append(candidate)
|
||
return found
|
||
|
||
|
||
def _deployed_roots(start_dir):
|
||
""".claude/ and .agents/ trees above start_dir — what a host really sees.
|
||
|
||
Consulted ONLY when no authoring root exists; see the section header. The
|
||
filesystem root is skipped for the same reason _apm_package_root skips it:
|
||
a stray /.claude/skills/ must not join every path's universe.
|
||
"""
|
||
found = []
|
||
current = os.path.abspath(start_dir)
|
||
for _ in range(10):
|
||
if _is_fs_root(current):
|
||
break
|
||
for name in ('.claude', '.agents'):
|
||
base = os.path.join(current, name)
|
||
if os.path.isdir(base):
|
||
found.append(base)
|
||
current = os.path.dirname(current)
|
||
return found
|
||
|
||
|
||
def known_targets(start_dir):
|
||
"""Every skill/agent name a boundary clause in start_dir may name."""
|
||
names = set()
|
||
start = os.path.abspath(start_dir)
|
||
|
||
# Siblings: a cache-installed plugin and a deployed .claude/skills/ tree
|
||
# both put peers one level up, with no plugins/ directory above them. The
|
||
# grandparent is guarded against the filesystem root exactly like the two
|
||
# walk-up loops above — for a start dir of /skills/<x> the grandparent is
|
||
# `/`, and collecting there picks up this machine's stray /.apm/skills/.
|
||
parent = os.path.dirname(start)
|
||
grandparent = os.path.dirname(parent)
|
||
if (os.path.basename(parent) in ('skills', 'agents')
|
||
and os.path.isdir(parent) and not _is_fs_root(grandparent)):
|
||
_collect_package(grandparent, names)
|
||
|
||
package = _apm_package_root(start)
|
||
if package:
|
||
_collect_package(package, names)
|
||
for dep_dir in _declared_dependency_dirs(package):
|
||
_collect_package(dep_dir, names)
|
||
|
||
root = _authoring_root(start)
|
||
if root:
|
||
_collect_authoring_root(root, names)
|
||
else:
|
||
for base in _deployed_roots(start):
|
||
_collect_package(base, names)
|
||
return names
|
||
|
||
|
||
# --- Extraction -----------------------------------------------------------
|
||
# False positives are the design constraint here, not recall. The rules:
|
||
# * A BARE target must be hyphenated AND sit in a boundary sentence (one
|
||
# carrying "do not"/"instead"/"rather than"/"not for"). Without the second
|
||
# condition, pc-run's "run pre-commit hooks" reads as a route to a
|
||
# non-existent `pre-commit` skill.
|
||
# * A BARE arrow target counts only in ADR-0020's compressed boundary form,
|
||
# `Not <thing> -> <skill-name>`. Without that, diagnose's process chain
|
||
# "fix -> regression-test" reads as a route to `regression-test`.
|
||
# * A backticked hyphenated token counts only inside a boundary sentence.
|
||
# Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard
|
||
# FAIL with no escape hatch. Gating it costs nothing (measured over this
|
||
# corpus: 54 targets before and after); DELETING it costs 7 real targets
|
||
# across three gitea skills, so it is gated, not removed.
|
||
# * SINGLE-WORD targets are deliberately NOT matchable bare — `research`,
|
||
# `triage`, `forge`, `prototype` and `tdd` are all real skill names and all
|
||
# ordinary English, so a bare-word rule would flag most of the corpus. A
|
||
# single-word target must be written `` `forge` `` or /forge to be seen.
|
||
# That is a known recall limitation, accepted over the false positives.
|
||
# Tool names (Read/Write/Edit) are excluded by the lowercase-only pattern; MCP
|
||
# tool names (issue_write) by its rejection of underscores; file names by its
|
||
# rejection of dots and slashes.
|
||
#
|
||
# ATTRIBUTIVE USE. The boundary-sentence gate above does NOT solve the
|
||
# `pre-commit` false positive, and the comment that claimed it did was wrong:
|
||
# "instead", "rather than", "do not" and "not for" are exactly the words a
|
||
# boundary clause uses, so the gate is open precisely where the risk is. All of
|
||
# these were hard dangling FAILs with no suppression:
|
||
# Use pre-commit hooks instead of ad-hoc scripts.
|
||
# Invoke the pull-request template instead of writing one by hand.
|
||
# Use conventional-commits formatting rather than free-form messages.
|
||
# Composes label-resolution logic instead of duplicating it.
|
||
# Do not use for X — run the `pre-push` hooks instead.
|
||
# What separates every one of them from a real route is grammar, not marking:
|
||
# the hyphenated token is a compound MODIFIER of the noun that follows it
|
||
# ("pre-commit hooks", "pull-request template"), where a route target is
|
||
# terminal — followed by punctuation, a conjunction, or a boundary word. So a
|
||
# target whose next token is an ordinary lowercase noun is CONFIRM-ONLY: it
|
||
# still resolves and still counts as a route when the name exists, but it can
|
||
# never raise a dangling error on its own.
|
||
#
|
||
# This is deliberately NOT the simpler "only marked targets may dangle" rule,
|
||
# which would have been wrong here: BOTH live true positives in this corpus are
|
||
# BARE — research's "(use neuledge-context)" and gitea-issues' "Composes
|
||
# gitea-labels-\n milestones", where the `>` fold yields "gitea-labels-
|
||
# milestones" and the trailing hyphen is what keeps it terminal. Marking is a
|
||
# poor proxy, so the follower token is the signal, and it is applied to
|
||
# backticked targets too.
|
||
#
|
||
# TERMINAL IS NOT ENOUGH — IN-SENTENCE CORROBORATION. The follower test clears
|
||
# `pre-push` in the example above only because that example happens to be
|
||
# followed by the noun "hooks". Move the same token into terminal position and
|
||
# it was a hard FAIL again, with no suppression mechanism anywhere in this gate:
|
||
# Do not use for running hooks — run `pre-commit` instead.
|
||
# Do not use for the commit message — see `commit-msg`.
|
||
# Do not use for type errors — run `type-check` first.
|
||
# Instead, use `semantic-release`.
|
||
# Do not use for the old flow — use the clean-up instead.
|
||
# Do not run end-to-end, run unit-tests.
|
||
# Every one of those is grammatically identical to a genuinely broken route:
|
||
# "route verb + hyphenated name + terminal" is also exactly how prose cites a
|
||
# tool, a hook, a file format or an English compound. Nothing local separates
|
||
# them, and the skills most exposed are the ones this contract sends authors
|
||
# back to rewrite first — pc-run, pc-author, vale-run, vale-config and the apm-*
|
||
# family are all ABOUT hyphenated tools.
|
||
#
|
||
# So the confidence to BLOCK a commit comes from the sentence, not the token: a
|
||
# prose-form target may raise a hard error only when its own sentence names at
|
||
# least one OTHER target that RESOLVES. A routing sentence proves itself by
|
||
# routing somewhere real; a lone unresolvable name proves nothing. That is not a
|
||
# rule fitted to the fixtures — it is the shape of both live true positives,
|
||
# which sit beside `write-docs` and `gitea-labels-milestones` respectively, and
|
||
# it changes this corpus's verdict by exactly nothing.
|
||
#
|
||
# An uncorroborated unresolvable target is NOT discarded: every caller reports
|
||
# it at its SUGGESTION tier, naming the target. The finding stays visible on
|
||
# every run; only the power to block a commit is withdrawn, which is the part
|
||
# that had no escape hatch.
|
||
#
|
||
# EXPLICIT ROUTE NOTATION is exempt from corroboration and always blocks:
|
||
# ADR-0020's compressed arrow (`Not <thing> -> <name>`) and Claude Code's
|
||
# invocation form (`/<name>`). Neither is ever how English cites a tool — nobody
|
||
# writes `-> pre-commit` or `/pre-commit` to mean the hook — so there is no
|
||
# ambiguity to resolve, and an author who wants a route checked unconditionally
|
||
# has two ways to say so.
|
||
#
|
||
# NAMESPACE: `plugin:skill` is live in this repo (native user-scope installs
|
||
# still resolve `gitea:gitea-prs`), so the patterns admit an optional
|
||
# `<plugin>:` prefix and normalize_target() strips it before resolution.
|
||
NS = r"(?:[a-z0-9]+(?:-[a-z0-9]+)*:)?"
|
||
NAME_ANY = NS + r"[a-z0-9]+(?:-[a-z0-9]+)*"
|
||
NAME_HYPH = NS + r"[a-z0-9]+(?:-[a-z0-9]+)+"
|
||
ROUTE_VERB = (r"(?:use|uses|using|run|runs|invoke|invokes|invoking|try|see"
|
||
r"|that'?s|compose|composes|call|calls"
|
||
r"|routes?\s+to|delegates?\s+to|prefers?|switch(?:es)?\s+to"
|
||
r"|hands?\s+off\s+to)")
|
||
MARKED_TARGET = r"(?:`/?(%s)`|(?<![\w./*-])/(%s)\b)" % (NAME_ANY, NAME_ANY)
|
||
ANY_TARGET = r"(?:%s|(%s)\b)" % (MARKED_TARGET, NAME_HYPH)
|
||
ROUTE_MARKED = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, MARKED_TARGET), re.I)
|
||
ROUTE_ANY = re.compile(r"\b%s\s+(?:the\s+|an?\s+)?%s" % (ROUTE_VERB, ANY_TARGET), re.I)
|
||
CONT_MARKED = re.compile(r"\s*(?:or|and|/|,)\s*%s" % MARKED_TARGET)
|
||
CONT_ANY = re.compile(r"\s*(?:or|and|/|,)\s*%s" % ANY_TARGET)
|
||
ARROW_MARKED = re.compile(r"(?:->|→)\s*%s" % MARKED_TARGET)
|
||
ARROW_BOUNDARY = re.compile(r"\bnot\b[^.;]*?(?:->|→)\s*(%s)\b" % NAME_HYPH, re.I)
|
||
BACKTICK = re.compile(r"`(%s)`" % NAME_HYPH)
|
||
# A boundary clause takes two shapes and BOTH count: the prose markers, and
|
||
# ADR-0020's compressed arrow form `Not <thing> -> <name>`.
|
||
BOUNDARY_MARKER = re.compile(r"\b(?:do\s+not|instead|rather\s+than|not\s+for)\b", re.I)
|
||
BOUNDARY_ARROW = re.compile(r"\bnot\b[^.;]*?(?:->|→)", re.I)
|
||
SENTENCE_SPLIT = re.compile(u'(?<=[.!?])\\s+(?=[A-Z"“(])')
|
||
|
||
# The token that may follow a route target without turning it into a compound
|
||
# modifier: punctuation, end of sentence, a conjunction, a boundary word, or a
|
||
# head noun that names the artifact itself ("the git-workflow skill"). Anything
|
||
# else — `hooks`, `template`, `formatting`, `logic` — is attributive prose.
|
||
FOLLOWER = re.compile(r"[`\s]*([a-z][a-z0-9]*)")
|
||
FOLLOWER_OK = frozenset("""
|
||
and or nor but for to when if unless while after before with from in on at by
|
||
of as than then instead rather directly first only always never also even
|
||
both either neither so because since per via plus alone here there this that
|
||
these those it its they them is are was were be been being has have had will
|
||
would can could should must may might does do did
|
||
skill skills agent agents plugin plugins command commands
|
||
""".split())
|
||
|
||
|
||
def normalize_target(target):
|
||
"""Comparison key: namespace stripped, lowercased.
|
||
|
||
Extraction is case-insensitive (re.I) but the universe is built from
|
||
lowercase directory names, so `Git-Commits` at the start of a sentence
|
||
resolved to nothing until this normalization existed.
|
||
"""
|
||
return target.split(':')[-1].lower()
|
||
|
||
|
||
def has_boundary_clause(description):
|
||
return bool(BOUNDARY_MARKER.search(description)
|
||
or BOUNDARY_ARROW.search(description))
|
||
|
||
|
||
def _first(match):
|
||
"""(name, start, end) offsets for the first group that matched."""
|
||
for index in range(1, (match.re.groups or 0) + 1):
|
||
if match.group(index):
|
||
return match.group(index), match.start(index), match.end(index)
|
||
return None, None, None
|
||
|
||
|
||
def _terminal(text, pos):
|
||
"""True if the token at pos does not make the preceding name a modifier."""
|
||
follower = FOLLOWER.match(text, pos)
|
||
return not follower or follower.group(1) in FOLLOWER_OK
|
||
|
||
|
||
def _notation(text, start, arrow):
|
||
"""True if the name is written in route NOTATION rather than in prose.
|
||
|
||
Two forms qualify: `/name` (Claude Code's invocation syntax, detected from
|
||
the character before the name) and `-> name` (ADR-0020's compressed boundary
|
||
form, passed in by the caller that matched the arrow). A backticked name
|
||
does NOT qualify — a code span is how a tool, a file and a skill are all
|
||
cited, so it carries no intent the follower test hasn't already read.
|
||
"""
|
||
return arrow or (start > 0 and text[start - 1] == '/')
|
||
|
||
|
||
def _add(out, text, name, start, end, strict=None, arrow=False):
|
||
if not name:
|
||
return
|
||
out.append((name,
|
||
_terminal(text, end) if strict is None else strict,
|
||
_notation(text, start, arrow)))
|
||
|
||
|
||
def _scan(text, route_re, cont_re, out):
|
||
for match in route_re.finditer(text):
|
||
name, start, end = _first(match)
|
||
if not name:
|
||
continue
|
||
_add(out, text, name, start, end)
|
||
# "use git-history or git-branches instead" / "use gitea-issues /
|
||
# gitea-prs" — keep consuming conjoined targets after the first.
|
||
pos = match.end()
|
||
while True:
|
||
cont = cont_re.match(text, pos)
|
||
if not cont:
|
||
break
|
||
_add(out, text, *_first(cont))
|
||
pos = cont.end()
|
||
|
||
|
||
def _extract_sentence(sentence):
|
||
"""[(name, may_dangle, notation)] for the routing targets in ONE sentence.
|
||
|
||
Kept separate from _extract() because corroboration is scoped to a single
|
||
sentence: a target's evidence is what stands beside it, not what the rest of
|
||
the description happens to mention.
|
||
"""
|
||
out = []
|
||
boundary = bool(BOUNDARY_MARKER.search(sentence))
|
||
_scan(sentence,
|
||
ROUTE_ANY if boundary else ROUTE_MARKED,
|
||
CONT_ANY if boundary else CONT_MARKED,
|
||
out)
|
||
for match in ARROW_MARKED.finditer(sentence):
|
||
# `-> name` and `-> /name` are route notation, not prose: nothing
|
||
# reads as a compound modifier after an arrow, so no follower test.
|
||
_add(out, sentence, *_first(match), strict=True, arrow=True)
|
||
for match in ARROW_BOUNDARY.finditer(sentence):
|
||
_add(out, sentence, match.group(1), match.start(1), match.end(1),
|
||
strict=True, arrow=True)
|
||
if boundary:
|
||
for match in BACKTICK.finditer(sentence):
|
||
_add(out, sentence, match.group(1), match.start(1), match.end(1))
|
||
return out
|
||
|
||
|
||
def _extract(description):
|
||
"""[(name, may_dangle, notation)] for every routing target."""
|
||
out = []
|
||
for sentence in SENTENCE_SPLIT.split(description):
|
||
out.extend(_extract_sentence(sentence))
|
||
return out
|
||
|
||
|
||
def boundary_targets(description):
|
||
"""Every routing target, for reporting and for confirming a route."""
|
||
return sorted({name for name, _, _ in _extract(description)})
|
||
|
||
|
||
def unresolved_targets(description, known):
|
||
"""Targets resolving to nothing, split into (blocking, reported).
|
||
|
||
`blocking` earns a hard error; `reported` is SUGGESTION tier — named on
|
||
every run, never fatal. Three conditions gate the promotion, and all of them
|
||
are documented at length in the ATTRIBUTIVE USE and CORROBORATION notes
|
||
above:
|
||
|
||
1. the target must be terminal, not a compound modifier ("pre-commit
|
||
hooks" is prose about a tool, not a route),
|
||
2. it must be written in route notation (`/name`, `-> name`), OR
|
||
3. its own sentence must name another target that DOES resolve.
|
||
|
||
Everything else is reported and left alone. `known` is the resolved
|
||
universe from known_targets(); passing an empty set is not meaningful —
|
||
callers check for that first and decline out loud instead.
|
||
"""
|
||
blocking, reported = set(), set()
|
||
for sentence in SENTENCE_SPLIT.split(description):
|
||
found = _extract_sentence(sentence)
|
||
resolved = {normalize_target(name) for name, _, _ in found
|
||
if normalize_target(name) in known}
|
||
for name, may_dangle, notation in found:
|
||
key = normalize_target(name)
|
||
if key in known or not may_dangle:
|
||
continue
|
||
if notation or (resolved - {key}):
|
||
blocking.add(name)
|
||
else:
|
||
reported.add(name)
|
||
return sorted(blocking), sorted(reported - blocking)
|
||
|
||
# --- Frontmatter ----------------------------------------------------------
|
||
# Tolerant on the way in, HARD-FAILING on the way out. A UTF-8 BOM, a leading
|
||
# blank line, trailing whitespace after either `---`, or CRLF line endings all
|
||
# defeated the old `^---\n(.*?)\n---`, and the miss was SILENT: every ADR-0020
|
||
# check was skipped and the file reported green (measured: a 550-character
|
||
# description with a 1,000-word body exited 0 behind a BOM). A file that cannot
|
||
# be measured must never report green, so every caller of these two ERRORs on a
|
||
# miss instead of moving on.
|
||
FRONTMATTER_RE = re.compile(
|
||
r'^[ \t\r\n]*---[ \t]*\r?\n(.*?)\r?\n[ \t]*---[ \t]*(?:\r?\n|\Z)', re.DOTALL)
|
||
|
||
|
||
def strip_bom(text):
|
||
return text[1:] if text.startswith(u'') else text
|
||
|
||
|
||
class FrontmatterError(Exception):
|
||
pass
|
||
|
||
|
||
def description_value(fm_text):
|
||
"""The description VALUE, with YAML folding resolved.
|
||
|
||
PyYAML is a HARD requirement, preflighted in bash. The hand-rolled fallback
|
||
this replaced diverged from a real parser across the FAIL boundary — one
|
||
corpus description measured 270 characters parsed and 412 unparsed, and a
|
||
quoted `"description"` key or an explicit `description: null` returned empty
|
||
from it, silently skipping the description AND routing checks. A gate that
|
||
disagrees with itself depending on which reader ran is worse than no gate.
|
||
|
||
This is the ONLY reader any of the three scripts may use to decide whether a
|
||
description is present. A line regex cannot: `description:` with no value
|
||
followed by `model: sonnet` lets `\\s*` cross the newline and captures the
|
||
NEXT key, which reads as a non-empty description, skips the "missing or
|
||
empty" failure, and then early-returns out of every ADR-0020 gate on the
|
||
genuinely empty folded value. That combination exited 0 with zero output on
|
||
a BLOCKING pre-push gate.
|
||
"""
|
||
try:
|
||
data = yaml.safe_load(fm_text)
|
||
except Exception as exc:
|
||
raise FrontmatterError(re.sub(r'\s+', ' ', str(exc)).strip())
|
||
if not isinstance(data, dict):
|
||
raise FrontmatterError('frontmatter is not a YAML mapping')
|
||
value = data.get('description')
|
||
if value is None:
|
||
return ''
|
||
if not isinstance(value, str):
|
||
value = str(value)
|
||
return re.sub(r'\s+', ' ', value).strip()
|
||
|
||
|
||
# --- Body-shape checks (skills only; agents have no references/ dir) -------
|
||
# Deterministic and countable, so they are enforced here. Whether a given
|
||
# gotcha is WARRANTED is semantic and stays the auditor's judgment, which is why
|
||
# both gotcha checks are SUGGESTION tier. A missing reference file is not a
|
||
# style opinion — it is a broken pointer — so that one is ERROR tier.
|
||
#
|
||
# Both read a FENCE-MASKED copy of the body. Scanning the raw body made a
|
||
# ```-fenced example a hard ERROR — and the skills most likely to carry one are
|
||
# skill-author and skill-audit, which DOCUMENT the references/ convention — and
|
||
# let a `## Gotchas` heading inside a fenced block stand in for the real
|
||
# section. Masking preserves every byte offset (content becomes spaces,
|
||
# newlines stay), so a span found in the mask slices the original.
|
||
GOTCHA_MAX_ENTRIES = 5
|
||
GOTCHA_MAX_BODY_FRACTION = 0.25
|
||
# The heading has to BE "Gotchas", not merely contain the word: `## Gotcha
|
||
# handling` and `## Why gotchas matter` are prose sections, and treating one as
|
||
# the Gotchas section measured a span that was never a gotcha list.
|
||
GOTCHA_HEADING = re.compile(r'^(#{1,6})[ \t]+(?:[^\n]*?[ \t])?gotchas?[ \t]*:?[ \t]*$',
|
||
re.I | re.M)
|
||
# Column 0 only. `^[ \t]{0,3}` counted a two-space-indented CHILD bullet as a
|
||
# top-level entry, so a five-entry section with sub-bullets reported nine.
|
||
GOTCHA_ENTRY = re.compile(r'^(?:[-*+]|\d+[.)])[ \t]+', re.M)
|
||
FENCE_OPEN = re.compile(r'^[ \t]{0,3}(`{3,}|~{3,})')
|
||
REFERENCE_POINTER = re.compile(
|
||
r'(?<![\w./-])(?:\./)?references/([A-Za-z0-9][A-Za-z0-9._/-]*\.md)')
|
||
# A pointer named in a sentence that says the file is GONE is a historical
|
||
# mention, not a dispatch entry: "the old `references/legacy.md` was removed in
|
||
# v2" is prose and must not be a hard ERROR. Narrow on purpose — a live
|
||
# dispatch table never describes its own target as removed, so this costs no
|
||
# recall.
|
||
REFERENCE_PAST = re.compile(
|
||
r'\b(?:removed|deleted|renamed|superseded|replaced|obsolete|deprecated'
|
||
r'|former|formerly|gone|no longer|used to)\b', re.I)
|
||
# A pointer QUALIFIED by another skill's name — "skill-audit's
|
||
# references/validation-scripts.md" — names a file that is deliberately NOT in
|
||
# this skill's directory. Requiring it on the local disk left NO legal spelling
|
||
# for a cross-skill reference at all: the only alternative, a full repo path
|
||
# (`plugins/kyberforge/.apm/skills/skill-audit/references/...`), is itself a
|
||
# FAIL under skill-audit's own file-structure rubric, because a path that climbs
|
||
# out of the skill directory stops resolving once the plugin is cache-installed.
|
||
# The possessive form is the sanctioned spelling, and it is skipped here. It is
|
||
# not checked further — this function has no way to locate another skill's
|
||
# directory, and inventing one would reintroduce exactly the cross-plugin path
|
||
# assumption the rubric forbids.
|
||
REFERENCE_QUALIFIER = re.compile(u"[A-Za-z0-9][A-Za-z0-9._-]*`?['’]s[ \t]+`?$")
|
||
|
||
|
||
def mask_fenced(text):
|
||
"""Body with fenced code blocks blanked out, byte offsets preserved."""
|
||
out = []
|
||
fence = None
|
||
for line in text.splitlines(keepends=True):
|
||
stripped = line.rstrip('\r\n')
|
||
opener = FENCE_OPEN.match(stripped)
|
||
marker = opener.group(1) if opener else None
|
||
if fence is None:
|
||
if marker:
|
||
fence = marker
|
||
out.append(' ' * len(stripped) + line[len(stripped):])
|
||
continue
|
||
out.append(line)
|
||
else:
|
||
out.append(' ' * len(stripped) + line[len(stripped):])
|
||
if (marker and marker[0] == fence[0] and len(marker) >= len(fence)
|
||
and not stripped.strip()[len(marker):].strip()):
|
||
fence = None
|
||
return ''.join(out)
|
||
|
||
|
||
def gotcha_stats(body):
|
||
"""(entry count, section word count) for the first Gotchas section, or None.
|
||
|
||
The section runs to the next heading at the same level or shallower.
|
||
Entries are top-level list items; a section written as subheadings instead
|
||
of a list counts those. Headings and entries are read from the fence mask;
|
||
the word count is taken from the original slice, because fenced lines are
|
||
real body words and the fraction is measured against the whole body.
|
||
"""
|
||
masked = mask_fenced(body)
|
||
match = GOTCHA_HEADING.search(masked)
|
||
if not match:
|
||
return None
|
||
level = len(match.group(1))
|
||
rest = masked[match.end():]
|
||
nxt = re.search(r'^#{1,%d}[ \t]+' % level, rest, re.M)
|
||
end = match.end() + (nxt.start() if nxt else len(rest))
|
||
section = masked[match.end():end]
|
||
entries = len(GOTCHA_ENTRY.findall(section))
|
||
if entries == 0 and level < 6:
|
||
entries = len(re.findall(r'^#{%d,6}[ \t]+' % (level + 1), section, re.M))
|
||
return entries, len(body[match.end():end].split())
|
||
|
||
|
||
def missing_reference_pointers(body, skill_dir):
|
||
"""references/<file>.md named in the body but absent from disk."""
|
||
masked = mask_fenced(body)
|
||
missing = set()
|
||
for match in REFERENCE_POINTER.finditer(masked):
|
||
start = masked.rfind('\n', 0, match.start()) + 1
|
||
end = masked.find('\n', match.end())
|
||
if end < 0:
|
||
end = len(masked)
|
||
if REFERENCE_PAST.search(masked[start:end]):
|
||
continue
|
||
if REFERENCE_QUALIFIER.search(masked[start:match.start()]):
|
||
continue
|
||
if not os.path.isfile(os.path.join(skill_dir, 'references', match.group(1))):
|
||
missing.add('references/' + match.group(1))
|
||
return sorted(missing)
|
||
# ===== END ADR-0020 SHARED BOUNDARY RESOLVER =====
|
||
|
||
|
||
# A leading BOM is stripped before anything is parsed or counted. It changes
|
||
# neither count below — it is not a line separator and str.split() does not
|
||
# treat it as whitespace — but it did defeat the frontmatter match.
|
||
try:
|
||
content = strip_bom(read_text(skill_md))
|
||
except EncodingError as exc:
|
||
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
|
||
f"hard failure, not a skip")
|
||
print("One or more checks failed.")
|
||
sys.exit(1)
|
||
|
||
# --- Parse frontmatter ---
|
||
fm_match = FRONTMATTER_RE.match(content)
|
||
if not fm_match:
|
||
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
|
||
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
|
||
"spaces after either marker and CRLF endings are all tolerated). Nothing "
|
||
"downstream can be measured, so this is a hard failure, not a skip")
|
||
print("One or more checks failed.")
|
||
sys.exit(1)
|
||
|
||
fm = fm_match.group(1)
|
||
body_start = fm_match.end()
|
||
|
||
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
|
||
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
|
||
# by `description: ...` captured the NEXT KEY as the name and reported a
|
||
# mismatch instead of an absence. Same class of bug as the `description:` one
|
||
# the shared resolver's description_value() docstring records.
|
||
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
|
||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||
|
||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||
# carry indentation and newlines that are not part of the value: every length
|
||
# measurement below is wrong unless the scalar is folded first.
|
||
try:
|
||
desc = description_value(fm)
|
||
except FrontmatterError as exc:
|
||
fail(f"frontmatter is not valid YAML ({exc}). Nothing downstream can be "
|
||
f"measured, so this is a hard failure, not a skip")
|
||
print("One or more checks failed.")
|
||
sys.exit(1)
|
||
|
||
dir_name = os.path.basename(skill_dir)
|
||
|
||
# --- Checks ---
|
||
|
||
# name present
|
||
if name:
|
||
ok(f"name present: '{name}'")
|
||
else:
|
||
fail("name field is missing or empty")
|
||
|
||
# name matches directory
|
||
if name and dir_name:
|
||
if name == dir_name:
|
||
ok(f"name '{name}' matches directory '{dir_name}'")
|
||
else:
|
||
fail(f"name '{name}' does not match directory '{dir_name}'")
|
||
|
||
# name length
|
||
if name:
|
||
if len(name) <= 64:
|
||
ok(f"name length {len(name)} chars (limit: 64)")
|
||
else:
|
||
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
|
||
|
||
# name format
|
||
if name:
|
||
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
|
||
ok("name format valid (kebab-case)")
|
||
else:
|
||
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
|
||
|
||
# description present
|
||
if desc:
|
||
ok("description present")
|
||
else:
|
||
fail("description field is missing or empty")
|
||
|
||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||
# sits underneath it rather than replacing it.
|
||
if desc:
|
||
dlen = len(desc)
|
||
if dlen <= 1024:
|
||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||
else:
|
||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||
|
||
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
|
||
# but not backtick-quoted references like `FILL IN:` used in instructions.
|
||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||
|
||
# description contains unfilled placeholder
|
||
if desc and PLACEHOLDER_RE.search(desc):
|
||
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
|
||
else:
|
||
if desc:
|
||
ok("description has no unfilled placeholders")
|
||
|
||
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
|
||
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
|
||
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
|
||
# outside the plugin directory once the plugin is cache-installed, so there is
|
||
# no single source to share. Keep the two in sync by hand: if they drift, this
|
||
# audit will report a skill ready to ship that the commit hook then rejects.
|
||
MAX_LINES = 500
|
||
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
|
||
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
|
||
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
|
||
# for the full measurement.
|
||
MAX_WORDS = 2770
|
||
|
||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
|
||
# description constants; per ADR-0020 agents take the description gates and
|
||
# deliberately take NO body word gate, because an agent body becomes the system
|
||
# prompt of a fresh context rather than competing with a live conversation.
|
||
#
|
||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||
# quality gate. Likewise the 1024-character description limit above is the
|
||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||
# underneath it.
|
||
DESC_SUGGEST_CHARS = 250
|
||
DESC_MAX_CHARS = 400
|
||
BODY_SUGGEST_WORDS = 600
|
||
BODY_MAX_WORDS = 900
|
||
|
||
line_count = len(content.splitlines())
|
||
if line_count <= MAX_LINES:
|
||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||
else:
|
||
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
|
||
|
||
# str.split() with no argument splits on runs of whitespace, matching the
|
||
# `wc -w` the hook uses, and counts the whole file including frontmatter.
|
||
word_count = len(content.split())
|
||
if word_count <= MAX_WORDS:
|
||
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
|
||
else:
|
||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||
|
||
body = content[body_start:]
|
||
|
||
# --- ADR-0020: description budget -----------------------------------------
|
||
if desc:
|
||
dlen = len(desc)
|
||
if dlen > DESC_MAX_CHARS:
|
||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||
f"composition notes and implementation detail to the body or README.md")
|
||
elif dlen > DESC_SUGGEST_CHARS:
|
||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||
else:
|
||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||
|
||
# --- ADR-0020: body budget -------------------------------------------------
|
||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||
body_word_count = len(body.split())
|
||
if body_word_count > BODY_MAX_WORDS:
|
||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||
f"and rationale prose to references/ behind an explicit "
|
||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||
else:
|
||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||
|
||
# --- Reference pointers must exist -----------------------------------------
|
||
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
|
||
# on disk is a hard break, and until this check existed nothing in the
|
||
# gate/audit/vale stack noticed it — all three exited 0.
|
||
missing_refs = missing_reference_pointers(body, skill_dir)
|
||
for ref in missing_refs:
|
||
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
|
||
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
|
||
if not missing_refs:
|
||
ok("all referenced references/ files exist")
|
||
|
||
# --- Gotchas discipline -----------------------------------------------------
|
||
# SUGGESTION on both counts: the measurement is deterministic, but whether a
|
||
# given gotcha earns its place in the body is the auditor's judgment.
|
||
gotchas = gotcha_stats(body)
|
||
if gotchas is not None:
|
||
gotcha_entries, gotcha_words = gotchas
|
||
if gotcha_entries > GOTCHA_MAX_ENTRIES:
|
||
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
|
||
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
|
||
f"missing references/ file or a design problem written up as a warning")
|
||
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
|
||
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
|
||
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
|
||
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
|
||
f"parts to references/ and keep the section for live traps")
|
||
|
||
# --- ADR-0020: boundary clause present -------------------------------------
|
||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||
# this particular skill warrants a boundary clause is judgment. Both accepted
|
||
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
|
||
if desc:
|
||
if has_boundary_clause(desc):
|
||
ok("description has a boundary clause")
|
||
else:
|
||
suggest("description has no boundary clause — add the prose form (\"Do not use "
|
||
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||
"so the router knows where NOT to send this skill")
|
||
|
||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||
# The resolution universe comes from the SKILL's own location: the authoring
|
||
# root above it (every sibling plugin in the monorepo), its own apm package, and
|
||
# the packages that package declares in apm.yml dependencies.apm. It is never
|
||
# derived from this script's own path, and — when an authoring root exists — it
|
||
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
|
||
# run `apm install` return the same verdict. See the shared resolver's header.
|
||
if desc:
|
||
routing_targets = boundary_targets(desc)
|
||
known = known_targets(skill_dir) if routing_targets else set()
|
||
if routing_targets and not known:
|
||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||
f"determined for this path (no authoring root above it, no apm package "
|
||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
|
||
elif routing_targets:
|
||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||
# route notation or its own sentence corroborates it by naming another
|
||
# target that resolves. See the shared resolver's CORROBORATION note.
|
||
unresolved, soft = unresolved_targets(desc, known)
|
||
for target in unresolved:
|
||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||
f"in this monorepo, in this package, or in a package it declares in "
|
||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||
f"target sends the router nowhere")
|
||
for target in soft:
|
||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||
f"in this monorepo, in this package, or in a package it declares in "
|
||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||
f"file format or an English compound. If it IS a route, write it as "
|
||
f"`/{target}` or `-> {target}` and it will be checked properly")
|
||
if not unresolved:
|
||
# Counts the targets that ACTUALLY resolve, not every target found:
|
||
# a confirm-only target (one used attributively — see the resolver's
|
||
# ATTRIBUTIVE USE note) is exempt from the failure above, so
|
||
# reporting it as resolved would be a false claim.
|
||
resolved = [t for t in routing_targets if normalize_target(t) in known]
|
||
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
|
||
f"{', '.join(resolved) if resolved else '(none)'}")
|
||
|
||
# Body unfilled placeholders
|
||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||
if fill_matches:
|
||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||
else:
|
||
ok("SKILL.md body has no unfilled placeholders")
|
||
|
||
# Interactive prompt heuristic.
|
||
#
|
||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||
# them is a false positive — one that has already cost two authors a
|
||
# contorted rewrite of working source:
|
||
#
|
||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||
# read -r X <<EOF here-doc
|
||
# read -r line < "$file" redirect from a file
|
||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||
# read -r X PREVIOUS line, not this one
|
||
#
|
||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||
# exists for.
|
||
def stdin_redirected(line, prev_line):
|
||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||
# must still fail.
|
||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||
|
||
def interactive_reads(source):
|
||
hits = []
|
||
prev_line = ''
|
||
for line in source.splitlines():
|
||
stripped = line.strip()
|
||
if re.match(r'read(\s|$)', stripped):
|
||
if not stdin_redirected(line, prev_line):
|
||
hits.append(stripped)
|
||
elif re.match(r'input\(', stripped):
|
||
hits.append(stripped)
|
||
# Blank lines and comments cannot carry the pipe that feeds a
|
||
# following `read`, so they never displace the previous line.
|
||
if stripped and not stripped.startswith('#'):
|
||
prev_line = line
|
||
return hits
|
||
|
||
# Scripts checks
|
||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||
if os.path.isdir(scripts_dir):
|
||
scripts = [f for f in os.listdir(scripts_dir)
|
||
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
|
||
for fname in scripts:
|
||
fpath = os.path.join(scripts_dir, fname)
|
||
try:
|
||
sc = read_text(fpath)
|
||
except EncodingError as exc:
|
||
# The executable-bit check below still runs — one unreadable byte
|
||
# must not silently drop a second, independent check.
|
||
sc = None
|
||
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
|
||
f"interactive prompts")
|
||
interactive = interactive_reads(sc) if sc is not None else []
|
||
if interactive:
|
||
fail(f"scripts/{fname}: may use interactive input "
|
||
f"(read/input from a terminal detected): {interactive[0]}")
|
||
elif sc is not None:
|
||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||
# Executable bit
|
||
if os.access(fpath, os.X_OK):
|
||
ok(f"scripts/{fname}: is executable")
|
||
else:
|
||
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
|
||
|
||
# Summary
|
||
print()
|
||
for s in suggestions:
|
||
print(f"SUGGESTION {s}")
|
||
if suggestions:
|
||
print()
|
||
if not failed:
|
||
if suggestions:
|
||
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
|
||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||
else:
|
||
print("All checks passed.")
|
||
sys.exit(0)
|
||
else:
|
||
print("One or more checks failed.")
|
||
sys.exit(1)
|
||
PYTHON
|