factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.
- Hook: JSON/shape/event-list checks mirroring the Copilot payload
validator, never-firing event casing, missing/escaping/non-executable
scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
(SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
(FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
ADR-0029 description length and trigger clause, dropped keys,
camelCase aliases, argument-hint with input (SUGGESTION). Whether a
prompt carries procedure is judgment in prompt-flow.md, not a script
heuristic.
Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.
Refs #94
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
280 lines
13 KiB
Bash
Executable File
280 lines
13 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# The ONE entry point for structural validation. It auto-detects whether the
|
|
# target is a skill directory, an agent definition file, or one of the three apm
|
|
# primitives with no container of their own (a hook, an instruction or a prompt)
|
|
# and runs the matching check suite. The skill and agent suites live in
|
|
# lib-checks-skill.sh and lib-checks-agent.sh and are unchanged from the
|
|
# skill-audit / agent-audit scripts they came from; the primitive suite lives in
|
|
# lib-checks-primitive.sh.
|
|
# The ADR-0020 boundary resolver both of them need is sourced once, from
|
|
# lib-boundary-resolver.sh, instead of being embedded twice.
|
|
#
|
|
# Detection never guesses. A target that matches no shape is a hard exit 2
|
|
# naming the mismatch, because the alternative — picking a mode and letting the
|
|
# suite fail on its own terms — reports a skill-shaped finding about an agent
|
|
# file, or the reverse, and sends the reader after the wrong problem.
|
|
|
|
# --- Path splitting, with bash builtins only -------------------------------
|
|
# dirname and basename are EXTERNAL commands, and every call below happens
|
|
# before the mode-specific python3/PyYAML preflight. Using them put coreutils
|
|
# ahead of python3 in the dependency order: on a PATH carrying neither, this
|
|
# script died at exit 127 naming `dirname` instead of reaching the preflight
|
|
# that names python3 — the exact failure tests/test-adr0020-contract.sh
|
|
# assertion 2 exists to prevent ("the two are checked separately so the message
|
|
# names the thing to install rather than the wrong one"). The pre-merge
|
|
# validate.sh was one self-contained file that reached its preflight on builtins
|
|
# alone; these two functions restore that property. `cd` and `pwd` are builtins
|
|
# and may stay.
|
|
#
|
|
# They reproduce dirname/basename semantics for the shapes this script sees:
|
|
# trailing slashes are stripped, a path with no slash yields "." / itself, and
|
|
# "/" yields "/".
|
|
_kf_dirname() {
|
|
local _p="$1"
|
|
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
|
if [[ "$_p" == "/" ]]; then
|
|
printf '%s' "/"
|
|
return 0
|
|
fi
|
|
if [[ "$_p" != */* ]]; then
|
|
printf '%s' "."
|
|
return 0
|
|
fi
|
|
_p="${_p%/*}"
|
|
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
|
if [[ -z "$_p" ]]; then
|
|
_p="/"
|
|
fi
|
|
printf '%s' "$_p"
|
|
}
|
|
|
|
_kf_basename() {
|
|
local _p="$1"
|
|
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
|
if [[ "$_p" == "/" ]]; then
|
|
printf '%s' "/"
|
|
return 0
|
|
fi
|
|
printf '%s' "${_p##*/}"
|
|
}
|
|
|
|
# --- The target's parent directory NAME, resolved -------------------------
|
|
# The agent rule tests the NAME of the target's parent directory. Reading that
|
|
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
|
|
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
|
|
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
|
|
# directory was refused as matching neither shape, by an error message naming
|
|
# that exact shape as valid. The pre-merge agent validator had no path-shape
|
|
# gate and worked from any working directory.
|
|
#
|
|
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
|
|
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
|
|
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
|
|
# still addressed as agents/, which is what the literal test always honoured.
|
|
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
|
|
# that cannot be entered — a typo'd path — falls back to the literal name, so
|
|
# the neither-shape error still fires for it.
|
|
_kf_parent_name() {
|
|
local _dir _resolved
|
|
_dir="$(_kf_dirname "$1")"
|
|
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
|
|
_kf_basename "$_resolved"
|
|
else
|
|
_kf_basename "$_dir"
|
|
fi
|
|
}
|
|
|
|
# --- This script's own directory, and the libraries beside it -------------
|
|
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
|
|
# CDPATH exported and the relative invocation the flow references prescribe
|
|
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
|
|
# and could resolve through CDPATH to an unrelated directory and source a
|
|
# same-named file from there. CDPATH is cleared for the one command, `--` ends
|
|
# option parsing for a directory named like a flag, and stdout is discarded so
|
|
# only `pwd` is captured.
|
|
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
|
|
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
|
|
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
|
|
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
|
|
exit 2
|
|
fi
|
|
|
|
# A sourced library that is missing or unreadable used to kill the script under
|
|
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
|
|
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
|
|
# partial install, or a copy or symlink of this one file taken out of scripts/,
|
|
# was therefore reported as a failing audit. Checked explicitly instead, and
|
|
# exit 2, which the same references read as "it never ran".
|
|
_kf_require_lib() {
|
|
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
|
|
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
|
|
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
|
|
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
|
|
exit 2
|
|
fi
|
|
}
|
|
|
|
usage() {
|
|
cat <<EOF
|
|
Usage: validate.sh <skill-dir | agent-file | hook-file | instruction-file | prompt-file>
|
|
|
|
Validate a skill directory against the agentskills.io specification, an agent
|
|
definition file against the agent definition spec, or an apm hook, instruction or
|
|
prompt file against what apm 0.28.0 actually deploys. The mode is detected from
|
|
the target:
|
|
|
|
skill mode the target is a directory (a skill directory contains SKILL.md),
|
|
or the target IS a SKILL.md file.
|
|
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
|
directory is named 'agents' (.apm/agents, .claude/agents,
|
|
.github/agents, .copilot/agents).
|
|
hook mode the target is a *.json file directly under a hooks/
|
|
directory (.apm/hooks, or a package's root hooks/).
|
|
instruction mode the target is a *.instructions.md file.
|
|
prompt mode the target is a *.prompt.md file.
|
|
|
|
Skill mode audits the directory named by <skill-dir>.
|
|
|
|
Agent mode: at plugin/APM scope, <agent-file> is a single vendor-neutral
|
|
.apm/agents/<name>.agent.md file with no counterpart. Its frontmatter allowlist
|
|
is not restated here: it is read at load time from the apm-agent-allowlist
|
|
section of references/agent-field-inventory.md, which is the authoritative list.
|
|
At project or user scope, <agent-file> is either half of a Claude Code .md /
|
|
Copilot .agent.md pair.
|
|
|
|
Arguments:
|
|
skill-dir Path to the skill directory containing SKILL.md.
|
|
agent-file Path to the agent file (or either half of a project/user-scope pair).
|
|
|
|
Exit codes:
|
|
0 All checks passed (may include SUGGESTIONs)
|
|
1 One or more checks failed
|
|
2 Nothing was audited (no argument, the target matches no shape, the
|
|
target does not exist, an unrecognized file extension, a missing
|
|
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
|
|
beside this script)
|
|
EOF
|
|
}
|
|
|
|
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
|
usage
|
|
exit 0
|
|
fi
|
|
|
|
if [[ $# -lt 1 ]]; then
|
|
echo "Error: a skill directory, an agent file, or a hook, instruction or prompt file is required." >&2
|
|
echo "" >&2
|
|
usage >&2
|
|
exit 2
|
|
fi
|
|
|
|
TARGET="$1"
|
|
|
|
# --- The target has to be there --------------------------------------------
|
|
# Only the directory branch below stats the target; the *.agent.md, the
|
|
# agents/-parent and the SKILL.md branches classify on NAME alone, so a typo'd
|
|
# path matching one of those shapes was handed to python3 and came back as a
|
|
# FAIL at exit 1 — the findings tier, for a target that was never there to have
|
|
# findings about. The tiers are: 2 nothing is at this path so no check ran, 1
|
|
# something is there and it is broken.
|
|
#
|
|
# This runs on the TYPED path, before the SKILL.md -> parent-directory rewrite
|
|
# further down: rewritten first, a missing `docs/SKILL.md` would be tested as
|
|
# `docs`, which exists, and the guard would miss it.
|
|
#
|
|
# `-L` deliberately rescues what `-e` rejects. A dangling symlink and a symlink
|
|
# loop are both FALSE to -e but TRUE to -L, and neither belongs here: something
|
|
# IS at that path, it just cannot be opened, and "exists but unreadable" is a
|
|
# real finding the agent suite's check_file reports as a FAIL naming the file.
|
|
# Catching them here would replace that FAIL with a false "does not exist".
|
|
if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then
|
|
echo "Error: '$TARGET' does not exist." >&2
|
|
echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2
|
|
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2
|
|
exit 2
|
|
fi
|
|
|
|
# --- Detect the mode -------------------------------------------------------
|
|
# Pure path and stat inspection, no interpreter and no external command needed,
|
|
# so it runs before the python3/PyYAML preflight — which is mode-specific,
|
|
# because each suite names the gates it would otherwise skip.
|
|
TARGET_BASE="$(_kf_basename "$TARGET")"
|
|
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
|
|
|
|
if [[ -d "$TARGET" ]]; then
|
|
if [[ -f "$TARGET/SKILL.md" ]]; then
|
|
MODE=skill
|
|
else
|
|
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
|
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2
|
|
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
|
exit 2
|
|
fi
|
|
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
|
MODE=skill
|
|
TARGET="$(_kf_dirname "$TARGET")"
|
|
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
|
MODE=agent
|
|
# The two primitive suffixes are tested before the agents/-parent rule: a
|
|
# *.prompt.md or *.instructions.md file is that primitive wherever it sits, and
|
|
# the parent-name rule would otherwise claim one that happened to sit in agents/.
|
|
elif [[ "$TARGET_BASE" == *.instructions.md ]]; then
|
|
MODE=instruction
|
|
elif [[ "$TARGET_BASE" == *.prompt.md ]]; then
|
|
MODE=prompt
|
|
elif [[ "$TARGET_BASE" == *.json && "$TARGET_PARENT" == "hooks" && ! -d "$TARGET" ]]; then
|
|
MODE=hook
|
|
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
|
MODE=agent
|
|
else
|
|
echo "Error: '$TARGET' matches no auditable shape." >&2
|
|
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents); hook mode needs a .json file directly under a hooks/ directory; instruction and prompt modes need a *.instructions.md or *.prompt.md file. Picking a mode anyway would audit this path against the wrong spec." >&2
|
|
echo " Fix: pass one of those shapes." >&2
|
|
exit 2
|
|
fi
|
|
|
|
# --- Run the matching suite ------------------------------------------------
|
|
# Each suite is reassembled in the order the resolver block sat in before the
|
|
# merge — preamble, resolver, body — so every check runs against exactly the
|
|
# names and the order it always did.
|
|
RC=0
|
|
case "$MODE" in
|
|
skill)
|
|
_kf_require_lib lib-boundary-resolver.sh
|
|
# shellcheck source=lib-boundary-resolver.sh
|
|
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
|
_kf_require_lib lib-checks-skill.sh
|
|
# shellcheck source=lib-checks-skill.sh
|
|
. "$SCRIPT_DIR/lib-checks-skill.sh"
|
|
kyberforge_skill_preflight
|
|
PROG="$KYBERFORGE_SKILL_PREAMBLE_PY
|
|
$KYBERFORGE_RESOLVER_PY
|
|
$KYBERFORGE_SKILL_BODY_PY"
|
|
python3 -u - "$TARGET" <<< "$PROG" || RC=$?
|
|
;;
|
|
agent)
|
|
_kf_require_lib lib-boundary-resolver.sh
|
|
# shellcheck source=lib-boundary-resolver.sh
|
|
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
|
_kf_require_lib lib-checks-agent.sh
|
|
# shellcheck source=lib-checks-agent.sh
|
|
. "$SCRIPT_DIR/lib-checks-agent.sh"
|
|
kyberforge_agent_preflight
|
|
PROG="$KYBERFORGE_AGENT_PREAMBLE_PY
|
|
$KYBERFORGE_RESOLVER_PY
|
|
$KYBERFORGE_AGENT_BODY_PY"
|
|
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
|
|
;;
|
|
hook | instruction | prompt)
|
|
_kf_require_lib lib-checks-primitive.sh
|
|
# shellcheck source=lib-checks-primitive.sh
|
|
. "$SCRIPT_DIR/lib-checks-primitive.sh"
|
|
kyberforge_primitive_preflight
|
|
python3 -u - "$TARGET" "$MODE" <<< "$KYBERFORGE_PRIMITIVE_PY" || RC=$?
|
|
;;
|
|
esac
|
|
|
|
exit "$RC"
|