feat(factory-audit): audit hooks, instructions and prompts

factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.

- Hook: JSON/shape/event-list checks mirroring the Copilot payload
  validator, never-firing event casing, missing/escaping/non-executable
  scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
  (SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
  (FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
  upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
  ADR-0029 description length and trigger clause, dropped keys,
  camelCase aliases, argument-hint with input (SUGGESTION). Whether a
  prompt carries procedure is judgment in prompt-flow.md, not a script
  heuristic.

Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.

Refs #94

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
2026-09-28 17:02:04 +00:00
parent 701e96d4b3
commit 70210d6a7e
14 changed files with 1062 additions and 29 deletions

View File

@@ -2,13 +2,16 @@
set -euo pipefail
# The ONE entry point for structural validation. It auto-detects whether the
# target is a skill directory or an agent definition file and runs the matching
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh
# and are unchanged from the skill-audit / agent-audit scripts they came from.
# target is a skill directory, an agent definition file, or one of the three apm
# primitives with no container of their own (a hook, an instruction or a prompt)
# and runs the matching check suite. The skill and agent suites live in
# lib-checks-skill.sh and lib-checks-agent.sh and are unchanged from the
# skill-audit / agent-audit scripts they came from; the primitive suite lives in
# lib-checks-primitive.sh.
# The ADR-0020 boundary resolver both of them need is sourced once, from
# lib-boundary-resolver.sh, instead of being embedded twice.
#
# Detection never guesses. A target that matches neither shape is a hard exit 2
# Detection never guesses. A target that matches no shape is a hard exit 2
# naming the mismatch, because the alternative — picking a mode and letting the
# suite fail on its own terms — reports a skill-shaped finding about an agent
# file, or the reverse, and sends the reader after the wrong problem.
@@ -115,17 +118,22 @@ _kf_require_lib() {
usage() {
cat <<EOF
Usage: validate.sh <skill-dir | agent-file>
Usage: validate.sh <skill-dir | agent-file | hook-file | instruction-file | prompt-file>
Validate a skill directory against the agentskills.io specification, or an agent
definition file against the agent definition spec. The mode is detected from the
target:
Validate a skill directory against the agentskills.io specification, an agent
definition file against the agent definition spec, or an apm hook, instruction or
prompt file against what apm 0.28.0 actually deploys. The mode is detected from
the target:
skill mode the target is a directory (a skill directory contains SKILL.md),
or the target IS a SKILL.md file.
agent mode the target is a *.agent.md file, or a *.md file whose parent
directory is named 'agents' (.apm/agents, .claude/agents,
.github/agents, .copilot/agents).
hook mode the target is a *.json file directly under a hooks/
directory (.apm/hooks, or a package's root hooks/).
instruction mode the target is a *.instructions.md file.
prompt mode the target is a *.prompt.md file.
Skill mode audits the directory named by <skill-dir>.
@@ -143,7 +151,7 @@ Arguments:
Exit codes:
0 All checks passed (may include SUGGESTIONs)
1 One or more checks failed
2 Nothing was audited (no argument, the target matches neither shape, the
2 Nothing was audited (no argument, the target matches no shape, the
target does not exist, an unrecognized file extension, a missing
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
beside this script)
@@ -156,7 +164,7 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
fi
if [[ $# -lt 1 ]]; then
echo "Error: a skill directory or an agent file is required." >&2
echo "Error: a skill directory, an agent file, or a hook, instruction or prompt file is required." >&2
echo "" >&2
usage >&2
exit 2
@@ -209,12 +217,21 @@ elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
TARGET="$(_kf_dirname "$TARGET")"
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
MODE=agent
# The two primitive suffixes are tested before the agents/-parent rule: a
# *.prompt.md or *.instructions.md file is that primitive wherever it sits, and
# the parent-name rule would otherwise claim one that happened to sit in agents/.
elif [[ "$TARGET_BASE" == *.instructions.md ]]; then
MODE=instruction
elif [[ "$TARGET_BASE" == *.prompt.md ]]; then
MODE=prompt
elif [[ "$TARGET_BASE" == *.json && "$TARGET_PARENT" == "hooks" && ! -d "$TARGET" ]]; then
MODE=hook
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
MODE=agent
else
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2
echo " Fix: pass one of those two shapes." >&2
echo "Error: '$TARGET' matches no auditable shape." >&2
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents); hook mode needs a .json file directly under a hooks/ directory; instruction and prompt modes need a *.instructions.md or *.prompt.md file. Picking a mode anyway would audit this path against the wrong spec." >&2
echo " Fix: pass one of those shapes." >&2
exit 2
fi
@@ -250,6 +267,13 @@ $KYBERFORGE_RESOLVER_PY
$KYBERFORGE_AGENT_BODY_PY"
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
;;
hook | instruction | prompt)
_kf_require_lib lib-checks-primitive.sh
# shellcheck source=lib-checks-primitive.sh
. "$SCRIPT_DIR/lib-checks-primitive.sh"
kyberforge_primitive_preflight
python3 -u - "$TARGET" "$MODE" <<< "$KYBERFORGE_PRIMITIVE_PY" || RC=$?
;;
esac
exit "$RC"