feat(factory-audit): audit hooks, instructions and prompts
factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.
- Hook: JSON/shape/event-list checks mirroring the Copilot payload
validator, never-firing event casing, missing/escaping/non-executable
scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
(SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
(FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
ADR-0029 description length and trigger clause, dropped keys,
camelCase aliases, argument-hint with input (SUGGESTION). Whether a
prompt carries procedure is judgment in prompt-flow.md, not a script
heuristic.
Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.
Refs #94
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
@@ -2,13 +2,16 @@
|
||||
set -euo pipefail
|
||||
|
||||
# The ONE entry point for structural validation. It auto-detects whether the
|
||||
# target is a skill directory or an agent definition file and runs the matching
|
||||
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh
|
||||
# and are unchanged from the skill-audit / agent-audit scripts they came from.
|
||||
# target is a skill directory, an agent definition file, or one of the three apm
|
||||
# primitives with no container of their own (a hook, an instruction or a prompt)
|
||||
# and runs the matching check suite. The skill and agent suites live in
|
||||
# lib-checks-skill.sh and lib-checks-agent.sh and are unchanged from the
|
||||
# skill-audit / agent-audit scripts they came from; the primitive suite lives in
|
||||
# lib-checks-primitive.sh.
|
||||
# The ADR-0020 boundary resolver both of them need is sourced once, from
|
||||
# lib-boundary-resolver.sh, instead of being embedded twice.
|
||||
#
|
||||
# Detection never guesses. A target that matches neither shape is a hard exit 2
|
||||
# Detection never guesses. A target that matches no shape is a hard exit 2
|
||||
# naming the mismatch, because the alternative — picking a mode and letting the
|
||||
# suite fail on its own terms — reports a skill-shaped finding about an agent
|
||||
# file, or the reverse, and sends the reader after the wrong problem.
|
||||
@@ -115,17 +118,22 @@ _kf_require_lib() {
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate.sh <skill-dir | agent-file>
|
||||
Usage: validate.sh <skill-dir | agent-file | hook-file | instruction-file | prompt-file>
|
||||
|
||||
Validate a skill directory against the agentskills.io specification, or an agent
|
||||
definition file against the agent definition spec. The mode is detected from the
|
||||
target:
|
||||
Validate a skill directory against the agentskills.io specification, an agent
|
||||
definition file against the agent definition spec, or an apm hook, instruction or
|
||||
prompt file against what apm 0.28.0 actually deploys. The mode is detected from
|
||||
the target:
|
||||
|
||||
skill mode the target is a directory (a skill directory contains SKILL.md),
|
||||
or the target IS a SKILL.md file.
|
||||
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
||||
directory is named 'agents' (.apm/agents, .claude/agents,
|
||||
.github/agents, .copilot/agents).
|
||||
hook mode the target is a *.json file directly under a hooks/
|
||||
directory (.apm/hooks, or a package's root hooks/).
|
||||
instruction mode the target is a *.instructions.md file.
|
||||
prompt mode the target is a *.prompt.md file.
|
||||
|
||||
Skill mode audits the directory named by <skill-dir>.
|
||||
|
||||
@@ -143,7 +151,7 @@ Arguments:
|
||||
Exit codes:
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
2 Nothing was audited (no argument, the target matches neither shape, the
|
||||
2 Nothing was audited (no argument, the target matches no shape, the
|
||||
target does not exist, an unrecognized file extension, a missing
|
||||
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
|
||||
beside this script)
|
||||
@@ -156,7 +164,7 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo "Error: a skill directory, an agent file, or a hook, instruction or prompt file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
@@ -209,12 +217,21 @@ elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
TARGET="$(_kf_dirname "$TARGET")"
|
||||
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
||||
MODE=agent
|
||||
# The two primitive suffixes are tested before the agents/-parent rule: a
|
||||
# *.prompt.md or *.instructions.md file is that primitive wherever it sits, and
|
||||
# the parent-name rule would otherwise claim one that happened to sit in agents/.
|
||||
elif [[ "$TARGET_BASE" == *.instructions.md ]]; then
|
||||
MODE=instruction
|
||||
elif [[ "$TARGET_BASE" == *.prompt.md ]]; then
|
||||
MODE=prompt
|
||||
elif [[ "$TARGET_BASE" == *.json && "$TARGET_PARENT" == "hooks" && ! -d "$TARGET" ]]; then
|
||||
MODE=hook
|
||||
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
||||
MODE=agent
|
||||
else
|
||||
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2
|
||||
echo " Fix: pass one of those two shapes." >&2
|
||||
echo "Error: '$TARGET' matches no auditable shape." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents); hook mode needs a .json file directly under a hooks/ directory; instruction and prompt modes need a *.instructions.md or *.prompt.md file. Picking a mode anyway would audit this path against the wrong spec." >&2
|
||||
echo " Fix: pass one of those shapes." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
@@ -250,6 +267,13 @@ $KYBERFORGE_RESOLVER_PY
|
||||
$KYBERFORGE_AGENT_BODY_PY"
|
||||
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
|
||||
;;
|
||||
hook | instruction | prompt)
|
||||
_kf_require_lib lib-checks-primitive.sh
|
||||
# shellcheck source=lib-checks-primitive.sh
|
||||
. "$SCRIPT_DIR/lib-checks-primitive.sh"
|
||||
kyberforge_primitive_preflight
|
||||
python3 -u - "$TARGET" "$MODE" <<< "$KYBERFORGE_PRIMITIVE_PY" || RC=$?
|
||||
;;
|
||||
esac
|
||||
|
||||
exit "$RC"
|
||||
|
||||
Reference in New Issue
Block a user