Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests 1 Actions Packages Projects Releases 3 Wiki Activity

feat(factory-audit): audit hooks, instructions and prompts

Browse Source 
factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.

- Hook: JSON/shape/event-list checks mirroring the Copilot payload
  validator, never-firing event casing, missing/escaping/non-executable
  scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
  (SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
  (FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
  upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
  ADR-0029 description length and trigger clause, dropped keys,
  camelCase aliases, argument-hint with input (SUGGESTION). Whether a
  prompt carries procedure is judgment in prompt-flow.md, not a script
  heuristic.

Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.

Refs #94

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
Defame1297
2026-09-28 17:02:04 +00:00
parent 701e96d4b3
commit 70210d6a7e
14 changed files with 1062 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
.claude-plugin/marketplace.json
View File

@@ -1,7 +1,7 @@
{ {
"name": "holocron", "name": "holocron",
"description": "AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows.", "description": "AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows.",
"version": "0.5.1", "version": "0.5.2",
"owner": { "owner": {
"name": "Defame1297", "name": "Defame1297",
"email": "defame1297@rkdr.net", "email": "defame1297@rkdr.net",
@@ -11,7 +11,7 @@
{ {
"name": "kyberforge", "name": "kyberforge",
"description": "Skills and agents for creating, maintaining, and managing an apm plugin marketplace for Claude Code and GitHub Copilot.", "description": "Skills and agents for creating, maintaining, and managing an apm plugin marketplace for Claude Code and GitHub Copilot.",
"version": "2.0.1", "version": "2.1.0",
"category": "Developer Tools", "category": "Developer Tools",
"source": "./plugins/kyberforge" "source": "./plugins/kyberforge"
}, },

6
apm.yml
View File

@@ -1,5 +1,5 @@
name: holocron name: holocron
version: 0.5.1 version: 0.5.2
description: AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows. description: AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows.
license: MIT license: MIT
@@ -61,7 +61,7 @@ dependencies:
# an apm mechanic. # an apm mechanic.
executables: executables:
allow: allow:
kyberforge#2.0.1: kyberforge#2.1.0:
hooks: true hooks: true
bin: true bin: true
@@ -71,7 +71,7 @@ marketplace:
# top-level apm.yml description:/version: above are NOT inherited into the # top-level apm.yml description:/version: above are NOT inherited into the
# compiled output despite being used elsewhere (e.g. by `apm audit`). # compiled output despite being used elsewhere (e.g. by `apm audit`).
description: AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows. description: AI development skills for Claude Code, and for GitHub Copilot through apm — factory, design, implement, review, and cross-cutting workflows.
version: 0.5.1 version: 0.5.2
owner: owner:
name: Defame1297 name: Defame1297
email: defame1297@rkdr.net email: defame1297@rkdr.net

23
plugins/kyberforge/.apm/skills/factory-audit/SKILL.md
View File

@@ -1,13 +1,13 @@
--- ---
name: factory-audit name: factory-audit
description: > description: >
Use when the user wants a skill directory or agent definition audited, Use when a skill, agent, or apm hook, instruction or prompt needs auditing,
including "is this ready to ship", or after hand-editing one outside its including "is this ready to ship", or after hand-editing one outside its
author skill. Not applying skill fixes -> skill-author. Not applying agent author skill. Not applying skill fixes -> skill-author.
fixes -> agent-author. Not applying agent fixes -> agent-author.
allowed-tools: Bash Read allowed-tools: Bash Read
metadata: metadata:
version: "1.0.5" version: "1.1.0"
category: factory category: factory
source_keys: source_keys:
- agentskills-home - agentskills-home
@@ -20,6 +20,8 @@ metadata:
- claude-code-subagents-docs - claude-code-subagents-docs
- context7-github-en-copilot - context7-github-en-copilot
- github-custom-agents-configuration - github-custom-agents-configuration
- apm-cli-installed-source
- apm-docs-llms-full
--- ---
## Gotchas ## Gotchas
@@ -30,21 +32,24 @@ metadata:
## Step 0 — Dispatch ## Step 0 — Dispatch
Resolve the flow from the target path **before running anything**. The two flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches. Resolve the flow from the target path **before running anything**. The flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches.
| Target | Flow | Read | | Target | Flow | Read |
|---|---|---| |---|---|---|
| A directory containing `SKILL.md` | skill | `references/skill-flow.md` | | A directory containing `SKILL.md` | skill | `references/skill-flow.md` |
| A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` | | A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` |
| A file named `*.agent.md` | agent | `references/agent-flow.md` | | A file named `*.agent.md` | agent | `references/agent-flow.md` |
| A file named `*.instructions.md` | instruction | `references/instruction-flow.md` |
| A file named `*.prompt.md` | prompt | `references/prompt-flow.md` |
| A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` | | A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` |
| A `.json` file whose immediate parent directory is `hooks/` (`.apm/hooks`, or a package's root `hooks/`) | hook | `references/hook-flow.md` |
| Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — | | Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — |
Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4. Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4.
On the last row, stop: run no validator and tell the user the two accepted shapes — a skill directory (or its `SKILL.md`), or an agent file (`*.agent.md`, or a `.md` directly under an `agents/` directory). Guessing a flow audits the path against the wrong spec. On the last row, stop: run no validator and tell the user the shapes the other rows accept. Guessing a flow audits the path against the wrong spec.
The scripts re-detect the flow from the path. If `validate.sh` reports on the other artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line. The scripts re-detect the flow from the path. If `validate.sh` reports on a different artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line.
## Step 4 — Report ## Step 4 — Report
@@ -64,6 +69,8 @@ Checked: structure · provider-safety · description · body · delegation · co
On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check. On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check.
Hook, instruction and prompt flows: the line their flow file ends with.
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed. Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
Each finding: Each finding:
@@ -74,4 +81,4 @@ FAIL/SUGGESTION <finding> — file:line
Fix: <exact change — quote before/after where applicable> Fix: <exact change — quote before/after where applicable>
``` ```
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow. Do not apply fixes — report and propose only. Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow, `Run primitive-author to address findings.` on the other three. Do not apply fixes — report and propose only.

8
plugins/kyberforge/.apm/skills/factory-audit/assets/vale/.vale.ini
View File

@@ -6,5 +6,13 @@ BasedOnStyles = Kyberforge
[**/agents/*.md] [**/agents/*.md]
BasedOnStyles = Kyberforge BasedOnStyles = Kyberforge
[**/*.instructions.md]
BasedOnStyles = Kyberforge
[**/*.prompt.md]
BasedOnStyles = Kyberforge
# Stays the last section: tests/test-vale-wrap.sh case 31 appends a rule
# override to the end of this file and relies on it landing here.
[**/*.agent.md] [**/*.agent.md]
BasedOnStyles = Kyberforge, KyberforgeCopilot BasedOnStyles = Kyberforge, KyberforgeCopilot

54
plugins/kyberforge/.apm/skills/factory-audit/references/hook-flow.md Normal file
View File

@@ -0,0 +1,54 @@
---
source_keys:
- apm-cli-installed-source
- apm-docs-llms-full
---
# Hook Flow
Steps 1 to 3 for an apm hook — the target Step 0 matched as a `.json` file directly under a
`hooks/` directory. Work them in order, then return to `SKILL.md` Step 4 to report.
## Gotchas
- apm checks almost nothing here. Invalid JSON is skipped without a word, an all-lowercase event deploys and never fires, and a missing script only warns — so `apm install` exiting 0 says nothing about whether the hook works. Never cite a clean install as evidence against a finding.
- Copilot receiving a Claude-shaped file is not a finding. apm renders one source for every target and documents that it owns the per-target shape; whether Copilot CLI honours a nested entry or `matcher` is unverified upstream, not a defect in the file.
## Step 1 — Deterministic checks
Resolve the path against this skill's own directory. Run exactly:
```bash
bash scripts/validate.sh <hook-file>
```
Its findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: JSON validity, the wrapped-or-naked shape, event lists and nested handler lists (the checks whose failure makes the Copilot install fail), event names that never fire, referenced scripts that are missing, outside the package or not executable, deprecated filename routing, and `${CLAUDE_PLUGIN_ROOT}` where `${PLUGIN_ROOT}` would do. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
There is no provenance and no Vale step: a hook carries no `source_keys` and no prose.
## Step 2 — Read the hook and its scripts
Read the hook file, every script it references, and the package's `apm.yml` `targets:` — reach is narrowed there, never in the hook file.
## Step 3 — Qualitative audit
Cite file and line for every finding.
**purpose** — apm's own rule is to reach for a skill, instruction or prompt first; a hook is for "this must always happen at this event".
- FAIL: the script carries procedure the agent should follow — instructions printed to the model, a multi-step workflow — rather than a runtime callback. That is a skill.
- SUGGESTION: the behaviour is harness-specific (a Claude-only event, a Claude-only matcher value) in a package whose `targets:` includes other harnesses, and nothing records that the other targets receiving it was accepted. The apm-native fix is a separate package with its own `targets:`, not a routing filename.
**handlers** — the research checklist's Should and audit-only items, which apm never checks:
- SUGGESTION: a handler without `"type": "command"` or an explicit numeric `timeout` in seconds.
- SUGGESTION: a tool event (`PreToolUse`, `PostToolUse`) or `SessionStart` with no `matcher` — Claude receives `"*"`. A `matcher` on an event Claude ignores it for (`Stop`, `UserPromptSubmit`) is inert, not wrong.
- SUGGESTION: a PascalCase event name that is not a real Claude Code event (a misspelling deploys verbatim and never fires; the script cannot tell a typo from an event it does not know).
- SUGGESTION: `bash`/`powershell`/`timeoutSec` keys in a Claude-shaped file — they render, but leave stray keys in `settings.json`.
- SUGGESTION: an unquoted script path that could contain spaces.
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
```text
Checked: structure · purpose · handlers
```

55
plugins/kyberforge/.apm/skills/factory-audit/references/instruction-flow.md Normal file
View File

@@ -0,0 +1,55 @@
---
source_keys:
- apm-cli-installed-source
- apm-docs-llms-full
---
# Instruction Flow
Steps 1 to 3 for an apm instruction — the target Step 0 matched as a `*.instructions.md` file.
Work them in order, then return to `SKILL.md` Step 4 to report.
## Gotchas
- `apm compile --validate` is not a gate. Every message `Instruction.validate()` produces is a warning, and it reports success on a file with no description and an empty body — never cite it as evidence against a finding.
- `description` never reaches Claude, and it is index text elsewhere, never a routing description. Do not hold it to the skill description contract: no trigger clause, no boundary clause. A Vale `Kyberforge.DescriptionOpener` alert here means rewrite it as a plain statement of what the rule covers.
## Step 1 — Deterministic checks
Resolve both paths against this skill's own directory. Run exactly:
```bash
bash scripts/validate.sh <instruction-file>
bash scripts/vale-wrap.sh <instruction-file>
```
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
There is no provenance step: an instruction carries no `source_keys`.
## Step 2 — Read the instruction and its context
Read the file, the package's `apm.yml`, and the repo's root `AGENTS.md`. For a scoped file, list the tracked files its `applyTo` matches (`rtk git ls-files` filtered by the glob).
## Step 3 — Qualitative audit
Cite file and line for every finding.
**scope** — an instruction applies when files matching `applyTo` are touched; with no `applyTo` it loads into every session of every repo that installs the package.
- FAIL: an always-on file whose content is a rule for this repo alone — it belongs in `AGENTS.md`, which is the repo's single always-on source, not in a package that ships it to every consumer.
- FAIL: an `applyTo` glob that matches no tracked file in any repo the package plausibly targets, so the rule never loads.
- SUGGESTION: an always-on file whose content is really file-type specific — narrow it with `applyTo`.
- SUGGESTION: a glob much broader than the content (`**` for a rule about Python).
**description**
- SUGGESTION: the description does not say what the rule covers, or contradicts the body. Any rationale Claude readers need belongs in the body, because Claude drops the description.
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
```text
Checked: structure · prose · scope · description
```

55
plugins/kyberforge/.apm/skills/factory-audit/references/prompt-flow.md Normal file
View File

@@ -0,0 +1,55 @@
---
source_keys:
- apm-cli-installed-source
- apm-docs-llms-full
---
# Prompt Flow
Steps 1 to 3 for an apm prompt — the target Step 0 matched as a `*.prompt.md` file. Work them in
order, then return to `SKILL.md` Step 4 to report.
## Gotchas
- A prompt is judged against ADR-0029, not against apm's framing. apm calls a prompt "a callable program"; this repo holds it to a single-intent, user-triggered message that steers existing skills or agents by name and carries no procedure of its own.
- A prompt's description is not a skill description. It is one plain user-facing sentence with no "Use when" trigger clause and no boundary clause — so never raise a missing trigger or boundary as a finding. A Vale `Kyberforge.DescriptionOpener` alert here means rewrite it as an imperative action ("Review the current PR with …"), not add a trigger.
## Step 1 — Deterministic checks
Resolve both paths against this skill's own directory. Run exactly:
```bash
bash scripts/validate.sh <prompt-file>
bash scripts/vale-wrap.sh <prompt-file>
```
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path and name, frontmatter, `description` presence, length and trigger clause, keys Claude drops, `input:` names and shapes, and `${input:x}` references against `input:`. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
There is no provenance step: a prompt carries no `source_keys`.
## Step 2 — Read the prompt and what it steers
Read the file end to end, then the description of every skill or agent its body names, and confirm each resolves in this repo or in a package the prompt's package declares.
## Step 3 — Qualitative audit
Cite file and line for every finding.
**role** — whether this is a prompt at all. Decide it by reading the body, not by its length or headings; there is no threshold.
- FAIL: the body clearly carries reusable procedure — steps, gotchas, domain know-how the agent could not act without — rather than steering skills or agents that hold it. Fix: move the procedure into a skill (new, or the one it belongs to) and reduce the prompt to the message that invokes it.
- FAIL: the body names a skill or agent that does not resolve, or one carrying `disable-model-invocation: true`, which the model cannot invoke.
- SUGGESTION: borderline — some how-to detail beyond steering, but not a full procedure.
- SUGGESTION: more than one intent in one prompt.
**description**
- SUGGESTION: the description does not read as one user-facing action, or does not name the skills the prompt steers. On Claude the description is model-visible and apm drops `disable-model-invocation`, so naming the steered skills keeps the router pointed at the capability rather than the wrapper.
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
```text
Checked: structure · prose · role · description
```

18
plugins/kyberforge/.apm/skills/factory-audit/references/sources.md
View File

@@ -10,6 +10,8 @@ source_keys:
- claude-code-subagents-docs - claude-code-subagents-docs
- context7-github-en-copilot - context7-github-en-copilot
- github-custom-agents-configuration - github-custom-agents-configuration
- apm-cli-installed-source
- apm-docs-llms-full
--- ---
# Sources # Sources
@@ -151,3 +153,19 @@ source_keys:
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling - **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
- **Contributing files:** (none) - **Contributing files:** (none)
- **Status:** `extracted` - **Status:** `extracted`
## apm-cli-installed-source
- **URL:** file:///root/.local/pipx/venvs/apm-cli/lib/python3.11/site-packages/apm_cli/
- **Research doc:** plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md
- **Description:** Installed apm-cli 0.28.0 source — ground truth for what apm deploys from a hook, instruction or prompt file and what it silently skips, warns on, or fails the install for; every deterministic check in `scripts/lib-checks-primitive.sh` traces to it via the research docs' Authoring checklists
- **Contributing files:** SKILL.md, references/hook-flow.md, references/instruction-flow.md, references/prompt-flow.md
- **Status:** `extracted`
## apm-docs-llms-full
- **URL:** https://microsoft.github.io/apm/llms-full.txt
- **Research doc:** plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md
- **Description:** Published apm docs bundle — the "Hooks and commands", "Instructions and agents" and "Author a prompt" guides: canonical hook shape and `${PLUGIN_ROOT}`, reach narrowed by `targets:` rather than filename routing, and "reach for a skill, instruction, or prompt first"
- **Contributing files:** SKILL.md, references/hook-flow.md, references/instruction-flow.md, references/prompt-flow.md
- **Status:** `extracted`

491
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-primitive.sh Executable file
View File

@@ -0,0 +1,491 @@
#!/usr/bin/env bash
# lib-checks-primitive.sh — SOURCED, never executed.
#
# The structural check suite for the three apm primitives that have no author
# skill of their own and no SKILL.md-shaped container: hooks
# (.apm/hooks/*.json), instructions (*.instructions.md) and prompts
# (*.prompt.md). validate.sh detects which one it was handed from the path and
# feeds $KYBERFORGE_PRIMITIVE_PY to python3 with the target as argv[1] and the
# primitive kind (hook | instruction | prompt) as argv[2].
#
# Every check here exists because apm itself does not make it. apm 0.28.0
# silently skips invalid hook JSON, only warns on an instruction with no
# description or body, and never validates a prompt's input: names against its
# ${input:x} references — so `apm install` and `apm compile --validate` both exit
# 0 on files that deploy nothing, or deploy something that never fires. The
# checks and their tiers come from the Authoring checklists at the end of
# plugins/kyberforge/docs/research/docs/microsoft-apm/{hooks,instructions,prompt}-primitive-schema.md,
# which trace each rule to the apm source that makes it matter.
#
# No boundary resolver and no word budgets: none of these files is routed on a
# description the way a skill is. A prompt's description IS model-visible on
# Claude, which is why it gets the two ADR-0029 SUGGESTIONs below — but whether
# a prompt body carries procedure that belongs in a skill is a judgment call the
# prompt flow makes by reading it, and deliberately has no heuristic here.
#
# Output follows lib-checks-agent.sh: FAIL lines on stderr, SUGGESTION and INFO
# on stdout, exit 1 on any FAIL, 0 otherwise.
#
# Consumed by: validate.sh, hook / instruction / prompt modes.
# shellcheck shell=bash
# shellcheck disable=SC2034
kyberforge_primitive_preflight() {
# Interpreter and library are checked separately so the message names the
# thing to install; see lib-checks-agent.sh for the history. PyYAML is needed
# for the two markdown kinds, and is required for hooks too so that one
# dependency set covers the whole suite rather than a hook audit passing on a
# machine where the next instruction audit cannot run.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: every primitive check parses the file; without python3 no check runs, and reporting that as a pass would be vacuous." >&2
echo " Fix: install python3." >&2
exit 2
fi
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
echo "Error: PyYAML is required but is not importable by python3." >&2
echo " Why: instruction and prompt frontmatter has to be parsed the way apm parses it; a hand-rolled reader would disagree with it on exactly the edge cases these checks exist for." >&2
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
exit 2
fi
}
IFS='' read -r -d '' KYBERFORGE_PRIMITIVE_PY <<'KYBERFORGE_PRIMITIVE' || true
import sys
import os
import re
import json
import yaml
for _stream in (sys.stdout, sys.stderr):
try:
_stream.reconfigure(encoding='utf-8')
except AttributeError: # pragma: no cover — Python < 3.7
pass
target = os.path.abspath(sys.argv[1])
kind = sys.argv[2]
fname = os.path.basename(target)
parent_dir = os.path.dirname(target)
failed = False
suggestions = []
def fail(msg):
global failed
failed = True
print(f"FAIL {msg}", file=sys.stderr)
def suggest(msg):
suggestions.append(msg)
def info(msg):
print(f"INFO {msg}")
def read_text(path):
try:
with open(path, encoding='utf-8') as f:
return f.read()
except UnicodeDecodeError as exc:
fail(f"not valid UTF-8 ({exc.reason} at byte {exc.start}) — apm reads primitives as UTF-8 — {fname}")
except OSError as exc:
fail(f"cannot be read ({exc.strerror}) — {fname}")
return None
def check_not_linked():
# apm's find_files_by_glob rejects symlinks and hardlinks (link count > 1),
# so a linked file is silently never deployed.
if os.path.islink(target):
fail(f"is a symlink — apm's discovery skips symlinks, so it is never deployed — {fname}")
return
try:
if os.stat(target).st_nlink > 1:
fail(f"is a hardlink (link count > 1) — apm's discovery rejects hardlinks, so it is never deployed — {fname}")
except OSError:
pass
def package_root_for(subdir):
# <pkg>/.apm/<subdir>/<file> -> <pkg>. Returns None for any other layout.
if os.path.basename(parent_dir) != subdir:
return None
apm_dir = os.path.dirname(parent_dir)
if os.path.basename(apm_dir) != '.apm':
return None
return os.path.dirname(apm_dir)
FRONTMATTER_RE = re.compile(r'\A---[ \t]*\r?\n(.*?)\r?\n---[ \t]*(?:\r?\n|\Z)', re.DOTALL)
def split_frontmatter(content):
"""Return (frontmatter dict | None, body, ok). ok is False on a parse FAIL."""
if content.startswith('\ufeff'):
content = content[1:]
m = FRONTMATTER_RE.match(content)
if not m:
fail(f"has no YAML frontmatter block (--- ... ---) — description and every other key live there — {fname}")
return None, content, False
try:
fm = yaml.safe_load(m.group(1))
except yaml.YAMLError as exc:
mark = getattr(exc, 'problem_mark', None)
where = f" at line {mark.line + 2}" if mark is not None else ''
fail(f"frontmatter is not valid YAML{where} — apm cannot read any key from it — {fname}")
return None, content[m.end():], False
if fm is None:
fm = {}
if not isinstance(fm, dict):
fail(f"frontmatter is not a YAML mapping — {fname}")
return None, content[m.end():], False
return fm, content[m.end():], True
def check_description(fm):
desc = fm.get('description')
if not isinstance(desc, str) or not desc.strip():
fail(f"'description' is missing or empty — apm does not require it, so nothing else will catch this — {fname}")
return None
return desc.strip()
# ---------------------------------------------------------------------------
# Hooks
# ---------------------------------------------------------------------------
ROUTING_TOKENS = ('copilot', 'vscode', 'cursor', 'claude', 'codex', 'gemini',
'antigravity', 'windsurf', 'kiro')
_TOK = '|'.join(ROUTING_TOKENS)
ROUTING_STEM_RE = re.compile(rf'^hooks-(?:{_TOK})$|(?:^|-)(?:{_TOK})-hooks$')
# Claude's rename map, 0.28.0: the only camelCase names that reach Claude as a
# native event. Any other camelCase name is deployed verbatim and never fires.
CLAUDE_MAPPED_CAMEL = {'preToolUse', 'postToolUse', 'sessionStart', 'agentStop'}
HOOK_COMMAND_KEYS = ('command', 'bash', 'powershell', 'windows', 'linux', 'osx')
ROOT_TOKENS = ('PLUGIN_ROOT', 'CLAUDE_PLUGIN_ROOT', 'CURSOR_PLUGIN_ROOT', 'KIRO_PLUGIN_ROOT')
ROOT_TOKEN_RE = re.compile(r'\$\{(' + '|'.join(ROOT_TOKENS) + r')\}')
def extract_script_refs(cmd):
"""Yield (kind, relpath, is_first_token) for each package-relative script
reference in a hook command string. kind is 'root' for a ${*_PLUGIN_ROOT}
token, 'rel' for a leading ./path."""
refs = []
for m in ROOT_TOKEN_RE.finditer(cmd):
start, end = m.start(), m.end()
opened = start > 0 and cmd[start - 1] in '"\''
quote = cmd[start - 1] if opened else None
rest = cmd[end:]
if opened and rest.startswith(quote):
# split-quoted form: "${PLUGIN_ROOT}"/scripts/my\ hook.sh
rest = rest[1:]
path = re.match(r'((?:\\.|[^\s"\'])*)', rest).group(1).replace('\\', '')
elif opened:
path = rest.split(quote, 1)[0]
else:
path = re.match(r'((?:\\.|[^\s"\'])*)', rest).group(1).replace('\\', '')
prefix = cmd[:start - 1] if opened else cmd[:start]
refs.append(('root', path.lstrip('/'), not prefix.strip()))
stripped = cmd.lstrip().lstrip('"\'')
if stripped.startswith('./'):
path = re.match(r'((?:\\.|[^\s"\'])*)', stripped).group(1).replace('\\', '')
refs.append(('rel', path, True))
return refs
def check_script(kind_, rel, first, pkg_root, where):
if not rel:
return
if '$' in rel or '`' in rel:
fail(f"script path '{rel}' contains '$' or a backtick — apm refuses to rewrite it for Claude — {where}")
return
candidates = []
if kind_ == 'root':
candidates.append(os.path.join(pkg_root, rel))
else:
candidates.append(os.path.join(parent_dir, rel))
candidates.append(os.path.join(pkg_root, rel))
real_root = os.path.realpath(pkg_root)
found = None
for c in candidates:
real = os.path.realpath(c)
if real != real_root and not real.startswith(real_root + os.sep):
fail(f"script '{rel}' resolves outside the package — apm confines hook scripts to the package root — {where}")
return
if os.path.isfile(c):
found = c
break
if found is None:
fail(f"script '{rel}' does not exist in the package — apm only warns, then deploys a hook that fails every time it fires — {where}")
return
if first and not os.access(found, os.X_OK):
fail(f"script '{rel}' is run directly but is not executable — chmod +x it, or invoke it through an interpreter — {where}")
def audit_hook():
check_not_linked()
stem = fname[:-len('.json')]
hooks_dir = os.path.basename(parent_dir)
if hooks_dir != 'hooks':
fail(f"is not directly in a hooks/ directory — apm discovers hook files only at .apm/hooks/*.json and hooks/*.json, non-recursively — {fname}")
if os.path.basename(os.path.dirname(parent_dir)) == '.apm':
pkg_root = os.path.dirname(os.path.dirname(parent_dir))
else:
pkg_root = os.path.dirname(parent_dir)
if not os.path.isfile(os.path.join(pkg_root, 'apm.yml')):
info(f"no apm.yml at the inferred package root {pkg_root} — script paths are resolved against it anyway — {fname}")
if ROUTING_STEM_RE.search(stem):
suggest(f"filename stem '{stem}' uses deprecated hook filename routing — name it plainly and narrow reach with target:/targets: in the package's apm.yml — {fname}")
content = read_text(target)
if content is None:
return
try:
doc = json.loads(content)
except json.JSONDecodeError as exc:
fail(f"is not valid JSON (line {exc.lineno}, column {exc.colno}) — apm skips an unparseable hook file silently — {fname}")
return
if not isinstance(doc, dict):
fail(f"top level is not a JSON object — {fname}")
return
if 'hooks' in doc:
events = doc['hooks']
if not isinstance(events, dict):
fail(f"'hooks' is not an object — apm skips the file, and the Copilot install fails outright — {fname}")
return
else:
stray = [k for k, v in doc.items() if not isinstance(v, list)]
if stray:
fail(f"naked settings-slice shape with non-list top-level key(s) {', '.join(sorted(stray))} — apm does not promote it, Claude gets nothing and Copilot gets a junk file; wrap events in {{\"hooks\": {{...}}}} — {fname}")
return
events = doc
if not events:
fail(f"contributes no hook entries — apm warns and deploys nothing — {fname}")
return
# A file is Claude-shaped when its entries nest handlers under "hooks" or
# its handlers use "command"; the flat bash/powershell form is Copilot's.
claude_shaped = False
shape_ok = True
for event, entries in events.items():
if not isinstance(entries, list):
fail(f"event '{event}' is not a list — the Copilot install fails on this payload — {fname}")
shape_ok = False
continue
for i, entry in enumerate(entries):
if not isinstance(entry, dict):
fail(f"event '{event}' entry {i} is not an object — the Copilot install fails on this payload — {fname}")
shape_ok = False
continue
if 'hooks' in entry:
claude_shaped = True
nested = entry['hooks']
if not isinstance(nested, list) or not all(isinstance(h, dict) for h in nested):
fail(f"event '{event}' entry {i}: nested 'hooks' is not a list of objects — the Copilot install fails on this payload — {fname}")
shape_ok = False
elif 'command' in entry:
claude_shaped = True
for event in events:
if not event.strip():
fail(f"empty event name — {fname}")
elif not any(c.isupper() for c in event):
fail(f"event '{event}' is all-lowercase — no target maps it and apm never warns, so it silently never fires — {fname}")
elif claude_shaped and event[0].islower() and event not in CLAUDE_MAPPED_CAMEL:
fail(f"event '{event}' is camelCase in a Claude-shaped file and Claude's map does not rename it — it deploys verbatim and never fires; write it in PascalCase — {fname}")
if not shape_ok:
return
uses_claude_token = False
for event, entries in events.items():
for i, entry in enumerate(entries):
handlers = entry['hooks'] if 'hooks' in entry else [entry]
for j, handler in enumerate(handlers):
where = f"{fname} {event}[{i}]" + (f".hooks[{j}]" if 'hooks' in entry else '')
for key in HOOK_COMMAND_KEYS:
cmd = handler.get(key)
if not isinstance(cmd, str):
continue
if '${CLAUDE_PLUGIN_ROOT}' in cmd:
uses_claude_token = True
for kind_, rel, first in extract_script_refs(cmd):
check_script(kind_, rel, first, pkg_root, where)
if uses_claude_token:
suggest(f"uses ${{CLAUDE_PLUGIN_ROOT}} — apm documents the target-neutral ${{PLUGIN_ROOT}}, which it rewrites identically for every target — {fname}")
# ---------------------------------------------------------------------------
# Instructions
# ---------------------------------------------------------------------------
INSTRUCTION_KEYS = {'description', 'applyTo', 'author', 'version'}
def audit_instruction():
check_not_linked()
stem = fname[:-len('.instructions.md')]
pkg_root = package_root_for('instructions')
if pkg_root is None:
fail(f"is not directly in a .apm/instructions/ directory — that is the authoring source; anything else is either deployed output or a legacy root file — {fname}")
else:
dup = os.path.join(pkg_root, fname)
if os.path.isfile(dup):
fail(f"stem '{stem}' also exists at the package root ({dup}) — both deploy to the same .claude/rules/{stem}.md, and one overwrites the other — {fname}")
content = read_text(target)
if content is None:
return
fm, body, ok = split_frontmatter(content)
if not ok:
return
check_description(fm)
if not body.strip():
fail(f"body is empty — apm deploys an empty rule without complaint — {fname}")
apply_to = fm.get('applyTo')
if apply_to is None or apply_to == '' or apply_to == []:
suggest(f"no applyTo — this loads into every session of every repo that installs the package; confirm always-on is intended, and that a rule for this repo alone is not really an AGENTS.md rule — {fname}")
elif isinstance(apply_to, list):
suggest(f"applyTo is a YAML list — Copilot receives the file verbatim and its handling of a list is unverified; use one comma-separated string — {fname}")
extra = sorted(k for k in fm if k not in INSTRUCTION_KEYS)
if extra:
suggest(f"frontmatter key(s) {', '.join(extra)} are read by no target and dropped on Claude — keep to description and applyTo (author, version optional) — {fname}")
# ---------------------------------------------------------------------------
# Prompts
# ---------------------------------------------------------------------------
PROMPT_KEYS = {'description', 'allowed-tools', 'model', 'argument-hint', 'input'}
PROMPT_CAMEL_ALIASES = {'allowedTools': 'allowed-tools', 'argumentHint': 'argument-hint'}
INPUT_NAME_RE = re.compile(r'^[A-Za-z][\w-]{0,63}$')
# apm's own rewrite pattern for ${input:x}, command_integrator.py.
INPUT_REF_RE = re.compile(r'\$\{\{?\s*input\s*:\s*([\w-]+)\s*\}?\}')
TRIGGER_RE = re.compile(r'\buse\s+(?:this\s+)?when\b', re.IGNORECASE)
PROMPT_DESC_SUGGEST_CHARS = 250
def prompt_input_names(spec):
"""Mirror apm's _extract_input_names, but FAIL on what it rejects or
misreads instead of warning. Returns the declared names."""
names = []
def accept(candidate):
if not isinstance(candidate, str):
fail(f"input entry {candidate!r} is not a string name — apm rejects it — {fname}")
return
s = candidate.strip()
if not s:
return
if not INPUT_NAME_RE.match(s):
fail(f"input name '{s}' does not match ^[A-Za-z][\\w-]{{0,63}}$ — apm rejects it, so the argument never exists — {fname}")
return
names.append(s)
if spec is None:
return names
if isinstance(spec, str):
accept(spec)
elif isinstance(spec, dict):
for k in spec:
accept(k)
elif isinstance(spec, list):
for item in spec:
if isinstance(item, dict):
if len(item) > 1:
keys = ', '.join(str(k) for k in item)
hint = (" — this is the upstream docs example's `- name: x` / `description:` form, which yields arguments [name, description]"
if 'name' in item else '')
fail(f"input entry {{{keys}}} is one map with several keys — apm reads every key as an argument name{hint}; write `- <name>: \"<desc>\"` — {fname}")
for k in item:
accept(k)
else:
accept(item)
else:
fail(f"input is neither a name, a list nor a map — apm extracts no arguments from it — {fname}")
return names
def audit_prompt():
check_not_linked()
stem = fname[:-len('.prompt.md')]
segs = stem.replace('\\', '/').split('/')
if not stem.strip() or any(s in ('.', '..', '') for s in segs) or '/' in stem.replace('\\', '/'):
fail(f"name '{stem}' is not a safe path segment — apm's validate_path_segments rejects it — {fname}")
pkg_root = package_root_for('prompts')
if pkg_root is None:
fail(f"is not directly in a .apm/prompts/ directory — that is the authoring source; anything else is either deployed output or a legacy root file — {fname}")
else:
dup = os.path.join(pkg_root, fname)
if os.path.isfile(dup):
fail(f"name '{stem}' also exists at the package root ({dup}) — both deploy as /{stem}, and they collide — {fname}")
content = read_text(target)
if content is None:
return
fm, body, ok = split_frontmatter(content)
if not ok:
return
desc = check_description(fm)
if desc is not None:
if len(desc) > PROMPT_DESC_SUGGEST_CHARS:
suggest(f"description is {len(desc)} characters (> {PROMPT_DESC_SUGGEST_CHARS}) — it is one user-facing sentence (ADR-0029) — {fname}")
if TRIGGER_RE.search(desc):
suggest(f"description carries a 'Use when' trigger clause — a prompt is user-triggered (ADR-0029); a trigger clause invites the model to route to it on Claude — {fname}")
for camel, kebab in PROMPT_CAMEL_ALIASES.items():
if camel in fm:
suggest(f"'{camel}' — use the kebab-case spelling '{kebab}' apm documents — {fname}")
extra = sorted(k for k in fm if k not in PROMPT_KEYS and k not in PROMPT_CAMEL_ALIASES)
if extra:
suggest(f"frontmatter key(s) {', '.join(extra)} are dropped on Claude (it keeps only {', '.join(sorted(PROMPT_KEYS))}) — keep them only if the Copilot-only behaviour is intended — {fname}")
declared = prompt_input_names(fm.get('input'))
used = []
for m in INPUT_REF_RE.finditer(body):
if m.group(1) not in used:
used.append(m.group(1))
if used and not declared:
fail(f"body uses {', '.join('${input:' + u + '}' for u in used)} but no input: is declared — apm rewrites references only when input: names them, so Claude receives the literal text — {fname}")
else:
for u in used:
if u not in declared:
fail(f"body uses ${{input:{u}}} but input: does not declare '{u}' — {fname}")
for d in declared:
if d not in used:
fail(f"input '{d}' is declared but the body never uses ${{input:{d}}} — the user is asked for an argument that goes nowhere — {fname}")
if declared and ('argument-hint' in fm or 'argumentHint' in fm):
suggest(f"argument-hint is set alongside input: — apm synthesises the hint from input: names; drop it unless that form is inadequate — {fname}")
if kind == 'hook':
audit_hook()
elif kind == 'instruction':
audit_instruction()
elif kind == 'prompt':
audit_prompt()
else:
print(f"Error: unknown primitive kind '{kind}'", file=sys.stderr)
sys.exit(2)
for s in suggestions:
print(f"SUGGESTION {s}")
sys.exit(1 if failed else 0)
KYBERFORGE_PRIMITIVE
KYBERFORGE_PRIMITIVE_PY="${KYBERFORGE_PRIMITIVE_PY%$'\n'}"

50
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh
View File

@@ -2,13 +2,16 @@
set -euo pipefail set -euo pipefail
# The ONE entry point for structural validation. It auto-detects whether the # The ONE entry point for structural validation. It auto-detects whether the
# target is a skill directory or an agent definition file and runs the matching # target is a skill directory, an agent definition file, or one of the three apm
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh # primitives with no container of their own (a hook, an instruction or a prompt)
# and are unchanged from the skill-audit / agent-audit scripts they came from. # and runs the matching check suite. The skill and agent suites live in
# lib-checks-skill.sh and lib-checks-agent.sh and are unchanged from the
# skill-audit / agent-audit scripts they came from; the primitive suite lives in
# lib-checks-primitive.sh.
# The ADR-0020 boundary resolver both of them need is sourced once, from # The ADR-0020 boundary resolver both of them need is sourced once, from
# lib-boundary-resolver.sh, instead of being embedded twice. # lib-boundary-resolver.sh, instead of being embedded twice.
# #
# Detection never guesses. A target that matches neither shape is a hard exit 2 # Detection never guesses. A target that matches no shape is a hard exit 2
# naming the mismatch, because the alternative — picking a mode and letting the # naming the mismatch, because the alternative — picking a mode and letting the
# suite fail on its own terms — reports a skill-shaped finding about an agent # suite fail on its own terms — reports a skill-shaped finding about an agent
# file, or the reverse, and sends the reader after the wrong problem. # file, or the reverse, and sends the reader after the wrong problem.
@@ -115,17 +118,22 @@ _kf_require_lib() {
usage() { usage() {
cat <<EOF cat <<EOF
Usage: validate.sh <skill-dir | agent-file> Usage: validate.sh <skill-dir | agent-file | hook-file | instruction-file | prompt-file>
Validate a skill directory against the agentskills.io specification, or an agent Validate a skill directory against the agentskills.io specification, an agent
definition file against the agent definition spec. The mode is detected from the definition file against the agent definition spec, or an apm hook, instruction or
target: prompt file against what apm 0.28.0 actually deploys. The mode is detected from
the target:
skill mode the target is a directory (a skill directory contains SKILL.md), skill mode the target is a directory (a skill directory contains SKILL.md),
or the target IS a SKILL.md file. or the target IS a SKILL.md file.
agent mode the target is a *.agent.md file, or a *.md file whose parent agent mode the target is a *.agent.md file, or a *.md file whose parent
directory is named 'agents' (.apm/agents, .claude/agents, directory is named 'agents' (.apm/agents, .claude/agents,
.github/agents, .copilot/agents). .github/agents, .copilot/agents).
hook mode the target is a *.json file directly under a hooks/
directory (.apm/hooks, or a package's root hooks/).
instruction mode the target is a *.instructions.md file.
prompt mode the target is a *.prompt.md file.
Skill mode audits the directory named by <skill-dir>. Skill mode audits the directory named by <skill-dir>.
@@ -143,7 +151,7 @@ Arguments:
Exit codes: Exit codes:
0 All checks passed (may include SUGGESTIONs) 0 All checks passed (may include SUGGESTIONs)
1 One or more checks failed 1 One or more checks failed
2 Nothing was audited (no argument, the target matches neither shape, the 2 Nothing was audited (no argument, the target matches no shape, the
target does not exist, an unrecognized file extension, a missing target does not exist, an unrecognized file extension, a missing
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
beside this script) beside this script)
@@ -156,7 +164,7 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
fi fi
if [[ $# -lt 1 ]]; then if [[ $# -lt 1 ]]; then
echo "Error: a skill directory or an agent file is required." >&2 echo "Error: a skill directory, an agent file, or a hook, instruction or prompt file is required." >&2
echo "" >&2 echo "" >&2
usage >&2 usage >&2
exit 2 exit 2
@@ -209,12 +217,21 @@ elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
TARGET="$(_kf_dirname "$TARGET")" TARGET="$(_kf_dirname "$TARGET")"
elif [[ "$TARGET_BASE" == *.agent.md ]]; then elif [[ "$TARGET_BASE" == *.agent.md ]]; then
MODE=agent MODE=agent
# The two primitive suffixes are tested before the agents/-parent rule: a
# *.prompt.md or *.instructions.md file is that primitive wherever it sits, and
# the parent-name rule would otherwise claim one that happened to sit in agents/.
elif [[ "$TARGET_BASE" == *.instructions.md ]]; then
MODE=instruction
elif [[ "$TARGET_BASE" == *.prompt.md ]]; then
MODE=prompt
elif [[ "$TARGET_BASE" == *.json && "$TARGET_PARENT" == "hooks" && ! -d "$TARGET" ]]; then
MODE=hook
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
MODE=agent MODE=agent
else else
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2 echo "Error: '$TARGET' matches no auditable shape." >&2
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2 echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents); hook mode needs a .json file directly under a hooks/ directory; instruction and prompt modes need a *.instructions.md or *.prompt.md file. Picking a mode anyway would audit this path against the wrong spec." >&2
echo " Fix: pass one of those two shapes." >&2 echo " Fix: pass one of those shapes." >&2
exit 2 exit 2
fi fi
@@ -250,6 +267,13 @@ $KYBERFORGE_RESOLVER_PY
$KYBERFORGE_AGENT_BODY_PY" $KYBERFORGE_AGENT_BODY_PY"
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$? python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
;; ;;
hook | instruction | prompt)
_kf_require_lib lib-checks-primitive.sh
# shellcheck source=lib-checks-primitive.sh
. "$SCRIPT_DIR/lib-checks-primitive.sh"
kyberforge_primitive_preflight
python3 -u - "$TARGET" "$MODE" <<< "$KYBERFORGE_PRIMITIVE_PY" || RC=$?
;;
esac esac
exit "$RC" exit "$RC"

7
plugins/kyberforge/.apm/skills/factory-audit/tests/README.md
View File

@@ -36,13 +36,16 @@ bats plugins/kyberforge/.apm/skills/factory-audit/tests/
| `validate-agent.bats` | `scripts/validate.sh` against agent files | | `validate-agent.bats` | `scripts/validate.sh` against agent files |
| `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories | | `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories |
| `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files | | `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files |
| `validate-primitive.bats` | `scripts/validate.sh` against apm hook, instruction and prompt files |
## Two scripts, four suites ## Two scripts, five suites
`factory-audit` merges what were two skills — `skill-audit` and `agent-audit` — `factory-audit` merges what were two skills — `skill-audit` and `agent-audit` —
each of which shipped its own `validate.sh` and `validate-provenance.sh`. The each of which shipped its own `validate.sh` and `validate-provenance.sh`. The
merged skill has **one** of each. Every suite here invokes one of those two merged skill has **one** of each. Every suite here invokes one of those two
scripts; the four files are two scripts × two artifact types, not four scripts. scripts; the four skill and agent files are two scripts × two artifact types, not four scripts.
`validate-primitive.bats` is a fifth suite over the same `scripts/validate.sh`, for hooks,
instructions and prompts, which have no provenance mode and so no provenance suite.
`validate-skill.bats` and `validate-agent.bats` run the same `validate-skill.bats` and `validate-agent.bats` run the same
`scripts/validate.sh` and differ only in the fixtures they point it at. The two `scripts/validate.sh` and differ only in the fixtures they point it at. The two

316
plugins/kyberforge/.apm/skills/factory-audit/tests/validate-primitive.bats Normal file
View File

@@ -0,0 +1,316 @@
#!/usr/bin/env bats
# scripts/validate.sh against the three apm primitives with no container of
# their own: hooks, instructions and prompts. Each rule under test traces to
# the Authoring checklist in
# plugins/kyberforge/docs/research/docs/microsoft-apm/<kind>-primitive-schema.md.
setup() {
REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../../../../../" && pwd)"
load "$REPO_ROOT/tests/test_helper/bats-support/load"
load "$REPO_ROOT/tests/test_helper/bats-assert/load"
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate.sh"
TMPDIR="$(mktemp -d)"
PKG="$TMPDIR/pkg"
mkdir -p "$PKG/.apm/hooks/scripts" "$PKG/.apm/instructions" "$PKG/.apm/prompts"
cat > "$PKG/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: hybrid
EOF
printf '#!/usr/bin/env bash\nexit 0\n' > "$PKG/.apm/hooks/scripts/check.sh"
chmod +x "$PKG/.apm/hooks/scripts/check.sh"
# Helper: write <content> as hook file <name> under .apm/hooks/.
write_hook() {
printf '%s\n' "$2" > "$PKG/.apm/hooks/$1"
}
# Helper: write an instruction <stem> with raw <frontmatter> and <body>.
write_instruction() {
printf -- '---\n%s\n---\n\n%s\n' "$2" "$3" > "$PKG/.apm/instructions/$1.instructions.md"
}
# Helper: write a prompt <stem> with raw <frontmatter> and <body>.
write_prompt() {
printf -- '---\n%s\n---\n\n%s\n' "$2" "$3" > "$PKG/.apm/prompts/$1.prompt.md"
}
}
teardown() {
rm -rf "$TMPDIR"
}
# ---------------------------------------------------------------------------
# Dispatch
# ---------------------------------------------------------------------------
@test "dispatch: a .json file outside a hooks/ directory matches no shape (exit 2)" {
printf '{}\n' > "$PKG/settings.json"
run bash "$SCRIPT" "$PKG/settings.json"
assert_equal "$status" 2
assert_output --partial "matches no auditable shape"
}
@test "dispatch: an *.instructions.md under an agents/ directory takes the instruction flow, not the agent flow" {
mkdir -p "$PKG/.apm/agents"
printf -- '---\ndescription: x\n---\n\nbody\n' > "$PKG/.apm/agents/x.instructions.md"
run bash "$SCRIPT" "$PKG/.apm/agents/x.instructions.md"
assert_failure
assert_output --partial "is not directly in a .apm/instructions/ directory"
refute_output --partial "counterpart"
}
# ---------------------------------------------------------------------------
# Hooks
# ---------------------------------------------------------------------------
@test "hook: canonical nested shape with \${PLUGIN_ROOT} passes clean" {
write_hook hooks.json '{"hooks":{"PreToolUse":[{"matcher":"Bash","hooks":[{"type":"command","command":"${PLUGIN_ROOT}/.apm/hooks/scripts/check.sh","timeout":10}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_success
refute_output --partial "FAIL"
refute_output --partial "SUGGESTION"
}
@test "hook: invalid JSON is a FAIL" {
write_hook hooks.json '{"hooks": {'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "is not valid JSON"
}
@test "hook: an event value that is not a list is a FAIL" {
write_hook hooks.json '{"hooks":{"PreToolUse":{"hooks":[]}}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "event 'PreToolUse' is not a list"
}
@test "hook: a naked slice with a stray scalar key is a FAIL" {
write_hook hooks.json '{"description":"x","PreToolUse":[{"hooks":[{"type":"command","command":"true"}]}]}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "naked settings-slice shape"
}
@test "hook: an all-lowercase event is a FAIL" {
write_hook hooks.json '{"hooks":{"stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "event 'stop' is all-lowercase"
}
@test "hook: camelCase userPromptSubmit in a Claude-shaped file is a FAIL; mapped sessionStart is not" {
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"hooks":[{"type":"command","command":"true"}]}],"sessionStart":[{"hooks":[{"type":"command","command":"true"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "event 'userPromptSubmit' is camelCase"
refute_output --partial "event 'sessionStart'"
}
@test "hook: a flat Copilot-shaped file may use camelCase events" {
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"type":"command","bash":"true","timeoutSec":5}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_success
}
@test "hook: a referenced script that does not exist is a FAIL" {
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/.apm/hooks/scripts/missing.sh"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "script '.apm/hooks/scripts/missing.sh' does not exist"
}
@test "hook: a directly-run script without the executable bit is a FAIL; the same script through an interpreter is not" {
chmod -x "$PKG/.apm/hooks/scripts/check.sh"
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"./scripts/check.sh"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "is run directly but is not executable"
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"bash ${PLUGIN_ROOT}/.apm/hooks/scripts/check.sh"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_success
}
@test "hook: a script path escaping the package is a FAIL" {
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/../outside.sh"}]}]}}'
touch "$TMPDIR/outside.sh"
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_failure
assert_output --partial "resolves outside the package"
}
@test "hook: \${CLAUDE_PLUGIN_ROOT} is a SUGGESTION, not a FAIL" {
write_hook hooks.json '{"hooks":{"SessionStart":[{"matcher":"startup","hooks":[{"type":"command","command":"${CLAUDE_PLUGIN_ROOT}/.apm/hooks/scripts/check.sh","timeout":5}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
assert_success
assert_output --partial "SUGGESTION uses \${CLAUDE_PLUGIN_ROOT}"
}
@test "hook: a deprecated filename-routing stem is a SUGGESTION" {
write_hook claude-hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
run bash "$SCRIPT" "$PKG/.apm/hooks/claude-hooks.json"
assert_success
assert_output --partial "deprecated hook filename routing"
}
@test "hook: a symlinked hook file is a FAIL" {
write_hook real.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
ln -s real.json "$PKG/.apm/hooks/link.json"
run bash "$SCRIPT" "$PKG/.apm/hooks/link.json"
assert_failure
assert_output --partial "is a symlink"
}
# ---------------------------------------------------------------------------
# Instructions
# ---------------------------------------------------------------------------
@test "instruction: description, applyTo and a body pass clean" {
write_instruction python 'description: Python style rules
applyTo: "**/*.py"' 'Use type hints on public functions.'
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "SUGGESTION"
}
@test "instruction: missing description is a FAIL" {
write_instruction python 'applyTo: "**/*.py"' 'Use type hints.'
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_failure
assert_output --partial "'description' is missing or empty"
}
@test "instruction: an empty body is a FAIL" {
write_instruction python 'description: x
applyTo: "**/*.py"' ''
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_failure
assert_output --partial "body is empty"
}
@test "instruction: invalid frontmatter YAML is a FAIL" {
write_instruction python 'description: [unclosed' 'body'
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_failure
assert_output --partial "frontmatter is not valid YAML"
}
@test "instruction: no applyTo is a SUGGESTION (deliberate always-on), not a FAIL" {
write_instruction general 'description: General rules' 'Be kind.'
run bash "$SCRIPT" "$PKG/.apm/instructions/general.instructions.md"
assert_success
assert_output --partial "SUGGESTION no applyTo"
}
@test "instruction: a YAML-list applyTo and unread keys are SUGGESTIONs" {
write_instruction python 'description: x
applyTo:
- "**/*.py"
name: python' 'body'
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_success
assert_output --partial "applyTo is a YAML list"
assert_output --partial "frontmatter key(s) name"
}
@test "instruction: the same stem at the package root is a FAIL" {
write_instruction python 'description: x
applyTo: "**/*.py"' 'body'
cp "$PKG/.apm/instructions/python.instructions.md" "$PKG/python.instructions.md"
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
assert_failure
assert_output --partial "also exists at the package root"
}
# ---------------------------------------------------------------------------
# Prompts
# ---------------------------------------------------------------------------
@test "prompt: declared and used inputs with a plain description pass clean" {
write_prompt review-pr 'description: Review a pull request with gitea-prs and factory-audit, then summarize.
input:
- pr_number: "The PR to review"' 'Review PR ${input:pr_number} with gitea-prs.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "SUGGESTION"
}
@test "prompt: missing description is a FAIL" {
write_prompt review-pr 'model: sonnet' 'Review the PR.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_failure
assert_output --partial "'description' is missing or empty"
}
@test "prompt: the upstream docs' - name: x / description: form is a FAIL" {
write_prompt review-pr 'description: Review a PR.
input:
- name: pr_number
description: The PR' 'Review ${input:pr_number}.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_failure
assert_output --partial "yields arguments [name, description]"
}
@test "prompt: an invalid input name is a FAIL" {
write_prompt review-pr 'description: Review a PR.
input: [1pr]' 'Review.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_failure
assert_output --partial "input name '1pr' does not match"
}
@test "prompt: an undeclared \${input:x} and an unused input are both FAILs" {
write_prompt review-pr 'description: Review a PR.
input: [pr_number]' 'Review ${input:branch}.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_failure
assert_output --partial "input: does not declare 'branch'"
assert_output --partial "input 'pr_number' is declared but the body never uses"
}
@test "prompt: \${input:x} with no input: declared is a FAIL" {
write_prompt review-pr 'description: Review a PR.' 'Review ${input:pr_number}.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_failure
assert_output --partial "but no input: is declared"
}
@test "prompt: a trigger clause, an over-long description, dropped keys and camelCase aliases are SUGGESTIONs" {
local long
long="Use when the user wants a PR reviewed. $(printf 'x%.0s' $(seq 1 240))"
write_prompt review-pr "description: $long
mode: agent
allowedTools: Bash" 'Review the PR.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_success
assert_output --partial "'Use when' trigger clause"
assert_output --partial "characters (> 250)"
assert_output --partial "frontmatter key(s) mode are dropped on Claude"
assert_output --partial "'allowedTools' — use the kebab-case spelling"
}
@test "prompt: argument-hint alongside input: is a SUGGESTION" {
write_prompt review-pr 'description: Review a PR.
argument-hint: <pr>
input: [pr_number]' 'Review ${input:pr_number}.'
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_success
assert_output --partial "argument-hint is set alongside input:"
}
@test "prompt: no procedure heuristic — a long, stepped body is not a script finding" {
write_prompt review-pr 'description: Review a PR.' "## Step 1
$(printf 'line\n%.0s' $(seq 1 80))
## Gotchas
- x"
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
assert_success
refute_output --partial "SUGGESTION"
}

2
plugins/kyberforge/apm.yml
View File

@@ -1,5 +1,5 @@
name: kyberforge name: kyberforge
version: 2.0.1 version: 2.1.0
description: Skills and agents for creating, maintaining, and managing an apm plugin marketplace for Claude Code and GitHub Copilot. description: Skills and agents for creating, maintaining, and managing an apm plugin marketplace for Claude Code and GitHub Copilot.
author: author:
name: Defame1297 name: Defame1297

2
tests/test-vale-wrap.sh
View File

@@ -1382,6 +1382,8 @@ plugins/demo/.apm/agents/demo.md|[**/agents/*.md]|isolating
.claude/agents/demo.md|[**/agents/*.md]|isolating .claude/agents/demo.md|[**/agents/*.md]|isolating
plugins/demo/.apm/agents/demo.agent.md|[**/*.agent.md]|overlapping plugins/demo/.apm/agents/demo.agent.md|[**/*.agent.md]|overlapping
copilot/demo.agent.md|[**/*.agent.md]|isolating copilot/demo.agent.md|[**/*.agent.md]|isolating
plugins/demo/.apm/instructions/demo.instructions.md|[**/*.instructions.md]|isolating
plugins/demo/.apm/prompts/demo.prompt.md|[**/*.prompt.md]|isolating
EOF_PROBE28 EOF_PROBE28
)" )"