feat(factory-audit): audit hooks, instructions and prompts

factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.

- Hook: JSON/shape/event-list checks mirroring the Copilot payload
  validator, never-firing event casing, missing/escaping/non-executable
  scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
  (SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
  (FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
  upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
  ADR-0029 description length and trigger clause, dropped keys,
  camelCase aliases, argument-hint with input (SUGGESTION). Whether a
  prompt carries procedure is judgment in prompt-flow.md, not a script
  heuristic.

Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.

Refs #94

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
2026-09-28 17:02:04 +00:00
parent 701e96d4b3
commit 70210d6a7e
14 changed files with 1062 additions and 29 deletions

View File

@@ -0,0 +1,55 @@
---
source_keys:
- apm-cli-installed-source
- apm-docs-llms-full
---
# Instruction Flow
Steps 1 to 3 for an apm instruction — the target Step 0 matched as a `*.instructions.md` file.
Work them in order, then return to `SKILL.md` Step 4 to report.
## Gotchas
- `apm compile --validate` is not a gate. Every message `Instruction.validate()` produces is a warning, and it reports success on a file with no description and an empty body — never cite it as evidence against a finding.
- `description` never reaches Claude, and it is index text elsewhere, never a routing description. Do not hold it to the skill description contract: no trigger clause, no boundary clause. A Vale `Kyberforge.DescriptionOpener` alert here means rewrite it as a plain statement of what the rule covers.
## Step 1 — Deterministic checks
Resolve both paths against this skill's own directory. Run exactly:
```bash
bash scripts/validate.sh <instruction-file>
bash scripts/vale-wrap.sh <instruction-file>
```
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
There is no provenance step: an instruction carries no `source_keys`.
## Step 2 — Read the instruction and its context
Read the file, the package's `apm.yml`, and the repo's root `AGENTS.md`. For a scoped file, list the tracked files its `applyTo` matches (`rtk git ls-files` filtered by the glob).
## Step 3 — Qualitative audit
Cite file and line for every finding.
**scope** — an instruction applies when files matching `applyTo` are touched; with no `applyTo` it loads into every session of every repo that installs the package.
- FAIL: an always-on file whose content is a rule for this repo alone — it belongs in `AGENTS.md`, which is the repo's single always-on source, not in a package that ships it to every consumer.
- FAIL: an `applyTo` glob that matches no tracked file in any repo the package plausibly targets, so the rule never loads.
- SUGGESTION: an always-on file whose content is really file-type specific — narrow it with `applyTo`.
- SUGGESTION: a glob much broader than the content (`**` for a rule about Python).
**description**
- SUGGESTION: the description does not say what the rule covers, or contradicts the body. Any rationale Claude readers need belongs in the body, because Claude drops the description.
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
```text
Checked: structure · prose · scope · description
```