Files

Defame1297 70210d6a7e feat(factory-audit): audit hooks, instructions and prompts

factory-audit gains three Step 0 rows and flows for the apm primitives
that have no container of their own: a .json file under hooks/, a
*.instructions.md and a *.prompt.md. apm validates almost none of them
(invalid hook JSON is skipped silently, instruction validate() only
warns, input: names are never checked against ${input:x}), so the
deterministic checks live in a new scripts/lib-checks-primitive.sh,
wired into validate.sh's path-shape detection. Each check and tier
traces to the Authoring checklists in the microsoft-apm research docs.

- Hook: JSON/shape/event-list checks mirroring the Copilot payload
  validator, never-firing event casing, missing/escaping/non-executable
  scripts (FAIL); deprecated filename routing and ${CLAUDE_PLUGIN_ROOT}
  (SUGGESTION).
- Instruction: location, frontmatter, description, body, stem clash
  (FAIL); missing or list applyTo and unread keys (SUGGESTION).
- Prompt: location/name, frontmatter, description, input names, the
  upstream `- name: x` docs bug, declared-vs-used ${input:x} (FAIL);
  ADR-0029 description length and trigger clause, dropped keys,
  camelCase aliases, argument-hint with input (SUGGESTION). Whether a
  prompt carries procedure is judgment in prompt-flow.md, not a script
  heuristic.

Vale now lints *.instructions.md and *.prompt.md with the Kyberforge
style; test-vale-wrap.sh gains their probe rows. New
tests/validate-primitive.bats (31 cases). kyberforge 2.0.1 -> 2.1.0 with
the executables.allow key, catalog 0.5.1 -> 0.5.2, marketplace.json
regenerated.

Refs #94

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi

2026-09-28 17:02:04 +00:00

2.8 KiB

Raw Blame History

source_keys

apm-cli-installed-source

apm-docs-llms-full

Instruction Flow

Steps 1 to 3 for an apm instruction — the target Step 0 matched as a *.instructions.md file. Work them in order, then return to SKILL.md Step 4 to report.

Gotchas

apm compile --validate is not a gate. Every message Instruction.validate() produces is a warning, and it reports success on a file with no description and an empty body — never cite it as evidence against a finding.
description never reaches Claude, and it is index text elsewhere, never a routing description. Do not hold it to the skill description contract: no trigger clause, no boundary clause. A Vale Kyberforge.DescriptionOpener alert here means rewrite it as a plain statement of what the rule covers.

Step 1 — Deterministic checks

Resolve both paths against this skill's own directory. Run exactly:

bash scripts/validate.sh <instruction-file>
bash scripts/vale-wrap.sh <instruction-file>

validate.sh findings become the ### Structure dimension, FAILs and SUGGESTIONs both, at the tier the script assigned. It exits 0 with no FAIL, 1 on real findings, 2 when it never ran — report that as ### Structure unverified, quoting the stderr reason.

vale-wrap.sh applies the bundled Kyberforge style. Every alert is a FAIL under ### Prose, cited by rule ID; do not re-derive it by judgment. 0 files scanned means NOT RUN, not clean — say so and judge prose by reading.

There is no provenance step: an instruction carries no source_keys.

Step 2 — Read the instruction and its context

Read the file, the package's apm.yml, and the repo's root AGENTS.md. For a scoped file, list the tracked files its applyTo matches (rtk git ls-files filtered by the glob).

Step 3 — Qualitative audit

Cite file and line for every finding.

scope — an instruction applies when files matching applyTo are touched; with no applyTo it loads into every session of every repo that installs the package.

FAIL: an always-on file whose content is a rule for this repo alone — it belongs in AGENTS.md, which is the repo's single always-on source, not in a package that ships it to every consumer.
FAIL: an applyTo glob that matches no tracked file in any repo the package plausibly targets, so the rule never loads.
SUGGESTION: an always-on file whose content is really file-type specific — narrow it with applyTo.
SUGGESTION: a glob much broader than the content (** for a rule about Python).

description

SUGGESTION: the description does not say what the rule covers, or contradicts the body. Any rationale Claude readers need belongs in the body, because Claude drops the description.

Then return to SKILL.md Step 4, opening the report with this coverage line:

Checked: structure · prose · scope · description

2.8 KiB Raw Blame History