fix(factory-audit): flag unbraced plugin-root tokens and close hook check gaps
Why: PR #144 review round 4 reproduced hooks referencing $PLUGIN_ROOT or ${PLUGIN_ROOT} without a path separator passing the audit, although apm only rewrites ${TOKEN}/ and the deployed hook points nowhere. - FAIL unbraced or unseparated plugin-root tokens - check the exec bit for scripts run via an interpreter -c string - skip env NAME=value prefixes when locating bare relative paths - correct input: and empty-frontmatter messages, depth-walk applyTo braces - INFO on unrecognised targets; failing-case tests for untested checks - document tiers, blind spots and crash exit 2; drop rtk from portable flow - restore the after-a-hand-edit trigger; pin upstream apm source URL Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
@@ -25,17 +25,41 @@ Resolve the path against this skill's own directory. Run exactly:
|
||||
bash scripts/validate.sh <hook-file>
|
||||
```
|
||||
|
||||
Its findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: JSON validity, the wrapped-or-naked shape, event lists and nested handler lists (the checks whose failure makes the Copilot install fail), a file contributing no entries (no events, only empty event lists, or an entry with no handler), event names that never fire, unfilled `FILL IN` or `FILL_IN_` template placeholders, a symlinked file or one under apm's deployed output rather than package source, referenced scripts that are missing, outside the package, not executable when run directly, or referenced by an absolute, bare relative, `../`, split-quoted, space-containing, or `$`/backtick-containing path apm will not bundle correctly, deprecated filename routing, and `${CLAUDE_PLUGIN_ROOT}` where `${PLUGIN_ROOT}` would do. Script references are read with apm 0.28.0's own patterns: `${PLUGIN_ROOT}/…` only when the path follows the token directly, up to the first whitespace or quote, and `./…` anywhere in the command. A `./` or `../` match is held to the script rules — a FAIL when missing — only in command position (the first token, or the first operand after `bash`, `sh`, `zsh`, `python`, `python3`, `node`, `pwsh`, `ruby` or `perl`, past its options such as `-e` or `-u`; after a `sh`-family `-c`, the first token of the command string; inline code such as `python3 -c` has none), when it ends in a script extension, or when it names a package entry that is not a file; any other match (`npx prettier --check ./src`, `printf '.\n'`) is at most a SUGGESTION, because apm only warns and it runs against the consumer's working directory as meant. Absolute and bare relative script paths are checked in the same command positions. Each event is judged per target the package root's `apm.yml` deploys to — no `target:`/`targets:`, `all`, or no `apm.yml` means every hook target — after apm's rename map for that target: it FAILs when a target with a published event list (Claude, Copilot) does not fire the renamed name, whatever its casing. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason. An INFO line is observational: report it under `### Structure` and count it in `· P info`.
|
||||
Its findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned.
|
||||
|
||||
Checks:
|
||||
|
||||
- JSON validity and UTF-8 encoding; a top level that is not a JSON object; the wrapped-or-naked shape; event lists and nested handler lists (the checks whose failure makes the Copilot install fail).
|
||||
- A file contributing no entries (no events, only empty event lists, or an entry with no handler), an empty event name, and event names that never fire.
|
||||
- Unfilled `FILL IN` or `FILL_IN_` template placeholders.
|
||||
- A symlinked file, or one under apm's deployed output or outside any package rather than package source (exit 1, a finding, not exit 2).
|
||||
- Referenced scripts that are missing, outside the package, not executable when run directly, or referenced by an absolute, bare relative, `../`, split-quoted, space-containing, or `$`/backtick-containing path apm will not bundle correctly.
|
||||
- A plugin-root token apm never rewrites: unbraced (`$PLUGIN_ROOT/x.sh`, `$CLAUDE_PLUGIN_ROOT`), or braced but not directly followed by `/` or `\` (`cd ${PLUGIN_ROOT} && …`).
|
||||
- Deprecated filename routing, and `${CLAUDE_PLUGIN_ROOT}` where `${PLUGIN_ROOT}` would do.
|
||||
- INFO: no `apm.yml` at an inferred `.apm/` package root, or a `targets:` naming no hook target apm recognises (`claude-code`), which leaves event names checked against no harness.
|
||||
|
||||
Script reference rules:
|
||||
|
||||
- Script references are read with apm 0.28.0's own patterns: `${PLUGIN_ROOT}/…` only when the path follows the token directly, up to the first whitespace or quote, and `./…` anywhere in the command.
|
||||
- A `./` or `../` match is held to the script rules — a FAIL when missing — only in command position, when it ends in a script extension, or when it names a package entry that is not a file; any other match (`npx prettier --check ./src`, `printf '.\n'`) is at most a SUGGESTION, because apm only warns and it runs against the consumer's working directory as meant.
|
||||
- Command position is the first token past any `NAME=value` assignments and `env` with its options, or the first operand after `bash`, `sh`, `zsh`, `python`, `python3`, `node`, `pwsh`, `ruby` or `perl`, past its options such as `-e` or `-u`; after a `sh`-family `-c`, the first token of the command string; inline code such as `python3 -c` has none. Absolute and bare relative script paths are checked in the same positions.
|
||||
- The exec bit is required of a script that runs directly: the first token, or the first token of a `-c` command string (`bash -c "${PLUGIN_ROOT}/x.sh"`). Through an interpreter it is not.
|
||||
- Each event is judged per target the package root's `apm.yml` deploys to — no `target:`/`targets:`, `all`, or no `apm.yml` means every hook target — after apm's rename map for that target: it FAILs when a target with a published event list (Claude, Copilot) does not fire the renamed name, whatever its casing.
|
||||
|
||||
Exit codes: **0** with no FAIL, **1** on real findings, **2** when it never ran, including a crash inside the checks — report that as `### Structure` unverified, quoting the stderr reason. An INFO line is observational: report it under `### Structure` and count it in `· P info`.
|
||||
|
||||
The command parser is a heuristic, not a shell. Known blind spots: a script after `&&`, `;` or a pipe inside one command, `env -S`, command substitution, and a quoted `-c` string that ends before the script are not in command position, so a bad reference there is at most a SUGGESTION or unseen. Read every command in Step 2 rather than taking a clean script run as proof.
|
||||
|
||||
There is no provenance and no Vale step: a hook carries no `source_keys` and no prose.
|
||||
|
||||
Five tiers deliberately differ from `primitive-author`'s checklist or the research's. Do not re-tier them by judgment:
|
||||
Six tiers deliberately differ from `primitive-author`'s checklist or the research's. Do not re-tier them by judgment:
|
||||
|
||||
- A hook file directly under a package-root `hooks/` — beside the package's `apm.yml`, or beside a `plugin.json` at any location apm's `find_plugin_json` reads (root, `.github/plugin/`, `.claude-plugin/`, `.cursor-plugin/`) — passes. apm discovers both `.apm/hooks/` and `hooks/`, installs a Claude plugin with no `apm.yml`, and this audit may target a third-party package; `primitive-author` authors only in `.apm/hooks/`. Any other `hooks/` directory (`.github/hooks/`, `.cursor/hooks/`, …) is apm's deployed output and FAILs.
|
||||
- A hook file directly under a package-root `hooks/` — beside the package's `apm.yml`, or beside a `plugin.json` at any location apm's `find_plugin_json` reads (root, `.github/plugin/`, `.claude-plugin/`, `.cursor-plugin/`) — passes. apm discovers both `.apm/hooks/` and `hooks/`, installs a Claude plugin with no `apm.yml`, and this audit may target a third-party package; `primitive-author` authors only in `.apm/hooks/`. Any other `hooks/` directory (`.github/hooks/`, `.cursor/hooks/`, …) is apm's deployed output, or no package source at all, and FAILs at exit 1.
|
||||
- An event that every listed target fires, but that reaches a target with no published event list (Cursor, Kiro, Gemini, Codex, Antigravity, Windsurf) in a non-PascalCase form after apm's rename, is a SUGGESTION, not the FAIL Must 4 implies: the script cannot tell a harness's native spelling (Cursor's `stop`, Windsurf's `pre_run_command`) from a typo. A Cursor-only `stop` therefore exits 0.
|
||||
- Copilot counts every name apm's own Copilot map emits as fired (`userPromptSubmit`, although Copilot documents `userPromptSubmitted`): the author cannot route around apm's rename, so that is not a finding in the file.
|
||||
- Deprecated filename routing is a SUGGESTION, matching the author's Should: the research allows it when deprecated routing is intended.
|
||||
- A non-executable script run as the command's first token is a FAIL, stricter than the research's Should, because it fails every time it fires.
|
||||
- A non-executable script run directly (the first token, or first in a `-c` string) is a FAIL, stricter than the research's Should, because it fails every time it fires.
|
||||
- `primitive-author` hook Must 5 bans an absolute or bare relative path "in any position"; this audit checks command positions only, because a later argument is data the script cannot tell from a path. Judge the rest by reading in Step 3.
|
||||
|
||||
## Step 2 — Read the hook and its scripts
|
||||
|
||||
|
||||
@@ -23,7 +23,7 @@ bash scripts/validate.sh <instruction-file>
|
||||
bash scripts/vale-wrap.sh <instruction-file>
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path, unfilled `FILL IN` or `FILL_IN_` template placeholders, frontmatter, `description`, body, an `applyTo` that is present but empty or has unbalanced braces or brackets, a missing or list-form `applyTo`, extra keys, and a stem duplicated at the package root. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path, a file that is not valid UTF-8, unfilled `FILL IN` or `FILL_IN_` template placeholders, frontmatter, `description`, body, an `applyTo` that is neither a string nor a list, present but empty, or has unbalanced braces or brackets (a closer before its opener counts), a missing or list-form `applyTo`, extra keys, and a stem duplicated at the package root. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran, including a crash inside the checks — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
|
||||
A missing `applyTo` is a SUGGESTION, not the FAIL `primitive-author`'s instruction Must 4 implies: absence is legal after the author Gate's explicit yes, which the audit cannot see. The **scope** dimension's always-on FAILs below cover the misuse. Do not re-tier it by judgment.
|
||||
|
||||
@@ -33,7 +33,7 @@ There is no provenance step: an instruction carries no `source_keys`.
|
||||
|
||||
## Step 2 — Read the instruction and its context
|
||||
|
||||
Read the file, the package's `apm.yml`, and the repo's root `AGENTS.md`. For a scoped file, list the tracked files its `applyTo` matches (`rtk git ls-files` filtered by the glob). List the instruction stems the installed dependencies ship (`apm_modules/**/.apm/instructions/*.instructions.md`) — the script checks only the package root for a duplicate.
|
||||
Read the file, the package's `apm.yml`, and the repo's root `AGENTS.md`. For a scoped file, list the tracked files its `applyTo` matches (`git ls-files` filtered by the glob). List the instruction stems the installed dependencies ship (`apm_modules/**/.apm/instructions/*.instructions.md`) — the script checks only the package root for a duplicate.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ order, then return to `SKILL.md` Step 4 to report.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- A prompt is judged against ADR-0029, not against apm's framing. apm calls a prompt "a callable program"; this repo holds it to a single-intent, user-triggered message that steers existing skills or agents by name and carries no procedure of its own.
|
||||
- A prompt is judged against ADR-0029, not against apm's framing. apm's docs call a prompt "a program for an LLM" (`apm-docs-llms-full`, "What is APM?" › "Secure by default"); this repo holds it to a single-intent, user-triggered message that steers existing skills or agents by name and carries no procedure of its own.
|
||||
- A prompt's description is not a skill description. It is one plain user-facing sentence with no "Use when" trigger clause and no boundary clause — so never raise a missing trigger or boundary as a finding. A Vale `Kyberforge.DescriptionOpener` alert here means rewrite it as an imperative action ("Review the current PR with …"), not add a trigger.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
@@ -24,7 +24,7 @@ bash scripts/validate.sh <prompt-file>
|
||||
bash scripts/vale-wrap.sh <prompt-file>
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path and name, unfilled `FILL IN` or `FILL_IN_` template placeholders, frontmatter, `description` presence, length, and trigger or `Not X -> Y` boundary clause, keys Claude drops, the camelCase spelling of `allowed-tools` or `argument-hint` and an `argument-hint` alongside `input:` (both SUGGESTION), `input:` names and the object form `- <name>: "<desc>"` (`primitive-author` prompt Must 3 — a bare name, a string list or a plain map is a FAIL even though apm reads them), and `${input:x}` references against `input:`. Keys Claude drops are a SUGGESTION, per `primitive-author` prompt Should 5: a Copilot-only key is legitimate when its Claude drop is intended, and only the author can say which. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path and name, a file that is not valid UTF-8, unfilled `FILL IN` or `FILL_IN_` template placeholders, frontmatter, `description` presence, length, and trigger or `Not X -> Y` boundary clause, keys Claude drops, the camelCase spelling of `allowed-tools` or `argument-hint` and an `argument-hint` alongside `input:` (both SUGGESTION), `input:` names and the object form `- <name>: "<desc>"` (`primitive-author` prompt Must 3 — a bare name, a string list or a plain map is a FAIL even though apm reads them), and `${input:x}` references against `input:`. Keys Claude drops are a SUGGESTION, per `primitive-author` prompt Should 5: a Copilot-only key is legitimate when its Claude drop is intended, and only the author can say which. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran, including a crash inside the checks — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. That includes the rules scoped to the `description` (`Kyberforge.DescriptionOpener`, `VagueWording`, `CompositionNote`), a deliberate deviation from `primitive-author`, which holds description wording to at most a Should: the house prose rules apply to every model- or user-visible description, and a deterministic rule does not change tier by file kind. Do not re-tier them. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
|
||||
|
||||
|
||||
@@ -159,7 +159,8 @@ source_keys:
|
||||
|
||||
## apm-cli-installed-source
|
||||
|
||||
- **URL:** file:///root/.local/pipx/venvs/apm-cli/lib/python3.11/site-packages/apm_cli/
|
||||
- **URL:** https://github.com/microsoft/apm/tree/v0.28.0/src/apm_cli/
|
||||
- **Note:** read locally from the pipx install at `~/.local/pipx/venvs/apm-cli/lib/python3.11/site-packages/apm_cli/` (apm-cli 0.28.0, tag `v0.28.0`)
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md
|
||||
- **Description:** Installed apm-cli 0.28.0 source — ground truth for what apm deploys from a hook, instruction or prompt file and what it silently skips, warns on, or fails the install for; every deterministic check in `scripts/lib-checks-primitive.sh` traces to it via the research docs' Authoring checklists
|
||||
- **Contributing files:** SKILL.md, references/hook-flow.md, references/instruction-flow.md, references/prompt-flow.md
|
||||
|
||||
Reference in New Issue
Block a user