fix(kyberforge): resolve PR #144 review and audit round 1
- factory-audit: no-op hooks, ./ after interpreters, split-quote and
spaced ${PLUGIN_ROOT} paths, camelCase events in Claude-targeted flat
files, case-insensitive routing stems, and non-string YAML keys are
now caught; input: forms and prompt boundary clauses align with
primitive-author; bats 347 -> 367
- primitive-author: routing forms, quoting guidance, install exit on
hidden Unicode, argument-hint exception
- forge: drop duplicated gotcha, fit description and body budgets (#143)
- skill-author: primitive-author boundary, Claude-only env vars
- hook: exit unless CLAUDE_PROJECT_DIR is set, so Copilot/Codex never
run apm update; ADR-0019 correction, ADR-0025 amendment, docs fixes
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KkT7RSDwDbmrM9T34b6sTi
This commit is contained in:
@@ -13,6 +13,7 @@ Steps 1 to 3 for an apm hook — the target Step 0 matched as a `.json` file dir
|
||||
|
||||
- apm checks almost nothing here. Invalid JSON is skipped without a word, an all-lowercase event deploys and never fires, and a missing script only warns — so `apm install` exiting 0 says nothing about whether the hook works. Never cite a clean install as evidence against a finding.
|
||||
- Copilot receiving a Claude-shaped file is not a finding. apm renders one source for every target and documents that it owns the per-target shape; whether Copilot CLI honours a nested entry or `matcher` is unverified upstream, not a defect in the file.
|
||||
- Quoting is not the fix for a script path with a space. apm rewrites a whole-token-quoted `"${PLUGIN_ROOT}/scripts/x.sh"`, but still stops reading the path at the space; the only fix is a path without one.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
@@ -22,15 +23,16 @@ Resolve the path against this skill's own directory. Run exactly:
|
||||
bash scripts/validate.sh <hook-file>
|
||||
```
|
||||
|
||||
Its findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: JSON validity, the wrapped-or-naked shape, event lists and nested handler lists (the checks whose failure makes the Copilot install fail), a file contributing no entries, event names that never fire, referenced scripts that are missing, outside the package, not executable when run directly, or referenced by an absolute or bare relative path apm will not bundle, deprecated filename routing, and `${CLAUDE_PLUGIN_ROOT}` where `${PLUGIN_ROOT}` would do. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
Its findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: JSON validity, the wrapped-or-naked shape, event lists and nested handler lists (the checks whose failure makes the Copilot install fail), a file contributing no entries (no events, only empty event lists, or an entry with no handler), event names that never fire, referenced scripts that are missing, outside the package, not executable when run directly, or referenced by an absolute, bare relative, `../`, split-quoted or space-containing path apm will not bundle correctly, deprecated filename routing, and `${CLAUDE_PLUGIN_ROOT}` where `${PLUGIN_ROOT}` would do. Script references are read with apm 0.28.0's own patterns: `${PLUGIN_ROOT}/…` only when the path follows the token directly, up to the first whitespace or quote, and `./…` anywhere in the command, including after an interpreter. A camelCase event outside Claude's rename map FAILs in a Claude-shaped file, and in a flat Copilot-shaped file too whenever the nearest `apm.yml` above the file targets Claude — no `target:`/`targets:` means every target. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
|
||||
There is no provenance and no Vale step: a hook carries no `source_keys` and no prose.
|
||||
|
||||
Three tiers deliberately differ from `primitive-author`'s checklist or the research's. Do not re-tier them by judgment:
|
||||
Four tiers deliberately differ from `primitive-author`'s checklist or the research's. Do not re-tier them by judgment:
|
||||
|
||||
- A hook file directly under a package-root `hooks/` passes. apm discovers both `.apm/hooks/` and `hooks/`, and this audit may target a third-party package; `primitive-author` authors only in `.apm/hooks/`.
|
||||
- Deprecated filename routing is a SUGGESTION, matching the author's Should: the research allows it when deprecated routing is intended.
|
||||
- A non-executable script run as the command's first token is a FAIL, stricter than the research's Should, because it fails every time it fires.
|
||||
- A split-quoted `"${PLUGIN_ROOT}"/x.sh` or a script path containing a space is a FAIL, stricter than the author's Should 10: apm leaves the first unrewritten and cuts the second at the space, so either deploys a hook that fails every time it fires.
|
||||
|
||||
## Step 2 — Read the hook and its scripts
|
||||
|
||||
@@ -51,7 +53,6 @@ Cite file and line for every finding.
|
||||
- SUGGESTION: a tool event (`PreToolUse`, `PostToolUse`) or `SessionStart` with no `matcher` — Claude receives `"*"`. A `matcher` on an event Claude ignores it for (`Stop`, `UserPromptSubmit`) is inert, not wrong.
|
||||
- SUGGESTION: a PascalCase event name that is not a real Claude Code event (a misspelling deploys verbatim and never fires; the script cannot tell a typo from an event it does not know).
|
||||
- SUGGESTION: `bash`/`powershell`/`timeoutSec` keys in a Claude-shaped file — they render, but leave stray keys in `settings.json`.
|
||||
- SUGGESTION: an unquoted script path that could contain spaces.
|
||||
- SUGGESTION: a helper `.json` file in the hook directory without a `hooks` key — Copilot's loader scans the bundled scripts directory and rejects it. Keep helper configuration non-JSON.
|
||||
|
||||
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
|
||||
|
||||
@@ -25,6 +25,8 @@ bash scripts/vale-wrap.sh <instruction-file>
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path, frontmatter, `description`, body, an `applyTo` that is present but empty or has unbalanced braces or brackets, a missing or list-form `applyTo`, extra keys, and a stem duplicated at the package root. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
|
||||
A missing `applyTo` is a SUGGESTION, not the FAIL `primitive-author`'s instruction Must 4 implies: absence is legal after the author Gate's explicit yes, which the audit cannot see. The **scope** dimension's always-on FAILs below cover the misuse. Do not re-tier it by judgment.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
|
||||
|
||||
There is no provenance step: an instruction carries no `source_keys`.
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
source_keys:
|
||||
- apm-cli-installed-source
|
||||
- apm-docs-llms-full
|
||||
- adr-0029-prompt-house-rule
|
||||
---
|
||||
|
||||
# Prompt Flow
|
||||
@@ -23,7 +24,7 @@ bash scripts/validate.sh <prompt-file>
|
||||
bash scripts/vale-wrap.sh <prompt-file>
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path and name, frontmatter, `description` presence, length and trigger clause, keys Claude drops, `input:` names and shapes, and `${input:x}` references against `input:`. Keys Claude drops are a SUGGESTION, not a FAIL, on purpose: a Copilot-only key is legitimate when its Claude drop is intended, and only the author can say which. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
`validate.sh` findings become the `### Structure` dimension, FAILs and SUGGESTIONs both, at the tier the script assigned: path and name, frontmatter, `description` presence, length, and trigger or `Not X -> Y` boundary clause, keys Claude drops, `input:` names and the object form `- <name>: "<desc>"` (`primitive-author` prompt Must 3 — a bare name, a string list or a plain map is a FAIL even though apm reads them), and `${input:x}` references against `input:`. Keys Claude drops are a SUGGESTION, not a FAIL, on purpose: a Copilot-only key is legitimate when its Claude drop is intended, and only the author can say which. It exits **0** with no FAIL, **1** on real findings, **2** when it never ran — report that as `### Structure` unverified, quoting the stderr reason.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style. Every alert is a FAIL under `### Prose`, cited by rule ID; do not re-derive it by judgment. `0 files` scanned means NOT RUN, not clean — say so and judge prose by reading.
|
||||
|
||||
@@ -48,7 +49,7 @@ Cite file and line for every finding.
|
||||
|
||||
**description**
|
||||
|
||||
- SUGGESTION: the description does not read as one user-facing action, or does not name the skills or agents the prompt steers. On Claude the description is model-visible and apm drops `disable-model-invocation`, so naming what it steers keeps the router pointed at the capability rather than the wrapper.
|
||||
- SUGGESTION: the description does not read as one user-facing action, carries a `Not X -> Y` boundary clause (`primitive-author` prompt Should 6), or does not name the skills or agents the prompt steers. On Claude the description is model-visible and apm drops `disable-model-invocation`, so naming what it steers keeps the router pointed at the capability rather than the wrapper.
|
||||
|
||||
Then return to `SKILL.md` Step 4, opening the report with this coverage line:
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ source_keys:
|
||||
- github-custom-agents-configuration
|
||||
- apm-cli-installed-source
|
||||
- apm-docs-llms-full
|
||||
- adr-0029-prompt-house-rule
|
||||
---
|
||||
|
||||
# Sources
|
||||
@@ -169,3 +170,12 @@ source_keys:
|
||||
- **Description:** Published apm docs bundle — the "Hooks and commands", "Instructions and agents" and "Author a prompt" guides: canonical hook shape and `${PLUGIN_ROOT}`, reach narrowed by `targets:` rather than filename routing, and "reach for a skill, instruction, or prompt first"
|
||||
- **Contributing files:** SKILL.md, references/hook-flow.md, references/instruction-flow.md, references/prompt-flow.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## adr-0029-prompt-house-rule
|
||||
|
||||
- **URL:** docs/adr/0029-prompts-are-thin-user-triggered-steering-messages.md
|
||||
- **Research doc:** none
|
||||
- **Basis:** docs/adr/0029-prompts-are-thin-user-triggered-steering-messages.md
|
||||
- **Description:** The repo's prompt house rule — a prompt is a single-intent, user-triggered steering message with no procedure — and its description contract: one user-facing sentence naming the steered skills, no trigger or boundary clause
|
||||
- **Contributing files:** references/prompt-flow.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
@@ -178,32 +178,66 @@ CLAUDE_MAPPED_CAMEL = {'preToolUse', 'postToolUse', 'sessionStart', 'agentStop'}
|
||||
HOOK_COMMAND_KEYS = ('command', 'bash', 'powershell', 'windows', 'linux', 'osx')
|
||||
ROOT_TOKENS = ('PLUGIN_ROOT', 'CLAUDE_PLUGIN_ROOT', 'CURSOR_PLUGIN_ROOT', 'KIRO_PLUGIN_ROOT')
|
||||
ROOT_TOKEN_RE = re.compile(r'\$\{(' + '|'.join(ROOT_TOKENS) + r')\}')
|
||||
# apm 0.28.0's own patterns, hook_integrator.py _rewrite_command_for_target:
|
||||
# the path must follow the token directly and ends at whitespace or a quote.
|
||||
# The ./ pattern is applied with finditer over the whole command, so it
|
||||
# matches after an interpreter (`bash ./x.sh`) too.
|
||||
APM_ROOT_REF_RE = re.compile(r'\$\{(?:' + '|'.join(ROOT_TOKENS) + r')\}([\\/][^\s"\']+)')
|
||||
APM_REL_REF_RE = re.compile(r'(\.[\\/][^\s"\']+)')
|
||||
|
||||
|
||||
def extract_script_refs(cmd):
|
||||
def _is_first(prefix):
|
||||
return not prefix.strip().strip('"\'').strip()
|
||||
|
||||
|
||||
def is_handler(h):
|
||||
# A handler runs something: a command key, or a non-command handler type
|
||||
# (Claude's prompt/agent/http hooks) whose payload is not a script.
|
||||
if not isinstance(h, dict):
|
||||
return False
|
||||
if any(isinstance(h.get(k), str) and h.get(k).strip() for k in HOOK_COMMAND_KEYS):
|
||||
return True
|
||||
return h.get('type') not in (None, 'command')
|
||||
|
||||
|
||||
def extract_script_refs(cmd, where):
|
||||
"""Yield (kind, relpath, is_first_token) for each package-relative script
|
||||
reference in a hook command string. kind is 'root' for a ${*_PLUGIN_ROOT}
|
||||
token, 'rel' for a leading ./path."""
|
||||
reference apm would rewrite, reading the command exactly as apm does. kind
|
||||
is 'root' for a ${*_PLUGIN_ROOT} token, 'rel' for a ./path. A token apm
|
||||
reads wrongly — split-quoted, or a path with a space — is a FAIL here,
|
||||
because apm leaves it unrewritten or cuts it short."""
|
||||
refs = []
|
||||
masked = cmd
|
||||
for m in ROOT_TOKEN_RE.finditer(cmd):
|
||||
start, end = m.start(), m.end()
|
||||
opened = start > 0 and cmd[start - 1] in '"\''
|
||||
quote = cmd[start - 1] if opened else None
|
||||
rest = cmd[end:]
|
||||
if opened and rest.startswith(quote):
|
||||
# split-quoted form: "${PLUGIN_ROOT}"/scripts/my\ hook.sh
|
||||
rest = rest[1:]
|
||||
path = re.match(r'((?:\\.|[^\s"\'])*)', rest).group(1).replace('\\', '')
|
||||
elif opened:
|
||||
path = rest.split(quote, 1)[0]
|
||||
if end < len(cmd) and cmd[end] in '"\'' and cmd[end + 1:end + 2] in ('/', '\\'):
|
||||
fail(f"script path '{cmd[start:]}' splits the quote after ${{{m.group(1)}}} — apm rewrites only a path that follows the token directly, so this one deploys unrewritten and unbundled; quote the whole token: \"${{PLUGIN_ROOT}}/<path>\" — {where}")
|
||||
for m in APM_ROOT_REF_RE.finditer(cmd):
|
||||
start, end = m.start(), m.end()
|
||||
opener = cmd[start - 1] if start > 0 and cmd[start - 1] in '"\'' else None
|
||||
path = m.group(1)
|
||||
nxt = cmd[end:end + 1]
|
||||
# A backslash-escaped space, or a quoted token whose script name only
|
||||
# completes past the whitespace apm stopped at ("…/my hook.sh").
|
||||
spaced = nxt.isspace() and path.endswith('\\')
|
||||
if not spaced and opener is not None and nxt.isspace():
|
||||
quoted = cmd[start:].split(opener, 1)[0]
|
||||
spaced = bool(SCRIPT_EXT_RE.search(quoted)) and not SCRIPT_EXT_RE.search(path)
|
||||
if spaced:
|
||||
fail(f"script path '{cmd[start:]}' contains a space — apm reads a ${{PLUGIN_ROOT}} path only up to the first whitespace or quote, so it bundles the wrong file and the hook fails; rename the script without spaces — {where}")
|
||||
else:
|
||||
path = re.match(r'((?:\\.|[^\s"\'])*)', rest).group(1).replace('\\', '')
|
||||
prefix = cmd[:start - 1] if opened else cmd[:start]
|
||||
refs.append(('root', path.lstrip('/'), not prefix.strip()))
|
||||
stripped = cmd.lstrip().lstrip('"\'')
|
||||
if stripped.startswith('./'):
|
||||
path = re.match(r'((?:\\.|[^\s"\'])*)', stripped).group(1).replace('\\', '')
|
||||
refs.append(('rel', path, True))
|
||||
prefix = cmd[:start - 1] if opener else cmd[:start]
|
||||
refs.append(('root', path.replace('\\', '/').lstrip('/'), _is_first(prefix)))
|
||||
masked = masked[:start] + ' ' * (end - start) + masked[end:]
|
||||
for m in APM_REL_REF_RE.finditer(masked):
|
||||
start = m.start()
|
||||
ref = m.group(1)
|
||||
if start > 0 and masked[start - 1] == '.':
|
||||
fail(f"script path '..{ref[1:]}' starts with ../ — apm reads it as ./{ref[2:]} from the hook directory, not the parent, so the wrong file (or none) is bundled; reference it as ${{PLUGIN_ROOT}}/<path> — {where}")
|
||||
continue
|
||||
opener = masked[start - 1] if start > 0 and masked[start - 1] in '"\'' else None
|
||||
prefix = masked[:start - 1] if opener else masked[:start]
|
||||
refs.append(('rel', ref[2:].replace('\\', '/'), _is_first(prefix)))
|
||||
return refs
|
||||
|
||||
|
||||
@@ -267,6 +301,45 @@ def check_script(kind_, rel, first, pkg_root, where):
|
||||
fail(f"script '{rel}' is run directly but is not executable — chmod +x it, or invoke it through an interpreter — {where}")
|
||||
|
||||
|
||||
def owning_apm_yml():
|
||||
"""The nearest apm.yml walking up from the hook file, or None."""
|
||||
d = parent_dir
|
||||
while True:
|
||||
cand = os.path.join(d, 'apm.yml')
|
||||
if os.path.isfile(cand):
|
||||
return cand
|
||||
up = os.path.dirname(d)
|
||||
if up == d:
|
||||
return None
|
||||
d = up
|
||||
|
||||
|
||||
def targets_claude():
|
||||
"""Whether apm renders this package's hooks to Claude. Mirrors
|
||||
parse_targets_field: no target:/targets: (or no apm.yml) means every
|
||||
target, and 'all' folds to every target. An unreadable apm.yml is treated
|
||||
as every target, the reading that keeps the stricter check on."""
|
||||
path = owning_apm_yml()
|
||||
if path is None:
|
||||
return True
|
||||
try:
|
||||
with open(path, encoding='utf-8') as f:
|
||||
data = yaml.safe_load(f)
|
||||
except (OSError, UnicodeDecodeError, yaml.YAMLError):
|
||||
return True
|
||||
if not isinstance(data, dict):
|
||||
return True
|
||||
raw = data.get('targets', data.get('target'))
|
||||
if raw is None:
|
||||
return True
|
||||
if isinstance(raw, list):
|
||||
tokens = [str(t).strip().lower() for t in raw]
|
||||
else:
|
||||
tokens = [t.strip().lower() for t in str(raw).split(',')]
|
||||
tokens = [t for t in tokens if t]
|
||||
return not tokens or 'claude' in tokens or 'all' in tokens
|
||||
|
||||
|
||||
def audit_hook():
|
||||
check_not_linked(hardlinks=False)
|
||||
stem = fname[:-len('.json')]
|
||||
@@ -280,7 +353,8 @@ def audit_hook():
|
||||
if not os.path.isfile(os.path.join(pkg_root, 'apm.yml')):
|
||||
info(f"no apm.yml at the inferred package root {pkg_root} — script paths are resolved against it anyway — {fname}")
|
||||
|
||||
if ROUTING_STEM_RE.search(stem):
|
||||
# apm lowercases the stem before routing (hook_file_routing.py).
|
||||
if ROUTING_STEM_RE.search(stem.lower()):
|
||||
suggest(f"filename stem '{stem}' uses deprecated hook filename routing — name it plainly and narrow reach with target:/targets: in the package's apm.yml — {fname}")
|
||||
|
||||
content = read_text(target)
|
||||
@@ -334,13 +408,31 @@ def audit_hook():
|
||||
elif 'command' in entry:
|
||||
claude_shaped = True
|
||||
|
||||
if shape_ok:
|
||||
# hook.md Must 3: the file contributes at least one entry. An empty
|
||||
# event list, or an entry with no handler, deploys nothing runnable.
|
||||
total = 0
|
||||
for event, entries in events.items():
|
||||
for i, entry in enumerate(entries):
|
||||
total += 1
|
||||
handlers = entry['hooks'] if 'hooks' in entry else [entry]
|
||||
if not any(is_handler(h) for h in handlers):
|
||||
fail(f"event '{event}' entry {i} has no handler — no command (or other handler type) to run, so it deploys nothing — {fname}")
|
||||
if total == 0:
|
||||
fail(f"contributes no hook entries — every event list is empty, so apm deploys nothing — {fname}")
|
||||
|
||||
# camelCase outside Claude's map never fires on Claude. A flat
|
||||
# Copilot-shaped file still renders to Claude whenever the package targets
|
||||
# it, so the exemption holds only for a package that does not.
|
||||
camel_checked = claude_shaped or targets_claude()
|
||||
for event in events:
|
||||
if not event.strip():
|
||||
fail(f"empty event name — {fname}")
|
||||
elif not any(c.isupper() for c in event):
|
||||
fail(f"event '{event}' is all-lowercase — no target maps it and apm never warns, so it silently never fires — {fname}")
|
||||
elif claude_shaped and event[0].islower() and event not in CLAUDE_MAPPED_CAMEL:
|
||||
fail(f"event '{event}' is camelCase in a Claude-shaped file and Claude's map does not rename it — it deploys verbatim and never fires; write it in PascalCase — {fname}")
|
||||
elif camel_checked and event[0].islower() and event not in CLAUDE_MAPPED_CAMEL:
|
||||
why = 'in a Claude-shaped file' if claude_shaped else "and the package's apm.yml targets Claude (no targets: means every target)"
|
||||
fail(f"event '{event}' is camelCase {why}, and Claude's map does not rename it — it deploys verbatim to Claude and never fires; write it in PascalCase — {fname}")
|
||||
|
||||
if not shape_ok:
|
||||
return
|
||||
@@ -357,7 +449,7 @@ def audit_hook():
|
||||
continue
|
||||
if '${CLAUDE_PLUGIN_ROOT}' in cmd:
|
||||
uses_claude_token = True
|
||||
refs = extract_script_refs(cmd)
|
||||
refs = extract_script_refs(cmd, where)
|
||||
for kind_, rel, first in refs:
|
||||
check_script(kind_, rel, first, pkg_root, where)
|
||||
if not any(first for _, _, first in refs):
|
||||
@@ -446,7 +538,7 @@ def audit_instruction():
|
||||
elif apply_to_ok and isinstance(apply_to, list):
|
||||
suggest(f"applyTo is a YAML list — Copilot receives the file verbatim and its handling of a list is unverified; use one comma-separated string — {fname}")
|
||||
|
||||
extra = sorted(k for k in fm if k not in INSTRUCTION_KEYS)
|
||||
extra = sorted(str(k) for k in fm if k not in INSTRUCTION_KEYS)
|
||||
if extra:
|
||||
suggest(f"frontmatter key(s) {', '.join(extra)} are read by no target and dropped on Claude — keep to description and applyTo (author, version optional) — {fname}")
|
||||
|
||||
@@ -461,6 +553,8 @@ INPUT_NAME_RE = re.compile(r'^[A-Za-z][\w-]{0,63}$')
|
||||
# apm's own rewrite pattern for ${input:x}, command_integrator.py.
|
||||
INPUT_REF_RE = re.compile(r'\$\{\{?\s*input\s*:\s*([\w-]+)\s*\}?\}')
|
||||
TRIGGER_RE = re.compile(r'\buse\s+(?:this\s+)?when\b', re.IGNORECASE)
|
||||
# The skill boundary form `Not <thing> -> <target>` (ASCII or Unicode arrow).
|
||||
BOUNDARY_RE = re.compile(r'\bnot\b[^.;]*?(?:->|\u2192)', re.IGNORECASE)
|
||||
PROMPT_DESC_SUGGEST_CHARS = 250
|
||||
|
||||
|
||||
@@ -481,15 +575,20 @@ def prompt_input_names(spec):
|
||||
return
|
||||
names.append(s)
|
||||
|
||||
form = "write each input as `- <name>: \"<description>\"` (primitive-author prompt Must 3)"
|
||||
if spec is None:
|
||||
return names
|
||||
if isinstance(spec, str):
|
||||
fail(f"input: is a bare name, not the object form — {form}, so every argument carries its description — {fname}")
|
||||
accept(spec)
|
||||
elif isinstance(spec, dict):
|
||||
fail(f"input: is a map, not the object form — {form} — {fname}")
|
||||
for k in spec:
|
||||
accept(k)
|
||||
elif isinstance(spec, list):
|
||||
for item in spec:
|
||||
if not isinstance(item, dict):
|
||||
fail(f"input entry {item!r} is a bare name, not the object form — {form} — {fname}")
|
||||
if isinstance(item, dict):
|
||||
if len(item) > 1:
|
||||
keys = ', '.join(str(k) for k in item)
|
||||
@@ -532,11 +631,13 @@ def audit_prompt():
|
||||
suggest(f"description is {len(desc)} characters (> {PROMPT_DESC_SUGGEST_CHARS}) — it is one user-facing sentence (ADR-0029) — {fname}")
|
||||
if TRIGGER_RE.search(desc):
|
||||
suggest(f"description carries a 'Use when' trigger clause — a prompt is user-triggered (ADR-0029); a trigger clause invites the model to route to it on Claude — {fname}")
|
||||
if BOUNDARY_RE.search(desc):
|
||||
suggest(f"description carries a 'Not X -> Y' boundary clause — a prompt is user-triggered (ADR-0029, primitive-author prompt Should 6); name the skills it steers instead — {fname}")
|
||||
|
||||
for camel, kebab in PROMPT_CAMEL_ALIASES.items():
|
||||
if camel in fm:
|
||||
suggest(f"'{camel}' — use the kebab-case spelling '{kebab}' apm documents — {fname}")
|
||||
extra = sorted(k for k in fm if k not in PROMPT_KEYS and k not in PROMPT_CAMEL_ALIASES)
|
||||
extra = sorted(str(k) for k in fm if k not in PROMPT_KEYS and k not in PROMPT_CAMEL_ALIASES)
|
||||
if extra:
|
||||
suggest(f"frontmatter key(s) {', '.join(extra)} are dropped on Claude (it keeps only {', '.join(sorted(PROMPT_KEYS))}) — keep them only if the Copilot-only behaviour is intended — {fname}")
|
||||
|
||||
@@ -559,15 +660,19 @@ def audit_prompt():
|
||||
suggest(f"argument-hint is set alongside input: — apm synthesises the hint from input: names; drop it unless that form is inadequate — {fname}")
|
||||
|
||||
|
||||
if kind == 'hook':
|
||||
audit_hook()
|
||||
elif kind == 'instruction':
|
||||
audit_instruction()
|
||||
elif kind == 'prompt':
|
||||
audit_prompt()
|
||||
else:
|
||||
AUDITS = {'hook': lambda: audit_hook(), 'instruction': lambda: audit_instruction(), 'prompt': lambda: audit_prompt()}
|
||||
if kind not in AUDITS:
|
||||
print(f"Error: unknown primitive kind '{kind}'", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
try:
|
||||
AUDITS[kind]()
|
||||
except Exception as exc: # an input shape no check anticipated
|
||||
# Exit 1 means findings; a crash means the checks never completed, which
|
||||
# is the never-ran tier, not a verdict on the file.
|
||||
print(f"Error: the {kind} checks crashed ({type(exc).__name__}: {exc}) and did not complete — {fname}", file=sys.stderr)
|
||||
print(" Why: a partial run reported as findings (exit 1) or as clean (exit 0) would be a verdict the checks never reached.", file=sys.stderr)
|
||||
print(" Fix: report ### Structure as unverified, and file the input shape against factory-audit's lib-checks-primitive.sh.", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
|
||||
@@ -196,7 +196,7 @@ TARGET="$1"
|
||||
if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then
|
||||
echo "Error: '$TARGET' does not exist." >&2
|
||||
echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2
|
||||
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2
|
||||
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md), agent file, hook file (.json under a hooks/ directory), *.instructions.md or *.prompt.md." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
@@ -212,8 +212,8 @@ if [[ -d "$TARGET" ]]; then
|
||||
MODE=skill
|
||||
else
|
||||
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and every other target (agent, hook, instruction, prompt) is a file, never a directory — so this path matches no mode and guessing one would report findings of the wrong kind." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, an agent file (<name>.agent.md, or a .md file under an agents/ directory), a hook file (.json under a hooks/ directory), a *.instructions.md or a *.prompt.md." >&2
|
||||
exit 2
|
||||
fi
|
||||
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
|
||||
@@ -53,15 +53,17 @@ provenance suites stand in the same relation to `scripts/validate-provenance.sh`
|
||||
Do not add a third script path here on the assumption that a differently named
|
||||
suite must mean a differently named script.
|
||||
|
||||
### Auto-detection is pinned across the pair
|
||||
### Auto-detection is pinned across the suites
|
||||
|
||||
Each entry point decides for itself what it was handed. ADR-0025 states the
|
||||
rule: a directory containing `SKILL.md` takes the skill flow; an `.agent.md`
|
||||
file, or a file under a directory named `agents/`, takes the agent flow.
|
||||
Anything else is rejected rather than guessed at. That behaviour is new with the
|
||||
merge — before it, each script was hard-wired to one artifact type and nothing
|
||||
about classification could be wrong — so it is asserted from both sides rather
|
||||
than in one place:
|
||||
skill and agent rule: a directory containing `SKILL.md` takes the skill flow; an
|
||||
`.agent.md` file, or a file under a directory named `agents/`, takes the agent
|
||||
flow. `scripts/validate.sh` adds three primitive shapes: a `*.instructions.md`
|
||||
or `*.prompt.md` file takes the instruction or prompt flow wherever it sits, and
|
||||
a `.json` file directly under a `hooks/` directory takes the hook flow. Any
|
||||
shape outside the five is rejected rather than guessed at. Classification could
|
||||
not be wrong before the merge — each script was hard-wired to one artifact type
|
||||
— so it is asserted from every side rather than in one place:
|
||||
|
||||
- the skill-side suites pin the skill-directory classification and the
|
||||
neither-shape rejection,
|
||||
@@ -69,6 +71,12 @@ than in one place:
|
||||
directory that is not `agents/`, and a plain `.md` under `.apm/agents/` — so
|
||||
that a detector implementing only one of them cannot pass both. Plus a control
|
||||
asserting an agent file never picks up a skill-only gate.
|
||||
- `validate-primitive.bats` pins the hook, instruction and prompt shapes,
|
||||
including the precedence that makes a `*.instructions.md` or `*.prompt.md`
|
||||
under `agents/` take the primitive flow rather than the agent flow, a hook
|
||||
file under a package-root `hooks/`, and a `.json` outside `hooks/` matching
|
||||
no shape. It also pins the primitive exit tiers: every negative case asserts
|
||||
exit 1 (`assert_failure 1`), and a missing python3 or PyYAML asserts exit 2.
|
||||
|
||||
Both skill-side suites additionally pin the `SKILL.md` **file** path, not just
|
||||
the directory: a pre-commit `files:` hook matches files, so every hook-driven
|
||||
|
||||
@@ -57,7 +57,7 @@ teardown() {
|
||||
mkdir -p "$PKG/.apm/agents"
|
||||
printf -- '---\ndescription: x\n---\n\nbody\n' > "$PKG/.apm/agents/x.instructions.md"
|
||||
run bash "$SCRIPT" "$PKG/.apm/agents/x.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is not directly in a .apm/instructions/ directory"
|
||||
refute_output --partial "counterpart"
|
||||
}
|
||||
@@ -77,49 +77,79 @@ teardown() {
|
||||
@test "hook: invalid JSON is a FAIL" {
|
||||
write_hook hooks.json '{"hooks": {'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is not valid JSON"
|
||||
}
|
||||
|
||||
@test "hook: an event value that is not a list is a FAIL" {
|
||||
write_hook hooks.json '{"hooks":{"PreToolUse":{"hooks":[]}}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'PreToolUse' is not a list"
|
||||
}
|
||||
|
||||
@test "hook: a naked slice with a stray scalar key is a FAIL" {
|
||||
write_hook hooks.json '{"description":"x","PreToolUse":[{"hooks":[{"type":"command","command":"true"}]}]}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "naked settings-slice shape"
|
||||
}
|
||||
|
||||
@test "hook: an all-lowercase event is a FAIL" {
|
||||
write_hook hooks.json '{"hooks":{"stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'stop' is all-lowercase"
|
||||
}
|
||||
|
||||
@test "hook: camelCase userPromptSubmit in a Claude-shaped file is a FAIL; mapped sessionStart is not" {
|
||||
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"hooks":[{"type":"command","command":"true"}]}],"sessionStart":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'userPromptSubmit' is camelCase"
|
||||
refute_output --partial "event 'sessionStart'"
|
||||
}
|
||||
|
||||
@test "hook: a flat Copilot-shaped file may use camelCase events" {
|
||||
@test "hook: a flat Copilot-shaped file may use camelCase events when the package does not target Claude" {
|
||||
printf 'targets:\n - copilot\n' >> "$PKG/apm.yml"
|
||||
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"type":"command","bash":"true","timeoutSec":5}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_success
|
||||
refute_output --partial "camelCase"
|
||||
}
|
||||
|
||||
@test "hook: camelCase in a flat file is a FAIL when the package targets Claude" {
|
||||
printf 'targets: [claude, copilot]\n' >> "$PKG/apm.yml"
|
||||
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"type":"command","bash":"true","timeoutSec":5}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'userPromptSubmit' is camelCase and the package's apm.yml targets Claude"
|
||||
}
|
||||
|
||||
@test "hook: camelCase in a flat file is a FAIL when apm.yml declares no targets (every target)" {
|
||||
write_hook hooks.json '{"hooks":{"userPromptSubmit":[{"type":"command","bash":"true","timeoutSec":5}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'userPromptSubmit' is camelCase"
|
||||
}
|
||||
|
||||
@test "hook: targets are read from the nearest apm.yml walking up, and target: all counts as Claude" {
|
||||
mkdir -p "$PKG/sub/hooks"
|
||||
printf 'name: sub\nversion: 0.1.0\ntarget: copilot\n' > "$PKG/sub/apm.yml"
|
||||
printf '%s\n' '{"hooks":{"userPromptSubmit":[{"type":"command","bash":"true","timeoutSec":5}]}}' > "$PKG/sub/hooks/hooks.json"
|
||||
run bash "$SCRIPT" "$PKG/sub/hooks/hooks.json"
|
||||
assert_success
|
||||
|
||||
printf 'name: sub\nversion: 0.1.0\ntarget: all\n' > "$PKG/sub/apm.yml"
|
||||
run bash "$SCRIPT" "$PKG/sub/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "is camelCase"
|
||||
}
|
||||
|
||||
@test "hook: a referenced script that does not exist is a FAIL" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/.apm/hooks/scripts/missing.sh"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "script '.apm/hooks/scripts/missing.sh' does not exist"
|
||||
}
|
||||
|
||||
@@ -127,7 +157,7 @@ teardown() {
|
||||
chmod -x "$PKG/.apm/hooks/scripts/check.sh"
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"./scripts/check.sh"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is run directly but is not executable"
|
||||
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"bash ${PLUGIN_ROOT}/.apm/hooks/scripts/check.sh"}]}]}}'
|
||||
@@ -139,7 +169,7 @@ teardown() {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/../outside.sh"}]}]}}'
|
||||
touch "$TMPDIR/outside.sh"
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "resolves outside the package"
|
||||
}
|
||||
|
||||
@@ -157,11 +187,26 @@ teardown() {
|
||||
assert_output --partial "deprecated hook filename routing"
|
||||
}
|
||||
|
||||
@test "hook: routing detection matches apm — case-insensitive, hooks-<target>, prefixed and combined stems" {
|
||||
local name
|
||||
for name in Claude-Hooks.json HOOKS-Copilot.json x-claude-hooks.json claude-codex-hooks.json; do
|
||||
write_hook "$name" '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/$name"
|
||||
assert_success
|
||||
assert_output --partial "deprecated hook filename routing"
|
||||
done
|
||||
|
||||
write_hook claude-hooks-extra.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/claude-hooks-extra.json"
|
||||
assert_success
|
||||
refute_output --partial "deprecated hook filename routing"
|
||||
}
|
||||
|
||||
@test "hook: a symlinked hook file is a FAIL" {
|
||||
write_hook real.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
ln -s real.json "$PKG/.apm/hooks/link.json"
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/link.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is a symlink"
|
||||
}
|
||||
|
||||
@@ -176,7 +221,7 @@ teardown() {
|
||||
@test "hook: an absolute script path is a FAIL; an absolute interpreter path is not" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"/usr/local/bin/check.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is an absolute path"
|
||||
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"/usr/bin/env true","timeout":5}]}]}}'
|
||||
@@ -188,7 +233,7 @@ teardown() {
|
||||
@test "hook: a bare relative path to a package script is a FAIL; a bare command is not" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":".apm/hooks/scripts/check.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is a bare relative path"
|
||||
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"npx some-tool --check","timeout":5}]}]}}'
|
||||
@@ -200,10 +245,99 @@ teardown() {
|
||||
@test "hook: a file contributing no entries is a FAIL" {
|
||||
write_hook hooks.json '{"hooks":{}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "contributes no hook entries"
|
||||
}
|
||||
|
||||
@test "hook: a file whose every event list is empty is a FAIL" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[],"PreToolUse":[]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "every event list is empty"
|
||||
}
|
||||
|
||||
@test "hook: an entry with no handler is a FAIL, whether nested hooks is empty or absent" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"matcher":"x"}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'Stop' entry 0 has no handler"
|
||||
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "event 'Stop' entry 0 has no handler"
|
||||
}
|
||||
|
||||
@test "hook: a non-command handler type (prompt) counts as a handler" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"prompt","prompt":"check","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_success
|
||||
refute_output --partial "no handler"
|
||||
}
|
||||
|
||||
@test "hook: a ./ script after an interpreter is existence-checked, as apm matches it" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"bash ./scripts/missing.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "script 'scripts/missing.sh' does not exist"
|
||||
|
||||
chmod -x "$PKG/.apm/hooks/scripts/check.sh"
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"bash ./scripts/check.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_success
|
||||
refute_output --partial "FAIL"
|
||||
}
|
||||
|
||||
@test "hook: a ../ script path is a FAIL — apm reads it as ./ from the hook directory" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"bash ../scripts/x.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "starts with ../"
|
||||
}
|
||||
|
||||
@test "hook: the whole-token-quoted \${PLUGIN_ROOT} form passes clean" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"\"${PLUGIN_ROOT}/.apm/hooks/scripts/check.sh\"","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_success
|
||||
refute_output --partial "FAIL"
|
||||
}
|
||||
|
||||
@test "hook: a split-quoted \"\${PLUGIN_ROOT}\"/path is a FAIL — apm never rewrites it" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"\"${PLUGIN_ROOT}\"/.apm/hooks/scripts/check.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "splits the quote"
|
||||
}
|
||||
|
||||
@test "hook: a script path with a space is a FAIL, quoted or backslash-escaped" {
|
||||
printf '#!/usr/bin/env bash\nexit 0\n' > "$PKG/.apm/hooks/scripts/my hook.sh"
|
||||
chmod +x "$PKG/.apm/hooks/scripts/my hook.sh"
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"\"${PLUGIN_ROOT}/.apm/hooks/scripts/my hook.sh\"","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "contains a space"
|
||||
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/.apm/hooks/scripts/my\\ hook.sh","timeout":5}]}]}}'
|
||||
run bash "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "contains a space"
|
||||
}
|
||||
|
||||
@test "hook: a hook file under a package-root hooks/ directory is audited, scripts resolved from the package root" {
|
||||
mkdir -p "$PKG/hooks" "$PKG/scripts"
|
||||
printf '#!/usr/bin/env bash\nexit 0\n' > "$PKG/scripts/run.sh"
|
||||
chmod +x "$PKG/scripts/run.sh"
|
||||
printf '%s\n' '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/scripts/run.sh","timeout":5}]}]}}' > "$PKG/hooks/hooks.json"
|
||||
run bash "$SCRIPT" "$PKG/hooks/hooks.json"
|
||||
assert_success
|
||||
refute_output --partial "FAIL"
|
||||
|
||||
printf '%s\n' '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"${PLUGIN_ROOT}/scripts/gone.sh","timeout":5}]}]}}' > "$PKG/hooks/hooks.json"
|
||||
run bash "$SCRIPT" "$PKG/hooks/hooks.json"
|
||||
assert_failure 1
|
||||
assert_output --partial "script 'scripts/gone.sh' does not exist"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Instructions
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -220,7 +354,7 @@ applyTo: "**/*.py"' 'Use type hints on public functions.'
|
||||
@test "instruction: missing description is a FAIL" {
|
||||
write_instruction python 'applyTo: "**/*.py"' 'Use type hints.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "'description' is missing or empty"
|
||||
}
|
||||
|
||||
@@ -228,14 +362,14 @@ applyTo: "**/*.py"' 'Use type hints on public functions.'
|
||||
write_instruction python 'description: x
|
||||
applyTo: "**/*.py"' ''
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "body is empty"
|
||||
}
|
||||
|
||||
@test "instruction: invalid frontmatter YAML is a FAIL" {
|
||||
write_instruction python 'description: [unclosed' 'body'
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "frontmatter is not valid YAML"
|
||||
}
|
||||
|
||||
@@ -261,7 +395,7 @@ name: python' 'body'
|
||||
write_instruction empty 'description: x
|
||||
applyTo: ""' 'body'
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/empty.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "applyTo is present but empty"
|
||||
refute_output --partial "SUGGESTION no applyTo"
|
||||
}
|
||||
@@ -270,10 +404,21 @@ applyTo: ""' 'body'
|
||||
write_instruction broken 'description: x
|
||||
applyTo: "**/*.{py"' 'body'
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/broken.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "unbalanced braces or brackets"
|
||||
}
|
||||
|
||||
@test "instruction: a non-string frontmatter key is a clean SUGGESTION, not a crash" {
|
||||
write_instruction python 'description: x
|
||||
applyTo: "**/*.py"
|
||||
1: a
|
||||
zeta: b' 'body'
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_success
|
||||
refute_output --partial "Traceback"
|
||||
assert_output --partial "frontmatter key(s) 1, zeta"
|
||||
}
|
||||
|
||||
@test "instruction: a top-level comma list with a brace group passes clean" {
|
||||
write_instruction multi 'description: x
|
||||
applyTo: "**/*.py, **/*.{pyi,pyx}"' 'body'
|
||||
@@ -287,7 +432,7 @@ applyTo: "**/*.py, **/*.{pyi,pyx}"' 'body'
|
||||
applyTo: "**/*.py"' 'body'
|
||||
cp "$PKG/.apm/instructions/python.instructions.md" "$PKG/python.instructions.md"
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "also exists at the package root"
|
||||
}
|
||||
|
||||
@@ -296,7 +441,7 @@ applyTo: "**/*.py"' 'body'
|
||||
applyTo: "**/*.py"' 'body'
|
||||
ln "$PKG/.apm/instructions/python.instructions.md" "$TMPDIR/python.instructions.md"
|
||||
run bash "$SCRIPT" "$PKG/.apm/instructions/python.instructions.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is a hardlink"
|
||||
}
|
||||
|
||||
@@ -317,7 +462,7 @@ input:
|
||||
@test "prompt: missing description is a FAIL" {
|
||||
write_prompt review-pr 'model: sonnet' 'Review the PR.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "'description' is missing or empty"
|
||||
}
|
||||
|
||||
@@ -327,23 +472,25 @@ input:
|
||||
- name: pr_number
|
||||
description: The PR' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "yields arguments [name, description]"
|
||||
}
|
||||
|
||||
@test "prompt: an invalid input name is a FAIL" {
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
input: [1pr]' 'Review.'
|
||||
input:
|
||||
- 1pr: "The PR"' 'Review.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "input name '1pr' does not match"
|
||||
}
|
||||
|
||||
@test "prompt: an undeclared \${input:x} and an unused input are both FAILs" {
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
input: [pr_number]' 'Review ${input:branch}.'
|
||||
input:
|
||||
- pr_number: "The PR"' 'Review ${input:branch}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "input: does not declare 'branch'"
|
||||
assert_output --partial "input 'pr_number' is declared but the body never uses"
|
||||
}
|
||||
@@ -351,7 +498,7 @@ input: [pr_number]' 'Review ${input:branch}.'
|
||||
@test "prompt: \${input:x} with no input: declared is a FAIL" {
|
||||
write_prompt review-pr 'description: Review a PR.' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "but no input: is declared"
|
||||
}
|
||||
|
||||
@@ -369,10 +516,58 @@ allowedTools: Bash" 'Review the PR.'
|
||||
assert_output --partial "'allowedTools' — use the kebab-case spelling"
|
||||
}
|
||||
|
||||
@test "prompt: input: forms other than the object form are FAILs (primitive-author Must 3)" {
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
input: [pr_number]' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure 1
|
||||
assert_output --partial "input entry 'pr_number' is a bare name"
|
||||
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
input: pr_number' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure 1
|
||||
assert_output --partial "input: is a bare name"
|
||||
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
input:
|
||||
pr_number: The PR' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure 1
|
||||
assert_output --partial "input: is a map"
|
||||
}
|
||||
|
||||
@test "prompt: a Not X -> Y boundary clause in the description is a SUGGESTION" {
|
||||
write_prompt review-pr 'description: Review the PR with gitea-prs. Not fixing it -> skill-author.' 'Review the PR with gitea-prs.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_success
|
||||
assert_output --partial "'Not X -> Y' boundary clause"
|
||||
}
|
||||
|
||||
@test "prompt: a non-string frontmatter key is a clean SUGGESTION, not a crash" {
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
1: a
|
||||
zeta: b' 'Review the PR.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_success
|
||||
refute_output --partial "Traceback"
|
||||
assert_output --partial "frontmatter key(s) 1, zeta"
|
||||
}
|
||||
|
||||
@test "prompt: a *.prompt.md under an agents/ directory takes the prompt flow, not the agent flow" {
|
||||
mkdir -p "$PKG/.apm/agents"
|
||||
printf -- '---\ndescription: x\n---\n\nbody\n' > "$PKG/.apm/agents/x.prompt.md"
|
||||
run bash "$SCRIPT" "$PKG/.apm/agents/x.prompt.md"
|
||||
assert_failure 1
|
||||
assert_output --partial "is not directly in a .apm/prompts/ directory"
|
||||
refute_output --partial "counterpart"
|
||||
}
|
||||
|
||||
@test "prompt: argument-hint alongside input: is a SUGGESTION" {
|
||||
write_prompt review-pr 'description: Review a PR.
|
||||
argument-hint: <pr>
|
||||
input: [pr_number]' 'Review ${input:pr_number}.'
|
||||
input:
|
||||
- pr_number: "The PR"' 'Review ${input:pr_number}.'
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_success
|
||||
assert_output --partial "argument-hint is set alongside input:"
|
||||
@@ -392,6 +587,34 @@ $(printf 'line\n%.0s' $(seq 1 80))
|
||||
write_prompt review-pr 'description: Review a pull request with gitea-prs.' 'Review the PR with gitea-prs.'
|
||||
ln "$PKG/.apm/prompts/review-pr.prompt.md" "$TMPDIR/review-pr.prompt.md"
|
||||
run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_failure
|
||||
assert_failure 1
|
||||
assert_output --partial "is a hardlink"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Never ran (exit 2)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "exit 2: a PATH with no python3 names the missing dependency in primitive mode" {
|
||||
write_hook hooks.json '{"hooks":{"Stop":[{"hooks":[{"type":"command","command":"true"}]}]}}'
|
||||
local emptybin="$TMPDIR/emptybin" cmd bash_bin
|
||||
mkdir -p "$emptybin"
|
||||
for cmd in cat sed; do
|
||||
ln -s "$(command -v "$cmd")" "$emptybin/$cmd"
|
||||
done
|
||||
bash_bin="$(command -v bash)"
|
||||
run env -i PATH="$emptybin" HOME="$HOME" "$bash_bin" "$SCRIPT" "$PKG/.apm/hooks/hooks.json"
|
||||
assert_equal "$status" 2
|
||||
assert_output --partial "python3 is required"
|
||||
}
|
||||
|
||||
@test "exit 2: a python3 that cannot import PyYAML names the missing library in primitive mode" {
|
||||
write_prompt review-pr 'description: Review a PR.' 'Review the PR.'
|
||||
local noyaml="$TMPDIR/noyaml"
|
||||
mkdir -p "$noyaml"
|
||||
# Shadow PyYAML: PYTHONPATH precedes site-packages, so `import yaml` fails.
|
||||
printf 'raise ImportError("PyYAML shadowed by the test")\n' > "$noyaml/yaml.py"
|
||||
PYTHONPATH="$noyaml" run bash "$SCRIPT" "$PKG/.apm/prompts/review-pr.prompt.md"
|
||||
assert_equal "$status" 2
|
||||
assert_output --partial "PyYAML is required"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user