feat(kyberforge): retarget forge skills to author/audit APM content #93

Merged
Defame1297 merged 14 commits from feat/89-apm-native-authoring into main 2026-08-12 11:48:50 +00:00
34 changed files with 2601 additions and 592 deletions

View File

@@ -70,6 +70,15 @@ repos:
pass_filenames: false
always_run: true
- id: check-scope-walkup-sync
name: Check scope walk-up implementations agree
description: Behaviorally cross-check validate.sh, validate-provenance.sh, new-agent.sh, and new-skill.sh's independent $HOME/.git/apm.yml walk-up ports against each other
entry: bash scripts/check-scope-walkup-sync.sh
language: system
stages: [pre-push]
pass_filenames: false
always_run: true
- id: check-release-needed
name: Check a release tag covers .pre-commit-hooks.yaml's paths
description: On push to main only, fail if files exposed via .pre-commit-hooks.yaml changed since the last tag

View File

@@ -44,3 +44,8 @@ separate single-provider skill, adding complexity with no benefit.
file now lives at `<plugin-root>/sources.md`, outside the `agents/` directory, because
`claude plugin validate --strict` auto-discovers every `.md` under `agents/` as an agent
requiring frontmatter. See ADR-0010 for the empirical finding and rationale.
**Update (ADR-0016):** the plugin-scope clause above is superseded. Plugin scope is no longer
detected via `plugin.json`, and no longer produces a Claude+Copilot file pair — a directory
containing `apm.yml` now gets a single vendor-neutral `.apm/agents/<name>.agent.md` file with
no provider-specific fields. Project scope and user scope are unaffected. See ADR-0016.

View File

@@ -14,3 +14,9 @@
- Scope detection walks up from the input file: first directory containing `plugin.json` → plugin scope; first directory containing `.git` without `plugin.json` → project scope; path under `~` with neither → user scope.
- At user scope the derivation crosses filesystem locations (`~/.claude/agents/` ↔ `~/.copilot/agents/`); the script must handle the home directory case explicitly.
- The invocation signature is the public contract. Changing it is a breaking change to any caller — treat it as such.
**Update (ADR-0016):** the plugin-scope clause above is superseded. Plugin scope is no longer
detected via `plugin.json`, and there is no counterpart to derive — a directory containing
`apm.yml` produces a single `.apm/agents/<name>.agent.md` file, and `agent-audit` validates it
directly with no pair-consistency check. Project scope and user scope keep the pair-derivation
mechanism described above unchanged. See ADR-0016.

View File

@@ -0,0 +1,92 @@
# Plugin-scope agent-author omits `tools:` and all Claude-only fields from `.apm/agents/*.agent.md`
This ADR is a narrower, downstream consequence discovered while designing issue #89's
implementation under ADR-0015's broader direction (Microsoft APM replaces hand-authored
plugin/marketplace authoring). It does not restate ADR-0015's rationale — see that ADR for
the parent decision.
## Context
APM's agent primitive (`.apm/agents/<name>.agent.md`) has no per-target integrator in
`apm compile` — confirmed via APM's own Python source (`integration/targets.py` and related
files, cited in `plugins/kyberforge/docs/research/docs/microsoft-apm/agent-primitive-schema.md`).
Compilation does a naive verbatim copy of the whole frontmatter and body to both the Claude
Code and Copilot CLI targets. This is unlike:
- The **skill** primitive, which is also a straight copy (confirmed in the same research doc)
but has no field semantics to conflict — `SKILL.md`'s content is target-agnostic already.
- The **prompt**, **instructions**, and **hooks** primitives, which each get real per-target
reconstruction through a dedicated integrator (field allowlisting, key renaming, dropped-field
warnings).
Because the agent primitive ships the same frontmatter unchanged to both harnesses, two
concrete incompatibilities surface:
1. **`tools:`** — Claude Code expects a space-separated tool-name string; Copilot CLI expects a
list drawn from its own alias vocabulary (`execute`/`read`/`edit`/`search`/`agent`/`web`). A
value correct for one harness is wrong for the other.
2. **Claude-only knobs with no Copilot equivalent** — `isolation`, `maxTurns`, `effort`,
`memory`, `permissionMode`. Writing any of these means Copilot's copy carries frontmatter
keys it doesn't recognize at all. Whether Copilot's agent loader ignores unknown keys or
errors on them is unconfirmed by research.
## Decision
At **plugin scope only** (destination package has an `apm.yml` at its root — an APM producer
package compiled via `apm compile`), `.apm/agents/<name>.agent.md` carries only `name`,
`description`, `model`, and the prose body. No `tools:` field, no Claude-only fields, at all.
Absent `tools:` means inherit-all-tools on both harnesses — the one value that is never wrong
on either target, unlike a present, harness-specific value that is guaranteed wrong on at least
one of them.
`agent-audit`, at plugin scope, is intended to flag — as a **SUGGESTION**, not a FAIL, since
this is an upstream schema limitation rather than an authoring mistake — any agent whose
description or body implies a need for tool restriction or a Claude-only behavior the
frontmatter can no longer express. This would give visibility into the gap without pretending
the schema can do something it can't. **Not yet implemented**: `check_apm_agent_file()` in
`validate.sh` currently validates only the field allowlist, `name`, `description`, and
body-emptiness/length — it has no heuristic for this case. Tracked as follow-up work.
### Scope boundary
This decision applies to **plugin-scope `agent-author` only**. Project scope (`.claude/agents/`
+ `.github/agents/`) and user scope (`~/.claude/agents/` + `~/.copilot/agents/`) are not APM
packages — neither goes through `apm compile` — so both keep today's dual-file Claude+Copilot
pair model exactly as ADR-0005 and ADR-0008 already describe. Those two ADRs remain fully
authoritative for project and user scope; only their plugin-scope clauses are affected by this
ADR (see the update notes appended to each).
## Considered options
**Pick one harness's vocabulary and accept breakage on the other (rejected).** E.g. always
write Claude's space-separated `tools:` string. Rejected because it ships a value that is
silently wrong (or possibly a hard error) on Copilot, and which harness "wins" would be an
arbitrary, undocumented asymmetry.
**Same as above, but `agent-audit` flags the cross-harness breakage as a tracked finding
(rejected).** Rejected for the same core reason — it still ships a wrong value to a real
harness. Tracking the breakage doesn't prevent it, and the chosen decision already gets
equivalent visibility (a SUGGESTION finding) without ever shipping the wrong value in the first
place.
## Consequences
- Every plugin-scope APM agent loses per-agent tool restriction and any Claude-only capability
(isolation, maxTurns, effort, memory, permissionMode) until APM ships a real per-target
integrator for the agent primitive. This is a known, accepted regression, not an oversight.
- **ADR-0005 is partially superseded** — its plugin-scope clause ("directory containing
`plugin.json` is plugin scope → both files land in `<root>/agents/`") no longer applies.
Plugin scope is now "directory containing `apm.yml` → single vendor-neutral file lands in
`<root>/.apm/agents/`." Project and user scope, and the rest of ADR-0005, are unaffected.
- **ADR-0008 is partially superseded** — its counterpart-derivation/pair-validation mechanism
no longer applies at plugin scope; `agent-audit` takes the single file directly there. Project
and user scope, where a real pair still exists, are unaffected.
- **ADR-0009 is not superseded.** The mechanism it established — `agent-audit` reading field
lists from `references/field-inventory.md` rather than hardcoding them, with a `source_keys`
provenance chain — survives and is reused. Only the *content shape* changes for plugin scope:
`field-inventory.md` shifts from two side-by-side CC-only/Copilot-only blocklists to one
vendor-neutral allowlist (`name`/`description`/`model`/`source_keys` — the last for provenance
tracking, validated separately by `validate-provenance.sh` against `sources.md`, not a
provider-specific field) for plugin-scope agents, while
continuing to serve its original two-blocklist role for project/user-scope validation.

View File

@@ -0,0 +1,66 @@
---
topic: agent-primitive-schema
source_keys:
- context7-microsoft-apm
- apm-github-repo
---
## Agent frontmatter is not a fixed APM-defined schema
APM's design principle (PRINCIPLES.md, P1 "No invented primitive frontmatter") is that APM emits to the canonical schema of each *upstream* harness and does not invent its own frontmatter keys. Consequently there is no single APM-owned "complete field list" for `.apm/agents/*.agent.md` the way there is for `apm.yml`. The docs describe six fields as commonly used, and treat everything else as pass-through content the author writes for whichever harness(es) they target:
| Field | Required | Purpose | Recognized by |
|---|---|---|---|
| `name` | Recommended | Display identifier; falls back to the filename stem if omitted | Universal, but Kiro derives identity from the deployed file path instead and strips this field |
| `description` | Yes | Used for agent discovery (e.g. by Cascade/Copilot) | Universal |
| `model` | Optional | Model the harness switches to when the agent is invoked | Copilot, Claude, Grok Build, Cursor, OpenCode (not Kiro's runtime behavior beyond passthrough; not meaningfully supported by Codex) |
| `tools` | Optional | Whitelist of callable tools/capabilities | Copilot, Claude, Grok Build, Cursor, OpenCode, Kiro (each with different accepted value shapes — see below) |
| `color` | Optional | Display color in harness UI | Copilot, Claude, OpenCode only |
| `handoffs` | Optional | Named agents (or structured objects) this agent can delegate to | Universal in principle; concretely modeled as a field on the internal `Chatmode` primitive (`list[str \| dict] \| None`), suggesting `.agent.md` is treated as a thin transform target rather than a fully independent, strongly-typed primitive internally |
**Claude Code fields NOT documented anywhere in APM's corpus:** `isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `disallowedTools`, `background`, `initialPrompt`. **Copilot CLI tool-alias shorthand** (`execute`/`read`/`edit`/`search`/`agent`/`web`) is also undocumented in APM's material — Copilot's agent `tools` are documented only as the `ToolName: true/false` map form. This is a genuine documentation gap, not an oversight on our part: APM's stated philosophy is to stay out of the way of harness-specific fields, so anything beyond the six-field table above is simply not modeled, validated, or discussed by APM — it would ride through compilation as opaque frontmatter for "verbatim" targets (see below) and be silently dropped for targets that do field-level reconstruction (Codex, Kiro).
## Compile-time mapping per target
`apm compile` reads `.apm/agents/*.agent.md` and reconciles per-target output according to the `targets`/`compilation` block in `apm.yml`. Behavior is verbatim-copy for most targets, with three targets doing real field-level transformation:
| Target | Output path | Field treatment |
|---|---|---|
| Copilot | `.github/agents/<name>.agent.md` | Verbatim |
| Claude | `.claude/agents/<name>.md` | Verbatim |
| Grok Build | `.grok/agents/<name>.md` | Verbatim |
| Cursor | `.cursor/agents/<name>.md` | Verbatim |
| OpenCode | `.opencode/agents/<name>.md` | Verbatim, but gated by strict validation (see below) |
| Codex | `.codex/agents/<name>.toml` | Reconstructed: only `name` and `description` map into TOML fields; the markdown body becomes `developer_instructions`. `model`/`tools` are **not yet** translated into Codex's MCP/tool transport config — effectively dropped |
| Kiro | `.kiro/agents/<relative-stem>.md` | Reconstructed: keeps only `description`, `model`, `tools`; **strips `name` and any unknown field**; identity comes from the file's path, not frontmatter |
| Windsurf | Not deployed as an agent | No agents primitive exists for Windsurf — Cascade auto-invokes skills by `description`, so a separate agent construct would collide; author as a skill instead |
| Gemini | Not deployed as an agent | Gemini has no agents primitive either; content is folded into `GEMINI.md` context instead |
**No `claude:`/`copilot:` provider-override or passthrough sub-key exists anywhere in APM's schema or docs.** This was checked specifically (frontmatter reference doc, manifest schema, compile reference, prompts pitfalls page) — the only analogous behavior found is for the *prompts* primitive, where APM documents that only five frontmatter keys are preserved across all targets and provider-only keys (`author`, `mcp`, `parameters`) are silently stripped on every provider except Copilot. No equivalent table exists for the agent primitive beyond the six-field summary above and the per-target rows in this table. There is no way for a single vendor-neutral `.agent.md` to carry a provider-only setting (e.g. Claude Code's `isolation` or `permissionMode`) through compilation to just one target — the field would either pass through verbatim (Claude/Copilot/Grok/Cursor, since those are unvalidated copies) or get silently dropped (Codex, Kiro, and OpenCode if it fails that target's strict shape check).
## The `tools:` field's shape is target-dependent, not fixed
APM does not enforce one canonical shape. Different targets expect different shapes, and since APM doesn't invent or normalize frontmatter (P1), the author is effectively writing in whichever shape their primary target consumes:
- **Map of booleans** — `tools: {Read: true, Grep: true}` — the form shown in APM's own canonical example and the form **required** by OpenCode's strict validator (`apm install -t opencode` warns at install time if `tools` is anything else, e.g. a list).
- **List of strings** — `tools: ["file-manager", "git-analysis"]` — appears in APM's own reference examples page; not glob-based and not `mcp__*`-namespaced in any example found.
- **Kiro capability tags** — a closed vocabulary distinct from tool names: `read`, `write`, `shell`, `web`, `subagent`, `knowledge`, `context`, `todo_list`, `@mcp`, `@builtin`, `*`. Kiro **fails closed** (refuses the whole write, no partial deploy) if any value in `tools` falls outside this set.
- No documented support for glob patterns or `mcp__*`-prefixed tool-name patterns as a `tools:` value shape anywhere in the corpus.
Net effect for retargeting `agent-author`: there's no single `tools:` normalization APM performs for you. If the skill needs to emit a `.apm/agents/*.agent.md` that survives compilation to both Claude Code and Copilot CLI cleanly, it should use the `{ToolName: true}` map form (both targets treat agents as verbatim copies, so whatever is written is what each harness receives directly — meaning the shape must already match what Claude Code's own agent frontmatter and Copilot's agent frontmatter expect natively, since APM performs no translation for either).
## Validation, naming, and size constraints
There is no dedicated `Agent`/`AgentPrimitive` validation function or dataclass in APM's own codebase (`apm_cli/models/validation.py`, `apm_cli/primitives/models.py`) — validation logic exists for `apm.yml`, `SKILL.md`, `plugin.json`, hooks, and other primitives, but `.agent.md` is not independently modeled or validated by generic APM code. What constraints exist are documented as authoring guidance rather than enforced checks, plus the per-target strict checks already covered above (OpenCode's tools/color shape check, Kiro's fail-closed tool allowlist):
- **Naming**: avoid the names `default` or `start` — they collide with `apm run` script name resolution.
- **Body size**: keep the markdown body under ~300 lines to preserve context window budget.
- **Body structure guidance**: open with role and scope in two sentences; define boundaries and expected output artifacts; don't duplicate global/workspace instructions the agent already inherits — override or extend only.
- **File location/extension**: must live at `.apm/agents/<name>.agent.md`.
- No documented uniqueness enforcement, required-field enforcement, or file-size hard limit beyond the above authoring guidance — this is a real gap, not an omission on our part: it simply isn't implemented as a checked rule anywhere in the source.
## Skill primitive: compilation is a straight copy, confirmed
`apm install`/`apm compile` never rewrite or rename `SKILL.md` per target — deployment is a literal directory copy (`shutil.copytree`) of the whole skill folder (`SKILL.md` plus `scripts/`, `references/`, `assets/`, `examples/`), with symlinks and cache markers filtered out. `SKILL.md`'s content is treated as immutable; only its *placement* varies by target. This is on top of, not a contradiction of, what's already in `configuration.md`/`examples.md`.
The one nuance not previously captured: **path routing changed via a "skills convergence."** Most targets (Copilot, Cursor, Codex, Gemini, OpenCode, Windsurf, and the explicit `agent-skills` target) now converge on a single shared `.agents/skills/<name>/SKILL.md` location to avoid duplicate copies across harnesses. Three targets still use harness-native paths because their default file scan looks only in their own directory: Claude (`.claude/skills/`), Kiro (`.kiro/skills/`), and Grok Build (`.grok/skills/`). The pre-convergence, per-harness-copy behavior can be restored with `--legacy-skill-paths` or `APM_LEGACY_SKILL_PATHS=1`. `includes:`/`compilation.placement:` (as already documented in `configuration.md`) apply to instructions/prompts placement, not to skills — no skill-specific effect from those keys was found anywhere in the corpus.

View File

@@ -0,0 +1,59 @@
---
topic: hooks-primitive-schema
source_keys:
- context7-microsoft-apm
- apm-github-repo
---
## File location, naming, and format — confirmed `.json`, not assumed
`.apm/hooks/*.json` (legacy fallback: bare `hooks/*.json` at package root, still discovered — `_has_hook_json()` checks both `hooks/` and `.apm/hooks/`). This is genuinely JSON, not YAML or Markdown-with-frontmatter like every other primitive — confirmed directly from source (`apm_cli/integration/hook_integrator.py` module docstring: "Integrates hook JSON files...") and from `apm_cli/models/validation.py`, which states a hook-only package's files define "hook handlers per the Claude Code hooks specification" — i.e. the canonical authoring shape APM expects is Claude Code's own native hook JSON shape, not an APM-invented one. This is consistent with APM's general P1 principle (no invented primitive frontmatter/format) extending even to hooks: author in whichever native harness shape you like, and APM normalizes.
**Accepted input shapes** (APM normalizes both into an internal vendor-neutral IR before rendering per target):
```json
// "Nested" wrapper (what the docs' canonical example shows)
{ "hooks": { "PreToolUse": [ { "hooks": [ {"type": "command", "command": "./scripts/validate.sh", "timeout": 10} ] } ] } }
// "Naked" top-level settings-slice (Claude Code settings.json shape, unwrapped)
{ "PreToolUse": [ { "hooks": [ {"type": "command", "command": "./scripts/validate.sh", "timeout": 10} ] } ] }
```
Both are accepted; APM's discovery/parsing layer detects and unwraps either. There is no separate `Hook`/`HookPrimitive` dataclass in `primitives/models.py` (unlike `Instruction`) — hooks are represented instead by a dedicated vendor-neutral IR (`apm_cli/integration/hook_ir.py`): `HookHandler(command, platform="all", timeout_seconds, provenance, metadata)` grouped into `HookBinding(event, handlers, matcher, provenance, metadata)` grouped into `HookDocument(bindings)`. This IR is populated during install-time integration, not during the generic primitive-discovery pass used for instructions/contexts/agents.
**Event names are case-convention-sensitive by target and get remapped, not just passed through.** Author in either PascalCase (Claude convention: `PreToolUse`, `PostToolUse`, `UserPromptSubmit`, `SessionStart`, `Stop`) or camelCase (Copilot convention: `preToolUse`, `postToolUse`, etc.) — `_HOOK_EVENT_MAP` per-target dictionaries translate between them during merge/deploy. An event name whose casing doesn't match the target's expected convention *and* has no explicit mapping entry triggers a non-fatal warning at install time (`_emit_hook_event_diagnostics`) — not a hard failure, but a real signal that the event likely won't fire.
**Script path placeholders** are rewritten per target during deploy: `${CLAUDE_PLUGIN_ROOT}/path`, `${CURSOR_PLUGIN_ROOT}/path`, `${PLUGIN_ROOT}/path`, and bare `./path` all get resolved relative to the package root and rewritten to whatever the target expects; bare system commands (no path separators) pass through unchanged.
## Compile-time mapping per target — both are real reconstruction, differently shaped
Neither Claude nor Copilot receives a byte-verbatim copy of the source hook JSON — this is a genuine, structural transform on both sides, driven by `apm_cli/integration/hook_native_formats.py` and `hook_integrator.py`.
**Claude Code — merged into `.claude/settings.json`, not a standalone file.** `claude` is registered in `_MERGE_HOOK_TARGETS` with `config_filename="settings.json"`, `schema_strict=True`. Behavior (per the integrator's own class docstring: "Claude: Merged into .claude/settings.json hooks key + .claude/hooks/<pkg>/"):
- Event bindings are merged into the `"hooks"` key of `.claude/settings.json`, using Claude's native nested-matcher-group shape (`{"hooks": {"PreToolUse": [{"hooks": [{"type": "command", "command": "...", "timeout": N}]}]}}`), with PascalCase event names.
- Any referenced script files are physically copied to `.claude/hooks/<package-name>/`, and the `command` field is rewritten to point at the copied location.
- An ownership sidecar (`apm-hooks.json`) tracks which entries in the shared `settings.json` were APM-installed, so `apm install`/uninstall can cleanly add/remove only its own entries without clobbering hand-authored hooks a user already had in that file.
**Copilot CLI — dedicated per-file deployment, flat/camelCase, field-renamed.** `copilot` is deliberately **not** in `_MERGE_HOOK_TARGETS` (confirmed in `_hook_dropped_targets.py`: "Names not registered in `_MERGE_HOOK_TARGETS` (e.g. `copilot`, which uses per-file, not merged, hook deployment...)"). Instead `PrimitiveMapping("hooks", ".json", "github_hooks")` deploys a dedicated file per source hook file. The native Copilot shape differs structurally from Claude's, per the module docstring:
```json
{
"version": 1,
"hooks": { "preToolUse": [ {"type": "command", "bash": "./scripts/validate.sh", "timeoutSec": 10} ] }
}
```
Differences from the Claude/source shape: flat arrays (no nested matcher-group wrapper), camelCase event keys, a required top-level `"version": 1`, and handler commands split by platform (`bash` / `powershell` keys) instead of a single `command` key, with `timeoutSec` replacing `timeout`.
## Compile-time file placement
| Target | Output location | Mechanism |
|---|---|---|
| Claude Code | `.claude/settings.json` (`"hooks"` key, merged) + scripts copied to `.claude/hooks/<pkg>/` | Merge into existing shared config file, ownership tracked via `apm-hooks.json` sidecar |
| Copilot CLI | `.github/hooks/<name>.json` | Dedicated per-file deploy, reshaped to Copilot's flat/camelCase/`version:1` schema |
## Validation constraints and gotchas
- **Copilot's native payload has an enforced shape** (`_validate_copilot_payload`): top-level `"version"` must equal `1`; `"hooks"` must be an object; each event's value must be a list; each entry must be an object; if an entry has a `"hooks"` key, its value must be a list of objects. These errors are collected and surfaced before any file is written (fail before mutation, not after).
- **Malformed existing config fails closed, not silently.** If `.claude/settings.json` (or another merge target's config) is unreadable/malformed JSON, APM leaves it **byte-identical** and logs an actionable warning rather than overwriting or corrupting it — the same fail-closed posture applies to orphaned `apm-hooks.json` sidecars when their native JSON counterpart is already gone.
- **Dropping a target from `apm.yml`'s `targets:` list does not auto-clean its merged hook entries** unless `apm install`/reconcile logic explicitly walks the complement set (`reconcile_dropped_targets`) — a real, documented gap the code works around rather than a design guarantee; relying on "just remove the target and hooks disappear" is not safe without a fresh `apm install`.
- **Event-casing mismatches are warnings, not errors** — a hook authored with the wrong casing for a target and no applicable rename mapping will silently not fire at runtime; APM only logs a warning at install time, it does not block the install or refuse to deploy the file.
- **No dedicated `Hook`/`HookPrimitive` validation dataclass** exists comparable to `Instruction.validate()` — validation is distributed across `_validate_copilot_payload` (Copilot-shape-specific) and general JSON-parseability checks, not a single primitive-level contract. This mirrors the same "no independent validation model" gap already documented for the agent primitive.

View File

@@ -0,0 +1,66 @@
---
topic: instructions-primitive-schema
source_keys:
- context7-microsoft-apm
- apm-github-repo
---
## File location, naming, and frontmatter
`.apm/instructions/*.instructions.md`. Confirmed as the genuine required extension (not assumed) via APM's own discovery glob in `apm_cli/primitives/discovery.py`: `**/.apm/instructions/*.instructions.md` (and the `.github/instructions/` mirror, plus a bare `**/*.instructions.md` fallback).
Unlike prompts and hooks, instructions **do** have a small, concretely modeled dataclass — `apm_cli.primitives.models.Instruction` — because instructions feed APM's own compile pipeline (they get folded into root context files), not just pass-through deployment:
```python
@dataclass
class Instruction:
name: str
file_path: Path
description: str
apply_to: str # from frontmatter key "applyTo"; empty means global/unconditional
content: str
author: str | None = None
version: str | None = None
source: str | None = None
```
Frontmatter fields: `description` (required by convention — its absence is a validation error) and `applyTo` (a glob or comma-separated glob list, or a YAML sequence — APM normalizes all three input shapes into one canonical comma-separated form internally via `normalize_apply_to`/`parse_apply_to`). No `applyTo` means the rule is treated as **unconditional** — folded into root context files as always-on guidance rather than scoped to specific paths.
`Instruction.validate()` produces these built-in errors/warnings:
- Missing `description` → error: `"Missing 'description' in frontmatter"`.
- Missing `applyTo` → warning-level: `"No 'applyTo' pattern specified -- instruction will apply globally"` (not fatal — it's accepted, just broad).
- Empty body → error: `"Empty content"`.
## Compile-time mapping: two entirely different mechanisms per target
This is the biggest divergence from the agent/skill/prompt primitives, and the one most likely to surprise: **Claude Code does not get a verbatim copy of the `.instructions.md` file at all.**
**Copilot CLI — verbatim, native primitive.** `PrimitiveMapping("instructions", ".instructions.md", "github_instructions")` on the `copilot` target has no `output_compare` flag, so `InstructionIntegrator` copies content through unchanged, preserving the original `applyTo:` frontmatter byte-for-byte (per the integrator's own docstring: "Copilot: `.github/instructions/` (verbatim, preserving applyTo:)"). This is deployed by `apm install`, not `apm compile`.
At **Copilot user scope only** (`~/.copilot/`), individual files are not deployed — Copilot CLI at user scope reads a single `copilot-instructions.md`, so APM concatenates all instructions into that one file instead (`user_primitive_overrides: {"instructions": PrimitiveMapping("", ".md", "copilot_user_instructions")}`). Project-scope behavior (per-file, `.github/instructions/`) is unaffected.
**Claude Code — real reconstruction into `.claude/rules/`, with field-dropping.** `PrimitiveMapping("rules", ".md", "claude_rules", output_compare=True)` marks this as one of APM's four "rule formats" (`RULE_FORMATS = {cursor_rules, claude_rules, windsurf_rules, kiro_steering}`) that transform their source rather than copy it. `InstructionIntegrator._convert_to_claude_rules()`:
- Parses the source frontmatter and pulls only `applyTo` — **`description` is dropped entirely**, not carried into the output in any form.
- Converts `applyTo` into a `paths:` YAML list (one `parse_apply_to()`-split glob per line), e.g. `applyTo: "**/*.py"` → `paths:\n - "**/*.py"`.
- If there was no `applyTo` (unconditional instruction), the output has **no frontmatter at all** — just the raw body, matching Claude's convention that files without `paths:` in `.claude/rules/` apply unconditionally.
- Filename is renamed: `<x>.instructions.md` → `<x>.md` (the primitive's `extension` field, `.md`, replaces the source suffix — this is the general rule for every `output_compare=True` "rule format").
This is architecturally the same category of lossy, real transformation the prior agent-primitive research found for Codex/Kiro agents — except here it's the default behavior for Claude specifically (not an opt-out edge case), and it applies even though Claude and Copilot are both first-class, actively-supported targets.
## Compile-time file placement
| Target | Output path | Transform |
|---|---|---|
| Copilot CLI (project scope) | `.github/instructions/<name>.instructions.md` | Verbatim byte copy, `applyTo:` preserved as-is |
| Copilot CLI (user scope, `~/.copilot/`) | `~/.copilot/copilot-instructions.md` | Concatenated — all instructions merged into one file, because Copilot CLI at user scope reads only that single file |
| Claude Code | `.claude/rules/<name>.md` | Reconstructed: `applyTo` → `paths:` YAML list; `description` dropped; no frontmatter at all if unconditional |
Additionally, **`apm compile`** (distinct from `apm install`) can also fold instruction content directly into root context files — `AGENTS.md` (single-file or per-directory "distributed" mode) and the Claude-specific parallel format `CLAUDE.md`/per-directory `CLAUDE.md` — grouped by directory using `applyTo` pattern analysis (`context_optimizer.optimize_instruction_placement`). To avoid duplicating content between the native `.claude/rules/`+`.github/instructions/` deployment (from `apm install`) and this root-context fold-in (from `apm compile`), a `skip_instructions` config flag (and `compilation.placement.min_instructions_per_file` in `apm.yml`) actively suppresses the redundant copy in AGENTS.md/CLAUDE.md once native per-target files exist — `apm compile --target claude --force-instructions` overrides this dedup when an author explicitly wants both.
## Validation constraints and gotchas
- **The `description` field is real for Copilot but silently discarded for Claude.** An author who relies on `description` to explain *why* a rule exists (common practice, since Copilot's `.instructions.md` UI can surface it) gets that context deleted on every Claude compile — there's no config to keep it as a comment or otherwise.
- **No content-level validation for the `paths:` conversion** — if `applyTo` contains a pattern `parse_apply_to` can't split sensibly, the resulting `paths:` list is whatever falls out; no dedicated schema check catches a malformed glob before deploy.
- **Directory-distribution logic for AGENTS.md/CLAUDE.md is heuristic, not declarative** — `context_optimizer.optimize_instruction_placement` picks placement directories from `applyTo` patterns algorithmically; `compilation.placement.min_instructions_per_file` in `apm.yml` (default effectively 1) is the only tuning knob, and setting it above 1 causes under-populated directories to have their instructions bubbled up to the parent directory rather than dropped.
- **Same "no dedicated primitive validation function" gap noted for agents** — `Instruction.validate()` in `primitives/models.py` is the only validation, and it is invoked as part of the generic primitive-discovery/compile pipeline, not as a standalone `apm audit` check comparable to what exists for `apm.yml` itself.

View File

@@ -0,0 +1,53 @@
---
topic: prompt-primitive-schema
source_keys:
- context7-microsoft-apm
- apm-github-repo
---
## File location, naming, and frontmatter
`.apm/prompts/*.prompt.md` (also discovered at the package root). Filename minus the `.prompt.md` suffix becomes the prompt's identity — used verbatim as the Copilot filename and, after transformation, as the Claude command name. No required-extension ambiguity: it is genuinely `.prompt.md`, confirmed both in docs and in APM's own `PromptIntegrator.find_prompt_files` (`*.prompt.md`) and `CommandIntegrator.find_prompt_files` (same glob).
There is no single closed frontmatter schema — APM's P1 "no invented primitive frontmatter" principle applies here too, so a prompt author writes whatever keys their primary target needs and APM passes or drops per-target. Keys seen in APM's own docs/examples:
| Field | Purpose |
|---|---|
| `description` | Shown in Copilot's prompt picker / used for discovery |
| `input` | List of parameter names (simple list, or list of `{name: description}` objects) referenced in body as `${input:name}` |
| `allowed-tools` (or `allowedTools`) | Tool allowlist for the prompt's execution |
| `argument-hint` (or `argumentHint`) | Human-readable hint for expected arguments |
| `model` | Model override when the prompt runs |
| `author`, `mcp`, `parameters` | Cursor/other-target-specific metadata — **not preserved** by the shared Claude/Cursor command transformer (see below) |
**Workflow-prompt-only keys** (Copilot App / Copilot Workflows, not Copilot CLI): `name`, `interval` (`manual`/`hourly`/`daily`/`weekly`), `schedule_hour` (0–23 UTC), `schedule_day` (0–6, weekly only), `mode` (`interactive`/`plan`), `reasoning_effort`. These are flat top-level keys on the same `.prompt.md` file, consumed only by the Copilot App scheduler integration — irrelevant to Claude Code / Copilot CLI compilation and should not be treated as universal prompt schema.
## Compile-time mapping: verbatim for Copilot, real reconstruction for Claude
**Copilot CLI target — verbatim copy.** `PrimitiveMapping("prompts", ".prompt.md", "github_prompt")` on the `copilot` target profile carries no `output_compare` flag, and `PromptIntegrator.copy_prompt()` reads the source file and writes it out unchanged (only markdown link targets get rewritten) via `copy_prompt: "Copy prompt file verbatim with link resolution."`. Every frontmatter key — including `author`, `mcp`, `parameters` — survives. Filename is untouched (`get_target_filename` returns `source_file.name`, "no -apm suffix").
**Claude Code target — real reconstruction into a slash command, with field-dropping.** There is no `prompts:` key at all in Claude's `TargetProfile.primitives` dict; instead prompts route through the shared `CommandIntegrator`, which transforms `.prompt.md` → Claude custom slash command markdown. `CommandIntegrator._transform_prompt_to_command()`:
- Strips the `.prompt.md` suffix from the filename to derive `command_name`.
- Builds an entirely new frontmatter object containing **only** these preserved keys: `description`, `allowed-tools` (accepts `allowedTools` alias), `model`, `argument-hint` (accepts `argumentHint` alias).
- Maps APM's `input:` list to Claude's `arguments:` list, and synthesizes `argument-hint` from it if not already set.
- Rewrites body placeholders `${input:name}` / `${{input:name}}` to Claude's native `$name` syntax via regex substitution.
- Computes `dropped_keys = source_frontmatter_keys - preserved_keys` and surfaces it as an install-time diagnostic warning — so `author`, `mcp`, `parameters`, and any other non-listed key are silently dropped from the compiled output but *not* silently dropped from the user's awareness (a warning fires).
- Cursor reuses this exact same transformer (`claude_command` format_id) — same preserved-key set, same drops.
## Compile-time file placement
| Target | Output path | Transform |
|---|---|---|
| Copilot CLI | `.github/prompts/<name>.prompt.md` | Verbatim byte copy (links resolved) |
| Claude Code | `.claude/commands/<name>.md` | Reconstructed: only `description`/`allowed-tools`/`model`/`argument-hint`/`arguments` survive; `input:` → `arguments:`; `${input:x}` → `$x` |
Invocation surface differs correspondingly: Copilot exposes it via the prompts picker UI (select by name); Claude exposes it as `/<name> <args>` (same pattern Cursor, OpenCode, Gemini CLI, and Windsurf's workflows menu use for their own compiled copies).
## Validation constraints and gotchas
- **Input-name validation is real, not just documentation.** `_extract_input_names()` enforces `[A-Za-z][\w-]{0,63}` on every name pulled from `input:`; anything that fails is dropped from `arguments:` and reported as a warning listing up to 5 rejected names (`input: rejected N invalid name(s) ... `). A malformed `input:` entry does not fail the install — it silently loses that one argument.
- **Filename-derived identity is security-checked.** `integrate_commands_for_target` calls `validate_path_segments(base_name, context="command filename")` specifically to reject a package shipping a `.prompt.md` file with a manipulated relative name (e.g. `../../evil.prompt.md`) that would otherwise escape the target commands directory.
- **The dropped-key warning is the only signal a Claude-only author gets** that Cursor-specific frontmatter (`author`, `mcp`, `parameters`) never reached the deployed file — there is no error, no hard failure, and no config flag to preserve those keys for Claude; the shared transformer's preserved-key list is fixed in code (`_PRESERVED_COMMAND_KEYS`), not configurable per package.
- **No dedicated `Prompt`/`PromptPrimitive` validation class exists** in `apm_cli/models/validation.py` or `apm_cli/primitives/models.py` — same gap pattern documented for the agent primitive. `apm.yml`'s `type: prompts` package-content-type ("Commands/prompts only, no instructions or skills") is validated at the package-type-detection level, not the individual-prompt level.
- Slash commands and prompts share one source directory and one glob (`.apm/prompts/*.prompt.md`) — there is no separate `.apm/commands/` primitive; "command" is purely a per-target compiled *name* for the same source file, not a distinct authoring primitive.

View File

@@ -4,5 +4,12 @@
- **URL:** context7:/microsoft/apm
- **Description:** Microsoft APM (Agent Package Manager) — open-source dependency manager for AI agent configuration (skills, prompts, instructions, agents, hooks, MCP/LSP deps), applying a declare/lock/install/audit workflow.
- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, examples.md, troubleshooting.md, testing-and-validation.md, marketplace-and-registries.md, monorepo-and-repo-shapes.md
- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, examples.md, troubleshooting.md, testing-and-validation.md, marketplace-and-registries.md, monorepo-and-repo-shapes.md, agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md
- **Status:** `extracted`
## apm-github-repo
- **URL:** https://github.com/microsoft/apm
- **Description:** APM's own Python source (`src/apm_cli/`) read directly for schema/compile-mapping ground truth where Context7's doc snippets were thin — specifically `primitives/models.py` and `primitives/discovery.py` (primitive dataclasses and discovery globs), `integration/prompt_integrator.py`, `integration/command_integrator.py`, `integration/instruction_integrator.py`, `integration/hook_integrator.py`, `integration/hook_native_formats.py`, `integration/hook_ir.py`, `integration/_hook_dropped_targets.py`, `integration/targets.py` (`KNOWN_TARGETS`/`PrimitiveMapping` per-target deploy config), `compilation/claude_formatter.py` and `compilation/distributed_compiler.py` (instruction fold-in to CLAUDE.md/AGENTS.md), and `models/validation.py`.
- **Contributing files:** agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md
- **Status:** `extracted`

View File

@@ -1,10 +1,29 @@
# agent-audit
Audits a Claude Code and Copilot agent definition file pair for correctness and quality.
Audits an agent definition for correctness and quality — a single vendor-neutral file at
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
## What it does
Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `sources.md` at the plugin root), then qualitative checks on description phrasing and system prompt quality. Step 1 also runs a Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the `Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is unavailable or reports `0 files` scanned. Produces a compact findings report in the same format as `skill-audit`.
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
vendor-neutral allowlist (`name`, `description`, `model`, `source_keys` — the last for
provenance tracking, checked separately by `validate-provenance.sh` against `sources.md`; see
ADR-0016), since `apm compile`
copies frontmatter verbatim to both harnesses and an unsafe field can't be silently dropped for
just one of them.
At **project/user scope**, accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives
the counterpart automatically, and validates both. Runs structural checks via `validate.sh`
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
Copilot-only fields in the CC file), provenance chain validation via `validate-provenance.sh`
(checks `source_keys` against `sources.md` at the plugin root — plugin/APM scope only), then
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the
`Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert
is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is
unavailable or reports `0 files` scanned. Produces a compact findings report in the same format
as `skill-audit`.
## Usage

View File

@@ -26,8 +26,8 @@ metadata:
## Gotchas
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
- The unit of authoring at project/user scope is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention at those scopes — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures. **At plugin/APM scope there is no pair** — the unit of authoring is a single vendor-neutral `.apm/agents/<name>.agent.md` file, and Pair Consistency does not apply there at all (see below).
- Scope is detected by walking up from the agent file's directory: at each level, if `apm.yml` exists AND contains a top-level `type: instructions|skill|hybrid|prompts` line, that directory is an APM package root — plugin/APM scope. A `type:`-less `apm.yml` is marketplace-only (see `docs/research/docs/microsoft-apm/monorepo-and-repo-shapes.md`) — skip it and keep walking up. Otherwise, if `.git` is a directory at that level, stop there — project scope. If neither is found before the filesystem root, fall back to user scope at `$HOME`. `plugin.json`/`.claude-plugin/plugin.json` are no longer scope signals for this skill — a directory with only a `plugin.json` and no `apm.yml` falls through to project (or user) scope.
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
- Do not output findings while auditing — gather internally, surface in Step 3 report.
@@ -36,16 +36,21 @@ metadata:
```bash
bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file>
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file> # project/user scope
scripts/vale-wrap.sh <path-to-apm-agent-file> # plugin/APM scope — single file
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
The script accepts either the CC file, the Copilot file, or (at plugin/APM scope) the single `.apm/agents/<name>.agent.md` file. It detects provider from extension and scope from the walk-up above, then runs the checks for that scope.
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". Run it against both files of the pair (not just the one passed in). `Kyberforge` applies to both files; `KyberforgeCopilot` applies to the `.agent.md` file only, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a Copilot description — there's nothing to flag in the CC file, so it isn't scoped there. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md` (`name`, `description`, `model`, `source_keys` — nothing else; `source_keys` is provenance metadata, not a provider-specific field, and is validated separately by `validate-provenance.sh` against `sources.md`). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
`validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside `name`/`description`/`model`/`source_keys` (read the `apm-agent-allowlist` section of `references/field-inventory.md`; `source_keys` carries provenance metadata, checked separately by `validate-provenance.sh` against `sources.md`).
## Step 2 — Qualitative checks
@@ -69,18 +74,29 @@ If a description finding is borderline, read `references/description-quality.md`
- Look for patterns like `# Optional. <long explanation>` or extensive inline guidance (more than 1–2 lines per field) that should be condensed or removed before shipping.
- This mirrors skill-audit's body-discipline check but applies to template documentation in the frontmatter — template guidance belongs in development; agent-ready files should have minimal comments.
**Pair consistency (cross-file):**
**Pair consistency (cross-file) — project/user scope only:**
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
- **Does not apply at plugin/APM scope** — there is only one file, by design; do not raise a Pair Consistency finding there under any circumstance.
**Unexpressable Claude-only behavior — plugin/APM scope only:**
- Read the description and body. If either implies a need the vendor-neutral frontmatter can no longer express — tool restriction, `isolation`, `memory`, or another Claude-only behavior that a hand-authored CC file could have declared — flag it as a SUGGESTION, never a FAIL. This is a known upstream schema limitation (APM's agent primitive has no per-target compile integrator, so `tools:`/`isolation`/etc. can't be emitted safely to both CC and Copilot — ADR-0016), not an authoring mistake. The finding exists to give the author visibility into the gap, not to imply the schema can be made to do something it can't.
- Example: a body that says "only use Read and Grep, never Edit" but the frontmatter has no `tools` field to enforce it — SUGGESTION, not FAIL.
## Step 3 — Report
Open with a coverage line:
Open with a coverage line. At project/user scope:
```text
Checked: structure · provider-safety · description · body · comment-discipline · pair-consistency · provenance
```
At plugin/APM scope, omit `pair-consistency` — it does not apply when there is no pair:
```text
Checked: structure · provider-safety · description · body · comment-discipline · provenance
```
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
For each finding:

View File

@@ -15,10 +15,6 @@ name description tools disallowedTools model effort maxTurns permissionMode skil
maxTurns isolation memory permissionMode effort hooks mcpServers disallowedTools skills initialPrompt color background
## plugin-silently-ignored-fields
hooks mcpServers permissionMode
## copilot-fields
name description tools target model disable-model-invocation user-invocable mcp-servers metadata
@@ -26,3 +22,7 @@ name description tools target model disable-model-invocation user-invocable mcp-
## copilot-only-fields
target disable-model-invocation user-invocable mcp-servers metadata
## apm-agent-allowlist
name description model source_keys

Allowlist contradicts agent-author's own instructions. apm-agent-allowlist omits source_keys, contradicting agent-author/SKILL.md's explicit instruction to add source_keys at plugin/APM scope when research-sourced — so a correctly-authored file fails validation.

Reproduced: a package-scope agent file with source_keys: fails validate.sh with FAIL field 'source_keys' is not in the vendor-neutral APM agent allowlist (description, model, name).

**Allowlist contradicts agent-author's own instructions.** `apm-agent-allowlist` omits `source_keys`, contradicting `agent-author/SKILL.md`'s explicit instruction to add `source_keys` at plugin/APM scope when research-sourced — so a correctly-authored file fails validation. Reproduced: a package-scope agent file with `source_keys:` fails `validate.sh` with `FAIL field 'source_keys' is not in the vendor-neutral APM agent allowlist (description, model, name)`.

View File

@@ -5,8 +5,10 @@ usage() {
cat <<EOF
Review

Documented parity with validate.sh doesn't hold. This usage text and agent-author/SKILL.md both claim this script walks up 'the same way validate.sh does', but it lacks validate.sh's $HOME-boundary and quote-tolerance handling — the documented behavioral parity is false (see the two paired findings above on this PR).

**Documented parity with validate.sh doesn't hold.** This usage text and `agent-author/SKILL.md` both claim this script walks up 'the same way `validate.sh` does', but it lacks `validate.sh`'s `$HOME`-boundary and quote-tolerance handling — the documented behavioral parity is false (see the two paired findings above on this PR).
Usage: validate-provenance.sh <agent-file>
Validate that an agent pair's sources provenance chain is complete and internally consistent.
Operates at plugin scope only — exits 0 silently for project and user scope agents.
Validate that an agent's sources provenance chain is complete and internally consistent.
Operates at plugin/APM scope only (a single vendor-neutral .apm/agents/<name>.agent.md
inside a package with a type:-bearing apm.yml) — exits 0 silently for project and user
scope agents.
Arguments:
agent-file Path to either the Claude Code .md or Copilot .agent.md agent file.
@@ -47,23 +49,46 @@ agent_file = os.path.abspath(sys.argv[1])
fname = os.path.basename(agent_file)
agent_dir = os.path.dirname(agent_file)
# --- Detect provider ---
if fname.endswith('.agent.md'):
provider = 'copilot'
name_stem = fname[:-len('.agent.md')]
elif fname.endswith('.md'):
provider = 'claude-code'
name_stem = fname[:-len('.md')]
else:
# --- Sanity-check extension (single vendor-neutral .agent.md file at plugin/APM scope) ---
if not (fname.endswith('.agent.md') or fname.endswith('.md')):
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
sys.exit(2)
# --- Find plugin root ---
# Matches a top-level `type:` line whose value is exactly one of the four

Not quote-tolerant, unlike validate.sh. TYPE_RE lacks the quote-tolerant ['\"]? group that validate.sh's APM_TYPE_RE has, so a quoted type: "skill" value is recognized as plugin scope by validate.sh but not by validate-provenance.sh (or new-agent.sh/new-skill.sh).

Reproduced: with type: "skill" in apm.yml, validate.sh correctly detects plugin scope; validate-provenance.sh on the same file exits 0 silently even with an unresolved source_keys entry that should have FAILed.

**Not quote-tolerant, unlike validate.sh.** `TYPE_RE` lacks the quote-tolerant `['\"]?` group that `validate.sh`'s `APM_TYPE_RE` has, so a quoted `type: "skill"` value is recognized as plugin scope by `validate.sh` but not by `validate-provenance.sh` (or `new-agent.sh`/`new-skill.sh`). Reproduced: with `type: "skill"` in `apm.yml`, `validate.sh` correctly detects plugin scope; `validate-provenance.sh` on the same file exits 0 silently even with an unresolved `source_keys` entry that should have FAILed.
# package content types — identical to validate.sh's APM_TYPE_RE. Group 1's
# optional quote must be closed by \1 (or nothing), and the value must be
# followed by whitespace/end-of-line so a malformed value like `prompts-only`
# doesn't false-match on the `prompts` prefix.
TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
# --- Find package root: walk up for the nearest ancestor apm.yml that

Silent no-op on plugin.json-only plugins. find_plugin_root() shares validate.sh's plugin.json blind spot, so provenance validation silently exits 0 (no check performed) for every existing plugin.json-only plugin.

Reproduced: validate-provenance.sh plugins/kyberforge/agents/apm-orchestrate.md exits 0 with no output even though the file has real source_keys and a matching sources.md exists — the check this script exists to run never fires, with no signal that it was skipped.

**Silent no-op on plugin.json-only plugins.** `find_plugin_root()` shares `validate.sh`'s plugin.json blind spot, so provenance validation silently exits 0 (no check performed) for every existing plugin.json-only plugin. Reproduced: `validate-provenance.sh plugins/kyberforge/agents/apm-orchestrate.md` exits 0 with no output even though the file has real `source_keys` and a matching `sources.md` exists — the check this script exists to run never fires, with no signal that it was skipped.
# declares a top-level type: field. An apm.yml with no type: field is a
# marketplace-only manifest (see monorepo-and-repo-shapes.md) — skip it and
# keep walking. Stop at a $HOME boundary, a .git boundary, or the filesystem
# root: none of these is plugin/APM scope, so this script has nothing to
# check there.
def find_plugin_root(start_dir):
home = os.path.expanduser('~')
current = os.path.abspath(start_dir)
while True:
if (os.path.isfile(os.path.join(current, 'plugin.json')) or os.path.isfile(os.path.join(current, '.claude-plugin', 'plugin.json'))):
return current
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml):
with open(apm_yml) as f:
if any(TYPE_RE.match(line) for line in f):
return current
# $HOME is a non-plugin-scope boundary — checked before the .git test
# below (mirrors validate.sh's detect_scope ordering), so a
# dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) can't shadow
# this check by being its own .git repo. Without this, the walk could
# continue past $HOME toward the filesystem root looking for a
# type-bearing apm.yml, misclassifying a user/project-scope file as
# plugin scope in rare ancestor layouts.
if current == home:
return None
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
# in a git worktree — exists() covers both.
if os.path.exists(os.path.join(current, '.git')):
return None
parent = os.path.dirname(current)
if parent == current:
return None
@@ -73,12 +98,6 @@ plugin_root = find_plugin_root(agent_dir)
if plugin_root is None:
sys.exit(0)
# --- Derive counterpart ---
if provider == 'copilot':
counterpart = os.path.join(agent_dir, name_stem + '.md')
else:
counterpart = os.path.join(agent_dir, name_stem + '.agent.md')
sources_md_path = os.path.join(plugin_root, 'sources.md')
# --- Helpers ---
@@ -166,15 +185,9 @@ def get_source_keys_from_file(fpath):
fm, _ = parse_frontmatter(content)
return parse_source_keys(fm)
# Plugin/APM scope is a single vendor-neutral file — no counterpart to merge.
given_keys = get_source_keys_from_file(agent_file)
counterpart_keys = get_source_keys_from_file(counterpart)
# Deduplicated union, preserving order
seen = set()
all_source_keys = []
for k in given_keys + counterpart_keys:
if k not in seen:
seen.add(k)
all_source_keys.append(k)
all_source_keys = given_keys
sources_md_exists = os.path.isfile(sources_md_path)
@@ -212,8 +225,8 @@ for line in sources_content.splitlines():
)
break
# --- Check 2: source_keys in agent files → slug exists in sources.md ---
for fpath, keys in [(agent_file, given_keys), (counterpart, counterpart_keys)]:
# --- Check 2: source_keys in the agent file → slug exists in sources.md ---
for fpath, keys in [(agent_file, given_keys)]:
if not keys:
continue
rel = os.path.relpath(fpath, plugin_root)

View File

@@ -5,10 +5,15 @@ usage() {
cat <<EOF
Usage: validate.sh <agent-file>
Validate a Claude Code or Copilot agent file pair against the agent definition spec.
Validate an agent definition file against the agent definition spec.
At plugin/APM scope, <agent-file> is a single vendor-neutral
.apm/agents/<name>.agent.md file (frontmatter allowlist: name, description,
model — no counterpart file). At project or user scope, <agent-file> is
either half of a Claude Code .md / Copilot .agent.md pair.
Arguments:
agent-file Path to either the Claude Code .md or Copilot .agent.md agent file.
agent-file Path to the agent file (or either half of a project/user-scope pair).
Exit codes:
0 All checks passed (may include SUGGESTIONs)
@@ -73,7 +78,7 @@ def parse_section_tokens(content, section_name):
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
plugin_ignored_fields = parse_section_tokens(inv_content, 'plugin-silently-ignored-fields')
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
# Tools the runtime withholds from subagents regardless of the tools field
SUBAGENT_UNAVAILABLE_TOOLS = {
@@ -83,48 +88,7 @@ SUBAGENT_UNAVAILABLE_TOOLS = {
# Copilot body length limit (chars) — content beyond this is silently truncated
COPILOT_BODY_LIMIT = 30000
# --- Detect scope ---
def detect_scope(start_dir):
current = os.path.abspath(start_dir)
while True:
if (os.path.isfile(os.path.join(current, 'plugin.json')) or
os.path.isfile(os.path.join(current, '.claude-plugin', 'plugin.json'))):
return 'plugin', current
if os.path.isdir(os.path.join(current, '.git')):
return 'project', current
parent = os.path.dirname(current)
if parent == current:
return 'user', os.path.expanduser('~')
current = parent
agent_dir = os.path.dirname(agent_file)
scope, scope_root = detect_scope(agent_dir)
# --- Derive counterpart path ---
if scope == 'plugin':
if provider == 'copilot':
counterpart = os.path.join(agent_dir, name_stem + '.md')
counterpart_provider = 'claude-code'
else:
counterpart = os.path.join(agent_dir, name_stem + '.agent.md')
counterpart_provider = 'copilot'
elif scope == 'project':
if provider == 'claude-code':
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
else: # user
home = os.path.expanduser('~')
if provider == 'claude-code':
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
# --- Helpers ---
# --- Helpers (shared by every scope) ---
failed = False
suggestions = []
@@ -167,7 +131,182 @@ def is_copilot_cloud_ide(fpath):
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
def check_file(fpath, file_provider, is_plugin_scope):
# --- Detect scope ---
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
# exactly one of the four package content types. Group 1 captures an optional
# opening quote; \1 requires the same character (or nothing) to close it, so
# "skill" and '"skill"' both match but a mismatched quote doesn't. The value
# must then be followed by whitespace or end-of-line — not just a non-word
# character — so a malformed value like `prompts-only` is correctly rejected
# instead of false-matching on the `prompts` prefix.
APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
def find_apm_package_root(apm_yml_path):
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
manifest, not a type:-less marketplace-only apm.yml)."""
with open(apm_yml_path) as f:
for line in f:
if APM_TYPE_RE.match(line):
Review

Misclassifies legacy plugin.json-only plugins. detect_scope() no longer recognizes plugin.json (only apm.yml), so existing plugin.json-only plugins are misclassified as project scope with the wrong counterpart path.

Reproduced: validate.sh plugins/kyberforge/agents/apm-orchestrate.md — a real, correctly-paired agent in this very plugin — fails with FAIL counterpart file not found: .github/agents/apm-orchestrate.agent.md, even though the real counterpart sits right next to it.

**Misclassifies legacy plugin.json-only plugins.** `detect_scope()` no longer recognizes `plugin.json` (only `apm.yml`), so existing plugin.json-only plugins are misclassified as project scope with the wrong counterpart path. Reproduced: `validate.sh plugins/kyberforge/agents/apm-orchestrate.md` — a real, correctly-paired agent in this very plugin — fails with `FAIL counterpart file not found: .github/agents/apm-orchestrate.agent.md`, even though the real counterpart sits right next to it.
return True
return False
def detect_scope(start_dir):
home = os.path.expanduser('~')
original_start = os.path.abspath(start_dir)
# Agent files conventionally live exactly two path segments below their
# scope root — <root>/.claude/agents, <root>/.github/agents,
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
# recovers the same root new-agent.sh would have been invoked with to
# produce this exact file, independent of how far the walk below has to

.git-as-file (worktrees) not recognized as a boundary. The project-boundary check uses isdir()/[[ -d ]] in all four rewritten walk-up implementations, which misses git worktrees where .git is a regular file (gitdir: ...), not a directory.

In a git worktree add checkout, none of the four walk-ups (validate.sh, validate-provenance.sh, new-agent.sh, new-skill.sh) recognize .git as a project boundary, so the walk continues past the intended project root for any agent/skill work done inside a worktree.

**`.git`-as-file (worktrees) not recognized as a boundary.** The project-boundary check uses `isdir()`/`[[ -d ]]` in all four rewritten walk-up implementations, which misses git worktrees where `.git` is a regular file (`gitdir: ...`), not a directory. In a `git worktree add` checkout, none of the four walk-ups (`validate.sh`, `validate-provenance.sh`, `new-agent.sh`, `new-skill.sh`) recognize `.git` as a project boundary, so the walk continues past the intended project root for any agent/skill work done inside a worktree.
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
# `root` vs `current` distinction even though validate.sh is handed a
# file's directory, not the scope root itself.

Root-fallback scope disagrees with new-agent.sh. detect_scope()'s filesystem-root fallback returns 'user' scope (pinned to real $HOME), while new-agent.sh's equivalent fallback returns 'project' scope rooted at the given path — the two scripts disagree on any directory outside $HOME with no .git/apm.yml above it.

Reproduced: new-agent.sh test-agent /tmp/scratch (outside $HOME, no .git anywhere above it) correctly creates a project-scope pair. Running validate.sh on the created file then falls back to user scope and looks for the counterpart at $HOME/.copilot/agents/test-agent.agent.md, failing with FAIL counterpart file not found even though the valid pair sits right there.

**Root-fallback scope disagrees with new-agent.sh.** `detect_scope()`'s filesystem-root fallback returns 'user' scope (pinned to real `$HOME`), while `new-agent.sh`'s equivalent fallback returns 'project' scope rooted at the given path — the two scripts disagree on any directory outside `$HOME` with no `.git`/`apm.yml` above it. Reproduced: `new-agent.sh test-agent /tmp/scratch` (outside `$HOME`, no `.git` anywhere above it) correctly creates a project-scope pair. Running `validate.sh` on the created file then falls back to user scope and looks for the counterpart at `$HOME/.copilot/agents/test-agent.agent.md`, failing with `FAIL counterpart file not found` even though the valid pair sits right there.
#
# That arithmetic is only trustworthy when the path actually has this
# shape: parent directory literally named "agents", grandparent one of
# the four known scope-dir names. A hand-placed or otherwise
# non-conventional agent file (never produced by new-agent.sh) has no
# such guarantee — blindly trusting two-segments-up there could point at
# an unrelated ancestor. conventional_shape gates every use of
# conventional_root below; when it's false, the walked-to `current`
# directory is used instead, the same fallback this function used before
# conventional_root existed.
scope_dir_name = os.path.basename(os.path.dirname(original_start))
conventional_shape = (
os.path.basename(original_start) == 'agents'
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
)
conventional_root = os.path.dirname(os.path.dirname(original_start))
current = original_start
while True:
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current
# $HOME is the user-scope boundary — checked before the .git test
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
# can't shadow user scope by being its own .git repo. 'user' scope
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
# conventional two-segments-below-root depth (i.e. $HOME IS that
# root, matching the real ~/.claude/agents or ~/.copilot/agents
# shape). Any other walk-up into $HOME — a marker-less directory
# nested deeper than that convention — resolves to project scope
# instead: a stray directory under $HOME can't be silently
# redirected into the shared global ~/.claude or ~/.copilot agent
# directories.
if current == home:
if original_start == home or (conventional_shape and conventional_root == home):
return 'user', home
return 'project', conventional_root if conventional_shape else current
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
# in a git worktree — exists() covers both. Returns conventional_root,
# not current: new-agent.sh's project-scope file placement always
# uses its `$ROOT` argument directly, never the walked-up `.git`
# location, so a <root> one or more levels below the repo's .git
# (a subdirectory of a larger git-tracked tree — explicitly a
# supported case per new-agent.sh's usage text) must resolve to the
# same root new-agent.sh actually wrote to, not to the .git dir —
# unless the path lacks the conventional shape, in which case that
# arithmetic isn't trustworthy and current is used instead.
if os.path.exists(os.path.join(current, '.git')):
return 'project', conventional_root if conventional_shape else current
parent = os.path.dirname(current)
if parent == current:
return 'project', conventional_root if conventional_shape else current
current = parent
agent_dir = os.path.dirname(agent_file)
scope, scope_root = detect_scope(agent_dir)
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
def check_apm_agent_file(fpath, allowlist, stem):
local_fname = os.path.basename(fpath)
with open(fpath) as f:
content = f.read()
fm, body = parse_frontmatter(content)
if fm is None:
fail(f"no valid YAML frontmatter (---...---) — {local_fname}")
return
# The apm-agent.md template embeds its authoring guidance as HTML
# comments inside the frontmatter block (so they render invisible in a
# Markdown preview but stay visible in the raw file). get_frontmatter_keys
# silently ignores any line that isn't a `key:` match, so a comment left
# behind at ship time would otherwise pass unnoticed — yet apm compile
# copies this frontmatter verbatim to both harnesses, and `<!-- -->` is
# not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016).
if re.search(r'<!--|-->', fm):
fail(f"frontmatter still contains template HTML comments (<!-- ... -->) "
f"— delete them before shipping — {local_fname}")
# Allowlist: only name/description/model may appear — no tools, no
# Claude-only or Copilot-only fields. apm compile verbatim-copies
# frontmatter to every target, so anything else is unsafe on at least
# one harness (ADR-0016).
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key not in allowlist:
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
f"({', '.join(sorted(allowlist))}) — {local_fname}")
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
name_val = extract_field(fm, 'name')
if not name_val:
fail(f"name field is missing or empty — {local_fname}")
else:
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
if name_val != stem:
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
# description — required, non-empty, no placeholder
desc_val = extract_field(fm, 'description')
if not desc_val:
fail(f"description field is missing or empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(desc_val):
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
# body — required, non-empty, no placeholder; same Copilot truncation risk
# applies since this file compiles verbatim into a real Copilot file downstream.
if not body.strip():
fail(f"system prompt body is empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(body):
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
if len(body) > COPILOT_BODY_LIMIT:
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
f"content beyond the limit is silently truncated by the Copilot runtime "
f"once apm compile emits it downstream — {local_fname}")
if scope == 'plugin':
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
for s in suggestions:
print(f"SUGGESTION {s}")
sys.exit(1 if failed else 0)
# --- Project/user scope: unchanged CC/Copilot pair validation ---
# --- Derive counterpart path ---
if scope == 'project':
if provider == 'claude-code':
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
else: # user
home = os.path.expanduser('~')
if provider == 'claude-code':
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
def check_file(fpath, file_provider):
local_fname = os.path.basename(fpath)
with open(fpath) as f:
content = f.read()
@@ -227,13 +366,6 @@ def check_file(fpath, file_provider, is_plugin_scope):
if key in copilot_only_fields:
fail(f"Copilot-only field '{key}' present in CC file — {local_fname}")
# Silently-ignored fields in plugin-scope CC file
if file_provider == 'claude-code' and is_plugin_scope:
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key in plugin_ignored_fields:
fail(f"plugin-silently-ignored field '{key}' present in plugin-scope CC file — {local_fname}")
# Subagent-unavailable tools listed in tools field
tools = extract_tools_list(fm)
unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS
@@ -246,9 +378,8 @@ if not os.path.isfile(counterpart):
sys.exit(1)
# --- Check both files ---
is_plugin = (scope == 'plugin')
check_file(agent_file, provider, is_plugin)
check_file(counterpart, counterpart_provider, is_plugin)
check_file(agent_file, provider)
check_file(counterpart, counterpart_provider)
for s in suggestions:
print(f"SUGGESTION {s}")

View File

@@ -8,26 +8,24 @@ setup() {
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate-provenance.sh"
TMPDIR="$(mktemp -d)"
# Helper: create a plugin root with plugin.json and an agents/ directory
make_plugin() {
# Helper: create an APM package root at <root> (apm.yml with a top-level
# type: line — a real package manifest, not marketplace-only) plus a
# single vendor-neutral agent file at <root>/.apm/agents/<name>.agent.md.
make_package() {
local root="$1"
mkdir -p "$root/agents"
echo '{"name":"test-plugin","version":"0.1.0"}' > "$root/plugin.json"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
}
# Helper: create a clean agent pair (no source_keys)
make_clean_pair() {
# Helper: create a clean agent file (no source_keys)
make_clean_agent() {
local root="$1"
local name="${2:-my-agent}"
cat > "$root/agents/${name}.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent.
EOF
cat > "$root/agents/${name}.agent.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
@@ -37,12 +35,12 @@ You are a test agent.
EOF
}
# Helper: create a CC agent file with source_keys
make_cc_with_source_keys() {
# Helper: create an agent file with source_keys
make_agent_with_source_keys() {
local root="$1"
local name="${2:-my-agent}"
local slug="${3:-my-source}"
cat > "$root/agents/${name}.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
@@ -50,37 +48,6 @@ source_keys:
- ${slug}
---
You are a test agent.
EOF
}
# Helper: create a Copilot agent file with source_keys
make_copilot_with_source_keys() {
local root="$1"
local name="${2:-my-agent}"
local slug="${3:-my-source}"
cat > "$root/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
source_keys:
- ${slug}
---
You are a test agent.
EOF
}
# Helper: create a minimal Copilot file without source_keys
make_copilot_clean() {
local root="$1"
local name="${2:-my-agent}"
cat > "$root/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent.
EOF
}
@@ -89,7 +56,7 @@ EOF
make_sources_md() {
local root="$1"
local slug="${2:-my-source}"
local contrib="${3:-agents/my-agent.md, agents/my-agent.agent.md}"
local contrib="${3:-.apm/agents/my-agent.agent.md}"
local research="${4:-(none)}"
cat > "$root/sources.md" <<EOF
# Sources
@@ -120,12 +87,56 @@ teardown() {
}
# ---------------------------------------------------------------------------
# Non-plugin scope → exit 0 silently
# Non-plugin/APM scope → exit 0 silently
# ---------------------------------------------------------------------------
@test "non-plugin scope: no plugin.json in tree → exit 0, no output" {
local dir="$TMPDIR/no-plugin"
@test "non-plugin scope: no apm.yml in tree → exit 0, no output" {
local dir="$TMPDIR/no-package"
mkdir -p "$dir/.apm/agents"
cat > "$dir/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: apm.yml present but type:-less (marketplace-only) → exit 0, no output" {
local dir="$TMPDIR/marketplace-only"
mkdir -p "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: root-marketplace
marketplace:
owner: someone
packages:
- ./packages/plugin-a
EOF
cat > "$dir/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: bare plugin.json (no apm.yml) is no longer a scope signal → exit 0, no output" {
local dir="$TMPDIR/old-plugin-json-only"
mkdir -p "$dir/agents"
echo '{"name":"test-plugin","version":"0.1.0"}' > "$dir/plugin.json"
cat > "$dir/agents/my-agent.md" <<EOF
---
name: my-agent
@@ -141,15 +152,100 @@ EOF
assert_output ""
}
@test "non-plugin scope: walk-up stops at .git boundary before reaching an ancestor apm.yml" {
local dir="$TMPDIR/repo"
mkdir -p "$dir/.git" "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
mkdir -p "$dir/sub/.apm/agents"
cat > "$dir/sub/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
# sub/ has no .git and no apm.yml of its own; the real package apm.yml
# lives at $dir, but $dir/.git means the walk from sub/ should stop at
# sub/ itself only if sub/ had a .git — here .git is at $dir, ABOVE
# sub/, so the walk from sub/ reaches $dir/apm.yml before any .git.
# This test instead verifies the walk finds that package root correctly
# (a positive case) — see the dedicated .git-stops-first test below for
# the negative case.
run bash "$SCRIPT" "$dir/sub/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "non-plugin scope: \$HOME boundary stops the walk before reaching an ancestor apm.yml above \$HOME" {
# A type-bearing apm.yml sits ABOVE the fake $HOME — if find_plugin_root
# didn't stop at $HOME, it would walk past it and misclassify this
# user/project-scope file as plugin scope, which would then FAIL on
# Check 0 (source_keys declared but sources.md absent) since sources.md
# doesn't exist at that ancestor apm.yml's location either.
local dir="$TMPDIR/anc"
mkdir -p "$dir"
cat > "$dir/apm.yml" <<EOF
name: outer-package
version: 0.1.0
type: skill
EOF
local fake_home="$dir/fakehome"
mkdir -p "$fake_home/.apm/agents"
cat > "$fake_home/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$fake_home/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: .git between the agent file and an ancestor apm.yml stops the walk first" {
local dir="$TMPDIR/repo2"
mkdir -p "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
mkdir -p "$dir/sub/.git" "$dir/sub/.apm/agents"
cat > "$dir/sub/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/sub/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
# ---------------------------------------------------------------------------
# Early exit: no sources.md, no source_keys → exit 0, no output
# ---------------------------------------------------------------------------
@test "clean pass: no sources.md and no source_keys → exit 0, no output" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@@ -158,22 +254,11 @@ EOF
# Check 0: source_keys present but sources.md absent → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: source_keys in CC file but sources.md absent" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_clean "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: source_keys in Copilot file but sources.md absent" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
make_copilot_with_source_keys "$root"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
@test "FAIL: source_keys in agent file but sources.md absent" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@@ -183,10 +268,9 @@ EOF
# ---------------------------------------------------------------------------
@test "FAIL: FILL IN: placeholder in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -194,23 +278,22 @@ EOF
- **URL:** FILL IN: add url
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FILL IN: inside backticks in sources.md does not fail" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root"
echo "Use \`FILL IN: value\` as example." >> "$root/sources.md"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}
@@ -218,62 +301,47 @@ EOF
# Check 2: source_keys slug missing from sources.md → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: source_keys slug in CC file not present as H2 in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root" "my-agent" "my-source"
make_copilot_clean "$root"
@test "FAIL: source_keys slug in agent file not present as H2 in sources.md" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root" "my-agent" "my-source"
make_sources_md "$root" "different-source" "(none)" "(none)"
run bash "$SCRIPT" "$root/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: source_keys slug in Copilot file not present as H2 in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
make_copilot_with_source_keys "$root" "my-agent" "my-source"
make_sources_md "$root" "different-source" "(none)" "(none)"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Check 4: Contributing file path doesn't exist → FAIL
# Check 3: Contributing file path doesn't exist → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Contributing file listed in sources.md does not exist" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
make_sources_md "$root" "my-source" "agents/nonexistent.md"
run bash "$SCRIPT" "$root/agents/my-agent.md"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root" "my-source" ".apm/agents/nonexistent.agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "pass: (none) in Contributing files is skipped" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root" "my-source" "(none — not used directly)"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}
# ---------------------------------------------------------------------------
# Check 6: Research doc field missing or placeholder → FAIL
# Check 5: Research doc field missing or placeholder → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Research doc field missing from sources.md entry" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -281,19 +349,18 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: Research doc field is FILL IN: placeholder" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -301,24 +368,24 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** FILL IN: path to research doc
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Check 5: Bidirectional — contributing file missing slug in source_keys → FAIL
# Check 4: Bidirectional — contributing file missing slug in source_keys → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Contributing file exists but does not list parent slug in source_keys" {
local root="$TMPDIR/plugin"
make_plugin "$root"
# CC file has source_keys: other-source (not my-source)
cat > "$root/agents/my-agent.md" <<EOF
local root="$TMPDIR/package"
make_package "$root"
# agent file has source_keys: other-source (not my-source)
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -328,8 +395,8 @@ source_keys:
You are a test agent.
EOF
make_copilot_clean "$root"
# sources.md says my-agent.md contributed to my-source, but my-agent.md doesn't list my-source
# sources.md says my-agent.agent.md contributed to my-source, but
# my-agent.agent.md doesn't list my-source
cat > "$root/sources.md" <<EOF
# Sources
@@ -337,7 +404,7 @@ EOF
- **URL:** https://example.com/other-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
@@ -345,39 +412,24 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** Another source.
- **Contributing files:** agents/my-agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Entry via Copilot file path
# ---------------------------------------------------------------------------
@test "accepts Copilot file path as entry point" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
make_sources_md "$root"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
assert_success
}
# ---------------------------------------------------------------------------
# Clean full pass
# ---------------------------------------------------------------------------
@test "clean full pass: all checks satisfied via CC file" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
@test "clean full pass: all checks satisfied" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}

View File

@@ -8,24 +8,27 @@ setup() {
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate.sh"
TMPDIR="$(mktemp -d)"
# Helper: create a plugin-scope pair in <dir> with given <name>
make_plugin_pair() {
local dir="$1"
# Helper: create an APM package root at <root> (apm.yml with a top-level
# type: line, marking it a real package manifest — not marketplace-only)
# plus a single vendor-neutral agent file at
# <root>/.apm/agents/<name>.agent.md. <extra_frontmatter>, if given, is
# inserted as additional raw frontmatter lines (used to inject fields
# under test).
make_apm_agent() {
local root="$1"
local name="$2"
mkdir -p "$dir"
echo '{}' > "$dir/plugin.json"
cat > "$dir/${name}.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
local extra_frontmatter="${3:-}"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$dir/${name}.agent.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
${extra_frontmatter}
---
You are a test agent. When invoked, do the thing.
@@ -38,17 +41,9 @@ teardown() {
}
# ---------------------------------------------------------------------------
# Passing cases
# Passing cases — project/user scope (unchanged)
# ---------------------------------------------------------------------------
@test "passes on a clean plugin-scope pair (CC file as input)" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
run bash "$SCRIPT" "$dir/my-agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "passes on a clean project-scope pair (CC file as input)" {
local root="$TMPDIR/project"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
@@ -73,29 +68,214 @@ EOF
refute_output --partial "FAIL"
}
@test "--help exits 0 and shows Usage:" {
run bash "$SCRIPT" --help
@test "user scope: \$HOME being a dotfiles .git repo does not shadow user scope" {
local fake_home="$TMPDIR/fakehome"
mkdir -p "$fake_home/.git" "$fake_home/.claude/agents" "$fake_home/.copilot/agents"
cat > "$fake_home/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$fake_home/.copilot/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$fake_home/.claude/agents/my-agent.md"
assert_success
assert_output --partial "Usage:"
refute_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Failing cases
# ---------------------------------------------------------------------------
@test "user scope: agent file directly in \$HOME (start dir IS exactly \$HOME, no walk-up) resolves to user scope" {
local fake_home="$TMPDIR/fakehome-direct"
mkdir -p "$fake_home" "$fake_home/.copilot/agents"
cat > "$fake_home/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
@test "fails when Copilot counterpart is missing" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
rm "$dir/my-agent.agent.md"
run bash "$SCRIPT" "$dir/my-agent.md"
You are a test agent. When invoked, do the thing.
EOF
cat > "$fake_home/.copilot/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$fake_home/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: a nested marker-less directory walked up into \$HOME resolves to project scope, not user scope (live repro of new-agent.sh's stray-directory case)" {
local fake_home="$TMPDIR/fakehome-nested"
local nested="$fake_home/scratch/testdir"
mkdir -p "$nested/.claude/agents" "$nested/.github/agents"
cat > "$nested/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$nested/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$nested/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: nested marker-less dir under \$HOME does NOT look for a counterpart under the shared \$HOME/.copilot or \$HOME/.github dirs" {
local fake_home="$TMPDIR/fakehome-nested2"
local nested="$fake_home/scratch/testdir"
mkdir -p "$nested/.claude/agents" "$fake_home/.copilot/agents"
cat > "$nested/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# Decoy counterpart at the *user*-scope location — if scope were
# misclassified as 'user' (the pre-fix bug), validate.sh would find this
# unrelated file and (wrongly) pass.
cat > "$fake_home/.copilot/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$nested/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "counterpart"
assert_output --partial "counterpart file not found"
}
@test "fails when CC-only field 'maxTurns' is in Copilot file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.agent.md" <<EOF
@test "project scope: filesystem-root fallback (no \$HOME in path, no markers found) resolves to project scope, not user scope" {
local unrelated_home="$TMPDIR/unrelated-home-never-reached"
local root="$TMPDIR/no-home-relation/deep/proj"
mkdir -p "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$unrelated_home" bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: <root> one level below a .git ancestor resolves scope to <root>, not to wherever .git was found (subdirectory of a larger git-tracked tree)" {
local repo="$TMPDIR/repo-with-subdir"
local root="$repo/subdir"
mkdir -p "$repo/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# new-agent.sh, invoked with <root> as its root argument, would place the
# counterpart at <root>/.github/agents — not at the repo root's
# .github/agents, even though .git lives at the repo root one level up.
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: a non-conventional path (agent file not directly under a literal 'agents' dir) falls back to the nearest .git boundary instead of two-segments-up arithmetic" {
local outer="$TMPDIR/outer-repo"
local pkg="$outer/pkgA"
mkdir -p "$pkg/.git" "$pkg/.github/agents" "$pkg/extra"
# Misplaced file: sits two path segments below $outer (pkgA/extra), which
# matches the conventional_root arithmetic by coincidence, but its
# immediate parent dir is "extra", not "agents" — conventional_shape is
# false, so the fix must fall back to the nearest .git boundary (pkgA),
# not trust $outer.
cat > "$pkg/extra/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# Counterpart at the nearest-.git root (pkgA), not at $outer — if the
# arithmetic were trusted here, validate.sh would look for a counterpart
# at $outer/.github/agents/my-agent.agent.md, which doesn't exist, and
# false-FAIL.
cat > "$pkg/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$pkg/extra/my-agent.md"
assert_success
refute_output --partial "counterpart file not found"
}
# ---------------------------------------------------------------------------
# Failing cases — project/user scope: CC/Copilot pair checks
# ---------------------------------------------------------------------------
@test "fails when a CC-only field ('maxTurns') is present in a project-scope Copilot file" {
local root="$TMPDIR/project"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -104,15 +284,338 @@ maxTurns: 10
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "CC-only field"
assert_output --partial "maxTurns"
}
@test "fails when a Copilot-only field ('target') is present in a project-scope CC file" {
local root="$TMPDIR/project"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
target: cli
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "Copilot-only field"
assert_output --partial "target"
}
@test "fails when the Copilot counterpart is missing at project scope" {
local root="$TMPDIR/project"
mkdir -p "$root/.git" "$root/.claude/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "counterpart file not found"
}
@test "--help exits 0 and shows Usage:" {
run bash "$SCRIPT" --help
assert_success
assert_output --partial "Usage:"
}
@test "fails when no arguments are given" {
run bash "$SCRIPT"
assert_failure
}
# ---------------------------------------------------------------------------
# Passing cases — plugin/APM scope
# ---------------------------------------------------------------------------
@test "passes on a clean plugin/APM-scope agent file (name/description only)" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "passes on a clean plugin/APM-scope agent file with optional model field" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "model: claude-opus-4"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Failing cases — plugin/APM scope: allowlist violations
# ---------------------------------------------------------------------------
@test "fails when 'tools' field is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "tools: Read Edit"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "tools"
}
@test "fails when a Claude-only field ('maxTurns') is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "maxTurns: 10"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "maxTurns"
}
@test "fails when plugin-silently-ignored field 'hooks' is in plugin-scope CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
@test "fails when a Copilot-only field ('target') is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "target: cli"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "target"
}
@test "fails when 'hooks' is present in a plugin/APM-scope agent file (outside allowlist)" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "hooks: {}"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "hooks"
}
# ---------------------------------------------------------------------------
# Failing cases — plugin/APM scope: structural checks
# ---------------------------------------------------------------------------
@test "fails when name is not kebab-case in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "kebab"
}
@test "fails when name does not match filename stem in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: wrong-name
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "does not match filename stem"
}
@test "fails when 'name' field is missing from a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
@test "fails when 'description' field is missing from a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
@test "fails when a template HTML comment is left in plugin/APM-scope frontmatter" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
<!-- model: sonnet
Optional. Omit to inherit the runtime default. -->
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "template HTML comments"
}
@test "fails when body contains unfilled FILL IN: placeholder in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
FILL IN: replace this with your system prompt.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
# ---------------------------------------------------------------------------
# Plugin/APM scope: no pair, no counterpart concept
# ---------------------------------------------------------------------------
@test "never raises a 'counterpart' FAIL on a clean plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "counterpart"
}
@test "never raises a 'counterpart' FAIL on a failing plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "tools: Read"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
refute_output --partial "counterpart"
}
# ---------------------------------------------------------------------------
# Scope-detection walk-up
# ---------------------------------------------------------------------------
@test "walk-up skips a type:-less apm.yml and finds a real package root further up" {
local root="$TMPDIR/case"
mkdir -p "$root/.apm/agents/nested/deeper"
cat > "$root/apm.yml" <<EOF
name: real-package
version: 1.0.0
type: skill
EOF
# Closer to the agent file than the real package root, but has no type:
# line — marketplace-only per monorepo-and-repo-shapes.md, must be skipped.
cat > "$root/.apm/agents/nested/apm.yml" <<EOF
name: not-a-package-manifest
version: 1.0.0
EOF
cat > "$root/.apm/agents/nested/deeper/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/nested/deeper/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "type:-less apm.yml is not treated as plugin scope — falls through to project scope" {
local root="$TMPDIR/proj-marketplace"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/apm.yml" <<EOF
name: marketplace-root
version: 1.0.0
marketplace:
packages: []
EOF
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "a bare plugin.json with no apm.yml is no longer plugin scope — falls through to project scope" {
local root="$TMPDIR/proj-legacy-plugin-json"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
echo '{}' > "$root/plugin.json"
# 'hooks' is plugin-silently-ignored only at (old) plugin scope; at
# project scope it's a legitimate CC field. If this directory were
# mis-detected as plugin scope (old plugin.json-based logic), this would
# FAIL with a plugin-silently-ignored-fields finding on 'hooks'.
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -124,95 +627,15 @@ hooks:
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
assert_output --partial "hooks"
}
@test "fails when CC file name is not kebab-case" {
local dir="$TMPDIR/agents"
mkdir -p "$dir"
echo '{}' > "$dir/plugin.json"
cat > "$dir/my-agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$dir/my-agent.agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
assert_output --partial "kebab"
}
@test "fails when 'name' field is missing from CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
---
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when 'description' field is missing from CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
---
name: my-agent
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when body contains unfilled FILL IN: placeholder" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
FILL IN: replace this with your system prompt.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when frontmatter name does not match filename stem" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.agent.md" <<EOF
---
name: wrong-name
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when no arguments are given" {
run bash "$SCRIPT"
assert_failure
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "hooks"
}

View File

@@ -4,7 +4,7 @@ Creates and improves agent definition files for Claude Code and GitHub Copilot C
## What it does
Scaffolds and fills in agent definition files (`.md` for Claude Code, `.agent.md` for Copilot CLI) at plugin, project, or user scope. Always generates both provider files in one pass. Also applies improvement signals — grill output, inline feedback, session context — to existing agent files. Bumps the plugin version in both manifests after every change (minor for new agents, patch for improvements).
Scaffolds and fills in agent definition files at plugin/APM, project, or user scope. Project and user scope always generate a Claude Code + Copilot CLI file pair (`.md` + `.agent.md`) in one pass. Plugin/APM scope generates a single vendor-neutral `.apm/agents/<name>.agent.md` file instead — no separate Claude Code / Copilot split, since `apm compile` has no per-target field integrator (see ADR-0016). Also applies improvement signals — grill output, inline feedback, session context — to existing agent files. Bumps the version after every change: the resolved package's `apm.yml` at plugin/APM scope (minor for new agents, patch for improvements); project/user scope has no manifest to bump.
## Before you start
@@ -21,7 +21,7 @@ Have ready: the agent's name (kebab-case), the root directory (plugin root, proj
bash scripts/new-agent.sh <agent-name> <root>
# Examples:
bash scripts/new-agent.sh code-reviewer plugins/kyberforge/
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
bash scripts/new-agent.sh deploy-assistant .
bash scripts/new-agent.sh security-reviewer ~
```
@@ -31,12 +31,13 @@ bash scripts/new-agent.sh security-reviewer ~
| File | Purpose |
|------|---------|
| `SKILL.md` | Skill instructions for agents |
| `scripts/new-agent.sh` | Scaffolds Claude Code and Copilot CLI agent files from templates |
| `references/deployment-modes.md` | Plugin vs project vs user scope: restrictions, scoped identifiers, path conventions |
| `scripts/new-agent.sh` | Scaffolds agent definition file(s) from templates — a single `.apm/agents/<name>.agent.md` at plugin/APM scope, or a Claude Code + Copilot CLI pair at project/user scope |
| `references/deployment-modes.md` | Plugin/APM vs project vs user scope: restrictions, scoped identifiers, path conventions |
| `references/scripts.md` | Conventions for new-agent.sh and any future scripts: contract, template variables, file placement, error messages |
| `references/sources.md` | Research provenance — sources that informed this skill |
| `assets/templates/claude-code.md` | Annotated Claude Code agent definition template |
| `assets/templates/copilot.agent.md` | Annotated Copilot CLI agent definition template |
| `assets/templates/claude-code.md` | Annotated Claude Code agent definition template (project/user scope) |
| `assets/templates/copilot.agent.md` | Annotated Copilot CLI agent definition template (project/user scope) |
| `assets/templates/apm-agent.md` | Annotated vendor-neutral APM agent definition template (plugin/APM scope) |
| `tests/new-agent.bats` | bats tests for `scripts/new-agent.sh` |
| `assets/README.md` | Directory meta-documentation for assets/ |
| `references/README.md` | Directory meta-documentation for references/ |

View File

@@ -3,14 +3,16 @@ name: agent-author
description: >
Use when the user wants to create a new agent definition file from scratch
("write an agent for X", "build a subagent that does Y", "create an agent
definition for Z"), or improve an existing one. Handles both Claude Code and
GitHub Copilot CLI agent formats, at plugin, project, and user scope — always
generating both provider files in one pass. Also use when the user provides
inline feedback about an agent's behavior and wants it applied, or when a
grill session has produced findings the user wants acted on — even if they
don't say "improve" explicitly. Do not use for read-only review — examine
agent files manually or run a grill session to generate improvement signals.
Do not use to author skills — use /skill-author instead.
definition for Z"), or improve an existing one. Handles agent definitions at
plugin/APM, project, and user scope. Project and user scope always generate
a Claude Code (`.md`) + Copilot CLI (`.agent.md`) file pair in one pass;
plugin/APM scope generates a single vendor-neutral `.apm/agents/<name>.agent.md`
file instead (no per-target Claude Code / Copilot split). Also use when the
user provides inline feedback about an agent's behavior and wants it applied,
or when a grill session has produced findings the user wants acted on — even
if they don't say "improve" explicitly. Do not use for read-only review —
examine agent files manually or run a grill session to generate improvement
signals. Do not use to author skills — use /skill-author instead.
allowed-tools: Bash Read Write Edit
metadata:
category: factory
@@ -26,18 +28,19 @@ metadata:
## Gotchas
- If the agent lives inside a plugin (its path contains a `plugin.json`), bump the plugin version after every change — in both `plugin.json` and `.claude-plugin/plugin.json` in the same edit pass. Convention: new agent → minor bump; improvement or fix → patch bump. Plugin consumers compare this version to detect updates; skipping it makes the change invisible.
- Plugin agents silently ignore `hooks`, `mcpServers`, and `permissionMode` — these fields have no effect and produce no warning. They only work in `.claude/agents/` or `~/.claude/agents/`.
- At plugin/APM scope, bump the resolved package's `apm.yml` `version` after every change — minor for a new agent, patch for a fix. Consumers compare this version to detect updates; skipping it hides the change.
- At plugin/APM scope, `tools` and all Claude-only fields (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `disallowedTools`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) are omitted entirely, not merely restricted (ADR-0016: `apm compile` copies frontmatter verbatim to both harnesses with no per-target integrator, so a harness-specific value is wrong on at least one). Only project/user scope supports these fields.
- An `apm.yml` with no top-level `type:` field is a marketplace-only manifest, not a package root — the walk-up skips it and keeps going.
- `AskUserQuestion`, `EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup`, and `WaitForMcpServers` are never available to any subagent regardless of the `tools` field. Exception: `ExitPlanMode` is available when the parent session runs in `permissionMode: plan`.
- Duplicate `name` values in the same scope: Claude Code silently discards one without warning. Always verify uniqueness before shipping.
- Plugin agents in subdirectories get scoped identifiers (`plugin:folder:name`). Keep agents flat in `agents/` to avoid this.
- Copilot CLI agent files **must** use the `.agent.md` extension. A plain `.md` file is not picked up by Copilot.
- Copilot has no `permissionMode`, `maxTurns`, `isolation`, or `memory` fields — do not include them in the Copilot file.
- Plugin agents in subdirectories get scoped identifiers (`plugin:folder:name`) — keep agents flat in `agents/` to avoid this. Applies to project/user-scope Claude Code agents only.
- Copilot CLI agent files **must** use the `.agent.md` extension — a plain `.md` file isn't picked up. The plugin/APM-scope single file also ends in `.agent.md` by convention, but it's vendor-neutral, not Copilot-only — it compiles to Claude Code too.
- Copilot has no `permissionMode`, `maxTurns`, `isolation`, or `memory` fields — do not include them in project/user-scope Copilot files.
- `model` resolution order for Claude Code: `CLAUDE_CODE_SUBAGENT_MODEL` env var → per-invocation parameter → frontmatter `model` → main session model. The frontmatter value is a low-priority default, not a guarantee.
## Route
If the destination is a plugin directory, read `references/deployment-modes.md`.
If the destination resolves to plugin/APM scope (scope detection in Step 1 finds a `type:`-bearing `apm.yml` at or above the root), read `references/deployment-modes.md`.
Determine which flow before touching the filesystem:
@@ -53,7 +56,7 @@ Signals: grill session output, inline user feedback, session context describing
Before touching the filesystem, confirm you have:
- [ ] Agent name (kebab-case, e.g. `code-reviewer`)
- [ ] Root directory (plugin root, project root, or `~` for user scope)
- [ ] Root directory (a path inside a package for plugin/APM scope, project root, or `~` for user scope)
- [ ] Agent purpose — one sentence describing the task this agent handles
- [ ] Trigger condition — when should the runtime delegate to this agent?
@@ -71,19 +74,25 @@ bash scripts/new-agent.sh <name> <root>
Examples:
```bash
bash scripts/new-agent.sh code-reviewer plugins/kyberforge/
bash scripts/new-agent.sh code-reviewer packages/my-package/ # plugin/APM scope if packages/my-package/apm.yml has a type: field
bash scripts/new-agent.sh deploy-assistant .
bash scripts/new-agent.sh security-reviewer ~
```
**Scope detection (script handles this automatically):**
- Root contains `plugin.json`, `.claude-plugin/plugin.json`, `.plugin/plugin.json`, or `.github/plugin/plugin.json` → plugin scope → creates `<root>/agents/<name>.md` + `<root>/agents/<name>.agent.md`
- Root is a project directory (no plugin marker) → creates `<root>/.claude/agents/<name>.md` + `<root>/.github/agents/<name>.agent.md`
- Root is `~` → creates `~/.claude/agents/<name>.md` + `~/.copilot/agents/<name>.agent.md`
**Scope detection (script handles this automatically).** The script walks up from `<root>` for a package boundary — same shape `agent-audit`'s `validate.sh` uses:
- Nearest ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) → **plugin/APM scope** → `<package-root>/.apm/agents/<name>.agent.md` (single vendor-neutral file). A `type:`-less `apm.yml` is marketplace-only — skipped, walk continues upward.
- No such `apm.yml`, `<root>` is a project directory → **project scope** (unchanged) → `<root>/.claude/agents/<name>.md` + `<root>/.github/agents/<name>.agent.md`
- `<root>` is exactly `~` (checked directly, no walk-up) → **user scope** (unchanged) → `~/.claude/agents/<name>.md` + `~/.copilot/agents/<name>.agent.md`
A bare `plugin.json` with no `apm.yml` no longer signals plugin scope — that path is fully replaced, not dual-mode; it falls through to project scope.
The script is file-by-file no-op — it skips any file that already exists.
### Step 2 — Fill in the Claude Code agent file (`<name>.md`)
### Step 2 — Fill in the agent file(s)
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Frontmatter carries ONLY `name`, `description`, optionally `model`, and optionally `source_keys` (provenance metadata, not a runtime field — see the template) — never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill in `name`, `description`, `model`, and the system prompt body per the guidance below; the rest of this step's field-by-field guidance (tools, maxTurns, effort, memory, isolation, disallowedTools, skills, color, initialPrompt, background) is project/user scope only. Skip Step 3 and go to Step 4.
**At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file.
Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Remove all template documentation comments from the YAML frontmatter after filling in required fields** — these are marked with `<!--` and `-->` and must be deleted before shipping.
@@ -95,9 +104,9 @@ Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Re
- Specific about the triggering condition and expertise domain
- Under 300 characters preferred
**`tools`** — restrict to what the agent actually needs. Omit to inherit all tools. Use `Agent(type1,type2)` to limit which subagent types this agent can spawn; omit `Agent` entirely to prevent spawning.
**`tools`** (project/user scope only — never at plugin/APM scope) — restrict to what the agent actually needs. Omit to inherit all tools. Use `Agent(type1,type2)` to limit which subagent types this agent can spawn; omit `Agent` entirely to prevent spawning.
**Optional fields worth considering:**
**Optional fields worth considering (project/user scope only — never at plugin/APM scope):**
- `model`: set when this agent needs a different capability tier (`haiku` for fast tasks, `opus` for deep reasoning)
- `maxTurns`: set a cap to prevent runaway agents on bounded tasks
- `effort`: set to `low` for single-lookup tasks, `high` or above for deep reasoning or multi-file analysis — overrides session effort level; omit to inherit
@@ -109,7 +118,7 @@ Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Re
- `initialPrompt`: auto-submitted as the first turn when this agent activates as the main session thread; only set when this agent is intended for main-thread activation
- `background`: set `true` to force background execution
**`source_keys`** — top-level list of research source slugs that informed this agent. Add only when research sources were used (i.e. entries with `` `extracted` `` status are in context from a prior `/research` session). Each slug must match an H2 heading in `sources.md` (plugin root). Omit entirely when no research was used.
**`source_keys`** — top-level list of research source slugs that informed this agent. Add only when research sources were used (i.e. entries with `` `extracted` `` status are in context from a prior `/research` session). Each slug must match an H2 heading in `sources.md` — see Step 4 for where that file lives (plugin/APM scope only). Omit entirely when no research was used.
```yaml
source_keys:
@@ -121,12 +130,14 @@ source_keys:
- Cover: inputs expected, process steps, output format, error handling
- One job per agent
### Step 3 — Fill in the Copilot agent file
### Step 3 — Fill in the Copilot agent file (project/user scope only)
Skip this step entirely at plugin/APM scope — there is no separate Copilot file there. The single `.apm/agents/<name>.agent.md` file from Step 2 already compiles to both Claude Code and Copilot CLI via `apm compile`.
**Two distinct Copilot agent formats** exist, with different paths and field sets. Choose one based on the deployment target:
**CLI format** (default — what the scaffold creates):
- Path: `.github/agents/<name>.agent.md` (project) or `<plugin>/agents/<name>.agent.md` (plugin)
- Path: `.github/agents/<name>.agent.md` (project) or `~/.copilot/agents/<name>.agent.md` (user)
- Extension: **must be `.agent.md`**
- Supported fields: `name` (required), `description` (required), `tools` (optional)
- `tools` uses Copilot aliases: `execute` (shell), `read`, `edit`, `search`, `agent`, `web`
@@ -145,14 +156,14 @@ source_keys:
The system prompt body should match the Claude Code version — the agent's task definition is the same across providers.
### Step 4 — Populate or delete `sources.md` (plugin scope only)
### Step 4 — Populate or delete `sources.md` (plugin/APM scope only)
Skip this step at project and user scope. The file lives at the plugin root, not inside `agents/` — `claude plugin validate --strict` auto-discovers every `.md` under `agents/` as an agent requiring frontmatter, and a flat provenance file would fail that check (see ADR-0010).
Skip at project/user scope. The file lives at the package root (alongside `apm.yml`), not inside `.apm/agents/` — otherwise tooling that scans that directory for agent definitions would treat it as an agent needing frontmatter (ADR-0010).
If a research `sources.md` is present in the conversation context:
1. Filter to entries with `` `extracted` `` status only.
2. For each entry, identify which agent files in the pair it contributed to.
3. Write `sources.md` at the plugin root using the format below. Paths in `Contributing files:` are relative to the plugin root.
2. For each entry, identify which agent file it contributed to.
3. Write `sources.md` at the package root using the format below. Paths in `Contributing files:` are relative to the package root.
```markdown
# Sources
@@ -162,11 +173,11 @@ If a research `sources.md` is present in the conversation context:
- **URL:** <source URL>
- **Research doc:** <path/to/research/sources.md relative to repo root>
- **Description:** <what this source covers>
- **Contributing files:** agents/<name>.md, agents/<name>.agent.md
- **Contributing files:** .apm/agents/<name>.agent.md
- **Status:** `extracted`
```
Each slug must match an H2 heading, and each slug must also appear in the `source_keys` list of every file listed under `Contributing files:`.
Each slug must match an H2 heading, and each slug must also appear in the `source_keys` list of the file listed under `Contributing files:`.
If no research sources are in context, delete `sources.md`.
@@ -174,15 +185,22 @@ If no research sources are in context, delete `sources.md`.
Run this checklist before invoking the audit:
**Claude Code file (`<name>.md`):**
**Plugin/APM scope — single file (`<name>.agent.md`):**
- [ ] `name` field present, kebab-case, unique in scope
- [ ] `description` field present and action-first
- [ ] If plugin scope: no `hooks`, `mcpServers`, or `permissionMode` (silently ignored at plugin scope)
- [ ] Frontmatter contains ONLY `name`, `description`, and optionally `model` (plus `source_keys` if research-sourced) — no `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `disallowedTools`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, or `mcpServers`
- [ ] System prompt body present and non-empty
- [ ] No `FILL IN:` placeholders remain
- [ ] No `<!-- -->` template comments remain in frontmatter
**Copilot CLI file (`<name>.agent.md`):**
**Project/user scope — Claude Code file (`<name>.md`):**
- [ ] `name` field present, kebab-case, unique in scope
- [ ] `description` field present and action-first
- [ ] System prompt body present and non-empty
- [ ] No `FILL IN:` placeholders remain
- [ ] No `<!-- -->` template comments remain in frontmatter
**Project/user scope — Copilot CLI file (`<name>.agent.md`):**
- [ ] File extension is `.agent.md` (not `.md`)
- [ ] `name` field matches the filename stem (e.g. `name: my-agent` in `my-agent.agent.md`)
- [ ] `description` field present
@@ -191,9 +209,9 @@ Run this checklist before invoking the audit:
- [ ] Body does not exceed 30,000 characters
- [ ] No `<!-- -->` template comments remain in frontmatter
If the destination is inside a plugin directory, apply a **minor bump** to the `version` field in both `plugin.json` and `.claude-plugin/plugin.json` at the plugin root in the same edit pass (e.g. `1.0.4` → `1.1.0`).
At plugin/APM scope, apply a **minor bump** to the resolved package's `apm.yml` `version` (single manifest, e.g. `1.0.4` → `1.1.0`).
Invoke the `kyberforge:agent-audit` skill directly on the created files to confirm the pair is valid before closing.
Invoke `kyberforge:agent-audit` on the created file(s) before closing — validates the pair at project/user scope, the single file at plugin/APM scope.
**Commit verification.** Capture `git log --oneline -1` before Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the new agent files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the agent as done once the hash has actually changed.
@@ -209,7 +227,7 @@ Verify `kyberforge:agent-audit` is available — it ships with the kyberforge pl
Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
**Partial state** — if one provider file exists but the other does not, scaffold the missing file first (run `bash scripts/new-agent.sh <name> <root>` — the file-by-file no-op means only the missing file is created), then continue with the improve flow on both files.
**Partial state (project/user scope only)** — if one provider file exists but not the other, scaffold the missing one (`bash scripts/new-agent.sh <name> <root>`, file-by-file no-op) then continue. Doesn't apply at plugin/APM scope — single file, no partial-pair state.
### Step 2 — Gather and group signals
@@ -230,7 +248,7 @@ Before editing, state which root causes were identified, what evidence supports
### Step 4 — Apply changes
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced.
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced. For a plugin/APM-scope single file, verify no field beyond `name`, `description`, `model`, and `source_keys` is introduced.
If the edit adds or removes research-sourced content, update `source_keys` in the edited file(s) and the corresponding entry in `sources.md` per Create flow's Step 4.
@@ -238,8 +256,8 @@ If the edit adds or removes research-sourced content, update `source_keys` in th
Re-run the validation checklist from the create flow's Step 5 on any edited file.
If the agent lives inside a plugin directory, apply a **patch bump** to the `version` field in both `plugin.json` and `.claude-plugin/plugin.json` at the plugin root in the same edit pass (e.g. `1.0.4` → `1.0.5`).
At plugin/APM scope, apply a **patch bump** to the resolved package's `apm.yml` `version` (e.g. `1.0.4` → `1.0.5`).
Invoke the `kyberforge:agent-audit` skill directly on the edited files to confirm no regressions before closing.
Invoke `kyberforge:agent-audit` on the edited file(s) to confirm no regressions — the pair at project/user scope, the single file at plugin/APM scope.
**Commit verification.** Capture `git log --oneline -1` at the start of Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the changed files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the improvement as done once the hash has actually changed.

View File

@@ -4,5 +4,6 @@
Annotated agent definition templates copied by `scripts/new-agent.sh` when scaffolding a new agent.
- **`claude-code.md`** — Claude Code agent definition template. Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders. Notes which fields are silently ignored for plugin agents.
- **`copilot.agent.md`** — Copilot CLI agent definition template (CLI format). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
- **`copilot.agent.md`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Only `name`, `description`, optional `model`, and optional `source_keys` (provenance metadata, not a runtime field) in frontmatter — no `tools` and no Claude-only fields, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator (ADR-0016).

View File

@@ -0,0 +1,54 @@
---
<!-- Vendor-neutral APM agent definition (plugin/APM scope).
Path: <package-root>/.apm/agents/<name>.agent.md — one file, no counterpart.
`apm compile` copies this frontmatter verbatim to BOTH the Claude Code and
Copilot CLI targets — there is no per-target field integrator. Claude's
`tools:` (space-separated string) and Copilot's `tools:` (alias list) are
incompatible vocabularies, and Claude-only fields (isolation, maxTurns,
effort, memory, permissionMode) have no Copilot equivalent. A value correct
for one harness is guaranteed wrong on the other, so this scope carries
ONLY the fields below — full stop (see ADR-0016). `source_keys` is
provenance metadata, not a runtime field, and is exempt from that rule.
Do NOT add: tools, isolation, maxTurns, effort, memory, permissionMode,
disallowedTools, skills, color, initialPrompt, background, hooks, or
mcpServers. Omitting `tools` means inherit-all-tools on both harnesses,
which is never wrong.
Fill in all FILL IN: placeholders. Delete template comments before shipping. -->
name: AGENT_NAME
<!-- Required. Lowercase letters and hyphens only. Must be unique within the scope. -->
description: FILL IN: Action-first description of what this agent does and when to invoke it.
<!-- Required. The primary signal for autonomous delegation.
Start with a verb: "Reviews...", "Analyzes...", "Generates..."
Be specific about the triggering condition and expertise domain.
Example: "Reviews pull request diffs for security issues. Use proactively after code changes." -->
<!-- model: sonnet
Optional. Aliases: sonnet, opus, haiku, fable. Or full model ID.
Omit to inherit the runtime default on whichever harness compiles this file. -->
<!-- source_keys:
- slug-name
Development-only. Add when research sources informed this agent (slugs must match
sources.md at the package root — same directory as apm.yml).
Omit when no research was used. Not a runtime field — silently ignored by both harnesses. -->
---
FILL IN: System prompt body. Write as a direct role instruction.
You are a FILL IN: role description. When invoked, FILL IN: primary action.
## Inputs
FILL IN: What inputs does this agent expect? (files, context, parameters)
## Process
FILL IN: Steps the agent takes. Be specific about ordering if it matters.
## Output
FILL IN: What does the agent produce? Format, location, structure.

View File

@@ -18,22 +18,15 @@ Agent definitions deploy at three scopes and behave differently at each. The sco
|---|---|---|---|
| User | `~/.claude/agents/` | `~/.copilot/agents/` | All sessions for this user |
| Project | `.claude/agents/` | `.github/agents/` or `.copilot/agents/` | This repo only |
| Plugin | `<plugin>/agents/` | `<plugin>/agents/` | Sessions with the plugin installed |
| Plugin/APM | `<package-root>/.apm/agents/<name>.agent.md` — single vendor-neutral file, `apm compile` emits it to both targets | *(same file)* | Sessions with the plugin/package installed |
When the same agent `name` appears at multiple scopes, **user scope wins over project scope wins over plugin scope** in Claude Code. In Copilot CLI, repo-level agents override enterprise and org-level; home-directory (user) agents override repo-level on name collision.
## Plugin scope restrictions
Plugin agents (agents shipped inside a plugin's `agents/` directory) have fewer capabilities than project or user agents:
Plugin/APM agents (`.apm/agents/<name>.agent.md`) carry only `name`, `description`, optionally `model`, and optionally `source_keys` (provenance metadata, not a runtime field — silently ignored by both harnesses) in frontmatter — full stop (see ADR-0016). `apm compile` copies this frontmatter verbatim to both the Claude Code and Copilot CLI compile targets with no per-target integrator: Claude's `tools:` (space-separated string) and Copilot's `tools:` (alias list) are incompatible vocabularies, and Claude-only fields have no Copilot equivalent, so any harness-specific value is guaranteed wrong on at least one target.
**Claude Code — silently ignored for plugin agents:**
- `hooks` — per-agent event handlers; copy the agent to `.claude/agents/` to use these
- `mcpServers` — inline MCP server definitions; not available in plugin context
- `permissionMode` — silently ignored; the parent session's permission mode applies
**Copilot CLI — unsupported or ignored:**
- `mcp-servers` — cloud agents only; ignored in VS Code and most IDEs
- `metadata` — key-value annotations; ignored in VS Code
This makes the old "silently ignored at plugin scope" framing moot. It's not that `hooks`, `mcpServers`, `permissionMode`, `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `disallowedTools`, `skills`, `color`, `initialPrompt`, or `background` are merely ignored at this scope — they are never written to the file at all. Copy the agent to `.claude/agents/` (project scope) or `~/.claude/agents/` (user scope) to use any of them.
## Scoped identifiers (Claude Code plugin agents only)
@@ -59,6 +52,6 @@ Agents at project or user scope are read directly from disk; cache isolation doe
|---|---|---|
| User | `~/.copilot/agents/<name>.agent.md` | Home directory |
| Project | `.github/agents/<name>.agent.md` | Standard; also `.copilot/agents/` |
| Plugin | `<plugin>/agents/<name>.agent.md` | Inside plugin root |
| Plugin/APM | `<package-root>/.apm/agents/<name>.agent.md` | Not a Copilot-only file — this is the single vendor-neutral source `apm compile` reads for the Copilot CLI target |
The `.agent.md` extension is **mandatory** — Copilot CLI does not pick up plain `.md` files in the `agents/` directory.
The `.agent.md` extension is **mandatory** for real Copilot CLI files (project/user scope) — Copilot CLI does not pick up plain `.md` files in the `agents/` directory. The plugin/APM source file also uses `.agent.md` by convention, since it compiles to Copilot CLI too, but it is not itself a Copilot file.

View File

@@ -24,7 +24,7 @@ Do not add additional substitution tokens unless you update both the template fi
## File placement
The script creates files at paths determined by scope detection (plugin / project / user). Scope is detected from the presence of `plugin.json`, `.claude-plugin/plugin.json`, `.plugin/plugin.json`, or `.github/plugin/plugin.json` in the root directory. If scope detection logic changes, update the `new-agent.sh` usage comment and `SKILL.md` Step 1 scope detection table in the same pass.
The script creates files at paths determined by scope detection (plugin/APM / project / user). Scope is resolved by walking up from the root directory: a `type:`-bearing `apm.yml` at or above the root marks the package root (plugin/APM scope, single file); an `apm.yml` without a `type:` field is a marketplace-only manifest and is skipped, the walk continues upward. If no such `apm.yml` is found, the root resolving to exactly `$HOME` is user scope; anything else is project scope. If scope detection logic changes, update the `new-agent.sh` usage comment and `SKILL.md` Step 1 scope detection description in the same pass.
## Error messages

View File

@@ -2,12 +2,12 @@
## new-agent.sh
Scaffolds agent definition files for both Claude Code and GitHub Copilot CLI from a single root directory input.
Scaffolds agent definition file(s) from a single root directory input.
```
Usage: new-agent.sh <agent-name> <root>
```
Detects scope from the root: `plugin.json` present → plugin scope (both files in `<root>/agents/`); `~` → user scope (`~/.claude/agents/` + `~/.copilot/agents/`); otherwise project scope (`.claude/agents/` + `.github/agents/`). Each file is a no-op if it already exists. See `--help` for full usage.
Resolves scope by walking up from `<root>`: a `type:`-bearing `apm.yml` found at or above `<root>` → plugin/APM scope (single file at `<package-root>/.apm/agents/<name>.agent.md`; an `apm.yml` without `type:` is a marketplace-only manifest and is skipped); `<root>` exactly `~` → user scope (`~/.claude/agents/` + `~/.copilot/agents/`); otherwise project scope (`<root>/.claude/agents/` + `<root>/.github/agents/`). Each file is a no-op if it already exists. See `--help` for full usage.
Tests: `tests/new-agent.bats` (requires `bats-support` and `bats-assert`).

View File

@@ -9,19 +9,29 @@ usage() {
cat <<EOF
Usage: new-agent.sh <agent-name> <root>
Scaffold agent definition files for Claude Code and GitHub Copilot CLI.
Scaffold agent definition file(s) for Claude Code, GitHub Copilot CLI, and/or
vendor-neutral APM packages.
Arguments:
agent-name Kebab-case agent identifier (e.g. code-reviewer, deploy-assistant).
root Root directory — determines scope:
plugin scope : root contains plugin.json
→ creates <root>/agents/<name>.md
→ creates <root>/agents/<name>.agent.md
→ creates <root>/sources.md (if absent)
project scope : root is a project directory (no plugin.json)
root Starting directory — scope is resolved by walking up from here:
plugin/APM scope : nearest ancestor (at/above root) whose apm.yml
has a top-level type: field (instructions,
skill, hybrid, or prompts) — an apm.yml
without type: is a marketplace-only manifest
and is skipped, the walk continues upward
→ creates <package-root>/.apm/agents/<name>.agent.md
(single vendor-neutral file — no tools,
isolation, maxTurns, effort, memory, or
permissionMode; apm compile has no per-target
field integrator, see ADR-0016)
→ creates <package-root>/sources.md (if absent)
project scope : no type:-bearing apm.yml found; root is a
project directory
→ creates <root>/.claude/agents/<name>.md
→ creates <root>/.github/agents/<name>.agent.md
user scope : root is ~ (home directory)
user scope : root is exactly ~ (home directory; checked
directly, no walk-up)
→ creates ~/.claude/agents/<name>.md
→ creates ~/.copilot/agents/<name>.agent.md
@@ -71,22 +81,105 @@ if [[ ! -d "$ROOT" ]]; then
echo "Error: root directory '$ROOT' does not exist." >&2
exit 1
fi
ROOT="$(cd "$ROOT" && pwd)"
# Detect scope
if [[ -f "$ROOT/plugin.json" || -f "$ROOT/.claude-plugin/plugin.json" || -f "$ROOT/.plugin/plugin.json" || -f "$ROOT/.github/plugin/plugin.json" ]]; then
SCOPE="plugin"
elif [[ "$ROOT" == "$HOME" ]]; then
SCOPE="user"
else
SCOPE="project"
fi
# True if apm_yml's top-level `type:` line names one of the four APM package
# types (instructions/skill/hybrid/prompts) — mirrors validate.sh's
# APM_TYPE_RE: an optional quote around the value must be closed by the
# *same* quote character (a mismatched or unterminated quote is rejected,
# not silently stripped), and the value must be followed by whitespace or
# end-of-line so `prompts-only` doesn't false-match on the `prompts` prefix.
# `|| [[ -n "$line" ]]` in the read condition also processes a final line
# that lacks a trailing newline, which `read` alone would otherwise skip.
is_apm_package_manifest() {
local apm_yml="$1" line
while IFS= read -r line || [[ -n "$line" ]]; do
if [[ "$line" =~ ^type:[[:space:]]*(instructions|skill|hybrid|prompts)([[:space:]]|$) ]]; then
return 0
fi

Scaffolds into the wrong directory for existing plugins. find_package_root() dropped the direct plugin.json check at $ROOT, so scaffolding a new agent inside an existing plugin.json-only plugin creates files in the wrong place.

Reproduced: new-agent.sh <name> plugins/kyberforge/ creates plugins/kyberforge/.claude/agents/<name>.md + .github/agents/<name>.agent.md instead of plugins/kyberforge/agents/<name>.md/.agent.md, diverging from where every other agent in that plugin actually lives.

**Scaffolds into the wrong directory for existing plugins.** `find_package_root()` dropped the direct `plugin.json` check at `$ROOT`, so scaffolding a new agent inside an existing plugin.json-only plugin creates files in the wrong place. Reproduced: `new-agent.sh <name> plugins/kyberforge/` creates `plugins/kyberforge/.claude/agents/<name>.md` + `.github/agents/<name>.agent.md` instead of `plugins/kyberforge/agents/<name>.md`/`.agent.md`, diverging from where every other agent in that plugin actually lives.
if [[ "$line" =~ ^type:[[:space:]]*([\"\'])(instructions|skill|hybrid|prompts)([\"\'])([[:space:]]|$) ]] \
&& [[ "${BASH_REMATCH[1]}" == "${BASH_REMATCH[3]}" ]]; then
return 0
fi
done < "$apm_yml"
return 1
}
# --- Walk-up package-root detection ---
#
# Mirrors agent-audit's validate.sh scope walk-up, with apm.yml + type: swapped
# in for the old plugin.json marker. Starting at ROOT, walk upward:
# - an apm.yml with a top-level `type:` field marks an APM package root
# (plugin/APM scope) — stop and return it.
# - an apm.yml with no `type:` field is a marketplace-only manifest — skip
# it, keep walking up.
# - user scope is checked directly at $HOME, no walk-up (see usage text
# above): ROOT itself being $HOME resolves to user scope, even if $HOME
# is itself a .git-tracked dotfiles directory (checked before the .git
# test below, so a dotfiles repo at $HOME can't shadow user scope).
# Walking *up into* $HOME from a nested directory with no apm.yml/.git
# of its own does NOT promote to user scope — it resolves to project
# scope instead, same as any other unmatched boundary, so a stray
# directory under $HOME can't be silently redirected into the shared
# global ~/.claude or ~/.copilot agent directories.
# - a .git file or directory marks the project-scope boundary (a worktree's
# .git is a file, not a directory) — stop.
# - filesystem root reached with neither found — project scope, same as
# any other unmatched boundary.
find_package_root() {
local root="$1" current="$1"
while true; do
if [[ -f "$current/apm.yml" ]] && is_apm_package_manifest "$current/apm.yml"; then
echo "plugin $current"
return
fi
if [[ "$current" == "$HOME" ]]; then
if [[ "$current" == "$root" ]]; then
echo "user $current"
return
fi
echo "project $current"
return
fi
if [[ -e "$current/.git" ]]; then
echo "project $current"
return
fi
local parent
parent="$(dirname "$current")"
if [[ "$parent" == "$current" ]]; then
echo "project $current"
return
fi
current="$parent"
done
}
# `read` consumes a single line, so kind and path are emitted on one
# space-separated line rather than two `echo`s — kind first (never contains
# spaces), path last (absorbs any spaces in the path safely).
WALK_RESULT="$(find_package_root "$ROOT")"
read -r WALK_KIND WALK_ROOT <<< "$WALK_RESULT"
PACKAGE_ROOT=""
case "$WALK_KIND" in
plugin)
SCOPE="plugin"
PACKAGE_ROOT="$WALK_ROOT"
;;
user)
SCOPE="user"
;;
project)
SCOPE="project"
;;
esac
# Determine file destinations
case "$SCOPE" in
plugin)
CC_DIR="$ROOT/agents"
CP_DIR="$ROOT/agents"
SOURCES_DIR="$ROOT"
APM_DIR="$PACKAGE_ROOT/.apm/agents"
SOURCES_DIR="$PACKAGE_ROOT"
;;
project)
CC_DIR="$ROOT/.claude/agents"
@@ -100,33 +193,47 @@ case "$SCOPE" in
;;
esac
CC_FILE="$CC_DIR/$AGENT_NAME.md"
CP_FILE="$CP_DIR/$AGENT_NAME.agent.md"
# Create directories
mkdir -p "$CC_DIR"
mkdir -p "$CP_DIR"
# Copy Claude Code template (no-op if exists)
created_any=false
if [[ -f "$CC_FILE" ]]; then
echo "Skipping '$CC_FILE' — already exists." >&2
if [[ "$SCOPE" == "plugin" ]]; then
APM_FILE="$APM_DIR/$AGENT_NAME.agent.md"
mkdir -p "$APM_DIR"
if [[ -f "$APM_FILE" ]]; then
echo "Skipping '$APM_FILE' — already exists." >&2
else
sed "s/AGENT_NAME/$AGENT_NAME/g" "$TEMPLATES_DIR/apm-agent.md" > "$APM_FILE"
echo "Created: $APM_FILE" >&2
created_any=true
fi
else
sed "s/AGENT_NAME/$AGENT_NAME/g" "$TEMPLATES_DIR/claude-code.md" > "$CC_FILE"
echo "Created: $CC_FILE" >&2
created_any=true
CC_FILE="$CC_DIR/$AGENT_NAME.md"
CP_FILE="$CP_DIR/$AGENT_NAME.agent.md"
mkdir -p "$CC_DIR"
mkdir -p "$CP_DIR"
# Copy Claude Code template (no-op if exists)
if [[ -f "$CC_FILE" ]]; then
echo "Skipping '$CC_FILE' — already exists." >&2
else
sed "s/AGENT_NAME/$AGENT_NAME/g" "$TEMPLATES_DIR/claude-code.md" > "$CC_FILE"
echo "Created: $CC_FILE" >&2
created_any=true
fi
# Copy Copilot template (no-op if exists)
if [[ -f "$CP_FILE" ]]; then
echo "Skipping '$CP_FILE' — already exists." >&2
else
sed "s/AGENT_NAME/$AGENT_NAME/g" "$TEMPLATES_DIR/copilot.agent.md" > "$CP_FILE"
echo "Created: $CP_FILE" >&2
created_any=true
fi
fi
# Copy Copilot template (no-op if exists)
if [[ -f "$CP_FILE" ]]; then
echo "Skipping '$CP_FILE' — already exists." >&2
else
sed "s/AGENT_NAME/$AGENT_NAME/g" "$TEMPLATES_DIR/copilot.agent.md" > "$CP_FILE"
echo "Created: $CP_FILE" >&2
created_any=true
fi
# Create sources.md at plugin scope (no-op if exists)
# Create sources.md at plugin/APM package root (no-op if exists)
if [[ -n "$SOURCES_DIR" ]]; then
SOURCES_FILE="$SOURCES_DIR/sources.md"
if [[ -f "$SOURCES_FILE" ]]; then
@@ -135,7 +242,7 @@ if [[ -n "$SOURCES_DIR" ]]; then
cat > "$SOURCES_FILE" <<'SOURCES'
# Sources
<!-- List research sources that informed agents in this directory.
<!-- List research sources that informed agents in this package.
Follow the format below. Only include entries with `extracted` status.
Delete this file if no research sources informed these agents. -->
@@ -143,7 +250,7 @@ if [[ -n "$SOURCES_DIR" ]]; then
- **URL:** <url>
- **Research doc:** <relative-path-to-upstream-research-sources-file>
- **Description:** <what this source covers>
- **Contributing files:** agents/<name>.md, agents/<name>.agent.md
- **Contributing files:** .apm/agents/<name>.agent.md
- **Status:** `extracted` -->
SOURCES
echo "Created: $SOURCES_FILE" >&2
@@ -158,12 +265,13 @@ else
echo "Scope: $SCOPE" >&2
echo "" >&2
echo "Next steps:" >&2
echo " 1. Fill in $CC_FILE — replace all FILL IN: placeholders" >&2
echo " 2. Fill in $CP_FILE — replace all FILL IN: placeholders" >&2
if [[ -n "$SOURCES_DIR" ]]; then
echo " 3. Populate $SOURCES_DIR/sources.md with research sources, or delete it" >&2
echo " 4. Validate: check required fields (name, description, system prompt) in both files" >&2
if [[ "$SCOPE" == "plugin" ]]; then
echo " 1. Fill in $APM_FILE — replace all FILL IN: placeholders (name, description, model, body only)" >&2
echo " 2. Populate $SOURCES_DIR/sources.md with research sources, or delete it" >&2
echo " 3. Validate: check required fields (name, description, system prompt) in the file" >&2
else
echo " 1. Fill in $CC_FILE — replace all FILL IN: placeholders" >&2
echo " 2. Fill in $CP_FILE — replace all FILL IN: placeholders" >&2
echo " 3. Validate: check required fields (name, description, system prompt) in both files" >&2
fi
fi

View File

@@ -24,62 +24,152 @@ teardown() {
}
# ---------------------------------------------------------------------------
# Plugin scope (plugin.json present at root)
# Plugin/APM scope (type:-bearing apm.yml at root)
# ---------------------------------------------------------------------------
@test "plugin scope: creates both agent files in agents/" {
touch "$ROOT/plugin.json"
@test "plugin/APM scope: creates single agent file in .apm/agents/" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -f "$ROOT/agents/my-agent.md" ]
assert [ -f "$ROOT/agents/my-agent.agent.md" ]
assert [ -f "$ROOT/.apm/agents/my-agent.agent.md" ]
}
@test "plugin scope: creates agents/ directory if missing" {
touch "$ROOT/plugin.json"
@test "plugin/APM scope: does not create the old dual-file pair" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -d "$ROOT/agents" ]
assert [ ! -f "$ROOT/agents/my-agent.md" ]
assert [ ! -f "$ROOT/agents/my-agent.agent.md" ]
}
@test "plugin scope: creates sources.md at plugin root" {
touch "$ROOT/plugin.json"
@test "plugin/APM scope: creates .apm/agents/ directory if missing" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -d "$ROOT/.apm/agents" ]
}
@test "plugin/APM scope: creates sources.md at package root" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -f "$ROOT/sources.md" ]
}
@test "plugin scope: no-op if claude code file already exists" {
touch "$ROOT/plugin.json"
mkdir -p "$ROOT/agents"
echo "existing" > "$ROOT/agents/my-agent.md"
@test "plugin/APM scope: no-op if agent file already exists" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
mkdir -p "$ROOT/.apm/agents"
echo "existing" > "$ROOT/.apm/agents/my-agent.agent.md"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
run grep "existing" "$ROOT/agents/my-agent.md"
run grep "existing" "$ROOT/.apm/agents/my-agent.agent.md"
assert_success
}
@test "plugin scope: no-op if copilot file already exists" {
touch "$ROOT/plugin.json"
mkdir -p "$ROOT/agents"
echo "existing" > "$ROOT/agents/my-agent.agent.md"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
run grep "existing" "$ROOT/agents/my-agent.agent.md"
@test "plugin/APM scope: frontmatter has no tools/isolation/maxTurns/effort/memory/permissionMode keys" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
bash "$SCRIPT" my-agent "$ROOT"
file="$ROOT/.apm/agents/my-agent.agent.md"
fm="$(sed -n '/^---$/,/^---$/p' "$file")"
# Column-0 key lines only — comment bodies in the template are indented,
# so this anchor naturally excludes commented-out example fields.
run grep -E '^(tools|isolation|maxTurns|effort|memory|permissionMode|disallowedTools|skills|color|initialPrompt|background|hooks|mcpServers):' <<< "$fm"
assert_failure
}
@test "plugin/APM scope: frontmatter carries only name, description, model, source_keys fields" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
bash "$SCRIPT" my-agent "$ROOT"
file="$ROOT/.apm/agents/my-agent.agent.md"
fm="$(sed -n '/^---$/,/^---$/p' "$file")"
keys="$(grep -oE '^[a-zA-Z][a-zA-Z0-9_-]*:' <<< "$fm" | sed 's/:$//' | sort -u)"
for key in $keys; do
if [[ "$key" != "name" && "$key" != "description" && "$key" != "model" && "$key" != "source_keys" ]]; then
fail "unexpected frontmatter key: $key"
fi
done
}
@test "plugin/APM scope: sources.md contributing-files template mentions the single-file path" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
bash "$SCRIPT" my-agent "$ROOT"
run grep ".apm/agents/<name>.agent.md" "$ROOT/sources.md"
assert_success
}
@test "plugin scope: still creates missing file when one already exists" {
touch "$ROOT/plugin.json"
mkdir -p "$ROOT/agents"
echo "existing" > "$ROOT/agents/my-agent.md"
@test "plugin/APM scope: template AGENT_NAME substituted" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
bash "$SCRIPT" my-agent "$ROOT"
run grep "my-agent" "$ROOT/.apm/agents/my-agent.agent.md"
assert_success
}
@test "plugin/APM scope: walk-up finds apm.yml at an ancestor directory, not just root arg" {
printf 'name: my-package\ntype: skill\n' > "$ROOT/apm.yml"
mkdir -p "$ROOT/nested/subdir"
run bash "$SCRIPT" my-agent "$ROOT/nested/subdir"
assert_success
assert [ -f "$ROOT/.apm/agents/my-agent.agent.md" ]
}
@test "plugin/APM scope: apm.yml without type: is skipped (marketplace-only manifest)" {
printf 'name: my-marketplace\n' > "$ROOT/apm.yml"
mkdir -p "$ROOT/.git"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -f "$ROOT/agents/my-agent.agent.md" ]
assert [ ! -f "$ROOT/.apm/agents/my-agent.agent.md" ]
assert [ -f "$ROOT/.claude/agents/my-agent.md" ]
}
@test "plugin/APM scope: type:-less apm.yml at leaf falls through to a type:-bearing apm.yml higher up" {
printf 'name: outer-package\ntype: skill\n' > "$ROOT/apm.yml"
mkdir -p "$ROOT/inner"
printf 'name: inner-marketplace\n' > "$ROOT/inner/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT/inner"
assert_success
assert [ -f "$ROOT/.apm/agents/my-agent.agent.md" ]
}
@test "plugin/APM scope: matched-quote type value ('skill') is recognized" {
printf 'name: my-package\ntype: "skill"\n' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -f "$ROOT/.apm/agents/my-agent.agent.md" ]
}
@test "plugin/APM scope: mismatched-quote type value is rejected, falls through to project scope" {
mkdir -p "$ROOT/.git"
printf "name: my-package\ntype: \"skill'\n" > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ ! -f "$ROOT/.apm/agents/my-agent.agent.md" ]
assert [ -f "$ROOT/.claude/agents/my-agent.md" ]
}
@test "plugin/APM scope: type: line is recognized even without a trailing newline on the file" {
printf 'name: my-package\ntype: skill' > "$ROOT/apm.yml"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ -f "$ROOT/.apm/agents/my-agent.agent.md" ]
}
# ---------------------------------------------------------------------------
# Non-plugin scope (no plugin.json)
# Old plugin.json marker is no longer recognized (full switch, no dual-mode)
# ---------------------------------------------------------------------------
@test "bare plugin.json (no apm.yml) is no longer detected as plugin scope — falls through to project scope" {
touch "$ROOT/plugin.json"
run bash "$SCRIPT" my-agent "$ROOT"
assert_success
assert [ ! -f "$ROOT/agents/my-agent.md" ]
assert [ ! -f "$ROOT/agents/my-agent.agent.md" ]
assert [ ! -f "$ROOT/.apm/agents/my-agent.agent.md" ]
assert [ -f "$ROOT/.claude/agents/my-agent.md" ]
assert [ -f "$ROOT/.github/agents/my-agent.agent.md" ]
}
# ---------------------------------------------------------------------------
# Non-plugin (project) scope
# ---------------------------------------------------------------------------
@test "non-plugin scope: creates claude code file in .claude/agents/" {
@@ -122,6 +212,51 @@ teardown() {
assert_success
}
@test "project scope: detected via .git present above root, output still relative to root arg" {
mkdir -p "$ROOT/repo/.git"
mkdir -p "$ROOT/repo/pkg"
run bash "$SCRIPT" my-agent "$ROOT/repo/pkg"
assert_success
assert [ -f "$ROOT/repo/pkg/.claude/agents/my-agent.md" ]
assert [ -f "$ROOT/repo/pkg/.github/agents/my-agent.agent.md" ]
}
# ---------------------------------------------------------------------------
# User scope
# ---------------------------------------------------------------------------
@test "user scope: root exactly \$HOME creates files under ~/.claude and ~/.copilot" {
FAKE_HOME="$(mktemp -d)"
run env HOME="$FAKE_HOME" bash "$SCRIPT" my-agent "~"
assert_success
assert [ -f "$FAKE_HOME/.claude/agents/my-agent.md" ]
assert [ -f "$FAKE_HOME/.copilot/agents/my-agent.agent.md" ]
rm -rf "$FAKE_HOME"
}
@test "user scope: \$HOME being a dotfiles .git repo does not shadow user scope" {
FAKE_HOME="$(mktemp -d)"
mkdir "$FAKE_HOME/.git"
run env HOME="$FAKE_HOME" bash "$SCRIPT" my-agent "~"
assert_success
assert [ -f "$FAKE_HOME/.claude/agents/my-agent.md" ]
assert [ -f "$FAKE_HOME/.copilot/agents/my-agent.agent.md" ]
refute [ -d "$FAKE_HOME/.github" ]
rm -rf "$FAKE_HOME"
}
@test "user scope is checked directly at \$HOME, no walk-up: a marker-less subdir under \$HOME resolves to project scope, not user scope" {
FAKE_HOME="$(mktemp -d)"
mkdir -p "$FAKE_HOME/scratch/testdir"
run env HOME="$FAKE_HOME" bash "$SCRIPT" my-agent "$FAKE_HOME/scratch/testdir"
assert_success
assert [ -f "$FAKE_HOME/scratch/testdir/.claude/agents/my-agent.md" ]
assert [ -f "$FAKE_HOME/scratch/testdir/.github/agents/my-agent.agent.md" ]
refute [ -f "$FAKE_HOME/.claude/agents/my-agent.md" ]
refute [ -f "$FAKE_HOME/.copilot/agents/my-agent.agent.md" ]
rm -rf "$FAKE_HOME"
}
# ---------------------------------------------------------------------------
# Name validation
# ---------------------------------------------------------------------------
@@ -164,21 +299,3 @@ teardown() {
run bash "$SCRIPT" my-agent "/nonexistent/path"
assert_failure
}
# ---------------------------------------------------------------------------
# Template content
# ---------------------------------------------------------------------------
@test "plugin scope: claude code template contains AGENT_NAME substituted" {
touch "$ROOT/plugin.json"
bash "$SCRIPT" my-agent "$ROOT"
run grep "my-agent" "$ROOT/agents/my-agent.md"
assert_success
}
@test "plugin scope: copilot template contains AGENT_NAME substituted" {
touch "$ROOT/plugin.json"
bash "$SCRIPT" my-agent "$ROOT"
run grep "my-agent" "$ROOT/agents/my-agent.agent.md"
assert_success
}

View File

@@ -65,18 +65,18 @@ When the intent spans multiple artifact types (e.g. a new skill inside a new plu
## Step 4 — Bump plugin version (if applicable)
After the routed skill finishes, check if the artifact was created or updated inside a plugin directory (any directory containing a `plugin.json`).
After the routed skill finishes, check if the artifact was created or updated inside a package by walking up from the artifact's path to the nearest ancestor `apm.yml` that declares a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`). An `apm.yml` with no `type:` field is a marketplace-only manifest (see `plugins/kyberforge/docs/research/docs/microsoft-apm/monorepo-and-repo-shapes.md`) — it does not count as a match; skip it and keep walking up.
**Skip this step if:**
- The artifact is standalone (not in any plugin) or scoped to user agent directories
- The author skill already bumped the plugin version (check the skill's audit output or completion message for version bump evidence)
- No ancestor `apm.yml` with a `type:` field is found (the artifact is standalone or scoped to user agent directories)
- The author skill already bumped the package version (check the skill's audit output or completion message for version bump evidence)
**If the artifact is in a plugin and no version bump was done:**
**If a typed `apm.yml` is found and no version bump was done:**
Invoke `kyberforge:plugin-author` as a **clean-context subagent** (fresh, not forked) with this brief:
Invoke `kyberforge:apm-workflow` as a **clean-context subagent** (fresh, not forked) with this brief:
> "The plugin at `<plugin-path>` gained a new `<artifact-type>` (`<artifact-name>`). Run the UPDATE flow to bump `version` in both manifests (Copilot and CC). Determine whether to bump minor (0.1.0) or patch (0.0.1) based on whether this is a new capability (minor) or a fix/refactor (patch). Do not release or tag — just update the manifests and commit."
> "The package at `<package-path>` gained a new `<artifact-type>` (`<artifact-name>`). Bump the `version` field in that package's `apm.yml`. Determine whether to bump minor (0.1.0) or patch (0.0.1) based on whether this is a new capability (minor) or a fix/refactor (patch). Do not release or tag — just update `apm.yml` and commit."
Use a clean-context subagent (not forked) so the version bump decision is made independently without anchoring to the earlier authoring context. This gives plugin-author a clear, isolated directive.
Use a clean-context subagent (not forked) so the version bump decision is made independently without anchoring to the earlier authoring context. This gives apm-workflow a clear, isolated directive.
Report completion to the user: "Updated `<plugin-name>` version from X.Y.Z to X.Y.Z to reflect the new `<artifact-name>`."
Report completion to the user: "Updated `<package-name>` version from X.Y.Z to X.Y.Z to reflect the new `<artifact-name>`."

View File

@@ -92,7 +92,7 @@ Check each pattern is appropriate and correctly formed:
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
- Optional directories contain real content — not just unfilled placeholder READMEs
- `README.md` present and accurately describes the skill and its files
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
- No cross-plugin path references in SKILL.md, scripts/, references/, or assets/ — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/<plugin>/skills/<other-skill>/`, or its APM-native equivalent `.apm/skills/<other-skill>/`) break when the plugin is installed to a cache; flag any found
- `references/sources.md` is exempt from the cross-plugin path check — `Research doc:` fields are development-only provenance pointers, not runtime references; they intentionally reference paths outside the skill directory and are expected to be non-resolvable after plugin install; `validate-provenance.sh` handles this gracefully by silently skipping upstream checks when those paths don't resolve
- `tests/` is exempt from the cross-plugin path check — test files are dev-only and may reference repo-level test infrastructure (e.g. a shared `tests/test_helper/`). This dependency must be declared in `tests/README.md`; flag if tests exist but `tests/README.md` is absent or does not document the dependency

View File

@@ -14,12 +14,14 @@ Routes to one of two flows based on context: if no skill directory exists at the
## Placement
| Type | Path |
|------|------|
| Direct (available immediately after install) | `~/.agents/skills/<name>/` |
| Plugin (installed via marketplace) | `plugins/<plugin>/skills/<name>/` |
`scripts/new-skill.sh` resolves the mode automatically by walking up from the given path — see `SKILL.md` Step 1 for the full algorithm.
If the destination is inside a plugin directory, read `references/deployment-modes.md` — cache isolation rules apply.
| Mode | Path | Chosen when |
|------|------|-------------|
| Standalone | `<path>/<name>/` | No `apm.yml` with a top-level `type:` field is found walking up from `<path>`, before hitting `.git` or the filesystem root |
| Package (APM) | `<package-root>/.apm/skills/<name>/` | A type-bearing `apm.yml` is found at or above `<path>` — `<path>` just needs to be somewhere inside the package |
If the destination resolves inside an APM package, read `references/deployment-modes.md` — self-containment rules apply to `apm compile` output the same way they applied to plugin cache isolation.
## Usage
@@ -33,8 +35,8 @@ If the destination is inside a plugin directory, read `references/deployment-mod
|------|---------|
| `README.md` | Human-readable overview of the skill and its files |
| `SKILL.md` | Skill instructions for agents |
| `scripts/new-skill.sh` | Copies annotated templates to the destination to scaffold a new skill |
| `references/deployment-modes.md` | Plugin vs standalone differences and cache isolation rules (loaded on demand) |
| `scripts/new-skill.sh` | Walks up from the given path to resolve package vs standalone mode, then copies annotated templates to the resolved destination |
| `references/deployment-modes.md` | APM package vs standalone differences and self-containment/cache-isolation rules (loaded on demand) |
| `references/scripts.md` | Package runners, inline dependency patterns, and full script contract (loaded on demand) |
| `references/sources.md` | Upstream research sources and which skill files each contributed to |
| `assets/templates/SKILL.md` | Annotated SKILL.md template |

View File

@@ -38,6 +38,11 @@ Determine which flow to follow before touching the filesystem:
Signals include: grill session output, `/skill-audit` findings (PASS/FAIL punch list), inline user feedback, session context describing what went wrong.
**Before running the scaffold script**, judge whether the destination is meant to be inside an APM package — the script can't tell "no package here" apart from "package not scaffolded yet":
- Package intent but no `type:`-bearing `apm.yml` found at/above the destination (e.g. "add to my apm package", or a sibling `.apm/`/`apm.yml` exists nearby) → **stop**, tell the user to run `/apm-workflow configure` (`apm plugin init`, from inside the package directory) first, then retry. Don't fall through to standalone mode.
- Otherwise (a `~/`-rooted destination, or no package context implied) → run `scripts/new-skill.sh`; it resolves package vs. standalone automatically (see Step 1).
## Creating a new skill
### Prerequisites
@@ -51,33 +56,42 @@ Design for one coherent user intent — skills too narrow force multiple loads p
- [ ] A clear purpose — what specific task will this skill handle?
- [ ] Trigger scenarios — when should an agent activate it, including indirect cases?
- [ ] Skill name (kebab-case) and destination path
- [ ] Capture `git log --oneline -1` now, before touching the filesystem — Step 6 needs it to verify a real commit landed
- [ ] Capture `git log --oneline -1` now, before touching the filesystem — Step 7 needs it to verify a real commit landed
If any are missing, stop and ask the user before proceeding.
**Requires `/skill-audit`** — used in Step 6 for final validation. Both skills ship in the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
**Requires `/skill-audit`** — used in Step 7 for final validation. Both skills ship in the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
### Step 1 — Scaffold
Run the copy script with the skill name and destination directory:
Run the copy script with the skill name and a path inside or at the target:
```bash
bash scripts/new-skill.sh <skill-name> <destination-dir>
bash scripts/new-skill.sh <skill-name> <path>
```
The script walks up from `<path>` for a package boundary: an ancestor `apm.yml` with a top-level `type:` field (`instructions`/`skill`/`hybrid`/`prompts`) means **package mode** — scaffolds into `<package-root>/.apm/skills/<skill-name>/`, not under `<path>` (a subdirectory of the package works fine as `<path>`). A `type:`-less `apm.yml` is a marketplace-only manifest, skipped. Hitting `.git` or the filesystem root first means **standalone mode** — scaffolds directly into `<path>/<skill-name>/`, same as before.
Examples:
```bash
# Package mode — packages/my-pkg/apm.yml already has `type: skill`
bash scripts/new-skill.sh my-tool packages/my-pkg/
# Standalone mode — no apm.yml/.git above ~/.agents/skills/
bash scripts/new-skill.sh my-tool ~/.agents/skills/
bash scripts/new-skill.sh data-analyzer plugins/myplugin/skills/
```
This creates `<destination-dir>/<skill-name>/` with annotated templates ready to fill in.
The script prints which mode it used and where the skill landed — read its output.
If the destination is inside a plugin directory (path contains a `plugin.json`), read `references/deployment-modes.md` before adding any file references to SKILL.md.
In package mode, read `references/deployment-modes.md` before adding any file references to SKILL.md.
### Step 2 — Fill in SKILL.md
### Step 2 — Update `apm.yml` includes (package mode only)
Open `<destination-dir>/<skill-name>/SKILL.md`. Replace every `FILL IN:` placeholder.
Skip in standalone mode. In package mode, check the resolved package's `apm.yml`: if `includes:` is an explicit list (not `auto`), append `.apm/skills/<skill-name>/` to it if not already present, preserving YAML formatting. If `includes: auto` or the field is absent, do nothing — `auto` already covers the new skill. Use Read/Edit directly on `apm.yml`; this isn't part of `scripts/new-skill.sh`.
### Step 3 — Fill in SKILL.md
Open the new skill's `SKILL.md` (the path Step 1 printed). Replace every `FILL IN:` placeholder.
**Frontmatter**
@@ -97,7 +111,7 @@ Open `<destination-dir>/<skill-name>/SKILL.md`. Replace every `FILL IN:` placeho
- `metadata` — key-value map; use `author`, `version`, `category`; add `source_keys` now (see below) if research sources are in context
- `allowed-tools` — space-separated pre-approved tools; reduces permission prompts (experimental — support varies by client)
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs here as you write the body; don't defer this to Step 5. Agents that fill in source_keys late tend to omit it entirely. Example:
**`metadata.source_keys`** — if research sources are in context, list the relevant slugs here as you write the body; don't defer this to Step 6. Agents that fill in source_keys late tend to omit it entirely. Example:
```yaml
metadata:
source_keys:
@@ -105,7 +119,7 @@ metadata:
- another-slug
```
**Embedding org-specific policy** — if a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`), inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing to the file's path. Plugins must be self-contained and portable — the org file may not exist wherever the plugin is installed, and in this repo such files are meant to be deleted once their content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the same `references/sources.md` schema as Step 5, noting in the `Research doc:` field that the source is an org convention rather than a plugin research corpus entry, so provenance survives after the source file is gone.
**Embedding org-specific policy** — if a skill encodes a rule sourced from an org convention file (e.g. `core/instructions/*.md`), inline that content directly into the skill (SKILL.md or a `references/` file) rather than pointing to the file's path. Plugins must be self-contained and portable — the org file may not exist wherever the plugin is installed, and in this repo such files are meant to be deleted once their content is fully embedded downstream. Tag the inlined content with a `source_keys` entry using the same `references/sources.md` schema as Step 6, noting in the `Research doc:` field that the source is an org convention rather than a plugin research corpus entry, so provenance survives after the source file is gone.
**Body — include only what the agent lacks**
@@ -175,7 +189,7 @@ Keep `SKILL.md` under 500 lines; 5,000 tokens is the recommended body budget. Wh
- Move reference material to `references/<topic>.md` and load it conditionally
- Bundle repeated executable logic into `scripts/` rather than reinventing each run
### Step 3 — Add scripts (if needed)
### Step 4 — Add scripts (if needed)
Place executable scripts in `scripts/`. Critical rule: **no interactive prompts** — agents run non-interactive; blocking on TTY input hangs indefinitely. Accept all input via flags, env vars, or stdin.
@@ -183,7 +197,7 @@ If adding a script, read `references/scripts.md` first — it covers the full co
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
### Step 4 — Add references, assets, and tests (if needed)
### Step 5 — Add references, assets, and tests (if needed)
**`references/`** — additional documentation loaded on demand. One topic per file.
Reference conditionally from SKILL.md: `If <condition>, read references/<file>.md`.
@@ -198,7 +212,7 @@ not in `scripts/`. See `tests/README.md` for setup instructions.
If not needed, delete the placeholder READMEs and their directories.
### Step 5 — Populate or delete `references/sources.md`
### Step 6 — Populate or delete `references/sources.md`
If a research `sources.md` is present in the conversation context:
@@ -210,7 +224,7 @@ If a research `sources.md` is present in the conversation context:
If no research `sources.md` is in context, delete `references/sources.md`.
### Step 6 — Validate and close
### Step 7 — Validate and close
Before running the audit, confirm:
- [ ] Skill name matches the directory name exactly
@@ -218,7 +232,7 @@ Before running the audit, confirm:
- [ ] Body has at least one non-empty section
- [ ] No `FILL IN:` placeholders remain in any file
Run `/skill-audit` on `<destination-dir>/<skill-name>`.
Run `/skill-audit` on the skill directory Step 1 reported — either `<package-root>/.apm/skills/<skill-name>/` or `<path>/<skill-name>/`.
All FAIL findings must be resolved before the skill is considered done.

View File

@@ -5,7 +5,7 @@ source_keys:
# Deployment Modes
Skills deploy in two modes. Both resolve relative paths from the skill root — the SKILL.md body works the same in either. Differences only arise when referencing files *outside* the skill directory.
Skills deploy standalone, or as part of a package — either a legacy plugin-mode cache install or an APM (`apm.yml`-governed `.apm/` tree, compiled via `apm compile`). All resolve relative paths from the skill root — the SKILL.md body works the same in any of them. Differences only arise when referencing files *outside* the skill directory.
## Cache isolation (plugin mode)
@@ -19,6 +19,18 @@ plugins/kyberforge/skills/other-skill/ # breaks
Fix: duplicate the file into the skill's own `scripts/` or `assets/`. There is no plugin-level `shared/` mechanism — the spec defines no cross-skill sharing, and `../` paths are broken by construction.
## Compiled output (APM package mode)
For a package (an `apm.yml`-governed `.apm/` source tree), the deployable artifact is generated by `apm compile` per target harness — not produced by copying the raw `.apm/` directory wholesale the way a plugin cache install copies a plugin directory. The same self-containment rule still applies at the skill level: **file references inside `.apm/skills/<name>/` must not reach outside that skill's own directory.**
```
../other-skill/validate.sh # breaks
.apm/skills/other-skill/ # breaks
../../shared/utils.sh # breaks
```
Fix: duplicate the file into the skill's own `scripts/` or `assets/`, same as plugin mode. `apm.yml`'s `includes:` list (when explicit, not `auto`) controls what gets published from the package, but it is not a cross-skill sharing mechanism — each skill directory must still stand alone.
## Env vars (plugin mode only)
These variables are injected when the plugin is loaded from an install cache. They are **not available in standalone mode.**
@@ -36,7 +48,7 @@ Deployed directly to `~/.agents/skills/<name>/`. No plugin context, no env vars
## Cross-tool portability
`SKILL.md` is portable — the same file works in Claude Code and Copilot CLI. Agent definitions and manifest files (`plugin.json`, `hooks.json`) are tool-specific and must be authored separately per tool.
`SKILL.md` is portable — the same file works in Claude Code and Copilot CLI, whether deployed standalone or compiled from an APM package. `apm.yml` is the source manifest: it is itself tool-agnostic (one file describes the package regardless of target), but `apm compile` produces per-target compiled output — a Claude Code plugin tree, a Copilot CLI tree, etc. — from it. Legacy hand-authored manifest files (`plugin.json`, `hooks.json`) are tool-specific and authored separately per tool; they sit outside the `apm.yml`-based flow.
## Shared assets between skills

View File

@@ -6,22 +6,42 @@ TEMPLATES_DIR="$SKILL_DIR/../assets/templates"
usage() {
cat <<EOF
Usage: new-skill.sh <skill-name> <destination-dir>
Usage: new-skill.sh <skill-name> <path>
Create a new skill scaffold by copying annotated templates to the destination.
Create a new skill scaffold by copying annotated templates to the resolved
destination. <path> is any existing path inside or at the target — a
package or a standalone location. It does not have to be a package root
itself.
The script walks up from <path> to pick one of two modes:
Package mode:
If an apm.yml with a top-level 'type:' field (instructions, skill,
hybrid, or prompts) is found at or above <path>, the skill is
scaffolded into <package-root>/.apm/skills/<skill-name>/ — not under
<path> itself. An apm.yml with no 'type:' field is a marketplace-only
manifest, not a package; it is skipped and the walk continues upward.
Standalone mode:
If the walk reaches a '.git' directory or the filesystem root without
finding a type-bearing apm.yml, the skill is scaffolded directly into
<path>/<skill-name>/, exactly as <path> was given.
Arguments:
skill-name Kebab-case skill identifier (e.g. my-tool, data-analyzer).
Must match the directory name exactly.
destination-dir Parent directory to create the skill in.
Examples: ~/.agents/skills/ plugins/myplugin/skills/
skill-name Kebab-case skill identifier (e.g. my-tool, data-analyzer).
Must match the directory name exactly.
path Any existing path inside/at the target. Used to locate the
package (package mode) or as the literal parent directory
(standalone mode). Must already exist.
Examples: ~/.agents/skills/ packages/my-pkg/some/subdir/
Output:
Creates <destination-dir>/<skill-name>/ with annotated templates ready to fill in.
Package mode: <package-root>/.apm/skills/<skill-name>/
Standalone mode: <path>/<skill-name>/
Exit codes:
0 Scaffold created successfully, or destination already exists (no-op)
1 Invalid arguments, missing destination parent, or templates not found
1 Invalid arguments, missing path, or templates not found
EOF
}
@@ -31,14 +51,14 @@ if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
fi
if [[ $# -lt 2 ]]; then
echo "Error: skill-name and destination-dir are required." >&2
echo "Error: skill-name and path are required." >&2
echo "" >&2
usage >&2
exit 1
fi
SKILL_NAME="$1"
DEST_DIR="$2"
TARGET_INPUT="$2"
# Validate skill name format
if ! echo "$SKILL_NAME" | grep -qE '^[a-z0-9]+(-[a-z0-9]+)*$'; then
@@ -55,13 +75,82 @@ if [[ ! -d "$TEMPLATES_DIR" ]]; then
exit 1
fi
# Validate destination exists
if [[ ! -d "$DEST_DIR" ]]; then
echo "Error: destination directory '$DEST_DIR' does not exist." >&2
# Validate path exists
if [[ ! -d "$TARGET_INPUT" ]]; then
echo "Error: path '$TARGET_INPUT' does not exist." >&2
exit 1
fi
TARGET="$DEST_DIR/$SKILL_NAME"
# True if apm_yml's top-level `type:` line names one of the four APM package
# types (instructions/skill/hybrid/prompts) — mirrors validate.sh's
# APM_TYPE_RE: an optional quote around the value must be closed by the
# *same* quote character (a mismatched or unterminated quote is rejected,
# not silently stripped), and the value must be followed by whitespace or
# end-of-line so `prompts-only` doesn't false-match on the `prompts` prefix.
# `|| [[ -n "$line" ]]` in the read condition also processes a final line

Missing $HOME boundary check. find_package_root() has no $HOME boundary check (unlike new-agent.sh's), so the walk-up can continue past $HOME and bind to an unrelated ancestor package.

Reproduced with a fake $HOME nested under a directory with a type-bearing apm.yml above it: new-skill.sh my-skill $HOME/skills walked past $HOME and scaffolded into the ancestor package's .apm/skills/my-skill/ instead of the intended standalone location under $HOME.

**Missing $HOME boundary check.** `find_package_root()` has no `$HOME` boundary check (unlike `new-agent.sh`'s), so the walk-up can continue past `$HOME` and bind to an unrelated ancestor package. Reproduced with a fake `$HOME` nested under a directory with a type-bearing `apm.yml` above it: `new-skill.sh my-skill $HOME/skills` walked past `$HOME` and scaffolded into the ancestor package's `.apm/skills/my-skill/` instead of the intended standalone location under `$HOME`.
# that lacks a trailing newline, which `read` alone would otherwise skip.
# Identical to agent-author's new-agent.sh copy of this helper.
is_apm_package_manifest() {
local apm_yml="$1" line
while IFS= read -r line || [[ -n "$line" ]]; do

Loose regex lets malformed type: values false-match. This type: regex uses \b word-boundary matching (also present in validate.sh/validate-provenance.sh) instead of new-agent.sh's stricter ([[:space:]]|$), so a malformed value like type: prompts-only false-matches as valid prompts in three scripts but is correctly rejected in new-agent.sh.

An apm.yml with type: prompts-only (typo) is treated as a valid type: prompts package by validate.sh, validate-provenance.sh, and new-skill.sh, but new-agent.sh walks past it looking for a different package root — an agent and a skill scaffolded from the same directory land in different roots for the same manifest.

**Loose regex lets malformed `type:` values false-match.** This `type:` regex uses `\b` word-boundary matching (also present in `validate.sh`/`validate-provenance.sh`) instead of `new-agent.sh`'s stricter `([[:space:]]|$)`, so a malformed value like `type: prompts-only` false-matches as valid `prompts` in three scripts but is correctly rejected in `new-agent.sh`. An `apm.yml` with `type: prompts-only` (typo) is treated as a valid `type: prompts` package by `validate.sh`, `validate-provenance.sh`, and `new-skill.sh`, but `new-agent.sh` walks past it looking for a different package root — an agent and a skill scaffolded from the same directory land in different roots for the same manifest.
if [[ "$line" =~ ^type:[[:space:]]*(instructions|skill|hybrid|prompts)([[:space:]]|$) ]]; then
return 0
fi
if [[ "$line" =~ ^type:[[:space:]]*([\"\'])(instructions|skill|hybrid|prompts)([\"\'])([[:space:]]|$) ]] \
&& [[ "${BASH_REMATCH[1]}" == "${BASH_REMATCH[3]}" ]]; then
return 0
fi
done < "$apm_yml"
return 1
}
# ---------------------------------------------------------------------------
# Walk up from <path> looking for a type-bearing apm.yml (package mode) or a
# .git boundary / filesystem root (standalone mode). An apm.yml with no
# top-level 'type:' field is a marketplace-only manifest — skip it and keep
# walking up. Prints one space-separated line: mode, then the resolved root.
# ---------------------------------------------------------------------------
find_package_root() {
local current
current="$(cd "$1" && pwd)"
while true; do
if [[ -f "$current/apm.yml" ]]; then
if is_apm_package_manifest "$current/apm.yml"; then
echo "package $current"
return 0
fi
# apm.yml exists but has no type: field — marketplace-only manifest.
# Not a package match; keep walking up.
fi
# .git is a directory in a normal checkout but a file (`gitdir: ...`) in
# a git worktree — -e covers both.
if [[ -e "$current/.git" ]]; then
echo "no-package $current"
return 0
fi
local parent
parent="$(dirname "$current")"
if [[ "$parent" == "$current" ]]; then
echo "no-package $current"
return 0
fi
current="$parent"
done
}
# `mapfile`/`readarray` are bash 4.0+ builtins with no fallback on macOS's
# stock /bin/bash 3.2 — read the single space-separated output line with a
# plain `read` instead (bash 3.2-safe). `read` consumes only one line, so
# mode and path must be on the same line: MODE first (never contains
# whitespace), PKG_ROOT last (safely absorbs a path containing spaces).
WALK_OUTPUT="$(find_package_root "$TARGET_INPUT")"
read -r MODE PKG_ROOT <<< "$WALK_OUTPUT"
if [[ "$MODE" == "package" ]]; then
TARGET="$PKG_ROOT/.apm/skills/$SKILL_NAME"
else
TARGET="$TARGET_INPUT/$SKILL_NAME"
fi
# Destination already exists — treat as a no-op so retries are safe
if [[ -d "$TARGET" ]]; then
@@ -69,6 +158,8 @@ if [[ -d "$TARGET" ]]; then
exit 0
fi
mkdir -p "$(dirname "$TARGET")"
# Copy templates to destination
cp -r "$TEMPLATES_DIR" "$TARGET"
@@ -77,7 +168,16 @@ sed -i "s/SKILL_NAME/$SKILL_NAME/g" "$TARGET/SKILL.md"
sed -i "s/SKILL_NAME/$SKILL_NAME/g" "$TARGET/README.md"
sed -i "s/SKILL_NAME/$SKILL_NAME/g" "$TARGET/tests/README.md"
echo "Scaffold created: $TARGET" >&2
if [[ "$MODE" == "package" ]]; then
echo "Mode: package — type-bearing apm.yml found at '$PKG_ROOT'" >&2
echo "Scaffold created: $TARGET" >&2
echo "" >&2
echo "Note: if '$PKG_ROOT/apm.yml' has an explicit 'includes:' list (not 'auto')," >&2
echo " add '.apm/skills/$SKILL_NAME/' to it." >&2
else
echo "Mode: standalone — no type-bearing apm.yml found above '$TARGET_INPUT'" >&2
echo "Scaffold created: $TARGET" >&2
fi
echo "" >&2
echo "Next steps:" >&2
echo " 1. Fill in $TARGET/SKILL.md — replace all FILL IN: placeholders" >&2

View File

@@ -117,3 +117,92 @@ teardown() {
assert_success
assert_output --partial "nothing to do"
}
# ---------------------------------------------------------------------------
# Mode detection: package vs standalone
# ---------------------------------------------------------------------------
@test "standalone mode is chosen when no apm.yml or .git is found above path" {
run bash "$SCRIPT" my-tool "$DEST"
assert_success
assert_output --partial "Mode: standalone"
assert [ -d "$DEST/my-tool" ]
}
@test "package mode: writes into <package-root>/.apm/skills/<name>, not under the given subdir" {
mkdir -p "$DEST/pkg/sub/deep"
cat > "$DEST/pkg/apm.yml" <<'EOF'
name: my-pkg
version: 1.0.0
type: skill
EOF
run bash "$SCRIPT" my-tool "$DEST/pkg/sub/deep"
assert_success
assert_output --partial "Mode: package"
assert [ -d "$DEST/pkg/.apm/skills/my-tool" ]
assert [ ! -d "$DEST/pkg/sub/deep/my-tool" ]
}
@test "package mode scaffold contains SKILL.md with name substituted" {
mkdir -p "$DEST/pkg"
cat > "$DEST/pkg/apm.yml" <<'EOF'
name: my-pkg
version: 1.0.0
type: hybrid
EOF
bash "$SCRIPT" my-tool "$DEST/pkg"
assert [ -f "$DEST/pkg/.apm/skills/my-tool/SKILL.md" ]
run grep "my-tool" "$DEST/pkg/.apm/skills/my-tool/SKILL.md"
assert_success
}
@test "walk-up skips a type-less apm.yml (marketplace-only) and finds a real package root further up" {
mkdir -p "$DEST/mid/sub"
cat > "$DEST/apm.yml" <<'EOF'
name: root-pkg
version: 1.0.0
type: skill
EOF
cat > "$DEST/mid/apm.yml" <<'EOF'
marketplace:
owner: acme
packages: []
EOF
run bash "$SCRIPT" my-tool "$DEST/mid/sub"
assert_success
assert_output --partial "Mode: package"
assert [ -d "$DEST/.apm/skills/my-tool" ]
assert [ ! -d "$DEST/mid/.apm" ]
}
@test "walk-up stops at .git boundary and uses standalone mode at the given path" {
mkdir -p "$DEST/repo/.git"
mkdir -p "$DEST/repo/sub"
run bash "$SCRIPT" my-tool "$DEST/repo/sub"
assert_success
assert_output --partial "Mode: standalone"
assert [ -d "$DEST/repo/sub/my-tool" ]
assert [ ! -d "$DEST/repo/my-tool" ]
}
@test "package mode: matched-quote type value ('skill') is recognized" {
mkdir -p "$DEST/pkg"
printf 'name: my-pkg\ntype: "skill"\n' > "$DEST/pkg/apm.yml"
run bash "$SCRIPT" my-tool "$DEST/pkg"
assert_success
assert_output --partial "Mode: package"
}
@test "mismatched-quote type value is rejected, falls through to standalone mode" {
printf "name: my-pkg\ntype: \"skill'\n" > "$DEST/apm.yml"
run bash "$SCRIPT" my-tool "$DEST"
assert_success
assert_output --partial "Mode: standalone"
}
@test "type: line is recognized even without a trailing newline on apm.yml" {
printf 'name: my-pkg\ntype: skill' > "$DEST/apm.yml"
run bash "$SCRIPT" my-tool "$DEST"
assert_success
assert_output --partial "Mode: package"
}

View File

@@ -0,0 +1,342 @@
#!/usr/bin/env bash
set -euo pipefail
# Behavioral consistency check for the four independent, semantically-equivalent
# ports of "walk up from a directory looking for a scope-defining marker" living
# in this repo:
#
# - plugins/kyberforge/skills/agent-audit/scripts/validate.sh (Python: detect_scope)
# - plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh (Python: find_plugin_root)
# - plugins/kyberforge/skills/agent-author/scripts/new-agent.sh (Bash: find_package_root)
# - plugins/kyberforge/skills/skill-author/scripts/new-skill.sh (Bash: find_package_root)
#
# Per ADR-0014's no-cross-skill-path rule, these can't be consolidated into a
# shared file (each skill's cache-install copies only its own files), so unlike
# check-vale-style-sync.sh (which diffs literal file copies) this can't be a
# text diff — the four implementations are hand-ported, not copied. Instead
# this builds a matrix of fixture directory trees and asserts the *observable
# behavior* agrees: whatever new-agent.sh/new-skill.sh actually create on disk,
# validate.sh/validate-provenance.sh must classify the same way when pointed at
# the result. Run from repo root or pass REPO_ROOT as arg.
REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"
if [[ ! -d "$REPO_ROOT" ]]; then
echo "Scope walk-up sync check failed: REPO_ROOT '$REPO_ROOT' is not a directory." >&2
exit 1
fi
REPO_ROOT="$(cd "$REPO_ROOT" && pwd)"
NEW_AGENT="$REPO_ROOT/plugins/kyberforge/skills/agent-author/scripts/new-agent.sh"
NEW_SKILL="$REPO_ROOT/plugins/kyberforge/skills/skill-author/scripts/new-skill.sh"
VALIDATE="$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate.sh"
VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh"
for f in "$NEW_AGENT" "$NEW_SKILL" "$VALIDATE" "$VALIDATE_PROVENANCE"; do
if [[ ! -f "$f" ]]; then
echo "Scope walk-up sync check: $f not found — kyberforge agent-author/agent-audit/skill-author skills not present, nothing to check." >&2
exit 0
fi
done
FAIL=0
err() { echo " FAIL: $1" >&2; FAIL=$((FAIL + 1)); }
ok() { echo " ok: $1"; }
FIXTURES=()
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
trap cleanup EXIT
# Fill a new-agent.sh-scaffolded pair's FILL IN: placeholders with valid
# content, isolating the scope/counterpart-lookup question from unrelated
# content-quality FAILs when cross-checking against validate.sh.
fill_agent_pair() {
local file="$1" name="$2"
cat > "$file" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
}
# ---------------------------------------------------------------------------
# Fixture 1: root exactly $HOME (no walk-up) — new-agent.sh's own documented
# user-scope case.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: root exactly \$HOME ---"
F1_HOME="$(mktemp -d)"
FIXTURES+=("$F1_HOME")
NAME1="probe-home-exact"
if ! env HOME="$F1_HOME" bash "$NEW_AGENT" "$NAME1" "$F1_HOME" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at root exactly \$HOME"
else
if [[ ! -f "$F1_HOME/.claude/agents/$NAME1.md" || ! -f "$F1_HOME/.copilot/agents/$NAME1.agent.md" ]]; then
err "new-agent.sh did not create the expected user-scope pair at \$HOME/.claude and \$HOME/.copilot"
else
fill_agent_pair "$F1_HOME/.claude/agents/$NAME1.md" "$NAME1"
fill_agent_pair "$F1_HOME/.copilot/agents/$NAME1.agent.md" "$NAME1"
if env HOME="$F1_HOME" bash "$VALIDATE" "$F1_HOME/.claude/agents/$NAME1.md" >/tmp/f1.out 2>&1; then
ok "validate.sh agrees: user scope, counterpart found under \$HOME/.copilot"
else
err "validate.sh disagreed with new-agent.sh's user-scope classification at root exactly \$HOME"
sed 's/^/ /' /tmp/f1.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 2: nested marker-less directory under $HOME — the live-repro shape.
# new-agent.sh's own docs call this out as deliberately project scope, not
# user scope (a stray directory under $HOME can't be silently redirected into
# the shared global ~/.claude or ~/.copilot agent directories).
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: nested marker-less directory under \$HOME ---"
F2_HOME="$(mktemp -d)"
FIXTURES+=("$F2_HOME")
F2_NESTED="$F2_HOME/scratch/testdir"
mkdir -p "$F2_NESTED"
NAME2="probe-home-nested"
if ! env HOME="$F2_HOME" bash "$NEW_AGENT" "$NAME2" "$F2_NESTED" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold under a nested marker-less \$HOME subdirectory"
else
if [[ ! -f "$F2_NESTED/.claude/agents/$NAME2.md" || ! -f "$F2_NESTED/.github/agents/$NAME2.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the nested dir (rooted at \$F2_NESTED, not \$HOME)"
elif [[ -f "$F2_HOME/.claude/agents/$NAME2.md" || -f "$F2_HOME/.copilot/agents/$NAME2.agent.md" ]]; then
err "new-agent.sh unexpectedly wrote into \$HOME/.claude or \$HOME/.copilot for a nested marker-less start dir"
else
ok "new-agent.sh: nested marker-less dir under \$HOME scaffolds project scope at the nested dir"
fill_agent_pair "$F2_NESTED/.claude/agents/$NAME2.md" "$NAME2"
fill_agent_pair "$F2_NESTED/.github/agents/$NAME2.agent.md" "$NAME2"
if env HOME="$F2_HOME" bash "$VALIDATE" "$F2_NESTED/.claude/agents/$NAME2.md" >/tmp/f2.out 2>&1; then
ok "validate.sh agrees: project scope, counterpart found at the nested dir (not \$HOME/.copilot)"
else
err "validate.sh disagreed with new-agent.sh: misclassified the nested marker-less \$HOME subdirectory"
sed 's/^/ /' /tmp/f2.out
fi
# new-skill.sh has no user/project distinction of its own (no $HOME
# awareness at all — see new-skill.sh's find_package_root), but it shares
# the same .git/apm.yml walk-up primitive. It must land its standalone
# scaffold at the given path too, not get redirected toward $HOME.
if env HOME="$F2_HOME" bash "$NEW_SKILL" probe-home-nested-skill "$F2_NESTED" >/tmp/f2skill.out 2>&1 \
&& [[ -d "$F2_NESTED/probe-home-nested-skill" ]]; then
ok "new-skill.sh agrees: standalone mode scaffolds at the nested dir, not redirected toward \$HOME"
else
err "new-skill.sh disagreed with new-agent.sh/validate.sh on the nested marker-less \$HOME subdirectory"
sed 's/^/ /' /tmp/f2skill.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 3: a .git boundary between the probe dir and $HOME must stop the
# walk before it ever reaches $HOME (so it can't be misclassified as user
# scope via the home-boundary path).
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: .git boundary short-circuits before reaching \$HOME ---"
F3_HOME="$(mktemp -d)"
FIXTURES+=("$F3_HOME")
# .git sits directly at the probe root (the conventional two-segments-above
# location .claude/agents and .github/agents are placed relative to). This
# fixture only exercises what it's meant to: that a .git ancestor stops the
# walk before it ever reaches $HOME. Fixture 3b below covers .git sitting
# higher up than the probe root.
F3_PROBE="$F3_HOME/myrepo"
mkdir -p "$F3_PROBE/.git"
NAME3="probe-git-boundary"
if ! env HOME="$F3_HOME" bash "$NEW_AGENT" "$NAME3" "$F3_PROBE" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a dir with a .git ancestor short of \$HOME"
else
if [[ ! -f "$F3_PROBE/.claude/agents/$NAME3.md" || ! -f "$F3_PROBE/.github/agents/$NAME3.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the probe dir"
else
fill_agent_pair "$F3_PROBE/.claude/agents/$NAME3.md" "$NAME3"
fill_agent_pair "$F3_PROBE/.github/agents/$NAME3.agent.md" "$NAME3"
if env HOME="$F3_HOME" bash "$VALIDATE" "$F3_PROBE/.claude/agents/$NAME3.md" >/tmp/f3.out 2>&1; then
ok "validate.sh agrees: .git boundary keeps this project scope, not promoted to user scope at \$HOME"
else
err "validate.sh disagreed with new-agent.sh on the .git-boundary-before-\$HOME fixture"
sed 's/^/ /' /tmp/f3.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 3b: .git sits one level ABOVE the probe root — a subdirectory of a
# larger git-tracked tree (e.g. a monorepo package dir). new-agent.sh always
# places project-scope files at its ROOT argument, never at the walked-up
# .git location, so validate.sh must resolve scope to the probe root too, not
# to the ancestor where .git happened to be found.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: .git ancestor sits above <root> (subdirectory of a larger git tree) ---"
F3B_REPO="$(mktemp -d)"
FIXTURES+=("$F3B_REPO")
mkdir -p "$F3B_REPO/.git"
F3B_PROBE="$F3B_REPO/subdir"
mkdir -p "$F3B_PROBE"
NAME3B="probe-git-above-root"
if ! bash "$NEW_AGENT" "$NAME3B" "$F3B_PROBE" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a dir one level below a .git ancestor"
else
if [[ ! -f "$F3B_PROBE/.claude/agents/$NAME3B.md" || ! -f "$F3B_PROBE/.github/agents/$NAME3B.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the probe dir (rooted at \$F3B_PROBE, not the repo root)"
else
fill_agent_pair "$F3B_PROBE/.claude/agents/$NAME3B.md" "$NAME3B"
fill_agent_pair "$F3B_PROBE/.github/agents/$NAME3B.agent.md" "$NAME3B"
if bash "$VALIDATE" "$F3B_PROBE/.claude/agents/$NAME3B.md" >/tmp/f3b.out 2>&1; then
ok "validate.sh agrees: scope root is <root>, not the .git ancestor above it"
else
err "validate.sh disagreed with new-agent.sh: resolved scope to the .git ancestor instead of <root>"
sed 's/^/ /' /tmp/f3b.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 4: a type-bearing apm.yml — plugin/APM scope. new-agent.sh and
# new-skill.sh must agree on the same package root, and validate.sh /
# validate-provenance.sh must both recognize it as plugin scope.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: type-bearing apm.yml (plugin/APM scope) ---"
F4_ROOT="$(mktemp -d)"
FIXTURES+=("$F4_ROOT")
printf 'name: test-package\nversion: 0.1.0\ntype: skill\n' > "$F4_ROOT/apm.yml"
NAME4="probe-plugin"
if ! bash "$NEW_AGENT" "$NAME4" "$F4_ROOT" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a type-bearing apm.yml root"
elif [[ ! -f "$F4_ROOT/.apm/agents/$NAME4.agent.md" ]]; then
err "new-agent.sh did not scaffold plugin scope at the type-bearing apm.yml root"
else
ok "new-agent.sh: plugin scope at type-bearing apm.yml root"
if bash "$NEW_SKILL" probe-plugin-skill "$F4_ROOT" >/tmp/f4skill.out 2>&1 \
&& [[ -d "$F4_ROOT/.apm/skills/probe-plugin-skill" ]]; then
ok "new-skill.sh agrees: package mode at the same apm.yml root"
else
err "new-skill.sh disagreed with new-agent.sh on the type-bearing apm.yml root"
sed 's/^/ /' /tmp/f4skill.out
fi
fill_agent_pair "$F4_ROOT/.apm/agents/$NAME4.agent.md" "$NAME4"
if bash "$VALIDATE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4validate.out 2>&1; then
ok "validate.sh agrees: plugin/APM scope, structural checks pass"
else
err "validate.sh disagreed with new-agent.sh: did not treat the type-bearing apm.yml root as plugin scope"
sed 's/^/ /' /tmp/f4validate.out
fi
# source_keys + a matching sources.md round-trips only if validate-provenance.sh
# resolves the SAME plugin root new-agent.sh/new-skill.sh did.
cat > "$F4_ROOT/.apm/agents/$NAME4.agent.md" <<EOF
---
name: ${NAME4}
description: A valid agent description.
source_keys:
- probe-source
---
You are a test agent.
EOF
cat > "$F4_ROOT/sources.md" <<EOF
# Sources
## probe-source
- **URL:** https://example.com/probe-source
- **Description:** A test source.
- **Contributing files:** .apm/agents/${NAME4}.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
if bash "$VALIDATE_PROVENANCE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4prov.out 2>&1; then
ok "validate-provenance.sh agrees: resolves the same plugin root, sources.md round-trips"
else
err "validate-provenance.sh disagreed on the plugin root for the type-bearing apm.yml fixture"
sed 's/^/ /' /tmp/f4prov.out
fi
fi
# ---------------------------------------------------------------------------
# Fixture 5: filesystem-boundary fallback — no $HOME relation, no marker
# anywhere. Both scripts must fall through to project scope, not user scope.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: filesystem-boundary fallback (no \$HOME relation, no markers) ---"
F5_UNRELATED_HOME_PARENT="$(mktemp -d)"
FIXTURES+=("$F5_UNRELATED_HOME_PARENT")
F5_UNRELATED_HOME="$F5_UNRELATED_HOME_PARENT/never-reached-$$"
F5_ROOT="$(mktemp -d)/deep/proj"
mkdir -p "$F5_ROOT"
FIXTURES+=("$(dirname "$(dirname "$F5_ROOT")")")
NAME5="probe-fs-boundary"
if ! env HOME="$F5_UNRELATED_HOME" bash "$NEW_AGENT" "$NAME5" "$F5_ROOT" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at the filesystem-boundary fixture"
else
if [[ ! -f "$F5_ROOT/.claude/agents/$NAME5.md" || ! -f "$F5_ROOT/.github/agents/$NAME5.agent.md" ]]; then
err "new-agent.sh did not scaffold project scope at the filesystem-boundary fixture"
else
fill_agent_pair "$F5_ROOT/.claude/agents/$NAME5.md" "$NAME5"
fill_agent_pair "$F5_ROOT/.github/agents/$NAME5.agent.md" "$NAME5"
if env HOME="$F5_UNRELATED_HOME" bash "$VALIDATE" "$F5_ROOT/.claude/agents/$NAME5.md" >/tmp/f5.out 2>&1; then
ok "validate.sh agrees: filesystem-boundary fallback resolves to project scope"
else
err "validate.sh disagreed with new-agent.sh on the filesystem-boundary fallback fixture"
sed 's/^/ /' /tmp/f5.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 6: a type-bearing apm.yml ABOVE $HOME must not be reached by
# validate-provenance.sh's walk-up from a nested, marker-less dir under $HOME
# — matches new-agent.sh, which also stops at $HOME before ever looking that
# far up.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: type-bearing apm.yml above \$HOME must not be reached ---"
F6_ANCESTOR="$(mktemp -d)"
FIXTURES+=("$F6_ANCESTOR")
printf 'name: outer-package\nversion: 0.1.0\ntype: skill\n' > "$F6_ANCESTOR/apm.yml"
F6_HOME="$F6_ANCESTOR/fakehome"
mkdir -p "$F6_HOME"
NAME6="probe-above-home"
if ! env HOME="$F6_HOME" bash "$NEW_AGENT" "$NAME6" "$F6_HOME" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold with a type-bearing apm.yml above \$HOME"
elif [[ -f "$F6_HOME/.apm/agents/$NAME6.agent.md" ]]; then
err "new-agent.sh walked past \$HOME and misclassified as plugin scope using the ancestor apm.yml"
elif [[ ! -f "$F6_HOME/.claude/agents/$NAME6.md" ]]; then
err "new-agent.sh did not scaffold user scope at root exactly \$HOME (with a type-bearing apm.yml above)"
else
ok "new-agent.sh: \$HOME boundary stops the walk before the ancestor apm.yml, user scope at \$HOME"
mkdir -p "$F6_HOME/.apm/agents"
cat > "$F6_HOME/.apm/agents/probe-prov.agent.md" <<'EOF'
---
name: probe-prov
description: A valid agent description.
source_keys:
- probe-source
---
You are a test agent.
EOF
# No sources.md exists anywhere under $F6_HOME or at the ancestor package
# root — if find_plugin_root walked past $HOME to the ancestor apm.yml,
# this would FAIL on Check 0 (source_keys declared but sources.md absent).
if env HOME="$F6_HOME" bash "$VALIDATE_PROVENANCE" "$F6_HOME/.apm/agents/probe-prov.agent.md" >/tmp/f6.out 2>&1 \
&& [[ -z "$(cat /tmp/f6.out)" ]]; then
ok "validate-provenance.sh agrees: \$HOME boundary stops the walk, exits 0 silently (not plugin scope)"
else
err "validate-provenance.sh walked past \$HOME to the ancestor apm.yml — disagrees with new-agent.sh"
sed 's/^/ /' /tmp/f6.out
fi
fi
echo ""
if [[ $FAIL -gt 0 ]]; then
echo "Scope walk-up sync check failed: $FAIL error(s). One of validate.sh's detect_scope, validate-provenance.sh's find_plugin_root, new-agent.sh's find_package_root, or new-skill.sh's find_package_root has drifted from the others' \$HOME/.git/apm.yml walk-up semantics. Re-read new-agent.sh's usage comment (the canonical description of the intended behavior) and bring the disagreeing script back in line." >&2
exit 1
fi
echo "Scope walk-up sync check passed: all four walk-up implementations agree on every fixture."

View File

@@ -0,0 +1,141 @@
#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$REPO_ROOT/scripts/check-scope-walkup-sync.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
FIXTURES=()
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
trap cleanup EXIT
# --- 1. Exits 0 against this repo's own (fixed) scripts ---
echo ""
echo "--- exits 0 against this repo's real scripts ---"
if bash "$SCRIPT" "$REPO_ROOT" > /tmp/check-scope-walkup-sync-clean.out 2>&1; then
pass "exits 0 against this repo's real scope walk-up scripts"
else
fail "exited non-zero against this repo's real (already-fixed) scripts"
sed 's/^/ /' /tmp/check-scope-walkup-sync-clean.out
fi
# --- 2. Exits 0 as a no-op when the kyberforge skills aren't present ---
echo ""
echo "--- exits 0 (no-op) when the target scripts don't exist ---"
FIXTURE_EMPTY="$(mktemp -d)"
FIXTURES+=("$FIXTURE_EMPTY")
if bash "$SCRIPT" "$FIXTURE_EMPTY" > /dev/null 2>&1; then
pass "exits 0 as a no-op when agent-author/agent-audit/skill-author aren't present"
else
fail "exited non-zero when the kyberforge skills are simply absent"
fi
# --- 3. Exits 1 against a REPO_ROOT that doesn't exist ---
echo ""
echo "--- exits 1 when REPO_ROOT does not exist ---"
if bash "$SCRIPT" "/nonexistent/path/$(date +%s)-$$" > /dev/null 2>&1; then
fail "exited 0 for a nonexistent REPO_ROOT — expected exit 1"
else
pass "exits non-zero for a nonexistent REPO_ROOT"
fi
# --- 4. Regression guard: reintroducing the $HOME-collapse bug into
# validate.sh's detect_scope must make the check fail. Builds a minimal
# REPO_ROOT (just the four scripts, at their real relative paths) so this
# doesn't depend on — or risk mutating — the real repo tree.
make_minimal_repo_root() {
local dir
dir="$(mktemp -d)"
local na="$dir/plugins/kyberforge/skills/agent-author/scripts"
local ns="$dir/plugins/kyberforge/skills/skill-author/scripts"
local aa="$dir/plugins/kyberforge/skills/agent-audit/scripts"
mkdir -p "$na" "$ns" "$aa"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-author/scripts/new-agent.sh" "$na/"
cp "$REPO_ROOT/plugins/kyberforge/skills/skill-author/scripts/new-skill.sh" "$ns/"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate.sh" "$aa/"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh" "$aa/"
# agent-author's templates are needed by new-agent.sh at runtime.
cp -R "$REPO_ROOT/plugins/kyberforge/skills/agent-author/assets" "$dir/plugins/kyberforge/skills/agent-author/"
cp -R "$REPO_ROOT/plugins/kyberforge/skills/skill-author/assets" "$dir/plugins/kyberforge/skills/skill-author/"
# validate.sh needs field-inventory.md
mkdir -p "$dir/plugins/kyberforge/skills/agent-audit/references"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/references/field-inventory.md" \
"$dir/plugins/kyberforge/skills/agent-audit/references/"
echo "$dir"
}
echo ""
echo "--- exits 1 when validate.sh's detect_scope collapses back to the \$HOME-walk-up bug ---"
FIXTURE_BUG="$(make_minimal_repo_root)"
FIXTURES+=("$FIXTURE_BUG")
python3 - "$FIXTURE_BUG/plugins/kyberforge/skills/agent-audit/scripts/validate.sh" <<'PYTHON'
import re, sys
path = sys.argv[1]
with open(path) as f:
content = f.read()
# Revert to the pre-fix collapsed logic: both the $HOME-boundary case and the
# filesystem-root fallback return 'user', home unconditionally.
old = """def detect_scope(start_dir):
home = os.path.expanduser('~')
original_start = os.path.abspath(start_dir)"""
assert old in content, "detect_scope signature not found — validate.sh has changed shape"
buggy = '''def detect_scope(start_dir):
home = os.path.expanduser('~')
current = os.path.abspath(start_dir)
while True:
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current
if current == home:
return 'user', home
if os.path.exists(os.path.join(current, '.git')):
return 'project', current
parent = os.path.dirname(current)
if parent == current:
return 'user', home
current = parent
'''
# Replace the whole function body up to (but not including) the next
# top-level `agent_dir = ` assignment that calls it.
pattern = re.compile(r"def detect_scope\(start_dir\):\n.*?\n(?=agent_dir = )", re.DOTALL)
assert pattern.search(content), "could not isolate detect_scope's full body"
content = pattern.sub(buggy + "\n", content)
with open(path, 'w') as f:
f.write(content)
PYTHON
if bash "$SCRIPT" "$FIXTURE_BUG" > /tmp/check-scope-walkup-sync-buggy.out 2>&1; then
fail "exited 0 against a validate.sh reverted to the \$HOME-collapse bug — expected exit 1"
else
pass "exits non-zero when validate.sh's detect_scope regresses to the \$HOME-collapse bug"
fi
echo ""
echo "--- exits 1 when validate-provenance.sh's find_plugin_root loses its \$HOME boundary check ---"
FIXTURE_BUG2="$(make_minimal_repo_root)"
FIXTURES+=("$FIXTURE_BUG2")
python3 - "$FIXTURE_BUG2/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh" <<'PYTHON'
import re, sys
path = sys.argv[1]
with open(path) as f:
content = f.read()
# Drop the `if current == home: return None` line — reverts to the pre-fix
# behavior of never checking a $HOME boundary at all.
pattern = re.compile(r"\n *# \$HOME is a non-plugin-scope boundary.*?\n *if current == home:\n *return None\n", re.DOTALL)
assert pattern.search(content), "could not find the \\$HOME boundary check to remove"
content = pattern.sub("\n", content)
with open(path, 'w') as f:
f.write(content)
PYTHON
if bash "$SCRIPT" "$FIXTURE_BUG2" > /tmp/check-scope-walkup-sync-buggy2.out 2>&1; then
fail "exited 0 against a validate-provenance.sh with no \$HOME boundary check — expected exit 1"
else
pass "exits non-zero when validate-provenance.sh's find_plugin_root loses its \$HOME boundary check"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]