feat(kyberforge): restructure agent-audit for plugin-scope apm agents

Validates the new single-file .apm/agents/<name>.agent.md shape agent-author
now produces at plugin/APM scope: frontmatter allowlist (name/description/
model only, from a new apm-agent-allowlist entry in field-inventory.md),
no counterpart derivation, and Pair Consistency dropped from that scope's
report entirely (nothing to pair by design). Adds a plugin/APM-scope-only
SUGGESTION when an agent's description/body implies a tool restriction or
Claude-only behavior the vendor-neutral frontmatter can no longer express
(ADR-0016).

Scope detection in both validate.sh and validate-provenance.sh switches
from a flat plugin.json/.claude-plugin/plugin.json check to a walk-up for
the nearest ancestor apm.yml with a top-level type: field, skipping
type:-less marketplace-only manifests — full switch, no dual-mode fallback
to the old plugin.json signal. validate-provenance.sh's walk-up was fixed
to match validate.sh's (it still used the old plugin.json check, and its
counterpart-merge logic was rewritten to read a single file's source_keys
instead of merging a CC+Copilot pair, since plugin/APM scope has no
counterpart). Project/user scope validation is unchanged in both scripts.

Refs: #89
This commit is contained in:
2026-08-11 18:05:33 +00:00
parent 8cd5c79c0a
commit 675ba40238
6 changed files with 615 additions and 370 deletions

View File

@@ -26,8 +26,8 @@ metadata:
## Gotchas
- The unit of authoring in this project is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures.
- Plugin scope is detected by the presence of `plugin.json` or `.claude-plugin/plugin.json` in the directory tree — not by the file path pattern. Walk up both paths at each level, don't guess.
- The unit of authoring at project/user scope is always a pair (CC `.md` + Copilot `.agent.md`). A missing counterpart is a FAIL under the kyberforge project convention at those scopes — neither the CC nor the Copilot platform itself requires a counterpart file. Label such findings as project convention violations, not platform spec failures. **At plugin/APM scope there is no pair** — the unit of authoring is a single vendor-neutral `.apm/agents/<name>.agent.md` file, and Pair Consistency does not apply there at all (see below).
- Scope is detected by walking up from the agent file's directory: at each level, if `apm.yml` exists AND contains a top-level `type: instructions|skill|hybrid|prompts` line, that directory is an APM package root — plugin/APM scope. A `type:`-less `apm.yml` is marketplace-only (see `docs/research/docs/microsoft-apm/monorepo-and-repo-shapes.md`) — skip it and keep walking up. Otherwise, if `.git` is a directory at that level, stop there — project scope. If neither is found before the filesystem root, fall back to user scope at `$HOME`. `plugin.json`/`.claude-plugin/plugin.json` are no longer scope signals for this skill — a directory with only a `plugin.json` and no `apm.yml` falls through to project (or user) scope.
- `references/field-inventory.md` must exist for `validate.sh` to run. The script exits with an error if it is missing.
- Do not output findings while auditing — gather internally, surface in Step 3 report.
@@ -36,16 +36,21 @@ metadata:
```bash
bash scripts/validate.sh <path-to-agent-file>
bash scripts/validate-provenance.sh <path-to-agent-file>
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file>
scripts/vale-wrap.sh <path-to-cc-file> <path-to-copilot-file> # project/user scope
scripts/vale-wrap.sh <path-to-apm-agent-file> # plugin/APM scope — single file
```
The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
The script accepts either the CC file, the Copilot file, or (at plugin/APM scope) the single `.apm/agents/<name>.agent.md` file. It detects provider from extension and scope from the walk-up above, then runs the checks for that scope.
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". Run it against both files of the pair (not just the one passed in). `Kyberforge` applies to both files; `KyberforgeCopilot` applies to the `.agent.md` file only, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a Copilot description — there's nothing to flag in the CC file, so it isn't scoped there. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md` (`name`, `description`, `model` — nothing else). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists).
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
`validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside `name`/`description`/`model` (read the `apm-agent-allowlist` section of `references/field-inventory.md`).
## Step 2 — Qualitative checks
@@ -69,18 +74,29 @@ If a description finding is borderline, read `references/description-quality.md`
- Look for patterns like `# Optional. <long explanation>` or extensive inline guidance (more than 1–2 lines per field) that should be condensed or removed before shipping.
- This mirrors skill-audit's body-discipline check but applies to template documentation in the frontmatter — template guidance belongs in development; agent-ready files should have minimal comments.
**Pair consistency (cross-file):**
**Pair consistency (cross-file) — project/user scope only:**
- Both files exist — FAIL if counterpart is missing (kyberforge project convention; not a platform requirement from either CC or Copilot — label as such)
- The following checks are covered automatically by `validate.sh`; apply them manually only when the script cannot run: both system prompt bodies non-empty — FAIL if either is empty
- **Does not apply at plugin/APM scope** — there is only one file, by design; do not raise a Pair Consistency finding there under any circumstance.
**Unexpressable Claude-only behavior — plugin/APM scope only:**
- Read the description and body. If either implies a need the vendor-neutral frontmatter can no longer express — tool restriction, `isolation`, `memory`, or another Claude-only behavior that a hand-authored CC file could have declared — flag it as a SUGGESTION, never a FAIL. This is a known upstream schema limitation (APM's agent primitive has no per-target compile integrator, so `tools:`/`isolation`/etc. can't be emitted safely to both CC and Copilot — ADR-0016), not an authoring mistake. The finding exists to give the author visibility into the gap, not to imply the schema can be made to do something it can't.
- Example: a body that says "only use Read and Grep, never Edit" but the frontmatter has no `tools` field to enforce it — SUGGESTION, not FAIL.
## Step 3 — Report
Open with a coverage line:
Open with a coverage line. At project/user scope:
```text
Checked: structure · provider-safety · description · body · comment-discipline · pair-consistency · provenance
```
At plugin/APM scope, omit `pair-consistency` — it does not apply when there is no pair:
```text
Checked: structure · provider-safety · description · body · comment-discipline · provenance
```
Then output only dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each dimension. Omit clean dimensions entirely. `### Provenance` findings are sourced verbatim from `validate-provenance.sh` output — copy them without rephrasing.
For each finding:

View File

@@ -26,3 +26,7 @@ name description tools target model disable-model-invocation user-invocable mcp-
## copilot-only-fields
target disable-model-invocation user-invocable mcp-servers metadata
## apm-agent-allowlist
name description model

View File

@@ -5,8 +5,10 @@ usage() {
cat <<EOF
Usage: validate-provenance.sh <agent-file>
Validate that an agent pair's sources provenance chain is complete and internally consistent.
Operates at plugin scope only — exits 0 silently for project and user scope agents.
Validate that an agent's sources provenance chain is complete and internally consistent.
Operates at plugin/APM scope only (a single vendor-neutral .apm/agents/<name>.agent.md
inside a package with a type:-bearing apm.yml) — exits 0 silently for project and user
scope agents.
Arguments:
agent-file Path to either the Claude Code .md or Copilot .agent.md agent file.
@@ -47,23 +49,28 @@ agent_file = os.path.abspath(sys.argv[1])
fname = os.path.basename(agent_file)
agent_dir = os.path.dirname(agent_file)
# --- Detect provider ---
if fname.endswith('.agent.md'):
provider = 'copilot'
name_stem = fname[:-len('.agent.md')]
elif fname.endswith('.md'):
provider = 'claude-code'
name_stem = fname[:-len('.md')]
else:
# --- Sanity-check extension (single vendor-neutral .agent.md file at plugin/APM scope) ---
if not (fname.endswith('.agent.md') or fname.endswith('.md')):
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
sys.exit(2)
# --- Find plugin root ---
TYPE_RE = re.compile(r'^type:\s*(instructions|skill|hybrid|prompts)\b')
# --- Find package root: walk up for the nearest ancestor apm.yml that
# declares a top-level type: field. An apm.yml with no type: field is a
# marketplace-only manifest (see monorepo-and-repo-shapes.md) — skip it and
# keep walking. Stop at a .git boundary or the filesystem root: neither is
# plugin/APM scope, so this script has nothing to check there.
def find_plugin_root(start_dir):
current = os.path.abspath(start_dir)
while True:
if (os.path.isfile(os.path.join(current, 'plugin.json')) or os.path.isfile(os.path.join(current, '.claude-plugin', 'plugin.json'))):
return current
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml):
with open(apm_yml) as f:
if any(TYPE_RE.match(line) for line in f):
return current
if os.path.isdir(os.path.join(current, '.git')):
return None
parent = os.path.dirname(current)
if parent == current:
return None
@@ -73,12 +80,6 @@ plugin_root = find_plugin_root(agent_dir)
if plugin_root is None:
sys.exit(0)
# --- Derive counterpart ---
if provider == 'copilot':
counterpart = os.path.join(agent_dir, name_stem + '.md')
else:
counterpart = os.path.join(agent_dir, name_stem + '.agent.md')
sources_md_path = os.path.join(plugin_root, 'sources.md')
# --- Helpers ---
@@ -166,15 +167,9 @@ def get_source_keys_from_file(fpath):
fm, _ = parse_frontmatter(content)
return parse_source_keys(fm)
# Plugin/APM scope is a single vendor-neutral file — no counterpart to merge.
given_keys = get_source_keys_from_file(agent_file)
counterpart_keys = get_source_keys_from_file(counterpart)
# Deduplicated union, preserving order
seen = set()
all_source_keys = []
for k in given_keys + counterpart_keys:
if k not in seen:
seen.add(k)
all_source_keys.append(k)
all_source_keys = given_keys
sources_md_exists = os.path.isfile(sources_md_path)
@@ -212,8 +207,8 @@ for line in sources_content.splitlines():
)
break
# --- Check 2: source_keys in agent files → slug exists in sources.md ---
for fpath, keys in [(agent_file, given_keys), (counterpart, counterpart_keys)]:
# --- Check 2: source_keys in the agent file → slug exists in sources.md ---
for fpath, keys in [(agent_file, given_keys)]:
if not keys:
continue
rel = os.path.relpath(fpath, plugin_root)

View File

@@ -5,10 +5,15 @@ usage() {
cat <<EOF
Usage: validate.sh <agent-file>
Validate a Claude Code or Copilot agent file pair against the agent definition spec.
Validate an agent definition file against the agent definition spec.
At plugin/APM scope, <agent-file> is a single vendor-neutral
.apm/agents/<name>.agent.md file (frontmatter allowlist: name, description,
model — no counterpart file). At project or user scope, <agent-file> is
either half of a Claude Code .md / Copilot .agent.md pair.
Arguments:
agent-file Path to either the Claude Code .md or Copilot .agent.md agent file.
agent-file Path to the agent file (or either half of a project/user-scope pair).
Exit codes:
0 All checks passed (may include SUGGESTIONs)
@@ -74,6 +79,7 @@ def parse_section_tokens(content, section_name):
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
plugin_ignored_fields = parse_section_tokens(inv_content, 'plugin-silently-ignored-fields')
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
# Tools the runtime withholds from subagents regardless of the tools field
SUBAGENT_UNAVAILABLE_TOOLS = {
@@ -83,48 +89,7 @@ SUBAGENT_UNAVAILABLE_TOOLS = {
# Copilot body length limit (chars) — content beyond this is silently truncated
COPILOT_BODY_LIMIT = 30000
# --- Detect scope ---
def detect_scope(start_dir):
current = os.path.abspath(start_dir)
while True:
if (os.path.isfile(os.path.join(current, 'plugin.json')) or
os.path.isfile(os.path.join(current, '.claude-plugin', 'plugin.json'))):
return 'plugin', current
if os.path.isdir(os.path.join(current, '.git')):
return 'project', current
parent = os.path.dirname(current)
if parent == current:
return 'user', os.path.expanduser('~')
current = parent
agent_dir = os.path.dirname(agent_file)
scope, scope_root = detect_scope(agent_dir)
# --- Derive counterpart path ---
if scope == 'plugin':
if provider == 'copilot':
counterpart = os.path.join(agent_dir, name_stem + '.md')
counterpart_provider = 'claude-code'
else:
counterpart = os.path.join(agent_dir, name_stem + '.agent.md')
counterpart_provider = 'copilot'
elif scope == 'project':
if provider == 'claude-code':
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
else: # user
home = os.path.expanduser('~')
if provider == 'claude-code':
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
# --- Helpers ---
# --- Helpers (shared by every scope) ---
failed = False
suggestions = []
@@ -167,6 +132,113 @@ def is_copilot_cloud_ide(fpath):
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
# --- Detect scope ---
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
# one of the four package content types. `[\'"]?` tolerates a quoted value; the
# pattern doesn't anchor the line end, so trailing whitespace/comments don't matter.
APM_TYPE_RE = re.compile(r"^type:\s*['\"]?(instructions|skill|hybrid|prompts)\b")
def find_apm_package_root(apm_yml_path):
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
manifest, not a type:-less marketplace-only apm.yml)."""
with open(apm_yml_path) as f:
for line in f:
if APM_TYPE_RE.match(line):
return True
return False
def detect_scope(start_dir):
current = os.path.abspath(start_dir)
while True:
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current
if os.path.isdir(os.path.join(current, '.git')):
return 'project', current
parent = os.path.dirname(current)
if parent == current:
return 'user', os.path.expanduser('~')
current = parent
agent_dir = os.path.dirname(agent_file)
scope, scope_root = detect_scope(agent_dir)
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
def check_apm_agent_file(fpath, allowlist, stem):
local_fname = os.path.basename(fpath)
with open(fpath) as f:
content = f.read()
fm, body = parse_frontmatter(content)
if fm is None:
fail(f"no valid YAML frontmatter (---...---) — {local_fname}")
return
# Allowlist: only name/description/model may appear — no tools, no
# Claude-only or Copilot-only fields. apm compile verbatim-copies
# frontmatter to every target, so anything else is unsafe on at least
# one harness (ADR-0016).
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key not in allowlist:
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
f"({', '.join(sorted(allowlist))}) — {local_fname}")
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
name_val = extract_field(fm, 'name')
if not name_val:
fail(f"name field is missing or empty — {local_fname}")
else:
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
if name_val != stem:
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
# description — required, non-empty, no placeholder
desc_val = extract_field(fm, 'description')
if not desc_val:
fail(f"description field is missing or empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(desc_val):
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
# body — required, non-empty, no placeholder; same Copilot truncation risk
# applies since this file compiles verbatim into a real Copilot file downstream.
if not body.strip():
fail(f"system prompt body is empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(body):
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
if len(body) > COPILOT_BODY_LIMIT:
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
f"content beyond the limit is silently truncated by the Copilot runtime "
f"once apm compile emits it downstream — {local_fname}")
if scope == 'plugin':
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
for s in suggestions:
print(f"SUGGESTION {s}")
sys.exit(1 if failed else 0)
# --- Project/user scope: unchanged CC/Copilot pair validation ---
# --- Derive counterpart path ---
if scope == 'project':
if provider == 'claude-code':
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
else: # user
home = os.path.expanduser('~')
if provider == 'claude-code':
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
def check_file(fpath, file_provider, is_plugin_scope):
local_fname = os.path.basename(fpath)
with open(fpath) as f:

View File

@@ -8,26 +8,24 @@ setup() {
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate-provenance.sh"
TMPDIR="$(mktemp -d)"
# Helper: create a plugin root with plugin.json and an agents/ directory
make_plugin() {
# Helper: create an APM package root at <root> (apm.yml with a top-level
# type: line — a real package manifest, not marketplace-only) plus a
# single vendor-neutral agent file at <root>/.apm/agents/<name>.agent.md.
make_package() {
local root="$1"
mkdir -p "$root/agents"
echo '{"name":"test-plugin","version":"0.1.0"}' > "$root/plugin.json"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
}
# Helper: create a clean agent pair (no source_keys)
make_clean_pair() {
# Helper: create a clean agent file (no source_keys)
make_clean_agent() {
local root="$1"
local name="${2:-my-agent}"
cat > "$root/agents/${name}.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent.
EOF
cat > "$root/agents/${name}.agent.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
@@ -37,12 +35,12 @@ You are a test agent.
EOF
}
# Helper: create a CC agent file with source_keys
make_cc_with_source_keys() {
# Helper: create an agent file with source_keys
make_agent_with_source_keys() {
local root="$1"
local name="${2:-my-agent}"
local slug="${3:-my-source}"
cat > "$root/agents/${name}.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
@@ -50,37 +48,6 @@ source_keys:
- ${slug}
---
You are a test agent.
EOF
}
# Helper: create a Copilot agent file with source_keys
make_copilot_with_source_keys() {
local root="$1"
local name="${2:-my-agent}"
local slug="${3:-my-source}"
cat > "$root/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
source_keys:
- ${slug}
---
You are a test agent.
EOF
}
# Helper: create a minimal Copilot file without source_keys
make_copilot_clean() {
local root="$1"
local name="${2:-my-agent}"
cat > "$root/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent.
EOF
}
@@ -89,7 +56,7 @@ EOF
make_sources_md() {
local root="$1"
local slug="${2:-my-source}"
local contrib="${3:-agents/my-agent.md, agents/my-agent.agent.md}"
local contrib="${3:-.apm/agents/my-agent.agent.md}"
local research="${4:-(none)}"
cat > "$root/sources.md" <<EOF
# Sources
@@ -120,12 +87,56 @@ teardown() {
}
# ---------------------------------------------------------------------------
# Non-plugin scope → exit 0 silently
# Non-plugin/APM scope → exit 0 silently
# ---------------------------------------------------------------------------
@test "non-plugin scope: no plugin.json in tree → exit 0, no output" {
local dir="$TMPDIR/no-plugin"
@test "non-plugin scope: no apm.yml in tree → exit 0, no output" {
local dir="$TMPDIR/no-package"
mkdir -p "$dir/.apm/agents"
cat > "$dir/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: apm.yml present but type:-less (marketplace-only) → exit 0, no output" {
local dir="$TMPDIR/marketplace-only"
mkdir -p "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: root-marketplace
marketplace:
owner: someone
packages:
- ./packages/plugin-a
EOF
cat > "$dir/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: bare plugin.json (no apm.yml) is no longer a scope signal → exit 0, no output" {
local dir="$TMPDIR/old-plugin-json-only"
mkdir -p "$dir/agents"
echo '{"name":"test-plugin","version":"0.1.0"}' > "$dir/plugin.json"
cat > "$dir/agents/my-agent.md" <<EOF
---
name: my-agent
@@ -141,15 +152,70 @@ EOF
assert_output ""
}
@test "non-plugin scope: walk-up stops at .git boundary before reaching an ancestor apm.yml" {
local dir="$TMPDIR/repo"
mkdir -p "$dir/.git" "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
mkdir -p "$dir/sub/.apm/agents"
cat > "$dir/sub/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
# sub/ has no .git and no apm.yml of its own; the real package apm.yml
# lives at $dir, but $dir/.git means the walk from sub/ should stop at
# sub/ itself only if sub/ had a .git — here .git is at $dir, ABOVE
# sub/, so the walk from sub/ reaches $dir/apm.yml before any .git.
# This test instead verifies the walk finds that package root correctly
# (a positive case) — see the dedicated .git-stops-first test below for
# the negative case.
run bash "$SCRIPT" "$dir/sub/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "non-plugin scope: .git between the agent file and an ancestor apm.yml stops the walk first" {
local dir="$TMPDIR/repo2"
mkdir -p "$dir/.apm/agents"
cat > "$dir/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
mkdir -p "$dir/sub/.git" "$dir/sub/.apm/agents"
cat > "$dir/sub/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run bash "$SCRIPT" "$dir/sub/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
# ---------------------------------------------------------------------------
# Early exit: no sources.md, no source_keys → exit 0, no output
# ---------------------------------------------------------------------------
@test "clean pass: no sources.md and no source_keys → exit 0, no output" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@@ -158,22 +224,11 @@ EOF
# Check 0: source_keys present but sources.md absent → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: source_keys in CC file but sources.md absent" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_clean "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: source_keys in Copilot file but sources.md absent" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
make_copilot_with_source_keys "$root"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
@test "FAIL: source_keys in agent file but sources.md absent" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@@ -183,10 +238,9 @@ EOF
# ---------------------------------------------------------------------------
@test "FAIL: FILL IN: placeholder in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -194,23 +248,22 @@ EOF
- **URL:** FILL IN: add url
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FILL IN: inside backticks in sources.md does not fail" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root"
echo "Use \`FILL IN: value\` as example." >> "$root/sources.md"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}
@@ -218,62 +271,47 @@ EOF
# Check 2: source_keys slug missing from sources.md → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: source_keys slug in CC file not present as H2 in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root" "my-agent" "my-source"
make_copilot_clean "$root"
@test "FAIL: source_keys slug in agent file not present as H2 in sources.md" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root" "my-agent" "my-source"
make_sources_md "$root" "different-source" "(none)" "(none)"
run bash "$SCRIPT" "$root/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: source_keys slug in Copilot file not present as H2 in sources.md" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_clean_pair "$root"
make_copilot_with_source_keys "$root" "my-agent" "my-source"
make_sources_md "$root" "different-source" "(none)" "(none)"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Check 4: Contributing file path doesn't exist → FAIL
# Check 3: Contributing file path doesn't exist → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Contributing file listed in sources.md does not exist" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
make_sources_md "$root" "my-source" "agents/nonexistent.md"
run bash "$SCRIPT" "$root/agents/my-agent.md"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root" "my-source" ".apm/agents/nonexistent.agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "pass: (none) in Contributing files is skipped" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root" "my-source" "(none — not used directly)"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}
# ---------------------------------------------------------------------------
# Check 6: Research doc field missing or placeholder → FAIL
# Check 5: Research doc field missing or placeholder → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Research doc field missing from sources.md entry" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -281,19 +319,18 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
@test "FAIL: Research doc field is FILL IN: placeholder" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
cat > "$root/sources.md" <<EOF
# Sources
@@ -301,24 +338,24 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md, agents/my-agent.agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** FILL IN: path to research doc
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Check 5: Bidirectional — contributing file missing slug in source_keys → FAIL
# Check 4: Bidirectional — contributing file missing slug in source_keys → FAIL
# ---------------------------------------------------------------------------
@test "FAIL: Contributing file exists but does not list parent slug in source_keys" {
local root="$TMPDIR/plugin"
make_plugin "$root"
# CC file has source_keys: other-source (not my-source)
cat > "$root/agents/my-agent.md" <<EOF
local root="$TMPDIR/package"
make_package "$root"
# agent file has source_keys: other-source (not my-source)
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -328,8 +365,8 @@ source_keys:
You are a test agent.
EOF
make_copilot_clean "$root"
# sources.md says my-agent.md contributed to my-source, but my-agent.md doesn't list my-source
# sources.md says my-agent.agent.md contributed to my-source, but
# my-agent.agent.md doesn't list my-source
cat > "$root/sources.md" <<EOF
# Sources
@@ -337,7 +374,7 @@ EOF
- **URL:** https://example.com/other-source
- **Description:** A test source.
- **Contributing files:** agents/my-agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
@@ -345,39 +382,24 @@ EOF
- **URL:** https://example.com/my-source
- **Description:** Another source.
- **Contributing files:** agents/my-agent.md
- **Contributing files:** .apm/agents/my-agent.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Entry via Copilot file path
# ---------------------------------------------------------------------------
@test "accepts Copilot file path as entry point" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
make_sources_md "$root"
run bash "$SCRIPT" "$root/agents/my-agent.agent.md"
assert_success
}
# ---------------------------------------------------------------------------
# Clean full pass
# ---------------------------------------------------------------------------
@test "clean full pass: all checks satisfied via CC file" {
local root="$TMPDIR/plugin"
make_plugin "$root"
make_cc_with_source_keys "$root"
make_copilot_with_source_keys "$root"
@test "clean full pass: all checks satisfied" {
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root"
run bash "$SCRIPT" "$root/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
}

View File

@@ -8,24 +8,27 @@ setup() {
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate.sh"
TMPDIR="$(mktemp -d)"
# Helper: create a plugin-scope pair in <dir> with given <name>
make_plugin_pair() {
local dir="$1"
# Helper: create an APM package root at <root> (apm.yml with a top-level
# type: line, marking it a real package manifest — not marketplace-only)
# plus a single vendor-neutral agent file at
# <root>/.apm/agents/<name>.agent.md. <extra_frontmatter>, if given, is
# inserted as additional raw frontmatter lines (used to inject fields
# under test).
make_apm_agent() {
local root="$1"
local name="$2"
mkdir -p "$dir"
echo '{}' > "$dir/plugin.json"
cat > "$dir/${name}.md" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
local extra_frontmatter="${3:-}"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$dir/${name}.agent.md" <<EOF
cat > "$root/.apm/agents/${name}.agent.md" <<EOF
---
name: ${name}
description: A valid agent description.
${extra_frontmatter}
---
You are a test agent. When invoked, do the thing.
@@ -38,17 +41,9 @@ teardown() {
}
# ---------------------------------------------------------------------------
# Passing cases
# Passing cases — project/user scope (unchanged)
# ---------------------------------------------------------------------------
@test "passes on a clean plugin-scope pair (CC file as input)" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
run bash "$SCRIPT" "$dir/my-agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "passes on a clean project-scope pair (CC file as input)" {
local root="$TMPDIR/project"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
@@ -79,40 +74,261 @@ EOF
assert_output --partial "Usage:"
}
# ---------------------------------------------------------------------------
# Failing cases
# ---------------------------------------------------------------------------
@test "fails when Copilot counterpart is missing" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
rm "$dir/my-agent.agent.md"
run bash "$SCRIPT" "$dir/my-agent.md"
@test "fails when no arguments are given" {
run bash "$SCRIPT"
assert_failure
assert_output --partial "counterpart"
}
@test "fails when CC-only field 'maxTurns' is in Copilot file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
maxTurns: 10
---
# ---------------------------------------------------------------------------
# Passing cases — plugin/APM scope
# ---------------------------------------------------------------------------
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
@test "passes on a clean plugin/APM-scope agent file (name/description only)" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "passes on a clean plugin/APM-scope agent file with optional model field" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "model: claude-opus-4"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
# ---------------------------------------------------------------------------
# Failing cases — plugin/APM scope: allowlist violations
# ---------------------------------------------------------------------------
@test "fails when 'tools' field is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "tools: Read Edit"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "tools"
}
@test "fails when a Claude-only field ('maxTurns') is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "maxTurns: 10"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "maxTurns"
}
@test "fails when plugin-silently-ignored field 'hooks' is in plugin-scope CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
@test "fails when a Copilot-only field ('target') is present in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "target: cli"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "target"
}
@test "fails when 'hooks' is present in a plugin/APM-scope agent file (outside allowlist)" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "hooks: {}"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "hooks"
}
# ---------------------------------------------------------------------------
# Failing cases — plugin/APM scope: structural checks
# ---------------------------------------------------------------------------
@test "fails when name is not kebab-case in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "kebab"
}
@test "fails when name does not match filename stem in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: wrong-name
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
assert_output --partial "does not match filename stem"
}
@test "fails when 'name' field is missing from a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
@test "fails when 'description' field is missing from a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
@test "fails when body contains unfilled FILL IN: placeholder in a plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
FILL IN: replace this with your system prompt.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
}
# ---------------------------------------------------------------------------
# Plugin/APM scope: no pair, no counterpart concept
# ---------------------------------------------------------------------------
@test "never raises a 'counterpart' FAIL on a clean plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "counterpart"
}
@test "never raises a 'counterpart' FAIL on a failing plugin/APM-scope agent file" {
local root="$TMPDIR/pkg"
make_apm_agent "$root" "my-agent" "tools: Read"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_failure
refute_output --partial "counterpart"
}
# ---------------------------------------------------------------------------
# Scope-detection walk-up
# ---------------------------------------------------------------------------
@test "walk-up skips a type:-less apm.yml and finds a real package root further up" {
local root="$TMPDIR/case"
mkdir -p "$root/.apm/agents/nested/deeper"
cat > "$root/apm.yml" <<EOF
name: real-package
version: 1.0.0
type: skill
EOF
# Closer to the agent file than the real package root, but has no type:
# line — marketplace-only per monorepo-and-repo-shapes.md, must be skipped.
cat > "$root/.apm/agents/nested/apm.yml" <<EOF
name: not-a-package-manifest
version: 1.0.0
EOF
cat > "$root/.apm/agents/nested/deeper/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/nested/deeper/my-agent.agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "type:-less apm.yml is not treated as plugin scope — falls through to project scope" {
local root="$TMPDIR/proj-marketplace"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/apm.yml" <<EOF
name: marketplace-root
version: 1.0.0
marketplace:
packages: []
EOF
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
}
@test "a bare plugin.json with no apm.yml is no longer plugin scope — falls through to project scope" {
local root="$TMPDIR/proj-legacy-plugin-json"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
echo '{}' > "$root/plugin.json"
# 'hooks' is plugin-silently-ignored only at (old) plugin scope; at
# project scope it's a legitimate CC field. If this directory were
# mis-detected as plugin scope (old plugin.json-based logic), this would
# FAIL with a plugin-silently-ignored-fields finding on 'hooks'.
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -124,95 +340,15 @@ hooks:
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
assert_output --partial "hooks"
}
@test "fails when CC file name is not kebab-case" {
local dir="$TMPDIR/agents"
mkdir -p "$dir"
echo '{}' > "$dir/plugin.json"
cat > "$dir/my-agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$dir/my-agent.agent.md" <<EOF
---
name: MyAgent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
assert_output --partial "kebab"
}
@test "fails when 'name' field is missing from CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
---
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when 'description' field is missing from CC file" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
---
name: my-agent
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when body contains unfilled FILL IN: placeholder" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.md" <<EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
FILL IN: replace this with your system prompt.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when frontmatter name does not match filename stem" {
local dir="$TMPDIR/agents"
make_plugin_pair "$dir" "my-agent"
cat > "$dir/my-agent.agent.md" <<EOF
---
name: wrong-name
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$dir/my-agent.md"
assert_failure
}
@test "fails when no arguments are given" {
run bash "$SCRIPT"
assert_failure
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "hooks"
}