2 Commits

Author SHA1 Message Date
eada85db99 docs(kyberforge): document source_keys as 4th apm-agent-allowlist field
field-inventory.md's apm-agent-allowlist and validate.sh's runtime
check already included source_keys as a 4th allowed field, and the
apm-agent.md template already instructed authors to add it for
provenance tracking — but SKILL.md (x2), README.md, ADR-0016, and
deployment-modes.md still described the allowlist as name/description/
model, "nothing else". The template itself even contradicted its own
source_keys guidance with a header claiming "ONLY the three fields
below — full stop" directly above it.

Updates all six locations to document source_keys as the intentional
4th field, resolving the contradiction.
2026-08-12 11:35:33 +00:00
044b2d3f08 fix(kyberforge): fix HOME/git scope-walkup false-FAILs in agent-audit
validate.sh's detect_scope() and validate-provenance.sh's
find_plugin_root() disagreed with new-agent.sh's already-correct,
documented walk-up semantics on three points, each causing validate.sh
to false-FAIL a legitimately-scaffolded project-scope agent pair:

- a marker-less directory walked up into $HOME (no .git/apm.yml of its
  own) was classified as user scope instead of project scope
- the .git-boundary branch returned the walked-to .git location instead
  of the conventional scope root, breaking any <root> that is a
  subdirectory of a larger git-tracked tree (monorepo package dirs)
- the new conventional-root arithmetic introduced to fix the above two
  cases had no guard against non-conventional/hand-placed file paths,
  which could point it at the wrong ancestor

Also adds scripts/check-scope-walkup-sync.sh, a behavioral drift-guard
(per ADR-0014's no-cross-skill-path precedent) that cross-checks the
four independently hand-ported walk-up implementations (validate.sh,
validate-provenance.sh, new-agent.sh, new-skill.sh) against real
fixture scaffolds, wired into .pre-commit-config.yaml at pre-push so
future drift between the ports is caught automatically.

Verified via bash tests/run-tests.sh (13/13) and targeted before/after
reproduction of each bug this closes.
2026-08-12 11:35:23 +00:00
14 changed files with 767 additions and 16 deletions

View File

@@ -70,6 +70,15 @@ repos:
pass_filenames: false pass_filenames: false
always_run: true always_run: true
- id: check-scope-walkup-sync
name: Check scope walk-up implementations agree
description: Behaviorally cross-check validate.sh, validate-provenance.sh, new-agent.sh, and new-skill.sh's independent $HOME/.git/apm.yml walk-up ports against each other
entry: bash scripts/check-scope-walkup-sync.sh
language: system
stages: [pre-push]
pass_filenames: false
always_run: true
- id: check-release-needed - id: check-release-needed
name: Check a release tag covers .pre-commit-hooks.yaml's paths name: Check a release tag covers .pre-commit-hooks.yaml's paths
description: On push to main only, fail if files exposed via .pre-commit-hooks.yaml changed since the last tag description: On push to main only, fail if files exposed via .pre-commit-hooks.yaml changed since the last tag

View File

@@ -86,5 +86,7 @@ place.
lists from `references/field-inventory.md` rather than hardcoding them, with a `source_keys` lists from `references/field-inventory.md` rather than hardcoding them, with a `source_keys`
provenance chain — survives and is reused. Only the *content shape* changes for plugin scope: provenance chain — survives and is reused. Only the *content shape* changes for plugin scope:
`field-inventory.md` shifts from two side-by-side CC-only/Copilot-only blocklists to one `field-inventory.md` shifts from two side-by-side CC-only/Copilot-only blocklists to one
vendor-neutral allowlist (`name`/`description`/`model`) for plugin-scope agents, while vendor-neutral allowlist (`name`/`description`/`model`/`source_keys` — the last for provenance
tracking, validated separately by `validate-provenance.sh` against `sources.md`, not a
provider-specific field) for plugin-scope agents, while
continuing to serve its original two-blocklist role for project/user-scope validation. continuing to serve its original two-blocklist role for project/user-scope validation.

View File

@@ -7,7 +7,9 @@ plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
vendor-neutral allowlist (`name`, `description`, `model` — see ADR-0016), since `apm compile` vendor-neutral allowlist (`name`, `description`, `model`, `source_keys` — the last for
provenance tracking, checked separately by `validate-provenance.sh` against `sources.md`; see
ADR-0016), since `apm compile`
copies frontmatter verbatim to both harnesses and an unsafe field can't be silently dropped for copies frontmatter verbatim to both harnesses and an unsafe field can't be silently dropped for
just one of them. just one of them.

View File

@@ -44,13 +44,13 @@ The script accepts either the CC file, the Copilot file, or (at plugin/APM scope
At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`. At **project/user scope** it derives the counterpart and runs the existing pair-based checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, body length, or subagent-unavailable tools → `### Provider safety`. A missing counterpart file → `### Pair consistency`.
At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md` (`name`, `description`, `model` — nothing else). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design. At **plugin/APM scope** there is no counterpart — the script instead checks the single file's frontmatter against the `apm-agent-allowlist` in `references/field-inventory.md` (`name`, `description`, `model`, `source_keys` — nothing else; `source_keys` is provenance metadata, not a provider-specific field, and is validated separately by `validate-provenance.sh` against `sources.md`). Findings about missing fields, bad name format, name/filename-stem mismatch, empty body, or missing frontmatter → `### Structure`, same as project/user scope. Findings about any field outside the allowlist (e.g. `tools`, or any Claude-only/Copilot-only field carried over from a hand-edit) and body length → `### Provider safety` — but the dimension's meaning shifts here: it is no longer a CC-vs-Copilot field-leakage check, it's a vendor-neutral-field-allowlist check, since `apm compile` verbatim-copies this file's frontmatter to every target and there is no per-target integrator to reconcile a CC-only or Copilot-only field (ADR-0016). `### Pair consistency` never applies at this scope — the script never emits a missing-counterpart FAIL here, because there is nothing to pair by design.
`vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered. `vale-wrap.sh` ships inside this skill's own `scripts/` — resolve it relative to this skill's directory the same way `scripts/validate.sh` is resolved above, so the invocation works whether this skill is running from this repo or from an installed plugin cache. Pass no `--config`: handed none, the wrapper loads its own sibling `assets/vale/.vale.ini`, located from the script's path rather than from the cwd. Adding an explicit relative `--config` breaks exactly the case the self-location covers — a resolved script path plus an unresolved config path yields `E100 Runtime error ... does not exist`, exit 2, which the fallback below then misreads as "vale unavailable". At project/user scope, run it against both files of the pair (not just the one passed in); at plugin/APM scope, run it against the single file. `Kyberforge` applies to all of these files via the `**/agents/*.md` glob; `KyberforgeCopilot` applies to any `*.agent.md` file — including the plugin/APM-scope file, which already has that extension — via the `**/*.agent.md` glob, since its one rule (`Use proactively`) flags CC-specific phrasing that's meaningless in a vendor-neutral or Copilot description. Every Vale alert is a `FAIL` — all rules are graded `error` — so report each one in the `### Description` / `### Body` dimensions citing its rule ID (e.g. `KyberforgeCopilot.ProactivePhrase`). Skip and fall back to Step 2 judgment if the `vale` binary is unavailable. If Vale reports `0 files` scanned, treat the pass as NOT RUN — not as clean — and fall back to full Step 2 judgment for the dimensions it would have covered.
`validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix. `validate-provenance.sh` operates at plugin/APM scope only — it walks up from the agent file's directory the same way `validate.sh` does (nearest ancestor `apm.yml` with a top-level `type:` field; skip a `type:`-less marketplace-only `apm.yml`; stop at `.git` or the filesystem root) and exits 0 silently if that walk doesn't land on a package root, or when no provenance data exists. When it does apply, it validates the chain between the single file's own `source_keys` and the package-scoped `sources.md` (package root — see ADR-0010). Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix.
If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside `name`/`description`/`model` (read the `apm-agent-allowlist` section of `references/field-inventory.md`). If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually. At project/user scope: counterpart file exists, required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). At plugin/APM scope: required fields present (`name`, `description`, non-empty body), `name` is kebab-case and matches the filename stem, no `FILL IN:` placeholders, no frontmatter field outside `name`/`description`/`model`/`source_keys` (read the `apm-agent-allowlist` section of `references/field-inventory.md`; `source_keys` carries provenance metadata, checked separately by `validate-provenance.sh` against `sources.md`).
## Step 2 — Qualitative checks ## Step 2 — Qualitative checks

View File

@@ -64,9 +64,11 @@ TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:
# --- Find package root: walk up for the nearest ancestor apm.yml that # --- Find package root: walk up for the nearest ancestor apm.yml that
# declares a top-level type: field. An apm.yml with no type: field is a # declares a top-level type: field. An apm.yml with no type: field is a
# marketplace-only manifest (see monorepo-and-repo-shapes.md) — skip it and # marketplace-only manifest (see monorepo-and-repo-shapes.md) — skip it and
# keep walking. Stop at a .git boundary or the filesystem root: neither is # keep walking. Stop at a $HOME boundary, a .git boundary, or the filesystem
# plugin/APM scope, so this script has nothing to check there. # root: none of these is plugin/APM scope, so this script has nothing to
# check there.
def find_plugin_root(start_dir): def find_plugin_root(start_dir):
home = os.path.expanduser('~')
current = os.path.abspath(start_dir) current = os.path.abspath(start_dir)
while True: while True:
apm_yml = os.path.join(current, 'apm.yml') apm_yml = os.path.join(current, 'apm.yml')
@@ -74,6 +76,15 @@ def find_plugin_root(start_dir):
with open(apm_yml) as f: with open(apm_yml) as f:
if any(TYPE_RE.match(line) for line in f): if any(TYPE_RE.match(line) for line in f):
return current return current
# $HOME is a non-plugin-scope boundary — checked before the .git test
# below (mirrors validate.sh's detect_scope ordering), so a
# dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) can't shadow
# this check by being its own .git repo. Without this, the walk could
# continue past $HOME toward the filesystem root looking for a
# type-bearing apm.yml, misclassifying a user/project-scope file as
# plugin scope in rare ancestor layouts.
if current == home:
return None
# .git is a directory in a normal checkout but a file (`gitdir: ...`) # .git is a directory in a normal checkout but a file (`gitdir: ...`)
# in a git worktree — exists() covers both. # in a git worktree — exists() covers both.
if os.path.exists(os.path.join(current, '.git')): if os.path.exists(os.path.join(current, '.git')):

View File

@@ -152,23 +152,68 @@ def find_apm_package_root(apm_yml_path):
def detect_scope(start_dir): def detect_scope(start_dir):
home = os.path.expanduser('~') home = os.path.expanduser('~')
current = os.path.abspath(start_dir) original_start = os.path.abspath(start_dir)
# Agent files conventionally live exactly two path segments below their
# scope root — <root>/.claude/agents, <root>/.github/agents,
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
# recovers the same root new-agent.sh would have been invoked with to
# produce this exact file, independent of how far the walk below has to
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
# `root` vs `current` distinction even though validate.sh is handed a
# file's directory, not the scope root itself.
#
# That arithmetic is only trustworthy when the path actually has this
# shape: parent directory literally named "agents", grandparent one of
# the four known scope-dir names. A hand-placed or otherwise
# non-conventional agent file (never produced by new-agent.sh) has no
# such guarantee — blindly trusting two-segments-up there could point at
# an unrelated ancestor. conventional_shape gates every use of
# conventional_root below; when it's false, the walked-to `current`
# directory is used instead, the same fallback this function used before
# conventional_root existed.
scope_dir_name = os.path.basename(os.path.dirname(original_start))
conventional_shape = (
os.path.basename(original_start) == 'agents'
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
)
conventional_root = os.path.dirname(os.path.dirname(original_start))
current = original_start
while True: while True:
apm_yml = os.path.join(current, 'apm.yml') apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml): if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current return 'plugin', current
# $HOME is the user-scope boundary — checked before the .git test # $HOME is the user-scope boundary — checked before the .git test
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) # below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
# can't shadow user scope by being its own .git repo. # can't shadow user scope by being its own .git repo. 'user' scope
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
# conventional two-segments-below-root depth (i.e. $HOME IS that
# root, matching the real ~/.claude/agents or ~/.copilot/agents
# shape). Any other walk-up into $HOME — a marker-less directory
# nested deeper than that convention — resolves to project scope
# instead: a stray directory under $HOME can't be silently
# redirected into the shared global ~/.claude or ~/.copilot agent
# directories.
if current == home: if current == home:
return 'user', home if original_start == home or (conventional_shape and conventional_root == home):
return 'user', home
return 'project', conventional_root if conventional_shape else current
# .git is a directory in a normal checkout but a file (`gitdir: ...`) # .git is a directory in a normal checkout but a file (`gitdir: ...`)
# in a git worktree — exists() covers both. # in a git worktree — exists() covers both. Returns conventional_root,
# not current: new-agent.sh's project-scope file placement always
# uses its `$ROOT` argument directly, never the walked-up `.git`
# location, so a <root> one or more levels below the repo's .git
# (a subdirectory of a larger git-tracked tree — explicitly a
# supported case per new-agent.sh's usage text) must resolve to the
# same root new-agent.sh actually wrote to, not to the .git dir —
# unless the path lacks the conventional shape, in which case that
# arithmetic isn't trustworthy and current is used instead.
if os.path.exists(os.path.join(current, '.git')): if os.path.exists(os.path.join(current, '.git')):
return 'project', current return 'project', conventional_root if conventional_shape else current
parent = os.path.dirname(current) parent = os.path.dirname(current)
if parent == current: if parent == current:
return 'user', home return 'project', conventional_root if conventional_shape else current
current = parent current = parent
agent_dir = os.path.dirname(agent_file) agent_dir = os.path.dirname(agent_file)

View File

@@ -183,6 +183,36 @@ EOF
assert_output --partial "FAIL" assert_output --partial "FAIL"
} }
@test "non-plugin scope: \$HOME boundary stops the walk before reaching an ancestor apm.yml above \$HOME" {
# A type-bearing apm.yml sits ABOVE the fake $HOME — if find_plugin_root
# didn't stop at $HOME, it would walk past it and misclassify this
# user/project-scope file as plugin scope, which would then FAIL on
# Check 0 (source_keys declared but sources.md absent) since sources.md
# doesn't exist at that ancestor apm.yml's location either.
local dir="$TMPDIR/anc"
mkdir -p "$dir"
cat > "$dir/apm.yml" <<EOF
name: outer-package
version: 0.1.0
type: skill
EOF
local fake_home="$dir/fakehome"
mkdir -p "$fake_home/.apm/agents"
cat > "$fake_home/.apm/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- my-source
---
You are a test agent.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$fake_home/.apm/agents/my-agent.agent.md"
assert_success
assert_output ""
}
@test "non-plugin scope: .git between the agent file and an ancestor apm.yml stops the walk first" { @test "non-plugin scope: .git between the agent file and an ancestor apm.yml stops the walk first" {
local dir="$TMPDIR/repo2" local dir="$TMPDIR/repo2"
mkdir -p "$dir/.apm/agents" mkdir -p "$dir/.apm/agents"

View File

@@ -92,6 +92,174 @@ EOF
refute_output --partial "FAIL" refute_output --partial "FAIL"
} }
@test "user scope: agent file directly in \$HOME (start dir IS exactly \$HOME, no walk-up) resolves to user scope" {
local fake_home="$TMPDIR/fakehome-direct"
mkdir -p "$fake_home" "$fake_home/.copilot/agents"
cat > "$fake_home/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$fake_home/.copilot/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$fake_home/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: a nested marker-less directory walked up into \$HOME resolves to project scope, not user scope (live repro of new-agent.sh's stray-directory case)" {
local fake_home="$TMPDIR/fakehome-nested"
local nested="$fake_home/scratch/testdir"
mkdir -p "$nested/.claude/agents" "$nested/.github/agents"
cat > "$nested/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$nested/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$nested/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: nested marker-less dir under \$HOME does NOT look for a counterpart under the shared \$HOME/.copilot or \$HOME/.github dirs" {
local fake_home="$TMPDIR/fakehome-nested2"
local nested="$fake_home/scratch/testdir"
mkdir -p "$nested/.claude/agents" "$fake_home/.copilot/agents"
cat > "$nested/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# Decoy counterpart at the *user*-scope location — if scope were
# misclassified as 'user' (the pre-fix bug), validate.sh would find this
# unrelated file and (wrongly) pass.
cat > "$fake_home/.copilot/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$fake_home" bash "$SCRIPT" "$nested/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "counterpart file not found"
}
@test "project scope: filesystem-root fallback (no \$HOME in path, no markers found) resolves to project scope, not user scope" {
local unrelated_home="$TMPDIR/unrelated-home-never-reached"
local root="$TMPDIR/no-home-relation/deep/proj"
mkdir -p "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run env HOME="$unrelated_home" bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: <root> one level below a .git ancestor resolves scope to <root>, not to wherever .git was found (subdirectory of a larger git-tracked tree)" {
local repo="$TMPDIR/repo-with-subdir"
local root="$repo/subdir"
mkdir -p "$repo/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# new-agent.sh, invoked with <root> as its root argument, would place the
# counterpart at <root>/.github/agents — not at the repo root's
# .github/agents, even though .git lives at the repo root one level up.
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_success
refute_output --partial "FAIL"
refute_output --partial "counterpart"
}
@test "project scope: a non-conventional path (agent file not directly under a literal 'agents' dir) falls back to the nearest .git boundary instead of two-segments-up arithmetic" {
local outer="$TMPDIR/outer-repo"
local pkg="$outer/pkgA"
mkdir -p "$pkg/.git" "$pkg/.github/agents" "$pkg/extra"
# Misplaced file: sits two path segments below $outer (pkgA/extra), which
# matches the conventional_root arithmetic by coincidence, but its
# immediate parent dir is "extra", not "agents" — conventional_shape is
# false, so the fix must fall back to the nearest .git boundary (pkgA),
# not trust $outer.
cat > "$pkg/extra/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
# Counterpart at the nearest-.git root (pkgA), not at $outer — if the
# arithmetic were trusted here, validate.sh would look for a counterpart
# at $outer/.github/agents/my-agent.agent.md, which doesn't exist, and
# false-FAIL.
cat > "$pkg/.github/agents/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$pkg/extra/my-agent.md"
assert_success
refute_output --partial "counterpart file not found"
}
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Failing cases — project/user scope: CC/Copilot pair checks # Failing cases — project/user scope: CC/Copilot pair checks
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------

View File

@@ -90,7 +90,7 @@ The script is file-by-file no-op — it skips any file that already exists.
### Step 2 — Fill in the agent file(s) ### Step 2 — Fill in the agent file(s)
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Frontmatter carries ONLY `name`, `description`, and optionally `model` — never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill in `name`, `description`, `model`, and the system prompt body per the guidance below; the rest of this step's field-by-field guidance (tools, maxTurns, effort, memory, isolation, disallowedTools, skills, color, initialPrompt, background) is project/user scope only. Skip Step 3 and go to Step 4. **At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Frontmatter carries ONLY `name`, `description`, optionally `model`, and optionally `source_keys` (provenance metadata, not a runtime field — see the template) — never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill in `name`, `description`, `model`, and the system prompt body per the guidance below; the rest of this step's field-by-field guidance (tools, maxTurns, effort, memory, isolation, disallowedTools, skills, color, initialPrompt, background) is project/user scope only. Skip Step 3 and go to Step 4.
**At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file. **At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file.

View File

@@ -6,4 +6,4 @@ Annotated agent definition templates copied by `scripts/new-agent.sh` when scaff
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders. - **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
- **`copilot.agent.md`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`). - **`copilot.agent.md`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Only `name`, `description`, and optional `model` in frontmatter — no `tools` and no Claude-only fields, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator (ADR-0016). - **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Only `name`, `description`, optional `model`, and optional `source_keys` (provenance metadata, not a runtime field) in frontmatter — no `tools` and no Claude-only fields, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator (ADR-0016).

View File

@@ -7,7 +7,8 @@
incompatible vocabularies, and Claude-only fields (isolation, maxTurns, incompatible vocabularies, and Claude-only fields (isolation, maxTurns,
effort, memory, permissionMode) have no Copilot equivalent. A value correct effort, memory, permissionMode) have no Copilot equivalent. A value correct
for one harness is guaranteed wrong on the other, so this scope carries for one harness is guaranteed wrong on the other, so this scope carries
ONLY the three fields below — full stop (see ADR-0016). ONLY the fields below — full stop (see ADR-0016). `source_keys` is
provenance metadata, not a runtime field, and is exempt from that rule.
Do NOT add: tools, isolation, maxTurns, effort, memory, permissionMode, Do NOT add: tools, isolation, maxTurns, effort, memory, permissionMode,
disallowedTools, skills, color, initialPrompt, background, hooks, or disallowedTools, skills, color, initialPrompt, background, hooks, or

View File

@@ -24,7 +24,7 @@ When the same agent `name` appears at multiple scopes, **user scope wins over pr
## Plugin scope restrictions ## Plugin scope restrictions
Plugin/APM agents (`.apm/agents/<name>.agent.md`) carry only `name`, `description`, and optionally `model` in frontmatter — full stop (see ADR-0016). `apm compile` copies this frontmatter verbatim to both the Claude Code and Copilot CLI compile targets with no per-target integrator: Claude's `tools:` (space-separated string) and Copilot's `tools:` (alias list) are incompatible vocabularies, and Claude-only fields have no Copilot equivalent, so any harness-specific value is guaranteed wrong on at least one target. Plugin/APM agents (`.apm/agents/<name>.agent.md`) carry only `name`, `description`, optionally `model`, and optionally `source_keys` (provenance metadata, not a runtime field — silently ignored by both harnesses) in frontmatter — full stop (see ADR-0016). `apm compile` copies this frontmatter verbatim to both the Claude Code and Copilot CLI compile targets with no per-target integrator: Claude's `tools:` (space-separated string) and Copilot's `tools:` (alias list) are incompatible vocabularies, and Claude-only fields have no Copilot equivalent, so any harness-specific value is guaranteed wrong on at least one target.
This makes the old "silently ignored at plugin scope" framing moot. It's not that `hooks`, `mcpServers`, `permissionMode`, `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `disallowedTools`, `skills`, `color`, `initialPrompt`, or `background` are merely ignored at this scope — they are never written to the file at all. Copy the agent to `.claude/agents/` (project scope) or `~/.claude/agents/` (user scope) to use any of them. This makes the old "silently ignored at plugin scope" framing moot. It's not that `hooks`, `mcpServers`, `permissionMode`, `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `disallowedTools`, `skills`, `color`, `initialPrompt`, or `background` are merely ignored at this scope — they are never written to the file at all. Copy the agent to `.claude/agents/` (project scope) or `~/.claude/agents/` (user scope) to use any of them.

View File

@@ -0,0 +1,342 @@
#!/usr/bin/env bash
set -euo pipefail
# Behavioral consistency check for the four independent, semantically-equivalent
# ports of "walk up from a directory looking for a scope-defining marker" living
# in this repo:
#
# - plugins/kyberforge/skills/agent-audit/scripts/validate.sh (Python: detect_scope)
# - plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh (Python: find_plugin_root)
# - plugins/kyberforge/skills/agent-author/scripts/new-agent.sh (Bash: find_package_root)
# - plugins/kyberforge/skills/skill-author/scripts/new-skill.sh (Bash: find_package_root)
#
# Per ADR-0014's no-cross-skill-path rule, these can't be consolidated into a
# shared file (each skill's cache-install copies only its own files), so unlike
# check-vale-style-sync.sh (which diffs literal file copies) this can't be a
# text diff — the four implementations are hand-ported, not copied. Instead
# this builds a matrix of fixture directory trees and asserts the *observable
# behavior* agrees: whatever new-agent.sh/new-skill.sh actually create on disk,
# validate.sh/validate-provenance.sh must classify the same way when pointed at
# the result. Run from repo root or pass REPO_ROOT as arg.
REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}"
if [[ ! -d "$REPO_ROOT" ]]; then
echo "Scope walk-up sync check failed: REPO_ROOT '$REPO_ROOT' is not a directory." >&2
exit 1
fi
REPO_ROOT="$(cd "$REPO_ROOT" && pwd)"
NEW_AGENT="$REPO_ROOT/plugins/kyberforge/skills/agent-author/scripts/new-agent.sh"
NEW_SKILL="$REPO_ROOT/plugins/kyberforge/skills/skill-author/scripts/new-skill.sh"
VALIDATE="$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate.sh"
VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh"
for f in "$NEW_AGENT" "$NEW_SKILL" "$VALIDATE" "$VALIDATE_PROVENANCE"; do
if [[ ! -f "$f" ]]; then
echo "Scope walk-up sync check: $f not found — kyberforge agent-author/agent-audit/skill-author skills not present, nothing to check." >&2
exit 0
fi
done
FAIL=0
err() { echo " FAIL: $1" >&2; FAIL=$((FAIL + 1)); }
ok() { echo " ok: $1"; }
FIXTURES=()
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
trap cleanup EXIT
# Fill a new-agent.sh-scaffolded pair's FILL IN: placeholders with valid
# content, isolating the scope/counterpart-lookup question from unrelated
# content-quality FAILs when cross-checking against validate.sh.
fill_agent_pair() {
local file="$1" name="$2"
cat > "$file" <<EOF
---
name: ${name}
description: A valid agent description.
---
You are a test agent. When invoked, do the thing.
EOF
}
# ---------------------------------------------------------------------------
# Fixture 1: root exactly $HOME (no walk-up) — new-agent.sh's own documented
# user-scope case.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: root exactly \$HOME ---"
F1_HOME="$(mktemp -d)"
FIXTURES+=("$F1_HOME")
NAME1="probe-home-exact"
if ! env HOME="$F1_HOME" bash "$NEW_AGENT" "$NAME1" "$F1_HOME" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at root exactly \$HOME"
else
if [[ ! -f "$F1_HOME/.claude/agents/$NAME1.md" || ! -f "$F1_HOME/.copilot/agents/$NAME1.agent.md" ]]; then
err "new-agent.sh did not create the expected user-scope pair at \$HOME/.claude and \$HOME/.copilot"
else
fill_agent_pair "$F1_HOME/.claude/agents/$NAME1.md" "$NAME1"
fill_agent_pair "$F1_HOME/.copilot/agents/$NAME1.agent.md" "$NAME1"
if env HOME="$F1_HOME" bash "$VALIDATE" "$F1_HOME/.claude/agents/$NAME1.md" >/tmp/f1.out 2>&1; then
ok "validate.sh agrees: user scope, counterpart found under \$HOME/.copilot"
else
err "validate.sh disagreed with new-agent.sh's user-scope classification at root exactly \$HOME"
sed 's/^/ /' /tmp/f1.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 2: nested marker-less directory under $HOME — the live-repro shape.
# new-agent.sh's own docs call this out as deliberately project scope, not
# user scope (a stray directory under $HOME can't be silently redirected into
# the shared global ~/.claude or ~/.copilot agent directories).
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: nested marker-less directory under \$HOME ---"
F2_HOME="$(mktemp -d)"
FIXTURES+=("$F2_HOME")
F2_NESTED="$F2_HOME/scratch/testdir"
mkdir -p "$F2_NESTED"
NAME2="probe-home-nested"
if ! env HOME="$F2_HOME" bash "$NEW_AGENT" "$NAME2" "$F2_NESTED" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold under a nested marker-less \$HOME subdirectory"
else
if [[ ! -f "$F2_NESTED/.claude/agents/$NAME2.md" || ! -f "$F2_NESTED/.github/agents/$NAME2.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the nested dir (rooted at \$F2_NESTED, not \$HOME)"
elif [[ -f "$F2_HOME/.claude/agents/$NAME2.md" || -f "$F2_HOME/.copilot/agents/$NAME2.agent.md" ]]; then
err "new-agent.sh unexpectedly wrote into \$HOME/.claude or \$HOME/.copilot for a nested marker-less start dir"
else
ok "new-agent.sh: nested marker-less dir under \$HOME scaffolds project scope at the nested dir"
fill_agent_pair "$F2_NESTED/.claude/agents/$NAME2.md" "$NAME2"
fill_agent_pair "$F2_NESTED/.github/agents/$NAME2.agent.md" "$NAME2"
if env HOME="$F2_HOME" bash "$VALIDATE" "$F2_NESTED/.claude/agents/$NAME2.md" >/tmp/f2.out 2>&1; then
ok "validate.sh agrees: project scope, counterpart found at the nested dir (not \$HOME/.copilot)"
else
err "validate.sh disagreed with new-agent.sh: misclassified the nested marker-less \$HOME subdirectory"
sed 's/^/ /' /tmp/f2.out
fi
# new-skill.sh has no user/project distinction of its own (no $HOME
# awareness at all — see new-skill.sh's find_package_root), but it shares
# the same .git/apm.yml walk-up primitive. It must land its standalone
# scaffold at the given path too, not get redirected toward $HOME.
if env HOME="$F2_HOME" bash "$NEW_SKILL" probe-home-nested-skill "$F2_NESTED" >/tmp/f2skill.out 2>&1 \
&& [[ -d "$F2_NESTED/probe-home-nested-skill" ]]; then
ok "new-skill.sh agrees: standalone mode scaffolds at the nested dir, not redirected toward \$HOME"
else
err "new-skill.sh disagreed with new-agent.sh/validate.sh on the nested marker-less \$HOME subdirectory"
sed 's/^/ /' /tmp/f2skill.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 3: a .git boundary between the probe dir and $HOME must stop the
# walk before it ever reaches $HOME (so it can't be misclassified as user
# scope via the home-boundary path).
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: .git boundary short-circuits before reaching \$HOME ---"
F3_HOME="$(mktemp -d)"
FIXTURES+=("$F3_HOME")
# .git sits directly at the probe root (the conventional two-segments-above
# location .claude/agents and .github/agents are placed relative to). This
# fixture only exercises what it's meant to: that a .git ancestor stops the
# walk before it ever reaches $HOME. Fixture 3b below covers .git sitting
# higher up than the probe root.
F3_PROBE="$F3_HOME/myrepo"
mkdir -p "$F3_PROBE/.git"
NAME3="probe-git-boundary"
if ! env HOME="$F3_HOME" bash "$NEW_AGENT" "$NAME3" "$F3_PROBE" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a dir with a .git ancestor short of \$HOME"
else
if [[ ! -f "$F3_PROBE/.claude/agents/$NAME3.md" || ! -f "$F3_PROBE/.github/agents/$NAME3.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the probe dir"
else
fill_agent_pair "$F3_PROBE/.claude/agents/$NAME3.md" "$NAME3"
fill_agent_pair "$F3_PROBE/.github/agents/$NAME3.agent.md" "$NAME3"
if env HOME="$F3_HOME" bash "$VALIDATE" "$F3_PROBE/.claude/agents/$NAME3.md" >/tmp/f3.out 2>&1; then
ok "validate.sh agrees: .git boundary keeps this project scope, not promoted to user scope at \$HOME"
else
err "validate.sh disagreed with new-agent.sh on the .git-boundary-before-\$HOME fixture"
sed 's/^/ /' /tmp/f3.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 3b: .git sits one level ABOVE the probe root — a subdirectory of a
# larger git-tracked tree (e.g. a monorepo package dir). new-agent.sh always
# places project-scope files at its ROOT argument, never at the walked-up
# .git location, so validate.sh must resolve scope to the probe root too, not
# to the ancestor where .git happened to be found.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: .git ancestor sits above <root> (subdirectory of a larger git tree) ---"
F3B_REPO="$(mktemp -d)"
FIXTURES+=("$F3B_REPO")
mkdir -p "$F3B_REPO/.git"
F3B_PROBE="$F3B_REPO/subdir"
mkdir -p "$F3B_PROBE"
NAME3B="probe-git-above-root"
if ! bash "$NEW_AGENT" "$NAME3B" "$F3B_PROBE" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a dir one level below a .git ancestor"
else
if [[ ! -f "$F3B_PROBE/.claude/agents/$NAME3B.md" || ! -f "$F3B_PROBE/.github/agents/$NAME3B.agent.md" ]]; then
err "new-agent.sh did not scaffold a project-scope pair at the probe dir (rooted at \$F3B_PROBE, not the repo root)"
else
fill_agent_pair "$F3B_PROBE/.claude/agents/$NAME3B.md" "$NAME3B"
fill_agent_pair "$F3B_PROBE/.github/agents/$NAME3B.agent.md" "$NAME3B"
if bash "$VALIDATE" "$F3B_PROBE/.claude/agents/$NAME3B.md" >/tmp/f3b.out 2>&1; then
ok "validate.sh agrees: scope root is <root>, not the .git ancestor above it"
else
err "validate.sh disagreed with new-agent.sh: resolved scope to the .git ancestor instead of <root>"
sed 's/^/ /' /tmp/f3b.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 4: a type-bearing apm.yml — plugin/APM scope. new-agent.sh and
# new-skill.sh must agree on the same package root, and validate.sh /
# validate-provenance.sh must both recognize it as plugin scope.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: type-bearing apm.yml (plugin/APM scope) ---"
F4_ROOT="$(mktemp -d)"
FIXTURES+=("$F4_ROOT")
printf 'name: test-package\nversion: 0.1.0\ntype: skill\n' > "$F4_ROOT/apm.yml"
NAME4="probe-plugin"
if ! bash "$NEW_AGENT" "$NAME4" "$F4_ROOT" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at a type-bearing apm.yml root"
elif [[ ! -f "$F4_ROOT/.apm/agents/$NAME4.agent.md" ]]; then
err "new-agent.sh did not scaffold plugin scope at the type-bearing apm.yml root"
else
ok "new-agent.sh: plugin scope at type-bearing apm.yml root"
if bash "$NEW_SKILL" probe-plugin-skill "$F4_ROOT" >/tmp/f4skill.out 2>&1 \
&& [[ -d "$F4_ROOT/.apm/skills/probe-plugin-skill" ]]; then
ok "new-skill.sh agrees: package mode at the same apm.yml root"
else
err "new-skill.sh disagreed with new-agent.sh on the type-bearing apm.yml root"
sed 's/^/ /' /tmp/f4skill.out
fi
fill_agent_pair "$F4_ROOT/.apm/agents/$NAME4.agent.md" "$NAME4"
if bash "$VALIDATE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4validate.out 2>&1; then
ok "validate.sh agrees: plugin/APM scope, structural checks pass"
else
err "validate.sh disagreed with new-agent.sh: did not treat the type-bearing apm.yml root as plugin scope"
sed 's/^/ /' /tmp/f4validate.out
fi
# source_keys + a matching sources.md round-trips only if validate-provenance.sh
# resolves the SAME plugin root new-agent.sh/new-skill.sh did.
cat > "$F4_ROOT/.apm/agents/$NAME4.agent.md" <<EOF
---
name: ${NAME4}
description: A valid agent description.
source_keys:
- probe-source
---
You are a test agent.
EOF
cat > "$F4_ROOT/sources.md" <<EOF
# Sources
## probe-source
- **URL:** https://example.com/probe-source
- **Description:** A test source.
- **Contributing files:** .apm/agents/${NAME4}.agent.md
- **Research doc:** (none)
- **Status:** \`extracted\`
EOF
if bash "$VALIDATE_PROVENANCE" "$F4_ROOT/.apm/agents/$NAME4.agent.md" >/tmp/f4prov.out 2>&1; then
ok "validate-provenance.sh agrees: resolves the same plugin root, sources.md round-trips"
else
err "validate-provenance.sh disagreed on the plugin root for the type-bearing apm.yml fixture"
sed 's/^/ /' /tmp/f4prov.out
fi
fi
# ---------------------------------------------------------------------------
# Fixture 5: filesystem-boundary fallback — no $HOME relation, no marker
# anywhere. Both scripts must fall through to project scope, not user scope.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: filesystem-boundary fallback (no \$HOME relation, no markers) ---"
F5_UNRELATED_HOME_PARENT="$(mktemp -d)"
FIXTURES+=("$F5_UNRELATED_HOME_PARENT")
F5_UNRELATED_HOME="$F5_UNRELATED_HOME_PARENT/never-reached-$$"
F5_ROOT="$(mktemp -d)/deep/proj"
mkdir -p "$F5_ROOT"
FIXTURES+=("$(dirname "$(dirname "$F5_ROOT")")")
NAME5="probe-fs-boundary"
if ! env HOME="$F5_UNRELATED_HOME" bash "$NEW_AGENT" "$NAME5" "$F5_ROOT" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold at the filesystem-boundary fixture"
else
if [[ ! -f "$F5_ROOT/.claude/agents/$NAME5.md" || ! -f "$F5_ROOT/.github/agents/$NAME5.agent.md" ]]; then
err "new-agent.sh did not scaffold project scope at the filesystem-boundary fixture"
else
fill_agent_pair "$F5_ROOT/.claude/agents/$NAME5.md" "$NAME5"
fill_agent_pair "$F5_ROOT/.github/agents/$NAME5.agent.md" "$NAME5"
if env HOME="$F5_UNRELATED_HOME" bash "$VALIDATE" "$F5_ROOT/.claude/agents/$NAME5.md" >/tmp/f5.out 2>&1; then
ok "validate.sh agrees: filesystem-boundary fallback resolves to project scope"
else
err "validate.sh disagreed with new-agent.sh on the filesystem-boundary fallback fixture"
sed 's/^/ /' /tmp/f5.out
fi
fi
fi
# ---------------------------------------------------------------------------
# Fixture 6: a type-bearing apm.yml ABOVE $HOME must not be reached by
# validate-provenance.sh's walk-up from a nested, marker-less dir under $HOME
# — matches new-agent.sh, which also stops at $HOME before ever looking that
# far up.
# ---------------------------------------------------------------------------
echo ""
echo "--- fixture: type-bearing apm.yml above \$HOME must not be reached ---"
F6_ANCESTOR="$(mktemp -d)"
FIXTURES+=("$F6_ANCESTOR")
printf 'name: outer-package\nversion: 0.1.0\ntype: skill\n' > "$F6_ANCESTOR/apm.yml"
F6_HOME="$F6_ANCESTOR/fakehome"
mkdir -p "$F6_HOME"
NAME6="probe-above-home"
if ! env HOME="$F6_HOME" bash "$NEW_AGENT" "$NAME6" "$F6_HOME" >/dev/null 2>&1; then
err "new-agent.sh failed to scaffold with a type-bearing apm.yml above \$HOME"
elif [[ -f "$F6_HOME/.apm/agents/$NAME6.agent.md" ]]; then
err "new-agent.sh walked past \$HOME and misclassified as plugin scope using the ancestor apm.yml"
elif [[ ! -f "$F6_HOME/.claude/agents/$NAME6.md" ]]; then
err "new-agent.sh did not scaffold user scope at root exactly \$HOME (with a type-bearing apm.yml above)"
else
ok "new-agent.sh: \$HOME boundary stops the walk before the ancestor apm.yml, user scope at \$HOME"
mkdir -p "$F6_HOME/.apm/agents"
cat > "$F6_HOME/.apm/agents/probe-prov.agent.md" <<'EOF'
---
name: probe-prov
description: A valid agent description.
source_keys:
- probe-source
---
You are a test agent.
EOF
# No sources.md exists anywhere under $F6_HOME or at the ancestor package
# root — if find_plugin_root walked past $HOME to the ancestor apm.yml,
# this would FAIL on Check 0 (source_keys declared but sources.md absent).
if env HOME="$F6_HOME" bash "$VALIDATE_PROVENANCE" "$F6_HOME/.apm/agents/probe-prov.agent.md" >/tmp/f6.out 2>&1 \
&& [[ -z "$(cat /tmp/f6.out)" ]]; then
ok "validate-provenance.sh agrees: \$HOME boundary stops the walk, exits 0 silently (not plugin scope)"
else
err "validate-provenance.sh walked past \$HOME to the ancestor apm.yml — disagrees with new-agent.sh"
sed 's/^/ /' /tmp/f6.out
fi
fi
echo ""
if [[ $FAIL -gt 0 ]]; then
echo "Scope walk-up sync check failed: $FAIL error(s). One of validate.sh's detect_scope, validate-provenance.sh's find_plugin_root, new-agent.sh's find_package_root, or new-skill.sh's find_package_root has drifted from the others' \$HOME/.git/apm.yml walk-up semantics. Re-read new-agent.sh's usage comment (the canonical description of the intended behavior) and bring the disagreeing script back in line." >&2
exit 1
fi
echo "Scope walk-up sync check passed: all four walk-up implementations agree on every fixture."

View File

@@ -0,0 +1,141 @@
#!/usr/bin/env bash
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
SCRIPT="$REPO_ROOT/scripts/check-scope-walkup-sync.sh"
PASS=0
FAIL=0
pass() { echo " PASS: $1"; PASS=$((PASS + 1)); }
fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); }
FIXTURES=()
cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; }
trap cleanup EXIT
# --- 1. Exits 0 against this repo's own (fixed) scripts ---
echo ""
echo "--- exits 0 against this repo's real scripts ---"
if bash "$SCRIPT" "$REPO_ROOT" > /tmp/check-scope-walkup-sync-clean.out 2>&1; then
pass "exits 0 against this repo's real scope walk-up scripts"
else
fail "exited non-zero against this repo's real (already-fixed) scripts"
sed 's/^/ /' /tmp/check-scope-walkup-sync-clean.out
fi
# --- 2. Exits 0 as a no-op when the kyberforge skills aren't present ---
echo ""
echo "--- exits 0 (no-op) when the target scripts don't exist ---"
FIXTURE_EMPTY="$(mktemp -d)"
FIXTURES+=("$FIXTURE_EMPTY")
if bash "$SCRIPT" "$FIXTURE_EMPTY" > /dev/null 2>&1; then
pass "exits 0 as a no-op when agent-author/agent-audit/skill-author aren't present"
else
fail "exited non-zero when the kyberforge skills are simply absent"
fi
# --- 3. Exits 1 against a REPO_ROOT that doesn't exist ---
echo ""
echo "--- exits 1 when REPO_ROOT does not exist ---"
if bash "$SCRIPT" "/nonexistent/path/$(date +%s)-$$" > /dev/null 2>&1; then
fail "exited 0 for a nonexistent REPO_ROOT — expected exit 1"
else
pass "exits non-zero for a nonexistent REPO_ROOT"
fi
# --- 4. Regression guard: reintroducing the $HOME-collapse bug into
# validate.sh's detect_scope must make the check fail. Builds a minimal
# REPO_ROOT (just the four scripts, at their real relative paths) so this
# doesn't depend on — or risk mutating — the real repo tree.
make_minimal_repo_root() {
local dir
dir="$(mktemp -d)"
local na="$dir/plugins/kyberforge/skills/agent-author/scripts"
local ns="$dir/plugins/kyberforge/skills/skill-author/scripts"
local aa="$dir/plugins/kyberforge/skills/agent-audit/scripts"
mkdir -p "$na" "$ns" "$aa"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-author/scripts/new-agent.sh" "$na/"
cp "$REPO_ROOT/plugins/kyberforge/skills/skill-author/scripts/new-skill.sh" "$ns/"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate.sh" "$aa/"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh" "$aa/"
# agent-author's templates are needed by new-agent.sh at runtime.
cp -R "$REPO_ROOT/plugins/kyberforge/skills/agent-author/assets" "$dir/plugins/kyberforge/skills/agent-author/"
cp -R "$REPO_ROOT/plugins/kyberforge/skills/skill-author/assets" "$dir/plugins/kyberforge/skills/skill-author/"
# validate.sh needs field-inventory.md
mkdir -p "$dir/plugins/kyberforge/skills/agent-audit/references"
cp "$REPO_ROOT/plugins/kyberforge/skills/agent-audit/references/field-inventory.md" \
"$dir/plugins/kyberforge/skills/agent-audit/references/"
echo "$dir"
}
echo ""
echo "--- exits 1 when validate.sh's detect_scope collapses back to the \$HOME-walk-up bug ---"
FIXTURE_BUG="$(make_minimal_repo_root)"
FIXTURES+=("$FIXTURE_BUG")
python3 - "$FIXTURE_BUG/plugins/kyberforge/skills/agent-audit/scripts/validate.sh" <<'PYTHON'
import re, sys
path = sys.argv[1]
with open(path) as f:
content = f.read()
# Revert to the pre-fix collapsed logic: both the $HOME-boundary case and the
# filesystem-root fallback return 'user', home unconditionally.
old = """def detect_scope(start_dir):
home = os.path.expanduser('~')
original_start = os.path.abspath(start_dir)"""
assert old in content, "detect_scope signature not found — validate.sh has changed shape"
buggy = '''def detect_scope(start_dir):
home = os.path.expanduser('~')
current = os.path.abspath(start_dir)
while True:
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current
if current == home:
return 'user', home
if os.path.exists(os.path.join(current, '.git')):
return 'project', current
parent = os.path.dirname(current)
if parent == current:
return 'user', home
current = parent
'''
# Replace the whole function body up to (but not including) the next
# top-level `agent_dir = ` assignment that calls it.
pattern = re.compile(r"def detect_scope\(start_dir\):\n.*?\n(?=agent_dir = )", re.DOTALL)
assert pattern.search(content), "could not isolate detect_scope's full body"
content = pattern.sub(buggy + "\n", content)
with open(path, 'w') as f:
f.write(content)
PYTHON
if bash "$SCRIPT" "$FIXTURE_BUG" > /tmp/check-scope-walkup-sync-buggy.out 2>&1; then
fail "exited 0 against a validate.sh reverted to the \$HOME-collapse bug — expected exit 1"
else
pass "exits non-zero when validate.sh's detect_scope regresses to the \$HOME-collapse bug"
fi
echo ""
echo "--- exits 1 when validate-provenance.sh's find_plugin_root loses its \$HOME boundary check ---"
FIXTURE_BUG2="$(make_minimal_repo_root)"
FIXTURES+=("$FIXTURE_BUG2")
python3 - "$FIXTURE_BUG2/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh" <<'PYTHON'
import re, sys
path = sys.argv[1]
with open(path) as f:
content = f.read()
# Drop the `if current == home: return None` line — reverts to the pre-fix
# behavior of never checking a $HOME boundary at all.
pattern = re.compile(r"\n *# \$HOME is a non-plugin-scope boundary.*?\n *if current == home:\n *return None\n", re.DOTALL)
assert pattern.search(content), "could not find the \\$HOME boundary check to remove"
content = pattern.sub("\n", content)
with open(path, 'w') as f:
f.write(content)
PYTHON
if bash "$SCRIPT" "$FIXTURE_BUG2" > /tmp/check-scope-walkup-sync-buggy2.out 2>&1; then
fail "exited 0 against a validate-provenance.sh with no \$HOME boundary check — expected exit 1"
else
pass "exits non-zero when validate-provenance.sh's find_plugin_root loses its \$HOME boundary check"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]