From 996d9be428b57522e6697dccbbb308bc1748f2a7 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Sun, 5 Jul 2026 09:01:06 +0000 Subject: [PATCH 1/2] fix(agents): relocate provenance sources.md outside agents/ dir `claude plugin validate --strict` auto-discovers every .md under a plugin's agents/ directory as an agent requiring frontmatter, so the provenance file there always needs fake agent frontmatter to pass validation. Confirmed empirically that an explicit `agents` manifest array can't suppress this discovery. Move the file to / sources.md instead, and update agent-author/agent-audit accordingly. Adds ADR-0010, partially superseding ADR-0005's `agents/sources.md` convention. Fixes #63. Co-Authored-By: Claude Sonnet 5 --- ...005-agent-author-dual-provider-scaffold.md | 5 ++ ...nt-sources-relocated-outside-agents-dir.md | 58 +++++++++++++++++++ plugins/git/.claude-plugin/plugin.json | 2 +- plugins/git/plugin.json | 2 +- plugins/git/{agents => }/sources.md | 6 -- plugins/kyberforge/.claude-plugin/plugin.json | 2 +- plugins/kyberforge/plugin.json | 2 +- .../kyberforge/skills/agent-audit/README.md | 4 +- .../kyberforge/skills/agent-audit/SKILL.md | 2 +- .../scripts/validate-provenance.sh | 52 ++++++++--------- .../tests/validate-provenance.bats | 36 ++++++------ .../kyberforge/skills/agent-author/SKILL.md | 10 ++-- .../assets/templates/claude-code.md | 2 +- .../assets/templates/copilot.agent.md | 2 +- .../skills/agent-author/scripts/new-agent.sh | 4 +- .../skills/agent-author/tests/new-agent.bats | 4 +- 16 files changed, 125 insertions(+), 68 deletions(-) create mode 100644 docs/adr/0010-agent-sources-relocated-outside-agents-dir.md rename plugins/git/{agents => }/sources.md (93%) diff --git a/docs/adr/0005-agent-author-dual-provider-scaffold.md b/docs/adr/0005-agent-author-dual-provider-scaffold.md index 5b1dd4e..6dc004d 100644 --- a/docs/adr/0005-agent-author-dual-provider-scaffold.md +++ b/docs/adr/0005-agent-author-dual-provider-scaffold.md @@ -39,3 +39,8 @@ separate single-provider skill, adding complexity with no benefit. - The file-by-file no-op in the script (skip existing files rather than overwriting) means partial state — one provider file exists, the other does not — is handled by routing in the skill body, not in the script. + +**Update (ADR-0010):** the `agents/sources.md` path above is superseded. The provenance +file now lives at `/sources.md`, outside the `agents/` directory, because +`claude plugin validate --strict` auto-discovers every `.md` under `agents/` as an agent +requiring frontmatter. See ADR-0010 for the empirical finding and rationale. diff --git a/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md b/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md new file mode 100644 index 0000000..3860472 --- /dev/null +++ b/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md @@ -0,0 +1,58 @@ +# Plugin-scope agent provenance file moves to `/sources.md` + +**Partially supersedes:** ADR-0005 (agent-author dual-provider scaffold) — specifically the +claim that "both files share a single `agents/sources.md` for provenance." The rest of +ADR-0005 (dual-provider generation, scope detection, single-root script interface) is +unaffected and remains in force. + +`claude plugin validate --strict` auto-discovers every `.md` file directly under a plugin's +`agents/` directory and treats it as an agent definition requiring YAML frontmatter (`name`, +`description`, etc.). A flat provenance file at `agents/sources.md` — no frontmatter, by +design, since it is not an agent — fails validation with a missing-frontmatter warning that +`--strict` promotes to an error. + +This was first hit in `plugins/git/agents/sources.md` (added by the git-plugin skill suite). +It failed the `validate-plugins` pre-push hook. The stopgap in commit `0239b00` added +throwaway agent frontmatter to unblock the push: + +```yaml +--- +name: git-agents-sources +description: Provenance record for the git plugin's agents, not an invokable agent. Do not invoke. +tools: none +--- +``` + +That workaround is now reverted — the file no longer lives where it needs to impersonate an +agent to pass validation. + +## Considered options + +**Exclude via an explicit `agents` manifest array (rejected)** — `plugin.json` supports +`"agents": ["./agents/reviewer.md"]` as an alternative to `"agents": "agents/"`. The +hypothesis was that listing only real agent files would stop the validator from also +discovering `sources.md` in the same directory. Tested empirically on a scratch copy of the +git plugin: `claude plugin validate --strict` still auto-discovered and failed on the +unlisted `sources.md`, regardless of the explicit array. The manifest field controls what +Claude Code loads as agents at runtime; it does not control what the validator scans on +disk. There is no manifest-level or CLI-flag mechanism to exclude a file from `agents/` +auto-discovery. + +**Keep the frontmatter workaround permanently (rejected)** — cheapest fix, already applied, +but semantically wrong: it makes a plain provenance record indistinguishable from a real +invokable agent to any tooling or UI that lists available agents (e.g. it could appear as a +callable agent in the `/agents` picker), which is confusing and incorrect. + +## Consequences + +- The provenance file moves to `/sources.md` — a flat file, plugin-root + relative, sitting outside any directory that Claude Code or its validator auto-scans. No + frontmatter is needed or added. +- `agent-author`'s `new-agent.sh` now writes `/sources.md` instead of + `/agents/sources.md` at plugin scope. +- `agent-audit`'s `validate-provenance.sh` now looks for `/sources.md` when + checking `source_keys` provenance chains. +- All doc and template references to `agents/sources.md` (agent-author `SKILL.md`, + agent-audit `SKILL.md`/`README.md`, both provider templates) are updated to `sources.md`. +- `plugins/git/agents/sources.md` is relocated to `plugins/git/sources.md` and the + `0239b00` frontmatter workaround is removed. diff --git a/plugins/git/.claude-plugin/plugin.json b/plugins/git/.claude-plugin/plugin.json index 77b5d8c..64d6d1d 100644 --- a/plugins/git/.claude-plugin/plugin.json +++ b/plugins/git/.claude-plugin/plugin.json @@ -13,5 +13,5 @@ ], "license": "MIT", "name": "git", - "version": "1.3.1" + "version": "1.3.2" } diff --git a/plugins/git/plugin.json b/plugins/git/plugin.json index 22cbf9f..5f4e94c 100644 --- a/plugins/git/plugin.json +++ b/plugins/git/plugin.json @@ -18,5 +18,5 @@ "skills": [ "skills/" ], - "version": "1.3.1" + "version": "1.3.2" } diff --git a/plugins/git/agents/sources.md b/plugins/git/sources.md similarity index 93% rename from plugins/git/agents/sources.md rename to plugins/git/sources.md index 4bed246..fef32b5 100644 --- a/plugins/git/agents/sources.md +++ b/plugins/git/sources.md @@ -1,9 +1,3 @@ ---- -name: git-agents-sources -description: Provenance record for the git plugin's agents, not an invokable agent. Do not invoke. -tools: none ---- - # Sources ## context7-git-htmldocs diff --git a/plugins/kyberforge/.claude-plugin/plugin.json b/plugins/kyberforge/.claude-plugin/plugin.json index 33a9d02..d1f1479 100644 --- a/plugins/kyberforge/.claude-plugin/plugin.json +++ b/plugins/kyberforge/.claude-plugin/plugin.json @@ -8,5 +8,5 @@ "keywords": [], "license": "MIT", "name": "kyberforge", - "version": "1.2.0" + "version": "1.2.1" } diff --git a/plugins/kyberforge/plugin.json b/plugins/kyberforge/plugin.json index 3cdec6f..6684331 100644 --- a/plugins/kyberforge/plugin.json +++ b/plugins/kyberforge/plugin.json @@ -13,5 +13,5 @@ "skills": [ "skills/" ], - "version": "1.2.0" + "version": "1.2.1" } diff --git a/plugins/kyberforge/skills/agent-audit/README.md b/plugins/kyberforge/skills/agent-audit/README.md index 6316d5f..9476dec 100644 --- a/plugins/kyberforge/skills/agent-audit/README.md +++ b/plugins/kyberforge/skills/agent-audit/README.md @@ -4,7 +4,7 @@ Audits a Claude Code and Copilot agent definition file pair for correctness and ## What it does -Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `agents/sources.md` at plugin scope), then qualitative checks on description phrasing and system prompt quality. Produces a compact findings report in the same format as `skill-audit`. +Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `sources.md` at the plugin root), then qualitative checks on description phrasing and system prompt quality. Produces a compact findings report in the same format as `skill-audit`. ## Usage @@ -25,7 +25,7 @@ Pass the path to either agent file as the argument. | `references/sources.md` | Research provenance for skill content | | `scripts/README.md` | Directory documentation for scripts/ | | `scripts/validate.sh` | Structural validation script for agent file pairs | -| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `agents/sources.md` | +| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `sources.md` (plugin root) | | `tests/README.md` | Bats test dependency and run instructions | | `tests/validate.bats` | Bats tests for validate.sh | | `tests/validate-provenance.bats` | Bats tests for validate-provenance.sh | diff --git a/plugins/kyberforge/skills/agent-audit/SKILL.md b/plugins/kyberforge/skills/agent-audit/SKILL.md index dbac5b7..baaa1a4 100644 --- a/plugins/kyberforge/skills/agent-audit/SKILL.md +++ b/plugins/kyberforge/skills/agent-audit/SKILL.md @@ -37,7 +37,7 @@ bash scripts/validate-provenance.sh The script accepts either the CC file or the Copilot file. It detects provider from extension, derives the counterpart, and runs all structural checks. Note FAILs and SUGGESTIONs for the `### Structure` and `### Provider safety` report dimensions. Findings about missing fields, bad name format, empty body, or missing frontmatter → `### Structure`. Findings about CC-only fields in a Copilot file, Copilot-only fields in a CC file, plugin-silently-ignored fields, body length, or subagent-unavailable tools → `### Provider safety`. -`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `agents/sources.md`. It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix. +`validate-provenance.sh` validates the provenance chain between the agent pair's `source_keys` and the plugin-scoped `sources.md` (plugin root — see ADR-0010). It exits 0 silently for non-plugin-scope agents and when no provenance data exists. Note FAILs from this script for the `### Provenance` dimension — surface them verbatim with Why and Fix. If the scripts cannot run (Bash denied, python3 unavailable), perform checks manually: required fields present (`name`, `description`, non-empty body), `name` is kebab-case, Copilot CLI `.agent.md` `name` must match filename stem (CC files are exempt — the CC platform does not require name to match filename), no `FILL IN:` placeholders, no CC-only fields in Copilot file, no Copilot-only fields in CC file (read `references/field-inventory.md` for the authoritative field lists). diff --git a/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh b/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh index 7099150..99c4e84 100755 --- a/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh +++ b/plugins/kyberforge/skills/agent-audit/scripts/validate-provenance.sh @@ -16,10 +16,10 @@ Exit codes: 1 One or more checks failed Checks performed: - 0 source_keys present in agent pair but agents/sources.md absent - 1 FILL IN: placeholders in agents/sources.md - 2 source_keys in agent files → slug exists in agents/sources.md - 3 Contributing files listed in agents/sources.md exist on disk (plugin-root relative) + 0 source_keys present in agent pair but sources.md absent + 1 FILL IN: placeholders in sources.md + 2 source_keys in agent files → slug exists in sources.md + 3 Contributing files listed in sources.md exist on disk (plugin-root relative) 4 Contributing files back-reference the parent slug in their source_keys 5 Research doc field present and not placeholder EOF @@ -78,7 +78,7 @@ if provider == 'copilot': else: counterpart = os.path.join(agent_dir, name_stem + '.agent.md') -sources_md_path = os.path.join(plugin_root, 'agents', 'sources.md') +sources_md_path = os.path.join(plugin_root, 'sources.md') # --- Helpers --- PLACEHOLDER_RE = re.compile(r'(?' to the '## {slug}' entry in agents/sources.md." + f"sources.md (## {slug})", + f"The '## {slug}' entry in sources.md has no '- **Research doc:**' line.", + f"Add '- **Research doc:** ' to the '## {slug}' entry in sources.md." ) elif rd_value == "" or PLACEHOLDER_RE.search(rd_value): emit_fail( "Research doc field is empty or placeholder", - f"agents/sources.md (## {slug})", + f"sources.md (## {slug})", f"The '## {slug}' entry has an unfilled Research doc value.", "Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable." ) diff --git a/plugins/kyberforge/skills/agent-audit/tests/validate-provenance.bats b/plugins/kyberforge/skills/agent-audit/tests/validate-provenance.bats index f79ce3c..f5627f4 100644 --- a/plugins/kyberforge/skills/agent-audit/tests/validate-provenance.bats +++ b/plugins/kyberforge/skills/agent-audit/tests/validate-provenance.bats @@ -85,13 +85,13 @@ You are a test agent. EOF } - # Helper: create a valid agents/sources.md with one entry + # Helper: create a valid sources.md with one entry make_sources_md() { local root="$1" local slug="${2:-my-source}" local contrib="${3:-agents/my-agent.md, agents/my-agent.agent.md}" local research="${4:-(none)}" - cat > "$root/agents/sources.md" < "$root/sources.md" < "$root/agents/sources.md" < "$root/sources.md" <> "$root/agents/sources.md" + echo "Use \`FILL IN: value\` as example." >> "$root/sources.md" run bash "$SCRIPT" "$root/agents/my-agent.md" assert_success } # --------------------------------------------------------------------------- -# Check 2: source_keys slug missing from agents/sources.md → FAIL +# Check 2: source_keys slug missing from sources.md → FAIL # --------------------------------------------------------------------------- -@test "FAIL: source_keys slug in CC file not present as H2 in agents/sources.md" { +@test "FAIL: source_keys slug in CC file not present as H2 in sources.md" { local root="$TMPDIR/plugin" make_plugin "$root" make_cc_with_source_keys "$root" "my-agent" "my-source" @@ -229,7 +229,7 @@ EOF assert_output --partial "FAIL" } -@test "FAIL: source_keys slug in Copilot file not present as H2 in agents/sources.md" { +@test "FAIL: source_keys slug in Copilot file not present as H2 in sources.md" { local root="$TMPDIR/plugin" make_plugin "$root" make_clean_pair "$root" @@ -244,7 +244,7 @@ EOF # Check 4: Contributing file path doesn't exist → FAIL # --------------------------------------------------------------------------- -@test "FAIL: Contributing file listed in agents/sources.md does not exist" { +@test "FAIL: Contributing file listed in sources.md does not exist" { local root="$TMPDIR/plugin" make_plugin "$root" make_cc_with_source_keys "$root" @@ -269,12 +269,12 @@ EOF # Check 6: Research doc field missing or placeholder → FAIL # --------------------------------------------------------------------------- -@test "FAIL: Research doc field missing from agents/sources.md entry" { +@test "FAIL: Research doc field missing from sources.md entry" { local root="$TMPDIR/plugin" make_plugin "$root" make_cc_with_source_keys "$root" make_copilot_with_source_keys "$root" - cat > "$root/agents/sources.md" < "$root/sources.md" < "$root/agents/sources.md" < "$root/sources.md" < "$root/agents/sources.md" < "$root/sources.md" <