docs: finish reconciling the agent write fence across the toolchain
The previous round taught agent-audit's validator to permit disallowedTools but left the skill that writes agents still forbidding it, in six places. Running agent-author on any of the three fenced orchestrators would have stripped the fence, and nothing would have caught it: the validator's allowlist is a permit list, so an absent field passes. The template was the worst of them, since its comment is copied verbatim into every new plugin-scope agent. Where a list had to be restated it is now a pointer to field-inventory.md's apm-agent-allowlist instead -- the same data validate.sh reads -- because a roster copied into a template goes stale one step further out than the roster itself. Where the text has to teach something it teaches the shape rule rather than the exception: tools is an allowlist whose vocabulary differs per harness, so verbatim copy makes one value wrong on one target; disallowedTools is a denylist, where an unrecognised name denies nothing, so the worst case is a missing fence rather than a wrongly granted capability. ADR-0016's amendment claimed an unrecognised key is inert on Copilot while the same ADR's Context says that behaviour is unconfirmed by research -- asserting as settled the exact thing it flags as unknown, and justifying it with apm's compile-time behaviour, which says nothing about Copilot's runtime. It is rewritten into labelled tiers: confirmed for Claude Code with citations, inferred by analogy for Copilot with the analogy's limits stated, unverified where it is unverified, and the residual risk accepted explicitly with its blast radius. It also no longer claims to restore a write sandbox: the denylist does not deny Bash, which these agents inherit and legitimately need. docs/hooks.md called the old root hooks.json a stale sync artifact -- it was added in the plugin's creating commit and pointed at by main's Copilot manifest -- and claimed both ecosystems now resolve hooks/hooks.json. Copilot does not: its hooks field has no default and no compiled manifest declares one, so it resolves nothing. Recorded as the gap it is, with re-injection noted as a follow-up rather than asserted away. Its event list is marked partial. Also: new-agent.bats asserted a hardcoded four-field allowlist and would have rejected a scaffolded agent carrying the field the ADR now blesses; it reads field-inventory.md too. And ADR-0016's premise that Claude's tools: is space-separated was wrong -- it takes a comma-separated string or a YAML list. The incompatibility with Copilot is the vocabulary, not the punctuation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
This commit is contained in:
@@ -29,7 +29,9 @@ metadata:
|
||||
## Gotchas
|
||||
|
||||
- At plugin/APM scope, bump the resolved package's `apm.yml` `version` after every change — minor for a new agent, patch for a fix. Consumers compare this version to detect updates; skipping it hides the change.
|
||||
- At plugin/APM scope, `tools` and all Claude-only fields (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `disallowedTools`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) are omitted entirely, not merely restricted (ADR-0016: `apm compile` copies frontmatter verbatim to both harnesses with no per-target integrator, so a harness-specific value is wrong on at least one). Only project/user scope supports these fields.
|
||||
- At plugin/APM scope, `tools` and all Claude-only fields (`isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, `mcpServers`) are omitted entirely, not merely restricted (ADR-0016: `apm compile` copies frontmatter verbatim to both harnesses with no per-target integrator, so a harness-specific value is wrong on at least one). Only project/user scope supports these fields.
|
||||
- `disallowedTools` is the one exception, on **shape**, not favouritism. `tools` is an *allowlist* whose vocabulary differs per harness (Claude tool names vs Copilot's `execute`/`read`/`edit`/`search`/`agent`/`web`), so verbatim copy makes one value wrong on one target. A *denylist* cannot fail that way: an unrecognised name denies nothing, so the worst case is a missing fence, never a wrong grant. Claude Code honours it for plugin subagents — `docs/research/docs/claude-code-plugins/agent-definition.md:99` lists the three fields plugin agents ignore (`hooks`, `mcpServers`, `permissionMode`) and this is not one. Write it on every read-only plugin-scope agent (ADR-0016's 2026-08-14 amendment).
|
||||
- That fence is partial: it denies only the tools it names. It does not deny `Bash`, which a plugin-scope agent with no `tools` inherits, so a shell redirect still writes. Say the agent is read-only in the body too.
|
||||
- An `apm.yml` with no top-level `type:` field is a marketplace-only manifest, not a package root — the walk-up skips it and keeps going.
|
||||
- `AskUserQuestion`, `EnterPlanMode`, `ExitPlanMode`, `ScheduleWakeup`, and `WaitForMcpServers` are never available to any subagent regardless of the `tools` field. Exception: `ExitPlanMode` is available when the parent session runs in `permissionMode: plan`.
|
||||
- Duplicate `name` values in the same scope: Claude Code silently discards one without warning. Always verify uniqueness before shipping.
|
||||
@@ -90,7 +92,7 @@ The script is file-by-file no-op — it skips any file that already exists.
|
||||
|
||||
### Step 2 — Fill in the agent file(s)
|
||||
|
||||
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Frontmatter carries ONLY `name`, `description`, optionally `model`, and optionally `source_keys` (provenance metadata, not a runtime field — see the template) — never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill in `name`, `description`, `model`, and the system prompt body per the guidance below; the rest of this step's field-by-field guidance (tools, maxTurns, effort, memory, isolation, disallowedTools, skills, color, initialPrompt, background) is project/user scope only. Skip Step 3 and go to Step 4.
|
||||
**At plugin/APM scope**, there is exactly one file: `<package-root>/.apm/agents/<name>.agent.md`. Its frontmatter allowlist is the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, read from there as data: `name`, `description`, `model`, `source_keys` (provenance metadata, not a runtime field — see the template), and `disallowedTools` for a read-only agent. Never `tools` or the other Claude-only fields listed in Gotchas (ADR-0016). Fill those in plus the system prompt body per the guidance below; the rest of this step's field-by-field guidance is project/user scope only. Skip Step 3 and go to Step 4.
|
||||
|
||||
**At project/user scope**, continue below to fill in both provider files — this step covers the Claude Code file (`<name>.md`); Step 3 covers the Copilot file.
|
||||
|
||||
@@ -106,13 +108,14 @@ Open the scaffolded Claude Code file. Replace every `FILL IN:` placeholder. **Re
|
||||
|
||||
**`tools`** (project/user scope only — never at plugin/APM scope) — restrict to what the agent actually needs. Omit to inherit all tools. Use `Agent(type1,type2)` to limit which subagent types this agent can spawn; omit `Agent` entirely to prevent spawning.
|
||||
|
||||
**Optional fields worth considering (project/user scope only — never at plugin/APM scope):**
|
||||
**`disallowedTools`** (all scopes, including plugin/APM) — denylist applied before `tools` and taking precedence over it; supports `mcp__<server>`, `mcp__<server>__*`, and `mcp__*` globs. `api-reference.md:40` types it `string / list` and `agent-definition.md:71` types it `string[]`, so a YAML list or a delimited string both work; this repo's plugin-scope agents use the comma-separated string (`disallowedTools: Edit, Write, NotebookEdit`) — match that.
|
||||
|
||||
**Optional fields worth considering (project/user scope only — never at plugin/APM scope, with the exception of `model`, which is allowed at every scope):**
|
||||
- `model`: set when this agent needs a different capability tier (`haiku` for fast tasks, `opus` for deep reasoning)
|
||||
- `maxTurns`: set a cap to prevent runaway agents on bounded tasks
|
||||
- `effort`: set to `low` for single-lookup tasks, `high` or above for deep reasoning or multi-file analysis — overrides session effort level; omit to inherit
|
||||
- `memory`: `user`, `project`, or `local` — only when cross-session state is genuinely needed
|
||||
- `isolation: worktree` — only when the agent modifies files and needs an isolated copy
|
||||
- `disallowedTools`: space-separated denylist applied before `tools`; supports `mcp__*` glob patterns (e.g. `disallowedTools: mcp__filesystem__*`)
|
||||
- `skills`: list of skill names preloaded at agent startup — different from the `source_keys` metadata field
|
||||
- `color`: UI color for the agent tile (`red`, `blue`, `green`, `yellow`, `purple`, `orange`, `pink`, `cyan`)
|
||||
- `initialPrompt`: auto-submitted as the first turn when this agent activates as the main session thread; only set when this agent is intended for main-thread activation
|
||||
@@ -152,7 +155,7 @@ Skip this step entirely at plugin/APM scope — there is no separate Copilot fil
|
||||
|
||||
**`source_keys`** — add the same top-level list as the CC file when research sources were used. Omit when no research was used.
|
||||
|
||||
**Remove all template documentation comments from the YAML frontmatter after filling in required fields** — these are marked with `<!--` and `-->` and must be deleted before shipping.
|
||||
**Delete the `<!-- -->` template comments from the frontmatter**, as in Step 2.
|
||||
|
||||
The system prompt body should match the Claude Code version — the agent's task definition is the same across providers.
|
||||
|
||||
@@ -183,37 +186,32 @@ If no research sources are in context, delete `sources.md`.
|
||||
|
||||
### Step 5 — Validate and close
|
||||
|
||||
Run this checklist before invoking the audit:
|
||||
Run this checklist before invoking the audit.
|
||||
|
||||
**Every file, at every scope:**
|
||||
- [ ] `name` present and kebab-case; `description` present
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] No `FILL IN:` placeholders and no `<!-- -->` template comments remain
|
||||
|
||||
**Plugin/APM scope — single file (`<name>.agent.md`):**
|
||||
- [ ] `name` field present, kebab-case, unique in scope
|
||||
- [ ] `description` field present and action-first
|
||||
- [ ] Frontmatter contains ONLY `name`, `description`, and optionally `model` (plus `source_keys` if research-sourced) — no `tools`, `isolation`, `maxTurns`, `effort`, `memory`, `permissionMode`, `disallowedTools`, `skills`, `color`, `initialPrompt`, `background`, `hooks`, or `mcpServers`
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] No `FILL IN:` placeholders remain
|
||||
- [ ] No `<!-- -->` template comments remain in frontmatter
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
- [ ] Every frontmatter field is in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the single source of truth, read as data by `validate.sh`. As of 2026-08-14: `name`, `description`, `model`, `source_keys`, `disallowedTools`. Nothing else — in particular no `tools`
|
||||
- [ ] A read-only agent carries `disallowedTools` **and** says so in the body
|
||||
|
||||
**Project/user scope — Claude Code file (`<name>.md`):**
|
||||
- [ ] `name` field present, kebab-case, unique in scope
|
||||
- [ ] `description` field present and action-first
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] No `FILL IN:` placeholders remain
|
||||
- [ ] No `<!-- -->` template comments remain in frontmatter
|
||||
- [ ] `name` unique in scope; `description` action-first
|
||||
|
||||
**Project/user scope — Copilot CLI file (`<name>.agent.md`):**
|
||||
- [ ] File extension is `.agent.md` (not `.md`)
|
||||
- [ ] `name` field matches the filename stem (e.g. `name: my-agent` in `my-agent.agent.md`)
|
||||
- [ ] `description` field present
|
||||
- [ ] `name` matches the filename stem (e.g. `name: my-agent` in `my-agent.agent.md`)
|
||||
- [ ] No Claude Code-only fields (`maxTurns`, `isolation`, `memory`, `permissionMode`, `effort`, `hooks`, `mcpServers`)
|
||||
- [ ] System prompt body present and non-empty
|
||||
- [ ] Body does not exceed 30,000 characters
|
||||
- [ ] No `<!-- -->` template comments remain in frontmatter
|
||||
|
||||
At plugin/APM scope, apply a **minor bump** to the resolved package's `apm.yml` `version` (single manifest, e.g. `1.0.4` → `1.1.0`).
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the created file(s) before closing — validates the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` before Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the new agent files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the agent as done once the hash has actually changed.
|
||||
**Commit verification.** Once the audit is clean, run `git add` and `git commit` for the new agent files — do not stop at staging. Then confirm `git log --oneline -1` differs from the hash captured before Step 1. A non-empty `git diff --stat` is not proof of completion: staged-but-uncommitted work is part of no commit and can be silently lost if the working tree is cleaned up. Only report the agent as done once the hash has actually changed.
|
||||
|
||||
## Improving an existing agent
|
||||
|
||||
@@ -223,9 +221,7 @@ Confirm the agent files exist and at least one improvement signal is present in
|
||||
|
||||
If no signals: "This skill applies existing signals to an agent. For a blind review, examine the files manually or run a grill session first."
|
||||
|
||||
Verify `kyberforge:agent-audit` is available — it ships with the kyberforge plugin and is co-installed with this skill. If unavailable, stop and tell the user to install the kyberforge plugin before continuing.
|
||||
|
||||
Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
Verify `kyberforge:agent-audit` is available, as in the create flow's Prerequisites. Capture `git log --oneline -1` now, before making any edits — Step 5 needs it to verify a real commit landed.
|
||||
|
||||
**Partial state (project/user scope only)** — if one provider file exists but not the other, scaffold the missing one (`bash scripts/new-agent.sh <name> <root>`, file-by-file no-op) then continue. Doesn't apply at plugin/APM scope — single file, no partial-pair state.
|
||||
|
||||
@@ -248,7 +244,7 @@ Before editing, state which root causes were identified, what evidence supports
|
||||
|
||||
### Step 4 — Apply changes
|
||||
|
||||
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced. For a plugin/APM-scope single file, verify no field beyond `name`, `description`, `model`, and `source_keys` is introduced.
|
||||
Edit any file the signals point to. Generalize the fix — find the underlying gap, not the specific example that failed. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused definition consistently outperforms an exhaustive one. For Copilot files, verify no Claude Code-only fields are introduced. For a plugin/APM-scope single file, verify every field is still in the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, and that an existing `disallowedTools` fence was not dropped by the edit.
|
||||
|
||||
If the edit adds or removes research-sourced content, update `source_keys` in the edited file(s) and the corresponding entry in `sources.md` per Create flow's Step 4.
|
||||
|
||||
@@ -260,4 +256,4 @@ At plugin/APM scope, apply a **patch bump** to the resolved package's `apm.yml`
|
||||
|
||||
Invoke `kyberforge:agent-audit` on the edited file(s) to confirm no regressions — the pair at project/user scope, the single file at plugin/APM scope.
|
||||
|
||||
**Commit verification.** Capture `git log --oneline -1` at the start of Step 1 and keep it. Once the audit is clean, run `git add` and `git commit` for the changed files — do not stop at staging. Then run `git log --oneline -1` again and confirm the hash changed from the one you captured at the start. A non-empty `git diff --stat` is not sufficient proof of completion: staged-but-uncommitted work isn't part of any commit and can be silently lost if the working tree is cleaned up before a commit lands. Only report the improvement as done once the hash has actually changed.
|
||||
**Commit verification.** Exactly as in the create flow's Step 5, against the hash captured at Step 1: commit the changed files once the audit is clean, and only report the improvement as done once `git log --oneline -1` shows a different hash.
|
||||
|
||||
Reference in New Issue
Block a user