Defame1297
ede3f06689
Diffing each retrofitted SKILL.md against its replacement references/ files found rules that existed on main and now existed nowhere — relocated in intent, deleted in fact. A trim that loses a rule is not progressive disclosure, it is data loss with a smaller word count. Three had no survivor. The least-privilege guidance for `tools` kept its mechanics and lost the "restrict to what the agent needs" half, so the remaining text read as encouragement to omit the field. The improve flow lost its regression check, so nothing compared the closing audit against the pre-edit state and a PASS quietly becoming a SUGGESTION went unnoticed — restored on both halves of the author pair, since agent-author had dropped its equivalent too. And agent bodies lost "would the agent get this wrong without it?", which mattered more than it looks: ADR-0020 deliberately sets no body word gate for agents, three of the four already sit between 933 and 1,199 words, and the delegation check only fires on procedure a skill already owns. That heuristic was the only brake left. Two more were reachable only from the wrong scope. agent-author tells the reader to load only the file for the resolved scope, but the mcp__ glob syntax for disallowedTools and the five tools no subagent ever receives had both landed in project-user-scope.md. disallowedTools is the ONLY permitted fence at plugin/APM scope, so the scope that needs the syntax most could not reach it, and a plugin-scope run could write a body telling the agent to ask the user a question. Two documents were actively wrong rather than merely thin. agent-audit told auditors that validate.sh resolves boundary targets for skills only; it runs at both scopes, so the auditor was hand-resolving what the script had already decided and could contradict it. And skill-audit routed to its script-troubleshooting reference whenever validate.sh "fails" — but it exits 1 on ordinary content FAILs, the normal outcome for the whole #99 population, so 1,302 words loaded on nearly every audit. A context-budget regression inside the skill that enforces the context budget. Finally, two illustrations taught the shape the gate ERRORs on, unfenced, while an adjacent rubric called it a hard ERROR. LESSONS.md records the reference-chain depth rule flipping from "one level deep" to "two hops, never three". ADR-0020 is silent on it and the reversal rode entirely on the diff; the looser rule is what mandatory dispatch requires. Refs: #99 ADR: 0020 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015W3iwF9ncfRZddGBxsMCYi
4.2 KiB
source_keys
| source_keys | |
|---|---|
|
Improving an existing agent
Return to SKILL.md Step 4 once Step 4 below is done — validation, the version bump and commit
verification are shared with the create flow and are not repeated here.
Step 1 — Verify inputs
Confirm the agent file (or, at project and user scope, the pair) exists and that at least one improvement signal is present in the conversation or in a referenced file.
If no signals are present, stop: "This skill applies existing signals to an agent. For a blind
review, run agent-audit instead."
agent-audit runs the validation in SKILL.md Step 4 and is co-installed with this skill; if
it is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
Partial pair — project and user scope only. If one provider file exists and the other does
not, scaffold the missing one with bash scripts/new-agent.sh <name> <root> (file-by-file no-op)
and continue. Plugin/APM scope is a single file and has no partial state.
Step 2 — Gather and group signals
Read the current file(s), then collect every signal from the conversation and from any path the user referenced.
Group signals by root cause, not by symptom. Patching per symptom is the default failure mode: three complaints often trace to one missing instruction. Ask: "What single gap in this agent causes this cluster?" One root cause, one fix.
Example:
- User feedback: the agent keeps trying to push to the remote
- Session context: no scope boundary in the system prompt
→ Root cause: the system prompt has no git scope constraint → fix: add an explicit boundary
Step 3 — Announce planned changes
Before editing, state which root causes were identified, what evidence supports each, and which files will change. Then proceed — edits are reversible via git, so no approval checkpoint is needed.
Step 4 — Apply changes
Edit whichever file the signals point to.
Generalize, do not patch. Fix the underlying gap, not the one example that failed. A fix scoped to the cases you have seen overfits and performs worse on new input.
Delegate rather than grow. An agent body has no word ceiling, but a body that restates a
procedure a skill it can invoke already owns is an agent-audit FAIL. When a signal reports a
missing procedure, check first whether an installed skill owns it and name that skill instead of
transcribing it. See references/contract.md.
The delegation check is not a length brake — it fires only on procedure an invocable skill already owns, and says nothing about original prose. That brake is judgment, and it is the only one left: for every sentence you add, ask "would the agent get this wrong without it?" and delete it if the answer is no.
Explain the why. Reasoning-based instructions outperform rigid directives. A rule written in all caps (ALWAYS/NEVER) is usually better reframed as why the behaviour matters, so the agent can apply judgment at the edges.
Retrofit before extending. Any agent predating ADR-0020 has to meet the description contract
before any other edit lands — the gates are hot and carry no baseline file, so a one-line fix to a
non-compliant agent cannot be committed until its description meets references/contract.md.
Treat that retrofit as part of the same change, not a follow-up.
Re-check the scope rules. Read the reference for the resolved scope (SKILL.md Step 2) and
confirm the edit introduced no field that scope forbids, and dropped no disallowedTools fence
that was already there.
If the edit adds or removes research-sourced content, update source_keys in the edited file and
the matching sources.md entry — the create flow's Step 3 has the rules.
Check for regressions before handing back. SKILL.md Step 4 tells you to resolve every FAIL,
which says nothing about a check that passed before these edits and no longer does. Compare the
closing agent-audit against the agent's pre-edit state — a PASS that has become a SUGGESTION, or
a SUGGESTION that has become a FAIL, is damage this flow caused and is in scope for it. Only the
improve flow can make that comparison; the create flow has no prior state to compare against.
Then return to SKILL.md Step 4.