Defame1297
ede3f06689
Diffing each retrofitted SKILL.md against its replacement references/ files found rules that existed on main and now existed nowhere — relocated in intent, deleted in fact. A trim that loses a rule is not progressive disclosure, it is data loss with a smaller word count. Three had no survivor. The least-privilege guidance for `tools` kept its mechanics and lost the "restrict to what the agent needs" half, so the remaining text read as encouragement to omit the field. The improve flow lost its regression check, so nothing compared the closing audit against the pre-edit state and a PASS quietly becoming a SUGGESTION went unnoticed — restored on both halves of the author pair, since agent-author had dropped its equivalent too. And agent bodies lost "would the agent get this wrong without it?", which mattered more than it looks: ADR-0020 deliberately sets no body word gate for agents, three of the four already sit between 933 and 1,199 words, and the delegation check only fires on procedure a skill already owns. That heuristic was the only brake left. Two more were reachable only from the wrong scope. agent-author tells the reader to load only the file for the resolved scope, but the mcp__ glob syntax for disallowedTools and the five tools no subagent ever receives had both landed in project-user-scope.md. disallowedTools is the ONLY permitted fence at plugin/APM scope, so the scope that needs the syntax most could not reach it, and a plugin-scope run could write a body telling the agent to ask the user a question. Two documents were actively wrong rather than merely thin. agent-audit told auditors that validate.sh resolves boundary targets for skills only; it runs at both scopes, so the auditor was hand-resolving what the script had already decided and could contradict it. And skill-audit routed to its script-troubleshooting reference whenever validate.sh "fails" — but it exits 1 on ordinary content FAILs, the normal outcome for the whole #99 population, so 1,302 words loaded on nearly every audit. A context-budget regression inside the skill that enforces the context budget. Finally, two illustrations taught the shape the gate ERRORs on, unfenced, while an adjacent rubric called it a hard ERROR. LESSONS.md records the reference-chain depth rule flipping from "one level deep" to "two hops, never three". ADR-0020 is silent on it and the reversal rode entirely on the diff; the looser rule is what mandatory dispatch requires. Refs: #99 ADR: 0020 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015W3iwF9ncfRZddGBxsMCYi
4.4 KiB
source_keys
| source_keys | |||
|---|---|---|---|
|
Improving an existing skill
Return to SKILL.md Step 4 once Step 4 below is done — validation, versioning and commit
verification are shared with the create flow and are not repeated here.
Step 1 — Verify inputs
Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
existing signals to a skill. For a blind review without signals, use /skill-audit instead."
Signals can come from anywhere in the conversation or referenced files:
- Grill session output (most common predecessor in the factory sequence)
/skill-auditfindings (PASS/FAIL/SUGGESTION punch list)- Human feedback (feedback.json, inline in conversation, PR or issue comments)
- Session context describing what went wrong
Also verify the name field in frontmatter matches the skill's directory name exactly.
Step 2 — Gather and group signals
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/,
tests/). Then collect all signals from the conversation and any file paths the user has
referenced.
Group signals by root cause, not symptom. Patching per symptom is the default failure mode: three eval failures may all trace to one missing instruction. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
Example:
- Session context: output format is wrong on every run
- Audit finding: no output template defined
- User feedback: "I always have to ask it to format the output"
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
Step 3 — Announce planned changes
Before editing, state:
- Which root causes were identified and what evidence supports each
- Which files will be changed and what will change in each
Then proceed — edits are reversible via git, no approval checkpoint needed.
Step 4 — Apply changes
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/,
references/, assets/, tests/, README.md.
Generalize, do not patch. Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you have seen will overfit and perform worse on new inputs.
Keep it lean. Remove instructions that are not pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
Explain the why. Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
Retrofit before extending. Any edit to a skill that predates ADR-0020 has to bring it into the
contract first — the gates are hot and carry no baseline file, so a one-line fix to a
non-compliant skill cannot be committed until the description and body meet
references/contract.md. Treat that retrofit as part of the same change, not a follow-up.
If the skill's description exceeds 250 characters, or its body-only word count exceeds 600, read
references/retrofit.md before editing. It carries the ordered cut procedure, the
mutually-exclusive-flows test, the reference-file conventions this flow needs, the collateral
checklist for README.md and references/sources.md, and a worked description retrofit. Do not
improvise the cuts — four dry runs invented six to ten different answers to the same questions.
If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
Check for regressions before handing back. SKILL.md Step 4 tells you to resolve every FAIL,
which says nothing about a check that passed before these edits and no longer does. Compare the
closing audit against the skill's pre-edit state — a PASS that has become a SUGGESTION, or a
SUGGESTION that has become a FAIL, is damage this flow caused and is in scope for it. Only the
improve flow can make that comparison; the create flow has no prior state to compare against.
Then return to SKILL.md Step 4.