ADR-0020 shipped its gates hot with no baseline file, so 26 of 39 descriptions and 9 of 39 bodies are over their FAIL tier and editing any of them for any reason requires bringing the skill into contract first. `references/improve.md` said exactly that and stopped there — it mandated a retrofit and supplied no procedure for one. Four dry-run retrofits confirmed what that costs. Asked the same questions — what to cut first, when a body is two flows rather than one, what else has to change alongside — they invented six to ten different answers, so the same skill retrofitted twice produced two different skills and neither run could be reviewed against anything. `references/retrofit.md` fixes the answers: an ordered cut list ranked by tokens removed against behaviour lost (inverting that order is how a retrofit deletes the instruction the skill existed to carry), the test for whether a body holds two mutually exclusive flows, the reference-file conventions, the collateral checklist for `README.md` and `references/sources.md`, and a worked description retrofit. It also states the trap the dry runs kept hitting: retrofit the skill in place, inside its package. The boundary-target universe is built by walking up from the file being checked, so a scratch copy has no authoring root above it, the check prints `INFO ... DID NOT RUN`, and the run still exits 0 — a line that reads as a pass and is not one. A retrofit signed off on a copy carries an unverified boundary target into the corpus. Loaded from the improve flow only when a budget is actually exceeded, so a routine improvement pays nothing for it. Refs: ADR-0020, #99
3.9 KiB
source_keys
| source_keys | |||
|---|---|---|---|
|
Improving an existing skill
Return to SKILL.md Step 4 once Step 4 below is done — validation, versioning and commit
verification are shared with the create flow and are not repeated here.
Step 1 — Verify inputs
Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
existing signals to a skill. For a blind review without signals, use /skill-audit instead."
Signals can come from anywhere in the conversation or referenced files:
- Grill session output (most common predecessor in the factory sequence)
/skill-auditfindings (PASS/FAIL/SUGGESTION punch list)- Human feedback (feedback.json, inline in conversation, PR or issue comments)
- Session context describing what went wrong
Also verify the name field in frontmatter matches the skill's directory name exactly.
Step 2 — Gather and group signals
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/,
tests/). Then collect all signals from the conversation and any file paths the user has
referenced.
Group signals by root cause, not symptom. Patching per symptom is the default failure mode: three eval failures may all trace to one missing instruction. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
Example:
- Session context: output format is wrong on every run
- Audit finding: no output template defined
- User feedback: "I always have to ask it to format the output"
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
Step 3 — Announce planned changes
Before editing, state:
- Which root causes were identified and what evidence supports each
- Which files will be changed and what will change in each
Then proceed — edits are reversible via git, no approval checkpoint needed.
Step 4 — Apply changes
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/,
references/, assets/, tests/, README.md.
Generalize, do not patch. Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you have seen will overfit and perform worse on new inputs.
Keep it lean. Remove instructions that are not pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
Explain the why. Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
Retrofit before extending. Any edit to a skill that predates ADR-0020 has to bring it into the
contract first — the gates are hot and carry no baseline file, so a one-line fix to a
non-compliant skill cannot be committed until the description and body meet
references/contract.md. Treat that retrofit as part of the same change, not a follow-up.
If the skill's description exceeds 250 characters, or its body-only word count exceeds 600, read
references/retrofit.md before editing. It carries the ordered cut procedure, the
mutually-exclusive-flows test, the reference-file conventions this flow needs, the collateral
checklist for README.md and references/sources.md, and a worked description retrofit. Do not
improvise the cuts — four dry runs invented six to ten different answers to the same questions.
If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
Then return to SKILL.md Step 4.