Files

Defame1297 2540e50fcc feat(kyberforge): give skill-author a procedure for the #99 retrofit

ADR-0020 shipped its gates hot with no baseline file, so 26 of 39 descriptions
and 9 of 39 bodies are over their FAIL tier and editing any of them for any
reason requires bringing the skill into contract first. `references/improve.md`
said exactly that and stopped there — it mandated a retrofit and supplied no
procedure for one.

Four dry-run retrofits confirmed what that costs. Asked the same questions —
what to cut first, when a body is two flows rather than one, what else has to
change alongside — they invented six to ten different answers, so the same skill
retrofitted twice produced two different skills and neither run could be reviewed
against anything.

`references/retrofit.md` fixes the answers: an ordered cut list ranked by tokens
removed against behaviour lost (inverting that order is how a retrofit deletes the
instruction the skill existed to carry), the test for whether a body holds two
mutually exclusive flows, the reference-file conventions, the collateral checklist
for `README.md` and `references/sources.md`, and a worked description retrofit.

It also states the trap the dry runs kept hitting: retrofit the skill in place,
inside its package. The boundary-target universe is built by walking up from the
file being checked, so a scratch copy has no authoring root above it, the check
prints `INFO ... DID NOT RUN`, and the run still exits 0 — a line that reads as a
pass and is not one. A retrofit signed off on a copy carries an unverified
boundary target into the corpus.

Loaded from the improve flow only when a budget is actually exceeded, so a routine
improvement pays nothing for it.

Refs: ADR-0020, #99

2026-08-16 16:40:19 +00:00

3.9 KiB

Raw Blame History

source_keys

agentskills-best-practices

agentskills-evaluating-skills

agentskills-optimizing-descriptions

Improving an existing skill

Return to SKILL.md Step 4 once Step 4 below is done — validation, versioning and commit verification are shared with the create flow and are not repeated here.

Step 1 — Verify inputs

Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.

If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use /skill-audit instead."

Signals can come from anywhere in the conversation or referenced files:

Grill session output (most common predecessor in the factory sequence)
/skill-audit findings (PASS/FAIL/SUGGESTION punch list)
Human feedback (feedback.json, inline in conversation, PR or issue comments)
Session context describing what went wrong

Also verify the name field in frontmatter matches the skill's directory name exactly.

Step 2 — Gather and group signals

Read the current skill files (SKILL.md and any files in scripts/, references/, assets/, tests/). Then collect all signals from the conversation and any file paths the user has referenced.

Group signals by root cause, not symptom. Patching per symptom is the default failure mode: three eval failures may all trace to one missing instruction. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.

Example:
- Session context: output format is wrong on every run
- Audit finding: no output template defined
- User feedback: "I always have to ask it to format the output"
→ Root cause: SKILL.md has no output format specification → one fix: add an output template

Step 3 — Announce planned changes

Before editing, state:

Which root causes were identified and what evidence supports each
Which files will be changed and what will change in each

Then proceed — edits are reversible via git, no approval checkpoint needed.

Step 4 — Apply changes

Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, tests/, README.md.

Generalize, do not patch. Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you have seen will overfit and perform worse on new inputs.

Keep it lean. Remove instructions that are not pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.

Explain the why. Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.

Retrofit before extending. Any edit to a skill that predates ADR-0020 has to bring it into the contract first — the gates are hot and carry no baseline file, so a one-line fix to a non-compliant skill cannot be committed until the description and body meet references/contract.md. Treat that retrofit as part of the same change, not a follow-up.

If the skill's description exceeds 250 characters, or its body-only word count exceeds 600, read references/retrofit.md before editing. It carries the ordered cut procedure, the mutually-exclusive-flows test, the reference-file conventions this flow needs, the collateral checklist for README.md and references/sources.md, and a worked description retrofit. Do not improvise the cuts — four dry runs invented six to ten different answers to the same questions.

If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.

Then return to SKILL.md Step 4.

3.9 KiB Raw Blame History