diff --git a/plugins/kyberforge/skills/skill-improve/README.md b/plugins/kyberforge/skills/skill-improve/README.md new file mode 100644 index 0000000..ac299b2 --- /dev/null +++ b/plugins/kyberforge/skills/skill-improve/README.md @@ -0,0 +1,25 @@ +# skill-improve + +Apply evidence-based improvements to an existing skill using signals from the current session. + +## What it does + +Takes an existing skill directory and one or more improvement signals — grill output, audit findings, eval failures, human feedback, or inline session context — and applies targeted edits to the skill files. Groups signals by root cause before editing to avoid per-symptom patching. Hands off to `/skill-audit` when done. + +## Before you start + +Have at least one improvement signal ready: a grill session output, `/skill-audit` report, eval results, feedback.json, or inline feedback describing what went wrong. If you have no signals and want a blind review, use `/skill-audit` instead. + +## Usage + +``` +/skill-improve +``` + +Provide the skill directory path and any signal files in the conversation. The skill gathers signals from context — you don't need to paste them in a specific format. + +## Files + +| File | Purpose | +|------|---------| +| `SKILL.md` | Skill instructions for agents | diff --git a/plugins/kyberforge/skills/skill-improve/SKILL.md b/plugins/kyberforge/skills/skill-improve/SKILL.md new file mode 100644 index 0000000..15d5317 --- /dev/null +++ b/plugins/kyberforge/skills/skill-improve/SKILL.md @@ -0,0 +1,77 @@ +--- +name: skill-improve +description: > + Apply evidence-based improvements to an existing skill. Use when the user + says "improve this skill", "fix this skill based on feedback", "apply these + audit findings", or "update the skill based on this grill output". Also use + when the user provides inline feedback about a skill's behavior and wants it + applied, or when a grill session, eval run, or audit has produced findings + the user wants acted on — even if they don't say "improve" explicitly. Do + not use when the user wants a read-only review only — use /skill-audit + instead. Do not use when no improvement signals are present in the + conversation or referenced files. +allowed-tools: Bash Read Write Edit +metadata: + category: factory +--- + +## Gotchas + +- Patching per symptom is the default failure mode. Three eval failures may all trace to one missing instruction — always identify the root cause before editing. +- Do not create new scripts unless a signal explicitly calls for it. Writing scripts from scratch requires transcript analysis that is out of scope here; flag the opportunity as a suggestion instead. +- If no improvement signals are present, do not proceed — redirect to `/skill-audit` for a blind review. + +## Step 1 — Verify inputs + +Confirm: +- [ ] Skill directory path is provided and the directory exists +- [ ] At least one improvement signal is present in the conversation or a referenced file + +If the skill dir is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use `/skill-audit` instead." + +Signals can come from anywhere in the conversation or referenced files: +- Grill session output (most common predecessor in the factory sequence) +- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list) +- Eval assertion failures (grading.json, benchmark.json) +- Human feedback (feedback.json, inline in conversation, PR or issue comments) +- Session context describing what went wrong + +## Step 2 — Gather and group signals + +Read the current skill files (SKILL.md and any files in scripts/, references/, assets/). Then collect all signals from the conversation and any file paths the user has referenced. + +Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom. + +Example: +- Eval fails because output format is wrong +- Audit finding: no output template defined +- User feedback: "I always have to ask it to format the output" +- Root cause: SKILL.md has no output format specification → one fix: add an output template + +## Step 3 — Announce planned changes + +Before editing, state: +- Which root causes were identified and what evidence supports each +- Which files will be changed and what will change in each + +Then proceed — edits are reversible via git, no approval checkpoint needed. + +## Step 4 — Apply changes + +Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, README.md. + +**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs. + +**Keep it lean.** Remove instructions that aren't pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one. + +**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases. + +**Fix the root, not the symptom.** If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md. + +**On scripts**: Fix and edit existing scripts freely when signals point to them. + +## Step 5 — Validate and close + +Run `/skill-audit` on the skill directory. Resolve any FAIL findings before considering the improvement complete. + +If the skill has no `evals/` directory, note it after the audit: "No evals found — consider running `/write-eval` to add test coverage so future improvement cycles have quantitative signals to work from."