feat(kyberforge): add skill-improve skill to factory workflow
Applies evidence-based improvements to existing skills using signals from grill sessions, audit reports, eval failures, and inline feedback. Groups signals by root cause before editing to avoid per-symptom patching. Hands off to /skill-audit on completion. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
25
plugins/kyberforge/skills/skill-improve/README.md
Normal file
25
plugins/kyberforge/skills/skill-improve/README.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# skill-improve
|
||||
|
||||
Apply evidence-based improvements to an existing skill using signals from the current session.
|
||||
|
||||
## What it does
|
||||
|
||||
Takes an existing skill directory and one or more improvement signals — grill output, audit findings, eval failures, human feedback, or inline session context — and applies targeted edits to the skill files. Groups signals by root cause before editing to avoid per-symptom patching. Hands off to `/skill-audit` when done.
|
||||
|
||||
## Before you start
|
||||
|
||||
Have at least one improvement signal ready: a grill session output, `/skill-audit` report, eval results, feedback.json, or inline feedback describing what went wrong. If you have no signals and want a blind review, use `/skill-audit` instead.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
/skill-improve
|
||||
```
|
||||
|
||||
Provide the skill directory path and any signal files in the conversation. The skill gathers signals from context — you don't need to paste them in a specific format.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
77
plugins/kyberforge/skills/skill-improve/SKILL.md
Normal file
77
plugins/kyberforge/skills/skill-improve/SKILL.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: skill-improve
|
||||
description: >
|
||||
Apply evidence-based improvements to an existing skill. Use when the user
|
||||
says "improve this skill", "fix this skill based on feedback", "apply these
|
||||
audit findings", or "update the skill based on this grill output". Also use
|
||||
when the user provides inline feedback about a skill's behavior and wants it
|
||||
applied, or when a grill session, eval run, or audit has produced findings
|
||||
the user wants acted on — even if they don't say "improve" explicitly. Do
|
||||
not use when the user wants a read-only review only — use /skill-audit
|
||||
instead. Do not use when no improvement signals are present in the
|
||||
conversation or referenced files.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
category: factory
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Patching per symptom is the default failure mode. Three eval failures may all trace to one missing instruction — always identify the root cause before editing.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing scripts from scratch requires transcript analysis that is out of scope here; flag the opportunity as a suggestion instead.
|
||||
- If no improvement signals are present, do not proceed — redirect to `/skill-audit` for a blind review.
|
||||
|
||||
## Step 1 — Verify inputs
|
||||
|
||||
Confirm:
|
||||
- [ ] Skill directory path is provided and the directory exists
|
||||
- [ ] At least one improvement signal is present in the conversation or a referenced file
|
||||
|
||||
If the skill dir is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Eval assertion failures (grading.json, benchmark.json)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/). Then collect all signals from the conversation and any file paths the user has referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
|
||||
|
||||
Example:
|
||||
- Eval fails because output format is wrong
|
||||
- Audit finding: no output template defined
|
||||
- User feedback: "I always have to ask it to format the output"
|
||||
- Root cause: SKILL.md has no output format specification → one fix: add an output template
|
||||
|
||||
## Step 3 — Announce planned changes
|
||||
|
||||
Before editing, state:
|
||||
- Which root causes were identified and what evidence supports each
|
||||
- Which files will be changed and what will change in each
|
||||
|
||||
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, README.md.
|
||||
|
||||
**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs.
|
||||
|
||||
**Keep it lean.** Remove instructions that aren't pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
|
||||
|
||||
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
|
||||
|
||||
**Fix the root, not the symptom.** If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
|
||||
|
||||
**On scripts**: Fix and edit existing scripts freely when signals point to them.
|
||||
|
||||
## Step 5 — Validate and close
|
||||
|
||||
Run `/skill-audit` on the skill directory. Resolve any FAIL findings before considering the improvement complete.
|
||||
|
||||
If the skill has no `evals/` directory, note it after the audit: "No evals found — consider running `/write-eval` to add test coverage so future improvement cycles have quantitative signals to work from."
|
||||
Reference in New Issue
Block a user