Defame1297
3c35f2fbd5
- Rewrite assets/templates/tests/README.md with correct repo-root context and SKILL_NAME placeholder in bats run command (was: `bats tests/`, wrong CWD) - Add sed substitution for tests/README.md in new-skill.sh so SKILL_NAME is replaced in scaffolded test docs; test added to new-skill.bats (red→green) - Remove /write-eval reference from skill-improve SKILL.md; reword as direct action since the skill does not exist in the kyberforge plugin - Remove self-referential README.md rows from skill-audit and skill-improve Files tables to match template and skill-write convention Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
77 lines
4.1 KiB
Markdown
77 lines
4.1 KiB
Markdown
---
|
|
name: skill-improve
|
|
description: >
|
|
Use when the user says "improve this skill", "fix this skill based on
|
|
feedback", "apply these audit findings", or "update the skill based on this
|
|
grill output". Also use when the user provides inline feedback about a skill's
|
|
behavior and wants it applied, or when a grill session, eval run, or audit has
|
|
produced findings the user wants acted on — even if they don't say "improve"
|
|
explicitly. Applies evidence-based improvements to an existing skill. Do not
|
|
use when the user wants a read-only review only — use /skill-audit instead.
|
|
Do not use when no improvement signals are present in the conversation or
|
|
referenced files.
|
|
allowed-tools: Bash Read Write Edit
|
|
metadata:
|
|
category: factory
|
|
---
|
|
|
|
## Gotchas
|
|
|
|
- Patching per symptom is the default failure mode. Three eval failures may all trace to one missing instruction — always identify the root cause before editing.
|
|
- Do not create new scripts unless a signal explicitly calls for it. Writing scripts from scratch requires transcript analysis that is out of scope here; flag the opportunity as a suggestion instead.
|
|
|
|
## Step 1 — Verify inputs
|
|
|
|
Confirm the skill directory path exists and that at least one improvement signal is present in the conversation or a referenced file.
|
|
|
|
If the skill dir is missing, ask for it. If no signals are present, stop: "This skill applies existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
|
|
|
Signals can come from anywhere in the conversation or referenced files:
|
|
- Grill session output (most common predecessor in the factory sequence)
|
|
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
|
- Eval assertion failures (grading.json, benchmark.json)
|
|
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
|
- Session context describing what went wrong
|
|
|
|
## Step 2 — Gather and group signals
|
|
|
|
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/, tests/). Then collect all signals from the conversation and any file paths the user has referenced.
|
|
|
|
Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
|
|
|
|
```text
|
|
Example:
|
|
- Eval fails because output format is wrong
|
|
- Audit finding: no output template defined
|
|
- User feedback: "I always have to ask it to format the output"
|
|
→ Root cause: SKILL.md has no output format specification → one fix: add an output template
|
|
```
|
|
|
|
## Step 3 — Announce planned changes
|
|
|
|
Before editing, state:
|
|
- Which root causes were identified and what evidence supports each
|
|
- Which files will be changed and what will change in each
|
|
|
|
Then proceed — edits are reversible via git, no approval checkpoint needed.
|
|
|
|
## Step 4 — Apply changes
|
|
|
|
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, tests/, README.md.
|
|
|
|
**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs.
|
|
|
|
**Keep it lean.** Remove instructions that aren't pulling their weight. For every sentence you add, ask: "Would the agent get this wrong without it?" A shorter, focused skill consistently outperforms an exhaustive one.
|
|
|
|
**Explain the why.** Reasoning-based instructions outperform rigid directives. If you find yourself writing a rule in all caps (ALWAYS/NEVER), reframe it: explain why the behavior matters so the agent can apply judgment in edge cases.
|
|
|
|
If a signal points to a script or reference file, edit that file directly rather than adding a workaround in SKILL.md.
|
|
|
|
**On scripts**: Fix and edit existing scripts freely when signals point to them.
|
|
|
|
## Step 5 — Validate and close
|
|
|
|
Run `/skill-audit` on the skill directory. Resolve any FAIL findings before considering the improvement complete.
|
|
|
|
If the skill has no `evals/` directory, note it after the audit: "No evals found — consider adding an `evals/` directory with assertion-based test cases to give future improvement cycles quantitative signals to work from."
|