Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat(kyberforge): give skill-author a procedure for the #99 retrofit

Browse Source 
ADR-0020 shipped its gates hot with no baseline file, so 26 of 39 descriptions
and 9 of 39 bodies are over their FAIL tier and editing any of them for any
reason requires bringing the skill into contract first. `references/improve.md`
said exactly that and stopped there — it mandated a retrofit and supplied no
procedure for one.

Four dry-run retrofits confirmed what that costs. Asked the same questions —
what to cut first, when a body is two flows rather than one, what else has to
change alongside — they invented six to ten different answers, so the same skill
retrofitted twice produced two different skills and neither run could be reviewed
against anything.

`references/retrofit.md` fixes the answers: an ordered cut list ranked by tokens
removed against behaviour lost (inverting that order is how a retrofit deletes the
instruction the skill existed to carry), the test for whether a body holds two
mutually exclusive flows, the reference-file conventions, the collateral checklist
for `README.md` and `references/sources.md`, and a worked description retrofit.

It also states the trap the dry runs kept hitting: retrofit the skill in place,
inside its package. The boundary-target universe is built by walking up from the
file being checked, so a scratch copy has no authoring root above it, the check
prints `INFO ... DID NOT RUN`, and the run still exits 0 — a line that reads as a
pass and is not one. A retrofit signed off on a copy carries an unverified
boundary target into the corpus.

Loaded from the improve flow only when a budget is actually exceeded, so a routine
improvement pays nothing for it.

Refs: ADR-0020, #99
This commit is contained in:
Defame1297
2026-08-16 16:40:19 +00:00
parent a85bdbed42
commit 2540e50fcc
8 changed files with 322 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
plugins/kyberforge/.apm/skills/skill-author/README.md
View File

@@ -47,6 +47,7 @@ If the destination resolves inside an APM package, read `references/deployment-m
| `references/create.md` | The create flow end to end — prerequisites, package-intent gate, scaffold, frontmatter, scripts, references, sources (loaded on demand) |
| `references/improve.md` | The improve flow end to end — signal verification, root-cause grouping, announcement, edits (loaded on demand) |
| `references/contract.md` | The ADR-0020 description and body contract, the Gotchas constraint, the two size gates, body patterns, and org-policy embedding (loaded on demand) |
| `references/retrofit.md` | Bringing a pre-ADR-0020 skill into contract — ordered cut procedure, the mutually-exclusive-flows test, reference-file conventions, the collateral checklist, and a worked description retrofit (loaded from the improve flow when a budget is exceeded) |
| `references/deployment-modes.md` | APM package vs standalone differences and self-containment/cache-isolation rules (loaded on demand) |
| `references/scripts.md` | Package runners, inline dependency patterns, and full script contract (loaded on demand) |
| `references/sources.md` | Upstream research sources and which skill files each contributed to |

6
plugins/kyberforge/.apm/skills/skill-author/references/improve.md
View File

@@ -76,6 +76,12 @@ contract first — the gates are hot and carry no baseline file, so a one-line f
non-compliant skill cannot be committed until the description and body meet
`references/contract.md`. Treat that retrofit as part of the same change, not a follow-up.
If the skill's description exceeds 250 characters, or its body-only word count exceeds 600, read
`references/retrofit.md` before editing. It carries the ordered cut procedure, the
mutually-exclusive-flows test, the reference-file conventions this flow needs, the collateral
checklist for `README.md` and `references/sources.md`, and a worked description retrofit. Do not
improvise the cuts — four dry runs invented six to ten different answers to the same questions.
If a signal points to a script or reference file, edit that file directly rather than adding a
workaround in SKILL.md.

152
plugins/kyberforge/.apm/skills/skill-author/references/retrofit.md Normal file
View File

@@ -0,0 +1,152 @@
---
source_keys:
- agentskills-best-practices
- agentskills-optimizing-descriptions
---
# Retrofitting a skill to the ADR-0020 contract
Read this when `references/improve.md` Step 4 sends you here: the skill you are editing is over
the description or body budget and has to come into contract before any other change can be
committed. The gates are hot and carry no baseline file, so a one-line fix to a non-compliant
skill is blocked until this is done.
Measure first. Do not guess which gate fired: run `/skill-audit` on the directory and read its
`### Structure` dimension, which reports the description characters and the **body-only** word
count separately from the whole-file spec backstop. Retrofit against the number that actually
fired — a skill can sit a thousand words inside the whole-file backstop while failing the body
budget.
**Validate in place.** Audit the skill's real directory inside its package. Never audit a copy in a
scratch directory, and never move a skill out to work on it: the boundary-target universe is built
by walking up *from the file being checked*, so a copy with no authoring root above it resolves
against nothing and the check declines rather than running —
```text
INFO boundary-target resolution DID NOT RUN — no skill universe could be determined for
this path ... Unchecked target(s): totally-fake-target
```
The run still exits 0, so that line reads as a pass and is not one. Treat `DID NOT RUN` as **not
checked**, always. A retrofit signed off on a scratch copy carries an unverified boundary target
into the corpus, which is precisely the failure this gate exists to catch.
## Cut in this order
Work the list top down and stop as soon as the gate clears. The order is by ratio of tokens
removed to behaviour lost — inverting it is how a retrofit ends up deleting the one instruction
the skill existed to carry.
1. **Gotchas that paraphrase a step in the body below.** Zero information, and already a FAIL on
its own. Delete the Gotcha, keep the step.
2. **Spec restatements** — text that repeats a published specification, a tool's `--help`, or a
ceiling the validator already enforces. The agent gets this right without it. Delete, or move
the table to `references/` if a flow genuinely needs to look it up.
3. **Capability enumeration** — in a description, the feature list after the trigger clause; in a
body, the paragraph that recites what the skill can do. One capability clause survives in the
description; the rest belongs in `README.md`.
4. **Per-flow prose** — anything only one branch of the procedure ever reaches. This is the
largest single win in most bodies, and it is a *move*, not a delete: each flow gets its own
self-contained `references/` file, wired from a dispatch table.
If the body is still over after all four, the skill is doing two jobs. Split it, and say so
rather than compressing prose until it stops being readable.
## What "mutually exclusive flows" means
Two or more flows that a single invocation cannot both take. The three-way test, copied verbatim
from the body-discipline rubric `/skill-audit` judges against — nothing to load, it is quoted in
full here:
> separate subcommands, separate input types, separate lifecycle stages
Any one of the three is enough. Two flows that differ only in a parameter value are one flow.
At two or more mutually exclusive flows a dispatch table is **mandatory** regardless of word
count, because every invocation otherwise pays for every branch it did not take.
## Reference-file conventions
The create flow owns these rules, and this flow is forbidden from reading `references/create.md`,
so what a retrofit needs is restated here:
- **One topic per file.** A file mixing two concerns gets loaded for one of them and spends the
caller's context on the other.
- **Kebab-case filenames**, named after the topic rather than the flow that reads it —
`body-discipline.md`, not `step-3.md`.
- **Wire every file with the literal conditional form** ``If <condition>, read
`references/<file>.md` ``. A generic pointer ("see `references/` for details") is a Vale error.
- **Two hops from `SKILL.md`, never three.** A flow file may route on to a shared contract file;
a file reachable only through two intermediates is rarely loaded when it is needed.
- **`source_keys` frontmatter.** If the content you are moving drew on a research source, the new
file needs top-level `source_keys:` frontmatter listing those slugs, and every slug must already
exist as an `## <slug>` heading in `references/sources.md`. Moving sourced content out of
`SKILL.md` without carrying its slugs across breaks the provenance chain, and `/skill-audit`
reports the new file as an INFO with no `source_keys`.
## Collateral is mandatory, not optional
Moving content out of a `SKILL.md` leaves three files describing a structure that no longer
exists. `/skill-audit`'s provenance check exits clean on all three of these, so nothing catches
them for you. After every retrofit that adds, removes or renames a file:
- [ ] **`README.md` file table** — a row for every new `references/` file, and no row left for a
file that is gone. Say what triggers the load, not just what the file contains.
- [ ] **`references/README.md`**, where the skill has one — same update, same reason.
- [ ] **`references/sources.md` → `Contributing files`** — add the new file to every slug whose
content moved into it, and remove any file the retrofit deleted. This is the one that gets
missed: `sources.md` keeps citing sections of `SKILL.md` that no longer exist, the
provenance check still exits 0, and the stale claim survives review.
- [ ] Re-run `/skill-audit` and confirm its `### Provenance` dimension does not report the new
file as missing `source_keys`.
## Worked example — a description retrofit
`gitea-issues` before, 827 characters, the single most common shape in the corpus:
```text
Use when reading or writing Gitea issues: listing repo issues, getting a single issue's details/
comments/labels, creating an issue, updating its state, adding or editing comments, applying
labels via issue_write, or searching issues/PRs across repositories. Triggers on "create an
issue", "what issues are open", "get issue #N", "close issue #N", "comment on issue #N", "search
issues for X" — even when the user doesn't say "Gitea" explicitly. Composes gitea-labels-
milestones for all label inference/resolution and milestone lookup — do not use this skill to
manage label or milestone definitions themselves (create/edit/delete a label, create/close a
milestone), that's gitea-labels-milestones directly. Do not use for pull requests (use gitea-prs)
or for local git branch/commit work (use gitea-branches or git-branches).
```
After, 240 characters:
```text
Use when reading or writing Gitea issues — list, read, create, comment on, label, close, or
search — even when the user does not say "Gitea". Not pull requests -> `gitea-prs`. Not label or
milestone definitions -> `gitea-labels-milestones`.
```
What came out, and why:
| Removed | Why |
|---|---|
| The second trigger register — `Triggers on "create an issue", "what issues are open", …` | The same triggers restated as quoted user phrasings. Two registers of one trigger list is a FAIL, not a suggestion. |
| `applying labels via issue_write` | Implementation detail. The router does not choose a skill by which MCP call it makes. |
| `Composes gitea-labels-milestones for all label inference/resolution and milestone lookup` | A composition note. It changes no routing decision and belongs in `README.md`. |
| The parenthetical `(create/edit/delete a label, create/close a milestone)` | Capability enumeration inside a boundary clause. The boundary needs the target, not its feature list. |
| The `gitea-branches` / `git-branches` boundary | Dropped entirely. Neither was ever going to win an issue request, so the clause defended against nothing — an invented boundary costs characters and buys no routing accuracy. |
| `Do not use for pull requests (use gitea-prs)` prose form | Kept, but rewritten as `Not pull requests -> \`gitea-prs\`.` The rewrite buys characters and one uniform shape for the router — not safety. Both forms are parsed **and** target-checked, so a typo in the prose form dangles exactly as an arrow typo does. |
What stayed: one trigger clause, one capability clause, the indirect trigger (genuinely warranted
here — people say "create an issue", not "create a Gitea issue"), and the boundary clauses.
## Two rules the gates enforce but the prose does not spell out
**Boundary clauses may be plural.** Write one per genuine near-miss — the example above carries
two, because two different skills could each steal activations. "A boundary clause" in the
contract means *at least one*, not *exactly one*. What is banned is a boundary clause invented for
a skill that was never going to compete, not a second real one.
**Never let a hyphenated routing target wrap across lines in a folded `>` scalar.** YAML folding
replaces the newline with a space, so `gitea-labels-` at the end of one line and `milestones` at
the start of the next fold into `gitea-labels- milestones`. `validate.sh` then reads the target as
`gitea-labels`, finds no such skill, and reports a dangling boundary target — the live finding on
`gitea-issues` today. Reflow the line so the whole name sits on one of them. The same applies to
any backticked skill or agent name in a description.

4
plugins/kyberforge/.apm/skills/skill-author/references/sources.md
View File

@@ -34,7 +34,7 @@ source_keys:
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md, references/retrofit.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
@@ -42,7 +42,7 @@ source_keys:
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md, references/retrofit.md
- **Status:** `extracted`
## agentskills-evaluating-skills

1
plugins/kyberforge/skills/skill-author/README.md
View File

@@ -47,6 +47,7 @@ If the destination resolves inside an APM package, read `references/deployment-m
| `references/create.md` | The create flow end to end — prerequisites, package-intent gate, scaffold, frontmatter, scripts, references, sources (loaded on demand) |
| `references/improve.md` | The improve flow end to end — signal verification, root-cause grouping, announcement, edits (loaded on demand) |
| `references/contract.md` | The ADR-0020 description and body contract, the Gotchas constraint, the two size gates, body patterns, and org-policy embedding (loaded on demand) |
| `references/retrofit.md` | Bringing a pre-ADR-0020 skill into contract — ordered cut procedure, the mutually-exclusive-flows test, reference-file conventions, the collateral checklist, and a worked description retrofit (loaded from the improve flow when a budget is exceeded) |
| `references/deployment-modes.md` | APM package vs standalone differences and self-containment/cache-isolation rules (loaded on demand) |
| `references/scripts.md` | Package runners, inline dependency patterns, and full script contract (loaded on demand) |
| `references/sources.md` | Upstream research sources and which skill files each contributed to |

6
plugins/kyberforge/skills/skill-author/references/improve.md
View File

@@ -76,6 +76,12 @@ contract first — the gates are hot and carry no baseline file, so a one-line f
non-compliant skill cannot be committed until the description and body meet
`references/contract.md`. Treat that retrofit as part of the same change, not a follow-up.
If the skill's description exceeds 250 characters, or its body-only word count exceeds 600, read
`references/retrofit.md` before editing. It carries the ordered cut procedure, the
mutually-exclusive-flows test, the reference-file conventions this flow needs, the collateral
checklist for `README.md` and `references/sources.md`, and a worked description retrofit. Do not
improvise the cuts — four dry runs invented six to ten different answers to the same questions.
If a signal points to a script or reference file, edit that file directly rather than adding a
workaround in SKILL.md.

152
plugins/kyberforge/skills/skill-author/references/retrofit.md Normal file
View File

@@ -0,0 +1,152 @@
---
source_keys:
- agentskills-best-practices
- agentskills-optimizing-descriptions
---
# Retrofitting a skill to the ADR-0020 contract
Read this when `references/improve.md` Step 4 sends you here: the skill you are editing is over
the description or body budget and has to come into contract before any other change can be
committed. The gates are hot and carry no baseline file, so a one-line fix to a non-compliant
skill is blocked until this is done.
Measure first. Do not guess which gate fired: run `/skill-audit` on the directory and read its
`### Structure` dimension, which reports the description characters and the **body-only** word
count separately from the whole-file spec backstop. Retrofit against the number that actually
fired — a skill can sit a thousand words inside the whole-file backstop while failing the body
budget.
**Validate in place.** Audit the skill's real directory inside its package. Never audit a copy in a
scratch directory, and never move a skill out to work on it: the boundary-target universe is built
by walking up *from the file being checked*, so a copy with no authoring root above it resolves
against nothing and the check declines rather than running —
```text
INFO boundary-target resolution DID NOT RUN — no skill universe could be determined for
this path ... Unchecked target(s): totally-fake-target
```
The run still exits 0, so that line reads as a pass and is not one. Treat `DID NOT RUN` as **not
checked**, always. A retrofit signed off on a scratch copy carries an unverified boundary target
into the corpus, which is precisely the failure this gate exists to catch.
## Cut in this order
Work the list top down and stop as soon as the gate clears. The order is by ratio of tokens
removed to behaviour lost — inverting it is how a retrofit ends up deleting the one instruction
the skill existed to carry.
1. **Gotchas that paraphrase a step in the body below.** Zero information, and already a FAIL on
its own. Delete the Gotcha, keep the step.
2. **Spec restatements** — text that repeats a published specification, a tool's `--help`, or a
ceiling the validator already enforces. The agent gets this right without it. Delete, or move
the table to `references/` if a flow genuinely needs to look it up.
3. **Capability enumeration** — in a description, the feature list after the trigger clause; in a
body, the paragraph that recites what the skill can do. One capability clause survives in the
description; the rest belongs in `README.md`.
4. **Per-flow prose** — anything only one branch of the procedure ever reaches. This is the
largest single win in most bodies, and it is a *move*, not a delete: each flow gets its own
self-contained `references/` file, wired from a dispatch table.
If the body is still over after all four, the skill is doing two jobs. Split it, and say so
rather than compressing prose until it stops being readable.
## What "mutually exclusive flows" means
Two or more flows that a single invocation cannot both take. The three-way test, copied verbatim
from the body-discipline rubric `/skill-audit` judges against — nothing to load, it is quoted in
full here:
> separate subcommands, separate input types, separate lifecycle stages
Any one of the three is enough. Two flows that differ only in a parameter value are one flow.
At two or more mutually exclusive flows a dispatch table is **mandatory** regardless of word
count, because every invocation otherwise pays for every branch it did not take.
## Reference-file conventions
The create flow owns these rules, and this flow is forbidden from reading `references/create.md`,
so what a retrofit needs is restated here:
- **One topic per file.** A file mixing two concerns gets loaded for one of them and spends the
caller's context on the other.
- **Kebab-case filenames**, named after the topic rather than the flow that reads it —
`body-discipline.md`, not `step-3.md`.
- **Wire every file with the literal conditional form** ``If <condition>, read
`references/<file>.md` ``. A generic pointer ("see `references/` for details") is a Vale error.
- **Two hops from `SKILL.md`, never three.** A flow file may route on to a shared contract file;
a file reachable only through two intermediates is rarely loaded when it is needed.
- **`source_keys` frontmatter.** If the content you are moving drew on a research source, the new
file needs top-level `source_keys:` frontmatter listing those slugs, and every slug must already
exist as an `## <slug>` heading in `references/sources.md`. Moving sourced content out of
`SKILL.md` without carrying its slugs across breaks the provenance chain, and `/skill-audit`
reports the new file as an INFO with no `source_keys`.
## Collateral is mandatory, not optional
Moving content out of a `SKILL.md` leaves three files describing a structure that no longer
exists. `/skill-audit`'s provenance check exits clean on all three of these, so nothing catches
them for you. After every retrofit that adds, removes or renames a file:
- [ ] **`README.md` file table** — a row for every new `references/` file, and no row left for a
file that is gone. Say what triggers the load, not just what the file contains.
- [ ] **`references/README.md`**, where the skill has one — same update, same reason.
- [ ] **`references/sources.md` → `Contributing files`** — add the new file to every slug whose
content moved into it, and remove any file the retrofit deleted. This is the one that gets
missed: `sources.md` keeps citing sections of `SKILL.md` that no longer exist, the
provenance check still exits 0, and the stale claim survives review.
- [ ] Re-run `/skill-audit` and confirm its `### Provenance` dimension does not report the new
file as missing `source_keys`.
## Worked example — a description retrofit
`gitea-issues` before, 827 characters, the single most common shape in the corpus:
```text
Use when reading or writing Gitea issues: listing repo issues, getting a single issue's details/
comments/labels, creating an issue, updating its state, adding or editing comments, applying
labels via issue_write, or searching issues/PRs across repositories. Triggers on "create an
issue", "what issues are open", "get issue #N", "close issue #N", "comment on issue #N", "search
issues for X" — even when the user doesn't say "Gitea" explicitly. Composes gitea-labels-
milestones for all label inference/resolution and milestone lookup — do not use this skill to
manage label or milestone definitions themselves (create/edit/delete a label, create/close a
milestone), that's gitea-labels-milestones directly. Do not use for pull requests (use gitea-prs)
or for local git branch/commit work (use gitea-branches or git-branches).
```
After, 240 characters:
```text
Use when reading or writing Gitea issues — list, read, create, comment on, label, close, or
search — even when the user does not say "Gitea". Not pull requests -> `gitea-prs`. Not label or
milestone definitions -> `gitea-labels-milestones`.
```
What came out, and why:
| Removed | Why |
|---|---|
| The second trigger register — `Triggers on "create an issue", "what issues are open", …` | The same triggers restated as quoted user phrasings. Two registers of one trigger list is a FAIL, not a suggestion. |
| `applying labels via issue_write` | Implementation detail. The router does not choose a skill by which MCP call it makes. |
| `Composes gitea-labels-milestones for all label inference/resolution and milestone lookup` | A composition note. It changes no routing decision and belongs in `README.md`. |
| The parenthetical `(create/edit/delete a label, create/close a milestone)` | Capability enumeration inside a boundary clause. The boundary needs the target, not its feature list. |
| The `gitea-branches` / `git-branches` boundary | Dropped entirely. Neither was ever going to win an issue request, so the clause defended against nothing — an invented boundary costs characters and buys no routing accuracy. |
| `Do not use for pull requests (use gitea-prs)` prose form | Kept, but rewritten as `Not pull requests -> \`gitea-prs\`.` The rewrite buys characters and one uniform shape for the router — not safety. Both forms are parsed **and** target-checked, so a typo in the prose form dangles exactly as an arrow typo does. |
What stayed: one trigger clause, one capability clause, the indirect trigger (genuinely warranted
here — people say "create an issue", not "create a Gitea issue"), and the boundary clauses.
## Two rules the gates enforce but the prose does not spell out
**Boundary clauses may be plural.** Write one per genuine near-miss — the example above carries
two, because two different skills could each steal activations. "A boundary clause" in the
contract means *at least one*, not *exactly one*. What is banned is a boundary clause invented for
a skill that was never going to compete, not a second real one.
**Never let a hyphenated routing target wrap across lines in a folded `>` scalar.** YAML folding
replaces the newline with a space, so `gitea-labels-` at the end of one line and `milestones` at
the start of the next fold into `gitea-labels- milestones`. `validate.sh` then reads the target as
`gitea-labels`, finds no such skill, and reports a dangling boundary target — the live finding on
`gitea-issues` today. Reflow the line so the whole name sits on one of them. The same applies to
any backticked skill or agent name in a description.

4
plugins/kyberforge/skills/skill-author/references/sources.md
View File

@@ -34,7 +34,7 @@ source_keys:
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md
- **Contributing files:** SKILL.md, references/create.md, references/improve.md, references/contract.md, references/retrofit.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
@@ -42,7 +42,7 @@ source_keys:
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md
- **Contributing files:** SKILL.md, references/improve.md, references/contract.md, references/retrofit.md
- **Status:** `extracted`
## agentskills-evaluating-skills