fix: write-skill governance review — model field, eval self-check, grill wiring
- Add model: sonnet to SKILL.md frontmatter (Claude Code extension; correct location per research — runtime config belongs in frontmatter, not META.md) - Document model field in SKILL-TEMPLATE.md with factory §9 routing guidance - Wire grill output explicitly to trigger description step (step 3 now references agreed name/category/purpose from grill rather than drafting independently) - Strengthen eval self-check: presence check → coverage check (trigger cases + at least one output case required) - Bump META.md to v1.2 (2026-05-26) - Add three LESSONS.md entries: META-TEMPLATE comment retention, overlap check scope (deployed dir not just repo), model field placement boundary Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
```yaml
|
```yaml
|
||||||
version: "1.1"
|
version: "1.2"
|
||||||
updated: 2026-05-18
|
updated: 2026-05-26
|
||||||
|
|
||||||
# when: describes when this skill is loaded — the full trigger context.
|
# when: describes when this skill is loaded — the full trigger context.
|
||||||
# More detail than the description field; not used for routing.
|
# More detail than the description field; not used for routing.
|
||||||
|
|||||||
@@ -11,6 +11,9 @@ description: <trigger description>
|
|||||||
metadata:
|
metadata:
|
||||||
category: <category — see CATEGORIES.md>
|
category: <category — see CATEGORIES.md>
|
||||||
# allowed-tools: <add only when the skill has a narrow, well-defined tool surface; omit otherwise>
|
# allowed-tools: <add only when the skill has a narrow, well-defined tool surface; omit otherwise>
|
||||||
|
# model: sonnet | opus | haiku — Claude Code extension; overrides session model for this skill's turn.
|
||||||
|
# Omit to inherit the active session model. Factory §9 routing: haiku=formatting/classification,
|
||||||
|
# sonnet=most coding/review, opus=adversarial/complex reasoning.
|
||||||
---
|
---
|
||||||
|
|
||||||
<requirements>
|
<requirements>
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ name: write-skill
|
|||||||
description: Use when the user wants to author a new skill file or convert an existing placeholder to the canonical authoring standard. Triggers: "write a new skill for X", "create a SKILL.md that does Y", "build a skill to handle Z". Do NOT use when fixing or updating an existing well-formed skill (use upgrade-skill), running existing evals (use write-eval), refactoring application code, or writing documentation for non-skill artifacts.
|
description: Use when the user wants to author a new skill file or convert an existing placeholder to the canonical authoring standard. Triggers: "write a new skill for X", "create a SKILL.md that does Y", "build a skill to handle Z". Do NOT use when fixing or updating an existing well-formed skill (use upgrade-skill), running existing evals (use write-eval), refactoring application code, or writing documentation for non-skill artifacts.
|
||||||
metadata:
|
metadata:
|
||||||
category: factory
|
category: factory
|
||||||
|
model: sonnet
|
||||||
---
|
---
|
||||||
|
|
||||||
<requirements>
|
<requirements>
|
||||||
@@ -32,11 +33,11 @@ Negative trigger cases are NOT a required input. The agent proposes them based o
|
|||||||
|
|
||||||
## Process
|
## Process
|
||||||
|
|
||||||
1. **Scan for overlap.** Check `.agents/skills/` for skills with similar purpose or trigger phrases. If overlap is found, surface it and wait for explicit direction — do not continue.
|
1. **Scan for overlap.** Check for skills with similar purpose or trigger phrases. If overlap is found, surface it and wait for explicit direction — do not continue.
|
||||||
|
|
||||||
2. **Grill.** Run a focused grill to reach shared understanding of: skill name, category, purpose, and use cases. One question at a time, with a recommendation for each.
|
2. **Grill.** Run a focused grill with the /grill-me skill to reach shared understanding of: skill name, category, purpose, and use cases. One question at a time, with a recommendation for each.
|
||||||
|
|
||||||
3. **Write and test the trigger description.** Draft `description:`. Propose negative trigger cases based on the skill's purpose and adjacent skills — get explicit user confirmation before running tests. Test all three cases and show per-case PASS/FAIL. A failed case means revise and retest — do not proceed.
|
3. **Write and test the trigger description.** Using the agreed name, category, purpose, and use cases from the grill, draft `description:`. Propose negative trigger cases based on the skill's purpose and adjacent skills — get explicit user confirmation before running tests. Test all three cases and show per-case PASS/FAIL. A failed case means revise and retest — do not proceed.
|
||||||
|
|
||||||
4. **Walk through each section.** For each section in `SKILL-TEMPLATE.md`: propose content, state where it comes from, present alternatives if they exist. Wait for explicit human confirmation before moving to the next section.
|
4. **Walk through each section.** For each section in `SKILL-TEMPLATE.md`: propose content, state where it comes from, present alternatives if they exist. Wait for explicit human confirmation before moving to the next section.
|
||||||
|
|
||||||
@@ -78,7 +79,7 @@ For placeholder conversions, `SKILL.md` replaces the existing file entirely —
|
|||||||
- [ ] Frontmatter contains only `name`, `description`, and `metadata.category` (plus `allowed-tools` if applicable)
|
- [ ] Frontmatter contains only `name`, `description`, and `metadata.category` (plus `allowed-tools` if applicable)
|
||||||
- [ ] Body is under 500 lines
|
- [ ] Body is under 500 lines
|
||||||
- [ ] For placeholder conversions: existing files read, all stale content removed, old directory deleted if renamed
|
- [ ] For placeholder conversions: existing files read, all stale content removed, old directory deleted if renamed
|
||||||
- [ ] `write-eval` invoked — eval file exists at correct path
|
- [ ] `write-eval` invoked — eval file exists at correct path, covers trigger cases (explicit, implicit, negative) and at least one output case
|
||||||
- [ ] User prompted for HITL behavioral test
|
- [ ] User prompted for HITL behavioral test
|
||||||
|
|
||||||
</checks>
|
</checks>
|
||||||
|
|||||||
12
LESSONS.md
12
LESSONS.md
@@ -70,6 +70,18 @@ When a skill produces a structured artifact like SKILL.md, the natural approach
|
|||||||
|
|
||||||
The agent-level HITL rule ("require explicit confirmation before irreversible shared-state operations") is being bypassed: the agent calls the tool and lets the permission dialog catch it. This means the rule is not firing in agent reasoning — it's the permission system acting as a safety net. If a user selects "don't ask again," the net disappears. Fix: the HITL rule needs to be framed as "do not call the tool" rather than "ask before proceeding" — the agent must ask first, then act only after explicit confirmation.
|
The agent-level HITL rule ("require explicit confirmation before irreversible shared-state operations") is being bypassed: the agent calls the tool and lets the permission dialog catch it. This means the rule is not firing in agent reasoning — it's the permission system acting as a safety net. If a user selects "don't ask again," the net disappears. Fix: the HITL rule needs to be framed as "do not call the tool" rather than "ask before proceeding" — the agent must ask first, then act only after explicit confirmation.
|
||||||
|
|
||||||
|
## 2026-05-26 — META-TEMPLATE uses YAML comments; META.md output retains them
|
||||||
|
|
||||||
|
META-TEMPLATE.md uses YAML `#` comments to explain fields inline. SKILL-TEMPLATE.md uses HTML comments inside XML tags, which the agent strips on fill. The structural difference means SKILL.md output is clean but META.md output retains the explanatory `#` lines — an inconsistency. Fix: restructure META-TEMPLATE.md so all explanatory guidance is prose above the code block (markdown, never copied into the output YAML), and the code block itself uses `<placeholder>` syntax with no `#` comment lines. This makes META.md fill behaviour deterministic for the same reason SKILL.md fill is: `<...>` markers are unambiguously replaceable; prose above the block is not part of the template.
|
||||||
|
|
||||||
|
## 2026-05-26 — Overlap checks must scan the deployed directory, not just the source repo
|
||||||
|
|
||||||
|
`write-a-skill` existed only in `~/.agents/skills/` (installed from a pre-refactor source) and was invisible during a repo-level scan of `.agents/skills/`. Governance reviews and overlap checks that only look at the source repo will miss skills added by install.sh from other sources or prior runs. Fix: overlap checks must scan the deployed `~/.agents/skills/` directory, not just the repo's `.agents/skills/`.
|
||||||
|
|
||||||
|
## 2026-05-26 — `model:` field belongs in SKILL.md frontmatter, not META.md
|
||||||
|
|
||||||
|
Claude Code supports `model:` as a provider extension in SKILL.md frontmatter — it overrides the session model for the skill's turn and reverts after. Attempting to put it in META.md was wrong: META.md is provenance/audit metadata, not runtime config. The boundary: if a field affects agent behaviour at invocation time, it belongs in SKILL.md frontmatter; if it serves upgrade reviews and audit trails, it belongs in META.md.
|
||||||
|
|
||||||
## 2026-05-18 — Planning meta-commentary does not belong in deployed artifacts
|
## 2026-05-18 — Planning meta-commentary does not belong in deployed artifacts
|
||||||
|
|
||||||
During write-skill refactor, an "open thread" note (about a deferred research step) was written directly into the SKILL.md Process section. The user caught it. The rule it violated: a deployed artifact (SKILL.md, a runtime file loaded by agents) must not contain planning meta-commentary — deferred items, open threads, and implementation notes belong in the issue file, which is the planning artifact. The skill body should contain only content relevant to runtime execution. If a decision is deferred, record it in the issue and leave no trace in the skill. The distinction: issue = planning record; skill = executable instruction.
|
During write-skill refactor, an "open thread" note (about a deferred research step) was written directly into the SKILL.md Process section. The user caught it. The rule it violated: a deployed artifact (SKILL.md, a runtime file loaded by agents) must not contain planning meta-commentary — deferred items, open threads, and implementation notes belong in the issue file, which is the planning artifact. The skill body should contain only content relevant to runtime execution. If a decision is deferred, record it in the issue and leave no trace in the skill. The distinction: issue = planning record; skill = executable instruction.
|
||||||
|
|||||||
Reference in New Issue
Block a user