refactor: write-skill phase 1 refactor — grill, implementation, lessons

Complete 2026-05-18 workstream for write-skill refactor (issue 0018 phase 1).

Grill output (handoff in docs/issues/0018-factory-write-skill.md):
- 6-section standard replaces 8: Role and When/When not dropped (not in
  agentskills.io spec; description carries trigger scope and negatives)
- Frontmatter reduced to 3 fields; provenance moves to META.md
  (progressive disclosure — not loaded at every skill scan)
- SKILL-TEMPLATE.md and META-TEMPLATE.md as authoritative copy-fill artifacts
- CATEGORIES.md as self-contained category reference in skill directory
- Copy and fill explicitly split into separate process steps for determinism

New files in .agents/skills/write-skill/:
- SKILL-TEMPLATE.md — 6-section template with XML blocks and inline examples per section
- META-TEMPLATE.md — provenance schema with inline-commented YAML; source vs references explicit
- CATEGORIES.md — self-contained category table (no runtime dependency on factory docs)
- META.md — write-skill's own provenance (v1.1, self-authored)

SKILL.md rewritten: 8-step process, XML blocks (<requirements>, <steps>, <checks>),
3-field frontmatter, copy-then-fill steps explicit.

Supporting docs updated:
- skill-implementation-workflow.md: SKILL-TEMPLATE.md as authoritative template;
  source:/references: now reference META.md; when: marked resolved
- factory-integration-decisions.md: frontmatter spec corrected
- ROADMAP.md: authoring standard description updated
- CONTEXT.md: Source field entry corrected; META.md added to glossary
- docs/spec/overview.md: 2026-05-18 refactor entry added
- docs/issues/0018: handoff status updated; acceptance criteria marked complete

LESSONS.md: 3 entries added in grill session (cargo-culted body sections,
provenance fields in frontmatter, copy-fill determinism) + 1 from implementation
(planning meta-commentary does not belong in deployed artifacts).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-18 06:48:46 +00:00
parent 2631142a0e
commit e3cc0b65f4
12 changed files with 593 additions and 127 deletions

View File

@@ -54,6 +54,22 @@ The per-skill workflow (and write-skill's own process step 4) requires testing t
write-eval's process requires presenting the full test plan and waiting for user confirmation before writing the file. When write-eval is invoked by passing pre-designed test cases directly to a write sub-agent, this gate is skipped — the file is written before the user sees the plan. This happened during write-docs (issue 0018 phase 2). Fix: when orchestrating write-eval as part of a larger workflow, split into two steps: (1) sub-agent proposes test cases and returns to the main conversation; (2) after user confirmation, sub-agent writes the file. Or: design cases in the main conversation, present them to the user, then spawn the write agent. The plan-then-write separation is the gate — collapsing it into a single sub-agent call silently removes it.
## 2026-05-18 — Skill body sections were cargo-culted, not spec-defined
The write-skill authoring standard required 8 body sections including Role and When/When not. These were assumed to be agentskills.io requirements. Checking the actual spec revealed the body has no format restrictions at all — recommended sections are step-by-step instructions, examples, and edge cases. Role and When/When not were added by convention without verifying the standard. Fix: before encoding any requirement as part of an authoring standard, check the upstream spec directly. The agentskills.io spec also confirmed that negative triggers belong in the description field — not in a separate body section — which eliminates a persistent duplication pattern across all skills.
## 2026-05-18 — Provenance fields in frontmatter are loaded on every skill scan
Fields like `source:`, `references:`, `version:`, `updated:`, and `when:` in SKILL.md frontmatter are loaded at agent startup alongside `name` and `description` for every installed skill. None of these are used for routing or runtime execution — they are audit and upgrade-cycle records. Loading them at startup violates progressive disclosure and wastes tokens proportional to the number of installed skills. Fix: move all non-routing frontmatter to a separate `META.md` file in the skill directory. Frontmatter keeps only `name`, `description`, `metadata.category`, and `allowed-tools` (when applicable) — the four fields the spec actually uses for routing and discovery.
## 2026-05-18 — Copy-fill is more deterministic than generate for structured skill artifacts
When a skill produces a structured artifact like SKILL.md, the natural approach is to generate it from internalized rules in the Process section. But this means section structure is only as reliable as the agent's instruction-following under token pressure. Copy-fill (copy the template to the target path, then fill in content) separates structure from content: the template mechanically enforces section order and presence, freeing the Process section to focus only on sequencing constraints (what order to decide things) rather than also policing structure. Side benefit: the template is a human-usable artifact that can be adopted independently of the skill. Fix applied in write-skill refactor: SKILL-TEMPLATE.md and META-TEMPLATE.md are the authoritative structure sources; the Process section no longer contains a body structure constraint — the template handles it.
## 2026-05-17 — HITL gap: agent delegates confirmation to permission system
The agent-level HITL rule ("require explicit confirmation before irreversible shared-state operations") is being bypassed: the agent calls the tool and lets the permission dialog catch it. This means the rule is not firing in agent reasoning — it's the permission system acting as a safety net. If a user selects "don't ask again," the net disappears. Fix: the HITL rule needs to be framed as "do not call the tool" rather than "ask before proceeding" — the agent must ask first, then act only after explicit confirmation.
## 2026-05-18 — Planning meta-commentary does not belong in deployed artifacts
During write-skill refactor, an "open thread" note (about a deferred research step) was written directly into the SKILL.md Process section. The user caught it. The rule it violated: a deployed artifact (SKILL.md, a runtime file loaded by agents) must not contain planning meta-commentary — deferred items, open threads, and implementation notes belong in the issue file, which is the planning artifact. The skill body should contain only content relevant to runtime execution. If a decision is deferred, record it in the issue and leave no trace in the skill. The distinction: issue = planning record; skill = executable instruction.