Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

refactor: write-skill phase 1 refactor — grill, implementation, lessons

Browse Source 
Complete 2026-05-18 workstream for write-skill refactor (issue 0018 phase 1).

Grill output (handoff in docs/issues/0018-factory-write-skill.md):
- 6-section standard replaces 8: Role and When/When not dropped (not in
  agentskills.io spec; description carries trigger scope and negatives)
- Frontmatter reduced to 3 fields; provenance moves to META.md
  (progressive disclosure — not loaded at every skill scan)
- SKILL-TEMPLATE.md and META-TEMPLATE.md as authoritative copy-fill artifacts
- CATEGORIES.md as self-contained category reference in skill directory
- Copy and fill explicitly split into separate process steps for determinism

New files in .agents/skills/write-skill/:
- SKILL-TEMPLATE.md — 6-section template with XML blocks and inline examples per section
- META-TEMPLATE.md — provenance schema with inline-commented YAML; source vs references explicit
- CATEGORIES.md — self-contained category table (no runtime dependency on factory docs)
- META.md — write-skill's own provenance (v1.1, self-authored)

SKILL.md rewritten: 8-step process, XML blocks (<requirements>, <steps>, <checks>),
3-field frontmatter, copy-then-fill steps explicit.

Supporting docs updated:
- skill-implementation-workflow.md: SKILL-TEMPLATE.md as authoritative template;
  source:/references: now reference META.md; when: marked resolved
- factory-integration-decisions.md: frontmatter spec corrected
- ROADMAP.md: authoring standard description updated
- CONTEXT.md: Source field entry corrected; META.md added to glossary
- docs/spec/overview.md: 2026-05-18 refactor entry added
- docs/issues/0018: handoff status updated; acceptance criteria marked complete

LESSONS.md: 3 entries added in grill session (cargo-culted body sections,
provenance fields in frontmatter, copy-fill determinism) + 1 from implementation
(planning meta-commentary does not belong in deployed artifacts).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Defame1297
2026-05-18 06:48:46 +00:00
parent 2631142a0e
commit e3cc0b65f4
12 changed files with 593 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files

354
docs/issues/0018-factory-write-skill.md
View File

@@ -70,12 +70,19 @@ Follow the per-skill workflow defined in `docs/notes/skill-implementation-workfl
- [ ] **HITL:** human reviews SKILL.md and eval before committing
- [x] Per-skill process followed for both phases (see `docs/notes/skill-implementation-workflow.md`)
- [x] Trigger description for each skill tested against explicit, implicit, and negative queries before body written
- [x] `when:` frontmatter field present in both SKILL.md files
- [x] `source:` and `references:` fields correctly populated or absent
- ~~[x] `when:` frontmatter field present in both SKILL.md files~~ — superseded by refactor: `when:` moves to META.md
- ~~[x] `source:` and `references:` fields correctly populated or absent~~ — superseded by refactor: both move to META.md
- [x] eval.yaml for each skill contains all 5 required test types
- [x] Body ≤500 lines for each skill
- [x] Phase 2 (`write-docs`) is the first skill produced end-to-end by the factory
- [x] `docs/spec/overview.md` updated to reflect both skills deployed
- [x] **Refactor:** `.agents/skills/write-skill/SKILL-TEMPLATE.md` exists — authoritative 6-section template with XML blocks
- [x] **Refactor:** `.agents/skills/write-skill/META-TEMPLATE.md` exists — YAML block with inline-commented source schema
- [x] **Refactor:** `.agents/skills/write-skill/CATEGORIES.md` exists — category table copied from factory-integration-decisions.md
- [x] **Refactor:** `.agents/skills/write-skill/META.md` exists — write-skill's own provenance (self-authored, no source, references agentskills.io)
- [x] **Refactor:** `write-skill/SKILL.md` rewritten — 6 sections, XML blocks, 3-field frontmatter, no Role, no When/When not
- [x] **Refactor:** `docs/notes/skill-implementation-workflow.md` updated — references SKILL-TEMPLATE.md instead of embedding inline template
- [ ] **Refactor HITL:** open fresh session, invoke "write a new skill for X", verify: overlap scan first, grill to gather, agent proposes negative cases, per-section explicit confirmation, copy-then-fill both files, write-eval invoked, HITL prompted
- [ ] **Phase 3:** `/grill-me` session completed; grill output committed
- [ ] **Phase 3:** `docs/notes/doc-convention.md` written and committed
- [ ] **Phase 3:** `write-docs` SKILL.md output format updated to reference the convention (via `upgrade-skill` if substantive)
@@ -134,3 +141,346 @@ Follow the per-skill workflow defined in `docs/notes/skill-implementation-workfl
**Open threads:**
- Documentation convention: scoped to Phase 3 of this issue — see "What to build" above. `write-docs` output format section will be updated once the convention is defined.
- HITL behavioral test: see above.
---
## Handoff — Phase 1 Refactor (write-skill)
**Status:** implementation complete — pending HITL behavioral test
**Files produced:**
- `.agents/skills/write-skill/SKILL.md` — rewritten (6 sections, XML blocks, 3-field frontmatter)
- `.agents/skills/write-skill/SKILL-TEMPLATE.md` — authoritative 6-section template with inline examples
- `.agents/skills/write-skill/META-TEMPLATE.md` — provenance schema with inline-commented YAML
- `.agents/skills/write-skill/CATEGORIES.md` — self-contained category table
- `.agents/skills/write-skill/META.md` — write-skill's own provenance (v1.1, self-authored)
**Context:** the Phase 1 write-skill was hand-authored as a bootstrap skill and does not follow the quality bar it is supposed to produce. A full grill session (2026-05-18) redesigned it from the ground up. The implementation session should produce all four files and update the authoring standard.
---
### What changes and why
The current write-skill is heavy, duplicates the agentskills.io spec incorrectly, embeds its own output template inline (28 lines), and loads provenance metadata that is never used at runtime. The refactor makes it:
- **Modular** — templates extracted to human-usable files; provenance separated into META.md
- **Spec-compliant** — frontmatter reduced to the four fields agentskills.io actually defines
- **Token-optimised** — provenance not loaded at runtime (progressive disclosure)
- **Clearer** — plain English constraints, numbered steps in improve-codebase-architecture tone, XML grouping
---
### New file structure
```
.agents/skills/write-skill/
├── SKILL.md ← rewritten (6 sections, XML-structured, lean frontmatter)
├── SKILL-TEMPLATE.md ← NEW: authoritative template for new skill bodies (copy-fill)
├── META-TEMPLATE.md ← NEW: authoritative template for new skill META.md files (copy-fill)
├── CATEGORIES.md ← NEW: category table (self-contained reference, not a runtime dependency)
└── META.md ← NEW: write-skill's own provenance record
```
---
### Frontmatter — new spec
**Before:**
```yaml
name: write-skill
description: ...
version: "1.0"
updated: 2026-05-17
when: ...
metadata:
category: factory
references:
- ...
```
**After:**
```yaml
name: write-skill
description: ...
metadata:
category: factory
```
`version`, `updated`, `when`, `source`, `references` all move to `META.md`. `allowed-tools` added only when the skill has a narrow, well-defined tool surface — write-skill does not, so omit.
**Rationale:** agentskills.io spec defines only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` as frontmatter fields. Everything else is a project extension. Project extensions that are audit/provenance records (not routing or runtime data) belong in META.md where they are not loaded on every skill scan.
---
### META.md — content and schema
META.md is a markdown file containing a single YAML code block. Content for write-skill:
```yaml
version: "1.1"
updated: 2026-05-18
when: invoked by explicit trigger ("write a new skill for X", "create a SKILL.md that does Y") or implicit request to author a skill file or convert an existing placeholder to the canonical authoring standard
# source: omitted — self-authored original; no upstream content adopted
# Absence of source means self-authored. If content is adopted from upstream,
# add a source entry per the META-TEMPLATE.md schema.
references:
- https://agentskills.io/specification.md
- https://agentskills.io/skill-creation/optimizing-descriptions
```
**The source vs references distinction — make this explicit in META-TEMPLATE.md:**
- `source:` — content you **adopted**. You read upstream code or docs, took text or logic, and incorporated it. Tracked at commit-level (repo slug, commit SHA, files with inline comments, updated date) so upgrade-skill can flag when upstream changed. **Absence means self-authored original.**
- `references:` — content you **cited**. It informed the skill but you took nothing verbatim. URLs, papers, standards, documentation.
Example: if you adapted Pocock's grill-me SKILL.md, that is `source:`. If you read agentskills.io best-practices and followed principles without copying text, that is `references:`.
---
### Description field — new requirements
Per agentskills.io spec and the optimizing-descriptions guide:
- **Routing only** — what the skill does, when to use it, negative triggers
- **Max 1024 characters**
- **Imperative phrasing** — "Use when..." not "This skill does..."
- **Include negative triggers** — the spec explicitly recommends this for preventing false activation on adjacent tasks
- **No behavioral/role framing** — that is the body's job
The `when:` frontmatter field moves to META.md. Any information it contained that is relevant to routing (trigger context, invocation conditions) must be incorporated into `description:`. The current description already covers most of this — review and ensure nothing from `when:` is lost.
---
### Dropped sections
**Role** — removed from the authoring standard entirely.
Rationale: not defined by agentskills.io spec. The three best-performing reference skills (grill-with-docs, tdd, improve-codebase-architecture) all work without it. The description + process carry the behavioral framing adequately. Chunk 5 agents will handle cognitive mode at session level. When Role is just a restatement of the description, it is dead weight (governance principle: minimum tokens to accomplish the task accurately).
**When to use / When not to use** — removed from the authoring standard.
Rationale: agentskills.io spec and the optimizing-descriptions guide both state that the description field is the correct place for trigger scope and negative cases. A separate body section repeating the same information violates DRY and the progressive disclosure principle (the description is read at startup; a body section is read only after activation — by which point the routing decision has already been made).
---
### Authoring standard update
Body sections drop from 8 to 6, in this order:
1. Required inputs
2. Constraints
3. Process
4. Output format
5. Failure handling
6. Self-check
`SKILL-TEMPLATE.md` becomes the authoritative template, superseding the inline template currently embedded in `docs/notes/skill-implementation-workflow.md`. Update that document to reference `SKILL-TEMPLATE.md` instead of duplicating it — single source of truth.
---
### XML structure
Three blocks wrapping the 6 sections:
```
<requirements>
## Required inputs
## Constraints
</requirements>
<steps>
## Process
## Output format
</steps>
<checks>
## Failure handling
## Self-check
</checks>
```
Permitted by factory rule: body will be >500 tokens with ≥3 logical sections. Named for plain-language clarity following grill-with-docs style.
---
### Required inputs (confirmed content)
- **Skill name** — inferred from description if not stated explicitly; ask if ambiguous
- **Category** — from the category table in `.agents/skills/write-skill/CATEGORIES.md` (see below)
- **Purpose + use cases** — what the skill does and what tasks it handles; source for the trigger description
- **For placeholder conversions:** existing SKILL.md path — read before writing
**Negative trigger cases are NOT a required input.** The agent proposes them based on the skill's purpose and adjacent skills found during the overlap scan. The user confirms or refines before trigger testing begins.
---
### Constraints (confirmed content)
Write in plain English, one rule per bullet, boundary condition stated inline:
- Write two files for every skill: `SKILL.md` at `.agents/skills/<name>/SKILL.md` and `META.md` alongside it
- Frontmatter has three fields only: `name`, `description`, and `metadata.category` — add `allowed-tools` only when the skill has a narrow, well-defined tool surface
- Keep the body under 500 lines — move anything longer into separate files in the skill directory
- Use XML tags only when the body has three or more logical sections and exceeds 500 tokens — default to plain prose
- Test the trigger description against all three cases — explicit, implicit, negative — before writing any body content. Hard gate: a failed case means revise and retest, not proceed
- Check for overlapping skills in `.agents/skills/` before writing anything — if overlap is found, surface it and wait for direction
- For placeholder conversions: read the existing SKILL.md first and remove all stale or outdated content
**Do not include a constraint about body section structure — the template enforces that mechanically.**
---
### Process (confirmed content)
Write in improve-codebase-architecture tone: short numbered steps, action verbs, side effects stated inline. No bureaucratic padding.
1. **Scan for overlap.** Check `.agents/skills/` for skills with similar purpose or trigger phrases. If overlap is found, surface it and wait for explicit direction — do not continue.
2. **Grill.** Run a focused grill to reach shared understanding of: skill name, category, purpose, and use cases. One question at a time, with a recommendation for each.
3. **Write and test the trigger description.** Draft `description:`. Propose negative trigger cases based on the skill's purpose and adjacent skills — get explicit user confirmation before running tests. Test all three cases and show per-case PASS/FAIL. A failed case means revise and retest — do not proceed.
4. **Walk through each section.** For each section in `SKILL-TEMPLATE.md`: propose content, state where it comes from, present alternatives if they exist. Wait for explicit human confirmation before moving to the next section.
5. **Copy both templates.** Copy `SKILL-TEMPLATE.md` to `.agents/skills/<name>/SKILL.md`. Copy `META-TEMPLATE.md` to `.agents/skills/<name>/META.md`. Do not modify content yet — copy first, fill second.
6. **Fill both files.** Fill in the copied `SKILL.md` with confirmed section content. Fill in the copied `META.md` with version, updated date, when, source (if applicable), and references (if applicable).
7. **Invoke `write-eval`.** Do not mark the skill complete without an eval file.
8. **Prompt for HITL.** Ask the user to open a fresh session, trigger the skill, and confirm output before committing.
**Open thread — research step:** a source discovery, source review, and governance conflict check step (per `docs/notes/skill-implementation-workflow.md` steps 1–3) belongs between step 1 (overlap scan) and step 2 (grill). Add this once the factory has enough maturity to support it. This is deliberately deferred, not forgotten.
Note: process now has 8 steps (copy and fill are explicitly split at steps 5 and 6).
---
### Output format (confirmed content)
Two files produced for every skill:
- `SKILL.md` — copy-filled from `SKILL-TEMPLATE.md` at `.agents/skills/<name>/SKILL.md`
- `META.md` — copy-filled from `META-TEMPLATE.md` at `.agents/skills/<name>/META.md`
For placeholder conversions, `SKILL.md` replaces the existing file entirely — no partial edits.
---
### Failure handling (confirmed content — lean, no overlap with constraints or process)
- Template file missing — stop, report the path searched, do not write from memory
- Existing SKILL.md not found for a placeholder conversion — stop, report the path searched
- `write-eval` fails or is unavailable — flag, do not mark the skill complete
---
### Self-check (confirmed content)
- [ ] Overlap check completed before any content was written
- [ ] Trigger description tested against all three cases — all passed before body content was written
- [ ] Negative trigger cases confirmed by user before testing
- [ ] Each section confirmed explicitly by user before SKILL.md was written
- [ ] SKILL.md copy-filled from `SKILL-TEMPLATE.md` at correct path
- [ ] `META.md` copy-filled from `META-TEMPLATE.md` at correct path
- [ ] Frontmatter contains only `name`, `description`, and `metadata.category` (plus `allowed-tools` if applicable)
- [ ] Body is under 500 lines
- [ ] For placeholder conversions: existing files read, all stale content removed, old directory deleted if renamed
- [ ] `write-eval` invoked — eval file exists at correct path
- [ ] User prompted for HITL behavioral test
---
### SKILL-TEMPLATE.md — what to produce
A complete, correctly-structured skeleton for a new skill body. Contains:
- Correct frontmatter block (3 fields only: name, description, metadata.category)
- All 6 body sections as `## ` headers in correct order
- Three XML blocks wrapping sections as documented above
- Placeholder comments in each section explaining what goes there and from which source
- No prose content — placeholders only
The template is the authoritative structure reference. If the section structure changes, update the template — not the skill body.
---
### CATEGORIES.md — what to produce
A reference file at `.agents/skills/write-skill/CATEGORIES.md` containing the canonical category table. The skill is self-contained — it must not reference `docs/notes/factory-integration-decisions.md` at runtime. The table is copied verbatim from that document:
| Category | Scope |
|---|---|
| `design` | grill-me, grill-with-docs, to-prd, prototype, architecture-review |
| `plan` | to-issues, triage |
| `implement` | tdd, diagnose, implement-feature, refactor, write-docs |
| `test` | write-tests, generate-test-data, review-test-coverage |
| `review` | improve-codebase-architecture, code-review, security-review, pr-description, changelog-entry |
| `deploy` | write-ci-pipeline, write-deployment-config, write-ai-review-workflow, deployment-checklist |
| `operate` | write-runbook, incident-diagnosis, post-mortem, inspect-deployment |
| `iac` | write-ansible-role, write-terraform-module, write-k8s-manifest, write-docker-compose, proxmox-vm-spec, iac-security-review, write-molecule-test |
| `cross-cutting` | zoom-out, caveman, session-handoff, governance-check, git-guardrails, git-commit-message |
| `factory` | write-skill, write-adr, write-workflow, write-eval, validate-skill, upgrade-skill, write-issue-spec |
| `roles` | architect, developer, reviewer, security, qa, ops — Chunk 5 |
---
### META-TEMPLATE.md — what to produce
A YAML code block inside a markdown file. The template must be self-explanatory — a reader should understand every field without consulting any other file. Produce exactly this structure with inline comments preserved:
```yaml
version: "1.0" # increment on meaningful changes to the skill
updated: YYYY-MM-DD # ISO date of last update
# when: describes when this skill is loaded — the full trigger context.
# More detail than the description field; not used for routing.
when: <describe the invocation conditions here>
# source: tracks content you ADOPTED from an upstream repo.
# Adopt = you read someone else's code or docs and incorporated text or logic directly.
# Omit this field entirely if the skill is self-authored — absence means original work.
# Present only when content was actually taken, tracked at commit-level for upgrade reviews.
source:
- repo: org/repo-name # GitHub slug — no URL, slug is stable and searchable
commit: <full SHA> # exact commit reviewed at time of adoption
files:
- path/to/file.md # inline comment: what was taken from this file
- path/to/other.md # inline comment: what was taken from this file
updated: YYYY-MM-DD # date this source entry was last reviewed
# references: tracks content you CITED but did not adopt verbatim.
# Cite = you read it and it informed the skill, but nothing was copied or adapted.
# Examples: a spec you followed, a paper that shaped the approach, external documentation.
# Distinct from source: source = took content; references = informed by content.
references:
- https://example.com/relevant-doc
```
---
### Open threads for future sessions
1. **Research step** — add source discovery, source review, and governance conflict check between overlap scan and grill once the factory supports it (documented above in Process)
2. **upgrade-skill** — when built, should reference `write-skill/SKILL-TEMPLATE.md` and `write-skill/META-TEMPLATE.md` rather than duplicating them. If templates being "owned" by write-skill feels awkward for upgrade-skill, move them to a shared factory location at that point. Do not act on this now — the templates' location is reversible and upgrade-skill doesn't exist yet.
3. **skill-implementation-workflow.md** — update to reference `SKILL-TEMPLATE.md` as the authoritative template instead of embedding its own inline copy. Single source of truth.
4. **write-eval** — follows the old 8-section standard. When write-skill is updated, write-eval should be reviewed and updated to the new 6-section standard in a follow-on session.
5. **All Chunk 3 skills** — any skills produced by write-skill going forward follow the new 6-section standard with META.md. Skills already produced (write-docs) should be reviewed against the new standard in issue 0028 (chunk 3 closure).
---
### Implementation order for next session
1. Read: `CONTEXT.md`, this issue file, current `.agents/skills/write-skill/SKILL.md`
2. Write `META-TEMPLATE.md` first — the source block schema with inline YAML comments must be explicit here before anything else references it
3. Write `SKILL-TEMPLATE.md` — 6 sections, XML blocks (`<requirements>`, `<steps>`, `<checks>`), correct frontmatter (3 fields only)
4. Write `CATEGORIES.md` — copy the category table from `docs/notes/factory-integration-decisions.md` verbatim
5. Rewrite `SKILL.md` — follow the new structure (write-skill does not copy-fill its own template; it models the same structure directly)
6. Write write-skill's own `META.md` — `version: "1.1"`, `updated: 2026-05-18`, no `source` (self-authored original), `references` cites agentskills.io spec and optimizing-descriptions
7. Update `docs/notes/skill-implementation-workflow.md` — reference `SKILL-TEMPLATE.md` instead of embedding its own inline template copy
8. Update acceptance criteria in this issue to reflect the new standard
9. HITL behavioral test — open a fresh session, invoke "write a new skill for X", verify: overlap scan first, grill used for gathering, agent proposes negative cases before trigger test, per-section explicit confirmation, both files produced via copy-then-fill, write-eval invoked, HITL prompted