feat: implement issue 0017 — factory/write-eval bootstrap skill

Adds write-eval, the first factory meta-skill. Produces eval.yaml test
files for skills following the two-section schema (trigger_tests +
output_tests) with provider-agnostic string assertions and show-plan-
then-merge-on-rerun behaviour. Hand-written bootstrap — subsequent
skills will use write-eval to produce their own evals.

Also tightens skill-implementation-workflow.md step 5b: per-section
options walk-through is now a named gate before writing, separate from
the synthesis grill. LESSONS.md entry added.

HITL behavioral test pending.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-17 17:52:02 +00:00
parent c705a38809
commit 83715018eb
7 changed files with 286 additions and 18 deletions

View File

@@ -94,8 +94,8 @@ Items consciously not resolved — to be addressed in the relevant chunk PRD or
- **AI coding factory integration** — grill complete. Decision record: `docs/notes/factory-integration-decisions.md`. ADRs: 0008 (factory boundary), 0009 (flat taxonomy), 0010 (role skills vs subagents). Follow-on issues: ~~0013 (LESSONS.md)~~ ✅, ~~0014 (docs/spec/ + VISION.md refactor)~~ ✅. Chunk 3 scope substantially expanded — skills rebuild, new skills, IaC/Gitea skills. See updated chunk table above.
- **`.gitkeep` files** — placeholder files exist in `core/agents/`, `core/workflows/`, `core/prompts/`, `docs/ard/`, `docs/bug/`. Remove each when the first real file is added to that directory. Each `.gitkeep` names the chunk that will populate it. (`docs/notes/.gitkeep` already removed — directory has real content.)
- **Skills pipeline verified** — `install.sh` deploys 12 skills to `~/.agents/skills/` and creates `~/.claude/skills/ → ~/.agents/skills/` symlink adapter. Tested idempotent. `skills-lock.json` removed (was a manual artifact). If `~/.claude/skills/` exists as a real directory on a machine being migrated, remove it manually and re-run install.
- **Skills pipeline verified** — `install.sh` deploys 13 skills to `~/.agents/skills/` and creates `~/.claude/skills/ → ~/.agents/skills/` symlink adapter. Tested idempotent. `skills-lock.json` removed (was a manual artifact). If `~/.claude/skills/` exists as a real directory on a machine being migrated, remove it manually and re-run install.
- **Chunk 2 behavioral tests** — run and fully resolved 2026-05-17. 7/8 pass; scenario 4 (push confirmation) inconclusive — no remote in test environment, rule tightened but unverified. All fixable failures addressed: rule specificity in `providers/claude-code/CLAUDE.md`; context-loading guarantee via `@import CONTEXT.md` in repo CLAUDE.md; standing rule in CONTEXT.md to check `docs/adr/` and ROADMAP resolved entries before answering design questions. Chunk 2 ✅ complete.
- **Governance Phase 1 behavioral tests** — run 2026-05-17. 3/4 testable scenarios pass. Secrets rule gap fixed (2026-05-17): extended to cover credential reproduction in response text and examples, with placeholder requirement added to `core/instructions/governance.md`. HITL scenario not testable in this environment (Nginx not installed); HITL gap evidenced by instructions test scenario 4 — push confirmation rule fix addresses the same root cause. Governance Phase 1 ✅ complete.
- **AI ethics/security workstream** — `docs/notes/ai-ethics-security-principles.md` exploration note is superseded. Governance Phase 1 (`core/instructions/governance.md`) covers all planned scope: credentials, data classification, HITL, scope discipline, agent autonomy, transparency, and security code review. Tier-placement architectural question resolved by the `@import` always-on model. No separate workstream needed.
- **Chunk 3 grill complete** — 2026-05-17. PRD at `docs/prd/chunk-3-skills-library.md`. Key decisions: 42-skill target library, AGENTS.md refactor as prerequisite issue (both CLAUDE.md files become thin adapters), git-cliff for changelog, provider-agnostic issue tracker abstraction, grill-me/grill-lean design phase split, factory bootstrap order (write-eval → write-skill → write-docs phase 2 → write-adr → remaining factory → design → parallel category groups). ADRs written: 0011 (provider-agnostic issue tracker), 0012 (AGENTS.md governance entry point, partially supersedes ADR-0005). Upstream review cadence: per-skill + quarterly post-roadmap (per-chunk-start changed to per-skill by issue 0016 grill). **Issues created 0015–0028** — all HITL; ~~0015 (AGENTS.md refactor, prerequisite)~~ ✅, ~~0016 (skill workflow grill, produces conventions for 0017–0028)~~ ✅, 0017–0018 (bootstrap skills: write-eval; write-skill + write-docs as phase 2), 0019 (remaining factory skills), 0020–0027 (design/implement/test/review/deploy/operate/iac/cross-cutting), 0028 (chunk closure). ~~Acceptance criteria for 0017–0028 to be refined after 0016 grill session.~~ ✅ Refined 2026-05-17 — see `docs/notes/skill-implementation-workflow.md`.
- **Chunk 3 grill complete** — 2026-05-17. PRD at `docs/prd/chunk-3-skills-library.md`. Key decisions: 42-skill target library, AGENTS.md refactor as prerequisite issue (both CLAUDE.md files become thin adapters), git-cliff for changelog, provider-agnostic issue tracker abstraction, grill-me/grill-lean design phase split, factory bootstrap order (write-eval → write-skill → write-docs phase 2 → write-adr → remaining factory → design → parallel category groups). ADRs written: 0011 (provider-agnostic issue tracker), 0012 (AGENTS.md governance entry point, partially supersedes ADR-0005). Upstream review cadence: per-skill + quarterly post-roadmap (per-chunk-start changed to per-skill by issue 0016 grill). **Issues created 0015–0028** — all HITL; ~~0015 (AGENTS.md refactor, prerequisite)~~ ✅, ~~0016 (skill workflow grill, produces conventions for 0017–0028)~~ ✅, ~~0017 (bootstrap skill: write-eval)~~ ⏳ HITL pending, 0018 (write-skill + write-docs as phase 2), 0019 (remaining factory skills), 0020–0027 (design/implement/test/review/deploy/operate/iac/cross-cutting), 0028 (chunk closure). ~~Acceptance criteria for 0017–0028 to be refined after 0016 grill session.~~ ✅ Refined 2026-05-17 — see `docs/notes/skill-implementation-workflow.md`.