holocron/LESSONS.md

# Lessons

Patterns observed during development of this repo. Three or more entries on the same pattern → promote to CONTEXT.md (or the relevant instruction file) as a standing rule.

**Graduation rule:** When three or more entries cover the same pattern, the human reviews and promotes it to the appropriate standing location: `CONTEXT.md` for domain-level principles, `core/instructions/coding.md` for coding conventions, `core/instructions/git.md` for git conventions, or `core/instructions/testing.md` for testing conventions. The graduated entries are marked `[graduated → target file]` rather than deleted (audit trail).

**Who writes here:** The session-handoff skill (Chunk 3) prompts LESSONS.md extraction before closing a session. The human may also write directly.

**What belongs here:** Non-obvious observations — a rule that was misapplied, a pattern that caused friction, a decision that turned out wrong in practice. Not summaries of what was built (that's git history) or planned changes (that's issues).

---

## 2026-05-17 — Workflow documents should prescribe sub-agent usage, not just allow it

When writing workflow documents (like `docs/notes/skill-implementation-workflow.md`), the natural tendency is to describe steps at a high level and leave sub-agent usage as an implementation detail. But if the workflow doesn't explicitly prescribe "spawn a sub-agent here," practitioners default to doing everything in the main context — accumulating token cost and losing the isolation benefit. Fix: make sub-agent usage a named step in the workflow, specifying what the agent receives, what it returns, and why it's isolated. This makes the workflow reproducible rather than dependent on the practitioner remembering to use agents.

## 2026-05-17 — Conflict check before synthesis grill, not during

When combining upstream sources into a skill, conflicts with governing documents (AI constitution, factory principles) tend to surface in the middle of the synthesis grill — disrupting the combining discussion and requiring context switches. Fix: run a dedicated conflict-check step before the grill. A sub-agent reads the governing documents, checks the upstream content against them, and returns a numbered list of tensions. The grill then starts with those items as explicit agenda points, making it faster and more systematic. An empty conflict list is also valuable — it confirms the upstreams are clean before co-writing begins.

## 2026-05-17 — Cross-references to "produced by issue N" rot before the session ends

Issue files frequently referenced "the workflow defined in `docs/notes/skill-implementation-workflow.md` (produced by issue 0016)." Within the same session that closes issue 0016, that parenthetical is already stale — the document exists and is the authoritative reference. Fix: reference the document path directly, not the issue that produced it. The git history records the producing issue; cross-references should point to the artifact that persists.

## 2026-05-17 — "Read at session start" is a behavioral hope, not a guarantee

The repo CLAUDE.md instructs agents to read CONTEXT.md and ROADMAP.md at session start, but agents skip this in practice — defaulting to reading only what's directly relevant to the immediate prompt (e.g. the skills folder). The governance.md works because `@import` is technically enforced by Claude Code. Fix: (1) add `@CONTEXT.md` to repo CLAUDE.md using `@import` to make it always-loaded; (2) add a "Key decisions" section to CONTEXT.md with one-line resolved-ADR summaries so locked choices are always in context. ROADMAP stays on-demand.

## 2026-05-17 — Instruction rules lose to RLHF defaults without specificity

Behavioral tests (2026-05-17) showed three communication/behavior rules failing: exploratory question format (gave verbose multi-bullet answer instead of 2-3 sentences), file edit intent (asked for clarification instead of stating intent and proceeding), and push confirmation (went straight to tool call instead of asking first). All three rules are present in `providers/claude-code/CLAUDE.md` as one-liner statements. The RLHF-trained defaults (thorough answers, risk-averse clarification seeking, fast execution) consistently outcompete thin rules. Fix: rewrite failing rules with specificity, a counter-example, and a boundary statement — not just a single-line imperative.

## 2026-05-17 — Secrets rule gap: response text not covered

The secrets prohibition in `core/instructions/governance.md` fired correctly when asked to write a password to a file, but the agent then reproduced the literal credential in its response text (in a shell `export` example). The rule was interpreted as "don't write to files" not "don't output at all." Fix: the rule needs to explicitly state "never produce the credential value in any output" and give an example showing placeholder usage (`export DB_PASSWORD='<your-password>'`).

## 2026-05-17 — Synthesis grill and SKILL.md co-write are two separate conversations

The synthesis grill (step 4) answers schema-level questions: how to combine upstreams, which eval schema to use, merge behaviour. Step 5b is a different conversation: how upstream content maps to each SKILL.md body section, what options each section had, and which was chosen. Collapsing them — writing the SKILL.md immediately after the grill without a per-section walk-through — means the human never sees the upstream options for the body and has no opportunity to redirect before the file is written. Fix: step 5b is now a named gate in the workflow. Walk through every body section one at a time, cite the upstream source, present alternatives, get confirmation. Only then write. Applies to both hand-written (bootstrap) and write-skill-produced skills.

## 2026-05-17 — Skill-calls-skill composition must be a named process step

When a skill invokes another skill as part of its work (e.g. write-skill invoking write-eval to produce the eval), that call must be a numbered step in the Process section — not left as an implicit external workflow step. If it isn't named, practitioners either forget it or do it manually outside the skill, breaking the composition chain. The user caught this during the write-skill co-write; it was absent from the process despite being in the workflow doc. Fix: when designing any skill that composes another, list each composed call explicitly as a numbered step with a "do not mark complete until X exists" constraint.

## 2026-05-17 — AGPL-3.0 repos appear prominently in community skill search results

When searching GitHub for agent skill upstreams, AGPL-3.0 repos (e.g. dceoy/speckit-agent-skills) appear alongside permissive-licensed ones without obvious visual distinction. AGPL imposes copyleft obligations on adopted content. Always run a licence check (GitHub API `/license` endpoint) before extracting any content from a new upstream. An AGPL finding is a hard exclude — record the repo, SHA, and licence in source review notes so future sessions don't re-review it.

## 2026-05-17 — Trigger description gate is not satisfied by embedding it in the section walk-through

The per-skill workflow (and write-skill's own process step 4) requires testing the trigger description against 3 cases — explicit, implicit, negative — as a standalone gate with explicit PASS/FAIL markers before any body content is written. During write-docs (issue 0018 phase 2), the trigger description was included in the section walk-through (step 5b) rather than tested first as a named gate. The gate never had explicit pass/fail output, which means neither the human nor the agent confirmed the trigger was sound before section content was written. Fix: treat the trigger test as a numbered standalone step with per-case PASS/FAIL output before step 5b begins. A section walk-through that happens to include the description field is not a substitute.

## 2026-05-17 — write-eval confirmation gate is bypassed when called via sub-agent with pre-designed cases

write-eval's process requires presenting the full test plan and waiting for user confirmation before writing the file. When write-eval is invoked by passing pre-designed test cases directly to a write sub-agent, this gate is skipped — the file is written before the user sees the plan. This happened during write-docs (issue 0018 phase 2). Fix: when orchestrating write-eval as part of a larger workflow, split into two steps: (1) sub-agent proposes test cases and returns to the main conversation; (2) after user confirmation, sub-agent writes the file. Or: design cases in the main conversation, present them to the user, then spawn the write agent. The plan-then-write separation is the gate — collapsing it into a single sub-agent call silently removes it.

## 2026-05-18 — Skill body sections were cargo-culted, not spec-defined

The write-skill authoring standard required 8 body sections including Role and When/When not. These were assumed to be agentskills.io requirements. Checking the actual spec revealed the body has no format restrictions at all — recommended sections are step-by-step instructions, examples, and edge cases. Role and When/When not were added by convention without verifying the standard. Fix: before encoding any requirement as part of an authoring standard, check the upstream spec directly. The agentskills.io spec also confirmed that negative triggers belong in the description field — not in a separate body section — which eliminates a persistent duplication pattern across all skills.

## 2026-05-18 — Provenance fields in frontmatter are loaded on every skill scan

Fields like `source:`, `references:`, `version:`, `updated:`, and `when:` in SKILL.md frontmatter are loaded at agent startup alongside `name` and `description` for every installed skill. None of these are used for routing or runtime execution — they are audit and upgrade-cycle records. Loading them at startup violates progressive disclosure and wastes tokens proportional to the number of installed skills. Fix: move all non-routing frontmatter to a separate `META.md` file in the skill directory. Frontmatter keeps only `name`, `description`, `metadata.category`, and `allowed-tools` (when applicable) — the four fields the spec actually uses for routing and discovery.

## 2026-05-18 — Copy-fill is more deterministic than generate for structured skill artifacts

When a skill produces a structured artifact like SKILL.md, the natural approach is to generate it from internalized rules in the Process section. But this means section structure is only as reliable as the agent's instruction-following under token pressure. Copy-fill (copy the template to the target path, then fill in content) separates structure from content: the template mechanically enforces section order and presence, freeing the Process section to focus only on sequencing constraints (what order to decide things) rather than also policing structure. Side benefit: the template is a human-usable artifact that can be adopted independently of the skill. Fix applied in write-skill refactor: SKILL-TEMPLATE.md and META-TEMPLATE.md are the authoritative structure sources; the Process section no longer contains a body structure constraint — the template handles it.

## 2026-05-17 — HITL gap: agent delegates confirmation to permission system

The agent-level HITL rule ("require explicit confirmation before irreversible shared-state operations") is being bypassed: the agent calls the tool and lets the permission dialog catch it. This means the rule is not firing in agent reasoning — it's the permission system acting as a safety net. If a user selects "don't ask again," the net disappears. Fix: the HITL rule needs to be framed as "do not call the tool" rather than "ask before proceeding" — the agent must ask first, then act only after explicit confirmation.

## 2026-05-26 — META-TEMPLATE uses YAML comments; META.md output retains them

META-TEMPLATE.md uses YAML `#` comments to explain fields inline. SKILL-TEMPLATE.md uses HTML comments inside XML tags, which the agent strips on fill. The structural difference means SKILL.md output is clean but META.md output retains the explanatory `#` lines — an inconsistency. Fix (deferred): restructure META-TEMPLATE.md so all explanatory guidance is prose above the code block (markdown, never copied into the output YAML), and the code block itself uses `<placeholder>` syntax with no `#` comment lines. This makes META.md fill behaviour deterministic for the same reason SKILL.md fill is: `<...>` markers are unambiguously replaceable; prose above the block is not part of the template. Do not apply until the human/copy-fill tradeoff is resolved — see 2026-05-26 session discussion.

## 2026-05-26 — Overlap checks must scan the deployed directory, not just the source repo

`write-a-skill` existed only in `~/.agents/skills/` (installed from a pre-refactor source) and was invisible during a repo-level scan of `.agents/skills/`. Governance reviews and overlap checks that only look at the source repo will miss skills added by install.sh from other sources or prior runs. Fix: overlap checks must scan the deployed `~/.agents/skills/` directory, not just the repo's `.agents/skills/`.

## 2026-05-26 — `model:` field belongs in SKILL.md frontmatter, not META.md

Claude Code supports `model:` as a provider extension in SKILL.md frontmatter — it overrides the session model for the skill's turn and reverts after. Attempting to put it in META.md was wrong: META.md is provenance/audit metadata, not runtime config. The boundary: if a field affects agent behaviour at invocation time, it belongs in SKILL.md frontmatter; if it serves upgrade reviews and audit trails, it belongs in META.md.

## 2026-05-26 — Research agents present synthesis as spec fact

When asked to research skill sub-file best practices, the research sub-agent reported "Process goes in SKILL.md. Context goes in reference files" as if it were verbatim from the Claude Code docs or the Agent Skills spec. Checking agentskills.io directly showed the spec says: "There are no format restrictions" on the body. The principle is a reasonable synthesis, not a quoted rule — but it nearly landed in write-skill's constraints as authoritative spec language. Fix: always verify research agent claims against the primary source before encoding them as rules, especially for spec or documentation claims. Plausible synthesis is the hardest fabrication to catch because it's often correct in spirit.

## 2026-05-18 — Planning meta-commentary does not belong in deployed artifacts

During write-skill refactor, an "open thread" note (about a deferred research step) was written directly into the SKILL.md Process section. The user caught it. The rule it violated: a deployed artifact (SKILL.md, a runtime file loaded by agents) must not contain planning meta-commentary — deferred items, open threads, and implementation notes belong in the issue file, which is the planning artifact. The skill body should contain only content relevant to runtime execution. If a decision is deferred, record it in the issue and leave no trace in the skill. The distinction: issue = planning record; skill = executable instruction.