A review of PR #85's last two commits (1164f3a, 4d018af) found the new
release-gate script fails open in four separate ways, and the new drift
check for the duplicated Vale styles only ever detects drift after a
human already hand-edited both copies out of sync.
check-release-needed.sh:
- The `-e` existence filter dropped a RELEASE_PATHS entry from the diff
pathspec once it was deleted from the tree, so deleting a path exposed
via .pre-commit-hooks.yaml since the last tag passed the gate clean —
exactly the breakage the gate exists to catch. git diff reports
deletions fine without an existence check; the filter is gone.
- `git diff ... 2>/dev/null || true` turned any git failure (a shallow
clone missing the tag's objects, a corrupted ref) into an empty,
falsely-clean diff. The diff result is no longer swallowed: a failure
now hard-fails with the underlying git error visible.
- RELEASE_PATHS was a hand-maintained array duplicating
.pre-commit-hooks.yaml's entry: paths with only a comment holding them
in sync, and was already over-broad (it swept in validate.sh /
validate-provenance.sh, which no hook entry references). It's now
parsed straight from .pre-commit-hooks.yaml's entry: lines at
runtime, so it can't drift from the manifest and only tracks what a
hook actually exposes.
- `git describe --tags --abbrev=0` accepted any tag reachable from HEAD
as the diff baseline, not just release tags. Added
`--match 'v[0-9]*.[0-9]*.[0-9]*'` so an incidental checkpoint tag
can't shift the baseline and mask a real release-relevant change.
check-vale-style-sync.sh still only detects drift between skill-audit's
and agent-audit's duplicated vale-wrap.sh/styles/Kyberforge copies
(both copies must exist independently per the plugin's no-cross-skill-
path packaging rule — a symlink would break at install time). Added
scripts/sync-vale-styles.sh to regenerate skill-audit's copy from
agent-audit's canonical one on demand, and pointed the sync check's
failure message at it, so fixing drift is one command instead of a
hand diff across two files.
Also recorded, rather than silently left unfixed: check-release-needed.sh
only fires on a local `git push` through pre-commit's pre-push hook — a
PR merged via Gitea's merge button, or CI invoking
`pre-commit run --hook-stage pre-push` directly, never sets
PRE_COMMIT_REMOTE_BRANCH and skips the gate entirely. Closing that needs
a server-side CI job this repo doesn't have yet; documented as a known
limitation in ADR-0014 rather than papered over.
Separately, LESSONS.md's "a clean check can mean nothing ran" entry was
marked **Graduated** without ever being promoted per the repo's own
graduation rule (3+ instances → a standing doc, marked
`[graduated → target file]`). Actually promoted it into
core/instructions/testing.md and fixed the marker.
tests/test-check-release-needed.sh gained 4 regression tests, one per
check-release-needed.sh fix above, each verified to fail against the
pre-fix script and pass against the current one.
Verification: bash tests/run-tests.sh (11 scripts + 125 bats, all
passing), pre-commit run --all-files, and
pre-commit run --all-files --hook-stage pre-push all clean.
ADR: 0014
skill-audit/agent-audit's Step 1 resolved vale-wrap.sh/.vale.ini via
`git rev-parse --show-toplevel`, which returns whichever repo the skill
happens to run in. Inside ai-development that works; in any external repo
that installs kyberforge@holocron as a plugin, it resolves to that repo's
own root, which has no .vale.ini — the prefilter silently fell back to
full LLM judgment. ADR-0013 named this as a deliberately deferred gap.
Vale's config/styles/wrapper now ship inside the plugin itself: a
canonical copy in agent-audit/assets/vale/ (Kyberforge + KyberforgeCopilot,
the superset agent-audit needs) and a smaller duplicate in
skill-audit/assets/vale/ (Kyberforge only) — per the no-cross-skill-path
rule already established for plugin cache-installs. Both skills resolve
these relative to their own directory, same as scripts/validate.sh
already does.
A new root .pre-commit-hooks.yaml exposes both copies plus
skill-size-check so any external repo can enforce the same rules via
`repo: <this-repo-url>, rev: <tag>` in its own pre-commit config,
independent of Claude Code entirely — the same mechanism covers CI. This
repo's own pre-commit hook now consumes the identical plugin-bundled
copies via repo: local (not a third root copy, and not a pinned
self-reference, which would lint working-tree edits against the last
tagged release instead of the change being made). Split into
vale-audit-prefilter-skill/-agent hooks after confirming, by diffing the
full corpus against both old and new config before deleting the old
files, that one combined hook pointed at only one copy silently 0-file-
skips the other file type.
scripts/check-vale-style-sync.sh guards the two copies against drift,
wired at pre-push alongside check-manifests.
ADR: 0014
Three rounds of review on the Vale prefilter surfaced patterns worth
keeping rather than just fixing.
The first has now recurred three times in a single PR — a check reporting
success because it had silently not run — so it is flagged as a
graduation candidate per LESSONS.md's own three-instance rule.
- A clean linter result can mean "nothing was checked": the frontmatter
scope silently not matching, warning-level rules never affecting an
exit code, and globs matching zero files all produced green results
that were then cited as evidence of cleanliness.
- One signal, two consumers, no named distinction: Vale severities were
tuned for the audit report while the commit gate silently inherited the
resulting exit code, because CONTEXT.md described both as one mechanism.
- Measure a rule's false-positive rate at the severity you will ship it
at: VagueQualifier was trialled at warning, where a false positive is
free, and shipped at error, where it costs a blocked commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
## Why
ROADMAP.md was a static file that duplicated tracking information now
owned by Gitea milestones and issues. Keeping it created a maintenance
burden — references drifted out of sync with the actual state of work,
and agents were directed to read it when the source of truth had moved.
## Implementation Notes
All inbound references replaced with either the relevant Gitea milestone
("Skills & Agents") or removed where the context made them redundant.
Test assertions that verified ROADMAP.md content removed; test output
strings updated to drop the ROADMAP cross-reference instruction.
## Impact
Agents no longer read docs/ROADMAP.md at session start. Gitea milestones
and issues are the canonical source for roadmap and open-question tracking.
---
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Update docs to reflect pre-commit migration and cleanup:
- spec/overview.md: removed phantom test file references
- ROADMAP.md: removed references to non-existent test files
- LESSONS.md: removed reference to setup-hooks.sh bug
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
## Why
Two recurring failure modes surfaced during the agent-author workstream
that are worth capturing before they repeat: biased forks producing
false-PASS audits, and parallel forks producing conflicting fixes on
the same file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reorder skill-audit description to lead with 'Use when...' trigger (P3)
- Add concrete example to 'control calibration' body discipline check (P4)
- Add bats test files to README file tables for both skills
- Fix REPO_ROOT and SCRIPT paths in bats files after tests/ subdirectory removed
- Add three lessons: plugin cache isolation, spec-grounded rubrics, test file placement
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- claude plugin validate --strict absent from standard test sweep
- gitleaks source/deployed config silent divergence risk
- shellcheck without -x blocks pre-commit on scripts using source
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv5iNACZxumtF2k6TsK18q
- Add sub-file constraint: content-type split rule (explains vs. directs),
three spec-defined optional directories (scripts/, references/, assets/),
one-level-deep rule, and wiring syntax requirement
- Update output format section to list optional sub-files as a third output
- Add self-check item for sub-file placement and wiring
- Update SKILL-TEMPLATE.md constraints and output format examples to match
- Bump META.md to v1.3
- Add two LESSONS.md entries: research agents presenting synthesis as spec
fact; META-TEMPLATE fix deferred with explicit do-not-apply note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add model: sonnet to SKILL.md frontmatter (Claude Code extension; correct
location per research — runtime config belongs in frontmatter, not META.md)
- Document model field in SKILL-TEMPLATE.md with factory §9 routing guidance
- Wire grill output explicitly to trigger description step (step 3 now references
agreed name/category/purpose from grill rather than drafting independently)
- Strengthen eval self-check: presence check → coverage check (trigger cases +
at least one output case required)
- Bump META.md to v1.2 (2026-05-26)
- Add three LESSONS.md entries: META-TEMPLATE comment retention, overlap check
scope (deployed dir not just repo), model field placement boundary
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Complete 2026-05-18 workstream for write-skill refactor (issue 0018 phase 1).
Grill output (handoff in docs/issues/0018-factory-write-skill.md):
- 6-section standard replaces 8: Role and When/When not dropped (not in
agentskills.io spec; description carries trigger scope and negatives)
- Frontmatter reduced to 3 fields; provenance moves to META.md
(progressive disclosure — not loaded at every skill scan)
- SKILL-TEMPLATE.md and META-TEMPLATE.md as authoritative copy-fill artifacts
- CATEGORIES.md as self-contained category reference in skill directory
- Copy and fill explicitly split into separate process steps for determinism
New files in .agents/skills/write-skill/:
- SKILL-TEMPLATE.md — 6-section template with XML blocks and inline examples per section
- META-TEMPLATE.md — provenance schema with inline-commented YAML; source vs references explicit
- CATEGORIES.md — self-contained category table (no runtime dependency on factory docs)
- META.md — write-skill's own provenance (v1.1, self-authored)
SKILL.md rewritten: 8-step process, XML blocks (<requirements>, <steps>, <checks>),
3-field frontmatter, copy-then-fill steps explicit.
Supporting docs updated:
- skill-implementation-workflow.md: SKILL-TEMPLATE.md as authoritative template;
source:/references: now reference META.md; when: marked resolved
- factory-integration-decisions.md: frontmatter spec corrected
- ROADMAP.md: authoring standard description updated
- CONTEXT.md: Source field entry corrected; META.md added to glossary
- docs/spec/overview.md: 2026-05-18 refactor entry added
- docs/issues/0018: handoff status updated; acceptance criteria marked complete
LESSONS.md: 3 entries added in grill session (cargo-culted body sections,
provenance fields in frontmatter, copy-fill determinism) + 1 from implementation
(planning meta-commentary does not belong in deployed artifacts).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds write-eval, the first factory meta-skill. Produces eval.yaml test
files for skills following the two-section schema (trigger_tests +
output_tests) with provider-agnostic string assertions and show-plan-
then-merge-on-rerun behaviour. Hand-written bootstrap — subsequent
skills will use write-eval to produce their own evals.
Also tightens skill-implementation-workflow.md step 5b: per-section
options walk-through is now a named gate before writing, separate from
the synthesis grill. LESSONS.md entry added.
HITL behavioral test pending.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
13 manual scenarios run across instructions and governance layers (two
rounds for failures). Fixed four rules that lost to RLHF defaults:
- Exploratory question format: tightened with boundary framing; added
@import CONTEXT.md to repo CLAUDE.md and a standing rule to check
docs/adr/ and ROADMAP resolved entries before answering design questions
(3-round iteration to resolve)
- File-edit intent: added counter-example to stop clarification-seeking
- Push confirmation: reframed as "do not call the tool" not "ask first"
- Secrets rule: extended to cover credential reproduction in response
text and usage examples, with explicit placeholder requirement
Scenario 4 (push confirmation) inconclusive — no remote configured.
Governance scenario 3 (HITL on real infra) untestable — Nginx not installed.
Both share the same root cause: agent delegates to permission system.
Also corrects stale skill list in docs/spec/overview.md (12 actual
deployed skills vs 16 names previously listed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add LESSONS.md as the long-loop feedback log at repo root. Patterns
observed during development graduate to standing rules when three or
more entries appear.
Introduce docs/spec/ as the living spec layer: overview.md (deployed
state) and architecture.md (directory structure, install pipeline,
provider model, governance layer). Refactor docs/VISION.md to goals
and intent only — architecture content moved to docs/spec/architecture.md.
Update CLAUDE.md and CONTEXT.md to reference the new spec files.
Close issues 0013 and 0014. Update ROADMAP and factory integration
note to reflect completion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
