The aggregate-assertion failure joins the "a clean result can mean nothing ran"
family as its fifth instance: a total over N subjects is satisfiable by a
proper subset, so it proves nothing about any individual subject. Records the
reverse mutation sweep — neuter each assertion, confirm exactly one case fails
— as standing practice for checks whose failure mode is silence.
The second entry is about accepted residuals: the U+2019 rewrite survived
review because its justification was documented in the same breath as the
workaround, and the covering test asserted the residual's presence rather than
the behaviour it cost. Documentation records a belief; a belief adjacent to a
workaround is the one most worth attacking.
Refs: #85
A config's local mode can prove nothing about the mode that ships:
repo: local collapses the clone prefix, cwd and repo root into one
directory, so a byte-identical entry: string worked locally for a
reason that exists only locally, through three review rounds.
Deleting a token from a shared artifact breaks whatever parses it,
silently: dropping --config killed the loop that gave the bundled
Vale styles release coverage, shrinking a derived path list with no
error and no failing test.
Kept separate from the adjacent "clean linter result" and "one signal,
two consumers" entries, which describe different failure modes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
A review of PR #85's last two commits (1164f3a, 4d018af) found the new
release-gate script fails open in four separate ways, and the new drift
check for the duplicated Vale styles only ever detects drift after a
human already hand-edited both copies out of sync.
check-release-needed.sh:
- The `-e` existence filter dropped a RELEASE_PATHS entry from the diff
pathspec once it was deleted from the tree, so deleting a path exposed
via .pre-commit-hooks.yaml since the last tag passed the gate clean —
exactly the breakage the gate exists to catch. git diff reports
deletions fine without an existence check; the filter is gone.
- `git diff ... 2>/dev/null || true` turned any git failure (a shallow
clone missing the tag's objects, a corrupted ref) into an empty,
falsely-clean diff. The diff result is no longer swallowed: a failure
now hard-fails with the underlying git error visible.
- RELEASE_PATHS was a hand-maintained array duplicating
.pre-commit-hooks.yaml's entry: paths with only a comment holding them
in sync, and was already over-broad (it swept in validate.sh /
validate-provenance.sh, which no hook entry references). It's now
parsed straight from .pre-commit-hooks.yaml's entry: lines at
runtime, so it can't drift from the manifest and only tracks what a
hook actually exposes.
- `git describe --tags --abbrev=0` accepted any tag reachable from HEAD
as the diff baseline, not just release tags. Added
`--match 'v[0-9]*.[0-9]*.[0-9]*'` so an incidental checkpoint tag
can't shift the baseline and mask a real release-relevant change.
check-vale-style-sync.sh still only detects drift between skill-audit's
and agent-audit's duplicated vale-wrap.sh/styles/Kyberforge copies
(both copies must exist independently per the plugin's no-cross-skill-
path packaging rule — a symlink would break at install time). Added
scripts/sync-vale-styles.sh to regenerate skill-audit's copy from
agent-audit's canonical one on demand, and pointed the sync check's
failure message at it, so fixing drift is one command instead of a
hand diff across two files.
Also recorded, rather than silently left unfixed: check-release-needed.sh
only fires on a local `git push` through pre-commit's pre-push hook — a
PR merged via Gitea's merge button, or CI invoking
`pre-commit run --hook-stage pre-push` directly, never sets
PRE_COMMIT_REMOTE_BRANCH and skips the gate entirely. Closing that needs
a server-side CI job this repo doesn't have yet; documented as a known
limitation in ADR-0014 rather than papered over.
Separately, LESSONS.md's "a clean check can mean nothing ran" entry was
marked **Graduated** without ever being promoted per the repo's own
graduation rule (3+ instances → a standing doc, marked
`[graduated → target file]`). Actually promoted it into
core/instructions/testing.md and fixed the marker.
tests/test-check-release-needed.sh gained 4 regression tests, one per
check-release-needed.sh fix above, each verified to fail against the
pre-fix script and pass against the current one.
Verification: bash tests/run-tests.sh (11 scripts + 125 bats, all
passing), pre-commit run --all-files, and
pre-commit run --all-files --hook-stage pre-push all clean.
ADR: 0014
skill-audit/agent-audit's Step 1 resolved vale-wrap.sh/.vale.ini via
`git rev-parse --show-toplevel`, which returns whichever repo the skill
happens to run in. Inside ai-development that works; in any external repo
that installs kyberforge@holocron as a plugin, it resolves to that repo's
own root, which has no .vale.ini — the prefilter silently fell back to
full LLM judgment. ADR-0013 named this as a deliberately deferred gap.
Vale's config/styles/wrapper now ship inside the plugin itself: a
canonical copy in agent-audit/assets/vale/ (Kyberforge + KyberforgeCopilot,
the superset agent-audit needs) and a smaller duplicate in
skill-audit/assets/vale/ (Kyberforge only) — per the no-cross-skill-path
rule already established for plugin cache-installs. Both skills resolve
these relative to their own directory, same as scripts/validate.sh
already does.
A new root .pre-commit-hooks.yaml exposes both copies plus
skill-size-check so any external repo can enforce the same rules via
`repo: <this-repo-url>, rev: <tag>` in its own pre-commit config,
independent of Claude Code entirely — the same mechanism covers CI. This
repo's own pre-commit hook now consumes the identical plugin-bundled
copies via repo: local (not a third root copy, and not a pinned
self-reference, which would lint working-tree edits against the last
tagged release instead of the change being made). Split into
vale-audit-prefilter-skill/-agent hooks after confirming, by diffing the
full corpus against both old and new config before deleting the old
files, that one combined hook pointed at only one copy silently 0-file-
skips the other file type.
scripts/check-vale-style-sync.sh guards the two copies against drift,
wired at pre-push alongside check-manifests.
ADR: 0014
Three rounds of review on the Vale prefilter surfaced patterns worth
keeping rather than just fixing.
The first has now recurred three times in a single PR — a check reporting
success because it had silently not run — so it is flagged as a
graduation candidate per LESSONS.md's own three-instance rule.
- A clean linter result can mean "nothing was checked": the frontmatter
scope silently not matching, warning-level rules never affecting an
exit code, and globs matching zero files all produced green results
that were then cited as evidence of cleanliness.
- One signal, two consumers, no named distinction: Vale severities were
tuned for the audit report while the commit gate silently inherited the
resulting exit code, because CONTEXT.md described both as one mechanism.
- Measure a rule's false-positive rate at the severity you will ship it
at: VagueQualifier was trialled at warning, where a false positive is
free, and shipped at error, where it costs a blocked commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
## Why
ROADMAP.md was a static file that duplicated tracking information now
owned by Gitea milestones and issues. Keeping it created a maintenance
burden — references drifted out of sync with the actual state of work,
and agents were directed to read it when the source of truth had moved.
## Implementation Notes
All inbound references replaced with either the relevant Gitea milestone
("Skills & Agents") or removed where the context made them redundant.
Test assertions that verified ROADMAP.md content removed; test output
strings updated to drop the ROADMAP cross-reference instruction.
## Impact
Agents no longer read docs/ROADMAP.md at session start. Gitea milestones
and issues are the canonical source for roadmap and open-question tracking.
---
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Update docs to reflect pre-commit migration and cleanup:
- spec/overview.md: removed phantom test file references
- ROADMAP.md: removed references to non-existent test files
- LESSONS.md: removed reference to setup-hooks.sh bug
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
## Why
Two recurring failure modes surfaced during the agent-author workstream
that are worth capturing before they repeat: biased forks producing
false-PASS audits, and parallel forks producing conflicting fixes on
the same file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reorder skill-audit description to lead with 'Use when...' trigger (P3)
- Add concrete example to 'control calibration' body discipline check (P4)
- Add bats test files to README file tables for both skills
- Fix REPO_ROOT and SCRIPT paths in bats files after tests/ subdirectory removed
- Add three lessons: plugin cache isolation, spec-grounded rubrics, test file placement
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- claude plugin validate --strict absent from standard test sweep
- gitleaks source/deployed config silent divergence risk
- shellcheck without -x blocks pre-commit on scripts using source
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gv5iNACZxumtF2k6TsK18q
- Add sub-file constraint: content-type split rule (explains vs. directs),
three spec-defined optional directories (scripts/, references/, assets/),
one-level-deep rule, and wiring syntax requirement
- Update output format section to list optional sub-files as a third output
- Add self-check item for sub-file placement and wiring
- Update SKILL-TEMPLATE.md constraints and output format examples to match
- Bump META.md to v1.3
- Add two LESSONS.md entries: research agents presenting synthesis as spec
fact; META-TEMPLATE fix deferred with explicit do-not-apply note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add model: sonnet to SKILL.md frontmatter (Claude Code extension; correct
location per research — runtime config belongs in frontmatter, not META.md)
- Document model field in SKILL-TEMPLATE.md with factory §9 routing guidance
- Wire grill output explicitly to trigger description step (step 3 now references
agreed name/category/purpose from grill rather than drafting independently)
- Strengthen eval self-check: presence check → coverage check (trigger cases +
at least one output case required)
- Bump META.md to v1.2 (2026-05-26)
- Add three LESSONS.md entries: META-TEMPLATE comment retention, overlap check
scope (deployed dir not just repo), model field placement boundary
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Complete 2026-05-18 workstream for write-skill refactor (issue 0018 phase 1).
Grill output (handoff in docs/issues/0018-factory-write-skill.md):
- 6-section standard replaces 8: Role and When/When not dropped (not in
agentskills.io spec; description carries trigger scope and negatives)
- Frontmatter reduced to 3 fields; provenance moves to META.md
(progressive disclosure — not loaded at every skill scan)
- SKILL-TEMPLATE.md and META-TEMPLATE.md as authoritative copy-fill artifacts
- CATEGORIES.md as self-contained category reference in skill directory
- Copy and fill explicitly split into separate process steps for determinism
New files in .agents/skills/write-skill/:
- SKILL-TEMPLATE.md — 6-section template with XML blocks and inline examples per section
- META-TEMPLATE.md — provenance schema with inline-commented YAML; source vs references explicit
- CATEGORIES.md — self-contained category table (no runtime dependency on factory docs)
- META.md — write-skill's own provenance (v1.1, self-authored)
SKILL.md rewritten: 8-step process, XML blocks (<requirements>, <steps>, <checks>),
3-field frontmatter, copy-then-fill steps explicit.
Supporting docs updated:
- skill-implementation-workflow.md: SKILL-TEMPLATE.md as authoritative template;
source:/references: now reference META.md; when: marked resolved
- factory-integration-decisions.md: frontmatter spec corrected
- ROADMAP.md: authoring standard description updated
- CONTEXT.md: Source field entry corrected; META.md added to glossary
- docs/spec/overview.md: 2026-05-18 refactor entry added
- docs/issues/0018: handoff status updated; acceptance criteria marked complete
LESSONS.md: 3 entries added in grill session (cargo-culted body sections,
provenance fields in frontmatter, copy-fill determinism) + 1 from implementation
(planning meta-commentary does not belong in deployed artifacts).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds write-eval, the first factory meta-skill. Produces eval.yaml test
files for skills following the two-section schema (trigger_tests +
output_tests) with provider-agnostic string assertions and show-plan-
then-merge-on-rerun behaviour. Hand-written bootstrap — subsequent
skills will use write-eval to produce their own evals.
Also tightens skill-implementation-workflow.md step 5b: per-section
options walk-through is now a named gate before writing, separate from
the synthesis grill. LESSONS.md entry added.
HITL behavioral test pending.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
13 manual scenarios run across instructions and governance layers (two
rounds for failures). Fixed four rules that lost to RLHF defaults:
- Exploratory question format: tightened with boundary framing; added
@import CONTEXT.md to repo CLAUDE.md and a standing rule to check
docs/adr/ and ROADMAP resolved entries before answering design questions
(3-round iteration to resolve)
- File-edit intent: added counter-example to stop clarification-seeking
- Push confirmation: reframed as "do not call the tool" not "ask first"
- Secrets rule: extended to cover credential reproduction in response
text and usage examples, with explicit placeholder requirement
Scenario 4 (push confirmation) inconclusive — no remote configured.
Governance scenario 3 (HITL on real infra) untestable — Nginx not installed.
Both share the same root cause: agent delegates to permission system.
Also corrects stale skill list in docs/spec/overview.md (12 actual
deployed skills vs 16 names previously listed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add LESSONS.md as the long-loop feedback log at repo root. Patterns
observed during development graduate to standing rules when three or
more entries appear.
Introduce docs/spec/ as the living spec layer: overview.md (deployed
state) and architecture.md (directory structure, install pipeline,
provider model, governance layer). Refactor docs/VISION.md to goals
and intent only — architecture content moved to docs/spec/architecture.md.
Update CLAUDE.md and CONTEXT.md to reference the new spec files.
Close issues 0013 and 0014. Update ROADMAP and factory integration
note to reflect completion.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
