Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

docs: make ADR-0020 match what actually shipped, and record what did not

Browse Source 
The ADR was written against base commit `f9b919d` and then not updated as the
implementation moved, so several of its numbers were measuring one thing and being
read as another — the exact conflation the ADR exists to stop, reproduced inside
it. Corrections, all reproducible now that each figure states its method:

- The preload tax is 23,427 chars / ~5,900 tokens, not 23,612 / ~6,200.
- `MAX_WORDS=2770` is a density proxy for the agentskills.io ~5,000-token ceiling,
  not "2× p90". Neither percentile reaches it: 2× the body-only p90 is 2,698 and
  2× the whole-file p90 is 3,052. Reading it as a percentile pairs a whole-file
  gate against a body-only distribution.
- `apm-workflow` is a 421-word body; 554 is its whole-file count. `skill-author`
  and `agent-author` were 2,623 and 2,582 body words — 2,760 and 2,758 whole-file,
  which is where "within twelve words of the gate" comes from. Two numbers for one
  file is the point, and only one of them is what either gate measures.
- Every `file:line` citation now says it resolves against `f9b919d`, since this
  change rewrites most of the cited files.

Three things the ADR asserted that no validator implemented are now filed by tier
in an exhaustive enforcement table — deterministic, prose-pattern, or auditor
judgment — because a rule filed under "Enforcement" that nothing enforces is the
failure mode this ADR is most exposed to. The Gotchas entry count moves to
SUGGESTION to match the script; the paraphrase FAIL is marked as an auditor's,
since semantic equivalence is not pattern-matchable.

Two gaps recorded rather than quietly left:

- The agent body-gate exemption lives in `agent-audit`'s validator and in the
  `skill-size-check` hook's `SKILL.md`-only `files:` pattern — *not* in
  `scripts/skill-size-check.sh`, which measures whatever path it is handed and
  today reports 900-word body FAILs on `git-orchestrate` (933),
  `gitea-orchestrate` (1,199) and `apm-orchestrate` (1,080). Agents escape by file
  pattern, not because the script knows the difference, so widening that pattern
  would silently enforce a gate this ADR declines to set.
- The `skill-audit`/`agent-audit` merge is deferred to #101. This change made the
  split deeper, not shallower: the dispatch retrofit took them from 3 and 4
  reference files to 7 and 8, and their two same-named `description-quality.md`
  files now differ on 100 of ~120 lines after normalising skill/agent. The merge
  reopens ADR-0008 and touches every call site in `skill-author`, `agent-author`
  and `forge`, so it is its own change. #100 carries the dangling-target fixes.

AGENTS.md and CONTEXT.md take the same corrections plus the two live setup
changes: PyYAML is now a hard requirement rather than an optional accelerator (a
fallback that mis-parses an unfamiliar scalar shape reports a clean pass on a file
it never measured), and `.claude/settings.json`'s `pretty-format-json` exclusion is
documented as load-bearing rather than as a tidy-up candidate.

LESSONS.md's autofix entry is corrected on its own provenance, which it got wrong
in both directions. `git log --date=iso` puts the introducing commit at 18:47 and
the fix at 21:54 — three hours, not "weeks" — and `git branch -a --contains` puts
the introducing commit on this branch only, not on main. It was manufactured
inside the same PR that diagnosed it. The added lesson is that "pre-existing" is a
claim about history and history is queryable: a defect found while working on a
branch feels inherited, and the feeling is not evidence.

Refs: ADR-0020, #99, #100, #101
This commit is contained in:
Defame1297
2026-08-16 16:41:45 +00:00
parent d02765d595
commit 64ffb9f35a
4 changed files with 254 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
LESSONS.md
View File

@@ -194,7 +194,11 @@ is the broken multi-`raw:` form, which `tests/test-vale-hooks-consumer.sh` now f
## 2026-08-14 — Un-anchoring a description rule to reach mid-sentence text is unshippable
Widening `DescriptionOpener` to catch `gitea-workflow`'s mid-description "This is the human-facing
entry point…" looked like a one-character change. Under `scope: text.frontmatter.description`, `^`
entry point…" looked like a one-character change. Both that skill and `gitea-labels-milestones`
*open* with "Use when…" and satisfy the opener rule; the offending clause sits at character 377 and
300 of the folded value respectively, so the rule was never violated and never silently passed — it
simply had no jurisdiction, which is a different defect and takes a different fix.
Under `scope: text.frontmatter.description`, `^`
anchors to the start of the whole description value — and `vale-wrap.sh` has already flattened that
value to one physical line, so `(?m)` changes nothing. Un-anchoring is therefore the only route to
mid-description text, and measured across the corpus it scores 5 hits and 5 false positives: skills
@@ -206,21 +210,38 @@ a new case does not fit.
## 2026-08-14 — A formatter in the commit path manufactures drift on a file with a clean git diff
`apm audit --ci` failed for weeks on `.claude/settings.json` while `git diff` on that file was empty —
the worst possible pairing of signals, because the file matched HEAD exactly and every instinct says
"nothing changed here". The content was identical to apm's output to the byte; only the JSON key
order differed. `pretty-format-json --autofix` sorts object keys unless `--no-sort-keys` is passed,
and its `exclude:` listed fifteen generated manifests but not this file, so from the commit that
first wrote a hook entry there (`2e395a4`) onward, apm's insertion-ordered output was silently
re-sorted on the way in. apm then replayed the install, produced its own order, and reported drift
against a file no human had touched.
`apm audit --ci` failed on `.claude/settings.json` while `git diff` on that file was empty — the worst
possible pairing of signals, because the file matched HEAD exactly and every instinct says "nothing
changed here". The content was identical to apm's output to the byte; only the JSON key order
differed. `pretty-format-json --autofix` sorts object keys unless `--no-sort-keys` is passed, and its
`exclude:` listed fifteen generated manifests but not this file, so from the commit that first wrote
a hook entry there onward, apm's insertion-ordered output was silently re-sorted on the way in. apm
then replayed the install, produced its own order, and reported drift against a file no human had
touched.
Two general points. First, a tool-owned generated file that passes through an autofixing formatter is
drifted by construction, and the diff that would reveal it never appears in `git diff` — it only
The provenance matters as much as the mechanism, and the first account of this entry got it wrong in
both directions. `git log --format='%h %ad %s' --date=iso` puts the introducing commit `2e395a4` at
2026-08-14 18:47 and the fix `7607522` at 21:54 — roughly three hours, not "weeks". And `2e395a4` is
the **first commit of the `refactor/trim-skills-agents-context` branch**, eleven minutes after the
base merge `f9b919d`; `git branch -a --contains 2e395a4` returns only that branch and its own
`remotes/origin/` tracking copy — two lines naming one branch, and `main` is not among them. So
this was not a latent defect inherited from `main`, it was manufactured inside the same PR that
diagnosed it, and the fixing commit's own message calling it "pre-existing … red at HEAD before
ADR-0020 work began" is the mis-attribution rather than the record. Two cheap commands would have
settled it before either sentence was written.
Three general points. First, a tool-owned generated file that passes through an autofixing formatter
is drifted by construction, and the diff that would reveal it never appears in `git diff` — it only
exists between the formatter's input and its output, which nothing stores. Second, the fix is
self-undoing unless the exclude lands in the same commit: correcting the file alone means the hook
re-breaks it as it is staged. Fix: when a tool declares ownership of a path, add that path to every
autofixing hook's `exclude` at the moment ownership is declared, not when the drift is noticed. This
repo gates marketplace-mirror, plugin-content and vale-style drift deterministically and has no
equivalent gate asserting tool-owned paths stay out of formatter scope — `.claude/settings.json` was
the sixteenth exclude and nothing prevents a seventeenth.
re-breaks it as it is staged. Third — the one this entry had to learn twice — "pre-existing" is a
claim about history, and history is queryable; a defect found while working on a branch feels
inherited, and the feeling is not evidence. A three-hour-old self-inflicted bug and a months-old
inherited one call for different responses, and writing the wrong one down converts a process failure
into a story about someone else's neglect. Fix: when a tool declares ownership of a path, add that
path to every autofixing hook's `exclude` at the moment ownership is declared, not when the drift is
noticed — and before describing any defect as pre-existing, run `git log -S` or
`git branch --contains` on the commit that introduced it. This repo gates marketplace-mirror,
plugin-content and vale-style drift deterministically and has no equivalent gate asserting tool-owned
paths stay out of formatter scope — `.claude/settings.json` was the sixteenth exclude and nothing
prevents a seventeenth.