Retrofits all 39 skills to ADR-0020's description/body context contract, then fixes what six rounds of independent review found in that retrofit — including four ways the hot gate itself failed open. Closes #99, #107, #108, #110, #111, #114, #115, #120. ## The retrofit (waves 1-5) | | Start | Now | |---|---|---| | Description FAILs (>400 chars) | 26 | **0** | | Body FAILs (>900 words, body-only) | 9 | **0** | | Dangling routing targets | 2 | **0** | | `Kyberforge.CompositionNote` | 10 | **0** | | Preload tax | 21,005 chars | **~10,500** | Under the 12,000-char success criterion. Per-wave detail is on #99. ## The review fixes **The gate failed open four ways, three of them found after the retrofit shipped.** An unrecognised follower token made a dangling target vanish. A skill directory with no `SKILL.md` resolved as a valid target, so a commit could be green locally and red in a fresh clone — three existing fixtures were relying on that, one of which made the install-leak A/B pass vacuously. Then the free-standing `/name` sweep turned out to be gated on the sentence carrying a boundary marker, so route notation in any other sentence was invisible — not an ERROR, not a SUGGESTION, not an INFO — which left the documented "`/name` always blocks" promise false from a second direction. All four fixed and pinned. **Two checks were silently not running.** `validate-provenance.sh` checks 7-8 were dead across nine skills. Waking them exposed a deeper problem: they assume `Research doc:` names a source index, but 30 of 121 entries point at topic content documents, so every new check-7 INFO was a false positive and check 8 was saved from a false-FAIL flood only by an *unannounced* skip. Checks 7/8 are now scoped to source indexes and every skip announces itself (#121). **The retrofit's own anti-goal, four times.** ADR-0020 warns that a blunt gate gets satisfied by deleting content rather than relocating it. `diagnose` and `skill-audit` relocated prose and then read it unconditionally; `prototype` and `vale-config` deleted rules outright that survived nowhere. All four addressed. ## Verification - `bash tests/run-tests.sh --strict` — 24 suites, 0 skipped, 0 failed - `bash tests/run-bats.sh` — 325 tests, 0 failures - `pre-commit run --all-files` — 17/17 - `pre-commit run --hook-stage pre-push --all-files` — 16/16, with `apm marketplace check` and `apm pack --check-clean` run against the remote, not skipped - `scripts/skill-size-check.sh` over all 39 skills — rc 0, 0 ERROR/FAIL, SUGGESTION-only - Preload tax measured at **10,498 chars**, max description 390 — both inside budget - Every new test proven non-vacuous by a deliberate mutation of the behaviour it covers **Per-commit sync, stated accurately:** the ten commits from the latest review round each pass `check-plugin-content-sync` in isolation, verified by checking each out in a detached worktree with a clean between. The earlier gitea window (`dfacf05..bedbd1d`, nine commits) does **not** — its mirror was regenerated in one batch at `bbc7300`. An earlier revision of this description claimed the property held for every commit; it does not, and a bisect through that window lands on a red commit. **Squash-merge** to collapse it, or accept that this range is not bisectable. ## Version bump Six plugins and the catalog take a **patch**, not a minor. The branch is **89 commits — 40 `fix` / 30 `refactor` / 12 `docs` / 5 `chore` / 2 `test` — zero `feat`, zero `!`, zero `BREAKING CHANGE`** — and adds no skill, agent, command or hook. (Two earlier revisions of this section cited a stale histogram, most recently 78 commits; the figures above are measured at HEAD.) Both rules this repo ships (`forge/references/version-bump.md`, landing in this PR, and `git-commits/references/conventional-commits-spec.md`) make that a patch, and the catalog set is unchanged at 7 entries. Not settled by that: four published files were removed from the installed tree, three moved, and `caveman` gained `disable-model-invocation`, retiring its old triggers. Under a strict reading those are major-class and currently ship under `refactor:` with no marker. Whether the deployed skill surface is a public contract is written down nowhere — worth deciding, but it outlives this PR. ## Deliberately not in scope #112 (cherry-pick ownership, now resolved in favour of `git-commits`), #113 (`rtk git` normalisation), #116 (research fan-out), #101 (audit-skill merge), #122 (non-spec skill-root files), #123 (no PRD producer) stay open. #117 is the one worth reading: the contract's remedy is to move prose into `references/`, which is exactly where neither the size gate nor Vale looks — and the blind spot is wider than #117 currently records, since there is no root `.vale.ini` at all, so every ADR, `CONTEXT.md` and `README.md` is unlinted too. That blind spot let this branch carry two `level: error` `Kyberforge.SentenceOpenerThereIs` violations into `references/` files it created — `provider-adapter-author/references/provider-matrix.md:31` and `agent-audit/references/finding-criteria.md:95`. Both are reworded in `afadaae`, confirmed by routing each file through the audit's own `vale-wrap.sh` (1 error each before, 0 after). Five further occurrences sit in `references/` files already on `main`; those are the pre-existing corpus and stay with #117, which is the real fix. Also unfixed and not this PR's: `apm install` appends a duplicate `SessionStart` entry to `.claude/settings.json`, so a fresh clone cannot get pre-push green without an edit AGENTS.md warns against. Reproduces identically on `main`. Co-authored-by: Defame1297 <gitea@rkdr.net> Reviewed-on: https://git.dev.rkdr.net/Defame1297/holocron/pulls/129 Co-authored-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net> Co-committed-by: Claude Code AI - Gitea MCP <claude@noreply.git.dev.rkdr.net>
99 lines
5.5 KiB
Markdown
99 lines
5.5 KiB
Markdown
---
|
|
source_keys:
|
|
- context7-websites-code-claude
|
|
- claude-code-plugins-docs
|
|
- claude-code-subagents-docs
|
|
- context7-github-en-copilot
|
|
- github-custom-agents-configuration
|
|
---
|
|
|
|
# Finding Criteria
|
|
|
|
Every FAIL and SUGGESTION criterion, for every qualitative dimension, and nothing else. The
|
|
reasoning each criterion stands on, its worked examples and its house rules stay in that
|
|
dimension's rubric, which Step 3 loads only for a dimension this file puts in play.
|
|
|
|
Two rules on using it:
|
|
|
|
- A criterion that plainly applies is a finding. Write it up citing file and line.
|
|
- A criterion that might apply, or whose call the wording here does not settle, is a reason to load
|
|
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
|
to read; it does not settle a close call on its own.
|
|
|
|
## description — `references/description-quality.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
|
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut. Agent
|
|
descriptions have no platform-documented ceiling of their own, so 400 is the only hard limit
|
|
there is — do not go looking for a backstop behind it.
|
|
- **Internal mechanics appear in the description.** Any of:
|
|
- capability enumeration or a feature list;
|
|
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
|
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
|
shared agent", "the human-facing entry point", "replaces the old flat invocation");
|
|
- implementation detail ("self-validates via a bundled deterministic script").
|
|
|
|
None of it can change a routing decision and all of it is preloaded.
|
|
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
|
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
|
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
|
as user phrasings, usually in the same order. One register, whichever routes better.
|
|
- **Descriptive rather than imperative phrasing** (`This agent ...`, `This is the ...`).
|
|
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
|
- **Vague capabilities** ("helps with agents" where "audits an agent definition pair" was
|
|
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
|
judgment.
|
|
- **`Use proactively` in a Copilot or vendor-neutral description.**
|
|
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
|
|
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
|
|
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0 of
|
|
`references/description-quality.md`.
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
|
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
|
- A near-miss exclusion is present but targets a weak near-miss.
|
|
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
|
|
|
**An unresolved boundary target is not graded here.** `validate.sh` resolves boundary targets for
|
|
agent files at both scopes and tiers the verdict itself — route notation (`/name`, an arrow form)
|
|
is an ERROR, the bare prose form a SUGGESTION unless a second target in the same sentence resolves.
|
|
Step 1 has already filed it under `### Structure` at that tier. Take the script's verdict rather
|
|
than re-resolving the name by hand, and do not re-grade it under description: a hand-walk over a
|
|
different universe can contradict the script, and re-grading puts one target in the report twice.
|
|
What is left to judgment is semantic and the script cannot reach it: whether a target that *does*
|
|
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
|
files manually") should have named one.
|
|
|
|
## body, delegation and comment-discipline — `references/body-and-delegation.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- The body restates a procedure owned by a skill the agent can invoke — Fix: invoke `<skill>`
|
|
instead
|
|
- A sentence answers "no" to the core test — it is padding
|
|
- A decision point presents a menu of options with no default
|
|
- An instruction repeats content already in the description
|
|
- Frontmatter comments are template scaffolding rather than instruction, or are HTML comments at
|
|
plugin/APM scope
|
|
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- The body does not open with a direct role instruction
|
|
- The body specifies no error handling — nothing tells the agent what to do with malformed,
|
|
missing or contradictory input
|
|
- The job the agent describes is unbounded, or bounded only implicitly
|
|
- A rationale is missing from a rule the agent is expected to enforce — present but unexplained
|
|
- Comments are useful but verbose enough to bury the field they annotate
|
|
|
|
**Never report an agent body as too long on a word count.** ADR-0020 gates a skill body at
|
|
600/900 words and deliberately gates an agent body at nothing, because an agent body *becomes* the
|
|
system prompt of a fresh context rather than competing with a live conversation. No number exists
|
|
to cite. The one length signal that applies is the Copilot runtime's 30,000-character body limit,
|
|
which `validate.sh` already reports as a SUGGESTION. Length is judged through the delegation FAIL
|
|
above instead.
|