Defame1297
fc305ba7d9
Five documents told authors that a prose-form dangling routing target blocks. The
gate reports it as a SUGGESTION and exits 0. Verified on fixtures: `-> name` and
`/name` are blocking ERRORs, the prose form is SUGGESTION-tier unless a second
resolving target in the same sentence corroborates it. ADR-0020 and gates.md were
right; contract.md, retrofit.md, description-quality.md, finding-criteria.md and
agent-author's contract.md were wrong — and they are what an author and an auditor
actually read. The whole 39-skill corpus was retrofitted against them.
skill-audit was also self-contradictory: it imports validate.sh's SUGGESTIONs into
the Structure dimension verbatim while its own rubric grades the same target a FAIL,
so one target got reported twice at two tiers. The script owns the grade; the rubric
now says so.
The YAML-fold trap that broke gitea-labels-milestones (#100) was warned about only
in retrofit.md, reachable only from the improve flow when a budget is exceeded. It
is now in both contract.md files, which SKILL.md mandates on the create flow too.
agent-audit loaded both rubrics unconditionally on every run — 3,323 words for a
clean audit against skill-audit's 1,636. dac9cad fixed exactly this in skill-audit
and edited agent-audit in the same commit without applying it. Same treatment: the
criteria move to a new finding-criteria.md and load per dimension. Clean run now
2,083 words, a 37% cut.
Routing: apm-workflow's description shed dependency installation while still owning
the flow, and apm-install's boundary did not exclude it, so "install my apm
dependencies" matched the CLI-binary skill with no route back. Fixed on both sides.
forge regains two of the three phrasings the retrofit deleted.
forge Step 1 called grill-with-docs unconditionally — a skill in plugins/bin, which
kyberforge does not declare as a dependency. It resolves here only because the
walk-up sweeps sibling plugins; a standalone install dead-ends. Step 1 now names
the cross-plugin dependency and gives an inline fallback. Declaring it properly in
apm.yml remains the better fix.
Also: both audit SKILL.md files now grade exit 2 as "did not run, dimension
unverified" rather than as findings; skill-audit's README row described content that
moved, which its own finding-criteria.md grades a FAIL; and body-discipline.md's
`git show <sha>:plugins/...` command is fenced, since an installed plugin cache has
no repo and file-structure.md makes a bare repo path a FAIL.
Refs: #100, #101, #125
ADR: 0020
139 lines
6.7 KiB
Markdown
139 lines
6.7 KiB
Markdown
---
|
|
source_keys:
|
|
- agentskills-spec
|
|
- agentskills-best-practices
|
|
- agentskills-optimizing-descriptions
|
|
- agentskills-using-scripts
|
|
---
|
|
|
|
# Finding Criteria
|
|
|
|
Every FAIL and SUGGESTION criterion, for every qualitative dimension, and nothing else. The
|
|
reasoning each criterion stands on, its worked examples and its house rules stay in that
|
|
dimension's rubric, which Step 3 loads only for a dimension this file puts in play.
|
|
|
|
Two rules on using it:
|
|
|
|
- A criterion that plainly applies is a finding. Write it up citing file and line.
|
|
- A criterion that might apply, or whose call the wording here does not settle, is a reason to load
|
|
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
|
to read; it does not settle a close call on its own.
|
|
|
|
## description — `references/description-quality.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
|
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
|
|
- **Internal mechanics appear in the description.** Any of:
|
|
- capability enumeration or a feature list;
|
|
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
|
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
|
shared skill", "the human-facing entry point", "replaces the old flat invocation");
|
|
- implementation detail ("self-validates via a bundled deterministic script").
|
|
|
|
None of it can change a routing decision and all of it is preloaded.
|
|
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
|
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
|
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
|
as user phrasings, usually in the same order. One register, whichever routes better.
|
|
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
|
|
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
|
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
|
|
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
|
judgment.
|
|
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of
|
|
`references/description-quality.md`.
|
|
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
|
of the 400-character house ceiling above.
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
|
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
|
- A near-miss exclusion is present but targets a weak near-miss.
|
|
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
|
|
|
**An unresolved boundary target is not graded here.** `validate.sh` owns that call and tiers it by
|
|
notation — `/name` or an arrow form is an ERROR, the bare prose form a SUGGESTION unless a second
|
|
target in the same sentence resolves — and Step 1 has already filed it under `### Structure` at that
|
|
tier. Re-grading it as a description FAIL puts one target in the report twice at two tiers. What is
|
|
left to judgment here is semantic and the script cannot reach it: whether a target that *does*
|
|
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
|
files manually") should have named one.
|
|
|
|
## body-discipline — `references/body-discipline.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- A sentence answers "no" to the core test — it is padding
|
|
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
|
|
- Two or more mutually exclusive flows are inlined instead of dispatched
|
|
- A Gotcha paraphrases a step in the body below it that every branch reaching the Gotcha also
|
|
reaches
|
|
- A decision point presents a menu of options with no default
|
|
- An instruction repeats content already in the description
|
|
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- The body exceeds 600 words counted body-only but stays at or under 900
|
|
- The Gotchas section carries more than five entries
|
|
- The Gotchas section exceeds 25% of the body
|
|
- A rationale is missing from an include/exclude rule — present but unexplained
|
|
- Gotchas are correct but placed late in the body rather than near the top
|
|
- Content that only one branch reaches is inlined where a `references/` file would serve
|
|
|
|
## patterns — `references/patterns.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
|
|
assumption
|
|
- An inner code fence is unescaped inside a markdown block, breaking the render
|
|
- A checklist wraps a single step
|
|
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
|
|
- The agent must produce a specific format and no output template is given
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- Gotchas are correctly formed but placed late in the body
|
|
- An output template is present but permissive where the consumer needs it exact
|
|
- A conditional reference names a trigger that is real but broader than the branch it guards
|
|
|
|
## file-structure and internal-consistency — `references/file-structure.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- A directory outside the four permitted ones exists
|
|
- Test files sit in `scripts/`
|
|
- A non-spec file sits at the skill root
|
|
- A path that resolves outside the skill directory appears outside the two exempt locations, in
|
|
prose rather than in a fenced example
|
|
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
|
- `README.md` is absent, or its file table has a missing or stale row
|
|
- `SKILL.md` describes a script invocation the script does not accept
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- An optional directory exists but holds only a placeholder README
|
|
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
|
|
|
|
## formatting and scripts — `references/formatting-and-scripts.md`
|
|
|
|
Flag as FAIL if:
|
|
|
|
- A script prompts interactively, in any form
|
|
- A script exposes no `--help`
|
|
- A destructive script has no `--dry-run`
|
|
- Data and diagnostics share a stream, so the output cannot be piped
|
|
- A relative path named in the body does not resolve
|
|
- Heading levels are inconsistent enough to break the document's structure
|
|
|
|
Flag as SUGGESTION if:
|
|
|
|
- Exit codes are meaningful but undocumented in `--help`
|
|
- A code block is untagged where a language applies
|
|
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
|
|
- List indentation or section spacing is inconsistent without breaking the render
|