Defame1297/holocron
0
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases 3 Wiki Activity

refactor(kyberforge): move audit guidance out of the per-run rubric load

Browse Source 
skill-audit loaded roughly 4,268 words of rubric on every run, most of it
criteria for findings a clean skill never triggers. The auditing guidance moves
into finding-criteria.md, read only when a finding is actually raised, cutting
a clean audit to about 999 words. The named-skill exemption is replaced with
properties, so the rubric stops carrying a list that ages the moment a skill
is renamed.

apm-workflow's `type:` trap sat in one flow while biting several, so it is
promoted to a common gate reachable from all of them; its claim to be
self-contained was untrue once it started routing to apm-install. skill-author's
contract had drifted from body-discipline.md and is realigned, and agent-audit's
field inventory is brought in line with the same split.
This commit is contained in:
Defame1297
2026-08-31 08:02:19 +00:00
parent e869912374
commit dac9cad912
42 changed files with 540 additions and 342 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
plugins/kyberforge/.apm/skills/skill-audit/references/body-discipline.md
View File

@@ -80,14 +80,28 @@ table** plus the gates common to every branch, and each flow lives in its own se
`references/` file. Inlining all of them is a FAIL regardless of word count, because every
invocation then pays for every branch it did not take.
The reference shape in this repo is `apm-workflow`: a **237-word body** dispatching to roughly
3,400 words of references across five mutually exclusive invocations. Its whole-file count is 304
words — cite 237 when calibrating a body, or the conflation this section warns against reappears
in the finding itself.
The reference shape in this repo is `apm-workflow`: a **294-word body** dispatching to 3,154 words
of references across five mutually exclusive flows. Its whole-file count is 348 words — cite 294
when calibrating a body, or the conflation this section warns against reappears in the finding
itself. The 3,154 counts the five flow files only; `references/sources.md` is a provenance record
and is never loaded at runtime, so counting it inflates the dispatched total.
Note its wiring: a three-column table (invocation, action, reference file) closed by one line,
*"Read only the reference file matching the requested action …"* That is the endorsed shape, and it
is why the literal-conditional requirement above exempts a body that dispatches. Do not flag it.
### What earns the wiring exemption
A dispatch table earns the exemption above on its properties, not on which skill it appears in.
Audit any dispatching body against these four:
- Every flow the skill handles has a row, and every row names a target file that exists on disk.
- Each row pairs a condition the agent can evaluate from the request with exactly one target. A row
keyed on a literal slash invocation fails this: a model-invoked activation never produces that
string, so the routing silently falls to whatever else the row carries.
- One line after the table tells the agent to read the file its row matched, and only that one.
- The gates every branch needs sit in the body, not inside one flow's file — see the reachability
precondition below.
A table missing any of the four is not exempt, and the literal-conditional requirement applies to it
as written. The exemption covers the wiring form only: every other rule in this file applies to a
dispatching skill exactly as it applies to any other.
## Gotchas sections
@@ -184,24 +198,5 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
```
## Auditing guidance
Flag as FAIL if:
- A sentence answers "no" to the core test — it is padding
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
- Two or more mutually exclusive flows are inlined instead of dispatched
- A Gotcha paraphrases a step in the body below it that every branch reaching the Gotcha also
reaches
- A decision point presents a menu of options with no default
- An instruction repeats content already in the description
- A prescriptive sequence is used where flexibility is fine, or the reverse
Flag as SUGGESTION if:
- The body exceeds 600 words counted body-only but stays at or under 900
- The Gotchas section carries more than five entries
- The Gotchas section exceeds 25% of the body
- A rationale is missing from an include/exclude rule — present but unexplained
- Gotchas are correct but placed late in the body rather than near the top
- Content that only one branch reaches is inlined where a `references/` file would serve
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
which Step 3 loads on every run.

37
plugins/kyberforge/.apm/skills/skill-audit/references/description-quality.md
View File

@@ -80,38 +80,5 @@ description: >
(`data-model` is illustrative. In a real description the target has to resolve.)
## Auditing guidance
Flag as FAIL if:
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
- **Internal mechanics appear in the description.** Any of:
- capability enumeration or a feature list;
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
shared skill", "the human-facing entry point", "replaces the old flat invocation");
- implementation detail ("self-validates via a bundled deterministic script").
None of it can change a routing decision and all of it is preloaded.
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
judgment. This is the rule that deflates a description, so apply it before reaching for length.
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
as user phrasings, usually in the same order. One register, whichever routes better.
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
judgment.
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
file in the authoring source. `validate.sh` reports the unresolved name.
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0.
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
of the 400-character house ceiling above.
Flag as SUGGESTION if:
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
tier only stops outliers. Report it rather than treating a 399-character description as clean.
- A near-miss exclusion is present but targets a weak near-miss.
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
which Step 3 loads on every run.

41
plugins/kyberforge/.apm/skills/skill-audit/references/file-structure.md
View File

@@ -24,9 +24,19 @@ knows to look at. Flag any other directory as a FAIL.
## Cross-plugin path references
A plugin is copied to a cache on install, and a path that climbs out of the skill directory stops
resolving there. Flag any `../`, `../../`, or absolute repo path (`plugins/<plugin>/skills/<other>/`
and its APM-native equivalent `.apm/skills/<other>/`) appearing in `SKILL.md`, `scripts/`,
`references/` or `assets/`.
resolving there. Flag a path in `SKILL.md`, `scripts/`, `references/` or `assets/` when it
**resolves outside the skill directory** — an absolute repo path
(`plugins/<plugin>/skills/<other>/` and its APM-native equivalent `.apm/skills/<other>/`), a
plugin-root path (`docs/`, `bin/`), or a `../` chain that leaves the skill root.
Resolve before flagging, twice over:
- **Resolve the path.** `$SKILL_DIR/../assets/templates` climbs one level from a `scripts/`
directory and lands back inside the same skill, so it resolves in a cache install and is not a
finding. A bare `../` is not the defect; leaving the skill is.
- **Skip fenced code blocks.** A path inside a fenced block is an example, and rubrics quote outside
paths deliberately as negative examples of what not to write. Flag a fenced path only when the
surrounding prose presents it as the form to copy.
**Referring to another skill's file.** There is one sanctioned spelling, and it is possessive:
`skill-audit's references/validation-scripts.md`. Write the skill by name and let the reader
@@ -39,9 +49,10 @@ on-disk check. Flag any other spelling of a cross-skill reference.
Two directories are exempt, and the exemptions are structural rather than discretionary:
- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers,
not runtime references. They are expected to be unresolvable after install, and
`validate-provenance.sh` handles that by skipping upstream checks silently when the path is
absent. Flagging them would make every correctly-provenanced skill fail.
not runtime references. They are expected to be unresolvable after install, so
`validate-provenance.sh` does not treat an absent path as a FAIL — it emits an INFO naming the
slug and stating that checks 7 and 8 did not run for it. Flagging them as broken references
would make every correctly-provenanced skill fail.
- **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared
`tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/`
exists and `tests/README.md` is absent or does not document it, that is a FAIL.
@@ -60,19 +71,5 @@ The skill has to agree with itself. Three checks:
A stale README row is the most common finding here and the easiest to miss from inside an
authoring pass, because the author knows what was intended and reads it into the gap.
## Auditing guidance
Flag as FAIL if:
- A directory outside the four permitted ones exists
- Test files sit in `scripts/`
- A non-spec file sits at the skill root
- A cross-plugin or parent-relative path appears outside the two exempt locations
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
- `README.md` is absent, or its file table has a missing or stale row
- `SKILL.md` describes a script invocation the script does not accept
Flag as SUGGESTION if:
- An optional directory exists but holds only a placeholder README
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
which Step 3 loads on every run.

132
plugins/kyberforge/.apm/skills/skill-audit/references/finding-criteria.md Normal file
View File

@@ -0,0 +1,132 @@
---
source_keys:
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
---
# Finding Criteria
Every FAIL and SUGGESTION criterion, for every qualitative dimension, and nothing else. The
reasoning each criterion stands on, its worked examples and its house rules stay in that
dimension's rubric, which Step 3 loads only for a dimension this file puts in play.
Two rules on using it:
- A criterion that plainly applies is a finding. Write it up citing file and line.
- A criterion that might apply, or whose call the wording here does not settle, is a reason to load
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
to read; it does not settle a close call on its own.
## description — `references/description-quality.md`
Flag as FAIL if:
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
- **Internal mechanics appear in the description.** Any of:
- capability enumeration or a feature list;
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
shared skill", "the human-facing entry point", "replaces the old flat invocation");
- implementation detail ("self-validates via a bundled deterministic script").
None of it can change a routing decision and all of it is preloaded.
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
judgment. This is the rule that deflates a description, so apply it before reaching for length.
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
as user phrasings, usually in the same order. One register, whichever routes better.
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
judgment.
- **A boundary clause naming a target that does not resolve** to a real skill directory or agent
file in the authoring source. `validate.sh` reports the unresolved name.
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of
`references/description-quality.md`.
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
of the 400-character house ceiling above.
Flag as SUGGESTION if:
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
tier only stops outliers. Report it rather than treating a 399-character description as clean.
- A near-miss exclusion is present but targets a weak near-miss.
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
## body-discipline — `references/body-discipline.md`
Flag as FAIL if:
- A sentence answers "no" to the core test — it is padding
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
- Two or more mutually exclusive flows are inlined instead of dispatched
- A Gotcha paraphrases a step in the body below it that every branch reaching the Gotcha also
reaches
- A decision point presents a menu of options with no default
- An instruction repeats content already in the description
- A prescriptive sequence is used where flexibility is fine, or the reverse
Flag as SUGGESTION if:
- The body exceeds 600 words counted body-only but stays at or under 900
- The Gotchas section carries more than five entries
- The Gotchas section exceeds 25% of the body
- A rationale is missing from an include/exclude rule — present but unexplained
- Gotchas are correct but placed late in the body rather than near the top
- Content that only one branch reaches is inlined where a `references/` file would serve
## patterns — `references/patterns.md`
Flag as FAIL if:
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
assumption
- An inner code fence is unescaped inside a markdown block, breaking the render
- A checklist wraps a single step
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
- The agent must produce a specific format and no output template is given
Flag as SUGGESTION if:
- Gotchas are correctly formed but placed late in the body
- An output template is present but permissive where the consumer needs it exact
- A conditional reference names a trigger that is real but broader than the branch it guards
## file-structure and internal-consistency — `references/file-structure.md`
Flag as FAIL if:
- A directory outside the four permitted ones exists
- Test files sit in `scripts/`
- A non-spec file sits at the skill root
- A path that resolves outside the skill directory appears outside the two exempt locations, in
prose rather than in a fenced example
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
- `README.md` is absent, or its file table has a missing or stale row
- `SKILL.md` describes a script invocation the script does not accept
Flag as SUGGESTION if:
- An optional directory exists but holds only a placeholder README
- `README.md` is accurate but describes a file's purpose more thinly than `SKILL.md` does
## formatting and scripts — `references/formatting-and-scripts.md`
Flag as FAIL if:
- A script prompts interactively, in any form
- A script exposes no `--help`
- A destructive script has no `--dry-run`
- Data and diagnostics share a stream, so the output cannot be piped
- A relative path named in the body does not resolve
- Heading levels are inconsistent enough to break the document's structure
Flag as SUGGESTION if:
- Exit codes are meaningful but undocumented in `--help`
- A code block is untagged where a language applies
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
- List indentation or section spacing is inconsistent without breaking the render

19
plugins/kyberforge/.apm/skills/skill-audit/references/formatting-and-scripts.md
View File

@@ -44,20 +44,5 @@ follow from that:
undocumented one is a coin flip.
- **`--dry-run` present for destructive operations.**
## Auditing guidance
Flag as FAIL if:
- A script prompts interactively, in any form
- A script exposes no `--help`
- A destructive script has no `--dry-run`
- Data and diagnostics share a stream, so the output cannot be piped
- A relative path named in the body does not resolve
- Heading levels are inconsistent enough to break the document's structure
Flag as SUGGESTION if:
- Exit codes are meaningful but undocumented in `--help`
- A code block is untagged where a language applies
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
- List indentation or section spacing is inconsistent without breaking the render
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
which Step 3 loads on every run.

18
plugins/kyberforge/.apm/skills/skill-audit/references/patterns.md
View File

@@ -50,19 +50,5 @@ forms are judgment.
`references/` when only one dispatch branch produces that output. A template inlined for a branch
most invocations never take is body-discipline padding.
## Auditing guidance
Flag as FAIL if:
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
assumption
- An inner code fence is unescaped inside a markdown block, breaking the render
- A checklist wraps a single step
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
- The agent must produce a specific format and no output template is given
Flag as SUGGESTION if:
- Gotchas are correctly formed but placed late in the body
- An output template is present but permissive where the consumer needs it exact
- A conditional reference names a trigger that is real but broader than the branch it guards
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
which Step 3 loads on every run.

8
plugins/kyberforge/.apm/skills/skill-audit/references/sources.md
View File

@@ -15,7 +15,7 @@
- **URL:** https://agentskills.io/specification.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md, references/validation-scripts.md
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
- **Status:** `extracted`
## agentskills-best-practices
@@ -23,7 +23,7 @@
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md, references/finding-criteria.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
@@ -31,7 +31,7 @@
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** SKILL.md, references/description-quality.md
- **Contributing files:** SKILL.md, references/description-quality.md, references/finding-criteria.md
- **Status:** `extracted`
## agentskills-evaluating-skills
@@ -47,7 +47,7 @@
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md, references/validation-scripts.md
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
- **Status:** `extracted`
## agentskills-quickstart