From 83277f4e5c3069ee7da5f315c4087fd5bb04e940 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Mon, 22 Jun 2026 19:53:45 +0000 Subject: [PATCH] feat(kyberforge): ground skill-audit qualitative rubric in agentskillsio spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add references/description-quality.md and references/body-discipline.md to skill-audit — condensed, rubric-focused extracts from the agentskills.io specification docs. Both files are loaded conditionally via progressive disclosure triggers added to Step 3 (Description and Body discipline dimensions), so the agent consults the spec source when a finding is borderline rather than relying solely on inline heuristics developed during the skill-write authoring cycle. Co-Authored-By: Claude Sonnet 4.6 --- .../kyberforge/skills/skill-audit/README.md | 2 + .../kyberforge/skills/skill-audit/SKILL.md | 4 + .../skill-audit/references/body-discipline.md | 82 +++++++++++++++++++ .../references/description-quality.md | 48 +++++++++++ 4 files changed, 136 insertions(+) create mode 100644 plugins/kyberforge/skills/skill-audit/references/body-discipline.md create mode 100644 plugins/kyberforge/skills/skill-audit/references/description-quality.md diff --git a/plugins/kyberforge/skills/skill-audit/README.md b/plugins/kyberforge/skills/skill-audit/README.md index 75c8dd4..bcb58f6 100644 --- a/plugins/kyberforge/skills/skill-audit/README.md +++ b/plugins/kyberforge/skills/skill-audit/README.md @@ -23,4 +23,6 @@ Provide the path to the skill directory to audit when invoking. |------|---------| | `SKILL.md` | Skill instructions for agents | | `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description length, line count, placeholder detection, script executable bit, and interactive-prompt detection | +| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline | +| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear | | `README.md` | This file | diff --git a/plugins/kyberforge/skills/skill-audit/SKILL.md b/plugins/kyberforge/skills/skill-audit/SKILL.md index 8ef52e8..54d57b5 100644 --- a/plugins/kyberforge/skills/skill-audit/SKILL.md +++ b/plugins/kyberforge/skills/skill-audit/SKILL.md @@ -38,6 +38,8 @@ Work through each dimension. Cite file and line number for every finding. - **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations? - **Length**: under 1024 characters? +If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`. + ### Body discipline For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding. @@ -46,6 +48,8 @@ For each sentence in the body, apply: *"Would the agent get this wrong without t - **Why rationale**: include/exclude rules explain why, not just what - **Control calibration**: prescriptive for fragile or critical sequences; flexible where multiple approaches are valid +If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`. + ### Patterns Check each pattern is appropriate and correctly formed: diff --git a/plugins/kyberforge/skills/skill-audit/references/body-discipline.md b/plugins/kyberforge/skills/skill-audit/references/body-discipline.md new file mode 100644 index 0000000..7f93144 --- /dev/null +++ b/plugins/kyberforge/skills/skill-audit/references/body-discipline.md @@ -0,0 +1,82 @@ +# Body Discipline Reference + +Source: agentskills.io — skill-authoring + +## The core test + +For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"** + +If no — cut it. The agent already knows it from general training. Adding it wastes tokens and dilutes the signal of what matters. + +## What belongs in the body + +Include content the agent lacks: +- Project-specific conventions and domain procedures it cannot infer +- Non-obvious edge cases and environment-specific gotchas +- The specific tools or sequences to use (not the full range of options) +- One default per decision point with one escape hatch + +Do not include: +- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is) +- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing +- Steps the agent handles independently — over-specifying leads to unproductive paths +- Restatements of the description — it's already in context + +## Calibrating control + +**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed: +```markdown +Run exactly: +\`\`\`bash +python scripts/migrate.py --verify --backup +\`\`\` +Do not modify the command or add additional flags. +``` + +**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives — agents make better decisions when they understand the purpose. + +## Defaults not menus + +Never present a list of equivalent options — pick one and mention the alternative briefly: +```markdown +# Too many options +Use pypdf, pdfplumber, PyMuPDF, or pdf2image... + +# Default with escape hatch +Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead. +``` + +## Gotchas sections + +Highest value content — environment-specific facts that defy reasonable assumptions. Place near the top of the body so the agent reads them before encountering the situation. + +```markdown +## Gotchas +- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`. +- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value. +``` + +Each entry must be a specific, surprising fact — not a general tip or reminder. + +## Progressive disclosure + +Keep `SKILL.md` under 500 lines. When more content is needed, move it to `references/` and load conditionally: + +```markdown +If the API returns a non-200 status, read `references/api-errors.md`. +``` + +"If X, read Y" is more useful than "see references/ for details." The agent loads on demand rather than up front. + +## Auditing guidance + +Flag as FAIL if: +- A sentence answers "no" to the core test (would agent get this wrong without it?) — it is padding +- Decision points present a menu of options with no default +- Instructions repeat content already in the description +- Prescriptive sequences are used where flexibility is fine, or vice versa + +Flag as SUGGESTION if: +- A rationale is missing from an include/exclude rule (present but unexplained) +- Gotchas are correct but placed late in the body rather than near the top +- A conditional reference trigger is vague ("see references/") rather than specific ("If X, read Y") diff --git a/plugins/kyberforge/skills/skill-audit/references/description-quality.md b/plugins/kyberforge/skills/skill-audit/references/description-quality.md new file mode 100644 index 0000000..1c4fd9f --- /dev/null +++ b/plugins/kyberforge/skills/skill-audit/references/description-quality.md @@ -0,0 +1,48 @@ +# Description Quality Reference + +Source: agentskills.io — optimizing-descriptions + +## How triggering works + +At startup, agents load only the `name` and `description` of each skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. **The description carries the entire triggering burden** — the body is never seen until after triggering. + +Agents typically consult skills only for tasks requiring knowledge beyond their defaults. Specialized knowledge — unfamiliar APIs, domain-specific workflows, uncommon formats — is where description wording makes the difference. + +## What a good description does + +- **Imperative phrasing** — "Use when..." not "This skill does...". The agent is deciding whether to act. +- **User intent, not mechanics** — describe what the user is trying to achieve, not how the skill works internally. +- **Err toward being pushy** — explicitly name contexts where the skill applies, including cases where the user doesn't name the domain: "even if they don't mention X explicitly." +- **Specificity over vagueness** — "parses and validates OpenAPI specs" beats "helps with APIs." +- **Near-miss exclusions** — add "Do not use when..." only if a near-miss skill exists that could steal activations. Use strong near-misses (queries that share keywords but need something different), not weak ones ("write a fibonacci function"). +- **Hard limit: 1024 characters** — descriptions grow during revision; check length before finalising. + +## Before / after + +```yaml +# Weak +description: Process CSV files. + +# Strong +description: > + Analyze CSV and tabular data files — compute summary statistics, + add derived columns, generate charts, and clean messy data. Use when + the user has a CSV, TSV, or Excel file and wants to explore, transform, + or visualize the data, even if they don't explicitly mention "CSV" or + "analysis." +``` + +The strong version names capabilities precisely and broadens applicability beyond explicit keyword matches. + +## Auditing guidance + +Flag as FAIL if: +- Phrasing is descriptive ("This skill...") not imperative ("Use when...") +- Capabilities are vague ("helps with APIs") — require precise verbs and nouns +- No indirect trigger coverage when indirect cases clearly exist +- No near-miss exclusions when a sibling skill could plausibly steal activations +- Length exceeds 1024 characters + +Flag as SUGGESTION if: +- Indirect trigger coverage exists but could be more specific +- Near-miss exclusions are present but target weak near-misses only