feat(kyberforge): ground skill-audit qualitative rubric in agentskillsio spec

Add references/description-quality.md and references/body-discipline.md to
skill-audit — condensed, rubric-focused extracts from the agentskills.io
specification docs. Both files are loaded conditionally via progressive
disclosure triggers added to Step 3 (Description and Body discipline
dimensions), so the agent consults the spec source when a finding is
borderline rather than relying solely on inline heuristics developed
during the skill-write authoring cycle.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-22 19:53:45 +00:00
parent 6a94ccc270
commit 83277f4e5c
4 changed files with 136 additions and 0 deletions

View File

@@ -23,4 +23,6 @@ Provide the path to the skill directory to audit when invoking.
|------|---------|
| `SKILL.md` | Skill instructions for agents |
| `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description length, line count, placeholder detection, script executable bit, and interactive-prompt detection |
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
| `README.md` | This file |

View File

@@ -38,6 +38,8 @@ Work through each dimension. Cite file and line number for every finding.
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?
- **Length**: under 1024 characters?
If a description finding is borderline or the distinction between PASS and FAIL is unclear, read `references/description-quality.md`.
### Body discipline
For each sentence in the body, apply: *"Would the agent get this wrong without this sentence?"* Flag any that answer "no" as padding.
@@ -46,6 +48,8 @@ For each sentence in the body, apply: *"Would the agent get this wrong without t
- **Why rationale**: include/exclude rules explain why, not just what
- **Control calibration**: prescriptive for fragile or critical sequences; flexible where multiple approaches are valid
If uncertain whether a sentence is padding or whether a control decision is correctly calibrated, read `references/body-discipline.md`.
### Patterns
Check each pattern is appropriate and correctly formed:

View File

@@ -0,0 +1,82 @@
# Body Discipline Reference
Source: agentskills.io — skill-authoring
## The core test
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and dilutes the signal of what matters.
## What belongs in the body
Include content the agent lacks:
- Project-specific conventions and domain procedures it cannot infer
- Non-obvious edge cases and environment-specific gotchas
- The specific tools or sequences to use (not the full range of options)
- One default per decision point with one escape hatch
Do not include:
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
- Exhaustive option lists — pick a default; the agent doesn't benefit from choosing
- Steps the agent handles independently — over-specifying leads to unproductive paths
- Restatements of the description — it's already in context
## Calibrating control
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be followed:
```markdown
Run exactly:
\`\`\`bash
python scripts/migrate.py --verify --backup
\`\`\`
Do not modify the command or add additional flags.
```
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives — agents make better decisions when they understand the purpose.
## Defaults not menus
Never present a list of equivalent options — pick one and mention the alternative briefly:
```markdown
# Too many options
Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
# Default with escape hatch
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
```
## Gotchas sections
Highest value content — environment-specific facts that defy reasonable assumptions. Place near the top of the body so the agent reads them before encountering the situation.
```markdown
## Gotchas
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
```
Each entry must be a specific, surprising fact — not a general tip or reminder.
## Progressive disclosure
Keep `SKILL.md` under 500 lines. When more content is needed, move it to `references/` and load conditionally:
```markdown
If the API returns a non-200 status, read `references/api-errors.md`.
```
"If X, read Y" is more useful than "see references/ for details." The agent loads on demand rather than up front.
## Auditing guidance
Flag as FAIL if:
- A sentence answers "no" to the core test (would agent get this wrong without it?) — it is padding
- Decision points present a menu of options with no default
- Instructions repeat content already in the description
- Prescriptive sequences are used where flexibility is fine, or vice versa
Flag as SUGGESTION if:
- A rationale is missing from an include/exclude rule (present but unexplained)
- Gotchas are correct but placed late in the body rather than near the top
- A conditional reference trigger is vague ("see references/") rather than specific ("If X, read Y")

View File

@@ -0,0 +1,48 @@
# Description Quality Reference
Source: agentskills.io — optimizing-descriptions
## How triggering works
At startup, agents load only the `name` and `description` of each skill. When a user's task matches a description, the agent reads the full `SKILL.md` into context. **The description carries the entire triggering burden** — the body is never seen until after triggering.
Agents typically consult skills only for tasks requiring knowledge beyond their defaults. Specialized knowledge — unfamiliar APIs, domain-specific workflows, uncommon formats — is where description wording makes the difference.
## What a good description does
- **Imperative phrasing** — "Use when..." not "This skill does...". The agent is deciding whether to act.
- **User intent, not mechanics** — describe what the user is trying to achieve, not how the skill works internally.
- **Err toward being pushy** — explicitly name contexts where the skill applies, including cases where the user doesn't name the domain: "even if they don't mention X explicitly."
- **Specificity over vagueness** — "parses and validates OpenAPI specs" beats "helps with APIs."
- **Near-miss exclusions** — add "Do not use when..." only if a near-miss skill exists that could steal activations. Use strong near-misses (queries that share keywords but need something different), not weak ones ("write a fibonacci function").
- **Hard limit: 1024 characters** — descriptions grow during revision; check length before finalising.
## Before / after
```yaml
# Weak
description: Process CSV files.
# Strong
description: >
Analyze CSV and tabular data files — compute summary statistics,
add derived columns, generate charts, and clean messy data. Use when
the user has a CSV, TSV, or Excel file and wants to explore, transform,
or visualize the data, even if they don't explicitly mention "CSV" or
"analysis."
```
The strong version names capabilities precisely and broadens applicability beyond explicit keyword matches.
## Auditing guidance
Flag as FAIL if:
- Phrasing is descriptive ("This skill...") not imperative ("Use when...")
- Capabilities are vague ("helps with APIs") — require precise verbs and nouns
- No indirect trigger coverage when indirect cases clearly exist
- No near-miss exclusions when a sibling skill could plausibly steal activations
- Length exceeds 1024 characters
Flag as SUGGESTION if:
- Indirect trigger coverage exists but could be more specific
- Near-miss exclusions are present but target weak near-misses only