refactor(kyberforge): make skill-write and skill-audit self-contained with shared resources
- Move validate.sh ownership to skill-audit/scripts/ — it is the canonical structural validator; skill-write now delegates Step 5 to /skill-audit - Add skill-write/references/scripts.md and deployment-modes.md for progressive disclosure of package runner patterns and plugin cache isolation rules - Fix skill-audit Step 1 cross-skill path reference (was repo-absolute, now skill-relative); add manual fallback for sandboxed/Bash-denied contexts - Scope Step 2 "read every file" to exclude binaries and unreferenced files - Fix new-skill.sh next-steps output to reference /skill-audit instead of the removed validate.sh - Remove stale Dependencies section from skill-audit README; flip dependency arrow — skill-write depends on skill-audit, not vice versa Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -4,7 +4,7 @@ Audit a skill directory against the agentskills.io specification. Runs structura
|
||||
|
||||
## What it does
|
||||
|
||||
1. Runs `validate.sh` from `skill-write` for structural checks (name format, description length, line count, placeholder detection, script rules)
|
||||
1. Runs `scripts/validate.sh` for structural checks (name format, description length, line count, placeholder detection, script rules)
|
||||
2. Reads all files in the skill directory
|
||||
3. Applies qualitative checks across seven dimensions
|
||||
4. Outputs a PASS/FAIL/SUGGESTION punch list with a specific fix proposal for every FAIL
|
||||
@@ -22,8 +22,5 @@ Provide the path to the skill directory to audit when invoking.
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `SKILL.md` | Skill instructions for agents |
|
||||
| `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description length, line count, placeholder detection, script executable bit, and interactive-prompt detection |
|
||||
| `README.md` | This file |
|
||||
|
||||
## Dependencies
|
||||
|
||||
Requires `skill-write` to be installed at `plugins/kyberforge/skills/skill-write/` for the structural validation step. If not present, the agent performs structural checks manually.
|
||||
|
||||
@@ -16,19 +16,15 @@ metadata:
|
||||
|
||||
## Step 1 — Structural validation
|
||||
|
||||
Locate the `validate.sh` script from `skill-write`:
|
||||
|
||||
```bash
|
||||
bash plugins/kyberforge/skills/skill-write/scripts/validate.sh <skill-dir>
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
```
|
||||
|
||||
If `validate.sh` is not found, note it and proceed — perform the checks it would cover manually.
|
||||
|
||||
List every FAIL from the structural check in the punch list before continuing.
|
||||
List every FAIL from the structural check in the punch list before continuing. If the script cannot execute (python3 unavailable, Bash denied, or permission error), perform structural checks manually: name format, name matches directory, description length ≤1024 chars, SKILL.md ≤500 lines, no unfilled `FILL IN:` placeholders, scripts executable and free of interactive prompts.
|
||||
|
||||
## Step 2 — Read all skill files
|
||||
|
||||
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, and `assets/`. Do not skip files — internal consistency checks require the full picture.
|
||||
Read every file in the skill directory: `SKILL.md`, `README.md` (if present), all files in `scripts/`, `references/`, and `assets/`. Skip binary files and files not directly referenced from `SKILL.md`. Do not skip text files — internal consistency checks require the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
@@ -66,6 +62,7 @@ Check each pattern is appropriate and correctly formed:
|
||||
- No non-spec files (e.g. META.md, extra config files)
|
||||
- Optional directories contain real content — not just unfilled placeholder READMEs
|
||||
- `README.md` present and accurately describes the skill and its files
|
||||
- No cross-plugin path references — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/kyberforge/skills/...`) break when the plugin is installed to a cache; flag any found
|
||||
|
||||
### Formatting
|
||||
|
||||
|
||||
179
plugins/kyberforge/skills/skill-audit/scripts/validate.sh
Executable file
179
plugins/kyberforge/skills/skill-audit/scripts/validate.sh
Executable file
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate.sh <skill-dir>
|
||||
|
||||
Validate a skill directory against the agentskills.io specification.
|
||||
|
||||
Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed
|
||||
1 One or more checks failed
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: skill-dir is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 -u - "$1" <<'PYTHON'
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
|
||||
if not os.path.isfile(skill_md):
|
||||
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with open(skill_md) as f:
|
||||
content = f.read()
|
||||
|
||||
failed = False
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
|
||||
def fail(msg):
|
||||
global failed
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
failed = True
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
|
||||
if not fm_match:
|
||||
fail("No valid YAML frontmatter block found (expected ---...---)")
|
||||
sys.exit(1)
|
||||
|
||||
fm = fm_match.group(1)
|
||||
body_start = fm_match.end()
|
||||
|
||||
# Extract name
|
||||
name_m = re.search(r'^name:\s*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — inline or block scalar (> or |)
|
||||
desc = ""
|
||||
desc_m = re.search(r'^description:\s*([>|])\n((?:[ \t]+.+\n?)+)', fm, re.MULTILINE)
|
||||
if desc_m:
|
||||
raw = desc_m.group(2)
|
||||
desc = re.sub(r'\s+', ' ', raw).strip()
|
||||
else:
|
||||
desc_inline = re.search(r'^description:\s*(.+)', fm, re.MULTILINE)
|
||||
if desc_inline:
|
||||
desc = desc_inline.group(1).strip()
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
# --- Checks ---
|
||||
|
||||
# name present
|
||||
if name:
|
||||
ok(f"name present: '{name}'")
|
||||
else:
|
||||
fail("name field is missing or empty")
|
||||
|
||||
# name matches directory
|
||||
if name and dir_name:
|
||||
if name == dir_name:
|
||||
ok(f"name '{name}' matches directory '{dir_name}'")
|
||||
else:
|
||||
fail(f"name '{name}' does not match directory '{dir_name}'")
|
||||
|
||||
# name length
|
||||
if name:
|
||||
if len(name) <= 64:
|
||||
ok(f"name length {len(name)} chars (limit: 64)")
|
||||
else:
|
||||
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
|
||||
|
||||
# name format
|
||||
if name:
|
||||
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
|
||||
ok(f"name format valid (kebab-case)")
|
||||
else:
|
||||
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
|
||||
|
||||
# description present
|
||||
if desc:
|
||||
ok(f"description present")
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
|
||||
# but not backtick-quoted references like `FILL IN:` used in instructions.
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
# description contains unfilled placeholder
|
||||
if desc and PLACEHOLDER_RE.search(desc):
|
||||
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
|
||||
else:
|
||||
if desc:
|
||||
ok("description has no unfilled placeholders")
|
||||
|
||||
# SKILL.md line count
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= 500:
|
||||
ok(f"SKILL.md line count {line_count} (limit: 500)")
|
||||
else:
|
||||
fail(f"SKILL.md line count {line_count} — exceeds 500-line limit")
|
||||
|
||||
# Body unfilled placeholders
|
||||
body = content[body_start:]
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
scripts = [f for f in os.listdir(scripts_dir)
|
||||
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
|
||||
for fname in scripts:
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
with open(fpath) as f:
|
||||
sc = f.read()
|
||||
# Interactive prompt heuristic
|
||||
if re.search(r'^\s*(read\s|input\()', sc, re.MULTILINE):
|
||||
fail(f"scripts/{fname}: may use interactive input (read/input detected)")
|
||||
else:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
if os.access(fpath, os.X_OK):
|
||||
ok(f"scripts/{fname}: is executable")
|
||||
else:
|
||||
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
|
||||
|
||||
# Summary
|
||||
print()
|
||||
if not failed:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
PYTHON
|
||||
Reference in New Issue
Block a user