fix(kyberforge): align skill factory with agentskills.io spec on directory rules
Move test infrastructure (validate.bats, new-skill.bats) from scripts/ to tests/ — the spec defines scripts/ as executable code agents can run, so test files don't belong there. Add tests/README.md placeholders with bats-support dependency declaration. Update skill-audit to permit tests/ and flag other unlisted directories, add scripts/ purpose check, and add /skill-improve near-miss exclusion. Update skill-improve and skill-write to cover tests/ in directory lists, scaffold template, and authoring guidance. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -25,5 +25,6 @@ Provide the path to the skill directory to audit when invoking.
|
||||
| `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description length, line count, placeholder detection, script executable bit, and interactive-prompt detection |
|
||||
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
|
||||
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
|
||||
| `scripts/validate.bats` | Bats test suite for validate.sh — dev tooling, not shipped with the plugin |
|
||||
| `tests/validate.bats` | Bats test suite for validate.sh |
|
||||
| `tests/README.md` | Setup instructions for bats-support and bats-assert test dependencies |
|
||||
| `README.md` | This file |
|
||||
|
||||
@@ -10,6 +10,7 @@ description: >
|
||||
(findings only, no PASS noise) with Why and Fix per finding, suitable for agent
|
||||
handoff to /skill-improve or human auditability. Do not use to run evals, fix
|
||||
application code bugs, or perform general code review unrelated to skill quality.
|
||||
Do not use when the user wants improvements applied — use /skill-improve instead.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
category: factory
|
||||
@@ -63,8 +64,9 @@ Check each pattern is appropriate and correctly formed:
|
||||
|
||||
### File structure
|
||||
|
||||
- Only spec-defined directories present: `scripts/`, `references/`, `assets/`
|
||||
- No non-spec files (e.g. META.md, extra config files)
|
||||
- Permitted directories: `scripts/`, `references/`, `assets/`, `tests/`; flag any other unlisted directory as FAIL — the spec allows additional dirs but this skill permits only these four to keep skills focused
|
||||
- `scripts/` contains only executable code agents can run; test files (`.bats`, `*_test.*`, `test_*.sh`) in `scripts/` are a FAIL — they belong in `tests/`
|
||||
- No non-spec files at the skill root (e.g. META.md, extra config files outside permitted directories)
|
||||
- Optional directories contain real content — not just unfilled placeholder READMEs
|
||||
- `README.md` present and accurately describes the skill and its files
|
||||
- No cross-plugin path references — paths using `../`, `../../`, or absolute repo paths (e.g. `plugins/kyberforge/skills/...`) break when the plugin is installed to a cache; flag any found
|
||||
|
||||
26
plugins/kyberforge/skills/skill-audit/tests/README.md
Normal file
26
plugins/kyberforge/skills/skill-audit/tests/README.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Tests require [bats-support](https://github.com/bats-core/bats-support) and
|
||||
[bats-assert](https://github.com/bats-core/bats-assert) installed under
|
||||
`tests/test_helper/`:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests:
|
||||
|
||||
```bash
|
||||
bats tests/
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `validate.bats` | Bats test suite for `scripts/validate.sh` |
|
||||
@@ -35,7 +35,7 @@ Signals can come from anywhere in the conversation or referenced files:
|
||||
|
||||
## Step 2 — Gather and group signals
|
||||
|
||||
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/). Then collect all signals from the conversation and any file paths the user has referenced.
|
||||
Read the current skill files (SKILL.md and any files in scripts/, references/, assets/, tests/). Then collect all signals from the conversation and any file paths the user has referenced.
|
||||
|
||||
Group signals by **root cause**, not symptom. Ask: "What single gap in the skill causes this cluster of failures?" One root cause → one fix. Do not make a separate edit for each symptom.
|
||||
|
||||
@@ -55,7 +55,7 @@ Then proceed — edits are reversible via git, no approval checkpoint needed.
|
||||
|
||||
## Step 4 — Apply changes
|
||||
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, README.md.
|
||||
Edit any file in the skill directory that the signals point to: SKILL.md, scripts/, references/, assets/, tests/, README.md.
|
||||
|
||||
**Generalize, don't patch.** Find the underlying gap, not the specific example that failed. A fix scoped only to the test cases you've seen will overfit and perform worse on new inputs.
|
||||
|
||||
|
||||
@@ -35,7 +35,8 @@ This skill produces its best output when you arrive with rich context:
|
||||
| `assets/templates/scripts/README.md` | Placeholder for bundled scripts |
|
||||
| `assets/templates/references/README.md` | Placeholder for reference docs |
|
||||
| `assets/templates/assets/README.md` | Placeholder for static assets |
|
||||
| `scripts/new-skill.bats` | Bats test suite for new-skill.sh — dev tooling, not shipped with the plugin |
|
||||
| `assets/templates/tests/README.md` | Placeholder for test files |
|
||||
| `tests/new-skill.bats` | Bats test suite for new-skill.sh — requires bats-support and bats-assert under `tests/test_helper/` |
|
||||
|
||||
## Placement
|
||||
|
||||
|
||||
@@ -138,7 +138,7 @@ If the skill needs scripts with external package dependencies or language-specif
|
||||
|
||||
If no scripts are needed, delete `scripts/README.md` and the `scripts/` directory.
|
||||
|
||||
## Step 4 — Add references and assets (if needed)
|
||||
## Step 4 — Add references, assets, and tests (if needed)
|
||||
|
||||
**`references/`** — additional documentation loaded on demand. One topic per file.
|
||||
Reference conditionally from SKILL.md: `If <condition>, read references/<file>.md`.
|
||||
@@ -146,6 +146,10 @@ Reference conditionally from SKILL.md: `If <condition>, read references/<file>.m
|
||||
**`assets/`** — static resources: templates, schemas, lookup tables.
|
||||
Reference by relative path from SKILL.md.
|
||||
|
||||
**`tests/`** — test files for scripts in `scripts/`. Use when scripts are complex
|
||||
enough to break silently. Test infrastructure (`.bats`, `*_test.*`) belongs here,
|
||||
not in `scripts/`. See `tests/README.md` for setup instructions.
|
||||
|
||||
If not needed, delete the placeholder READMEs and their directories.
|
||||
|
||||
## Step 5 — Validate
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## When to add tests
|
||||
|
||||
Add tests here when the skill has scripts in `scripts/` that are complex enough
|
||||
to break silently — validators, parsers, generators, anything with branching
|
||||
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
|
||||
belongs here, not in `scripts/`.
|
||||
|
||||
## Test runner
|
||||
|
||||
Tests are run with [Bats](https://bats-core.readthedocs.io) for shell scripts.
|
||||
Install support libraries under `tests/test_helper/`:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests:
|
||||
|
||||
```bash
|
||||
bats tests/
|
||||
```
|
||||
|
||||
## If no tests are needed
|
||||
|
||||
Delete this README and the `tests/` directory entirely.
|
||||
@@ -84,4 +84,5 @@ echo " 1. Fill in $TARGET/SKILL.md — replace all FILL IN: placeholders"
|
||||
echo " 2. Add scripts to scripts/ if needed (or delete the directory)"
|
||||
echo " 3. Add docs to references/ if needed (or delete the directory)"
|
||||
echo " 4. Add resources to assets/ if needed (or delete the directory)"
|
||||
echo " 5. Validate: run /skill-audit on $TARGET"
|
||||
echo " 5. Add tests to tests/ if the skill has scripts (or delete the directory)"
|
||||
echo " 6. Validate: run /skill-audit on $TARGET"
|
||||
|
||||
Reference in New Issue
Block a user