3 Commits

Author SHA1 Message Date
6910f1b5a5 fix(lint): reject checkpoint-suffixed tags as the release-gate baseline
git describe --match is a shell glob, not a regex: the trailing `*`s
in 'v[0-9]*.[0-9]*.[0-9]*' match any suffix, so a tag like
v1.2.3-checkpoint or v1.2.3-rc1 satisfied the pattern and could be
picked as LAST_TAG instead of the true last release. That silently
shifts the diff baseline and can let a push skip a required release.
--exclude '*-*' rules out any tag carrying a hyphenated suffix.

Added a regression test that tags a release-relevant change with a
v1.0.1-checkpoint tag right after v1.0.0 and asserts the gate still
fires — confirmed it fails against the pre-fix script and passes
against the fix.
2026-08-09 19:55:36 +00:00
050aec4c80 docs(kyberforge): document Vale-wiring files in skill-audit/agent-audit READMEs
Both READMEs' file tables predate #85's Vale wiring and never picked
up scripts/vale-wrap.sh or the assets/vale/ style tree, so a reader
of either README had no way to find where the new Step 1 sub-check
actually lives. List the new files and note the Vale sub-check in
"What it does" for both skills.
2026-08-09 19:55:27 +00:00
0a41b2c7d3 fix(kyberforge): restore skill-audit's action-verb opening check
skill-audit's Description dimension implied Vale's DescriptionOpener
alert fully covers imperative-phrasing checks, but that rule only
matches the literal "This skill..." pattern. agent-audit kept its
equivalent manual "does the description open with a verb" fallback
bullet; skill-audit's got dropped when Vale wiring landed in #85,
leaving other non-imperative openers (gerunds, passive phrasing) to
sail through unflagged. Restore the parallel check.
2026-08-09 19:55:19 +00:00
5 changed files with 40 additions and 3 deletions

View File

@@ -4,7 +4,7 @@ Audits a Claude Code and Copilot agent definition file pair for correctness and
## What it does
Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `sources.md` at the plugin root), then qualitative checks on description phrasing and system prompt quality. Produces a compact findings report in the same format as `skill-audit`.
Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `sources.md` at the plugin root), then qualitative checks on description phrasing and system prompt quality. Step 1 also runs a Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the `Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is unavailable or reports `0 files` scanned. Produces a compact findings report in the same format as `skill-audit`.
## Usage
@@ -19,6 +19,12 @@ Pass the path to either agent file as the argument.
| File | Purpose |
|------|---------|
| `SKILL.md` | Skill instructions for agents |
| `assets/vale/.vale.ini` | Vale config: scopes `Kyberforge` to `**/agents/*.md`, `Kyberforge`+`KyberforgeCopilot` to `**/*.agent.md` |
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Flags descriptions opening with "This skill/agent" instead of an imperative "Use when..." |
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Flags generic "see references/ for info" pointers instead of specific file references |
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Flags sentences opening with "There is/are" instead of naming the subject directly |
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Flags vague capability wording ("helps with", "utilize", "assists with", "used for") in descriptions |
| `assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml` | Flags CC-specific "Use proactively" phrasing with no effect in Copilot descriptions |
| `references/README.md` | Directory documentation for references/ |
| `references/description-quality.md` | Qualitative guide for borderline description findings |
| `references/field-inventory.md` | Authoritative list of valid CC and Copilot agent fields |
@@ -26,6 +32,7 @@ Pass the path to either agent file as the argument.
| `scripts/README.md` | Directory documentation for scripts/ |
| `scripts/validate.sh` | Structural validation script for agent file pairs |
| `scripts/validate-provenance.sh` | Provenance chain validation script for agent pairs against `sources.md` (plugin root) |
| `scripts/vale-wrap.sh` | Drop-in `vale` wrapper that works around a frontmatter-description NLP scope limitation |
| `tests/README.md` | Bats test dependency and run instructions |
| `tests/validate.bats` | Bats tests for validate.sh |
| `tests/validate-provenance.bats` | Bats tests for validate-provenance.sh |

View File

@@ -4,7 +4,7 @@ Audit a skill directory against the agentskills.io specification. Runs structura
## What it does
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks
1. Runs `scripts/validate.sh` and `scripts/validate-provenance.sh` for structural and provenance checks, plus `scripts/vale-wrap.sh` — a Vale prefilter that deterministically flags known-bad description openers, vague wording, padding phrases, and "There is/are" sentence openers
2. Reads all files in the skill directory
3. Applies qualitative checks across seven dimensions
4. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix — and a result block with handoff to /skill-improve
@@ -24,6 +24,12 @@ Provide the path to the skill directory to audit when invoking.
| `SKILL.md` | Skill instructions for agents |
| `scripts/validate.sh` | Structural validator — checks name format, name matches directory, description length, line count, placeholder detection, script executable bit, and interactive-prompt detection |
| `scripts/validate-provenance.sh` | Provenance validator — checks sources.md completeness, source_keys/slug consistency, Contributing files existence, bidirectional linkage, Research doc: fields, and upstream research doc alignment |
| `scripts/vale-wrap.sh` | Vale prefilter wrapper — runs the bundled `Kyberforge` Vale styles against SKILL.md and reports alerts as deterministic FAILs ahead of Step 3's qualitative review |
| `assets/vale/.vale.ini` | Vale configuration — points Vale at the bundled `Kyberforge` style path, self-located relative to `vale-wrap.sh` |
| `assets/vale/styles/Kyberforge/DescriptionOpener.yml` | Vale rule — flags literal "This skill..."/"This agent..." description openers |
| `assets/vale/styles/Kyberforge/PaddingPhrase.yml` | Vale rule — flags generic "see references/" padding phrasing in conditional references |
| `assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml` | Vale rule — flags body sentences starting with "There is"/"There are" |
| `assets/vale/styles/Kyberforge/VagueWording.yml` | Vale rule — flags known filler wording (e.g. "helps with", "utilize") |
| `references/description-quality.md` | Spec-grounded rubric for description auditing — loaded when a finding is borderline |
| `references/body-discipline.md` | Spec-grounded rubric for body discipline auditing — loaded when padding vs necessity is unclear |
| `references/sources.md` | Provenance record — agentskills.io sources that informed this skill and which files each contributed to |

View File

@@ -55,6 +55,7 @@ Work through each dimension internally. Collect findings only; report them in St
Vale's `Kyberforge.DescriptionOpener` ("This skill..." openers) and `Kyberforge.VagueWording` (filler like "helps with", "utilize") alerts from Step 1 — both FAILs — cover imperative phrasing and known vague-wording filler directly; report them as findings without re-deriving by judgment. The rest is still a judgment call:
- **Action-verb opening**: does the description start with a verb ("Audits...", "Reviews...", "Validates...")? Vale's `Kyberforge.DescriptionOpener` alert only catches the literal "This skill..." pattern — confirming an arbitrary opening word is genuinely a strong verb still requires judgment.
- **Specificity beyond the filler blocklist**: are capabilities stated precisely ("parses OpenAPI specs") or genuinely vaguely ("handles files")?
- **Indirect triggers**: does it cover cases where the user doesn't name the domain directly?
- **Near-miss exclusions**: are "Do not use when..." clauses present if a near-miss skill could steal activations?

View File

@@ -55,7 +55,12 @@ fi
# experiment tag reachable from the pushed ref must not shift the diff baseline.
# The tag is resolved from $PUSHED_REF, not HEAD, for the same reason the diff
# is: a tag reachable only from HEAD is not part of the history being pushed.
LAST_TAG="$(git describe --tags --abbrev=0 --match 'v[0-9]*.[0-9]*.[0-9]*' "$PUSHED_REF" 2>/dev/null || true)"
# --match is a shell glob, not a regex: its trailing `*`s match any suffix, so
# without --exclude a pre-release/checkpoint tag like v1.2.3-checkpoint or
# v1.2.3-rc1 also satisfies 'v[0-9]*.[0-9]*.[0-9]*' and could be picked over the
# true last release tag. --exclude is glob syntax too, so '*-*' is what actually
# rules out any tag carrying a hyphenated suffix, leaving only bare vMAJOR.MINOR.PATCH.
LAST_TAG="$(git describe --tags --abbrev=0 --match 'v[0-9]*.[0-9]*.[0-9]*' --exclude '*-*' "$PUSHED_REF" 2>/dev/null || true)"
if [[ -z "$LAST_TAG" ]]; then
echo "FAIL: no release tag exists yet, but .pre-commit-hooks.yaml already exposes hooks to external consumers." >&2

View File

@@ -419,6 +419,24 @@ else
fail "the repo's own .pre-commit-hooks.yaml no longer satisfies the entry constraints: $OUT20"
fi
# --- 21. A vX.Y.Z-suffixed checkpoint tag must not satisfy the release gate ---
# git describe --match uses shell-glob semantics, not regex: the trailing `*` in
# 'v[0-9]*.[0-9]*.[0-9]*' matches any suffix, so a pre-release/checkpoint tag like
# v1.0.1-checkpoint also satisfies the glob and can be picked as LAST_TAG instead
# of the true last release tag — hiding a real release-relevant change that landed
# before the checkpoint tag from the diff.
echo ""
echo "--- ignores a vX.Y.Z-checkpoint tag and still flags the change since the real release tag ---"
FIXTURE21="$(make_tagged_fixture)"; track "$FIXTURE21"
echo "v2" > "$FIXTURE21/scripts/skill-size-check.sh"
(cd "$FIXTURE21" && git add -A && git commit -q -m "real release-relevant change" && git tag v1.0.1-checkpoint)
OUT21=$(run_check "$FIXTURE21" "refs/heads/main" || true)
if echo "$OUT21" | grep -q "skill-size-check.sh"; then
pass "still flags the release-relevant change since v1.0.0, ignoring the vX.Y.Z-checkpoint tag"
else
fail "a vX.Y.Z-checkpoint tag satisfied the glob and hid a real release-relevant change"
fi
echo ""
echo "Results: $PASS passed, $FAIL failed"
[[ $FAIL -eq 0 ]]