fix(gates): close six PR #135 review findings in gates and their tests
B1: check-skill-version-bump.sh resolves every merge-base with `git merge-base
--all` instead of the single base git happens to pick. A criss-cross history has
two, so the verdict turned on that choice: a skill byte-identical to main's tip
could still be reported "not above merge-base" / "not above main tip" and fail a
push that should pass. A skill now counts as changed only when it differs from
EVERY base, and its version must exceed the version at every base it exists at
as well as at the main tip; with more than one base the failure names which one.
Case 40 in tests/test-skill-version-bump.sh builds the criss-cross fixture and
pins both directions.
B2: check-apm-current.sh no longer assumes the remote default branch is `main`
when origin/HEAD is unset. A checkout whose default is `master` was standing on
its default branch and being told "this is a feature branch, so discard it" --
to throw away a real lock update. With origin/HEAD unset nothing is asserted and
the neutral advice stands. tests/test-apm-current-hook.sh covers the unset case
on both `main` and `master`.
#4: the required-frontmatter checks folded into skill-size-check.sh by c8a7c9e
were untested apart from the leading-zero shape -- mutating the missing-version
ERROR into a no-op left every suite green. tests/test-adr0020-frontmatter.sh now
pins name presence and non-emptiness, metadata.version presence and semver
shape, and the four grep defects the deleted test-skill-frontmatter.sh named.
#5: nothing asked whether a Vale rule still MATCHES anything -- rewriting
CompositionNote.yml's tokens to match nothing left test-vale-wrap.sh at 63/63.
Case 35 enumerates the rule files under the Kyberforge* style directories at run
time, requires an alert from each on its own fixture, and fails when a
discovered rule has no fixture row. The stale comment at case 31 is corrected.
#6: tests/run-tests.sh --strict exited 0 when discovery found no test-*.sh at
all; strictness only ever acted on skips, and with no suites there were none. It
now cross-checks the git index the way run-bats.sh does and fails
unconditionally on an empty set, naming the search root.
N9: the skill-size-check hook description in .pre-commit-config.yaml covered
only the size, context-budget and boundary-target gates. It now also names the
required frontmatter fields, matching docs/spec/gates.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NwD8Egs5r4ndqeFLmhusX2
This commit is contained in:
@@ -215,7 +215,7 @@ repos:
|
||||
- id: skill-size-check
|
||||
stages: ['pre-commit']
|
||||
name: SKILL.md size and context-budget ceilings
|
||||
description: Enforce agentskills.io's 500-line/2,770-whole-file-word spec ceilings AND ADR-0020's context budget -- description 250 chars SUGGESTION / 400 FAIL, body-only 600 words SUGGESTION / 900 FAIL, and every boundary-clause routing target resolving to a real skill or agent under plugins/*/.apm/
|
||||
description: Enforce agentskills.io's 500-line/2,770-whole-file-word spec ceilings AND ADR-0020's context budget -- description 250 chars SUGGESTION / 400 FAIL, body-only 600 words SUGGESTION / 900 FAIL, and every boundary-clause routing target resolving to a real skill or agent under plugins/*/.apm/ -- plus the required frontmatter fields folded in from the former skill-frontmatter hook, namely name, a non-empty description, and a metadata.version matching three-part semver (1.0.0)
|
||||
entry: scripts/skill-size-check.sh
|
||||
language: script
|
||||
files: '^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$'
|
||||
|
||||
@@ -56,12 +56,21 @@ emit() {
|
||||
# is churn unrelated to the branch and should be discarded. The branch name only
|
||||
# selects between fixed strings and is never interpolated. Outside a git checkout,
|
||||
# or on a detached HEAD, the neutral advice stands.
|
||||
#
|
||||
# So does an UNSET origin/HEAD, which is the common state: git only writes it on
|
||||
# clone, and `git remote add` never does. The fallback here used to be `main`,
|
||||
# which is a guess, and it is wrong in exactly the repos that would notice — a
|
||||
# checkout whose default branch is `master` was told "this is a feature branch,
|
||||
# so discard it" while standing on its default branch, i.e. told to throw away a
|
||||
# real lock update. There is no cheap way to learn the remote's default without
|
||||
# the network, so nothing is asserted: the advice stays neutral and the reader
|
||||
# decides.
|
||||
lock_advice="commit it or discard it deliberately."
|
||||
current_branch="$(git symbolic-ref --short -q HEAD 2> /dev/null || true)"
|
||||
if [[ -n "$current_branch" ]]; then
|
||||
default_branch="$(git symbolic-ref --short -q refs/remotes/origin/HEAD 2> /dev/null || true)"
|
||||
default_branch="${default_branch#origin/}"
|
||||
if [[ "$current_branch" == "${default_branch:-main}" ]]; then
|
||||
default_branch="$(git symbolic-ref --short -q refs/remotes/origin/HEAD 2> /dev/null || true)"
|
||||
default_branch="${default_branch#origin/}"
|
||||
if [[ -n "$current_branch" && -n "$default_branch" ]]; then
|
||||
if [[ "$current_branch" == "$default_branch" ]]; then
|
||||
lock_advice="this is the default branch, so commit it or discard it deliberately."
|
||||
else
|
||||
lock_advice="this is a feature branch, so discard it: git checkout -- apm.lock.yaml && apm install"
|
||||
|
||||
@@ -6,16 +6,29 @@ set -euo pipefail
|
||||
# is the gate that holds the rule, since skill-size-check only checks presence
|
||||
# and shape.
|
||||
#
|
||||
# Baseline: `git merge-base <main> <pushed commit>`, where <main> is origin/main
|
||||
# when it resolves and the local `main` branch otherwise. Readers install
|
||||
# skills from main, so "changed" means changed relative to what main ships, not
|
||||
# relative to the remote branch's current tip. Diffing from PRE_COMMIT_FROM_REF
|
||||
# would let the second push of a feature branch excuse a change the first push
|
||||
# already carried unbumped. The check runs on every push whatever the target
|
||||
# branch — nothing here reads PRE_COMMIT_REMOTE_BRANCH — so it also runs under
|
||||
# a manual `pre-commit run --hook-stage pre-push` (against HEAD, since no
|
||||
# PRE_COMMIT_TO_REF is set). A missing bump is cheapest to fix on the branch,
|
||||
# before review.
|
||||
# Baseline: `git merge-base --all <main> <pushed commit>`, where <main> is
|
||||
# origin/main when it resolves and the local `main` branch otherwise. Readers
|
||||
# install skills from main, so "changed" means changed relative to what main
|
||||
# ships, not relative to the remote branch's current tip. Diffing from
|
||||
# PRE_COMMIT_FROM_REF would let the second push of a feature branch excuse a
|
||||
# change the first push already carried unbumped. The check runs on every push
|
||||
# whatever the target branch — nothing here reads PRE_COMMIT_REMOTE_BRANCH — so
|
||||
# it also runs under a manual `pre-commit run --hook-stage pre-push` (against
|
||||
# HEAD, since no PRE_COMMIT_TO_REF is set). A missing bump is cheapest to fix on
|
||||
# the branch, before review.
|
||||
#
|
||||
# `--all`, not the single base git would otherwise pick for it. A criss-cross
|
||||
# history — main merges a branch while that branch merges a main commit — has
|
||||
# TWO merge bases, and which one `git merge-base` prints is an implementation
|
||||
# detail. Picking one made the verdict a coin flip: a skill byte-identical to
|
||||
# main's tip was still reported "not above merge-base" whenever the losing base
|
||||
# happened to be chosen, so an already-merged bump failed the push it should
|
||||
# have passed. So a skill counts as CHANGED only when it differs from EVERY
|
||||
# base — differing from none of them, or from only some, means one base already
|
||||
# carries the pushed content — and a changed skill's version must exceed the
|
||||
# version at every base it exists at. Both directions are conservative: the
|
||||
# intersection cannot exempt a skill that genuinely changed since all of main's
|
||||
# reachable history, and requiring every base keeps the ratchet.
|
||||
#
|
||||
# Second baseline: the tip of that same <main> ref. A changed skill's pushed
|
||||
# version must exceed its version there too (ADR-0022, second 2026-09-16
|
||||
@@ -134,38 +147,70 @@ if [[ -z "$MAIN_REF" ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! BASELINE="$(git merge-base "$MAIN_REF" "$PUSHED_COMMIT" 2>/dev/null)"; then
|
||||
BASES=()
|
||||
while IFS= read -r base; do
|
||||
[[ -n "$base" ]] && BASES+=("$base")
|
||||
done < <(git merge-base --all "$MAIN_REF" "$PUSHED_COMMIT" 2>/dev/null || true)
|
||||
|
||||
if [[ ${#BASES[@]} -eq 0 ]]; then
|
||||
echo "FAIL: no merge-base between $MAIN_REF and $PUSHED_REF, so there is no baseline to compare skill versions against." >&2
|
||||
echo " Fix: ensure full history is available (e.g. git fetch --unshallow) and retry." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ "$MAIN_REF" == "main" && "$BASELINE" == "$PUSHED_COMMIT" ]]; then
|
||||
echo "FAIL: origin/main does not resolve and $PUSHED_REF is already contained in local main, so local main cannot serve as an independent baseline — the diff would be empty by construction." >&2
|
||||
echo " Fix: git fetch origin main and retry." >&2
|
||||
exit 1
|
||||
# A pushed commit that is an ancestor of <main> is itself the only merge-base,
|
||||
# so this fires on exactly the case it always did.
|
||||
if [[ "$MAIN_REF" == "main" ]]; then
|
||||
for base in ${BASES[@]+"${BASES[@]}"}; do
|
||||
if [[ "$base" == "$PUSHED_COMMIT" ]]; then
|
||||
echo "FAIL: origin/main does not resolve and $PUSHED_REF is already contained in local main, so local main cannot serve as an independent baseline — the diff would be empty by construction." >&2
|
||||
echo " Fix: git fetch origin main and retry." >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
CHANGED_FILE="$(mktemp)"
|
||||
trap 'rm -f "$CHANGED_FILE"' EXIT
|
||||
|
||||
if ! git diff -z --no-renames --name-only "$BASELINE" "$PUSHED_COMMIT" -- plugins > "$CHANGED_FILE"; then
|
||||
echo "FAIL: could not diff $BASELINE..$PUSHED_REF (see git error above)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
SKILL_PATH_RE='^(plugins/[^/]+/\.apm/skills/[^/]+)/(.+)$'
|
||||
SKILL_DIRS=()
|
||||
while IFS= read -r -d '' path; do
|
||||
[[ "$path" =~ $SKILL_PATH_RE ]] || continue
|
||||
[[ "${BASH_REMATCH[2]}" == tests/* ]] && continue
|
||||
dir="${BASH_REMATCH[1]}"
|
||||
seen=false
|
||||
for existing in ${SKILL_DIRS[@]+"${SKILL_DIRS[@]}"}; do
|
||||
[[ "$existing" == "$dir" ]] && { seen=true; break; }
|
||||
|
||||
# skill_dirs_at <base>: sets SKILL_DIRS_ONE to the skill directories differing
|
||||
# between <base> and the pushed commit.
|
||||
skill_dirs_at() {
|
||||
local path dir existing seen
|
||||
SKILL_DIRS_ONE=()
|
||||
if ! git diff -z --no-renames --name-only "$1" "$PUSHED_COMMIT" -- plugins > "$CHANGED_FILE"; then
|
||||
echo "FAIL: could not diff $1..$PUSHED_REF (see git error above)." >&2
|
||||
exit 1
|
||||
fi
|
||||
while IFS= read -r -d '' path; do
|
||||
[[ "$path" =~ $SKILL_PATH_RE ]] || continue
|
||||
[[ "${BASH_REMATCH[2]}" == tests/* ]] && continue
|
||||
dir="${BASH_REMATCH[1]}"
|
||||
seen=false
|
||||
for existing in ${SKILL_DIRS_ONE[@]+"${SKILL_DIRS_ONE[@]}"}; do
|
||||
[[ "$existing" == "$dir" ]] && { seen=true; break; }
|
||||
done
|
||||
$seen || SKILL_DIRS_ONE+=("$dir")
|
||||
done < "$CHANGED_FILE"
|
||||
}
|
||||
|
||||
# Intersected across every base: a skill matching any ONE base is already
|
||||
# shipped by that base and has nothing left to bump.
|
||||
skill_dirs_at "${BASES[0]}"
|
||||
SKILL_DIRS=(${SKILL_DIRS_ONE[@]+"${SKILL_DIRS_ONE[@]}"})
|
||||
for ((i = 1; i < ${#BASES[@]}; i++)); do
|
||||
[[ ${#SKILL_DIRS[@]} -eq 0 ]] && break
|
||||
skill_dirs_at "${BASES[i]}"
|
||||
KEPT=()
|
||||
for dir in ${SKILL_DIRS[@]+"${SKILL_DIRS[@]}"}; do
|
||||
for existing in ${SKILL_DIRS_ONE[@]+"${SKILL_DIRS_ONE[@]}"}; do
|
||||
[[ "$existing" == "$dir" ]] && { KEPT+=("$dir"); break; }
|
||||
done
|
||||
done
|
||||
$seen || SKILL_DIRS+=("$dir")
|
||||
done < "$CHANGED_FILE"
|
||||
SKILL_DIRS=(${KEPT[@]+"${KEPT[@]}"})
|
||||
done
|
||||
|
||||
[[ ${#SKILL_DIRS[@]} -eq 0 ]] && exit 0
|
||||
|
||||
@@ -238,35 +283,64 @@ in_tree() {
|
||||
|
||||
OFFENDERS=()
|
||||
for dir in ${SKILL_DIRS[@]+"${SKILL_DIRS[@]}"}; do
|
||||
at_base=false
|
||||
# Bases the skill exists at, in merge-base order, with the version read at
|
||||
# each. Index-matched arrays rather than one map: bash 3.2 has no `declare -A`.
|
||||
base_at=()
|
||||
base_vers=()
|
||||
first_base_ver=""
|
||||
for base in ${BASES[@]+"${BASES[@]}"}; do
|
||||
in_tree "$base" "$dir/SKILL.md" || continue
|
||||
version_at "$base" "$dir/SKILL.md"
|
||||
base_at+=("$base")
|
||||
base_vers+=("$VERSION")
|
||||
[[ -n "$first_base_ver" ]] || first_base_ver="$VERSION"
|
||||
done
|
||||
|
||||
# When main has not moved since a merge-base, the tip IS that baseline and the
|
||||
# skill is checked once.
|
||||
at_tip=false
|
||||
in_tree "$BASELINE" "$dir/SKILL.md" && at_base=true
|
||||
# When main has not moved since the merge-base, the tip is the same baseline.
|
||||
[[ "$MAIN_TIP" != "$BASELINE" ]] && in_tree "$MAIN_TIP" "$dir/SKILL.md" && at_tip=true
|
||||
# Absent at both baselines: new, renamed-to, or merged-into. Exempt.
|
||||
$at_base || $at_tip || continue
|
||||
tip_is_base=false
|
||||
for base in ${BASES[@]+"${BASES[@]}"}; do
|
||||
if [[ "$MAIN_TIP" == "$base" ]]; then
|
||||
tip_is_base=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [[ "$tip_is_base" == false ]] && in_tree "$MAIN_TIP" "$dir/SKILL.md"; then
|
||||
at_tip=true
|
||||
fi
|
||||
# Absent at every baseline: new, renamed-to, or merged-into. Exempt.
|
||||
[[ ${#base_at[@]} -gt 0 ]] || $at_tip || continue
|
||||
# Directory absent at pushed commit: deleted or renamed-from. Exempt.
|
||||
[[ "$(git cat-file -t "$PUSHED_COMMIT:$dir" 2>/dev/null)" == "tree" ]] || continue
|
||||
|
||||
base_ver=""
|
||||
tip_ver=""
|
||||
if $at_base; then version_at "$BASELINE" "$dir/SKILL.md"; base_ver="$VERSION"; fi
|
||||
if $at_tip; then version_at "$MAIN_TIP" "$dir/SKILL.md"; tip_ver="$VERSION"; fi
|
||||
|
||||
if ! in_tree "$PUSHED_COMMIT" "$dir/SKILL.md"; then
|
||||
OFFENDERS+=("$dir: SKILL.md missing at $PUSHED_REF (baseline: ${base_ver:-none})")
|
||||
OFFENDERS+=("$dir: SKILL.md missing at $PUSHED_REF (baseline: ${first_base_ver:-none})")
|
||||
continue
|
||||
fi
|
||||
version_at "$PUSHED_COMMIT" "$dir/SKILL.md"
|
||||
cur_ver="$VERSION"
|
||||
|
||||
if [[ -z "$cur_ver" ]]; then
|
||||
OFFENDERS+=("$dir: metadata.version missing or not MAJOR.MINOR.PATCH at $PUSHED_REF (baseline: ${base_ver:-none})")
|
||||
OFFENDERS+=("$dir: metadata.version missing or not MAJOR.MINOR.PATCH at $PUSHED_REF (baseline: ${first_base_ver:-none})")
|
||||
continue
|
||||
fi
|
||||
if [[ -n "$base_ver" ]] && ! semver_gt "$cur_ver" "$base_ver"; then
|
||||
OFFENDERS+=("$dir: $base_ver -> $cur_ver (not above merge-base)")
|
||||
fi
|
||||
# Named by sha only when there is more than one base to tell apart; a
|
||||
# criss-cross history is the only case where "which merge-base" is a question
|
||||
# the reader cannot answer from the branch alone.
|
||||
for ((i = 0; i < ${#base_at[@]}; i++)); do
|
||||
base_ver="${base_vers[i]}"
|
||||
[[ -n "$base_ver" ]] || continue
|
||||
semver_gt "$cur_ver" "$base_ver" && continue
|
||||
if [[ ${#BASES[@]} -gt 1 ]]; then
|
||||
OFFENDERS+=("$dir: $base_ver -> $cur_ver (not above merge-base ${base_at[i]})")
|
||||
else
|
||||
OFFENDERS+=("$dir: $base_ver -> $cur_ver (not above merge-base)")
|
||||
fi
|
||||
done
|
||||
if [[ -n "$tip_ver" ]] && ! semver_gt "$cur_ver" "$tip_ver"; then
|
||||
OFFENDERS+=("$dir: $tip_ver -> $cur_ver (not above $MAIN_REF tip)")
|
||||
fi
|
||||
|
||||
@@ -169,6 +169,67 @@ done < <(
|
||||
| sort
|
||||
)
|
||||
|
||||
# Discovering NOTHING is never a clean run, and it used to be the quietest
|
||||
# possible pass: the loops below iterate zero times, nothing is printed between
|
||||
# the bats block and the summary, and `Summary: 0 passed, 0 failed` exits 0 --
|
||||
# under --strict too, because strictness only ever turned SKIPS into failures
|
||||
# and there were no suites to skip. A wrong TEST_DIR, a mistyped `find` pattern,
|
||||
# an exclusion that grew to swallow tests/, and a gutted checkout all land here.
|
||||
# tests/run-bats.sh has carried this guard for its own .bats discovery; this is
|
||||
# the same guard one file over, and the run-tests pre-push hook is the caller
|
||||
# that needs it.
|
||||
#
|
||||
# Two checks, in the same order and for the same reasons as run-bats.sh's.
|
||||
# First, the derived one: every test-*.sh in the git index must have been
|
||||
# discovered. The direction matters -- a discovered file need NOT be tracked
|
||||
# (work in progress is ordinary), and a file removed with `git rm` leaves the
|
||||
# index, so a deliberate removal passes while an accidental disappearance
|
||||
# fails. It only runs when SEARCH_ROOT is itself the git worktree root, which
|
||||
# is what keeps it off the mktemp fixture trees in tests/test-run-tests.sh --
|
||||
# those hold one or two test-*.sh files by design and git resolves no worktree
|
||||
# for them. The same exclusions are reapplied to the index listing so both
|
||||
# sides cover the same universe.
|
||||
EXPECTED_SCRIPTS=()
|
||||
GIT_TOPLEVEL="$(git -C "$SEARCH_ROOT" rev-parse --show-toplevel 2> /dev/null || true)"
|
||||
if [[ -n "$GIT_TOPLEVEL" && "$GIT_TOPLEVEL" == "$SEARCH_ROOT" ]]; then
|
||||
while IFS= read -r f; do
|
||||
[[ -n "$f" ]] && EXPECTED_SCRIPTS+=("$SEARCH_ROOT/$f")
|
||||
done < <(
|
||||
git -C "$SEARCH_ROOT" ls-files -- 'test-*.sh' '*/test-*.sh' \
|
||||
| grep -Ev '(^|/)\.claude/worktrees/|(^|/)apm_modules/|(^|/)\.claude/skills/' \
|
||||
| sort || true
|
||||
)
|
||||
fi
|
||||
|
||||
if [[ ${#EXPECTED_SCRIPTS[@]} -gt 0 ]]; then
|
||||
MISSING_SCRIPTS=()
|
||||
for expected in ${EXPECTED_SCRIPTS[@]+"${EXPECTED_SCRIPTS[@]}"}; do
|
||||
found=false
|
||||
for actual in ${SCRIPTS[@]+"${SCRIPTS[@]}"}; do
|
||||
if [[ "$actual" == "$expected" ]]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
[[ "$found" == true ]] || MISSING_SCRIPTS+=("${expected#"$SEARCH_ROOT"/}")
|
||||
done
|
||||
if [[ ${#MISSING_SCRIPTS[@]} -gt 0 ]]; then
|
||||
echo "Error: ${#MISSING_SCRIPTS[@]} of ${#EXPECTED_SCRIPTS[@]} tracked test-*.sh file(s) were not discovered under $SEARCH_ROOT — they were deleted without being removed from the index, or the search path/exclusions above no longer reach them:" >&2
|
||||
for m in ${MISSING_SCRIPTS[@]+"${MISSING_SCRIPTS[@]}"}; do
|
||||
echo " $m" >&2
|
||||
done
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Second, unconditional and separate: a tree with nothing tracked (a tarball
|
||||
# export, a fresh scaffold) still must not run on an empty set and call it
|
||||
# green.
|
||||
if [[ ${#SCRIPTS[@]} -eq 0 ]]; then
|
||||
echo "Error: found 0 test-*.sh file(s) under $SEARCH_ROOT — the search path is wrong or the suite has been gutted" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Each test-*.sh is independent (fixtures live under its own mktemp dir, none
|
||||
# write back into the live repo tree -- verified before adding this), so they
|
||||
# run concurrently in fixed-size batches instead of one at a time. Dispatch and
|
||||
|
||||
@@ -441,6 +441,231 @@ else
|
||||
fail "validate.sh agent mode exited $SILENT_RC with output '${SILENT_OUT:-<empty>}' — the original defect was exit 0 and total silence on a blocking pre-push gate"
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. The REQUIRED-FIELD checks, folded in from the deleted `skill-frontmatter`
|
||||
# ---------------------------------------------------------------------------
|
||||
# Commit c8a7c9e retired the standalone `skill-frontmatter` hook and moved its
|
||||
# two presence checks — `name` non-empty, `metadata.version` present and
|
||||
# three-part semver — into skill-size-check.sh, beside the ADR-0020 gates. The
|
||||
# hook's own suite went with it, and only the leading-zero shape was left
|
||||
# covered (tests/test-skill-size-check.sh). Measured: mutating the
|
||||
# missing-version ERROR to a no-op left every suite in the repo green. These
|
||||
# cases are that behaviour pinned back down, on the same fixtures the deleted
|
||||
# suite used.
|
||||
#
|
||||
# Probed against the HOOK alone, deliberately. validate.sh's skill mode has its
|
||||
# own metadata.version check with its own wording, and its agent mode has none
|
||||
# at all — ADR-0022 binds skills, not agents — so probe_all's "all three must
|
||||
# agree" contract does not hold for this family and asserting it would be
|
||||
# asserting something the ADRs contradict.
|
||||
#
|
||||
# The four grep defects the deleted suite named are kept as cases because the
|
||||
# YAML-parsed implementation must not regress into any of them: a `metadata:`
|
||||
# block quoted in the BODY, a `version:` under a following `source:` list, a
|
||||
# deeper-indented `version:`, and the mirror image — a `version:` far down a
|
||||
# long metadata block, which the old `-A10` grep reported MISSING.
|
||||
|
||||
# write_required <name> <frontmatter> [body] — a SKILL.md whose only interesting
|
||||
# property is its frontmatter. The description and body sit well inside every
|
||||
# ADR-0020 ceiling, so a finding here is the required-field check and nothing
|
||||
# else; a fixture that also tripped a ceiling would satisfy "exits non-zero" for
|
||||
# the wrong reason.
|
||||
write_required() {
|
||||
local name="$1" frontmatter="$2" body="${3:-Body text.}"
|
||||
local dir="$TMPDIR_T/required/$name"
|
||||
mkdir -p "$dir"
|
||||
{
|
||||
printf -- '---\n'
|
||||
printf '%s\n' "$frontmatter"
|
||||
printf -- '---\n\n'
|
||||
printf '%s\n' "$body"
|
||||
} > "$dir/SKILL.md"
|
||||
printf '%s' "$dir/SKILL.md"
|
||||
}
|
||||
|
||||
# write_required_raw <name> <whole-file> — for the shapes that must NOT have a
|
||||
# closing marker written for them.
|
||||
write_required_raw() {
|
||||
local name="$1"
|
||||
local dir="$TMPDIR_T/required/$name"
|
||||
mkdir -p "$dir"
|
||||
printf '%s' "$2" > "$dir/SKILL.md"
|
||||
printf '%s' "$dir/SKILL.md"
|
||||
}
|
||||
|
||||
REQ_DESC='Use when probing the required-field checks. Do not use for anything else.'
|
||||
|
||||
# require_finding <label> <file> <needle> — non-zero exit AND the named message.
|
||||
# The needle is the message, not the exit code: the mutation this case exists to
|
||||
# catch turns the ERROR into a no-op, and a file that also failed some other gate
|
||||
# would still exit non-zero with the check gone.
|
||||
require_finding() {
|
||||
local label="$1" file="$2" needle="$3" out status=0
|
||||
set +e
|
||||
out="$(bash "$HOOK" "$file" 2>&1)"
|
||||
status=$?
|
||||
set -e
|
||||
if [[ $status -eq 0 ]]; then
|
||||
fail "$label — the hook exited 0: ${out:-<no output>}"
|
||||
elif [[ "$out" != *"$needle"* ]]; then
|
||||
fail "$label — the hook failed but never said '$needle': $out"
|
||||
else
|
||||
pass "$label"
|
||||
fi
|
||||
}
|
||||
|
||||
# require_clean <label> <file> — exits 0 with no finding at all.
|
||||
require_clean() {
|
||||
local label="$1" file="$2" out status=0
|
||||
set +e
|
||||
out="$(bash "$HOOK" "$file" 2>&1)"
|
||||
status=$?
|
||||
set -e
|
||||
if [[ $status -eq 0 ]]; then
|
||||
pass "$label"
|
||||
else
|
||||
fail "$label — expected exit 0, got $status: ${out:-<no output>}"
|
||||
fi
|
||||
}
|
||||
|
||||
echo ""
|
||||
echo "--- control: a SKILL.md carrying both required fields passes ---"
|
||||
# Without this, every case below could be passing because the fixture generator
|
||||
# is broken rather than because the checks fire.
|
||||
require_clean "a well-formed name + metadata.version passes" \
|
||||
"$(write_required valid "name: valid
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version: \"1.0.0\"")"
|
||||
|
||||
echo ""
|
||||
echo "--- metadata.version missing, in each of the shapes that used to satisfy the old grep ---"
|
||||
require_finding "no metadata block at all is reported missing" \
|
||||
"$(write_required no-metadata "name: no-metadata
|
||||
description: $REQ_DESC")" \
|
||||
"metadata.version field is missing"
|
||||
require_finding "a metadata block with other keys but no version is reported missing" \
|
||||
"$(write_required metadata-no-version "name: metadata-no-version
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
author: someone")" \
|
||||
"metadata.version field is missing"
|
||||
# `version:` with no value parses to None, which is absent, not malformed —
|
||||
# reporting it as a bad VALUE would send the author looking for a typo in a
|
||||
# value that is not there.
|
||||
require_finding "a valueless 'version:' is reported missing, not malformed" \
|
||||
"$(write_required empty-version "name: empty-version
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version:")" \
|
||||
"metadata.version field is missing"
|
||||
# skill-author's own docs quote a metadata block verbatim; under the old
|
||||
# whole-file grep that quotation satisfied the check for the file quoting it.
|
||||
require_finding "a metadata block quoted in the BODY does not satisfy the check" \
|
||||
"$(write_required fenced-metadata "name: fenced-metadata
|
||||
description: $REQ_DESC" '# Fenced
|
||||
|
||||
Skills declare their version like this:
|
||||
|
||||
```yaml
|
||||
metadata:
|
||||
version: "1.0.0"
|
||||
```')" \
|
||||
"metadata.version field is missing"
|
||||
# `-A10` ran ten lines past `metadata:` regardless of where the block ended, and
|
||||
# write-docs and research both carry a `source:` list immediately after it.
|
||||
require_finding "a version: belonging to a following source[] does not satisfy the check" \
|
||||
"$(write_required source-list "name: source-list
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
author: someone
|
||||
source:
|
||||
- name: upstream
|
||||
version: \"2.3.4\"")" \
|
||||
"metadata.version field is missing"
|
||||
# `grep -q \" version:\"` was an unanchored substring match, so any indentation
|
||||
# of two spaces or more matched.
|
||||
require_finding "a four-space-indented version: one level deeper does not satisfy the check" \
|
||||
"$(write_required deep-indent "name: deep-indent
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
provenance:
|
||||
version: \"1.0.0\"")" \
|
||||
"metadata.version field is missing"
|
||||
# The mirror image, and the reason this one asserts a PASS: the old grep's
|
||||
# ten-line window reported a real version missing once the block grew past it.
|
||||
require_clean "a version: thirteen lines into the metadata block is found" \
|
||||
"$(write_required long-metadata "name: long-metadata
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
a: 1
|
||||
b: 2
|
||||
c: 3
|
||||
d: 4
|
||||
e: 5
|
||||
f: 6
|
||||
g: 7
|
||||
h: 8
|
||||
i: 9
|
||||
j: 10
|
||||
k: 11
|
||||
version: \"1.0.0\"")"
|
||||
|
||||
echo ""
|
||||
echo "--- present is not well formed: a non-semver metadata.version is its own finding ---"
|
||||
# plugins/bin/.apm/skills/write-docs/SKILL.md carried `version: "1.0"` through a
|
||||
# whole PR under a presence-only check: present, well-nested, and not a version.
|
||||
# The value is quoted back so the author does not have to guess which key.
|
||||
require_finding "'1.0' is rejected as malformed and the message quotes it" \
|
||||
"$(write_required two-part "name: two-part
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version: \"1.0\"")" \
|
||||
"metadata.version is malformed ('1.0')"
|
||||
require_finding "'latest' is rejected as malformed and the message quotes it" \
|
||||
"$(write_required word-version "name: word-version
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version: latest")" \
|
||||
"metadata.version is malformed ('latest')"
|
||||
|
||||
echo ""
|
||||
echo "--- the name field is required and must not be empty ---"
|
||||
require_finding "an absent name is reported" \
|
||||
"$(write_required no-name "description: $REQ_DESC
|
||||
metadata:
|
||||
version: \"1.0.0\"")" \
|
||||
"name field is missing or empty"
|
||||
require_finding "an empty name is reported" \
|
||||
"$(write_required empty-name "name: \"\"
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version: \"1.0.0\"")" \
|
||||
"name field is missing or empty"
|
||||
|
||||
echo ""
|
||||
echo "--- a file whose frontmatter block cannot be read reports THAT, not a missing field ---"
|
||||
# The required-field checks run downstream of the frontmatter match, so a file
|
||||
# with no readable block must land on the parse error rather than being reported
|
||||
# as a skill that merely forgot its version — and must never report green.
|
||||
require_finding "an unterminated frontmatter block is a parse error" \
|
||||
"$(write_required_raw unterminated "---
|
||||
name: unterminated
|
||||
description: $REQ_DESC
|
||||
metadata:
|
||||
version: \"1.0.0\"
|
||||
|
||||
Body text.
|
||||
")" \
|
||||
"no parseable YAML frontmatter block"
|
||||
require_finding "an empty '---/---' block is a parse error" \
|
||||
"$(write_required_raw empty-block "---
|
||||
---
|
||||
|
||||
Body text.
|
||||
")" \
|
||||
"no parseable YAML frontmatter block"
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
@@ -150,6 +150,23 @@ if command -v git > /dev/null 2>&1; then
|
||||
commit -q --allow-empty -m init
|
||||
touch "$REPO/apm.lock.yaml"
|
||||
|
||||
# origin/HEAD is unset here — git writes it on clone and `git remote add` does
|
||||
# not — so the hook cannot know what the default branch is and must not guess.
|
||||
# It used to assume `main`, which is why the `master` case further down was a
|
||||
# live defect.
|
||||
out="$(run_hook_in "$REPO" "$REPO")"
|
||||
advice="$(advice_of "$out")"
|
||||
grep -q "commit it or discard it deliberately" <<< "$advice" \
|
||||
&& pass "with origin/HEAD unset, keeps the neutral lock advice" \
|
||||
|| fail "with origin/HEAD unset the advice should stay neutral: $advice"
|
||||
# Needles are the two DECISION phrases, not the bare words: the fixed prefix
|
||||
# of every notice already says "behind the remote default branch".
|
||||
grep -qE "this is (the default|a feature) branch, so" <<< "$advice" \
|
||||
&& fail "with origin/HEAD unset the hook must not claim to know which branch this is: $advice" \
|
||||
|| pass "with origin/HEAD unset, claims nothing about which branch this is"
|
||||
|
||||
git -C "$REPO" update-ref refs/remotes/origin/main HEAD
|
||||
git -C "$REPO" symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/main
|
||||
out="$(run_hook_in "$REPO" "$REPO")"
|
||||
grep -q "default branch, so commit it or discard it deliberately" <<< "$(advice_of "$out")" \
|
||||
&& pass "on main, says to commit or discard the lock deliberately" \
|
||||
@@ -174,6 +191,26 @@ if command -v git > /dev/null 2>&1; then
|
||||
grep -q "default branch, so commit it" <<< "$(advice_of "$out")" \
|
||||
&& pass "reads the default branch from origin/HEAD when it is set" \
|
||||
|| fail "should treat origin/HEAD's branch as the default: $(advice_of "$out")"
|
||||
|
||||
# The regression the `${default_branch:-main}` fallback caused: a repo whose
|
||||
# default branch is `master`, with origin/HEAD unset (no clone wrote it), was
|
||||
# standing on its DEFAULT branch and was told to discard the lock as feature
|
||||
# churn. Assuming `main` is the only way to reach that verdict, so the case is
|
||||
# pinned on the branch name that makes the assumption wrong.
|
||||
MASTER_REPO="$WORK/master-repo"
|
||||
mkdir -p "$MASTER_REPO"
|
||||
git -C "$MASTER_REPO" init -q -b master
|
||||
git -C "$MASTER_REPO" -c user.email=probe@example.invalid -c user.name=probe \
|
||||
commit -q --allow-empty -m init
|
||||
touch "$MASTER_REPO/apm.lock.yaml"
|
||||
out="$(run_hook_in "$MASTER_REPO" "$MASTER_REPO")"
|
||||
advice="$(advice_of "$out")"
|
||||
grep -qF "feature branch, so discard it" <<< "$advice" \
|
||||
&& fail "on master with origin/HEAD unset the hook assumed main and told the reader to discard a real lock update: $advice" \
|
||||
|| pass "on master with origin/HEAD unset, does not misread the default branch as a feature branch"
|
||||
grep -q "commit it or discard it deliberately" <<< "$advice" \
|
||||
&& pass "on master with origin/HEAD unset, falls back to the neutral lock advice" \
|
||||
|| fail "on master with origin/HEAD unset the advice should be neutral: $advice"
|
||||
else
|
||||
echo " (git not on PATH — branch-specific advice cases not run)"
|
||||
fi
|
||||
|
||||
@@ -703,6 +703,74 @@ else
|
||||
pass "a root under .claude/worktrees/ runs its own suites and skips nested worktrees"
|
||||
fi
|
||||
|
||||
# --- 13. Discovering ZERO test-*.sh files is a hard error, not a green run ---
|
||||
# The quietest vacuous pass in the dispatcher, and the one --strict did not
|
||||
# reach: strictness only ever turned SKIPS into failures, so with no suites to
|
||||
# skip there was nothing for it to act on. The loops iterated zero times, the
|
||||
# run printed `Summary: 0 passed, 0 skipped, 0 failed` and exited 0 -- from the
|
||||
# run-tests pre-push hook, which prints nothing for a passing hook, that is
|
||||
# indistinguishable from every suite passing. A wrong TEST_DIR, a mistyped find
|
||||
# pattern, an exclusion that grew to swallow tests/, and a gutted checkout all
|
||||
# land here. tests/run-bats.sh has guarded its own discovery this way since it
|
||||
# hit the same hole; this is the sibling.
|
||||
#
|
||||
# The bats runner is the healthy stub, so the only thing wrong with the fixture
|
||||
# is that it has no case scripts -- a failure here cannot be the bats leg.
|
||||
echo ""
|
||||
echo "--- discovering zero test-*.sh files fails the run and names the search root ---"
|
||||
DIR13="$(make_fake_repo)"
|
||||
FIXTURES+=("$DIR13")
|
||||
install_healthy_bats_runner "$DIR13"
|
||||
run_fake "$DIR13"
|
||||
if [[ $FAKE_RC -eq 0 ]]; then
|
||||
fail "an empty search root exited 0 — a run that verified nothing reports the same as a run that verified everything: $FAKE_OUT"
|
||||
elif ! grep -q "found 0 test-\*\.sh file(s)" <<< "$FAKE_OUT"; then
|
||||
fail "an empty search root failed without saying that nothing was discovered: $FAKE_OUT"
|
||||
elif ! grep -qF "$DIR13/cases" <<< "$FAKE_OUT"; then
|
||||
fail "the zero-discovery error did not name the root it searched, which is the one fact needed to fix it: $FAKE_OUT"
|
||||
elif grep -q "^=== Summary:" <<< "$FAKE_OUT"; then
|
||||
fail "an empty search root still printed a summary line, so a reader scanning for the verdict sees a green one: $FAKE_OUT"
|
||||
else
|
||||
pass "zero discovered test-*.sh files fails the run, names the search root, and prints no summary"
|
||||
fi
|
||||
|
||||
# And under --strict too. Asserted separately because the ONLY lever --strict
|
||||
# had was the skip list, so "it fails now" and "it fails under the gate's own
|
||||
# invocation" were genuinely different questions here.
|
||||
echo ""
|
||||
echo "--- ... and under --strict, which previously had no lever on this at all ---"
|
||||
DIR13B="$(make_fake_repo)"
|
||||
FIXTURES+=("$DIR13B")
|
||||
install_healthy_bats_runner "$DIR13B"
|
||||
run_fake "$DIR13B" --strict
|
||||
if [[ $FAKE_RC -eq 0 ]]; then
|
||||
fail "an empty search root exited 0 under --strict, the invocation the run-tests pre-push hook uses: $FAKE_OUT"
|
||||
elif grep -q "found 0 test-\*\.sh file(s)" <<< "$FAKE_OUT"; then
|
||||
pass "--strict also fails a run that discovered nothing"
|
||||
else
|
||||
fail "--strict failed an empty search root for some other reason: $FAKE_OUT"
|
||||
fi
|
||||
|
||||
# The control: one discovered case is enough to get past the guard, so the two
|
||||
# cases above fail on the count and not on something else the fixture lacks.
|
||||
echo ""
|
||||
echo "--- ... while a single discovered case still passes ---"
|
||||
DIR13C="$(make_fake_repo)"
|
||||
FIXTURES+=("$DIR13C")
|
||||
install_healthy_bats_runner "$DIR13C"
|
||||
add_case "$DIR13C" test-one.sh <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
echo "one case ran"
|
||||
EOF
|
||||
run_fake "$DIR13C"
|
||||
if [[ $FAKE_RC -ne 0 ]]; then
|
||||
fail "a fixture with exactly one case failed: $FAKE_OUT"
|
||||
elif grep -q "found 0 test-\*\.sh file(s)" <<< "$FAKE_OUT"; then
|
||||
fail "the zero-discovery guard fired on a root that has one case script: $FAKE_OUT"
|
||||
else
|
||||
pass "one discovered case script is enough to get past the zero-discovery guard"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
@@ -588,6 +588,51 @@ expect_fail "unbumped skill with leading blank lines is held to its baseline ver
|
||||
write_skill "$F" demo alpha 'version: "1.0.1"' "lead body 2"; blank_lead "$F" alpha; commit "$F"
|
||||
expect_pass "bumped skill with leading blank lines passes" "$F"
|
||||
|
||||
echo ""
|
||||
echo "--- 40. a criss-cross history is judged against EVERY merge-base ---"
|
||||
# Two merge bases, and which one plain `git merge-base` prints is git's choice,
|
||||
# not a property of the history. The gate used to take that single answer, so
|
||||
# the verdict turned on it: here `git diff main feature -- plugins` is EMPTY
|
||||
# (main already carries the bump, via its merge of the feature branch) and the
|
||||
# push still failed with "not above main tip", because the base git picked was
|
||||
# the one that predates the bump. `--all` plus the intersection rule makes the
|
||||
# answer the same whichever base git would have named.
|
||||
#
|
||||
# C0 alpha 1.0.0
|
||||
# +-- feature: F1 bumps alpha to 1.0.1
|
||||
# +-- main: M1 unrelated, then M2 merges F1
|
||||
# feature: F2 merges M1 -> merge bases {M1, F1}
|
||||
F="$(mktemp -d)"; CLEANUP_DIRS+=("$F")
|
||||
(cd "$F" && git init -q -b main && git config user.email t@t.t && git config user.name t)
|
||||
write_skill "$F" demo alpha 'version: "1.0.0"'; commit "$F" C0
|
||||
(cd "$F" && git checkout -q -b feature)
|
||||
write_skill "$F" demo alpha 'version: "1.0.1"' "new body"; commit "$F" F1
|
||||
F1_SHA="$(cd "$F" && git rev-parse HEAD)"
|
||||
(cd "$F" && git checkout -q main)
|
||||
echo unrelated > "$F/m1.txt"; commit "$F" M1
|
||||
M1_SHA="$(cd "$F" && git rev-parse HEAD)"
|
||||
(cd "$F" && git merge -q --no-edit "$F1_SHA" -m M2 > /dev/null)
|
||||
(cd "$F" && git checkout -q feature && git merge -q --no-edit "$M1_SHA" -m F2 > /dev/null)
|
||||
|
||||
BASES40="$(cd "$F" && git merge-base --all main feature | sort)"
|
||||
if [[ "$(printf '%s\n' "$BASES40" | wc -l)" -eq 2 ]]; then
|
||||
pass "fixture check: the history really does have two merge bases"
|
||||
else
|
||||
fail "fixture check: expected two merge bases, got: $BASES40"
|
||||
fi
|
||||
if [[ -z "$(cd "$F" && git diff main feature -- plugins)" ]]; then
|
||||
pass "fixture check: nothing under plugins/ differs between main and the branch"
|
||||
else
|
||||
fail "fixture check: plugins/ differs between main and the branch, so this is not the case under test"
|
||||
fi
|
||||
expect_pass "a skill identical to main's tip passes whichever merge-base git would pick" "$F"
|
||||
|
||||
# And the ratchet still holds on the same shape: a further edit with no bump
|
||||
# differs from BOTH bases, so it is not excused by the criss-cross.
|
||||
write_skill "$F" demo alpha 'version: "1.0.1"' "later body"; commit "$F" F3
|
||||
expect_fail "an unbumped edit on a criss-cross branch still fails, naming the baseline sha" \
|
||||
"alpha: 1\.0\.1 -> 1\.0\.1 \(not above merge-base [0-9a-f]{40}\)" "$F"
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]]
|
||||
|
||||
@@ -1783,7 +1783,14 @@ fi
|
||||
# merge took out with the script, and cases 28-30 cannot backstop it. They key
|
||||
# on `Kyberforge.VagueWording` and `KyberforgeCopilot.ProactivePhrase`, so
|
||||
# DescriptionOpener, PaddingPhrase, SentenceOpenerThereIs and CompositionNote
|
||||
# can each be retired underneath a passing probe.
|
||||
# are invisible to them.
|
||||
#
|
||||
# Which is a statement about the LEVEL of each rule, and only that. It used to
|
||||
# read as though those four rules were uncovered outright, and they were: case
|
||||
# 35 at the end of this file is what closed that, enumerating the style
|
||||
# directories at run time and demanding an alert from every rule it finds. The
|
||||
# two cases are complementary and neither subsumes the other — 35 proves a rule
|
||||
# still matches text, this one proves the match is still blocking.
|
||||
#
|
||||
# The original's comments, verbatim:
|
||||
#
|
||||
@@ -2364,6 +2371,124 @@ EOF_MUT34
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- 35. Every shipped rule actually fires on a fixture ---------------------
|
||||
#
|
||||
# The last coverage class the deleted scripts/check-vale-style-sync.sh and its
|
||||
# consumer suite took with them (ADR-0025). Cases 28-31 keep a rule LOADED, at
|
||||
# `error`, and in scope; none of them asks whether the rule still MATCHES
|
||||
# anything. Measured: rewriting CompositionNote.yml's tokens so they match no
|
||||
# text left this suite at 63/63 passed — the rule shipped, was loaded, was
|
||||
# blocking, and was inert.
|
||||
#
|
||||
# The rule list is discovered from the style directories at run time, never
|
||||
# hardcoded, and a discovered rule with no fixture row is a FAILURE rather than
|
||||
# a silent skip. That direction is the one that decays: a hardcoded list lets
|
||||
# rule #7 ship uncovered, and a fixture table read as "check the rows I have"
|
||||
# does exactly the same.
|
||||
VALE_ASSETS35="$FACTORY_AUDIT/assets/vale"
|
||||
|
||||
# `<Style>.<Rule>` for every rule file under styles/Kyberforge*/, which is how
|
||||
# vale itself names an alert.
|
||||
discovered_rules35() {
|
||||
local dir rule style name
|
||||
for dir in "$VALE_ASSETS35"/styles/Kyberforge*/; do
|
||||
[[ -d "$dir" ]] || continue
|
||||
style="${dir%/}"
|
||||
style="${style##*/}"
|
||||
for rule in "$dir"*.yml; do
|
||||
[[ -f "$rule" ]] || continue
|
||||
name="${rule##*/}"
|
||||
printf '%s.%s\n' "$style" "${name%.yml}"
|
||||
done
|
||||
done | sort
|
||||
}
|
||||
|
||||
# `<Style>.<Rule>|<path>|<description>|<body>`. The path decides which [glob]
|
||||
# section of .vale.ini applies, so KyberforgeCopilot's row has to be an
|
||||
# .agent.md file — that style is loaded nowhere else (ADR-0013). Each fixture
|
||||
# carries exactly the one trigger its rule is about; the rest of the text is
|
||||
# deliberately clean, so an alert for the wrong rule cannot satisfy the row.
|
||||
RULE_FIXTURES35="$(
|
||||
cat << 'EOF_FIX35'
|
||||
Kyberforge.CompositionNote|composition/SKILL.md|Use when the caller wants a cross-cutting probe. Do not use for anything else.|Body text.
|
||||
Kyberforge.DescriptionOpener|opener/SKILL.md|This is the description opener under test. Do not use for anything else.|Body text.
|
||||
Kyberforge.PaddingPhrase|padding/SKILL.md|Use when the caller wants a probe. Do not use for anything else.|See references for more info.
|
||||
Kyberforge.SentenceOpenerThereIs|sentence-opener/SKILL.md|Use when the caller wants a probe. Do not use for anything else.|There is a defect here.
|
||||
Kyberforge.VagueWording|vague/SKILL.md|Use when the caller helps with a probe. Do not use for anything else.|Body text.
|
||||
KyberforgeCopilot.ProactivePhrase|proactive/probe.agent.md|Use when the caller wants a probe. Use proactively.|Body text.
|
||||
EOF_FIX35
|
||||
)"
|
||||
|
||||
echo ""
|
||||
echo "--- every shipped Vale rule has a fixture, and every fixture names a shipped rule (no vale needed) ---"
|
||||
|
||||
DISCOVERED35="$(discovered_rules35)"
|
||||
UNCOVERED35=""
|
||||
STALE_FIXTURES35=""
|
||||
while IFS= read -r RULE35; do
|
||||
[[ -n "$RULE35" ]] || continue
|
||||
FOUND35=false
|
||||
while IFS='|' read -r FID35 _; do
|
||||
[[ "$FID35" == "$RULE35" ]] && { FOUND35=true; break; }
|
||||
done <<EOF_COV35
|
||||
$RULE_FIXTURES35
|
||||
EOF_COV35
|
||||
[[ "$FOUND35" == true ]] || UNCOVERED35+="$RULE35 "
|
||||
done <<EOF_RULES35
|
||||
$DISCOVERED35
|
||||
EOF_RULES35
|
||||
while IFS='|' read -r FID35 _; do
|
||||
[[ -n "$FID35" ]] || continue
|
||||
grep -qxF "$FID35" <<< "$DISCOVERED35" || STALE_FIXTURES35+="$FID35 "
|
||||
done <<EOF_STALE35
|
||||
$RULE_FIXTURES35
|
||||
EOF_STALE35
|
||||
|
||||
if [[ -z "$DISCOVERED35" ]]; then
|
||||
fail "no rule files were found under $VALE_ASSETS35/styles/Kyberforge*/ — either the styles were gutted or this discovery no longer reaches them, and every assertion below would be vacuous"
|
||||
elif [[ -n "$UNCOVERED35" ]]; then
|
||||
fail "shipped Vale rule(s) have no fixture row, so nothing proves they still match anything: $UNCOVERED35"
|
||||
elif [[ -n "$STALE_FIXTURES35" ]]; then
|
||||
fail "fixture row(s) name a rule that no longer ships, so those rows prove nothing about the live styles: $STALE_FIXTURES35"
|
||||
else
|
||||
pass "all $(printf '%s\n' "$DISCOVERED35" | wc -l | tr -d ' ') shipped rule(s) have a fixture row, and every row names a live rule"
|
||||
fi
|
||||
|
||||
if [[ "$VALE_READY" == true ]]; then
|
||||
echo ""
|
||||
echo "--- and each of those fixtures actually raises its own rule's alert ---"
|
||||
TREE35="$(mktemp -d)"
|
||||
new_fixture "$TREE35"
|
||||
INERT35=""
|
||||
while IFS='|' read -r RID35 RPATH35 RDESC35 RBODY35; do
|
||||
[[ -n "$RID35" ]] || continue
|
||||
mkdir -p "$TREE35/$(dirname "$RPATH35")"
|
||||
{
|
||||
echo "---"
|
||||
echo "name: probe"
|
||||
echo "description: $RDESC35"
|
||||
echo "---"
|
||||
echo ""
|
||||
echo "$RBODY35"
|
||||
} > "$TREE35/$RPATH35"
|
||||
# vale exits non-zero merely for HAVING alerts, which is the expected
|
||||
# outcome for every row here, hence the `|| true`.
|
||||
REPORT35="$( { (cd "$TREE35" && vale --config "$VALE_ASSETS35/.vale.ini" "$RPATH35" 2>&1) | sed -E 's/\x1b\[[0-9;]*m//g'; } || true)"
|
||||
if ! grep -qE 'in [0-9]+ files?\.' <<< "$REPORT35"; then
|
||||
INERT35+="[$RID35: vale printed no summary line for $RPATH35, so it did not run: ${REPORT35:-<no output>}] "
|
||||
elif ! grep -qF "$RID35" <<< "$REPORT35"; then
|
||||
INERT35+="[$RID35 raised no alert on its own fixture: ${REPORT35:-<no output>}] "
|
||||
fi
|
||||
done <<EOF_FIRE35
|
||||
$RULE_FIXTURES35
|
||||
EOF_FIRE35
|
||||
if [[ -n "$INERT35" ]]; then
|
||||
fail "a shipped rule matched nothing on the fixture written for it — it is loaded and blocking but inert, which is indistinguishable from a passing file: $INERT35"
|
||||
else
|
||||
pass "every shipped Vale rule raises its own alert on the fixture written for it"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Results: $PASS passed, $FAIL failed"
|
||||
[[ $FAIL -eq 0 ]] || exit 1
|
||||
|
||||
Reference in New Issue
Block a user