2 Commits

Author SHA1 Message Date
e62f68a1cc refactor(lint): cache manifest parsing, single-pass size check
Two more efficiency findings from the same code-review pass:

- check-vale-style-sync.sh's hook_file_regexes() reparsed both
  pre-commit manifests from scratch on every call. The final
  validation loop calls it once per probe (3 probes: skill-audit once,
  agent-audit twice for its two file shapes), so agent-audit's regex
  set was being parsed twice for no reason. Now cached per skill in a
  lazily-populated associative array, with a separate "seen" map so an
  empty result isn't mistaken for "not yet computed."
- skill-size-check.sh read the target file twice (separate awk and
  wc -w calls) to get line and word counts; now a single awk pass
  returns both. Also documented, next to MAX_LINES/MAX_WORDS, why
  those constants are duplicated against skill-audit/scripts/
  validate.sh's Python implementation rather than unified — same
  cross-language/cross-context tradeoff as vale-wrap.sh's duplication,
  guarded by tests/test-skill-size-check.sh's drift check.

Verified: test-check-vale-style-sync.sh 20/20, test-skill-size-check.sh
9/9, full suite 12/12, pre-commit --all-files clean.
2026-08-09 20:14:36 +00:00
680aa4f43c refactor(kyberforge): consolidate vale-wrap.sh's config parsing and subprocess spawns
Two efficiency findings from a code-review pass:
- The separated (--config X) and joined (--config=X) argument branches
  duplicated ~20 lines of absolutize-if-relative path logic. Extracted
  into abs_config_value(), used by both branches; the redundant
  --config=/* special case falls out since the helper already passes
  absolute paths through unchanged.
- The common single-file path spawned python3 twice per file (once for
  abspath resolution, once for flatten()). flatten() now optionally
  takes a tmpdir arg and does both in one process. The per-file loop
  under a directory argument is unchanged — that path wasn't flagged.

agent-audit's copy is canonical; skill-audit's copy was regenerated via
scripts/sync-vale-styles.sh, not hand-edited, to guarantee byte parity.
No hardening (bash 3.2 compat, surrogateescape, symlink guards) touched.
Verified: tests/test-vale-wrap.sh 39/39, check-vale-style-sync.sh clean,
full suite 12/12.
2026-08-09 20:14:23 +00:00
4 changed files with 167 additions and 80 deletions

View File

@@ -77,6 +77,17 @@ is_builtin_output() {
*) false ;; *) false ;;
esac esac
} }
# Absolutizes a `--config` value against the caller's cwd. Shared by both
# argument forms below — separated (`--config X`) and joined (`--config=X`)
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
# one place instead of being duplicated per form.
abs_config_value() {
if [[ "$1" == /* ]]; then
printf '%s' "$1"
else
printf '%s' "$cwd/$1"
fi
}
for arg in "$@"; do for arg in "$@"; do
if [[ -n "$pending_flag" ]]; then if [[ -n "$pending_flag" ]]; then
# Value of a separated two-argv flag. It is never a lint target, however # Value of a separated two-argv flag. It is never a lint target, however
@@ -85,11 +96,7 @@ for arg in "$@"; do
case "$pending_flag" in case "$pending_flag" in
--config) --config)
# Always a path, and required to exist. # Always a path, and required to exist.
if [[ "$arg" == /* ]]; then vale_args+=("$(abs_config_value "$arg")")
vale_args+=("$arg")
else
vale_args+=("$cwd/$arg")
fi
;; ;;
--output|--path) --output|--path)
# See `is_builtin_output` above for why the built-in `--output` names # See `is_builtin_output` above for why the built-in `--output` names
@@ -117,13 +124,8 @@ for arg in "$@"; do
config_given=true config_given=true
continue continue
;; ;;
--config=/*)
vale_args+=("$arg")
config_given=true
continue
;;
--config=*) --config=*)
vale_args+=("--config=$cwd/${arg#--config=}") vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
config_given=true config_given=true
continue continue
;; ;;
@@ -204,11 +206,33 @@ abspath() {
} }
flatten() { flatten() {
python3 - "$1" "$2" <<'PYTHON' # Two call shapes: `flatten src dest` (dest already resolved and inside the
# scratch tree — the per-markdown-file calls in the directory branch below)
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
# branch further down) additionally resolves `raw_dest` the way a separate
# `abspath` call used to, applies the same sandbox-escape guard, and prints
# the resolved path — folding two python3 spawns per file into one.
python3 - "$@" <<'PYTHON'
import os
import re import re
import sys import sys
src, dest = sys.argv[1], sys.argv[2] src, dest_input = sys.argv[1], sys.argv[2]
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
if tmpdir is None:
dest = dest_input
else:
dest = os.path.abspath(dest_input)
if not dest.startswith(tmpdir + os.sep):
print(
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
f"land outside {tmpdir}",
file=sys.stderr,
)
sys.exit(2)
os.makedirs(os.path.dirname(dest), exist_ok=True)
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument) # surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
# a byte-for-byte round trip instead of aborting the whole run on a decode error. # a byte-for-byte round trip instead of aborting the whole run on a decode error.
with open(src, encoding='utf-8', errors='surrogateescape') as fh: with open(src, encoding='utf-8', errors='surrogateescape') as fh:
@@ -435,6 +459,9 @@ if header_m:
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh: with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
fh.write(content) fh.write(content)
if tmpdir is not None:
print(dest)
PYTHON PYTHON
} }
@@ -449,23 +476,23 @@ mkdir -p "$mirror"
argv_paths=() argv_paths=()
for arg in ${path_args[@]+"${path_args[@]}"}; do for arg in ${path_args[@]+"${path_args[@]}"}; do
if [[ "$arg" == /* ]]; then if [[ "$arg" == /* ]]; then
dest="$tmpdir$arg" raw_dest="$tmpdir$arg"
else else
dest="$mirror/$arg" raw_dest="$mirror/$arg"
fi fi
dest="$(abspath "$dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
if [[ -d "$arg" ]]; then if [[ -d "$arg" ]]; then
dest="$(abspath "$raw_dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
# A directory is mirrored whole — vale applies its own format filtering to # A directory is mirrored whole — vale applies its own format filtering to
# the tree, so any file dropped here would be silently unlinted — and then # the tree, so any file dropped here would be silently unlinted — and then
# every markdown file in the copy is flattened in place. `.git` is pruned: # every markdown file in the copy is flattened in place. `.git` is pruned:
@@ -484,7 +511,9 @@ for arg in ${path_args[@]+"${path_args[@]}"}; do
flatten "$md" "$md" flatten "$md" "$md"
done < <(find "$dest" -type f -name '*.md' -print0) done < <(find "$dest" -type f -name '*.md' -print0)
else else
flatten "$arg" "$dest" # `abspath` + `flatten` folded into one python3 process — see the comment
# atop `flatten` above.
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
fi fi
if [[ "$arg" == /* ]]; then if [[ "$arg" == /* ]]; then
argv_paths+=("$dest") argv_paths+=("$dest")

View File

@@ -77,6 +77,17 @@ is_builtin_output() {
*) false ;; *) false ;;
esac esac
} }
# Absolutizes a `--config` value against the caller's cwd. Shared by both
# argument forms below — separated (`--config X`) and joined (`--config=X`)
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
# one place instead of being duplicated per form.
abs_config_value() {
if [[ "$1" == /* ]]; then
printf '%s' "$1"
else
printf '%s' "$cwd/$1"
fi
}
for arg in "$@"; do for arg in "$@"; do
if [[ -n "$pending_flag" ]]; then if [[ -n "$pending_flag" ]]; then
# Value of a separated two-argv flag. It is never a lint target, however # Value of a separated two-argv flag. It is never a lint target, however
@@ -85,11 +96,7 @@ for arg in "$@"; do
case "$pending_flag" in case "$pending_flag" in
--config) --config)
# Always a path, and required to exist. # Always a path, and required to exist.
if [[ "$arg" == /* ]]; then vale_args+=("$(abs_config_value "$arg")")
vale_args+=("$arg")
else
vale_args+=("$cwd/$arg")
fi
;; ;;
--output|--path) --output|--path)
# See `is_builtin_output` above for why the built-in `--output` names # See `is_builtin_output` above for why the built-in `--output` names
@@ -117,13 +124,8 @@ for arg in "$@"; do
config_given=true config_given=true
continue continue
;; ;;
--config=/*)
vale_args+=("$arg")
config_given=true
continue
;;
--config=*) --config=*)
vale_args+=("--config=$cwd/${arg#--config=}") vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
config_given=true config_given=true
continue continue
;; ;;
@@ -204,11 +206,33 @@ abspath() {
} }
flatten() { flatten() {
python3 - "$1" "$2" <<'PYTHON' # Two call shapes: `flatten src dest` (dest already resolved and inside the
# scratch tree — the per-markdown-file calls in the directory branch below)
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
# branch further down) additionally resolves `raw_dest` the way a separate
# `abspath` call used to, applies the same sandbox-escape guard, and prints
# the resolved path — folding two python3 spawns per file into one.
python3 - "$@" <<'PYTHON'
import os
import re import re
import sys import sys
src, dest = sys.argv[1], sys.argv[2] src, dest_input = sys.argv[1], sys.argv[2]
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
if tmpdir is None:
dest = dest_input
else:
dest = os.path.abspath(dest_input)
if not dest.startswith(tmpdir + os.sep):
print(
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
f"land outside {tmpdir}",
file=sys.stderr,
)
sys.exit(2)
os.makedirs(os.path.dirname(dest), exist_ok=True)
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument) # surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
# a byte-for-byte round trip instead of aborting the whole run on a decode error. # a byte-for-byte round trip instead of aborting the whole run on a decode error.
with open(src, encoding='utf-8', errors='surrogateescape') as fh: with open(src, encoding='utf-8', errors='surrogateescape') as fh:
@@ -435,6 +459,9 @@ if header_m:
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh: with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
fh.write(content) fh.write(content)
if tmpdir is not None:
print(dest)
PYTHON PYTHON
} }
@@ -449,23 +476,23 @@ mkdir -p "$mirror"
argv_paths=() argv_paths=()
for arg in ${path_args[@]+"${path_args[@]}"}; do for arg in ${path_args[@]+"${path_args[@]}"}; do
if [[ "$arg" == /* ]]; then if [[ "$arg" == /* ]]; then
dest="$tmpdir$arg" raw_dest="$tmpdir$arg"
else else
dest="$mirror/$arg" raw_dest="$mirror/$arg"
fi fi
dest="$(abspath "$dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
if [[ -d "$arg" ]]; then if [[ -d "$arg" ]]; then
dest="$(abspath "$raw_dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
# A directory is mirrored whole — vale applies its own format filtering to # A directory is mirrored whole — vale applies its own format filtering to
# the tree, so any file dropped here would be silently unlinted — and then # the tree, so any file dropped here would be silently unlinted — and then
# every markdown file in the copy is flattened in place. `.git` is pruned: # every markdown file in the copy is flattened in place. `.git` is pruned:
@@ -484,7 +511,9 @@ for arg in ${path_args[@]+"${path_args[@]}"}; do
flatten "$md" "$md" flatten "$md" "$md"
done < <(find "$dest" -type f -name '*.md' -print0) done < <(find "$dest" -type f -name '*.md' -print0)
else else
flatten "$arg" "$dest" # `abspath` + `flatten` folded into one python3 process — see the comment
# atop `flatten` above.
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
fi fi
if [[ "$arg" == /* ]]; then if [[ "$arg" == /* ]]; then
argv_paths+=("$dest") argv_paths+=("$dest")

View File

@@ -80,26 +80,45 @@ done
# Prints the `files:` regex of every hook, in either manifest, whose entry is # Prints the `files:` regex of every hook, in either manifest, whose entry is
# $1's vale-wrap.sh. Records are delimited by their `- id:` line, so the check # $1's vale-wrap.sh. Records are delimited by their `- id:` line, so the check
# does not depend on `entry:` preceding `files:` within a record. # does not depend on `entry:` preceding `files:` within a record.
#
# Cached per skill (in HOOK_REGEX_CACHE, populated lazily) because the final
# validation loop below probes agent-audit twice — once for its CC agent-file
# shape, once for its Copilot .agent.md shape — and both probes need the same
# regex set. Without the cache, that pair of calls would each re-parse both
# manifest files from scratch for no new information. HOOK_REGEX_CACHE_SEEN is
# a separate array so a skill with no matching hooks (empty result) is still
# recognized as already computed, rather than re-parsed every call.
declare -A HOOK_REGEX_CACHE=()
declare -A HOOK_REGEX_CACHE_SEEN=()
hook_file_regexes() { hook_file_regexes() {
local skill="$1" manifest raw local skill="$1" manifest raw result
for manifest in "$REPO_ROOT/.pre-commit-hooks.yaml" "$REPO_ROOT/.pre-commit-config.yaml"; do if [[ -n "${HOOK_REGEX_CACHE_SEEN[$skill]:-}" ]]; then
[[ -f "$manifest" ]] || continue printf '%s' "${HOOK_REGEX_CACHE[$skill]}"
awk -v skill="$skill" ' return
function flush() { fi
if (entry ~ skill "/scripts/vale-wrap.sh" && files != "") print files result="$(
entry = ""; files = "" for manifest in "$REPO_ROOT/.pre-commit-hooks.yaml" "$REPO_ROOT/.pre-commit-config.yaml"; do
} [[ -f "$manifest" ]] || continue
/^[ \t]*-[ \t]*id:/ { flush() } awk -v skill="$skill" '
/^[ \t]*entry:/ { entry = $0 } function flush() {
/^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) } if (entry ~ skill "/scripts/vale-wrap.sh" && files != "") print files
END { flush() } entry = ""; files = ""
' "$manifest" }
done | while IFS= read -r raw; do /^[ \t]*-[ \t]*id:/ { flush() }
# Strip the surrounding YAML quotes; the regex itself never carries them. /^[ \t]*entry:/ { entry = $0 }
raw="${raw%\'}"; raw="${raw#\'}" /^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) }
raw="${raw%\"}"; raw="${raw#\"}" END { flush() }
printf '%s\n' "$raw" ' "$manifest"
done done | while IFS= read -r raw; do
# Strip the surrounding YAML quotes; the regex itself never carries them.
raw="${raw%\'}"; raw="${raw#\'}"
raw="${raw%\"}"; raw="${raw#\"}"
printf '%s\n' "$raw"
done
)"
HOOK_REGEX_CACHE[$skill]="$result"
HOOK_REGEX_CACHE_SEEN[$skill]=1
printf '%s' "$result"
} }
# Asks vale — the thing that actually applies these globs — whether a config # Asks vale — the thing that actually applies these globs — whether a config

View File

@@ -35,6 +35,13 @@ set -euo pipefail
# tokenization and does not replace one. Re-measure the corpus before treating # tokenization and does not replace one. Re-measure the corpus before treating
# any of these numbers as still current. # any of these numbers as still current.
# These constants are intentionally duplicated in
# skill-audit/scripts/validate.sh (Python) rather than shared from one file:
# this script is a standalone bash pre-commit hook, that one is an in-skill
# Python validator invoked in a different context (same rationale as
# vale-wrap.sh's per-plugin duplication — see its own header comment).
# tests/test-skill-size-check.sh asserts both files agree on these values, so
# drift between them fails CI rather than silently diverging.
MAX_LINES=500 MAX_LINES=500
MAX_WORDS=2770 MAX_WORDS=2770
FAIL=0 FAIL=0
@@ -42,16 +49,19 @@ FAIL=0
for f in "$@"; do for f in "$@"; do
[[ -f "$f" ]] || continue [[ -f "$f" ]] || continue
# awk's NR counts the final line even without a trailing newline, matching # Single awk pass computes both line count and word count, avoiding a
# Python's splitlines() semantics (used by skill-audit/scripts/validate.sh # second read of the file. NR counts the final line even without a
# for its own line count) — `wc -l` undercounts by 1 in that case. # trailing newline, matching Python's splitlines() semantics (used by
lines=$(awk 'END{print NR}' "$f") # skill-audit/scripts/validate.sh for its own line count) — `wc -l`
# undercounts by 1 in that case. Word count uses awk's default
# whitespace-splitting NF, matching `wc -w` semantics.
read -r lines words <<< "$(awk '{w += NF} END{print NR, w+0}' "$f")"
if (( lines > MAX_LINES )); then if (( lines > MAX_LINES )); then
echo "ERROR: $f has $lines lines, exceeding the $MAX_LINES-line ceiling (agentskills.io skill-authoring.md)" >&2 echo "ERROR: $f has $lines lines, exceeding the $MAX_LINES-line ceiling (agentskills.io skill-authoring.md)" >&2
FAIL=1 FAIL=1
fi fi
words=$(wc -w < "$f")
if (( words > MAX_WORDS )); then if (( words > MAX_WORDS )); then
echo "ERROR: $f has $words words (proxy for tokens), exceeding the $MAX_WORDS-word ceiling (~5,000 tokens, agentskills.io skill-authoring.md)" >&2 echo "ERROR: $f has $words words (proxy for tokens), exceeding the $MAX_WORDS-word ceiling (~5,000 tokens, agentskills.io skill-authoring.md)" >&2
FAIL=1 FAIL=1