Why The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser (93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line test suite. Sync-by-gate had already failed once: at484357athe two parser copies drifted into different spellings of the bullet loop while a docstring asserted they were identical. That drift was behaviour-neutral and was re-unified by hand at598a7c3, so the copies were identical at merge time — but nothing had caught it, and the next drift need not be neutral. Implementation Notes Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching across skill directories, which is why two separate skills needed embedded copies; two files inside ONE skill may source a third. That is the whole reason the merge removes duplication rather than relocating it. The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path before any validation, and its table mirrors validate.sh's detection exactly: a directory holding SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory (agent); anything else stops without running a validator. Steps 1-3 live in references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the other artifact type, the body restarts at Step 0. Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that matches a live sibling reads as ownership rather than membership. The description carries one arrow per boundary target, because ADR-0020 resolves only the first target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated "audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16% of the body: no size SUGGESTIONs. The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both. tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources the resolver and that no third copy exists. The entry scripts classify the target from its resolved parent directory, so a bare agent filename typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is missing, rather than dying with exit 1, the tier the flows relay as real findings. The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's status (`f || RC=$?`) disables errexit for its entire body, and no subshell or `set -e` inside can re-arm it once the call sits in a condition context (measured, both spellings). Their error paths use `exit`, which is unaffected either way; this keeps errexit armed for anything added later. Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's dev environment is root -- so the guard could never fire where it exists to fire. A read attempt is also the stricter question, catching EIO. This is the reasoning scripts/check-vale-style-sync.sh carried before this commit deleted it; the hazard did not go with it. All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared, the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two remaining bare cd calls take absolute paths, which CDPATH is never consulted for. Impact BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to 2.0.0 (catalog 0.4.7). Check logic is unchanged: differential runs of the old and new validators across every skill and agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename. One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change, deliberately (full table in ADR-0025): - a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is by path shape, so a shape-matching path that is simply absent used to reach the validator and come back as a FAIL against a file that never existed; - an unshaped target exits 2 under the generic "matches neither" message, and a directory with no SKILL.md under a third, distinct one -- three exit-2 messages, not one; - a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding about the artifact rather than a usage error; - a SKILL.md file path is audited as its skill directory instead of refused; - a .md agent outside an agents/ directory is refused rather than audited; - a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2. validate-provenance.sh already exited 2 for the last two; validate.sh now matches it. .pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both files: regexes are unchanged; only entry: and description: moved. scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21), tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite. Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8. ADR: 0025 BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by factory-audit, which auto-detects whether it was handed a skill directory or an agent file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
536 lines
22 KiB
Bash
Executable File
536 lines
22 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# Works around a Vale limitation: the `text.frontmatter.description` NLP scope
|
|
# silently stops matching once the `description:` value spans 2+ physical lines
|
|
# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block
|
|
# scalar (the style used by most skills/agents in this repo), a plain scalar
|
|
# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped
|
|
# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed
|
|
# value keeps exactly the line breaks the source has, and vale matches it fine
|
|
# (verified against vale 3.15.2), so literal blocks are deliberately left alone.
|
|
# This script flattens an affected description to a one-line scalar in a scratch
|
|
# copy — or, for the rare value no inline scalar can spell out verbatim, to a
|
|
# `|-` literal block with a single content line, which vale matches just as well
|
|
# (padding with blank lines so every other line number is unchanged), then
|
|
# runs the real `vale` binary against the copies. Drop-in replacement for calling
|
|
# `vale` directly: same args, same exit code, bar the two documented divergences
|
|
# below.
|
|
#
|
|
# "Same args" means relative paths — path arguments and the values of the
|
|
# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against
|
|
# the caller's current directory, exactly as bare `vale` resolves them. The flag
|
|
# values are rewritten to absolute form because the run ends up `cd`'d into the
|
|
# scratch mirror, where a relative one would no longer resolve. (An earlier
|
|
# version resolved path arguments against the repo root, an invented convention
|
|
# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse,
|
|
# silently dropped file arguments that didn't happen to resolve from the repo
|
|
# root — skipping the flattening this script exists for.)
|
|
#
|
|
# Divergence 1: with no `--config` at all, this script's own sibling
|
|
# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit
|
|
# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in
|
|
# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and
|
|
# hard-fail (E100) for every external consumer. The manifest therefore passes the
|
|
# script alone, and an explicit `--config` from any other caller still wins.
|
|
#
|
|
# Divergence 2: a path-shaped argument that does not exist is a hard error
|
|
# (exit 2). Bare vale drops it, falls back to reading stdin, and prints
|
|
# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable
|
|
# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather
|
|
# than clean, and `in stdin` does not match that guard, so the silent form would
|
|
# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way
|
|
# to keep that guard honest. Linting prose piped on stdin is therefore
|
|
# unsupported here — it already was, since the no-path handoff closes stdin so
|
|
# vale can't block on a pipe that will never carry content.
|
|
#
|
|
# Vale prints each path exactly as it was handed to it, so the scratch tree
|
|
# mirrors the caller's absolute cwd: a relative path argument is passed through
|
|
# verbatim and resolves to its flattened copy, keeping the report byte-identical
|
|
# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that
|
|
# property. Only an absolute path outside the cwd is rewritten to its scratch
|
|
# copy and so reports a scratch path — unavoidable, since a file can only be
|
|
# read from where it actually is.
|
|
|
|
cwd="$(pwd -P)"
|
|
|
|
# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 —
|
|
# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"`
|
|
# on an empty array as an unbound variable under `set -u`. No expansion site is
|
|
# reachable while empty on today's control flow, so this is insurance against a
|
|
# later edit breaking that invariant, not a live fix.
|
|
vale_args=()
|
|
path_args=()
|
|
pending_flag=""
|
|
config_given=false
|
|
|
|
# `--output` takes either one of vale's built-in style names or a template file
|
|
# path. Only the file form needs absolutizing, and the built-in names have to be
|
|
# excluded by name *before* the existence test below: a file or directory
|
|
# literally called `line` in the caller's cwd would otherwise rewrite the
|
|
# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template]
|
|
# Runtime error`) where bare vale just uses the built-in. `--path` has no such
|
|
# names — it is always a path — so the check is keyed on the flag too.
|
|
is_builtin_output() {
|
|
case "$2" in
|
|
line|JSON|CLI) [[ "$1" == "--output" ]] ;;
|
|
*) false ;;
|
|
esac
|
|
}
|
|
# Absolutizes a `--config` value against the caller's cwd. Shared by both
|
|
# argument forms below — separated (`--config X`) and joined (`--config=X`)
|
|
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
|
|
# one place instead of being duplicated per form.
|
|
abs_config_value() {
|
|
if [[ "$1" == /* ]]; then
|
|
printf '%s' "$1"
|
|
else
|
|
printf '%s' "$cwd/$1"
|
|
fi
|
|
}
|
|
for arg in "$@"; do
|
|
if [[ -n "$pending_flag" ]]; then
|
|
# Value of a separated two-argv flag. It is never a lint target, however
|
|
# file-like it looks. The run ends up `cd`'d into the scratch mirror, so a
|
|
# value naming a file has to be absolutized here or it stops resolving.
|
|
case "$pending_flag" in
|
|
--config)
|
|
# Always a path, and required to exist.
|
|
vale_args+=("$(abs_config_value "$arg")")
|
|
;;
|
|
--output|--path)
|
|
# See `is_builtin_output` above for why the built-in `--output` names
|
|
# are excluded first. Anything that names nothing is passed through and
|
|
# left for vale to interpret.
|
|
if is_builtin_output "$pending_flag" "$arg"; then
|
|
vale_args+=("$arg")
|
|
elif [[ "$arg" != /* && -e "$arg" ]]; then
|
|
vale_args+=("$cwd/$arg")
|
|
else
|
|
vale_args+=("$arg")
|
|
fi
|
|
;;
|
|
*)
|
|
vale_args+=("$arg")
|
|
;;
|
|
esac
|
|
pending_flag=""
|
|
continue
|
|
fi
|
|
case "$arg" in
|
|
--config)
|
|
vale_args+=("$arg")
|
|
pending_flag="$arg"
|
|
config_given=true
|
|
continue
|
|
;;
|
|
--config=*)
|
|
vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
|
|
config_given=true
|
|
continue
|
|
;;
|
|
# Same cwd-relative resolution for the `--flag=value` spelling of the two
|
|
# other path-valued flags.
|
|
--output=*|--path=*)
|
|
flag_val="${arg#*=}"
|
|
if is_builtin_output "${arg%%=*}" "$flag_val"; then
|
|
vale_args+=("$arg")
|
|
elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then
|
|
vale_args+=("${arg%%=*}=$cwd/$flag_val")
|
|
else
|
|
vale_args+=("$arg")
|
|
fi
|
|
continue
|
|
;;
|
|
# Vale's remaining value-taking flags, per `vale --help` (3.x). In the
|
|
# separated two-argv form the value must not be classified as a lint target
|
|
# — `--output tmpl.tmpl` names a real template file, and treating it as
|
|
# input both lints the template and reorders argv so vale sees
|
|
# `--output --no-wrap`. The `--flag=value` form needs no entry here: it
|
|
# starts with `-` and falls through to vale untouched. A value flag added by
|
|
# some future vale release is simply absent from this list and lands back on
|
|
# today's behaviour, so this list going stale is never worse than not having
|
|
# it.
|
|
--ext|--filter|--glob|--minAlertLevel|--output|--path)
|
|
vale_args+=("$arg")
|
|
pending_flag="$arg"
|
|
continue
|
|
;;
|
|
# Vale's subcommands are bare words that name no file, so they would trip
|
|
# the not-found error below. A lint target literally named `sync` (no
|
|
# extension, no slash) is misread as the subcommand — accepted, because the
|
|
# alternative is failing every `vale-wrap.sh ls-config`.
|
|
ls-config|ls-dirs|ls-metrics|ls-vars|sync)
|
|
vale_args+=("$arg")
|
|
continue
|
|
;;
|
|
esac
|
|
if [[ "$arg" == -* ]]; then
|
|
vale_args+=("$arg")
|
|
continue
|
|
fi
|
|
# Everything left is a lint target: `vale [options] [input...]` has no third
|
|
# kind of argument. See divergence 2 above for why a missing one is fatal here.
|
|
if [[ ! -e "$arg" ]]; then
|
|
echo "vale-wrap.sh: no such file or directory: $arg" >&2
|
|
exit 2
|
|
fi
|
|
# An absolute path inside the caller's cwd is relativized so the report cites
|
|
# a path that resolves against the real tree. Left absolute, it would be
|
|
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
|
|
# path to a file that is deleted on exit, which reads as a bug in any report
|
|
# quoting it. Absolute paths outside the cwd have no relative form and keep
|
|
# the scratch-path behaviour documented above.
|
|
if [[ "$arg" == "$cwd"/* ]]; then
|
|
path_args+=("${arg#"$cwd"/}")
|
|
else
|
|
path_args+=("$arg")
|
|
fi
|
|
done
|
|
|
|
if [[ "$config_given" == false ]]; then
|
|
# `scripts/../assets/vale` begins with neither `/` nor `.`, so `cd` consults
|
|
# CDPATH for it — and when a CDPATH entry supplies the directory, `cd` PRINTS
|
|
# the directory it chose. A bare `$(cd ... && pwd)` therefore captured TWO
|
|
# lines, and the chosen directory could be an unrelated tree entirely: with
|
|
# CDPATH=/tmp/decoy and /tmp/decoy/scripts present, this resolved to
|
|
# /tmp/decoy/assets/vale and vale died on a two-line --config path. CDPATH is
|
|
# cleared for the one command, `--` ends option parsing for a directory named
|
|
# like a flag, and stdout is discarded so only `pwd` is captured. Same fix as
|
|
# validate.sh and validate-provenance.sh apply to their SCRIPT_DIR.
|
|
vale_args+=(--config "$(CDPATH='' cd -- "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" > /dev/null && pwd)/.vale.ini")
|
|
fi
|
|
|
|
if [[ ${#path_args[@]} -eq 0 ]]; then
|
|
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
|
|
# block waiting on a pipe that will never carry content.
|
|
exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null
|
|
fi
|
|
|
|
# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing)
|
|
# is a GNU extension the BSD realpath on macOS doesn't have — and every dest
|
|
# below is a path that doesn't exist yet. python3 is already a hard dependency.
|
|
abspath() {
|
|
python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1"
|
|
}
|
|
|
|
flatten() {
|
|
# Two call shapes: `flatten src dest` (dest already resolved and inside the
|
|
# scratch tree — the per-markdown-file calls in the directory branch below)
|
|
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
|
|
# branch further down) additionally resolves `raw_dest` the way a separate
|
|
# `abspath` call used to, applies the same sandbox-escape guard, and prints
|
|
# the resolved path — folding two python3 spawns per file into one.
|
|
python3 - "$@" <<'PYTHON'
|
|
import os
|
|
import re
|
|
import sys
|
|
|
|
src, dest_input = sys.argv[1], sys.argv[2]
|
|
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
|
|
|
|
if tmpdir is None:
|
|
dest = dest_input
|
|
else:
|
|
dest = os.path.abspath(dest_input)
|
|
if not dest.startswith(tmpdir + os.sep):
|
|
print(
|
|
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
|
|
f"land outside {tmpdir}",
|
|
file=sys.stderr,
|
|
)
|
|
sys.exit(2)
|
|
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
|
|
|
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
|
|
# a byte-for-byte round trip instead of aborting the whole run on a decode error.
|
|
with open(src, encoding='utf-8', errors='surrogateescape') as fh:
|
|
content = fh.read()
|
|
|
|
# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\<newline>` is handled
|
|
# separately in unescape_double because it also swallows the next indentation.
|
|
DQ_ESCAPES = {
|
|
'0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n',
|
|
'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"',
|
|
'/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028',
|
|
'P': '\u2029',
|
|
}
|
|
|
|
# First characters that make a plain (unquoted) scalar mean something other than
|
|
# text: YAML's c-indicator set.
|
|
PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`'
|
|
|
|
|
|
def unescape_double(text):
|
|
"""Decode a double-quoted YAML scalar's body to the string YAML parses."""
|
|
out = []
|
|
i = 0
|
|
while i < len(text):
|
|
char = text[i]
|
|
if char != '\\':
|
|
out.append(char)
|
|
i += 1
|
|
continue
|
|
i += 1
|
|
if i >= len(text):
|
|
break
|
|
esc = text[i]
|
|
if esc == '\n':
|
|
i += 1
|
|
while i < len(text) and text[i] in ' \t':
|
|
i += 1
|
|
continue
|
|
if esc in 'xuU':
|
|
width = {'x': 2, 'u': 4, 'U': 8}[esc]
|
|
digits = text[i + 1:i + 1 + width]
|
|
if len(digits) == width:
|
|
try:
|
|
out.append(chr(int(digits, 16)))
|
|
except ValueError:
|
|
pass
|
|
else:
|
|
i += 1 + width
|
|
continue
|
|
out.append(DQ_ESCAPES.get(esc, esc))
|
|
i += 1
|
|
return ''.join(out)
|
|
|
|
|
|
def close_quote(text, quote):
|
|
"""Index of the closing `quote` in `text`, which starts just past the
|
|
opening one. None while the scalar is still unterminated."""
|
|
i = 0
|
|
while i < len(text):
|
|
char = text[i]
|
|
if quote == '"' and char == '\\':
|
|
i += 2
|
|
continue
|
|
if char == quote:
|
|
if quote == "'" and text[i + 1:i + 2] == "'":
|
|
i += 2
|
|
continue
|
|
return i
|
|
i += 1
|
|
return None
|
|
|
|
|
|
def continuation_lines(rest):
|
|
"""Yield the physical lines of `rest` that continue the value started on the
|
|
`description:` line. Indentation-based and blank-line-tolerant, per YAML:
|
|
a blank line (any amount of whitespace) always stays inside; the indent is
|
|
set by the first content line; the value ends at the first line indented
|
|
less than that, at any line flush with the key (that is the next mapping
|
|
key, not a continuation), or at EOF."""
|
|
indent = None
|
|
for line in rest.splitlines(keepends=True):
|
|
text = line.rstrip('\n')
|
|
if text.strip() == '':
|
|
yield line
|
|
continue
|
|
line_indent = len(text) - len(text.lstrip(' \t'))
|
|
if line_indent == 0:
|
|
return
|
|
if indent is None:
|
|
indent = line_indent
|
|
elif line_indent < indent:
|
|
return
|
|
yield line
|
|
|
|
|
|
def emit(value):
|
|
"""Render `value` as a YAML scalar whose source text spells the value out
|
|
verbatim. Vale locates the description by matching the parsed value back
|
|
against the source, so a scalar carrying any escape — `''` in a
|
|
single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the
|
|
whole `text.frontmatter.description` scope vanish, the same failure this
|
|
script exists to work around. Verbatim forms only, therefore, tried in
|
|
descending order of fidelity. The first three occupy one physical line; the
|
|
`|-` fallback occupies two, which the caller accounts for when padding."""
|
|
if (value
|
|
and value[0] not in PLAIN_UNSAFE_FIRST
|
|
and ': ' not in value
|
|
and not value.endswith(':')
|
|
and ' #' not in value):
|
|
return value # plain: nothing needs escaping at all
|
|
if "'" not in value:
|
|
return "'" + value + "'" # single-quoted: only `'` would escape
|
|
if '"' not in value and '\\' not in value:
|
|
return '"' + value + '"' # double-quoted: only `"`/`\` would
|
|
# Last resort: the value needs quoting AND holds an apostrophe AND a double
|
|
# quote or backslash, so no *inline* scalar can carry it verbatim. A `|-`
|
|
# literal block can — a block scalar's body has no escape syntax at all, so
|
|
# `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches
|
|
# the description scope against it (the header above says the same of the
|
|
# `|` blocks this script deliberately leaves alone; verified against vale
|
|
# 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed
|
|
# value is exactly `value` with no trailing newline.
|
|
return '|-\n ' + value
|
|
|
|
|
|
fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL)
|
|
if fm_match:
|
|
fm = fm_match.group(2)
|
|
header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE)
|
|
else:
|
|
header_m = None
|
|
|
|
if header_m:
|
|
head_start = header_m.start()
|
|
value_start = header_m.end()
|
|
header_end = fm.find('\n', value_start)
|
|
header_end = len(fm) if header_end == -1 else header_end
|
|
first = fm[value_start:header_end]
|
|
body_start = header_end + 1
|
|
indicator = first.rstrip()
|
|
|
|
block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator)
|
|
if block_m and block_m.group(1) == '|':
|
|
kind = None # literal blocks keep their line breaks; vale is fine
|
|
elif block_m:
|
|
kind = 'block' # folded (`>`): the value starts on the next line
|
|
elif indicator == '':
|
|
kind = 'block' # bare `description:`: a plain scalar on later lines
|
|
elif first[:1] == '"':
|
|
kind = 'double'
|
|
elif first[:1] == "'":
|
|
kind = 'single'
|
|
elif first[:1] in '#&*!':
|
|
kind = None # comment, anchor, alias or tag — not a plain scalar
|
|
else:
|
|
kind = 'plain'
|
|
|
|
text = ''
|
|
value_end = value_start
|
|
value_lines = 0
|
|
if kind in ('block', 'plain'):
|
|
body = ''.join(continuation_lines(fm[body_start:]))
|
|
value_end = body_start + len(body)
|
|
if kind == 'block':
|
|
text = body
|
|
value_lines = body.count('\n')
|
|
else:
|
|
text = fm[value_start:value_end]
|
|
value_lines = 1 + body.count('\n')
|
|
if ' #' in text or text.lstrip().startswith('#'):
|
|
# A `#` opens a comment inside a plain scalar. Folding it in
|
|
# would lint text YAML never treats as part of the value, so
|
|
# leave the file alone rather than lint the wrong string.
|
|
kind = None
|
|
elif kind in ('double', 'single'):
|
|
quote = '"' if kind == 'double' else "'"
|
|
inner_start = value_start + 1
|
|
acc = fm[inner_start:body_start]
|
|
idx = close_quote(acc, quote)
|
|
lines = continuation_lines(fm[body_start:])
|
|
while idx is None:
|
|
try:
|
|
acc += next(lines)
|
|
except StopIteration:
|
|
break
|
|
idx = close_quote(acc, quote)
|
|
if idx is None:
|
|
kind = None # unterminated quote: invalid YAML, leave it to vale
|
|
else:
|
|
inner = acc[:idx]
|
|
value_end = inner_start + idx + 1
|
|
text = unescape_double(inner) if quote == '"' else inner.replace("''", "'")
|
|
value_lines = 1 + inner.count('\n')
|
|
|
|
flat = re.sub(r'\s+', ' ', text).strip()
|
|
if kind and flat and value_lines >= 2:
|
|
# `value_end` can land mid-line, just past a closing quote, so extend to
|
|
# the end of that physical line and carry whatever follows (a trailing
|
|
# comment) across unchanged.
|
|
if value_end > 0 and fm[value_end - 1] == '\n':
|
|
span_end = value_end
|
|
trailer = ''
|
|
else:
|
|
newline = fm.find('\n', value_end)
|
|
span_end = len(fm) if newline == -1 else newline + 1
|
|
trailer = fm[value_end:span_end].rstrip('\n')
|
|
scalar = emit(flat)
|
|
# A trailing comment carried across from the original line stays on the
|
|
# `description:` line itself: after a block scalar's `|-` header it is
|
|
# still a comment, but inside the block body it would become part of the
|
|
# value.
|
|
head, newline_sep, block_body = scalar.partition('\n')
|
|
# The replacement displaces the whole span, so the blank-line pad makes
|
|
# up the difference between the lines it displaced and the lines it
|
|
# occupies — every later line number is unchanged. That is one line for
|
|
# the three inline forms and two for the `|-` block; the span itself is
|
|
# at least two lines here (`value_lines >= 2` is a precondition), so the
|
|
# pad count never goes negative.
|
|
pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n'))
|
|
new_fm = (fm[:head_start] + 'description: ' + head + trailer
|
|
+ newline_sep + block_body + '\n' + pad + fm[span_end:])
|
|
content = (fm_match.group(1) + new_fm + fm_match.group(3)
|
|
+ content[fm_match.end():])
|
|
|
|
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
|
|
fh.write(content)
|
|
|
|
if tmpdir is not None:
|
|
print(dest)
|
|
PYTHON
|
|
}
|
|
|
|
tmpdir="$(cd "$(mktemp -d)" && pwd -P)"
|
|
trap 'rm -rf "$tmpdir"' EXIT
|
|
|
|
# Mirror of the caller's cwd inside the scratch tree; relative path arguments
|
|
# are resolved from here.
|
|
mirror="$tmpdir$cwd"
|
|
mkdir -p "$mirror"
|
|
|
|
argv_paths=()
|
|
for arg in ${path_args[@]+"${path_args[@]}"}; do
|
|
if [[ "$arg" == /* ]]; then
|
|
raw_dest="$tmpdir$arg"
|
|
else
|
|
raw_dest="$mirror/$arg"
|
|
fi
|
|
if [[ -d "$arg" ]]; then
|
|
dest="$(abspath "$raw_dest")"
|
|
# A path argument with enough leading `..` to climb past the mirror root would
|
|
# write outside the scratch dir. The real filesystem clamps such a path at
|
|
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
|
|
case "$dest" in
|
|
"$tmpdir"/*) ;;
|
|
*)
|
|
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
|
|
exit 2
|
|
;;
|
|
esac
|
|
mkdir -p "$(dirname "$dest")"
|
|
# A directory is mirrored whole — vale applies its own format filtering to
|
|
# the tree, so any file dropped here would be silently unlinted — and then
|
|
# every markdown file in the copy is flattened in place. `.git` is pruned:
|
|
# vale never lints it and copying it can dwarf the rest of the tree.
|
|
# `find -L` follows symlinks because vale does: it lints both a symlinked
|
|
# file and a file under a symlinked directory, and a bare `-type f` walk
|
|
# would report "0 files" where bare vale reports one. (A symlink loop makes
|
|
# `find` warn on stderr and carry on, which is also what vale does.) The
|
|
# second walk needs no `-L`: the mirror is all real files by construction.
|
|
mkdir -p "$dest"
|
|
while IFS= read -r -d '' rel; do
|
|
mkdir -p "$dest/$(dirname "$rel")"
|
|
cp "$arg/$rel" "$dest/$rel"
|
|
done < <(CDPATH='' cd -- "$arg" && find -L . -name .git -prune -o -type f -print0)
|
|
while IFS= read -r -d '' md; do
|
|
flatten "$md" "$md"
|
|
done < <(find "$dest" -type f -name '*.md' -print0)
|
|
else
|
|
# `abspath` + `flatten` folded into one python3 process — see the comment
|
|
# atop `flatten` above.
|
|
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
|
|
fi
|
|
if [[ "$arg" == /* ]]; then
|
|
argv_paths+=("$dest")
|
|
else
|
|
argv_paths+=("$arg")
|
|
fi
|
|
done
|
|
|
|
cd "$mirror"
|
|
vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"}
|