docs: reconcile the ADRs, gates and audit log with the shipped behaviour

Why

A six-agent review of the two preceding commits found their code sound -- the
differential claim holds, the published hook contract is byte-unchanged -- but
their prose drifted from it in three ways: statements of fact the code
contradicts, markers in a convention this repo does not use, and figures that
went stale when the merge changed what they counted.

Implementation Notes

ADR-0025's edge-path table is rewritten around one stated doctrine: exit 0 is
audited and clean, exit 1 is audited with findings or a target present but
unreadable, exit 2 is that nothing was audited. Its old row 1 promised "one
generic matches-neither message" for three different inputs; there are three
distinct messages, and the missing-path case exited 1 until the preceding commit
fixed it. Rows are added for the preflight and CDPATH changes, because a table
claiming to enumerate every entry-point behaviour change reproduces its own
"an earlier revision of this ADR said they were behaviour-neutral" failure if it
omits any.

ADR-0025 also gains a Consequences supersession record in ADR-0016's form:
partially-superseded entries for 0008, 0014, 0020 and 0021, and explicit
"is not superseded" entries with reasoning for the rest. Twelve ADRs are amended
and it previously listed none.

ADR-0008 moves from an amendment note to partially superseded. Its contract
genuinely narrowed -- an agent .md outside an agents/ directory was audited
before the merge and is refused now -- and ADR-0020 already recorded that the
merge "reopens ADR-0008". Its detector description said "a path under
.apm/agents/", the phrasing ADR-0025 rejects as wider than the script and
circular; the shipped rule is a .md whose immediate parent is named agents/, at
any scope.

ADR-0020's amendment claimed the boundary resolver is sourced by
validate-provenance.sh. It is not, and never was; only validate.sh sources it,
once per mode branch. Three Home-column entries pointed at reference filenames
the merge renamed, one of which now resolves to two files because its row covers
skills and agents.

Five ADRs opened with "Skill renamed per ADR-0025", a form this repo does not
use, in the same commit that used the conventional "Amended by ADR-0025" twice.
They are normalized. "Renamed" was also wrong: the BREAKING-CHANGE trailer says
the skills were removed and their flows merged.

SIMPLIFICATION-AUDIT.md had 2026-09-15 notes attached to headlines that were
never updated, against its own convention of correcting in place with
strikethrough. Every figure here was re-derived at HEAD by command, and several
differed from the review's own numbers, so the notes record the basis rather
than the result alone.

LESSONS.md asserted the two review-time suite failures were the SIGPIPE race.
The commit that fixed that race explicitly declined to claim it -- the suite was
running while agents edited live config files -- so the hedge is restored.

Impact

No code, test or configuration change; documentation only. Suites stay 20/20
strict with 0 skipped and 374/374 bats. No gate parses ADR or gates.md content,
so nothing here is load-bearing for a hook.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
2026-09-16 08:54:21 +00:00
parent ffcbed6c41
commit 69119f4754
14 changed files with 284 additions and 77 deletions

View File

@@ -367,6 +367,16 @@ into the single `factory-audit/scripts/lib-boundary-resolver.sh`, sourced by tha
The two remaining copies must still stay byte-identical — a plugin script cannot source the root
one, which is the constraint that forces a copy to exist at all.
`tests/test-adr0020-contract.sh` pins that arrangement, and one of its assertions was green on a
defect it named. "`validate.sh` sources the resolver in **both mode branches**" was implemented as a
file-wide `grep -Ec … -ge 2`, which cannot see a branch at all: delete the `agent)` arm's source line
and duplicate the `skill)` arm's, and the file-wide count is still 2 and the assertion still passes,
with the agent path running no resolver or some other one. It is now a **per-arm structural check** —
each arm of `validate.sh`'s `case "$MODE" in` block must carry exactly one `source` line inside its
own body, and the file must carry exactly those two — with a mutation self-test that performs that
exact count-preserving edit on a copy and requires the check to fail on it. The suite went 25 → 28
cases.
### `python3` and PyYAML are hard requirements
Both, and neither is a best-effort accelerator.
@@ -426,6 +436,58 @@ the hook definitions filter on `SKILL.md`
"extend" that hook's `files:` pattern to cover agents** on the assumption that the script already
knows the difference; doing so silently enforces a gate ADR-0020 declines to set.
### The three exit tiers of `factory-audit`'s validators
`validate.sh` and `validate-provenance.sh` use the exit code to say **whether an audit happened**,
not only what it found. The skill's flow references tell the auditor to surface a non-zero run's
output verbatim as real findings, so the distinction is load-bearing rather than cosmetic:
| Exit | Means |
|---|---|
| **0** | audited, and clean |
| **1** | audited, and there are findings — or the target is present but cannot be opened |
| **2** | **nothing was audited.** The checks never ran |
Exit 2 covers: no arguments at all; a target that does not exist, with its own "does not exist"
message; a target matching neither the skill nor the agent shape; a missing or unreadable `lib-*.sh`
beside the entry script; and a missing `python3` or PyYAML. Every one of those used to exit 1, so
an install problem or a typo'd path arrived at the auditor as findings about the artifact. A missing
target was the sharpest case: only the directory branch stats the target, so a typo'd `*.agent.md`
path was classified on name alone, handed to `python3`, and came back as a FAIL about a file that was
never there.
**A dangling symlink or a symlink loop stays exit 1, deliberately.** Both are false to `-e` and true
to `-L`, and the existence guard tests both: something *is* at that path, it just cannot be opened,
and "exists but unreadable" is a real finding the check suite reports as a FAIL naming the file.
Catching them in the existence guard would replace that FAIL with a false "does not exist".
`--help` is exit 0 and now works with a library missing. `validate-provenance.sh` sourced both
provenance libraries at the top of the file, so a partial install turned the one command that
explains how to use the script into an exit 2; the libraries are now loaded only when a mode's half
of the usage is actually printed, and when one is gone that half says so instead of the whole command
failing. It also separates an **empty** positional from **no** positional — `validate-provenance.sh
""`, an unquoted variable that expanded to nothing, used to be reported as "only flags were given",
which sent the reader after a flag they never typed. Both are exit 2, with different messages.
**All three entry scripts are CDPATH-safe.** `cd` **prints** the directory whenever CDPATH supplied
it, so a bare `cd` on a path starting with neither `/` nor `.` can both emit an extra line into
whatever captures it *and* resolve to an unrelated tree. Every such site now does
`CDPATH='' cd -- …`: `SCRIPT_DIR` and the target's parent-directory name in `validate.sh` and
`validate-provenance.sh`, and **both** sites in `vale-wrap.sh` — its default `--config` resolution,
and the directory-mirror walk below. The two bare `cd`s left in `vale-wrap.sh` take absolute paths
(`mktemp -d` output, and the mirror root derived from it), which CDPATH is never consulted for.
The second of `vale-wrap.sh`'s two sites is not an asset lookup: its directory mirroring runs
`cd "$arg" && find -L . … -print0` on a *path argument*, which is relative whenever the caller passed
a relative one. With an exported CDPATH holding a same-named directory, the `cd` emitted the decoy's
path into the `-print0` stream and the mirror was built from the decoy's files. It is cleared the
same way now. Only the `cd` changed: the walk still mirrors the whole tree, prunes `.git`, and
follows symlinks with `find -L` because vale does.
**That one was latent, not a live gate defect.** Neither published Vale hook reaches it: both filter
on `files:` patterns that match single markdown files, and only the `-d "$arg"` branch mirrors a
directory. The exposed caller is the hand-invoked `vale-wrap.sh <dir>`.
## Current retrofit status
The ADR-0020 gates ship hot, with no baseline file — a shrinking baseline was considered and
@@ -614,6 +676,22 @@ existed.
every path it matches must be in that hook's own artifact class. A hook narrowed to zero files never
runs, and pre-commit reports no error.
**Case 34** asks, statically and with no Vale binary, whether the shipped `.vale.ini` could load a
style at all. Four assertions: every `[glob]` section declares a **non-empty** `BasedOnStyles`; every
style any section names resolves to a real directory under `StylesPath`; that `StylesPath` is **not
absolute**; and at least one `[glob]` section exists, so the check cannot pass vacuously on a config
with nothing in it. A section whose `BasedOnStyles` is empty is the silent case — Vale lints every
file that glob matches with no rule loaded, prints `0 errors` and exits 0. The absolute-path clause
is the one that is not obvious: an absolute `StylesPath` passes on the machine that wrote it and
hard-fails for every external consumer of `.pre-commit-hooks.yaml`, which is the only reason those
styles ship at all. Part B is a mutation self-test against the same function Part A calls — it empties
each section's `BasedOnStyles` in a copy of the assets, and absolutizes `StylesPath` in another
pointed at that copy's own real `styles/` directory, and requires each to fail by name.
**It is not case 0 again.** Case 0 reads `BasedOnStyles` lines file-wide and tolerates an absolute
`StylesPath`; case 34 reads them **per `[glob]` section** and rejects one. Case 0 checks the config
loads; case 34 checks that loading it arms anything.
### What Vale owns, and what stays LLM judgment
Six rule files, six distinct rules:
@@ -830,8 +908,8 @@ hook — is deleted with the second Vale copy (ADR-0025). The six glob probes su
`test-vale-wrap.sh` without Vale skips only its Vale-dependent cases, not the whole suite. The cases
that are plain greps and awk over the config and the two hook manifests still run: case 0, 16, 26,
27, the static halves of 28, 31 Parts A and B, 32 and 33. A static failure exits 1, because a real
defect is not a setup error. Only an all-static-pass run exits 77.
27, the static halves of 28, 31 Parts A and B, 32, 33 and 34. A static failure exits 1, because a
real defect is not a setup error. Only an all-static-pass run exits 77.
### Mentioning banned phrasing without tripping the rule
@@ -866,11 +944,47 @@ what to install. (It was three until `test-check-vale-style-sync.sh` was deleted
`echo "$OUT" | grep -q PATTERN`. Under `set -o pipefail` the pipe form fails depending on timing:
`grep -q` exits on its first match, `echo` takes SIGPIPE on its next write, and pipefail reports that
as the pipeline failing, so output that matched reads as "no match". It showed up as a push gate that
failed about once in 670 runs, on a different suite each time. `tests/test-no-pipefail-early-exit-grep.sh`
scans every tracked shell file that sets pipefail and fails on the pipe form. It does this for `echo`
or `printf` piped into `grep` with `-q`, `-m`, `-l`, `-L`, `--quiet` or `--silent`. It checks its own
scanner against fixtures before trusting a clean result. Pipes from other commands are out of scope.
In practice they either absorb the writer's exit status with `|| true` or write only once, at exit.
failed about once in 670 runs, on a different suite each time.
`tests/test-no-pipefail-early-exit-grep.sh` is the static guard. **What it scans** is the tracked
`*.sh`, `*.bats` and `*.bash` files, minus itself — `git ls-files -- '*.sh' '*.bats' '*.bash'` — and,
within each one that sets pipefail, it fails on the pipe form. Every tracked shell file in this tree
carries one of those three extensions today, so "every tracked shell file" is true in effect, but it
is a property of the tree and not of the scan: a shell script tracked under any other name is not
reached. It checks its own scanner against fixtures before trusting a clean result.
**What the pattern models**, after a review widened it on four axes and narrowed it on one:
- **The reader** is `grep`, `egrep` or `fgrep` — behind a path prefix (`/bin/grep`), a `command`
prefix, or env-var assignments (`LC_ALL=C grep`) — taking `-q`, `-m`, `-l`, `-L`, `--quiet`,
`--silent`, `--max-count` or `--files-with`. (The last two match the regex and were missing from
this list.)
- **Intermediate stages are seen.** `echo x | filter | grep -q y` is a site; only the two-stage form
used to be.
- **Both line continuations are joined** before matching: a trailing backslash, and a trailing `|`,
which is equally legal in a pipeline. The hit is reported at the line the command starts on.
- **`pipefail` may sit anywhere in a `set` line**, so `set -o errexit -o pipefail` arms the file. It
previously had to follow the *first* `-o`, and a file-level miss skips every site in that file
rather than one — which is why that test is deliberately loose.
- **A bare `&` ends a segment**, so `echo ok && other | grep -q x` — whose writer is `other`, not the
`echo` — is not a site. `&` followed by a digit is kept, so `2>&1` does not end one.
The widening turned up **five live vulnerable sites in `tests/test-apm-current-hook.sh`** that the
narrower scanner never saw while the suite reported green; all five are converted, and the scanner
now reports zero over the tree. Its fixtures went from 4 vulnerable spellings to **12**, plus
near-miss negatives it must leave alone, and the suite from 5 cases to **7**.
**Remit: early-exiting GREP readers only.** `head`, `sed -n 1p` and a bare `read` exit early too, and
an `echo` or `printf` feeding any of them is the same race. Those are guarded by **convention** —
absorb the writer's status with `|| true`, or take the verdict from a here-string — and deliberately
not by this test: most legitimate uses of them in this tree are already absorbed, and the scanner
cannot see absorption from the pipeline text alone, so flagging them would be noise. Two live
`grep … | head -1` sites (`tests/test-vale-wrap.sh:620` and `:1046`) were fixed by hand with that
idiom. Pipes from a non-builtin writer (`run_wrap … | grep -q`) are out of scope for the same reason:
in practice they either absorb the writer's exit status with `|| true` or write only once, at exit.
**Known limitation: heredoc bodies are scanned as code.** A `cat <<'EOF'` body containing a
vulnerable-looking line would be reported as a real site. There are none in the tree today.
`tests/run-bats.sh` derives the set of `.bats` files it expects from `git ls-files`, so a `.bats`
file deleted from the worktree but still tracked in the index fails the run rather than silently