7607522 fixed the symptom in the wrong place. It made `test-run-tests.sh`'s
`run_fake()` spawn fixtures via `env -u RUN_TESTS_STRICT`, which stops that one
suite inheriting strictness — and leaves every future suite to defend itself the
same way. The variable's only job is done the moment `run-tests.sh` latches it
into the `STRICT` shell local, so it is unset there now and the leak is gone for
every child. The `env -u` stays as this suite's own defence in depth rather than
as the fix.
Two corrections to that commit's account of the bug, both overstated and both
cheap to have checked:
- The blast radius was two assertions, cases 10c and 10g, not six. Nothing else
in the repo reads `RUN_TESTS_STRICT`.
- The pre-push gate was never red. It invokes `bash tests/run-tests.sh --strict`,
and the flag sets a shell local that is never exported, so the flag spelling
never leaked at all. Only the env-var spelling did.
That asymmetry between the two documented spellings is the real finding, and
nothing asserted against it. Case 10b compared the parent's verdict, which is the
half that already matched; the halves that differed were the environments the two
spellings handed every dispatched suite. New case 10i asks a child directly —
`${VAR+set}`, so an exported empty value still counts as a leak — and asserts the
two observations equal each other rather than a hardcoded expectation, so they
cannot drift apart in a direction the case did not anticipate.
Why:
The repo published apm packages but consumed them the old way — `claude plugin install
<name>@holocron`, six plugins enabled per project. Dogfooding stopped one layer short of the
install tooling kyberforge itself ships.
Implementation notes:
- Root apm.yml declares the six packages as dependencies.apm git+path objects against the
holocron remote. Object form over `<name>@holocron` aliases on purpose: an alias first needs
`apm marketplace add`, which writes to ~/.apm/marketplaces.json — user scope, absent on a fresh
clone. Unpinned against the default branch, matching the autoUpdate the native install had.
- apm.lock.yaml is committed; .claude/skills/, .claude/agents/ and apm_modules/ are gitignored
regenerable install output. Committing the deployed skills would add a third mirror of content
ADR-0017 already governs two copies of.
- .mcp.json is generated by apm from plugins/bin/.mcp.json, so the obsidian MCP server survives
the switch.
- .claude/settings.json is reduced to {"hooks": {}}. apm replays the install into a scratch tree
and diffs, so any repo-owned key there is permanent drift that fails apm-audit-ci. Nothing was
lost: enabledPlugins was empty after the uninstall and the only hooks entry was PreToolUse: [].
- tests/run-bats.sh and tests/run-tests.sh exclude apm_modules/. It holds a full copy of every
plugin, and a copied .bats file resolves its helpers against the dependency root rather than
this repo — 334 tests, 167 failures before the exclusion.
Impact:
Skills are now unnamespaced — `git-commits`, not `git:git-commits` — because apm deploys plain
project skills with no plugin to prefix. AGENTS.md, CONTEXT.md and docs/spec/architecture.md are
updated accordingly. Root apm.yml now declares dependencies, which arms apm-audit-ci's
lockfile-exists check for the root manifest. External consumers are unaffected: the marketplace
manifests are untouched and `apm pack --check-clean` stays clean. Project scope only.
ADR: 0018
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
Three gaps left by the previous round, all the same shape: a gate reporting
success having verified less than it appears to.
run_bats() hard-failed on a missing or non-executable runner but never checked
that the runner produced anything. An empty, executable run-bats.sh exits 0,
and the dispatcher printed a green summary with 166 bats tests silently absent.
It now requires an "N tests, M failures" line with a non-zero count.
run-tests.sh's skip listing is swallowed by pre-commit on a pass, so on a
machine without vale three suites exited 77 and the pre-push gate went green
having run 14 of 17. The hook now invokes it as --strict, where a skip fails
and the error names each suite and the reason it skipped. An ad-hoc local run
still skips gracefully -- at pre-push a skip means a documented dependency is
missing, which is a setup error, not a legitimate state. Deliberately not wired
to the vale downgrade's env var: one flag must not disarm two gates.
BATS_FILE_FLOOR is replaced by an expectation derived from git ls-files. A
floor of 8 against a real count of 10 let two files and eleven tests disappear
green, and the number needed an edit whenever a plugin was added. The derived
set needs no number, and catches an addition as well as a removal -- a .bats
file staged into the index and deleted from disk is now demanded back.
The vale opt-out announced its downgrade to nobody: pre-commit prints nothing
for a passing hook, so the summary line AGENTS.md tells the reader to check was
unreachable in exactly the situation it exists for. The hook is now verbose.
Also corrects the PROBES_CHECKED guard, whose commit message described a state
that cannot occur -- the .vale.ini loop errs first. Its two reachable triggers,
a gutted probe heredoc and a probe row naming a missing directory, had no test;
they do now, each asserting the guard is the sole cause.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
run_bats() was an if with no else, so a missing or non-executable run-bats.sh
made all 166 bats tests disappear with no diagnostic while the run printed a
green summary and exited 0. --bats-only became a total no-op. This is the same
defect the PR fixed one level down, left open in the dispatcher that pre-push
actually invokes -- and nothing tested run-tests.sh at all.
run-bats.sh's aggregation was asserted by nothing. Three separate mutations to
its failure-detection line all survived the existing suite, because real bats
emits both a nonzero exit and "not ok" lines, so each signal masked the other.
The new cases produce each signal without the other; all three mutants now die.
Also in this pass:
- zero discovered .bats files exited 0, so a widened path exclusion retired the
suite silently. Replaced with a file-count floor: a collapse to one or two
files is the same failure as a collapse to zero
- an existing-but-empty status file counted as a pass, because [[ "" -eq 0 ]]
is arithmetic-true and the || echo 1 fallback only covered a missing file.
The repro is deterministic: the stub truncates its own status file, then
kill -9s its parent so the real exit-code write never happens
Adds tests/test-run-tests.sh, 9 cases pinning the exit 0 / 1 / 77 three-way
split against a single summary line.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
`tests/run-tests.sh` declared `source=lib/batch-run.sh`, which resolves
to neither the repo root nor the script's own directory. A directive
that does not resolve is silent: it blinds test-vale-wrap.sh's
`sourced_files()` seeding exemption, and shellcheck's own SC1091 is
`info` while .pre-commit-config.yaml pins `--severity=warning`.
Issue #97 names run-bats.sh's `../scripts/lib/batch-run.sh` as the
correct spelling. It is not. Directives resolve against the source-path,
which under pre-commit is the repo root, so `../scripts/...` escapes the
repo and trips SC1091 exactly as `lib/...` does -- verified directly.
The spelling satisfying both shellcheck and `sourced_files()`'s
two-candidate rule is repo-root-relative, matching scripts/install.sh.
Fixes all three: run-tests.sh, run-bats.sh, and check-manifests.sh,
the last unmentioned by the issue. Every directive in the repo now
resolves, which the previous commit's case 27 asserts.
Refs #97
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
--check's throwaway pack copy seeded .claude-plugin/plugin.json and
.github/plugin/plugin.json from the real plugin dir, then packed without
--force -- apm pack silently skips regenerating a plugin.json that already
exists, so the diff always compared the copy against itself and never caught
drift in the compiled name/version/description/mcpServers. --force is now
always passed; in check mode it forces regeneration inside the throwaway copy
only, which sync_plugin_manifest() then diffs against the real committed
manifest.
sync_hooks_json() returned early whenever .apm/hooks/ was missing, without
checking whether a stale hooks.json was still sitting at the plugin root from
a prior sync -- unlike sync_dir(), which already detects that kind of orphaned
mirrored output. It now mirrors sync_dir()'s shape: flagged as drift in
--check, removed on a real sync.
Running the corrected --check --all against this repo's own plugins surfaced
3 real orphans: plugins/{git,gitea,core}/hooks.json, empty stubs added in
4edaaac only to satisfy an old plugin.json pointer-field check that no longer
exists (their compiled plugin.json has never had a hooks field, and none of
the three ever had .apm/hooks/). Removed as part of this fix since they're
exactly the drift the corrected check now catches -- leaving them would break
the sync-plugin-content pre-push gate on this branch.
Also extracts two shared helpers into scripts/lib/, sourced by this script and
others so a future bug fix doesn't need hand-applying three times:
- marketplace-plugins.sh: walks marketplace.json for local plugin dirs (this
script's --all branch and check-manifests.sh had near-identical copies)
- batch-run.sh: the bounded-batch concurrent job runner (this script,
tests/run-tests.sh, and tests/run-bats.sh each hand-rolled the same
core-count-capped wait loop independently)
Extended tests/test-sync-plugin-content.sh with coverage for both drift cases
(plugin.json version-bump drift, orphaned-hooks.json drift), including that a
re-sync clears each.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
Both runners spawned one process at a time and let it finish before starting
the next, so wall time scaled with file count even though each file's own
work (bats: mostly repeated apm/git subprocess startup; the vale-heavy
shell scripts: repeated vale binary startup) rarely used a full core.
run-bats.sh now backgrounds one `bats` invocation per file, bounded by core
count, buffering each file's TAP output so concurrent streams can't
interleave, then flushes in stable sorted order once every job finishes.
run-tests.sh does the same for test-*.sh in fixed-size batches (plain `wait`,
not `wait -n`, to stay on the bash-3.2-safe path test-vale-wrap.sh already
enforces for this file) -- verified beforehand that every test-*.sh keeps its
fixtures under its own mktemp dir rather than mutating the live repo tree, so
running them concurrently is safe.
Confirmed correct on both the pass and fail paths (a deliberately failing
bats test and a scratch TEST_DIR with pass/fail/skip-77 scripts) before
measuring: full bats suite 26-30s -> 7.5s, tests/run-tests.sh 68s -> ~45-50s,
full pre-push gate 2m40s -> 1m12s.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7GvKuJfy2WrdBmUttV4DT
MAX_WORDS=2900 was calibrated to the corpus median density and carried no
margin: at the densest observed 7.22 chars/word (~1.81 tokens/word) it permits
~5,240 tokens against the 5,000 it proxies for. 2770 holds the worst observed
density under the ceiling. The largest SKILL.md is 2,489 words, so the change
costs nothing today — 281 words of margin — and the header comment now argues
the new calibration rather than swapping the digits.
Both enforcement points move together, and a new test asserts they agree, since
a SKILL.md passing its own audit while the commit hook blocks it is the
disagreement this pair exists to prevent.
CONTEXT.md is deliberately left ungated: it is 2,816 words, and gating it would
block the build. Recorded here so the omission reads as a decision rather than
an oversight.
skill-audit's manual-fallback path listed only the line ceiling, so an agent
taking that path passed an oversized SKILL.md the hook then rejected. The word
ceiling is now named alongside it. agent-audit is deliberately unchanged: the
size hook scopes to SKILL.md only and agent-audit's validate.sh has no word
gate, so claiming it there would be false.
The Vale research doc still showed the MDX {/* vale off */} form under a
Markdown heading, contradicting CONTEXT.md and vale-run's troubleshooting
reference — that form suppresses nothing in plain .md. Fixed in both places it
appeared.
tests/run-tests.sh used mapfile (bash 4.0+) with unguarded array expansion,
though AGENTS.md tells contributors to run it and macOS ships bash 3.2. It now
collects via a while-read loop over process substitution and guards every
expansion. The newline-delimited find|sort pipeline is kept rather than -print0
with sort -z, whose BSD portability is the weaker link, and which matches
mapfile -t's previous behaviour exactly.
Refs: #85
ADR: 0013
pre-commit prefixes only entry[0] with the hook-repo clone path
(cmd = (prefix.path(cmd[0]), *cmd[1:])), so the --config argument in
.pre-commit-hooks.yaml resolved against the *consuming* repo's root
and hard-failed every external run with E100. Two of the three hooks
ADR-0014 promises were unusable.
vale-wrap.sh now self-locates its config from BASH_SOURCE when no
--config is supplied; an explicit --config still wins in all three
argv forms and stays cwd-relative. Both manifests drop the argument
and are kept byte-identical: the local repo: local config resolved
--config correctly only because the consuming repo *was* this repo,
and that divergence is why three review rounds missed the defect.
Also in the wrapper:
- replace GNU-only `realpath -m` with a portable abspath helper; -m is
load-bearing (dest does not exist yet), so BSD realpath aborted the
script under set -e on macOS
- walk directory arguments instead of passing them through unflattened,
which reported a clean 0-error run for files that fail when named
explicitly
- read/write with errors='surrogateescape' so one non-UTF-8 .md under a
directory argument cannot abort the hook
New test-vale-hooks-consumer.sh builds the hook repo from the working
tree and points a file:// consumer at it, covering the manifest as a
hook repo for the first time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MCQ648fLSFXPHGZdQ8gn58
- Add tests/run-tests.sh: discovers and runs all test-*.sh (including
plugin subdirs) and the bats suite; replaces per-script pre-push calls
- Add tests/run-tests.bats: TDD coverage for run-tests.sh behaviours
- Update setup-hooks.sh: pre-push block now calls run-tests.sh
- Fix test-install.sh: remove provider adapter symlink tests (adapter
removed in marketplace migration), guard skills loops on dir existence
- Fix test-instructions-and-docs.sh: content index checks now point to
core/AGENTS.md (where it lives), remove ard/bug dir assertions
- Fix test-setup-hooks.sh: assert pre-push hook calls run-tests.sh
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
