docs: reconcile the ADRs, gates and audit log with the shipped behaviour
Why A six-agent review of the two preceding commits found their code sound -- the differential claim holds, the published hook contract is byte-unchanged -- but their prose drifted from it in three ways: statements of fact the code contradicts, markers in a convention this repo does not use, and figures that went stale when the merge changed what they counted. Implementation Notes ADR-0025's edge-path table is rewritten around one stated doctrine: exit 0 is audited and clean, exit 1 is audited with findings or a target present but unreadable, exit 2 is that nothing was audited. Its old row 1 promised "one generic matches-neither message" for three different inputs; there are three distinct messages, and the missing-path case exited 1 until the preceding commit fixed it. Rows are added for the preflight and CDPATH changes, because a table claiming to enumerate every entry-point behaviour change reproduces its own "an earlier revision of this ADR said they were behaviour-neutral" failure if it omits any. ADR-0025 also gains a Consequences supersession record in ADR-0016's form: partially-superseded entries for 0008, 0014, 0020 and 0021, and explicit "is not superseded" entries with reasoning for the rest. Twelve ADRs are amended and it previously listed none. ADR-0008 moves from an amendment note to partially superseded. Its contract genuinely narrowed -- an agent .md outside an agents/ directory was audited before the merge and is refused now -- and ADR-0020 already recorded that the merge "reopens ADR-0008". Its detector description said "a path under .apm/agents/", the phrasing ADR-0025 rejects as wider than the script and circular; the shipped rule is a .md whose immediate parent is named agents/, at any scope. ADR-0020's amendment claimed the boundary resolver is sourced by validate-provenance.sh. It is not, and never was; only validate.sh sources it, once per mode branch. Three Home-column entries pointed at reference filenames the merge renamed, one of which now resolves to two files because its row covers skills and agents. Five ADRs opened with "Skill renamed per ADR-0025", a form this repo does not use, in the same commit that used the conventional "Amended by ADR-0025" twice. They are normalized. "Renamed" was also wrong: the BREAKING-CHANGE trailer says the skills were removed and their flows merged. SIMPLIFICATION-AUDIT.md had 2026-09-15 notes attached to headlines that were never updated, against its own convention of correcting in place with strikethrough. Every figure here was re-derived at HEAD by command, and several differed from the review's own numbers, so the notes record the basis rather than the result alone. LESSONS.md asserted the two review-time suite failures were the SIGPIPE race. The commit that fixed that race explicitly declined to claim it -- the suite was running while agents edited live config files -- so the hedge is restored. Impact No code, test or configuration change; documentation only. Suites stay 20/20 strict with 0 skipped and 374/374 bats. No gate parses ADR or gates.md content, so nothing here is load-bearing for a hook. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
@@ -1,9 +1,9 @@
|
||||
# Add INFO as a third finding level in skill-audit reports
|
||||
|
||||
**Skill renamed per ADR-0025 (2026-09-15):** `skill-audit` and `agent-audit` merged into
|
||||
`factory-audit`, which dispatches to a skill flow and an agent flow at Step 0. Read `skill-audit`
|
||||
below as `factory-audit`'s skill flow. The decision itself is unchanged — ADR-0025 carried every
|
||||
audit criterion, tier and finding level across as-is.
|
||||
**Amended by ADR-0025 (2026-09-15).** `skill-audit` and `agent-audit` were removed and their flows
|
||||
merged into `factory-audit`, which dispatches to a skill flow and an agent flow at Step 0. Read
|
||||
`skill-audit` below as `factory-audit`'s skill flow. The decision itself is unchanged — ADR-0025
|
||||
carried every audit criterion, tier and finding level across as-is.
|
||||
|
||||
`skill-audit` shipped with two finding levels: FAIL (blocks shipping) and
|
||||
SUGGESTION (optional improvement). Provenance validation introduced observations
|
||||
|
||||
@@ -1,10 +1,17 @@
|
||||
# agent-audit takes a single file path and derives the counterpart by scope detection
|
||||
|
||||
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
|
||||
**Partially superseded by ADR-0025 (2026-09-15).** `agent-audit` merged with `skill-audit` into
|
||||
`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow. The single-file
|
||||
invocation contract this ADR sets survives the merge intact — `factory-audit` dispatches at Step 0
|
||||
on the target path, and an `*.agent.md` or a path under `.apm/agents/` takes the agent flow, so the
|
||||
caller still names one file and the script still derives the rest.
|
||||
invocation contract this ADR sets survives — `factory-audit` dispatches at Step 0 on the target
|
||||
path, so the caller still names one file and the script still derives the rest — but the set of
|
||||
paths that contract accepts is **narrower**. The agent flow is taken by an `*.agent.md` file, or by
|
||||
a `.md` file whose *immediate* parent directory is named `agents/`, at any scope:
|
||||
`.claude/agents/x.md` is accepted, `.apm/agents/sub/x.md` is refused. "A path under `.apm/agents/`"
|
||||
is not the rule — ADR-0025 rejects that phrasing as both wider than the script and circular. An
|
||||
agent `.md` outside an `agents/` directory was audited before the merge and now stops at exit 2,
|
||||
the never-ran tier, because a single auto-detecting entry point classifies on the path alone and
|
||||
detection never guesses.
|
||||
ADR-0020 recorded the merge as reopening this ADR; that is the clause it reopened.
|
||||
|
||||
`agent-audit` validates agent definition file pairs (Claude Code `.md` + Copilot `.agent.md`). The skill accepts a path to either file and derives the counterpart using scope detection rather than requiring the caller to name both files or supply a root directory.
|
||||
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
# agent-audit reads field lists from a reference file, not hardcoded script arrays
|
||||
|
||||
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
|
||||
`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow; the reference file this
|
||||
ADR is about is now `factory-audit/references/agent-field-inventory.md`. The decision is unchanged —
|
||||
the field lists still live in a reference file read at runtime, not in script arrays.
|
||||
**Amended by ADR-0025 (2026-09-15).** `agent-audit` was removed and its flow merged with
|
||||
`skill-audit`'s into `factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow; the
|
||||
reference file this ADR is about is now `factory-audit/references/agent-field-inventory.md`. The
|
||||
decision is unchanged — the field lists still live in a reference file read at runtime, not in
|
||||
script arrays.
|
||||
|
||||
`agent-audit`'s `validate.sh` checks for Claude Code-only fields in Copilot files and
|
||||
silently-ignored fields in plugin agents. Rather than hardcoding those field lists in the
|
||||
|
||||
@@ -15,9 +15,10 @@ ADR's own conclusion is unaffected by that move: the provenance file still belon
|
||||
auto-scans, and `.apm/agents/` is, if anything, further removed from plugin-root than the old
|
||||
flat `agents/` directory was, so the reasoning below still holds. References below to
|
||||
`<plugin-root>/agents/` describe the pre-APM layout in effect when this decision was made.
|
||||
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
|
||||
`factory-audit`; read the `agent-audit` references below as `factory-audit`'s agent flow, whose
|
||||
`validate-provenance.sh` still resolves `<plugin-root>/sources.md` exactly as this ADR decided.
|
||||
**Amended by ADR-0025 (2026-09-15).** `agent-audit` was removed and its flow merged with
|
||||
`skill-audit`'s into `factory-audit`; read the `agent-audit` references below as `factory-audit`'s
|
||||
agent flow, whose `validate-provenance.sh` still resolves `<plugin-root>/sources.md` exactly as this
|
||||
ADR decided.
|
||||
**Scope boundary (per ADR-0016):** this path change is plugin scope only. Project scope
|
||||
(`.claude/agents/` + `.github/agents/`) and user scope (`~/.claude/agents/` +
|
||||
`~/.copilot/agents/`) are unaffected — they are not APM packages and keep the dual-file
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
Three skills in the `core` plugin (`core`'s first active skills):
|
||||
|
||||
- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern (**skill renamed per ADR-0025, 2026-09-15:** `skill-audit` is now `factory-audit`'s skill flow; the author-then-audit pattern is unchanged). When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition.
|
||||
- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern (**Amended by ADR-0025, 2026-09-15:** `skill-audit` was removed and its flow is now `factory-audit`'s skill flow; the author-then-audit pattern is unchanged). When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition.
|
||||
- **`agentsmd-audit`** — a single combined pass checking three mandatory baselines against `AGENTS.md` only: secrets/credentials (governance.md hard prohibition), structural completeness (common-sections checklist from the agents.md spec), and accuracy/drift (do referenced commands/paths resolve against the repo). Never inspects provider adapter files.
|
||||
- **`provider-adapter-author`** — detects and converts a provider-specific instruction file into a thin adapter that imports `AGENTS.md` (mirroring this repo's own two-tier `CLAUDE.md` pattern). Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`) rather than a separate paired audit skill, since the check (import present, no duplicated headings, size threshold) is mechanical.
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
# Vale audit prefilter expands into a plugin-content harness, scoped to prose-pattern rules only
|
||||
|
||||
Issue #84 wired Vale as a deterministic prefilter for `skill-audit`/`agent-audit`, scoped to
|
||||
Issue #84 wired Vale as a deterministic prefilter for `skill-audit`/`agent-audit` (merged into
|
||||
`factory-audit`'s two flows by ADR-0025; read every mention of the pair below that way), scoped to
|
||||
exactly four pattern-matchable checks (imperative description opener, vague capability wording,
|
||||
generic reference-pointer padding, Copilot's dead `Use proactively` phrasing), documented only in
|
||||
CONTEXT.md's "Vale audit prefilter" section — never its own ADR — and explicitly excluding body
|
||||
@@ -41,7 +42,8 @@ already handled elsewhere (gitleaks) or genuinely out of scope for a plugin-cont
|
||||
and `github-copilot-plugins/agent-definition.md`, found that the existing four Kyberforge rules
|
||||
already cover the pattern-matchable surface those specs describe. The remaining spec guidance —
|
||||
calibrating control vs. giving freedom, avoiding menus of options, coherent skill scope, moderate
|
||||
detail level — is semantic judgment, already `skill-audit`'s job via LLM review, not new lintable
|
||||
detail level — is semantic judgment, already `skill-audit`'s job via LLM review (now
|
||||
`factory-audit`'s skill flow, see ADR-0025), not new lintable
|
||||
rules. One confirmation surfaced: Claude Code's `Use proactively` phrasing is meaningful for `.md`
|
||||
agent files (it triggers auto-invocation), unlike Copilot's `.agent.md` files where it's dead
|
||||
phrasing — so `KyberforgeCopilot/ProactivePhrase`'s existing `.agent.md`-only scope is correct and
|
||||
|
||||
@@ -164,6 +164,7 @@ doesn't wonder if it was overlooked.
|
||||
avoids for this repo's own dev-time gate. A tag not existing at all is also a hard fail on
|
||||
`main`, covering the very first release. This is deterministic tooling, not a standing
|
||||
instruction to remember — consistent with `check-manifests.sh`/`check-vale-style-sync.sh`
|
||||
(the latter deleted by ADR-0025, see the amendment at the top of this file)
|
||||
already using the same pre-push, main-agnostic-elsewhere pattern.
|
||||
- **Known limitation, not yet closed:** `check-release-needed.sh` only fires when a human runs
|
||||
`git push` locally with pre-commit's hooks installed — `PRE_COMMIT_REMOTE_BRANCH` is set by
|
||||
|
||||
@@ -164,7 +164,9 @@ whether a field is safe under verbatim copy in a single vendor-neutral file.
|
||||
no longer applies at plugin scope; `agent-audit` takes the single file directly there. Project
|
||||
and user scope, where a real pair still exists, are unaffected.
|
||||
- **ADR-0009 is not superseded.** The mechanism it established — `agent-audit` reading field
|
||||
lists from `references/field-inventory.md` rather than hardcoding them, with a `source_keys`
|
||||
lists from `references/field-inventory.md` (now
|
||||
`factory-audit/references/agent-field-inventory.md`, see ADR-0025) rather than hardcoding them,
|
||||
with a `source_keys`
|
||||
provenance chain — survives and is reused. Only the *content shape* changes for plugin scope:
|
||||
`field-inventory.md` shifts from two side-by-side CC-only/Copilot-only blocklists to one
|
||||
vendor-neutral allowlist for plugin-scope agents, while continuing to serve its original
|
||||
|
||||
@@ -65,9 +65,10 @@ Three sub-decisions inside that:
|
||||
## Consequences
|
||||
|
||||
**Skills gain an unnamespaced name.** apm deploys plain project skills, so `git:git-commits` also
|
||||
answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit` (**skill renamed per ADR-0025,
|
||||
2026-09-15:** that skill is now `factory-audit`, so the live example is `kyberforge:factory-audit` to
|
||||
`factory-audit`; the rule is unchanged). This is not configurable —
|
||||
answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit` (**Amended by ADR-0025,
|
||||
2026-09-15:** `skill-audit` was removed and its flow merged into `factory-audit`, so the live
|
||||
example is `kyberforge:factory-audit` to `factory-audit`; the rule is unchanged). This is not
|
||||
configurable —
|
||||
a project skill has no plugin to prefix. `AGENTS.md` and `CONTEXT.md` are updated to name the bare
|
||||
form, which is what apm deploys and the only form a repo consuming holocron through apm gets.
|
||||
|
||||
|
||||
@@ -17,7 +17,9 @@ three: `scripts/skill-size-check.sh`, `skill-audit/scripts/validate.sh` and
|
||||
`scripts/skill-size-check.sh`, which still embeds the 1,061-line block between `BEGIN`/`END ADR-0020
|
||||
SHARED BOUNDARY RESOLVER` markers, and one plugin copy — extracted out of the merged validator into
|
||||
`plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh` and sourced by
|
||||
`factory-audit`'s `validate.sh` and `validate-provenance.sh` rather than pasted into each. The
|
||||
`factory-audit`'s `validate.sh`, once in each of its two mode branches, rather than pasted into
|
||||
both. `validate-provenance.sh` is not a third reader: it sources `lib-contributing-files.sh` and one
|
||||
of `lib-provenance-skill.sh`/`lib-provenance-agent.sh`, and never touches the resolver at all. The
|
||||
Enforcement table's "constants mirrored in `skill-audit/scripts/validate.sh` and
|
||||
`agent-audit/scripts/validate.sh`" is one path now, `factory-audit/scripts/validate.sh`, which
|
||||
auto-detects the artifact type; the skills/agents columns are unaffected, since the merged validator
|
||||
@@ -275,7 +277,7 @@ which tier each rule is in, because the failure this ADR is most exposed to is a
|
||||
| Check | Applies to | Tier | Home |
|
||||
|---|---|---|---|
|
||||
| description characters (250 SUGGESTION † / 400 FAIL) | skills, agents | deterministic | `scripts/skill-size-check.sh`; constants mirrored in `skill-audit/scripts/validate.sh` and `agent-audit/scripts/validate.sh` |
|
||||
| body-only words (600 SUGGESTION / 900 FAIL) | skills | deterministic | `skill-size-check.sh`, `skill-audit/scripts/validate.sh` |
|
||||
| body-only words (600 SUGGESTION / 900 FAIL) | skills | deterministic | `skill-size-check.sh`, `skill-audit/scripts/validate.sh` (now `factory-audit/scripts/validate.sh`, see ADR-0025) |
|
||||
| description present and non-empty (ERROR) | skills, agents | deterministic | same |
|
||||
| boundary target resolves to a real skill or agent — **three** verdicts, not two (ERROR when written in route notation — `/name`, or any arrow form; or when a *terminal* bare name's own sentence names another target that resolves. SUGGESTION otherwise. INFO, "DID NOT RUN", exit 0, when no skill universe could be determined for the path at all — no authoring root above it, no apm package root, no declared apm dependencies, no deployed `.claude/` or `.agents/` tree: the targets are named and left unchecked) | skills, agents | deterministic | same |
|
||||
| boundary clause absent — `absent` (SUGGESTION) † | skills, agents | deterministic | same |
|
||||
@@ -285,10 +287,10 @@ which tier each rule is in, because the failure this ADR is most exposed to is a
|
||||
| Gotchas over 25% of the body (SUGGESTION) | skills | deterministic | same |
|
||||
| every `references/<file>.md` a body names exists (ERROR) | skills | deterministic | same |
|
||||
| description opener, composition notes in a description | skills, agents | prose pattern | `plugins/kyberforge/.apm/skills/*/assets/vale/styles/Kyberforge/` |
|
||||
| a Gotcha paraphrasing a body step | skills | **auditor judgment** | `references/body-discipline.md` |
|
||||
| dispatch at two or more mutually exclusive flows | skills | **auditor judgment** | `references/body-discipline.md` |
|
||||
| a Gotcha paraphrasing a body step | skills | **auditor judgment** | `references/body-discipline.md` (now `references/skill-body-discipline.md`, see ADR-0025) |
|
||||
| dispatch at two or more mutually exclusive flows | skills | **auditor judgment** | `references/body-discipline.md` (now `references/skill-body-discipline.md`, see ADR-0025) |
|
||||
| delegation: an agent body restating a skill's procedure | agents | **auditor judgment** | `agent-audit` (now `factory-audit`'s agent flow, see ADR-0025) |
|
||||
| capability enumeration, restatement, trigger quality | skills, agents | **auditor judgment** | `references/description-quality.md` |
|
||||
| capability enumeration, restatement, trigger quality | skills, agents | **auditor judgment** | `references/description-quality.md` (now two files, `references/skill-description-quality.md` and `references/agent-description-quality.md`, see ADR-0025 — this row applies to both artifact types, and the merge splits every flow-specific reference by flow) |
|
||||
|
||||
The rows in bold are stated as FAILs in the Decision above and are FAILs an *auditor* issues. None of
|
||||
them is countable: "does this Gotcha paraphrase step 4", "are these two flows mutually exclusive" and
|
||||
@@ -524,7 +526,8 @@ Upstream citations below are relative to
|
||||
growth. Would have made the retrofit a visible burn-down instead of a wall. Rejected in favour of
|
||||
hot gates.
|
||||
- **A sync gate over the duplicated spans** instead of a merge rule — generalising
|
||||
`scripts/check-vale-style-sync.sh` to cover shared prose so duplication persists but drift cannot.
|
||||
`scripts/check-vale-style-sync.sh` (live when this was written; ADR-0025 deleted it along with the
|
||||
second copy it diffed) to cover shared prose so duplication persists but drift cannot.
|
||||
Rejected for the audit pair in favour of merging, which removes the duplication rather than
|
||||
policing it, and removes a mutually-excluding near-miss pair from the router at the same time. It
|
||||
remains the only available answer for the author pair.
|
||||
|
||||
@@ -142,7 +142,10 @@ exactly the shapes `scripts/validate.sh` detects:
|
||||
the skill flow.
|
||||
- A `*.agent.md` file, or a `.md` file whose *immediate* parent directory is `agents/`, takes the
|
||||
agent flow.
|
||||
- Anything else stops, runs no validator, and names the two accepted shapes.
|
||||
- Anything else stops, runs no validator, and names the two accepted shapes. `validate.sh`
|
||||
distinguishes **three** cases here and says which one applies — the path does not exist, it is a
|
||||
directory with no `SKILL.md`, or it matches neither shape — so a typo is never reported as a
|
||||
spec violation.
|
||||
|
||||
Putting the dispatch after any deterministic check would mean running the wrong validator first and
|
||||
reading its output as a finding. An earlier revision of the body carried a two-row table with no
|
||||
@@ -158,9 +161,12 @@ script and circular.
|
||||
one of `scripts/lib-provenance-skill.sh` / `scripts/lib-provenance-agent.sh`.
|
||||
|
||||
Two things justify this shape. First, **self-containment binds between skills, not within one.** The
|
||||
resolver had to be embedded verbatim in three copies because three skill directories cannot read
|
||||
each other's files; two files inside one skill directory have no such problem. Sourcing is available
|
||||
the moment the directory boundary between them disappears. Second, **a single auto-detecting entry
|
||||
resolver had to be embedded verbatim in three copies because its three homes could not read each
|
||||
other's files: two of them were the separate `skill-audit` and `agent-audit` directories, and the
|
||||
third is `scripts/skill-size-check.sh` at the repo root, which is consumed through
|
||||
`.pre-commit-hooks.yaml` and so can reach into no plugin path at all (point 6 below). Two files
|
||||
inside one skill directory have no such problem. Sourcing is available the moment the directory
|
||||
boundary between them disappears. Second, **a single auto-detecting entry
|
||||
point makes a Step 0 misdispatch detectable.** The script re-detects the flow from the target, so
|
||||
even after a misdispatch it runs the right checks and its finding tiers are correct. That alone does
|
||||
not make the misdispatch self-correcting, and an earlier revision of this ADR wrongly said it did.
|
||||
@@ -218,17 +224,31 @@ directory and agent file matched stdout, stderr and exit code.
|
||||
|
||||
**The entry points are not behaviour-neutral, and an earlier revision of this ADR said they were.**
|
||||
Those differential runs used valid targets only, so they could not see that the new detection layer
|
||||
changed what happens to *invalid* ones. Every change below is deliberate:
|
||||
changed what happens to *invalid* ones.
|
||||
|
||||
**The exit tiers are what every row below is measured against**, so state them once: **0** is
|
||||
audited and clean, **1** is audited and has findings — a target that is *present but unreadable*
|
||||
counts here — and **2** is that **nothing was audited**. Exit 2 is the never-ran tier, so the flow
|
||||
files report the section as unverified and quote the reason instead of reading the run as a verdict.
|
||||
The pre-merge scripts did not hold that line: several never-ran conditions exited 1, which puts a
|
||||
target that was never opened into the findings tier. The doctrine is now applied uniformly across
|
||||
both entry points, and that is what most of the table is. Every change below is deliberate:
|
||||
|
||||
| Input | Pre-merge | Now |
|
||||
|---|---|---|
|
||||
| a missing path, a directory with no `SKILL.md`, a non-agent `.md` (e.g. `README.md`) | exit 1, or a mode-specific exit-2 message | **exit 2** with one generic "matches neither" Error/Why/Fix. Exit 2 is the never-ran tier, so the flow files report the section as unverified and quote the reason. |
|
||||
| a missing path, of any shape | exit 1 — the `*.agent.md`, `agents/`-parent and `SKILL.md` branches classify on the *name*, so a typo'd path went to the validator and its absence came back as a FAIL | **exit 2**, with its own dedicated "does not exist" Error/Why/Fix. The test runs on the typed path *before* the `SKILL.md` → parent-directory rewrite, so a missing `docs/SKILL.md` is not silently tested as `docs`. |
|
||||
| a dangling symlink, or a symlink loop | exit 1 | **unchanged: exit 1**, FAIL "could not be read". The guard in the row above deliberately stops short of these — `-L` rescues what `-e` rejects, because something *is* at that path and merely cannot be opened. Present-but-broken is a real finding, and reclassifying it as "does not exist" would send the reader after a typo instead of after the link. |
|
||||
| a directory with no `SKILL.md` | a mode-specific exit-2 message | **exit 2**, with its own distinct message — a skill directory is identified by its `SKILL.md`, and an agent target is a file, never a directory. |
|
||||
| anything else that matches neither shape (e.g. `README.md`) | exit 1, or a mode-specific exit-2 message | **exit 2**, the generic "matches neither a skill directory nor an agent file" Error/Why/Fix. |
|
||||
| a `SKILL.md` file path | exit 1 or 2 (`…/SKILL.md/SKILL.md not found`, "not a directory") | **accepted**; its parent directory is audited |
|
||||
| an agent `.md` *not* under an `agents/` directory (e.g. `~/drafts/my-agent.md`) | audited | **refused, exit 2**. Detection never guesses. No tracked file in this repo is affected. |
|
||||
| an agent `.md` whose *immediate* parent directory is not `agents/` (e.g. `~/drafts/my-agent.md`, or `.apm/agents/sub/x.md`) | audited | **refused, exit 2**. Detection never guesses. No tracked file in this repo is affected. |
|
||||
| a bare or `./`-relative agent filename, run from inside its `agents/` directory | audited | audited. The parent directory's name is read from the real path, not the typed string. |
|
||||
| a `lib-*.sh` missing or unreadable, or the script directory unresolvable | did not apply (single file) | **exit 2** with Error/Why/Fix, never a raw bash error at exit 1, which is the real-findings tier |
|
||||
| `CDPATH` exported | did not apply (no `cd`) | no effect. `SCRIPT_DIR` resolves with `CDPATH=''` and `cd -- … >/dev/null`. |
|
||||
| no argument | `Error: skill-dir is required.` / `agent-file is required.` | one combined message and usage block; exit code unchanged (1 from `validate.sh`, 2 from `validate-provenance.sh`) |
|
||||
| `python3` missing, or PyYAML not importable | exit 1 | **exit 2** with Error/Why/Fix, and the interpreter is checked separately from the library so the message names the right one. No interpreter means no check ran, which is the never-ran tier and not a finding about the target. `validate.sh`'s preflight is mode-specific and runs after detection, so it names the gates the *selected* flow would skip; `validate-provenance.sh` needs `python3` but not PyYAML, and already exited 2 for it. |
|
||||
| `CDPATH` exported | `validate.sh`/`validate-provenance.sh`: did not apply (no `cd`). `vale-wrap.sh`: **broken** — a `CDPATH` entry with a `scripts/` child made `cd` print the directory it resolved, so `--config` was handed a two-line argument and vale died | no effect on any of the three entry scripts. Every `cd` now runs as `CDPATH='' cd -- … >/dev/null`. This is a fix in the wrapper, not a no-op: an earlier revision of this ADR implied all three were already safe. |
|
||||
| no argument | `Error: skill-dir is required.` / `agent-file is required.`, at exit 1 from `validate.sh` and exit 2 from `validate-provenance.sh` | one combined message and usage block, and **exit 2 from both**. Nothing was audited, so the never-ran tier is the correct one; `validate.sh`'s exit 1 was the outlier and is corrected rather than carried across. |
|
||||
| `--help` | exit 0 | exit 0, and from `validate-provenance.sh` it now works **even with a library missing**: the help text needs no library, so failing the preflight first made `--help` unusable exactly when a reader most needed the usage block. |
|
||||
| an empty-string positional (`validate-provenance.sh ""`) | reported as "only flags were given" | reported accurately — an empty argument is an argument, and misnaming it sent the reader to the wrong fix. |
|
||||
|
||||
A single `validate.sh` copied or symlinked out of its `scripts/` directory still does not work,
|
||||
because its libraries are not beside it. It now fails at exit 2 and says so.
|
||||
@@ -237,10 +257,12 @@ because its libraries are not beside it. It now fails at exit 2 and says so.
|
||||
|
||||
**Keep two skills and rely on the byte-identity contract test alone (rejected).** This is the status
|
||||
quo: `tests/test-adr0020-contract.sh` already hashes the resolver across copies, and
|
||||
`check-vale-style-sync.sh` already diffs the Vale halves at pre-push. Only 6 of its 17 assertion
|
||||
sites exist because there are two copies. The other 11 do other work, and are rehomed or ported above rather than being an
|
||||
argument for the status quo. On the duplication itself it polices drift rather than removing the thing that drifts,
|
||||
and it pays 2,934 lines plus a pre-push hook to do so. It also leaves
|
||||
`check-vale-style-sync.sh` already diffs the Vale halves at pre-push. Only **2** of its 17
|
||||
assertion sites actually diff the two copies, and four more exist solely so the script can locate
|
||||
them — a real `REPO_ROOT`, non-stale `.apm/` paths, both copies present. The other 11 do other
|
||||
work, and are rehomed or ported above rather than being an argument for the status quo. On the
|
||||
duplication itself it polices drift rather than removing the thing that drifts, and it pays 2,934
|
||||
lines plus a pre-push hook to do so. It also leaves
|
||||
the router carrying a mutually-excluding near-miss pair whose two descriptions each spend a boundary
|
||||
clause pointing at the other — a routing cost the merge removes for free. ADR-0020 already weighed
|
||||
this option for this pair and chose merging; nothing measured since changes the balance.
|
||||
@@ -259,8 +281,10 @@ today, and folding each into one file with a skill section and an agent section
|
||||
outcome. It defeats the dispatch. The entire point of moving Steps 1-3 into `references/` is that an
|
||||
invocation loads one flow's content and not the other's; a two-section reference file re-inflates
|
||||
per-invocation context to the full 1,532-word span the body ceiling forced out. ADR-0020 measured
|
||||
these same files at 100 of ~120 differing lines after normalising `skill`/`agent`, so the merged file
|
||||
would also be mostly disjoint text under one heading.
|
||||
exactly one of the three pairs — the two same-named `references/description-quality.md` files — at
|
||||
100 of ~120 differing lines after normalising `skill`/`agent`. The other two pairs are unmeasured,
|
||||
and the figure should not be read onto them; the one that was measured is enough to make the point,
|
||||
because a merged file built out of it would be mostly disjoint text under one heading.
|
||||
|
||||
**Naming it `audit` (rejected).** Shortest available name and an accurate one. It collides with
|
||||
`agentsmd-audit`, which audits a repo's `AGENTS.md` and has nothing to do with the factory. A bare
|
||||
@@ -374,3 +398,37 @@ resetting to `0.1.0` would signal an immaturity that the merged content does not
|
||||
above to Step 0, the gotchas and the description are an improve pass, so under `skill-author`'s
|
||||
patch-bump rule it is now **`1.0.1`**. The plugin itself goes from `1.6.2` to **`2.0.0`**, because
|
||||
removing two invocable skills breaks anyone calling them by name.
|
||||
|
||||
**Twelve earlier ADRs are amended, and the split between the ones that are narrowed and the ones
|
||||
that only change a name is the load-bearing part.** A reader who takes "merged" to mean every
|
||||
decision about either skill is reopened will re-litigate settled ground; a reader who takes it to
|
||||
mean nothing changed will miss the one contract that did.
|
||||
|
||||
- **ADR-0008 is partially superseded** — its single-file invocation contract stands, but the set of
|
||||
accepted target shapes is narrower. An agent `.md` whose *immediate* parent directory is not
|
||||
`agents/` was audited pre-merge and is now refused at exit 2, because one auto-detecting entry
|
||||
point has to classify on the path alone and detection never guesses. Scope detection, counterpart
|
||||
derivation and the "name one file, the script derives the rest" signature are unaffected. ADR-0020
|
||||
anticipated exactly this when it recorded that the merge "reopens ADR-0008".
|
||||
- **ADR-0014 is partially superseded** — its two skill-scoped Vale copies, its split of `.vale.ini`
|
||||
into a skill config and an agent config, and the `check-vale-style-sync.sh` pre-push gate that
|
||||
diffed them, no longer apply. The reasoning is not reversed; the condition it operated on is gone.
|
||||
Its "one hook per file-scope" finding is untouched, and is why both exported hook IDs survive.
|
||||
- **ADR-0020 is partially superseded**, and only in the *count and location of the scripts* that
|
||||
carry it. Every tier, threshold and verdict rule stands. "All three validators" is two, and the
|
||||
mirrored constants live at one plugin path. Its deferred merge rule for this pair is discharged
|
||||
here rather than superseded — that is what this ADR implements.
|
||||
- **ADR-0021 is partially superseded** on that same count: "the three ADR-0020 validators" is two.
|
||||
The rule it decides — a published description states a domain boundary and never enumerates the
|
||||
skills behind it — is untouched.
|
||||
- **ADR-0004 is not superseded.** FAIL/SUGGESTION/INFO, and the rule that INFO never moves the
|
||||
pass/fail verdict, are carried into the skill flow unchanged. No audit criterion changes here.
|
||||
- **ADR-0009 is not superseded.** Its mechanism — field lists read from a reference file at runtime
|
||||
rather than hardcoded into script arrays, with a `source_keys` provenance chain back to the
|
||||
research corpus — survives and is reused as-is. Only the filename moves, `field-inventory.md` to
|
||||
`agent-field-inventory.md`, and that rename is byte-identical.
|
||||
- **ADR-0010, ADR-0012, ADR-0013, ADR-0015, ADR-0016 and ADR-0018 are not superseded.** Each names
|
||||
`skill-audit` or `agent-audit`, or a path beneath one, while deciding something else: a provenance
|
||||
file's location, a plugin boundary, Vale rule scope, an allowlist's home, a namespacing example.
|
||||
Every one of those references resolves through `factory-audit` now, and no claim any of them makes
|
||||
moves. They carry pointer annotations rather than amendments for that reason.
|
||||
|
||||
@@ -367,6 +367,16 @@ into the single `factory-audit/scripts/lib-boundary-resolver.sh`, sourced by tha
|
||||
The two remaining copies must still stay byte-identical — a plugin script cannot source the root
|
||||
one, which is the constraint that forces a copy to exist at all.
|
||||
|
||||
`tests/test-adr0020-contract.sh` pins that arrangement, and one of its assertions was green on a
|
||||
defect it named. "`validate.sh` sources the resolver in **both mode branches**" was implemented as a
|
||||
file-wide `grep -Ec … -ge 2`, which cannot see a branch at all: delete the `agent)` arm's source line
|
||||
and duplicate the `skill)` arm's, and the file-wide count is still 2 and the assertion still passes,
|
||||
with the agent path running no resolver or some other one. It is now a **per-arm structural check** —
|
||||
each arm of `validate.sh`'s `case "$MODE" in` block must carry exactly one `source` line inside its
|
||||
own body, and the file must carry exactly those two — with a mutation self-test that performs that
|
||||
exact count-preserving edit on a copy and requires the check to fail on it. The suite went 25 → 28
|
||||
cases.
|
||||
|
||||
### `python3` and PyYAML are hard requirements
|
||||
|
||||
Both, and neither is a best-effort accelerator.
|
||||
@@ -426,6 +436,58 @@ the hook definitions filter on `SKILL.md`
|
||||
"extend" that hook's `files:` pattern to cover agents** on the assumption that the script already
|
||||
knows the difference; doing so silently enforces a gate ADR-0020 declines to set.
|
||||
|
||||
### The three exit tiers of `factory-audit`'s validators
|
||||
|
||||
`validate.sh` and `validate-provenance.sh` use the exit code to say **whether an audit happened**,
|
||||
not only what it found. The skill's flow references tell the auditor to surface a non-zero run's
|
||||
output verbatim as real findings, so the distinction is load-bearing rather than cosmetic:
|
||||
|
||||
| Exit | Means |
|
||||
|---|---|
|
||||
| **0** | audited, and clean |
|
||||
| **1** | audited, and there are findings — or the target is present but cannot be opened |
|
||||
| **2** | **nothing was audited.** The checks never ran |
|
||||
|
||||
Exit 2 covers: no arguments at all; a target that does not exist, with its own "does not exist"
|
||||
message; a target matching neither the skill nor the agent shape; a missing or unreadable `lib-*.sh`
|
||||
beside the entry script; and a missing `python3` or PyYAML. Every one of those used to exit 1, so
|
||||
an install problem or a typo'd path arrived at the auditor as findings about the artifact. A missing
|
||||
target was the sharpest case: only the directory branch stats the target, so a typo'd `*.agent.md`
|
||||
path was classified on name alone, handed to `python3`, and came back as a FAIL about a file that was
|
||||
never there.
|
||||
|
||||
**A dangling symlink or a symlink loop stays exit 1, deliberately.** Both are false to `-e` and true
|
||||
to `-L`, and the existence guard tests both: something *is* at that path, it just cannot be opened,
|
||||
and "exists but unreadable" is a real finding the check suite reports as a FAIL naming the file.
|
||||
Catching them in the existence guard would replace that FAIL with a false "does not exist".
|
||||
|
||||
`--help` is exit 0 and now works with a library missing. `validate-provenance.sh` sourced both
|
||||
provenance libraries at the top of the file, so a partial install turned the one command that
|
||||
explains how to use the script into an exit 2; the libraries are now loaded only when a mode's half
|
||||
of the usage is actually printed, and when one is gone that half says so instead of the whole command
|
||||
failing. It also separates an **empty** positional from **no** positional — `validate-provenance.sh
|
||||
""`, an unquoted variable that expanded to nothing, used to be reported as "only flags were given",
|
||||
which sent the reader after a flag they never typed. Both are exit 2, with different messages.
|
||||
|
||||
**All three entry scripts are CDPATH-safe.** `cd` **prints** the directory whenever CDPATH supplied
|
||||
it, so a bare `cd` on a path starting with neither `/` nor `.` can both emit an extra line into
|
||||
whatever captures it *and* resolve to an unrelated tree. Every such site now does
|
||||
`CDPATH='' cd -- …`: `SCRIPT_DIR` and the target's parent-directory name in `validate.sh` and
|
||||
`validate-provenance.sh`, and **both** sites in `vale-wrap.sh` — its default `--config` resolution,
|
||||
and the directory-mirror walk below. The two bare `cd`s left in `vale-wrap.sh` take absolute paths
|
||||
(`mktemp -d` output, and the mirror root derived from it), which CDPATH is never consulted for.
|
||||
|
||||
The second of `vale-wrap.sh`'s two sites is not an asset lookup: its directory mirroring runs
|
||||
`cd "$arg" && find -L . … -print0` on a *path argument*, which is relative whenever the caller passed
|
||||
a relative one. With an exported CDPATH holding a same-named directory, the `cd` emitted the decoy's
|
||||
path into the `-print0` stream and the mirror was built from the decoy's files. It is cleared the
|
||||
same way now. Only the `cd` changed: the walk still mirrors the whole tree, prunes `.git`, and
|
||||
follows symlinks with `find -L` because vale does.
|
||||
|
||||
**That one was latent, not a live gate defect.** Neither published Vale hook reaches it: both filter
|
||||
on `files:` patterns that match single markdown files, and only the `-d "$arg"` branch mirrors a
|
||||
directory. The exposed caller is the hand-invoked `vale-wrap.sh <dir>`.
|
||||
|
||||
## Current retrofit status
|
||||
|
||||
The ADR-0020 gates ship hot, with no baseline file — a shrinking baseline was considered and
|
||||
@@ -614,6 +676,22 @@ existed.
|
||||
every path it matches must be in that hook's own artifact class. A hook narrowed to zero files never
|
||||
runs, and pre-commit reports no error.
|
||||
|
||||
**Case 34** asks, statically and with no Vale binary, whether the shipped `.vale.ini` could load a
|
||||
style at all. Four assertions: every `[glob]` section declares a **non-empty** `BasedOnStyles`; every
|
||||
style any section names resolves to a real directory under `StylesPath`; that `StylesPath` is **not
|
||||
absolute**; and at least one `[glob]` section exists, so the check cannot pass vacuously on a config
|
||||
with nothing in it. A section whose `BasedOnStyles` is empty is the silent case — Vale lints every
|
||||
file that glob matches with no rule loaded, prints `0 errors` and exits 0. The absolute-path clause
|
||||
is the one that is not obvious: an absolute `StylesPath` passes on the machine that wrote it and
|
||||
hard-fails for every external consumer of `.pre-commit-hooks.yaml`, which is the only reason those
|
||||
styles ship at all. Part B is a mutation self-test against the same function Part A calls — it empties
|
||||
each section's `BasedOnStyles` in a copy of the assets, and absolutizes `StylesPath` in another
|
||||
pointed at that copy's own real `styles/` directory, and requires each to fail by name.
|
||||
|
||||
**It is not case 0 again.** Case 0 reads `BasedOnStyles` lines file-wide and tolerates an absolute
|
||||
`StylesPath`; case 34 reads them **per `[glob]` section** and rejects one. Case 0 checks the config
|
||||
loads; case 34 checks that loading it arms anything.
|
||||
|
||||
### What Vale owns, and what stays LLM judgment
|
||||
|
||||
Six rule files, six distinct rules:
|
||||
@@ -830,8 +908,8 @@ hook — is deleted with the second Vale copy (ADR-0025). The six glob probes su
|
||||
|
||||
`test-vale-wrap.sh` without Vale skips only its Vale-dependent cases, not the whole suite. The cases
|
||||
that are plain greps and awk over the config and the two hook manifests still run: case 0, 16, 26,
|
||||
27, the static halves of 28, 31 Parts A and B, 32 and 33. A static failure exits 1, because a real
|
||||
defect is not a setup error. Only an all-static-pass run exits 77.
|
||||
27, the static halves of 28, 31 Parts A and B, 32, 33 and 34. A static failure exits 1, because a
|
||||
real defect is not a setup error. Only an all-static-pass run exits 77.
|
||||
|
||||
### Mentioning banned phrasing without tripping the rule
|
||||
|
||||
@@ -866,11 +944,47 @@ what to install. (It was three until `test-check-vale-style-sync.sh` was deleted
|
||||
`echo "$OUT" | grep -q PATTERN`. Under `set -o pipefail` the pipe form fails depending on timing:
|
||||
`grep -q` exits on its first match, `echo` takes SIGPIPE on its next write, and pipefail reports that
|
||||
as the pipeline failing, so output that matched reads as "no match". It showed up as a push gate that
|
||||
failed about once in 670 runs, on a different suite each time. `tests/test-no-pipefail-early-exit-grep.sh`
|
||||
scans every tracked shell file that sets pipefail and fails on the pipe form. It does this for `echo`
|
||||
or `printf` piped into `grep` with `-q`, `-m`, `-l`, `-L`, `--quiet` or `--silent`. It checks its own
|
||||
scanner against fixtures before trusting a clean result. Pipes from other commands are out of scope.
|
||||
In practice they either absorb the writer's exit status with `|| true` or write only once, at exit.
|
||||
failed about once in 670 runs, on a different suite each time.
|
||||
|
||||
`tests/test-no-pipefail-early-exit-grep.sh` is the static guard. **What it scans** is the tracked
|
||||
`*.sh`, `*.bats` and `*.bash` files, minus itself — `git ls-files -- '*.sh' '*.bats' '*.bash'` — and,
|
||||
within each one that sets pipefail, it fails on the pipe form. Every tracked shell file in this tree
|
||||
carries one of those three extensions today, so "every tracked shell file" is true in effect, but it
|
||||
is a property of the tree and not of the scan: a shell script tracked under any other name is not
|
||||
reached. It checks its own scanner against fixtures before trusting a clean result.
|
||||
|
||||
**What the pattern models**, after a review widened it on four axes and narrowed it on one:
|
||||
|
||||
- **The reader** is `grep`, `egrep` or `fgrep` — behind a path prefix (`/bin/grep`), a `command`
|
||||
prefix, or env-var assignments (`LC_ALL=C grep`) — taking `-q`, `-m`, `-l`, `-L`, `--quiet`,
|
||||
`--silent`, `--max-count` or `--files-with`. (The last two match the regex and were missing from
|
||||
this list.)
|
||||
- **Intermediate stages are seen.** `echo x | filter | grep -q y` is a site; only the two-stage form
|
||||
used to be.
|
||||
- **Both line continuations are joined** before matching: a trailing backslash, and a trailing `|`,
|
||||
which is equally legal in a pipeline. The hit is reported at the line the command starts on.
|
||||
- **`pipefail` may sit anywhere in a `set` line**, so `set -o errexit -o pipefail` arms the file. It
|
||||
previously had to follow the *first* `-o`, and a file-level miss skips every site in that file
|
||||
rather than one — which is why that test is deliberately loose.
|
||||
- **A bare `&` ends a segment**, so `echo ok && other | grep -q x` — whose writer is `other`, not the
|
||||
`echo` — is not a site. `&` followed by a digit is kept, so `2>&1` does not end one.
|
||||
|
||||
The widening turned up **five live vulnerable sites in `tests/test-apm-current-hook.sh`** that the
|
||||
narrower scanner never saw while the suite reported green; all five are converted, and the scanner
|
||||
now reports zero over the tree. Its fixtures went from 4 vulnerable spellings to **12**, plus
|
||||
near-miss negatives it must leave alone, and the suite from 5 cases to **7**.
|
||||
|
||||
**Remit: early-exiting GREP readers only.** `head`, `sed -n 1p` and a bare `read` exit early too, and
|
||||
an `echo` or `printf` feeding any of them is the same race. Those are guarded by **convention** —
|
||||
absorb the writer's status with `|| true`, or take the verdict from a here-string — and deliberately
|
||||
not by this test: most legitimate uses of them in this tree are already absorbed, and the scanner
|
||||
cannot see absorption from the pipeline text alone, so flagging them would be noise. Two live
|
||||
`grep … | head -1` sites (`tests/test-vale-wrap.sh:620` and `:1046`) were fixed by hand with that
|
||||
idiom. Pipes from a non-builtin writer (`run_wrap … | grep -q`) are out of scope for the same reason:
|
||||
in practice they either absorb the writer's exit status with `|| true` or write only once, at exit.
|
||||
|
||||
**Known limitation: heredoc bodies are scanned as code.** A `cat <<'EOF'` body containing a
|
||||
vulnerable-looking line would be reported as a real site. There are none in the tree today.
|
||||
|
||||
`tests/run-bats.sh` derives the set of `.bats` files it expects from `git ls-files`, so a `.bats`
|
||||
file deleted from the worktree but still tracked in the index fails the run rather than silently
|
||||
|
||||
Reference in New Issue
Block a user