refactor!: carry out the simplification audit across gates, tests, plugins and docs #135

Merged
Defame1297 merged 85 commits from docs/simplification-audit into main 2026-09-20 19:14:03 +00:00
119 changed files with 6308 additions and 5487 deletions
Showing only changes of commit 620f20b0fd - Show all commits

View File

@@ -1,7 +1,7 @@
{
"name": "holocron",
"description": "AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows.",
"version": "0.4.6",
"version": "0.4.7",
"owner": {
"name": "Defame1297",
"email": "defame1297@rkdr.net",
@@ -11,7 +11,7 @@
{
"name": "kyberforge",
"description": "Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace.",
"version": "1.6.2",
"version": "2.0.0",
"category": "Developer Tools",
"source": "./plugins/kyberforge"
},

View File

@@ -138,7 +138,7 @@ repos:
- id: check-apm-agents-valid
name: Validate real APM agent files
description: Run agent-audit's validate.sh over every plugins/*/.apm/agents/*.agent.md file in this repo -- the artifacts it governs, not fixtures
description: Run factory-audit's validate.sh over every plugins/*/.apm/agents/*.agent.md file in this repo -- the artifacts it governs, not fixtures
entry: bash scripts/check-apm-agents-valid.sh
language: system
stages: [pre-push]
@@ -163,24 +163,16 @@ repos:
pass_filenames: false
always_run: true
- id: check-vale-style-sync
name: Check Vale style copies are in sync
description: Diff skill-audit's Vale copy against agent-audit's canonical copy
entry: bash scripts/check-vale-style-sync.sh
language: system
stages: [pre-push]
pass_filenames: false
always_run: true
# verbose so the DOWNGRADED run is audible. This hook can pass while
# having verified strictly less than its name claims:
# CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 skips all six glob probes
# and says so on a `passed (text-level only, vale unavailable)` line.
# pre-commit prints nothing at all for a passing hook, so without this
# the opt-out reinstated exactly the silent vacuous pass the script was
# written to kill, one level up -- the run showed a bare `Passed` and
# the documented instruction to read that summary line was impossible to
# follow in the one situation the opt-out exists for. The script's clean
# output is a single line, so this costs one line per push.
# check-vale-style-sync was removed by ADR-0025. Only 6 of its 17
# assertions diffed skill-audit's Vale copy against agent-audit's; the
# merge into factory-audit leaves one copy, so those are moot. The other
# 11 moved into tests/test-vale-wrap.sh (case 0, cases 28-31, its
# Vale-absent skip, and case 33 for cross-manifest files: agreement),
# which run-tests runs here at
# pre-push, so do not re-add the hook to restore coverage. Do not
# confuse its removal with check-scope-walkup-sync below, which survives:
# that one cross-checks four hand-ported walk-up implementations, only two
# of which lived in the audit pair.
- id: check-scope-walkup-sync
name: Check scope walk-up implementations agree
@@ -236,7 +228,7 @@ repos:
files: '^plugins/[^/]+/\.apm/(skills/.*\.md|agents/.*\.agent\.md)$'
# README.md is excluded on purpose, not by oversight. A skill-directory
# README is consumer-facing prose that no agent ever loads, and the
# `git clone` lines in the seven tests/README.md files are setup
# `git clone` lines in the six tests/README.md files are setup
# instructions for a third party who has no rtk installed. Prefixing
# those would be actively wrong -- see ADR-0023's consumer section.
exclude: '(^|/)README\.md$'
@@ -245,17 +237,23 @@ repos:
- id: vale-audit-prefilter-skill
stages: ['pre-commit']
name: Vale audit prefilter (SKILL.md)
description: Run Vale against SKILL.md files as a deterministic prefilter for skill-audit, via skill-audit's own bundled copy
entry: plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh
description: Run Vale against SKILL.md files as a deterministic prefilter for factory-audit's skill flow, via factory-audit's own bundled copy
entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
language: script
files: '^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$'
pass_filenames: true
# Two hook IDs pointing at ONE vale-wrap.sh is deliberate, not leftover
# duplication. ADR-0014 measured a single hook entry silently scanning 0
# files of the other type, and ADR-0025 carried that finding across the
# merge: the two `files:` regexes are what keep the SKILL.md scope and the
# agent-file scope independently addressable. The script self-locates its
# config via ${BASH_SOURCE[0]}, so one copy serves both.
- id: vale-audit-prefilter-agent
stages: ['pre-commit']
name: Vale audit prefilter (agent files)
description: Run Vale against agent markdown files as a deterministic prefilter for agent-audit, via agent-audit's own bundled copy
entry: plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh
description: Run Vale against agent markdown files as a deterministic prefilter for factory-audit's agent flow, via factory-audit's own bundled copy
entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
language: script
files: '^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$'
pass_filenames: true

View File

@@ -1,14 +1,22 @@
# PUBLISHED CONTRACT. External repos consume these IDs with `rev: <tag>`, so an
# ID or a `files:` regex here may not change without breaking them on upgrade.
# ADR-0025 merged skill-audit and agent-audit into factory-audit and re-pointed
# both `entry:` paths at its single vale-wrap.sh; both IDs and both regexes are
# unchanged, deliberately. Collapsing them into one was considered and rejected:
# it breaks every consumer pinning kyberforge-vale-audit-agent, and it re-creates
# ADR-0014's measured failure where one hook against one config silently scanned
# 0 files of the other type. Two IDs are what keep both file scopes addressable.
- id: kyberforge-vale-audit-skill
name: Kyberforge Vale prose audit (SKILL.md)
description: Deterministic prose-pattern prefilter for kyberforge's skill-audit, via its own bundled Vale config/styles
entry: plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh
description: Deterministic prose-pattern prefilter for kyberforge's factory-audit skill flow, via its own bundled Vale config/styles
entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
language: script
files: '(^|/)SKILL\.md$'
- id: kyberforge-vale-audit-agent
name: Kyberforge Vale prose audit (agent files)
description: Deterministic prose-pattern prefilter for kyberforge's agent-audit, via its own bundled Vale config/styles
entry: plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh
description: Deterministic prose-pattern prefilter for kyberforge's factory-audit agent flow, via its own bundled Vale config/styles
entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
language: script
files: '(^|/)agents/[^/]+\.md$|\.agent\.md$'

View File

@@ -77,7 +77,7 @@ _Avoid_: the marketplace, upstream
**Provenance chain**:
The three-stage traceability record linking a skill back to its research inputs: `/research` produces
topic docs and a `sources.md`; the author skill records which sources informed which files in
`references/sources.md` and `source_keys` frontmatter; `skill-audit` validates the chain is complete
`references/sources.md` and `source_keys` frontmatter; `factory-audit` validates the chain is complete
and internally consistent.
_Avoid_: sources, citations, attribution
@@ -122,7 +122,7 @@ _Avoid_: repo root, project root
**Near-miss**:
A query that shares keywords with this skill but needs a different one — and, by extension, the
sibling that would wrongly answer it; boundary clauses exist to exclude genuine near-misses rather
than to enumerate siblings. Detail: `skill-audit/references/description-quality.md`.
than to enumerate siblings. Detail: `factory-audit/references/skill-description-quality.md`.
_Avoid_: overlap, similar skill
**Issue**:
@@ -130,6 +130,12 @@ The cross-provider term for a tracked unit of work. Gitea is this repo's canonic
(ADR-0007), but skills say "linked issue" generically rather than naming a provider.
_Avoid_: ticket, card, task
**Family prefix**:
The shared first segment of a skill name (`git-`, `gitea-`, `apm-`, `agentsmd-`) marking a group of
siblings. No bare skill name is a **Family prefix** of another: `forge` exists, so no skill is named
`forge-*`, because a prefix that matches a live sibling reads as ownership rather than membership.
_Avoid_: namespace, category
## Relationships
- An **apm package** bundles one or more **Skills** and agents; a **Plugin marketplace** lists
@@ -140,7 +146,7 @@ _Avoid_: ticket, card, task
type to the matching author skill — an already-specified fix (file, line, and change known) calls
that author skill directly, because each routing hop re-derives instructions from a shorter brief
and has been observed to drop hard constraints handed down the chain.
- A **Skill** built on research carries a **Provenance chain**; `skill-audit` fails it when broken.
- A **Skill** built on research carries a **Provenance chain**; `factory-audit` fails it when broken.
- **LESSONS.md** feeds the standing files: three or more entries on one pattern graduate the pattern
into the relevant standing document.
@@ -175,6 +181,8 @@ _Avoid_: ticket, card, task
the compounds keep the word and are not being renamed.
- "context" means both the model's live token window and the bounded domain this file describes —
resolved: unqualified "context" in this repo means the token window.
- "audit" was used for both an author skill's inline closeout and `forge`'s independent
- "audit" was used for both an author skill's inline closeout and the audit skill's independent
clean-context recheck — resolved: these are two distinct layers, kept separate precisely because
an audit running in the same context as the work it checks shares that work's blind spots.
an audit running in the same context as the work it checks shares that work's blind spots. The
recheck belongs to `factory-audit`, not to `forge`, which routes only to the author
skills and never to an audit.

View File

@@ -54,7 +54,7 @@ Skills sharing a resource (e.g. `validate.sh`) via a `shared/` directory and rel
## 2026-06-22 — Qualitative rubrics should be grounded in upstream spec docs, not in-repo usage
`skill-audit`'s description and body-discipline rubrics were derived from `skill-write`'s own conventions — circular, so drift in one silently propagated to the other. Fix: extract condensed reference files directly from the upstream spec (agentskills.io) into the audit skill, so the rubric is independent of in-repo convention drift.
`skill-audit`'s (now `factory-audit`'s skill flow, per ADR-0025: `references/skill-description-quality.md` and `references/skill-body-discipline.md`) description and body-discipline rubrics were derived from `skill-write`'s own conventions — circular, so drift in one silently propagated to the other. Fix: extract condensed reference files directly from the upstream spec (agentskills.io) into the audit skill, so the rubric is independent of in-repo convention drift.
## 2026-06-22 — Test files in scripts/ are dev tooling; document them in README as non-spec
@@ -62,7 +62,7 @@ The agentskills.io spec defines `scripts/` for bundled executables, not test inf
## 2026-06-27 — Clean-context audit catches what biased forks miss
A fresh-context skill-audit caught two FAILs (an incomplete README table, invalid cache paths) that the implementing fork's own audit missed — the fork that built the artifact knows what was intended and fills gaps silently. Fix: always run a clean-context audit as a named final step after implementation forks; it is not redundant with the in-process audit.
A fresh-context skill-audit (now `factory-audit`, per ADR-0025) caught two FAILs (an incomplete README table, invalid cache paths) that the implementing fork's own audit missed — the fork that built the artifact knows what was intended and fills gaps silently. Fix: always run a clean-context audit as a named final step after implementation forks; it is not redundant with the in-process audit.
## 2026-06-27 — Parallel forks on the same file produce conflicts requiring a third fork to reconcile

View File

@@ -32,13 +32,13 @@ Install all of these before setting up. Each one is a hard dependency of a git h
| --- | --- | --- |
| `apm` CLI | Two pre-push hooks shell out to it (`apm-audit-ci` and `apm-pack-check-clean`) | The `apm-install` skill, or `curl -sSL https://aka.ms/apm-unix \| sh`. Verify with `apm --version` |
| `python3` + PyYAML | Required by `scripts/skill-size-check.sh` (the `skill-size-check` pre-commit hook), which reads folded YAML frontmatter | `python3` is usually present — pre-commit is itself a Python application. `pip install pyyaml` if the hook reports PyYAML missing |
| `vale` | Required by the `vale-audit-prefilter-skill` / `-agent` pre-commit hooks and the `check-vale-style-sync` pre-push hook | `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or https://vale.sh/docs/vale-cli/installation/ |
| `vale` | Required by the `vale-audit-prefilter-skill` / `-agent` pre-commit hooks, and by the `test-vale-wrap.sh` / `test-vale-hooks-consumer.sh` suites that `run-tests --strict` runs at pre-push | `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or https://vale.sh/docs/vale-cli/installation/ |
| `claude` CLI | Required by the `validate-marketplace` pre-push hook | Claude Code |
Two notes worth reading before you skip one:
- **PyYAML is a hard requirement, not an optional accelerator.** The hand-rolled fallback frontmatter reader was removed deliberately: a reader that mis-parses an unfamiliar scalar shape reports a clean pass on a file it never measured.
- **No `vale sync` is needed.** The `Kyberforge` styles are committed under `plugins/kyberforge/.apm/skills/{skill-audit,agent-audit}/assets/vale/styles/`, not downloaded packages (ADR-0014).
- **No `vale sync` is needed.** The `Kyberforge` styles are committed under `plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/`, not downloaded packages (ADR-0014, ADR-0025).
## Setup

View File

@@ -76,13 +76,15 @@ Where the 276 s goes (each suite run alone, sequential):
| ~~`test-sync-plugin-content.sh`~~ deleted | ~~83 s~~ | 14 temp trees, 2 `git init`, repeated `apm pack` |
| all 351 bats tests (10 files, kyberforge and core validators) | 64 s | mostly `validate.sh` / `validate-provenance.sh` fixtures |
| `test-adr0020-differential.sh` | 29 s | 12 assertions; re-runs two validators over the live corpus and a fixture tree |
| `test-check-vale-style-sync.sh` | 25 s | guards a byte-identical copy |
| ~~`test-check-vale-style-sync.sh`~~ deleted | ~~25 s~~ | guards a byte-identical copy |
| `test-vale-wrap.sh` | 14 s | |
| `test-adr0020-frontmatter.sh` + `-targets.sh` | 25 s | |
| Remaining 20 suites | 36 s | 12 of them run in under 2 s each |
Five suites account for 215 s of 276 s. Three of those five (sync-plugin-content, vale-style-sync, adr0020-differential) test tooling that findings 2, 7, and 14 propose to delete or shrink, so the fastest path to a quick pre-push is removing the duplication those tests guard rather than optimising the tests.
> **Also struck (2026-09-15):** `test-check-vale-style-sync.sh` (25 s) went with the `check-vale-style-sync` hook in finding 14's merge (ADR-0025). Measured at HEAD: `tests/` holds **19** `test-*.sh` suites and `tests/run-tests.sh` reports `19 passed, 0 skipped, 0 failed`. Same basis as the note below — arithmetic on the 2026-09-10 baseline minus the struck rows, not a fresh timing run.
> **Done (2026-09-14):** see commit `718c79a` on `docs/simplification-audit`. The three struck-through rows are gone: `test-sync-plugin-content.sh` (83 s, 1,289 lines, 92 cases), `check-plugin-content-sync` (4.5 s) and `validate-plugins` (4.9 s). Expected, not re-measured: roughly 92 s comes off every push (83 + 4.5 + 4.9 = 92.4 s) (~83 s of it out of `run-tests`, which loses its single slowest suite), on the arithmetic of the 2026-09-10 figures alone. The remaining rows have not been re-timed since, so treat every number in this section as the 2026-09-10 baseline minus those three, not as a fresh measurement.
## 3. Enforcement layer: hooks, tests, scripts
@@ -95,7 +97,7 @@ This is the area you named as hardest to understand and slowest. Root cause: mos
> **Corrected and closed (2026-09-14, at `a6434e0`):** two things above went stale within hours of being written, and the finding was never given a marker.
>
> - **"Keep the two `claude plugin validate` hooks"** is void. `718c79a` (ADR-0024) deleted `validate-plugins` — the ADR's own reasoning is that `claude plugin validate` reads manifests only and could never detect the empty-content defect it was credited with guarding, and with the per-plugin manifests gone it has nothing left to read. Only **`validate-marketplace`** survives, over the one manifest this repo still ships (`.claude-plugin/marketplace.json`). Of the six hooks this finding named, three now exist: `validate-marketplace`, `apm-pack-check-clean`, `apm-audit-ci`. Verified against `.pre-commit-config.yaml`: 27 `- id:` entries, 9 with `stages: [pre-push]`, no `validate-plugins` entry.
> - **The gates.md figures above ("13→11 self-authored / 15→13 total") were correct for `0dffff3` and are no longer current.** `718c79a` removed two more pre-push hooks after that commit, and `docs/spec/gates.md:24` now reads **11 reported / 9 self-authored**. Read the count from that file, not from this note.
> - **The gates.md figures above ("13→11 self-authored / 15→13 total") were correct for `0dffff3` and are no longer current.** `718c79a` removed two more pre-push hooks after that commit, and `docs/spec/gates.md:24` read **11 reported / 9 self-authored** when this note was written; finding 14's merge has since removed `check-vale-style-sync`, and it now reads **10 reported / 8 self-authored**. Read the count from that file, not from this note.
>
> Marked `[x]`: all three of this finding's decisions are resolved — `check-manifests` deleted (`e647f14`), `apm-audit-ci` kept on the grill above, `apm-marketplace-check` removed (`0dffff3`).
@@ -104,6 +106,7 @@ This is the area you named as hardest to understand and slowest. Root cause: mos
- `check-vale-style-sync`: 413 lines + 798 test lines guarding a byte-identical 526-line `vale-wrap.sh` and style directory copied between skill-audit and agent-audit. About 350 of its lines run Vale glob probes against the hook file patterns. Disappears if the two audit skills merge (finding 14); the probes belong in `test-vale-wrap.sh`.
- `check-scope-walkup-sync`: 365 lines cross-checking four independent ports of the same package-root walk-up. Disappears if the ports share one script or the skills merge.
> **Grilled, held (2026-09-14):** both of the above are gated on findings 14/15 (merging skill-audit+agent-audit and skill-author+agent-author), deliberately held for a separate session rather than decided here. Correction for that session: the audit's §8 grouping is wrong — these merges don't need ADR-0012 revisited (that ADR governs the unrelated `core` plugin's three `agentsmd-*` skills). The actual constraint is ADR-0014 (no-cross-skill file sharing on plugin cache-install), and merging sidesteps it rather than requiring it be reversed. The open question for that session is a design one — a shared skill's `description` carrying both skill- and agent-audit trigger phrases — not an ADR supersession. ADR-0012 revisit is needed only for finding 24.
> **Settled (2026-09-15) — split verdict, and the first bullet held in full.** Finding 14 landed as `factory-audit` (ADR-0025). **`check-vale-style-sync` is deleted**, hook, script and test, exactly as the first bullet predicted — and its probes **were** rehomed into `test-vale-wrap.sh`, as cases 28-30 (case 31 carries the override allowlist), so both halves of that bullet are closed. `docs/spec/gates.md` records the rehoming, not an open gap. *(Updated later on 2026-09-15.)* The one assertion this note used to call still uncovered — cross-manifest *agreement* between `.pre-commit-hooks.yaml`'s and `.pre-commit-config.yaml`'s `files:` regexes — is now ported as case 33, which pairs the hooks by `id:`. Case 32 covers the separate zero-match question. It was a real gap while it lasted: narrowing the local skill hook to `^plugins/kyberforge/` left 6 of 38 skills prefiltered and the suite green. `bash tests/test-vale-wrap.sh` now reports `61 passed, 0 failed` (it was 56 before cases 0 and 33 and the Part B mutation self-tests). The bullet's "about 350 of its lines run Vale glob probes" overstates the probe half: at `a5962ba` the script is **413 lines**, of which the `.vale.ini` coverage section is **332** (`67..398`) and the machinery that actually invokes vale against a probe path is **204** (`195..398`). The balance of that section is `StylesPath`, `BasedOnStyles` and per-rule-override greps — text assertions, not probes. (Its test file is **797** lines, as the note above says, not the 798 the bullet carries.) **`check-scope-walkup-sync` stays**, and the second bullet's "or the skills merge" is wrong: two of its four walk-up ports are in the *author* skills (`new-agent.sh`, `new-skill.sh`), which this merge does not touch, and the audit-side pair is Python against the author-side pair's Bash, so the gate can never degrade into a text diff. Full reasoning in §10's 2026-09-15 note. Finding 15 would not remove it either.
- [x] ~~`check-marketplace-mirror-sync`: guards `.github/plugin/marketplace.json`. The script header calls it Copilot's legacy convention path and says Copilot also accepts the Claude path; the vendored Copilot docs list it as primary. Verify against current Copilot CLI before deleting hook, script, test, and mirror file.~~
> **Grilled and done (2026-09-14):** verified against GitHub's current Copilot CLI plugin docs (not the vendored copy, which risked drift). Copilot CLI's marketplace discovery checks paths in order — `marketplace.json`, `.plugin/marketplace.json`, `.github/plugin/marketplace.json`, `.claude-plugin/marketplace.json` — falling through to whichever exists first. `.claude-plugin/marketplace.json` (apm's own `claude` output) already satisfies that chain's last step, so the dedicated `.github/plugin/marketplace.json` mirror bought Copilot users its *preferred* discovery path rather than a required one. Decided against reopening ADR-0018 (native install for both Claude Code and Copilot CLI stays supported) to justify this — the deletion holds either way, since Copilot's own fallback covers it. Deleted `.github/plugin/marketplace.json`, `scripts/sync-marketplace-mirror.sh` (81 lines), `tests/test-sync-marketplace-mirror.sh` (304 lines), and the `check-marketplace-mirror-sync` pre-push hook; removed the dangling references to the deleted script in `scripts/sync-plugin-content.sh` and `tests/test-sync-plugin-content.sh` (both had comments citing its reasoning by name), and updated `docs/spec/architecture.md`'s description of the marketplace-manifest compile step. `tests/test-sync-plugin-content.sh` (92 cases) still passes in full.
>
@@ -135,6 +138,8 @@ This is the area you named as hardest to understand and slowest. Root cause: mos
> **Corrected (2026-09-14):** two of the six named targets no longer exist — `validate-plugins` and `check-plugin-content-sync` were deleted in commit `718c79a` (finding 7, ADR-0024). Actual state today: **9 repo-authored pre-push hooks** — `run-tests`, `check-executables-allow-sync`, `apm-audit-ci`, `check-apm-agents-valid`, `apm-pack-check-clean`, `check-vale-style-sync`, `check-scope-walkup-sync`, `check-release-needed`, `validate-marketplace` — plus the 2 pre-commit `meta` hooks that also run at this stage, so 11 are reported at pre-push. `validate-marketplace` was kept: the root `marketplace:` block in `apm.yml` and the root `.claude-plugin/marketplace.json` stay, because apm's own marketplace consumers read that same catalogue and `<name>@holocron` short names depend on it. (That manifest is the only tracked file under `.claude-plugin/` — `git ls-files .claude-plugin` returns it alone. The sibling `.claude-plugin/plugin.json` is a local `apm pack` byproduct, has never been tracked on any branch, and is ignored at `.gitignore:59`; it was not "kept", because it was never there.)
> **Superseded count (2026-09-15):** finding 14 deleted `check-vale-style-sync` with the merge into `factory-audit` (ADR-0025), so pre-push is now **8 repo-authored hooks** (10 reported). The dated note above is the state on 2026-09-14; see finding 14's note for the correction.
Pre-commit stays roughly as is minus `skill-frontmatter`, and minus `check-ast` once finding 9 removes the only `.py` files. ~~Tests 26 files to about 10 (12,400 to about 5,000 lines).~~ Keep bats and its three submodules; the 351 bats tests ship inside plugins and are the right tool there. Do not port the bash suites to bats; delete them instead.
> **Re-measured (2026-09-14, at `a6434e0`):** the tests target was stated against the 2026-09-10 baseline and both its numbers are stale. `tests/` now holds **20 `test-*.sh` suites totalling 9,123 lines** (plus the two runners, 490). Six suites have gone since the baseline: `test-check-manifests.sh` (`e647f14`), `test-skill-frontmatter.sh` (`c8a7c9e`), `test-governance-layer.sh` and `test-instructions-and-docs.sh` (`5f9f2b3`), `test-sync-marketplace-mirror.sh` (`0dffff3`), `test-sync-plugin-content.sh` (`718c79a`). Restated on the same basis the target is **20 files to about 10, 9,123 to about 5,000 lines** — the file half of the target is now the closer half, and finding 9's `check-ast` clause is moot anyway, since finding 9 is not proceeding.
@@ -156,7 +161,7 @@ The shared pattern: per-skill `README.md` files no model reads, a `docs/research
>
> Corrected figures: **46 `sources.md` files / 1,752 lines** in three distinct classes — 29 skill `references/sources.md` (1,217 lines), 13 research indexes (435), 4 plugin-root files (100, ADR-0010). The finding does **not** double-count: it states two disjoint classes additively ("32 plugin and skill `sources.md` files (about 1,300 lines) **plus** 9 research indexes"), and that plugin-and-skill subtotal is really **33 files / 1,317 lines**, matching its "about 1,300" exactly — had the 32 swept in the research indexes the figure would have been ~1,750. Its real errors there are an off-by-one (32 should be 33) and an omission: it missed the 4 vendored example indexes under `kyberforge/docs/research/examples/skill-write/`, so 9 should be 13. Carriers of `source_keys` in YAML frontmatter: **196** — 168 at column 0 and 28 nested two spaces under `metadata:` — so the finding's 216 is closer to the truth than it looks. (219 files merely *mention* the string. A naive `^[[:space:]]*source_keys:` grep returns 200, but 4 of those are heredoc or fixture text rather than frontmatter: both `validate-provenance.bats` copies, `scripts/check-scope-walkup-sync.sh`, and a fenced example in `plugins/bin/.apm/skills/research/references/file-format.md`.) Checks: **16 across the two copies** (skill-audit 0–9, agent-audit 0–5), not ten. Validator line counts (1,198 / 632) and 125 bats tests are exact.
>
> **"Touches every skill's frontmatter" is roughly right.** **28 of the 39 real skills carry `source_keys` in frontmatter**, nested under `metadata:` — see `plugins/git/.apm/skills/git-commits/SKILL.md:10-17`, where `metadata:` → `source_keys:` carries four slugs. (44 tracked files match `*SKILL.md`; subtract `skill-author/assets/templates/SKILL.md` and the 4 vendored under `kyberforge/docs/research/examples/skill-write/`, leaving 39 real skills.) The 11 without it are exactly the `plugins/bin/` skills. Check 2 in the skill-audit validator (SKILL.md `source_keys` → slug in `sources.md`) is correspondingly **live**, not dead code: `parse_source_keys()` at `plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh:242-270` handles both spellings explicitly — the metadata-nested branch at `:257`, the top-level branch at `:260`, and a docstring that says "handles metadata.source_keys and top-level" — check 2 at `:766` runs against all 28 carrier skills, every one of which has a `references/sources.md`, and bats pins it at `validate-provenance.bats:222` ("FAIL: source_keys slug in SKILL.md not present as H2 in sources.md") and `:1313` (a BOM must not silently disable check 2). The imbalance the finding names is real and **worse** than claimed: 4,641 validator+bats lines against 1,752 of metadata, a 2.6:1 ratio.
> **"Touches every skill's frontmatter" is roughly right.** **28 of the 39 real skills carry `source_keys` in frontmatter**, nested under `metadata:` — see `plugins/git/.apm/skills/git-commits/SKILL.md:10-17`, where `metadata:` → `source_keys:` carries four slugs. (44 tracked files match `*SKILL.md`; subtract `skill-author/assets/templates/SKILL.md` and the 4 vendored under `kyberforge/docs/research/examples/skill-write/`, leaving 39 real skills.) The 11 without it are exactly the `plugins/bin/` skills. Check 2 in the skill-side validator (SKILL.md `source_keys` → slug in `sources.md`) is correspondingly **live**, not dead code: `parse_source_keys()` at `plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh:277-305` handles both spellings explicitly — the metadata-nested branch at `:292`, the top-level branch at `:295`, and a docstring that says "handles metadata.source_keys and top-level" — check 2 at `:712` runs against all 28 carrier skills, every one of which has a `references/sources.md`, and bats pins it at `plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats:222` ("FAIL: source_keys slug in SKILL.md not present as H2 in sources.md") and `:1337` (a BOM must not silently disable check 2). (Paths and line numbers re-derived at HEAD: ADR-0025's merge moved this code out of `skill-audit/scripts/validate-provenance.sh` into the shared skill-side library, so the figures this note carried at `062ca47` — `:242-270`, `:257`, `:260`, `:766`, `:1313` — no longer resolve.) The imbalance the finding names is real and **worse** than claimed: 4,641 validator+bats lines against 1,752 of metadata, a 2.6:1 ratio.
>
> **Omitted entirely: the chain has a producer.** `plugins/bin/.apm/skills/research/` *specifies* the `sources.md` + `source_keys:` output format, and `plugins/bin/evals/research/research/eval.yaml` carries three criteria asserting it. **This is the blocking decision: does `research` keep emitting `sources.md`?** If yes, the chain is not dropped — only unenforced, and the finding collapses to "delete the validators." If no, the research skill's output contract and its evals need redesigning.
>
@@ -172,7 +177,14 @@ The shared pattern: per-skill `README.md` files no model reads, a `docs/research
### 4.2 kyberforge (290 files, 44,568 lines incl. mirror; the 7 SKILL.md bodies are 333 lines, under 1%)
14. **Merge `skill-audit` + `agent-audit` into one `audit` skill (removes about 3,300 lines and two pre-push hooks).** `vale-wrap.sh` is byte-identical in both; five Vale rules byte-identical (agent-audit carries one extra, so it is the superset); `validate.sh` shares a 1,061-line boundary-target resolver block that diffs as zero lines; SKILL.md steps 1, 3, 4 and the gotchas are the same text. Each copy is hard-wired to one mode, so the merged script needs a path switch. The duplication exists because a plugin-cache install copies only each skill's own files (the rule ADR-0014 follows), so a script cannot be shared across skills; merging the skills is the only way to remove the copy. Effort M.
14. [x] ~~**Merge `skill-audit` + `agent-audit` into one `audit` skill (removes about 3,300 lines and two pre-push hooks).** `vale-wrap.sh` is byte-identical in both; five Vale rules byte-identical (agent-audit carries one extra, so it is the superset); `validate.sh` shares a 1,061-line boundary-target resolver block that diffs as zero lines; SKILL.md steps 1, 3, 4 and the gotchas are the same text. Each copy is hard-wired to one mode, so the merged script needs a path switch. The duplication exists because a plugin-cache install copies only each skill's own files (the rule ADR-0014 follows), so a script cannot be shared across skills; merging the skills is the only way to remove the copy. Effort M.~~
> **Done (2026-09-15), with three of its claims corrected.** Merged into **`factory-audit`**, not `audit` — the name states the domain (the artifact factory's own output) rather than the verb. See `docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md`. The repo goes from 39 skills to 38. Entry scripts are `scripts/validate.sh`, `scripts/validate-provenance.sh` and `scripts/vale-wrap.sh`. Only the first two auto-detect the artifact type they were handed and dispatch to a per-type library. `vale-wrap.sh` does not and never did: it is byte-identical to both pre-merge copies (`diff` clean against each at `a5962ba`) and names neither `SKILL.md` nor `.agent.md` anywhere in its 526 lines. Its scoping comes from outside it — the `.vale.ini` glob sections and the `files:` regexes of the two hooks that call it.
>
> - **Yield: 2,934 lines and ONE pre-push hook, not ~3,300 and two.** The hook is `check-vale-style-sync`, deleted with its script (413 lines) and `tests/test-check-vale-style-sync.sh` (797). **They did *not* exist only to diff the two now-merged Vale copies** — an earlier revision of this bullet said so and it was wrong, as this document's own ":104" measurement already implied. The script has **17** assertion sites (13 `err` calls and 4 hard-fail exits; ADR-0025 maps each one). Only **6** are genuinely moot: two diffed the copies and four guarded the script's ability to locate them. **10** were rehomed into `tests/test-vale-wrap.sh`: case 0 (config loads), cases 28–30 (glob probes, style loading, Copilot scoping), case 31 (override allowlist) and the suite-level exit 77. **1**, the cross-manifest `files:` drift check, is ported as case 33, pairing hooks by `id:` since both now share one `entry:`. *(Corrected later on 2026-09-15.)* An earlier revision of this bullet said 18 / 6 / 11 / 1. It called the cross-manifest check knowingly dropped and "seven of them stronger". None of that survives a recount. Two text greps became behavioural Vale probes, not seven, and case 32 alone never covered the narrowing that case 33 now catches. `check-scope-walkup-sync` **survives**; see the §10 correction below for why. Pre-push goes 9 repo-authored hooks to 8 (11 reported to 10). The rest of the saving is the second embedded resolver (1,061), the second `vale-wrap.sh` (526), the second `assets/vale/styles/Kyberforge/` copy (44), and the Contributing-files parser embedded in both `validate-provenance.sh` copies (93). **413 + 797 + 1,061 + 526 + 44 + 93 = 2,934**, which is the headline. An earlier revision of this bullet listed 413 + 797 + 526 + 48 + 1,061 = 2,845: it dropped the 93-line parser outright, and its **48** for the Vale copy is the five byte-identical style rules (13 + 7 + 7 + 7 + 10 = **44**) plus skill-audit's 4-line `.vale.ini`. ADR-0025 counts **44** on purpose — the two `.vale.ini` files were deliberately *not* identical (agent-audit's carried the extra `[**/*.agent.md]` section and the `KyberforgeCopilot` style), so that file is a deleted file rather than a removed duplicate, and folding it in would make the headline 2,938. All six figures measured at `a5962ba`.
> - **"Each copy is hard-wired to one mode" was false**, and it is the claim that made this look like a bigger win than it is. The two `validate.sh` files are not one script parameterised per mode: outside the shared 1,061-line resolver they hold **1,293 lines between them** (616 skill-side, 677 agent-side) and share **91** of those. That 91 is ADR-0025's figure and it is exactly reproducible: strip the marked resolver block from each copy at `a5962ba` (`115..1175` skill-side, `189..1249` agent-side, 1,061 lines each), then take the size of the intersection of the two *distinct raw line sets* — 510 distinct lines skill-side, 530 agent-side, 91 in common. An earlier revision of this bullet said "about 115", which matches no counting rule that has been reproduced: dropping blank lines gives 90 and dropping comments as well gives 64. The merged validator dispatches on artifact type over two largely independent bodies of checks; it does not collapse them.
> - **The §8 blocker was a non-issue.** The design question held open there — whether one `description` could carry both skills' trigger sets without breaching the ADR-0020 ceiling — was answered against the **400-character FAIL**, which the merged description clears. Read the number from the shipped file, not from a draft. *(Corrected later on 2026-09-15.)* The shipped description is **241 characters**, inside the 250-character SUGGESTION target, and `bash scripts/skill-size-check.sh plugins/kyberforge/.apm/skills/factory-audit/SKILL.md` prints nothing for it. A first cut shipped at **319** and accepted the SUGGESTION as the cost of carrying both artifact types' trigger phrases. That reasoning was wrong. The quoted phrases (`audit this skill`, `review my SKILL.md`, `audit this agent`, `review my agent file`) restated the "skill directory or agent definition audited" trigger in a second register, which ADR-0020 makes a FAIL. Removing them, and keeping both boundary arrows, gives 241. An earlier "241" in this document and ADR-0025's "240" came from a hypothetical single-arrow draft that was never reproduced. That today's figure is also 241 is a coincidence, not a confirmation of it. The real ceiling was the other one: a single body covering both artifact types ran past the **900-word body FAIL**. Solved the way ADR-0020 prescribes — a dispatch body that routes to per-type references, with the 16 per-type reference files namespaced `skill-*` and `agent-*` (plus the shared `sources.md`).
>
> **Finding 18 was deliberately kept out of scope.** Its re-scoped remainder is prose trimming inside these same files and would have made the merge diff unreviewable; it stays open against `factory-audit`'s files.
15. **Merge `skill-author` + `agent-author` likewise.** `contract.md` shares most of its Description section; `new-skill.sh` and `new-agent.sh` implement the same package-root walk-up with different mode names; step 1 dispatch tables and step 3 gates are near-identical. Keep the agent scope logic (plugin vs project/user) as its own reference. Effort M.
@@ -489,6 +501,14 @@ The recurring failure mode is worth naming, because it has now produced six wron
**Where the real remaining opportunity is:** finding 14 (merge `skill-audit` + `agent-audit`) at **−1,587 lines with zero coverage loss**, which is also where finding 16's savings actually live. Its blocker is the design question in §8 — one `description` carrying both skills' trigger phrases — not an ADR supersession.
> **Executed, and one knock-on claim corrected (2026-09-15).** Finding 14 landed as `factory-audit` (ADR-0025); yield **2,934 lines and one pre-push hook**, and the §8 blocker turned out to be a non-issue. The body was the binding ceiling, not the description, which ships at 241 characters, under the 250 target, once a duplicated trigger register was removed. See finding 14's own note for the corrections.
>
> **The merge does not unblock `check-scope-walkup-sync`, and nothing in this audit should be read as saying it does.** §3's finding 2 bullet says that gate "disappears if the ports share one script or the skills merge"; the second half of that is wrong, and the first is unreachable. The gate cross-checks **four** independent `$HOME`/`.git`/`apm.yml` walk-up ports, and only two of them are in the audit pair (`validate.sh`'s `detect_scope`, `validate-provenance.sh`'s `find_plugin_root`). The other two — `new-agent.sh`'s and `new-skill.sh`'s `find_package_root` — live in the **author** skills, which finding 15 has not merged and which could not be merged into the audit skill in any case. Four ports go to four ports.
>
> It cannot degrade into a text diff either, which is the shape that would let it be deleted rather than merely shrunk: the two audit-side ports are **Python** (`def detect_scope`, `def find_plugin_root`, inside heredocs) and the two author-side ports are **Bash** functions. Byte-comparing them is not an option at any point on this path, so the behavioural fixture cross-check is the only available form of the gate. It survives finding 15 too.
>
> `check-vale-style-sync` was the only one of the two "keep two copies in sync" gates that finding 14 could remove, which is why the yield is one hook and not two.
### Two defects to fix independently of any finding
- **~~A live bug in always-on context.~~ Fixed (2026-09-15).** The deployed `core/instructions/governance.md` cited `docs/HUMANS.md`, which does not exist — the file is `docs/wiki/HUMANS.md`. Five occurrences across three files (`governance.md:82`, which was self-inconsistent against its own correct line 73; `CONTROLS.md:5,101,106`; `ai-constitution.md:238`), in a file `@`-imported into every session in every project. All five now point at `docs/wiki/HUMANS.md`. Note the deployed copy under `~/.claude/` no longer matches the repo until `scripts/install.sh` re-runs.

View File

@@ -1,5 +1,5 @@
name: holocron
version: 0.4.6
version: 0.4.7
description: AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows.
license: MIT
@@ -42,7 +42,7 @@ dependencies:
# after a kyberforge release, check this first.
executables:
allow:
kyberforge#1.6.2:
kyberforge#2.0.0:
hooks: true
bin: true
@@ -52,7 +52,7 @@ marketplace:
# top-level apm.yml description:/version: above are NOT inherited into the
# compiled output despite being used elsewhere (e.g. by `apm audit`).
description: AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows.
version: 0.4.6
version: 0.4.7
owner:
name: Defame1297
email: defame1297@rkdr.net
@@ -77,7 +77,7 @@ marketplace:
- name: kyberforge
description: Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace.
source: ./plugins/kyberforge
version: 1.6.2
version: 2.0.0
category: Developer Tools
- name: bin

View File

@@ -1,5 +1,10 @@
# Add INFO as a third finding level in skill-audit reports
**Skill renamed per ADR-0025 (2026-09-15):** `skill-audit` and `agent-audit` merged into
`factory-audit`, which dispatches to a skill flow and an agent flow at Step 0. Read `skill-audit`
below as `factory-audit`'s skill flow. The decision itself is unchanged — ADR-0025 carried every
audit criterion, tier and finding level across as-is.
`skill-audit` shipped with two finding levels: FAIL (blocks shipping) and
SUGGESTION (optional improvement). Provenance validation introduced observations
that are worth surfacing but not actionable: a `references/*.md` file with no

View File

@@ -1,5 +1,11 @@
# agent-audit takes a single file path and derives the counterpart by scope detection
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow. The single-file
invocation contract this ADR sets survives the merge intact — `factory-audit` dispatches at Step 0
on the target path, and an `*.agent.md` or a path under `.apm/agents/` takes the agent flow, so the
caller still names one file and the script still derives the rest.
`agent-audit` validates agent definition file pairs (Claude Code `.md` + Copilot `.agent.md`). The skill accepts a path to either file and derives the counterpart using scope detection rather than requiring the caller to name both files or supply a root directory.
## Considered options

View File

@@ -1,5 +1,10 @@
# agent-audit reads field lists from a reference file, not hardcoded script arrays
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow; the reference file this
ADR is about is now `factory-audit/references/agent-field-inventory.md`. The decision is unchanged —
the field lists still live in a reference file read at runtime, not in script arrays.
`agent-audit`'s `validate.sh` checks for Claude Code-only fields in Copilot files and
silently-ignored fields in plugin agents. Rather than hardcoding those field lists in the
script, the script reads `references/field-inventory.md` at runtime. This keeps field list

View File

@@ -15,6 +15,9 @@ ADR's own conclusion is unaffected by that move: the provenance file still belon
auto-scans, and `.apm/agents/` is, if anything, further removed from plugin-root than the old
flat `agents/` directory was, so the reasoning below still holds. References below to
`<plugin-root>/agents/` describe the pre-APM layout in effect when this decision was made.
**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into
`factory-audit`; read the `agent-audit` references below as `factory-audit`'s agent flow, whose
`validate-provenance.sh` still resolves `<plugin-root>/sources.md` exactly as this ADR decided.
**Scope boundary (per ADR-0016):** this path change is plugin scope only. Project scope
(`.claude/agents/` + `.github/agents/`) and user scope (`~/.claude/agents/` +
`~/.copilot/agents/`) are unaffected — they are not APM packages and keep the dual-file

View File

@@ -6,7 +6,7 @@
Three skills in the `core` plugin (`core`'s first active skills):
- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern. When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition.
- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern (**skill renamed per ADR-0025, 2026-09-15:** `skill-audit` is now `factory-audit`'s skill flow; the author-then-audit pattern is unchanged). When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition.
- **`agentsmd-audit`** — a single combined pass checking three mandatory baselines against `AGENTS.md` only: secrets/credentials (governance.md hard prohibition), structural completeness (common-sections checklist from the agents.md spec), and accuracy/drift (do referenced commands/paths resolve against the repo). Never inspects provider adapter files.
- **`provider-adapter-author`** — detects and converts a provider-specific instruction file into a thin adapter that imports `AGENTS.md` (mirroring this repo's own two-tier `CLAUDE.md` pattern). Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`) rather than a separate paired audit skill, since the check (import present, no duplicated headings, size threshold) is mechanical.

View File

@@ -109,7 +109,8 @@ every rule to `level: error` is what actually implements this decision.
1.81 tokens per word, so a worst-case `SKILL.md` at the ceiling still lands under 5,000 tokens —
`wc -w` is not BPE tokenization). Either one exceeded fails the hook. Both are
inclusive: a file at exactly 500 lines or exactly 2,770 words passes, and only one past a ceiling
fails. `skill-audit/scripts/validate.sh` enforces the same pair on the same inclusive terms, so
fails. `skill-audit/scripts/validate.sh` (now `factory-audit/scripts/validate.sh`, see ADR-0025)
enforces the same pair on the same inclusive terms, so
the audit and the commit hook cannot disagree about whether a given `SKILL.md` is over size.
- `styles/KyberforgeTrial/` and `.vale.trial.ini` were deliberately not created — noted here so a
future reader doesn't wonder if a trial tier was forgotten.

View File

@@ -5,6 +5,28 @@ out of the repo root was deliberately deferred there, not fixed. ADR-0013's othe
(rule scope, `level: error` model, `SentenceOpenerThereIs`/`VagueQualifier` trial outcomes) is
unaffected and remains in force.
**Amended by ADR-0025 (2026-09-15).** The reasoning below is not reversed; its *precondition* is
gone. The two skill-scoped Vale copies this ADR mandates — `agent-audit/assets/vale/` (canonical)
and `skill-audit/assets/vale/` (subset) — existed because the no-cross-skill-sharing rule made it
impossible for one audit skill to read the other's config. ADR-0025 merges the two skills into
`factory-audit`, so there is no boundary left to duplicate across: there is now **one** copy, at
`plugins/kyberforge/.apm/skills/factory-audit/assets/vale/`, carrying both styles and the
single-file `.vale.ini` — `[**/SKILL.md]`, `[**/agents/*.md]`, `[**/*.agent.md]` — that this ADR's
"One hook per file-scope" section had split in two. `scripts/check-vale-style-sync.sh`, decided on
below and wired at pre-push, is deleted with the copy it diffed. Nothing it asserted about the
config was lost. Its six-row glob-coverage probe table is now `tests/test-vale-wrap.sh` cases 28-30,
run against the merged config. Case 31 carries across the per-rule override allowlist, and case 0
carries across the "config loads" guards. Its **cross-manifest `files:` drift check** is ported as
case 33. The original keyed each hook's record on `entry:`, which stopped working once both vale
hooks shared one entry, so the port pairs the hooks by `id:` instead. Of the script's 17 assertion
sites, 6 compared the two copies and are moot, 10 are rehomed and 1 is ported. ADR-0025 gives the
per-assertion mapping; read the "six" here as probe *rows*, not as a share of those 17.
What does **not** change: the two exported hook IDs, `kyberforge-vale-audit-skill` and
`kyberforge-vale-audit-agent`, keep their IDs and their `files:` regexes — external consumers pin
them by name — and the argument-free `entry:` contract is untouched. Read the two-copy table, the
sync-check paragraph, and the `tests/test-vale-wrap.sh` Consequences bullet below as the state this
ADR established, not as current layout.
`skill-audit`/`agent-audit`'s Step 1 called
`"$(git rev-parse --show-toplevel)/scripts/vale-wrap.sh" --config "$(git rev-parse --show-toplevel)/.vale.ini"`
— which resolves to whichever repo the skill happens to be running in. Inside `ai-development`
@@ -128,7 +150,9 @@ doesn't wonder if it was overlooked.
`.pre-commit-config.yaml` stays byte-identical to the shipped manifest on those `entry:` lines
so the local gate keeps exercising the same resolution path a consumer does.
- `tests/test-vale-wrap.sh` now exercises skill-audit's copy specifically — its fixtures are all
`SKILL.md`-shaped, and only skill-audit's `.vale.ini` has the matching glob section.
`SKILL.md`-shaped, and only skill-audit's `.vale.ini` has the matching glob section. (State as of
this ADR. Since ADR-0025 there is one `vale-wrap.sh` and one `.vale.ini` under `factory-audit/`,
and that suite exercises all three glob sections of the merged config — see cases 28-30.)
- The first `vX.Y.Z` tag is cut once this change and its tests pass, giving external
`.pre-commit-hooks.yaml` consumers something to pin.
- **Cutting the tag is not left to memory.** `scripts/check-release-needed.sh`, wired at

View File

@@ -123,7 +123,8 @@ correction) sorted what they document into three buckets:
- ADR-0016 (a narrower decision discovered while designing issue #89) turned out to gate how
issue #90 had to re-author plugin-scope agents: `.apm/agents/*.agent.md` compiles verbatim to
both Claude and Copilot, so those files carry only the fields in the `apm-agent-allowlist` section
of `plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (as amended
of `plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (now
`factory-audit/references/agent-field-inventory.md`, see ADR-0025) (as amended
2026-08-14: `name`/`description`/`model`/`source_keys`/`disallowedTools`) — existing dual-file
`<name>.md`+`<name>.agent.md` pairs could not be raw-moved, only re-authored.
- Two follow-up issues tracked the remaining work: #89 (`skill-author`/`agent-author` routing

View File

@@ -143,7 +143,8 @@ below is narrowed accordingly.
Enforcement follows the decision: `agent-audit`'s plugin-scope validator reads its allowlist as
data from the `apm-agent-allowlist` section of
`plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md`, and that line now reads
`plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (now
`factory-audit/references/agent-field-inventory.md`, see ADR-0025), and that line now reads
`name description model source_keys disallowedTools`. `disallowedTools` also stays in that file's
`claude-code-only-fields` list, which is not a contradiction — that list governs whether a field
may cross the CC/Copilot boundary in a real project/user-scope *pair*, a different question from

View File

@@ -65,7 +65,9 @@ Three sub-decisions inside that:
## Consequences
**Skills gain an unnamespaced name.** apm deploys plain project skills, so `git:git-commits` also
answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit`. This is not configurable —
answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit` (**skill renamed per ADR-0025,
2026-09-15:** that skill is now `factory-audit`, so the live example is `kyberforge:factory-audit` to
`factory-audit`; the rule is unchanged). This is not configurable —
a project skill has no plugin to prefix. `AGENTS.md` and `CONTEXT.md` are updated to name the bare
form, which is what apm deploys and the only form a repo consuming holocron through apm gets.

View File

@@ -8,6 +8,23 @@ gates that hold them.
**Status: accepted (2026-08-14).**
**Amended by ADR-0025 (2026-09-15).** The contract, the tiers and every verdict rule below stand
unchanged. What moved is the **number and location of the scripts that carry them**. This ADR names
three: `scripts/skill-size-check.sh`, `skill-audit/scripts/validate.sh` and
`agent-audit/scripts/validate.sh` — "all three validators" (Decision), "all three scripts"
(Enforcement table footnote), "`scripts/skill-size-check.sh` and its two mirrored copies"
(the `_add()` amendment). ADR-0025 merged the two audit skills, so there are now **two**: the root
`scripts/skill-size-check.sh`, which still embeds the 1,061-line block between `BEGIN`/`END ADR-0020
SHARED BOUNDARY RESOLVER` markers, and one plugin copy — extracted out of the merged validator into
`plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh` and sourced by
`factory-audit`'s `validate.sh` and `validate-provenance.sh` rather than pasted into each. The
Enforcement table's "constants mirrored in `skill-audit/scripts/validate.sh` and
`agent-audit/scripts/validate.sh`" is one path now, `factory-audit/scripts/validate.sh`, which
auto-detects the artifact type; the skills/agents columns are unaffected, since the merged validator
applies the body tiers on the skill path only. The two copies must still stay byte-identical — a
plugin script cannot source the root one, which is why a second copy exists at all. Read every
"three" below as the count at the time of writing.
## Context
Every `file:line` citation in this ADR is against the base commit the decision was taken on,
@@ -189,7 +206,8 @@ Agents take the same description gates — they are preloaded identically — an
A skill body is loaded into the caller's context, competing with the live conversation; an agent body
becomes the system prompt of a fresh context. The rationale for the 900-word FAIL does not transfer.
That exemption is expressed in `agent-audit/scripts/validate.sh`, which has no body constant, and in
That exemption is expressed in `agent-audit/scripts/validate.sh` (now `factory-audit`'s
auto-detecting `validate.sh` on its agent path, see ADR-0025), which has no body constant, and in
the `files:` pattern of the `skill-size-check` pre-commit hook, which is `SKILL.md`-only. It is *not*
expressed in `scripts/skill-size-check.sh` itself, which measures whatever path it is handed —
running it directly over `plugins/*/.apm/agents/*.agent.md` exits 1 with 900-word body FAILs on
@@ -201,7 +219,8 @@ file pattern, not by the script knowing the difference. Anyone widening that pat
would silently enforce a gate this ADR declines to set.
A plugin-scope agent is a single file with no sibling `references/` directory, so it cannot disclose
to itself — it can only delegate to skills. `agent-audit` therefore gains a **delegation check**: an
to itself — it can only delegate to skills. `agent-audit` (now `factory-audit`'s agent flow, see
ADR-0025) therefore gains a **delegation check**: an
agent body that restates a procedure owned by a skill it can invoke is a FAIL, with the fix being
"invoke `<skill>` instead". Length falls out of delegation rather than being gated directly.
@@ -232,18 +251,20 @@ type of input they take should be **one skill with a dispatch table**. This catc
one-or-two-file agent pair, per ADR-0005 and ADR-0016) and their overlap is in the improve flow
rather than the core job.
**DEFERRED — not implemented in the change that carries this ADR. Tracked as issue #101.** Both
skills still exist separately, and this change made the split deeper rather than shallower: retrofit
**DEFERRED when this ADR was written — not implemented in the change that carries it. Tracked as
issue #101. IMPLEMENTED by ADR-0025 (2026-09-15), which merged the pair into `factory-audit` with a
Step 0 dispatch and closed the deferral.** At the time of writing both
skills still existed separately, and this change made the split deeper rather than shallower: retrofit
to the dispatch pattern took `skill-audit` from 3 reference files to 7 and `agent-audit` from 4 to 8,
and their two same-named `references/description-quality.md` files now differ on 100 of ~120 lines
after normalising `skill`/`agent`, where before they were closer. It has kept deepening since: the
#99 retrofit added `finding-criteria.md` to `skill-audit`, drawing it level with `agent-audit`. Both
figures move with the next retrofit, so measure rather than quote —
`ls plugins/kyberforge/.apm/skills/<name>/references/ | grep -c '\.md$'`. The merge stays the
`ls plugins/kyberforge/.apm/skills/factory-audit/references/ | grep -c '\.md$'`. The merge stayed the
decision; it reopens ADR-0008 (agent-audit's single-file invocation contract) and touches every call
site in `skill-author`, `agent-author` and `forge`, which is why it is its own change and not a rider
on this one. Recorded here rather than dropped, so the gap between the rule and the tree is deliberate
and dated instead of discovered later.
site in `skill-author`, `agent-author` and `forge`, which is why it was its own change and not a rider
on this one. Recorded here rather than dropped, so the gap between the rule and the tree was deliberate
and dated instead of discovered later — and ADR-0025 is where it was closed.
### Enforcement and rollout
@@ -266,7 +287,7 @@ which tier each rule is in, because the failure this ADR is most exposed to is a
| description opener, composition notes in a description | skills, agents | prose pattern | `plugins/kyberforge/.apm/skills/*/assets/vale/styles/Kyberforge/` |
| a Gotcha paraphrasing a body step | skills | **auditor judgment** | `references/body-discipline.md` |
| dispatch at two or more mutually exclusive flows | skills | **auditor judgment** | `references/body-discipline.md` |
| delegation: an agent body restating a skill's procedure | agents | **auditor judgment** | `agent-audit` |
| delegation: an agent body restating a skill's procedure | agents | **auditor judgment** | `agent-audit` (now `factory-audit`'s agent flow, see ADR-0025) |
| capability enumeration, restatement, trigger quality | skills, agents | **auditor judgment** | `references/description-quality.md` |
The rows in bold are stated as FAILs in the Decision above and are FAILs an *auditor* issues. None of
@@ -399,16 +420,18 @@ over the same 39 files now reports 0 errors, 0 warnings and 0 suggestions, so
independent of `skill-size-check`, so a new description can reintroduce it; `skill-size-check` does
not cover the Vale half, and no `references/` file is linted by anything (`docs/spec/gates.md` has
both causes, issue #117 tracks them). Re-derive rather than quote —*
`bash plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh plugins/*/.apm/skills/*/SKILL.md`.
`bash plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh plugins/*/.apm/skills/*/SKILL.md`
*(path re-pointed by ADR-0025; the `skill-audit` copy this ADR originally named no longer exists).*
**A ceiling does not produce an average.** If every author writes to the 400-character FAIL, the
preload lands at 39 × 400 = 15,600 chars — a 33% cut off 23,427, not the ~50% intended. Writing to
the 250-character SUGGESTION instead lands at 9,750, a 58% cut. The halving depends entirely on the
250-character SUGGESTION tier being visible and respected. That tier works here in a way it does not
elsewhere in this repo: `skill-audit` already reports `PASS (N suggestions)` as a first-class
elsewhere in this repo: `skill-audit` (now `factory-audit`'s skill flow, see ADR-0025) already
reports `PASS (N suggestions)` as a first-class
outcome. This is explicitly **not** the failure ADR-0013 records — Vale warnings are invisible
because vale's exit code keys on `error` alone, but these gates live in `validate.sh` and
`skill-audit`, where a SUGGESTION reaches the report. Realistic landing is somewhere in that 33-58%
because vale's exit code keys on `error` alone, but these gates live in `validate.sh` and the
audit skill itself, where a SUGGESTION reaches the report. Realistic landing is somewhere in that 33-58%
band, not a guaranteed 50%.
**A word gate cannot detect the defect it is standing in for.** `git-commits` carries twelve Gotchas

View File

@@ -83,7 +83,8 @@ unnamed in `git`'s corrected description, though `65bac15`'s own commit message
**Nothing checks any of this.** `scripts/check-manifests.sh` does not contain the string
`description`. The three ADR-0020 validators (`scripts/skill-size-check.sh` and skill-audit's and
agent-audit's `validate.sh`) gate on SKILL.md and agent frontmatter; they do open `apm.yml`, but only
agent-audit's `validate.sh` — two since ADR-0025 merged the audit pair into `factory-audit`, whose
single auto-detecting `validate.sh` carries both) gate on SKILL.md and agent frontmatter; they do open `apm.yml`, but only
to read `dependencies.apm` when resolving the boundary-target universe — none of them reads the
`description:` key, and their hook globs match `SKILL.md` and `*.agent.md` only. `apm audit --ci`,
`apm pack --check-clean` and `scripts/sync-plugin-content.sh --check --all` all compare compiled

View File

@@ -0,0 +1,376 @@
# `skill-audit` and `agent-audit` merge into one `factory-audit` with a Step 0 dispatch
**Status: accepted (2026-09-15).** Implements ADR-0020's "Merging siblings" rule, which named this
exact pair, scoped itself to them, and then deferred the work as issue #101. The deferral is closed
here. `skill-author` and `agent-author` stay separate — ADR-0020 excluded the author pair
deliberately, and nothing in this change touches that exclusion.
## Context
Every figure below was measured against the worktree on 2026-09-15. Re-derive rather than quote; the
commands are given where a number is load-bearing.
The two skills duplicate content because they cannot share a file.
`plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md`, sourced from the
agentskills.io spec, states the constraint for APM package mode: file references inside
`.apm/skills/<name>/` must not reach outside that skill's own directory, and the spec defines no
cross-skill sharing mechanism. apm deploys skills flat into `.claude/skills/<name>/` with no plugin
tier above them, so there is no directory a second skill could read from. ADR-0024 re-confirmed this
after deleting the native install path, specifically to stop the constraint being re-litigated as a
Claude-Code artifact. It is not one. It binds under the only install path that survives.
What that constraint costs, between these two skills:
| Duplicated artifact | Lines | Verification |
|---|---|---|
| ADR-0020 boundary resolver, embedded in both `validate.sh` copies | 1,061 | marker block `115..1175` (skill) and `189..1249` (agent); `tests/test-adr0020-contract.sh` assertion 1 hashes them |
| `scripts/vale-wrap.sh` | 526 | `diff -q` clean |
| `assets/vale/styles/Kyberforge/`, five rules | 44 | `diff -r` clean |
| Contributing-files parser, embedded in both `validate-provenance.sh` copies | 93 | marker block `300..392` (skill) and `294..386` (agent); `diff -q` clean on the extracted blocks |
| **Removable by merging** | **1,724** | |
On top of that, `scripts/check-vale-style-sync.sh` (413 lines) and
`tests/test-check-vale-style-sync.sh` (797 lines) go with the merge. That is **not because the whole
gate was a copy diff**; it was not, and saying so would overstate the case for deleting it. The
script has **17 assertion sites**: 13 `err` calls and 4 hard-fail exits. Its closing
`exit 1` only reports the `err` count, so it is not an assertion. Count them with
`git show 61b0b9c^:scripts/check-vale-style-sync.sh`. An earlier revision of this ADR said 18. No
reproducible counting rule gives 18, and it is corrected here.
| Class | Old line | What it asserted | Now |
|---|---|---|---|
| **Moot (6)** | 19 | `REPO_ROOT` is a directory | nothing to guard; no script |
| | 42 | the `.apm/` paths are not stale | no copies to locate |
| | 52, 56 | neither copy is missing | one copy |
| | 60 | the two `vale-wrap.sh` copies are identical | one copy |
| | 64 | the two `styles/Kyberforge/` copies are identical | one copy |
| **Rehomed (10)** | 95, 113 | `.vale.ini` exists and is readable | case 0 |
| | 122 | `StylesPath = styles` is set | case 0 |
| | 127 | some section's `BasedOnStyles` names `Kyberforge` | case 28 (Part B proves it fails) |
| | 174 | no Kyberforge rule is overridden below a bare `YES`/`error` | case 31 |
| | 191 | `KyberforgeCopilot` ships and is loaded | case 30 |
| | 308 | `vale` is installed | suite-level: exit 77, which `run-tests --strict` fails |
| | 341 | every probe path matches some vale hook's `files:` regex | case 28 |
| | 347 | every probe path raises a Kyberforge alert under the config | cases 28 and 29 |
| | 397 | at least one probe row was checked | case 28's section floor |
| **Ported (1)** | 343 | local and published `files:` regexes agree per probe | case 33 |
Six are moot. Two diffed the copies, and four guarded the script's own ability to locate them: a
real `REPO_ROOT`, non-stale `.apm/` paths, and both copies present. With one copy and no script
there is nothing left to diff or locate. The other ten read `.vale.ini`, the style directory and
the hook manifests on their own terms, so they are **rehomed rather than retired**, into `tests/test-vale-wrap.sh`, which already owns the
wrapper's behaviour against this config.
Two rehomed checks got stronger, because a text grep became a behavioural Vale probe:
- line 127: case 28 Part B drops `Kyberforge` from a copy and requires vale to report the style as
not loaded;
- line 191: case 30 requires the Copilot rule to fire on `.agent.md` and nowhere else, and Part B
proves both an unload and a leak fail.
The rest moved at equal strength. Case 31 is the same grep as before. An earlier revision claimed
"7 of 11 stronger"; that claim is withdrawn.
**The cross-manifest check is ported, not dropped.** It extracts each vale hook's `files:` regex
from `.pre-commit-hooks.yaml` (the external-facing manifest) and from `.pre-commit-config.yaml`
(this repo's own copy of the same hooks) *independently*. It then asserts that a probe path in scope
of one is in scope of the other. That catches this repo narrowing its local hook without narrowing
the published one, or the reverse.
The original selected each hook's record by matching `entry:` against the owning skill's
`scripts/vale-wrap.sh` path. After the merge both vale hooks point at the same entry, so that
selector can no longer tell them apart. Case 33 pairs the hooks by `id:` instead, from an explicit
table: `kyberforge-vale-audit-skill` ↔ `vale-audit-prefilter-skill`, and
`kyberforge-vale-audit-agent` ↔ `vale-audit-prefilter-agent`. It carries the original six probe rows
unchanged. It also fails by name on a missing hook id, and on a class with no shared probe.
An earlier revision of this ADR shipped *without* that port and called the gap half-closed by case
32. It was not. Narrowing `vale-audit-prefilter-skill` from `^plugins/[^/]+/...` to
`^plugins/kyberforge/...` still matches tracked files of the right class. That clears case 32 while
silently dropping every other plugin's skills from this repo's prefilter, and it was measured
leaving the whole suite green. Case 33's Part B now makes exactly that mutation, the agent-hook
equivalent and a renamed hook id, and requires each to fail.
Case 32 stays, for the separate zero-match question: each local hook must still select at least one
tracked file, and only files of its own artifact class.
**Line count.** Merging removes the 1,724 duplicated lines above. Deleting the two sync-gate files
(413 + 797 = 1,210) removes more, for **2,934 lines** in total, plus one pre-push hook,
`check-vale-style-sync`, formerly at `.pre-commit-config.yaml:166`. Two smaller deletions are not in
that figure:
- `scripts/sync-vale-styles.sh` (21 lines), the helper that regenerated skill-audit's copy from
agent-audit's, now has nothing to sync.
- `agent-audit/scripts/README.md` (47 lines) has no successor. Nothing referenced it, and the only
README `references/skill-file-structure.md` mandates is `tests/README.md`, which survives.
The duplication is not symmetrical across the whole tree, and the asymmetry is what shapes the
decision. Outside the shared resolver the two `validate.sh` copies total 1,293 lines (616 skill, 677
agent) and have **91 distinct lines** in common. The two validators are not one script with a mode
flag; they are two genuinely different scripts that happen to embed one identical block.
The bodies are the binding constraint on the merge. `skill-audit`'s body is 724 words and
`agent-audit`'s is 808 — 1,532 together against `BODY_MAX_WORDS = 900`. Only 211 words are common
to both (47 byte-identical body lines). A merged body that simply concatenated the two flows would
fail the gate its own plugin enforces by a factor of 1.7, and there is no trimming route to 900:
1,321 of the 1,532 words are flow-specific.
Both skills already carry `category: factory` in their metadata, and both carry five
`source_keys` — ten in total, disjoint, because they audit against different specs.
## Decision
**The two skills become one, named `factory-audit`.**
**1. The name.** `factory` is what both already declare as their `category`, so the merged skill is
named for the thing it audits rather than for the two input types it now dispatches between. Two
alternatives were live and both are rejected below for naming reasons rather than substance:
`audit` collides with the unrelated `agentsmd-audit`, and `forge-audit` makes a bare skill name a
family prefix of a live sibling — `forge` exists at
`plugins/kyberforge/.apm/skills/forge/`. No bare skill name may be a family prefix of another.
**2. `SKILL.md` becomes a dispatch body.** Steps 1-3 move out to `references/skill-flow.md` and
`references/agent-flow.md`. The body carries the Gotchas that apply to both branches, the dispatch
table, and Step 4 — Report, which is shared. This is ADR-0020's own rule ("Dispatch is mandatory at
two or more mutually exclusive flows") applied to the file that defines it, and the word arithmetic
above is why it is mandatory here rather than stylistic.
**Dispatch happens at Step 0, keyed on the target path, before Step 1 runs.** The table accepts
exactly the shapes `scripts/validate.sh` detects:
- A directory containing `SKILL.md`, or a `SKILL.md` file (its parent directory is audited), takes
the skill flow.
- A `*.agent.md` file, or a `.md` file whose *immediate* parent directory is `agents/`, takes the
agent flow.
- Anything else stops, runs no validator, and names the two accepted shapes.
Putting the dispatch after any deterministic check would mean running the wrong validator first and
reading its output as a finding. An earlier revision of the body carried a two-row table with no
fallback row. It could not route a `SKILL.md` file path, a trigger its own description advertised.
Its agent row ("a path under `.apm/agents/`… or an agent markdown file") was both wider than the
script and circular.
**3. One entry point per script, auto-detecting, with the mode-specific half sourced.**
- `scripts/validate.sh` detects the target type itself, then sources `scripts/lib-boundary-resolver.sh`
and one of `scripts/lib-checks-skill.sh` / `scripts/lib-checks-agent.sh`.
- `scripts/validate-provenance.sh` does the same, sourcing `scripts/lib-contributing-files.sh` and
one of `scripts/lib-provenance-skill.sh` / `scripts/lib-provenance-agent.sh`.
Two things justify this shape. First, **self-containment binds between skills, not within one.** The
resolver had to be embedded verbatim in three copies because three skill directories cannot read
each other's files; two files inside one skill directory have no such problem. Sourcing is available
the moment the directory boundary between them disappears. Second, **a single auto-detecting entry
point makes a Step 0 misdispatch detectable.** The script re-detects the flow from the target, so
even after a misdispatch it runs the right checks and its finding tiers are correct. That alone does
not make the misdispatch self-correcting, and an earlier revision of this ADR wrongly said it did.
The flow file drives Steps 2-4, so a misdispatched audit would still apply the wrong Step 3 rubrics,
print the wrong coverage line and recommend the wrong author skill. The body closes that gap with an
explicit guard under the Step 0 table: if `validate.sh` reports on the other artifact type than the
row taken, discard the run and restart at Step 0.
**4. Reference files are prefixed by flow, with one exception.** Every flow-specific file becomes
`skill-*` or `agent-*` — `skill-description-quality.md`, `agent-description-quality.md`,
`skill-finding-criteria.md`, `agent-finding-criteria.md`, and so on. The exception is `sources.md`,
which stays singular and carries all ten `source_keys`, because the skill-side provenance check
hard-codes `os.path.join(skill_dir, "references", "sources.md")` (pre-merge
`skill-audit/scripts/validate-provenance.sh:180`, now `scripts/lib-provenance-skill.sh:215`). A per-flow sources file would mean
changing the provenance contract to get a cosmetic gain.
**5. Both exported Vale hook IDs survive unchanged.** `.pre-commit-hooks.yaml` keeps
`kyberforge-vale-audit-skill` and `kyberforge-vale-audit-agent`, keeps both `files:` regexes
(`(^|/)SKILL\.md$` and `(^|/)agents/[^/]+\.md$|\.agent\.md$`), and re-points both `entry:` lines at
the one surviving `vale-wrap.sh`. Nothing in the published hook-repo contract changes: an external
consumer's `.pre-commit-config.yaml` keeps working byte-for-byte across the merge. Two IDs pointing
at one script is not a redundancy — it is what keeps the two `files:` scopes addressable
independently, which is exactly ADR-0014's "one hook per file-scope" finding.
**6. `tests/test-adr0020-contract.sh` changes in three ways, and the third is a conversion, not a
deletion.** Assertion 1 drops from three resolver copies to two: the merged `factory-audit` holds
one, and `scripts/skill-size-check.sh` keeps its embedded copy. A new assertion 1a gives the
resolver the same protection 1b already gave the parser. It asserts that `validate.sh` sources
`lib-boundary-resolver.sh` in both mode branches, and that the resolver's BEGIN marker and
`def _authoring_root(` appear in exactly those two files and nowhere else. A byte-identity hash alone
would miss a third pasted copy, or an entry point that quietly stopped sourcing the library. Sourcing the resolver from the
plugin tree into `skill-size-check.sh` was considered and refuted — that script is a repo-root hook
consumed through `.pre-commit-hooks.yaml`, where `entry[0]` is the only token pre-commit rewrites,
so it cannot reach a file inside the plugin at a path any consumer has. Assertion 1b is **converted**:
it stops pinning that two `validate-provenance.sh` copies of the Contributing-files parser are
byte-identical, and starts pinning that `lib-contributing-files.sh` is a single sourced copy that has
not been re-inlined into either mode library. The claim it protects is the same one — the parser has
exactly one authority — stated against the new structure. The drift history behind it is smaller than
an earlier revision of this ADR implied. `484357a` (2026-08-30) added the bullet-form parser to both
copies with two different spellings of the loop: a temporary `rest` in skill-audit and an inline
slice in agent-audit. The two were behaviourally identical. `598a7c3` (2026-09-01) unified the
spellings and added the `SHARED CONTRIBUTING-FILES PARSER` markers that 1b hashed. From then until
the merge's parent the two marker blocks were byte-identical (`md5 0857272d…` both). So the parser
never *parsed* differently. What the gate never covered was the prose around the block, and a
docstring there asserted identity the loop did not have. One sourced library removes the question.
**7. Two things this change does not do.** `skill-author` and `agent-author` are **not** merged
here. That remains an open finding and it is unmeasured; ADR-0020 excluded the pair on the grounds
that they emit genuinely different artifacts, and nothing measured in this session revisits that.
And **no audit criterion changes.** Every check, tier, threshold, regex and branch is carried across
as-is. The Python payloads reassembled from the new libraries differ from the pre-merge heredocs only
in comments. The one exception is three lines naming `references/agent-field-inventory.md`, a
byte-identical rename of `field-inventory.md`. Byte-level differential runs over every live skill
directory and agent file matched stdout, stderr and exit code.
**The entry points are not behaviour-neutral, and an earlier revision of this ADR said they were.**
Those differential runs used valid targets only, so they could not see that the new detection layer
changed what happens to *invalid* ones. Every change below is deliberate:
| Input | Pre-merge | Now |
|---|---|---|
| a missing path, a directory with no `SKILL.md`, a non-agent `.md` (e.g. `README.md`) | exit 1, or a mode-specific exit-2 message | **exit 2** with one generic "matches neither" Error/Why/Fix. Exit 2 is the never-ran tier, so the flow files report the section as unverified and quote the reason. |
| a `SKILL.md` file path | exit 1 or 2 (`…/SKILL.md/SKILL.md not found`, "not a directory") | **accepted**; its parent directory is audited |
| an agent `.md` *not* under an `agents/` directory (e.g. `~/drafts/my-agent.md`) | audited | **refused, exit 2**. Detection never guesses. No tracked file in this repo is affected. |
| a bare or `./`-relative agent filename, run from inside its `agents/` directory | audited | audited. The parent directory's name is read from the real path, not the typed string. |
| a `lib-*.sh` missing or unreadable, or the script directory unresolvable | did not apply (single file) | **exit 2** with Error/Why/Fix, never a raw bash error at exit 1, which is the real-findings tier |
| `CDPATH` exported | did not apply (no `cd`) | no effect. `SCRIPT_DIR` resolves with `CDPATH=''` and `cd -- … >/dev/null`. |
| no argument | `Error: skill-dir is required.` / `agent-file is required.` | one combined message and usage block; exit code unchanged (1 from `validate.sh`, 2 from `validate-provenance.sh`) |
A single `validate.sh` copied or symlinked out of its `scripts/` directory still does not work,
because its libraries are not beside it. It now fails at exit 2 and says so.
## Considered options
**Keep two skills and rely on the byte-identity contract test alone (rejected).** This is the status
quo: `tests/test-adr0020-contract.sh` already hashes the resolver across copies, and
`check-vale-style-sync.sh` already diffs the Vale halves at pre-push. Only 6 of its 17 assertion
sites exist because there are two copies. The other 11 do other work, and are rehomed or ported above rather than being an
argument for the status quo. On the duplication itself it polices drift rather than removing the thing that drifts,
and it pays 2,934 lines plus a pre-push hook to do so. It also leaves
the router carrying a mutually-excluding near-miss pair whose two descriptions each spend a boundary
clause pointing at the other — a routing cost the merge removes for free. ADR-0020 already weighed
this option for this pair and chose merging; nothing measured since changes the balance.
**One monolithic dispatching `validate.sh` (rejected).** Dropping one resolver copy from the
concatenation of the two current files gives roughly 2,354 lines in a single script. It is the
straightforward reading of "merge the scripts", and it is wrong on the evidence: the two validators
share only 91 distinct lines outside the resolver, so a monolith would be two near-disjoint
implementations behind one `if`, with every future edit to either half requiring a reader to hold
both in context. Sourcing per-mode libraries gets the same single entry point and keeps the halves
readable apart.
**Genuinely merging the three colliding reference files into two-section files (rejected).**
`description-quality.md`, `finding-criteria.md` and `validation-scripts.md` exist under both skills
today, and folding each into one file with a skill section and an agent section is the tidier-looking
outcome. It defeats the dispatch. The entire point of moving Steps 1-3 into `references/` is that an
invocation loads one flow's content and not the other's; a two-section reference file re-inflates
per-invocation context to the full 1,532-word span the body ceiling forced out. ADR-0020 measured
these same files at 100 of ~120 differing lines after normalising `skill`/`agent`, so the merged file
would also be mostly disjoint text under one heading.
**Naming it `audit` (rejected).** Shortest available name and an accurate one. It collides with
`agentsmd-audit`, which audits a repo's `AGENTS.md` and has nothing to do with the factory. A bare
`audit` alongside it reads as the general case of a skill it is unrelated to, which is precisely the
routing confusion a merge is supposed to reduce.
**Naming it `forge-audit` (rejected).** It matches the plugin and reads well. `forge` is a live skill
in the same plugin, so `forge-audit` makes one bare skill name a prefix of another — a router asked
to distinguish `forge` from `forge-audit` is being asked to disambiguate on a suffix, and a user
typing `forge` gets an ambiguity that does not exist today.
**Collapsing the two exported Vale hook IDs into one (rejected).** With a single `vale-wrap.sh` and a
single `.vale.ini`, one hook ID looks sufficient. It is a breaking change to a published hook-repo
contract: any external repo pinning `kyberforge-vale-audit-agent` breaks on upgrade, for no gain.
It also re-creates ADR-0014's measured failure in a new place — that ADR confirmed empirically that a
single hook entry pointed at one config silently scanned 0 files of the other type. Two IDs cost two
manifest stanzas and keep both file scopes explicit.
## Consequences
**The single-file `.vale.ini` comes back, and this does not reverse ADR-0014.** ADR-0014 split one
root config into two skill-scoped copies because two skills each needed their own, and no
plugin-level shared directory exists to hold one. Its reasoning is untouched; the merge removes the
condition that reasoning operated on. One skill needs one config, so the union is written back into
one file. **The union is behaviour-neutral and this was checked rather than assumed:** skill-audit's
config has a single `[**/SKILL.md]` section, agent-audit's has `[**/agents/*.md]` and
`[**/*.agent.md]`, and no file in the corpus matches more than one of the three. Where an overlap is
constructible at all (`agents/SKILL.md`), both matching sections assign `BasedOnStyles = Kyberforge`,
so even then no verdict moves. `KyberforgeCopilot` stays scoped to `[**/*.agent.md]` exactly as it is
now, which is what keeps the merged config from widening Copilot-specific rules onto `SKILL.md`.
**`scripts/check-scope-walkup-sync.sh` survives, and confusing it with `check-vale-style-sync.sh` is
the obvious mistake here.** The two look like the same kind of gate and are not. The walk-up checker
covers **four** independent ports of the scope walk-up, and only two of them live in the audit pair:
the other two are `agent-author/scripts/new-agent.sh` and `skill-author/scripts/new-skill.sh`, which
this change does not touch. They are also Bash where the audit pair's are Python, so as its own header
records, it can never become a text diff — it asserts behavioural agreement across a fixture matrix
instead. Merging two of four ports leaves three ports and the same job.
**Roughly 71 files carry inbound references to the two skill names and must be re-pointed.**
Derived as `git grep -l -E "skill-audit|agent-audit" | wc -l` — it includes ADRs, `LESSONS.md`,
`docs/spec/gates.md`, both author skills' routing targets, `forge`'s dispatch, the test suite and the
two manifests. Boundary clauses naming `skill-audit` or `agent-audit` are the sharp end: ADR-0020's
resolvable-target check is a blocking ERROR on a dangling route, so a missed rename fails the push
rather than degrading quietly. Historical references inside ADRs describing the pre-merge state stay
as they are; the resolver reads boundary clauses in descriptions, not ADR prose.
**The dispatch body carries only the gotchas common to both flows, and ships with no SUGGESTION.** An
earlier revision of this change shipped the Gotchas section at **229 of 548 body words, 42%**,
against `GOTCHA_MAX_BODY_FRACTION = 0.25`. It accepted that as standing output, arguing that moving
a gotcha to `references/` meant an extra file read on every invocation. That argument was wrong for
the two gotchas that were over budget, because neither was shared:
- the `Agent flow, plugin/APM scope only` provider-safety bullet names its one branch in its own
text;
- the 112-word body-word-gate bullet was two separate pre-merge gotchas welded together, a skill half
and an agent half.
A dispatch body is the dispatch table *plus the gates common to every branch* (CONTEXT.md; the
skill-flow rubric `references/skill-body-discipline.md`). Keeping a single-branch gotcha in it
contradicts that definition. Moving it into its flow file costs no read either, because the body
already loads exactly one flow file on every invocation by construction.
So the skill half now sits under `## Gotchas` in `references/skill-flow.md`. The agent half and the
provider-safety bullet sit under `## Gotchas` in `references/agent-flow.md`. The body keeps three
gotchas: the no-narration rule, the `disable-model-invocation` exemption and the Vale `0 files`
trap. Measured with `scripts/skill-size-check.sh` thresholds zeroed to force the figures out, the
section is now **91 of 555 body words, 16%**.
**Every invocation now reads one extra `references/` file.** The dispatch body names the flow file
and the agent loads it, where today Steps 1-3 arrive with the body. This is the cost the progressive-
disclosure trade always carries, and it is paid against a saving: an invocation loads the dispatch
body plus one flow instead of a body that would have to carry both. It is also the reason the
two-section reference file was rejected above.
**The original audit's figures for this finding were wrong in three ways, and each is worth naming
so the correction is not re-derived from scratch later.**
- It claimed roughly 3,300 duplicated lines and **two** pre-push hooks. The measured removal is
**2,934 lines and one hook**. The second hook it counted was `check-scope-walkup-sync`, which
survives for the reason above.
- It claimed the two validators were one script hard-wired per mode. They are not. Outside the shared
resolver they total 1,293 lines with 91 distinct lines in common. That error matters because it is
what made the monolithic `validate.sh` look like the obvious implementation.
- It named the **merged `description`** as the blocker on merging. It is not. Merging deletes
description content rather than accumulating it: the `Not a skill directory -> skill-audit` clause
loses its referent, and the `"is this ready to ship"` trigger was duplicated verbatim across both.
The two descriptions it replaces measure **239** (skill-audit) and **250** (agent-audit) at
`61b0b9c^`. The description this skill ships measures **241**, inside the 250 SUGGESTION target.
It carries one arrow per boundary target (`Not applying skill fixes -> skill-author. Not applying
agent fixes -> agent-author.`), because ADR-0020 resolves only the first target after an arrow, so
a one-arrow form would leave `agent-author` checked by nothing. The real blocker was the body: 1,532
words against `BODY_MAX_WORDS = 900`, with only 211 words shared. Diagnosing the description would
have produced a merge with a concatenated body that failed its own plugin's gate.
**Correction to an earlier revision of this bullet.** It shipped the description at **319**
characters and accepted the SUGGESTION. It said the excess paid for the second arrow and for
"both flows' artifact-specific trigger phrases carried in full". Only the arrow was worth it.
The trigger phrases stated one trigger twice in two registers: "a skill directory or agent
definition audited", then quoted `audit this skill`, `review my SKILL.md`, `audit this agent` and
`review my agent file`. ADR-0020 makes that a FAIL ("Stating the same trigger twice in two
registers is a FAIL"), so it was not a cost of merging. Dropping the quoted duplicates, and keeping
the one indirect trigger that omits the domain word (`is this ready to ship`), gives 241 with both
arrows kept. The same revision's "240 characters" figure for a hypothetical single-arrow merge was
never reproduced, and is withdrawn rather than re-derived.
**`factory-audit` shipped at `metadata.version: "1.0.0"`, not ADR-0022's `0.1.0` for a new skill.**
It is a new directory, but not a new skill in the sense ADR-0022's starting version encodes: it
carries every check, rubric and reference of two skills that were both already at `1.0.0`, and
resetting to `0.1.0` would signal an immaturity that the merged content does not have. The fixes
above to Step 0, the gotchas and the description are an improve pass, so under `skill-author`'s
patch-bump rule it is now **`1.0.1`**. The plugin itself goes from `1.6.2` to **`2.0.0`**, because
removing two invocable skills breaks anyone calling them by name.

View File

@@ -79,7 +79,7 @@ This repo also has a `CLAUDE.md` at its root — the Claude Code entry point for
## Reference conventions
The stated convention is that files referencing other files declare those references explicitly: the referencing file carries the forward reference (the content index in `core/AGENTS.md`, `references:` in frontmatter), the referenced file carries a `when:` field describing when it is loaded, and divergence between the two signals staleness. It is aspirational, not a description of the repo today — no file under `core/instructions/` carries frontmatter at all, `when:` appears in exactly one of the 39 `SKILL.md` sources under `plugins/*/.apm/skills/`, and the reference scanner script meant to derive the reverse map ("what files reference this file?") does not exist; `docs/notes/skill-implementation-workflow.md` still lists it as unbuilt work. Treat it as intent for instruction files, skills, and workflow documents, not as a rule the repo enforces.
The stated convention is that files referencing other files declare those references explicitly: the referencing file carries the forward reference (the content index in `core/AGENTS.md`, `references:` in frontmatter), the referenced file carries a `when:` field describing when it is loaded, and divergence between the two signals staleness. It is aspirational, not a description of the repo today — no file under `core/instructions/` carries frontmatter at all, `when:` appears in exactly one of the 38 `SKILL.md` sources under `plugins/*/.apm/skills/`, and the reference scanner script meant to derive the reverse map ("what files reference this file?") does not exist; `docs/notes/skill-implementation-workflow.md` still lists it as unbuilt work. Treat it as intent for instruction files, skills, and workflow documents, not as a rule the repo enforces.
## Provider model

View File

@@ -21,24 +21,24 @@ Install hooks via `pc-run`, wiring **all three stages**. This repo's `.pre-commi
`default_install_hook_types`, so a plain install silently skips `commit-msg` (Conventional Commits)
and `pre-push` (everything below).
The pre-push command reports **11** hooks, not 9. The extra two are pre-commit's own `meta` hooks,
The pre-push command reports **10** hooks, not 8. The extra two are pre-commit's own `meta` hooks,
`check-hooks-apply` and `check-useless-excludes`: they declare no `stages:`, so they run at every
stage including this one. Both are declared in this repo's `.pre-commit-config.yaml` like everything
else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Nine
else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Eight
is the count of hooks this repo authors itself.
**The caveat: one of those 9 is a silent no-op under that invocation.**
**The caveat: one of those 8 is a silent no-op under that invocation.**
`check-release-needed` exits 0 immediately unless `PRE_COMMIT_REMOTE_BRANCH` equals
`refs/heads/main`, and pre-commit exports that variable only from the real pre-push git hook during
an actual `git push`. Running the stage by hand — or from a CI runner — therefore reports it
`Passed` having checked nothing. That is by design for feature branches — pushing WIP must not be
blocked on cutting a premature tag — but it means `--hook-stage pre-push --all-files` is a full
rehearsal of 8 hooks and a skip of the ninth. The script's own header records the same gap for
rehearsal of 7 hooks and a skip of the eighth. The script's own header records the same gap for
a PR merged through Gitea's merge button, where no local push happens at all.
## The pre-push gate
Nine hooks, grouped below by what they guard rather than by the order `.pre-commit-config.yaml` declares them in.
Eight hooks, grouped below by what they guard rather than by the order `.pre-commit-config.yaml` declares them in.
**Core checks**
@@ -50,7 +50,6 @@ Nine hooks, grouped below by what they guard rather than by the order `.pre-comm
| Hook | Guards |
|---|---|
| `check-vale-style-sync` | skill-audit's Vale copy matches agent-audit's canonical copy, plus six glob-coverage probes (see [Vale](#vale)) |
| `check-scope-walkup-sync` | `validate.sh`, `validate-provenance.sh`, `new-agent.sh` and `new-skill.sh`'s four independent `$HOME`/`.git`/`apm.yml` walk-up ports still agree behaviorally |
| `check-executables-allow-sync` | root `apm.yml`'s `executables.allow` key names kyberforge's actual version (see [apm gates](#apm-gates)) |
@@ -61,7 +60,7 @@ drift in generated text.
| Hook | Guards |
|---|---|
| `check-apm-agents-valid` | runs agent-audit's `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md` (see [Agent files](#agent-files-take-the-description-gates-not-the-body-gate)) |
| `check-apm-agents-valid` | runs `factory-audit`'s `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md` (see [Agent files](#agent-files-take-the-description-gates-not-the-body-gate)) |
**apm's own gates**
@@ -204,9 +203,9 @@ gets promoted.
### Target resolution walk
Resolution walks up **from the file being checked** — never from the script's own location. Deriving
it from `${BASH_SOURCE}` leaked holocron's 39-skill universe into every consumer repo running the
hook through pre-commit, so a consumer skill routing to `skill-audit` resolved against a plugin it
had never installed.
it from `${BASH_SOURCE}` leaked holocron's own skill universe into every consumer repo running the
hook through pre-commit, so a consumer skill routing to a holocron skill (`skill-audit` at the time,
now `factory-audit`) resolved against a plugin it had never installed.
The walk finds an **authoring root**: the nearest ancestor holding `plugins/*/.apm/skills` or
`plugins/*/.apm/agents`, falling back to the nearest ancestor holding `.git`. **Two passes, not one
@@ -354,14 +353,19 @@ findings.
### Duplicated constants
`skill-audit`'s `validate.sh` holds a second copy of the four ADR-0020 constants
(`DESC_SUGGEST_CHARS` / `DESC_MAX_CHARS` / `BODY_SUGGEST_WORDS` / `BODY_MAX_WORDS`), and
`agent-audit`'s `validate.sh` holds a third copy of the two description constants. They are copied
rather than imported because a cache-installed plugin's scripts cannot read files outside their own
plugin directory. `tests/test-skill-size-check.sh` asserts the copies agree, so drift fails CI rather
than silently letting an audit bless a skill the commit hook then rejects. The shared boundary
resolver block is embedded verbatim in all three scripts between `BEGIN`/`END ADR-0020 SHARED
BOUNDARY RESOLVER` markers and must stay byte-identical.
`factory-audit`'s `validate.sh` holds a second copy of the four ADR-0020 constants
(`DESC_SUGGEST_CHARS` / `DESC_MAX_CHARS` / `BODY_SUGGEST_WORDS` / `BODY_MAX_WORDS`) — the two
description constants apply to both artifact types it handles, the two body constants only to
skills. They are copied rather than imported because a cache-installed plugin's scripts cannot read
files outside their own plugin directory. `tests/test-skill-size-check.sh` asserts the copies agree,
so drift fails CI rather than silently letting an audit bless a skill the commit hook then rejects.
**The shared boundary resolver is now two copies, not three** (ADR-0025). `scripts/skill-size-check.sh`
still carries it embedded between `BEGIN`/`END ADR-0020 SHARED BOUNDARY RESOLVER` markers; the two
plugin copies that used to sit inside `skill-audit`'s and `agent-audit`'s `validate.sh` collapsed
into the single `factory-audit/scripts/lib-boundary-resolver.sh`, sourced by that skill's scripts.
The two remaining copies must still stay byte-identical — a plugin script cannot source the root
one, which is the constraint that forces a copy to exist at all.
### `python3` and PyYAML are hard requirements
@@ -387,7 +391,7 @@ fold.
## Agent files take the description gates, not the body gate
`check-apm-agents-valid` runs agent-audit's `validate.sh` over every real
`check-apm-agents-valid` runs `factory-audit`'s `validate.sh` over every real
`plugins/*/.apm/agents/*.agent.md`. It derives its expected file set from `git ls-files` — the pattern
`tests/run-bats.sh` established — so an agent file deleted from the worktree but still tracked fails
the run, and **discovering zero agent files is an error, not a pass**. An untracked *new* agent file
@@ -399,12 +403,13 @@ against synthetic `mktemp` fixtures — it had never run against the agent files
how ADR-0016 could be amended to bless a `disallowedTools` frontmatter field while `validate.sh`'s
allowlist still rejected it: spec and enforcer disagreed and every gate stayed green.
Agents take the ADR-0020 **description** gates (agent-audit's `validate.sh` holds its own copy of
Agents take the ADR-0020 **description** gates (`factory-audit`'s `validate.sh` holds its own copy of
those two constants) and, deliberately, **no body word gate**. A skill body is loaded into the
caller's context and competes with the live conversation; an agent body becomes the system prompt of
a *fresh* context. The rationale for the 900-word FAIL does not transfer. A bats test pins that
absence in agent-audit's validator — adding a body gate there contradicts the ADR rather than fixing
an inconsistency.
absence for the agent path of `factory-audit`'s validator — adding a body gate there contradicts the
ADR rather than fixing an inconsistency. The merge did not change this: the validator auto-detects
the target type, and the body gate applies on the skill path only.
**Be precise about the scope of that guarantee: it holds for the *validator*, not for the shared
script.** `scripts/skill-size-check.sh` applies its body gate to whatever path it is handed, and
@@ -424,7 +429,7 @@ knows the difference; doing so silently enforces a gate ADR-0020 declines to set
## Current retrofit status
The ADR-0020 gates ship hot, with no baseline file — a shrinking baseline was considered and
rejected. The corpus is currently clean on both: 0 of 39 descriptions/bodies exceed their FAIL tier,
rejected. The corpus is currently clean on both: 0 of 38 descriptions/bodies exceed their FAIL tier,
0 dangling targets, 0 `Kyberforge.CompositionNote` (Vale) errors. History: issue #99.
Nothing is grandfathered — a new skill, or an edit that crosses a FAIL tier, is blocked on first
@@ -503,7 +508,7 @@ boundary, and a stricter form would only move the same trust to a different stri
in list items, not fences. Those are clause-1 sites the gate cannot see, because it cannot
distinguish them from clause-2 mentions in the same list.
- **`README.md`, excluded by pattern.** A skill-directory README is consumer-facing prose no agent
loads, and the `git clone https://github.com/bats-core/…` lines in the seven `tests/README.md`
loads, and the `git clone https://github.com/bats-core/…` lines in the six `tests/README.md`
files are setup instructions for a third party who has no `rtk`. Prefixing those would be actively
wrong, not merely noisy — see ADR-0023's consumer section.
- **Quoting.** The line splitter breaks on `;`, `|`, `&&`, `||`, `$(` and backticks without tracking
@@ -526,29 +531,92 @@ it was written for.
Install the `vale` binary — `brew install vale` (macOS), `snap install vale` (Linux),
`choco install vale` (Windows), or see <https://vale.sh/docs/vale-cli/installation/>. No `vale sync`
is needed: the `Kyberforge` styles are **committed** under
`plugins/kyberforge/.apm/skills/{skill-audit,agent-audit}/assets/vale/styles/`, not downloaded
packages (ADR-0014).
`plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/`, not downloaded packages
(ADR-0014).
### Two copies, one canonical
### One copy, one config
Wiring Vale as a deterministic prefilter for `skill-audit`/`agent-audit`'s Description dimension
(motivation: issue #84) is repo-specific, not part of the generic `lint` plugin, so it does not live
in `plugins/lint/` — and per ADR-0014 it no longer lives at the repo root either. It lives **twice**,
one copy per skill, both under `plugins/kyberforge/.apm/skills/`:
Wiring Vale as a deterministic prefilter for `factory-audit`'s Description dimension (motivation:
issue #84) is repo-specific, not part of the generic `lint` plugin, so it does not live in
`plugins/lint/` — and per ADR-0014 it no longer lives at the repo root either. It lives **once**,
under `plugins/kyberforge/.apm/skills/factory-audit/assets/vale/`, carrying both the `Kyberforge`
and `KyberforgeCopilot` styles and a single `.vale.ini` with all three glob sections:
`[**/SKILL.md]`, `[**/agents/*.md]`, `[**/*.agent.md]`.
| Copy | Styles | `.vale.ini` sections |
|---|---|---|
| `agent-audit/assets/vale/` — **canonical** | `Kyberforge`, `KyberforgeCopilot` | `[**/agents/*.md]`, `[**/*.agent.md]` |
| `skill-audit/assets/vale/` — smaller duplicate | `Kyberforge` | `[**/SKILL.md]` |
ADR-0014 split this into two skill-scoped copies because a plugin's cache-install copies only each
skill's own files and `skill-audit` could not reach across the skill boundary into `agent-audit`'s
copy. Merging the two audit skills removed the boundary, so the copy went with it and the single-file
`.vale.ini` ADR-0014 split apart is restored (ADR-0025). `KyberforgeCopilot` stays scoped to
`.agent.md` files alone, for the Copilot-only "`Use proactively` has no effect" check.
Duplicated rather than shared because a plugin's cache-install copies only each skill's own files —
there is no cross-skill sharing to point at. `check-vale-style-sync` at pre-push is what keeps them
from drifting; `KyberforgeCopilot` is the one deliberate inequality, being scoped only to `.agent.md`
files for the Copilot-only "`Use proactively` has no effect" check.
With one copy there is nothing left to diff, so the `check-vale-style-sync` pre-push hook,
`scripts/check-vale-style-sync.sh` and `tests/test-check-vale-style-sync.sh` are deleted — one hook
off the push gate. **Read what went with it, not just what became vacuous.** The script had 17
assertion sites. ADR-0025 maps each one; the short version follows.
**Genuinely moot (6):**
- the `vale-wrap.sh` diff and the `styles/Kyberforge/` diff, which compared two copies that are now
one;
- the four hard-fail guards that located those copies (`REPO_ROOT` is a directory, the `.apm/` paths
are not stale, neither copy is missing).
Its `StylesPath` and `BasedOnStyles` checks were **not** diffs. They were per-file greps of each
`.vale.ini`, so they survive: case 0 below checks that the config loads, and case 28 checks that the
`Kyberforge` style is actually loaded.
**Rehomed or ported (11).** The largest group is the **six-row glob-coverage probe table**, which
invoked `vale --config` on one representative path per file shape. It was the only assertion
anywhere that catches a `.vale.ini` glob typo (`[**/SKILL.md]` → `[**/SKILLS.md]`), the failure mode
where every other check stays clean while Vale lints zero files. One config does not make that
impossible: a typo in any one of the three sections still 0-file-skips that shape.
**Case 0** runs before any Vale-dependent case and needs no Vale binary. It asserts that the shipped
`.vale.ini` exists and is readable, sets a `StylesPath` that resolves to a directory, and names only
styles that ship. A config that cannot load used to surface as nine generic "vale printed no summary
line" failures across cases 28–31. It now fails once, names the cause, and holds the Vale-dependent
cases back.
The probes now live in `tests/test-vale-wrap.sh` (cases 28–30), rehomed against the merged config:
one representative path per file shape, each asserted to produce a Vale scan of more than zero files
*and* a Kyberforge alert (case 28). Case 28 also checks that each probe path is in scope of a
published vale hook, and that every `.vale.ini` section has a probe row. Its Part B drops
`Kyberforge` from each section's `BasedOnStyles` in a copy and requires that section's probes to
fail as "style not loaded". Case 29 is a mutation case: it typos each section in a copy of the
assets and requires that section's isolating probes to drop to zero. Case 30 asserts that
`KyberforgeCopilot` reaches `.agent.md` files alone. Its Part B requires both an unload (dropped from
`[**/*.agent.md]`) and a leak (added to `[**/SKILL.md]`) to fail. Case **31** is the third class that went with the
script and is not a glob probe at all: the per-rule override allowlist, which pins every Kyberforge
rule at a blocking bare `YES`/`error`. It is not redundant with the probes above — those key on
`Kyberforge.VagueWording` and `KyberforgeCopilot.ProactivePhrase`, so the other four rules
(`DescriptionOpener`, `PaddingPhrase`, `SentenceOpenerThereIs`, `CompositionNote`) can each be
overridden out of `error` underneath a passing probe. That gap is closed.
Two cases cover the hook manifests.
**Case 33** is the original's cross-manifest `files:` drift check, ported. It extracts each vale
hook's `files:` regex from `.pre-commit-hooks.yaml` and from `.pre-commit-config.yaml`
*independently*, compares them per hook and never as a union, and asserts that each shared probe path
is in scope of both or neither. The original selected each hook's record by matching `entry:`
against the owning skill's `vale-wrap.sh` path. After the merge both hook IDs share one `entry:`, so
the port pairs them by `id:` from an explicit table: `kyberforge-vale-audit-skill` ↔
`vale-audit-prefilter-skill`, and `kyberforge-vale-audit-agent` ↔ `vale-audit-prefilter-agent`. A
missing hook id or a class with no shared probe fails by name. Part B requires three mutations to
fail: the skill hook narrowed to one plugin, the agent hook narrowed the same way, and a renamed
local hook id.
This was briefly a real hole. Narrowing `vale-audit-prefilter-skill` from `^plugins/[^/]+/...` to
`^plugins/kyberforge/...` left 6 of 38 skills prefiltered, and the whole suite green, before case 33
existed.
**Case 32** covers the separate zero-match question on the local manifest alone. Each
`.pre-commit-config.yaml` vale hook's `files:` regex must still match at least one tracked file, and
every path it matches must be in that hook's own artifact class. A hook narrowed to zero files never
runs, and pre-commit reports no error.
### What Vale owns, and what stays LLM judgment
Eleven rule files across the two copies, six distinct rules:
Six rule files, six distinct rules:
| Rule | Vale scope | Bans | From |
|---|---|---|---|
@@ -591,7 +659,7 @@ analogue here — Vale has no tier to make audible.
### External consumers: the root `.pre-commit-hooks.yaml`
The root `.pre-commit-hooks.yaml` exposes both Vale copies (`kyberforge-vale-audit-skill`,
The root `.pre-commit-hooks.yaml` exposes two Vale hook IDs (`kyberforge-vale-audit-skill`,
`kyberforge-vale-audit-agent`) plus `kyberforge-skill-size-check`, so any external repo can enforce
the same rules with `repo: <this-repo-url>, rev: <tag>` in its own `.pre-commit-config.yaml`.
pre-commit clones the pinned rev into its own cache, independent of whether Claude Code or the
@@ -599,8 +667,14 @@ pre-commit clones the pinned rev into its own cache, independent of whether Clau
--all-files`. `skill-size-check` has no external asset dependency, so it needed no relocation under
ADR-0014 — only exposure.
**The two IDs survive the merge even though they now point at the same wrapper.** Both
`kyberforge-vale-audit-skill` and `kyberforge-vale-audit-agent` keep their IDs and their `files:`
regexes, because an external repo pins them by name in its own `.pre-commit-config.yaml` and
collapsing them to one would break every such consumer silently. What changed is only the `entry:`
target: both now name `factory-audit/scripts/vale-wrap.sh`.
This repo's own `vale-audit-prefilter-skill` / `-agent` hooks consume the **identical**
plugin-bundled copies via `repo: local`. Deliberately not a third root copy, and deliberately **not a
plugin-bundled copy via `repo: local`. Deliberately not a second root copy, and deliberately **not a
pinned self-reference** — a pinned self-reference would lint working-tree edits against the last
tagged release rather than against the change being made.
@@ -619,18 +693,22 @@ vendored research-corpus `SKILL.md` files match neither pattern (see
for `skill-size-check`), so prose findings surface only when you edit a file this repo actually
authors. Without the binary the hooks fail with a bare "command not found" and no install pointer.
**Two hooks, not one combined hook.** Both manifests split the prefilter in two precisely because a
single hook can point at only one copy, and that copy would silently 0-file-skip the other file
shape (see [A 0-file Vale run is NOT RUN](#a-0-file-vale-run-is-not-run)).
**Two hooks, not one combined hook — for a different reason than ADR-0014 gave.** The original
reason was mechanical: with a config per skill, a single hook could point at only one copy and would
silently 0-file-skip the other file shape (see
[A 0-file Vale run is NOT RUN](#a-0-file-vale-run-is-not-run)). One `.vale.ini` carrying all three
sections removes that constraint. The split stays anyway because the two IDs are an exported
contract external consumers pin by name, and because the `files:` regexes still have to differ —
each hook hands Vale only the file shape it is scoped to.
### The `.vale.ini` globs do no scoping
Each `.vale.ini`'s section globs are **path-agnostic** — `[**/SKILL.md]` for skill-audit's copy,
`[**/agents/*.md]` and `[**/*.agent.md]` for agent-audit's — and constrain filename *shape*, not
The `.vale.ini`'s section globs are **path-agnostic** — `[**/SKILL.md]`, `[**/agents/*.md]` and
`[**/*.agent.md]` — and constrain filename *shape*, not
location: Vale's `*` crosses `/`. A `SKILL.md` outside `plugins/` (a project-scope
`.claude/skills/foo/SKILL.md`, say) still matches `[**/SKILL.md]` and gets linted normally.
All scoping therefore comes from the pre-commit hook's own `files:` regex and from the audit skills
All scoping therefore comes from the pre-commit hook's own `files:` regex and from `factory-audit`
passing one explicit file per invocation. The two manifests scope **differently on purpose**:
| Manifest | `-skill` | `-agent` |
@@ -638,8 +716,10 @@ passing one explicit file per invocation. The two manifests scope **differently
| `.pre-commit-config.yaml` (pins this repo's layout) | `^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$` | `^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$` |
| `.pre-commit-hooks.yaml` (layout-agnostic for consumers) | `(^\|/)SKILL\.md$` | `(^\|/)agents/[^/]+\.md$\|\.agent\.md$` |
Narrowing a `.vale.ini` glob to a `plugins/`-shaped path to "tighten" it breaks the consumer case,
and `check-vale-style-sync`'s probe set is built to catch exactly that.
Narrowing a `.vale.ini` glob to a `plugins/`-shaped path to "tighten" it breaks the consumer case.
`check-vale-style-sync`'s probe set was built to catch exactly that; it moved to
`tests/test-vale-wrap.sh` with the hook's deletion, and two of the six probes exist specifically to
pin this location independence — see [One copy, one config](#one-copy-one-config).
### The blind spot: `references/` is unlinted, for two independent reasons
@@ -647,16 +727,19 @@ Every `references/*.md` file in the corpus is outside the prose gate. Count them
`git ls-files | grep -cE '^plugins/[^/]+/\.apm/skills/[^/]+/references/.*\.md$'` rather than reading
a figure here; it moves with every retrofit. This is the gap that matters most, because the context
contract's own remedy for an over-long body is to move prose **into** `references/` — the gate pushes
text across its own boundary and then stops watching it.
text across its own boundary and then stops watching it. `factory-audit` is the live example. Its
dispatch body keeps only the gotchas common to both flows, and the flow-specific gotchas live under
`## Gotchas` in `references/skill-flow.md` and `references/agent-flow.md` (ADR-0025). Handing both
flow files to `vale-wrap.sh` prints `0 errors … in 0 files` and exits 0.
**Closing either cause alone changes nothing.** There are two, and they are independent:
| Cause | Where | Effect on a `references/` file |
|---|---|---|
| the `Kyberforge` style is scoped `[**/SKILL.md]` | `skill-audit/assets/vale/.vale.ini` | matches no section, so Vale lints 0 files and exits 0 |
| the `Kyberforge` style is scoped `[**/SKILL.md]` | `factory-audit/assets/vale/.vale.ini` | matches no section, so Vale lints 0 files and exits 0 |
| the hook's `files:` regex is `^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$` | `vale-audit-prefilter-skill` in `.pre-commit-config.yaml` | the file is never handed to Vale at all |
Verified both ways. Handing skill-audit's `vale-wrap.sh` a reference file directly — bypassing
Verified both ways. Handing `factory-audit`'s `vale-wrap.sh` a reference file directly — bypassing
pre-commit entirely, so only the style scope is in play — prints `0 errors … in 0 files` and exits 0,
where the same wrapper on a `SKILL.md` reports `in 1 file`. And the hook's `files:` regex, applied to
`git ls-files`, selects only the skill-directory `SKILL.md` files scoped at the top of this page, so
@@ -672,8 +755,8 @@ The consumer manifest is a third axis and does not rescue this either: `.pre-com
### `vale-wrap.sh`, never bare `vale`
Both audit skills' Step 1 and both pre-commit hooks call **each copy's own**
`scripts/vale-wrap.sh`, not `vale`. It works around a confirmed **Vale 3.15.2** limitation:
`factory-audit`'s Step 1 and both pre-commit hooks call
`factory-audit/scripts/vale-wrap.sh`, not `vale`. It works around a confirmed **Vale 3.15.2** limitation:
`text.frontmatter.description` silently stops matching on most — not all — multi-line descriptions.
Verified by reproduction on a deliberately-bad fixture, not assumed:
@@ -718,8 +801,9 @@ alongside it would resolve against the cwd instead, yielding `E100 Runtime error
and exit 2 — which both skills' fallback misreads as "vale unavailable" and silently downgrades to
full LLM judgment.
`tests/test-vale-wrap.sh` regression-tests this against **skill-audit's** copy specifically: its
fixtures are all `SKILL.md`-shaped, and only skill-audit's `.vale.ini` carries that glob section.
`tests/test-vale-wrap.sh` regression-tests this against `factory-audit`'s copy — the only one left.
Its fixtures are all `SKILL.md`-shaped, and that copy's `.vale.ini` carries the matching glob section
along with the two agent ones.
### A 0-file Vale run is NOT RUN
@@ -736,25 +820,18 @@ clean.
### Pre-push
`vale` is a **pre-push** dependency too, not only pre-commit. `check-vale-style-sync` runs **six
glob-coverage probes** by invoking `vale --config` — one representative path per file shape the
prefilter is supposed to cover. They are the only assertions in the script that catch a `.vale.ini`
glob typo (`[**/SKILL.md]` → `[**/SKILLS.md]`), the failure mode where every text-level check stays
clean while vale lints zero files. As a warning this self-disabled on exactly that mutation and
exited 0, and since pre-commit swallows a passing hook's output the stderr line was never seen — the
hook reported `Passed`. Missing `vale` is therefore a hard failure here.
`vale` is still a **pre-push** dependency, but no longer through a hook of its own.
`check-vale-style-sync` — the hook that ran the six glob probes, and whose
`CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1` opt-out downgraded them audibly rather than skipping the
hook — is deleted with the second Vale copy (ADR-0025). The six glob probes survive it inside
`test-vale-wrap.sh`, so `run-tests --strict` is now the gate that runs them. That is also what keeps
`vale` a pre-push requirement: `test-vale-hooks-consumer.sh` exits 77 without the binary, and so does
`test-vale-wrap.sh` once its static cases pass, and a skip fails the push.
The opt-out is `CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1`, and **it is not `SKIP=`**: the hook
still runs and still asserts everything verifiable from file text, but the six probes do not, and its
summary says so explicitly —
```
Vale style sync check passed (text-level only, vale unavailable): … 0 glob probe(s) verified.
```
Use it only on a machine that genuinely cannot install `vale`, and read that line as "the glob axis
was not checked", not as a pass. The hook is `verbose: true` for exactly that reason — its clean
output is a single line, so it costs one line per push.
`test-vale-wrap.sh` without Vale skips only its Vale-dependent cases, not the whole suite. The cases
that are plain greps and awk over the config and the two hook manifests still run: case 0, 16, 26,
27, the static halves of 28, 31 Parts A and B, 32 and 33. A static failure exits 1, because a real
defect is not a setup error. Only an all-static-pass run exits 77.
### Mentioning banned phrasing without tripping the rule
@@ -780,9 +857,10 @@ run. The pre-push hook invokes the same script as `--strict` (`RUN_TESTS_STRICT=
where a skip **does** fail the push: at pre-push a skip means one of the documented dependencies is
absent on this machine, so the gate would otherwise report success having run fewer suites than it
appears to. Without `--strict` the gate once went green having verified 15 of 17 suites on a
vale-less PATH, with the skip list swallowed. Without vale, three suites skip —
`test-check-vale-style-sync.sh`, `test-vale-hooks-consumer.sh`, `test-vale-wrap.sh` — and the strict
failure names each one and what to install.
vale-less PATH, with the skip list swallowed. Without vale, two suites skip —
`test-vale-hooks-consumer.sh` and `test-vale-wrap.sh` — and the strict failure names each one and
what to install. (It was three until `test-check-vale-style-sync.sh` was deleted with its hook; see
[One copy, one config](#one-copy-one-config).)
`tests/run-bats.sh` derives the set of `.bats` files it expects from `git ls-files`, so a `.bats`
file deleted from the worktree but still tracked in the index fails the run rather than silently
@@ -903,6 +981,9 @@ this remote before any network call.
- `docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md`,
`docs/adr/0014-vale-prefilter-ships-from-the-plugin.md` — apm-generated manifests, committed Vale
styles
- `docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md` — the audit-pair merge that
collapsed the two Vale copies to one, removed the `check-vale-style-sync` hook, and took the shared
boundary resolver from three copies to two. It amends ADR-0014 and ADR-0020 on those points
- `docs/spec/architecture.md` — directory structure, install pipeline, what is generated and what is
hand-authored
- `.pre-commit-config.yaml` — the hooks themselves, with inline rationale comments

View File

@@ -1,90 +0,0 @@
---
name: agent-audit
description: >
Use when the user wants an agent definition audited — "audit this agent",
"review my agent file", "is this ready to ship" — or after hand-editing an
agent outside agent-author. Not applying fixes -> agent-author. Not a skill
directory -> skill-audit.
allowed-tools: Bash Read
metadata:
version: "1.0.2"
category: factory
source_keys:
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
## Gotchas
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
- Agents take the same 250/400-character description gates as skills and **no body word gate at all** — an agent body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer. Judge an over-long agent body through the delegation check, never by word count.
- At plugin/APM scope the agent is a single vendor-neutral file by design, so provider safety stops meaning Claude-Code-versus-Copilot field leakage there.
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
## Step 1 — Deterministic checks
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
```bash
bash scripts/validate.sh <agent-file>
bash scripts/validate-provenance.sh <agent-file>
bash scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
```
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, `references/field-inventory.md` missing — read `references/validation-scripts.md`; what these scripts measure is not reproducible by reading.
`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — a bad argument or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same 2 tier.
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
| Rule | Dimension |
|---|---|
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
## Step 2 — Read the agent and load its scope contract
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
| Scope | Read |
|---|---|
| plugin/APM | `references/scope-plugin-apm.md` |
| project, user | `references/scope-project-user.md` |
## Step 3 — Qualitative audit
Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
| Dimension | Rubric |
|---|---|
| description | `references/description-quality.md` |
| body, delegation, comment-discipline | `references/body-and-delegation.md` |
Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding.
## Step 4 — Report
Open with a coverage line naming every dimension checked. At project/user scope:
```text
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
```
At plugin/APM scope, drop `pair-consistency` — there is no pair to check.
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
Each finding:
```text
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
```
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run agent-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.

View File

@@ -1,90 +0,0 @@
---
source_keys:
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
# Sources
## context7-websites-code-claude
- **URL:** context7:/websites/code_claude
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
- **Status:** `extracted`
## claude-code-plugins-docs
- **URL:** https://code.claude.com/docs/en/plugins
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/validation-scripts.md
- **Status:** `extracted`
## claude-code-subagents-docs
- **URL:** https://code.claude.com/docs/en/sub-agents
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
- **Status:** `extracted`
## context7-github-en-copilot
- **URL:** context7:/websites/github_en_copilot
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
- **Status:** `extracted`
## github-custom-agents-configuration
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
- **Status:** `extracted`
## github-cli-plugin-reference
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-creating
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-finding-installing
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-marketplace
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands
- **Contributing files:** (none)
- **Status:** `extracted`
## github-sdk-custom-agents
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
- **Contributing files:** (none)
- **Status:** `extracted`

View File

@@ -1,47 +0,0 @@
# scripts/
Executable code bundled with this skill. Agents run scripts in this directory
to perform repeatable operations rather than reinventing the logic each run.
## When to add a script
Add a script when agents independently reinvent the same logic across runs —
building the same parser, chart, or validation routine from scratch each time.
Bundle it here once, tested and reliable.
## Script requirements (agentskills.io)
Scripts must be designed for non-interactive, agentic execution:
- **No interactive prompts** — agents run in non-interactive shells.
Accept all input via flags, env vars, or stdin. A script that blocks on
TTY input hangs indefinitely.
- **Expose `--help`** — this is how agents learn your script's interface.
Keep the output concise; it enters the agent's context window.
- **Structured output** — write data (JSON, CSV, TSV) to stdout.
Write progress, warnings, and diagnostics to stderr.
- **Idempotent** — prefer "create if not exists" over "create and fail on
duplicate". Agents may retry on failure.
- **Meaningful exit codes** — `0` for success, non-zero for failure.
Use distinct codes for different failure types; document them in `--help`.
- **Dry-run support** — add `--dry-run` for destructive operations.
## Self-contained scripts
Bundle dependencies inline so the agent can run the script with a single command.
Python (PEP 723 + uv):
```python
# /// script
# dependencies = ["requests>=2.31,<3"]
# requires-python = ">=3.11"
# ///
import requests
```
```bash
uv run scripts/my-script.py
```
## If no scripts are needed
Delete this README and the `scripts/` directory entirely.

File diff suppressed because it is too large Load Diff

View File

@@ -1,33 +0,0 @@
# tests/
Test files for scripts bundled with this skill.
## When to add tests
Add tests here when the skill has scripts in `scripts/` that are complex enough
to break silently — validators, parsers, generators, anything with branching
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
belongs here, not in `scripts/`.
## Dependencies
Tests require [bats-support](https://github.com/bats-core/bats-support) and
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
helpers from the repo root's `tests/test_helper/`.
From the repo root:
```bash
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
```
Run all tests for this skill (from the repo root):
```bash
bats <destination-dir>/agent-audit/tests/
```
## If no tests are needed
Delete this README and the `tests/` directory entirely.

View File

@@ -3,7 +3,7 @@ name: agent-author
description: >
Use when the user wants to create a new agent definition file from scratch, or
apply grill findings, audit findings, or inline feedback to an existing one.
Not read-only review -> `agent-audit`. Not skills -> `skill-author`.
Not read-only review -> `factory-audit`. Not skills -> `skill-author`.
allowed-tools: Bash Read Write Edit
metadata:
version: "1.0.2"
@@ -18,7 +18,7 @@ metadata:
- At plugin/APM scope `tools` and every Claude-only field are omitted entirely, not merely ignored: `apm compile` copies frontmatter verbatim to both harnesses, so fencing a read-only agent with `tools:` is wrong on one of them. `disallowedTools` is the one restriction that survives (ADR-0016).
- That fence is partial. It denies only the tools it names, never `Bash`, which a plugin-scope agent inherits — a shell redirect still writes. State the read-only boundary in the body too.
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is an `agent-audit` FAIL.
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is a `factory-audit` FAIL.
- Duplicate `name` values in one scope: Claude Code discards one silently. Verify uniqueness before shipping.
## Step 1 — Dispatch
@@ -29,7 +29,7 @@ metadata:
| A file exists, at least one improvement signal present | Improve | `references/improve.md` |
| A file exists, no signals | Stop and ask | — |
Signals: grill output, `agent-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
Signals: grill output, `factory-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
Read only the reference for the resolved flow. Capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it.
@@ -48,7 +48,7 @@ Read only the file for the resolved scope; the other describes fields this run c
Before writing or editing a `description`, or restructuring a body, read `references/contract.md` — the three-part shape, banned content, the delegation rule and the body pattern.
Gates `agent-audit` enforces at every scope:
Gates `factory-audit` enforces at every scope:
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <name>` that resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only: an agent's `name` and `description` is preloaded into every session exactly as a skill's is.
- **Body** — no word gate, and a delegation check in its place: name the skill to invoke rather than restating what it does.
@@ -58,7 +58,7 @@ At every scope, five tools reach no subagent whatever `tools` says — `AskUserQ
## Step 4 — Validate and close
Invoke `agent-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
Invoke `factory-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
At plugin/APM scope bump the resolved package's `apm.yml` `version` — **minor** on create, **patch** on improve — because consumers compare it to detect updates. Project and user scope have no manifest.

View File

@@ -7,4 +7,4 @@ All three scaffold the `description` in the three-part ADR-0020 shape — a `Use
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
- **`copilot.agent.md.template`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the authoritative list, read from there as data by `agent-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment).
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md` — the authoritative list, read from there as data by `factory-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment).

View File

@@ -6,8 +6,8 @@
anything, so a harness-specific value is wrong on at least one of them.
This template does not restate the permitted-field list. The authoritative
list is the `apm-agent-allowlist` section of agent-audit's
references/field-inventory.md, which agent-audit's validate.sh reads from
list is the `apm-agent-allowlist` section of factory-audit's
references/agent-field-inventory.md, which factory-audit's validate.sh reads from
there as data — a list copied into a template goes stale one step further
out than the list itself. Every field scaffolded below is on it; before
adding any other field, check that section.
@@ -37,7 +37,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
deleted.
Never write "Use proactively" here. It steers the Claude Code runtime and does
nothing anywhere else, and this file compiles to a Copilot `.agent.md` too, where
agent-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL.
factory-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL.
The phrase is CC-only; at this scope, a precise trigger clause does that job.
Example: "Use when a diff needs checking for injected credentials before it
merges. Not prose or style linting -> `lint-runner`." -->
@@ -69,7 +69,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
agent. Delete this comment before shipping. -->
## Inputs

View File

@@ -18,7 +18,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
"Use proactively" is valid HERE and only here: it steers the Claude Code runtime
to offer this agent unprompted. Add it only if that is what you want. If you add
it, leave it OUT of the Copilot half of the pair — the phrase does nothing there
and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The
and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The
pair must describe the same job; it does not have to be byte-identical.
Example: "Use when a diff needs checking for injected credentials before it
merges. Not prose or style linting -> `lint-runner`." -->
@@ -31,7 +31,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
Omit Agent entirely to prevent this agent from spawning subagents.
Never available to subagents regardless of tools field:
AskUserQuestion, EnterPlanMode, ExitPlanMode, ScheduleWakeup, WaitForMcpServers
Listing any of them is a finding: agent-audit enforces the flat rule. -->
Listing any of them is a finding: factory-audit enforces the flat rule. -->
<!-- model: sonnet
Optional. Aliases: sonnet, opus, haiku, fable. Or full model ID.
@@ -90,7 +90,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
agent. Delete this comment before shipping. -->
## Inputs

View File

@@ -20,8 +20,8 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
250 characters is the target, 400 the hard ceiling (ADR-0020).
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was deleted.
Never write "Use proactively" here. It steers the Claude Code runtime and does nothing
in Copilot, and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL.
Otherwise keep the wording matched to the Claude Code half of the pair: agent-audit
in Copilot, and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL.
Otherwise keep the wording matched to the Claude Code half of the pair: factory-audit
checks that both halves describe the same job, not that they are byte-identical, so
dropping the CC-only phrase here is not a pair-consistency finding.
Example: "Use when a diff needs checking for injected credentials before it
@@ -58,7 +58,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
agent. Delete this comment before shipping. -->
## Inputs

View File

@@ -7,7 +7,7 @@ source_keys:
# The agent description and body contract
House contract. The counts and the boundary targets are enforced by
`agent-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
`factory-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
judgment calls by its reference files.
## Why the budget exists
@@ -54,7 +54,7 @@ appear depends on the file:
| Vendor-neutral `.apm/agents/<name>.agent.md` (plugin/APM scope) | **Never.** Same Vale rule, same hard FAIL — the file matches the `**/*.agent.md` glob, and it compiles to a real Copilot agent downstream. |
A pair whose Claude Code half carries the phrase and whose Copilot half omits it is correct, not
inconsistent: `agent-audit` checks that both halves describe the same job, not that they match
inconsistent: `factory-audit` checks that both halves describe the same job, not that they match
word for word.
Indirect triggers ("even if the user doesn't say X") take a similar conditional at every scope:
@@ -126,7 +126,7 @@ One job per agent. An agent covering two jobs gets delegated to for the wrong on
**Delegation discipline replaces the word gate.** A plugin/APM agent is a single file with no
sibling `references/` directory: it cannot disclose progressively to itself, so its only way to
stay short is to *invoke* rather than *restate*. A body that transcribes a procedure a skill it
can invoke already owns is an `agent-audit` FAIL, and the fix is one line — "invoke `<skill>`".
can invoke already owns is a `factory-audit` FAIL, and the fix is one line — "invoke `<skill>`".
- Restating: "To commit, check the message against Conventional Commits: type, scope,
description; header under 100 chars; …"

View File

@@ -22,7 +22,7 @@ Before touching the filesystem, confirm you have:
If any are missing, stop and ask before proceeding.
`agent-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
`factory-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
is co-installed with this skill; if it is unavailable, stop and ask the user to install
kyberforge before continuing.

View File

@@ -27,7 +27,7 @@ When the same agent `name` appears at multiple scopes, **user scope wins over pr
Field rules are per scope and live with the scope: `references/plugin-scope.md` for the single
vendor-neutral file, `references/project-user-scope.md` for the Claude Code / Copilot pair. Read
one, not both. The short version is that plugin/APM frontmatter is an allowlist read from
`agent-audit`'s `references/field-inventory.md`, narrow because `apm compile` copies frontmatter
`factory-audit`'s `references/agent-field-inventory.md`, narrow because `apm compile` copies frontmatter
verbatim to every target (ADR-0016), while project and user scope carry the full per-provider
field sets.

View File

@@ -14,9 +14,9 @@ Confirm the agent file (or, at project and user scope, the pair) exists and that
improvement signal is present in the conversation or in a referenced file.
If no signals are present, stop: "This skill applies existing signals to an agent. For a blind
review, run `agent-audit` instead."
review, run `factory-audit` instead."
`agent-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
`factory-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
it is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
**Partial pair — project and user scope only.** If one provider file exists and the other does
@@ -53,7 +53,7 @@ Edit whichever file the signals point to.
scoped to the cases you have seen overfits and performs worse on new input.
**Delegate rather than grow.** An agent body has no word ceiling, but a body that restates a
procedure a skill it can invoke already owns is an `agent-audit` FAIL. When a signal reports a
procedure a skill it can invoke already owns is a `factory-audit` FAIL. When a signal reports a
missing procedure, check first whether an installed skill owns it and name that skill instead of
transcribing it. See `references/contract.md`.
@@ -81,7 +81,7 @@ the matching `sources.md` entry — the create flow's Step 3 has the rules.
**Check for regressions before handing back.** `SKILL.md` Step 4 tells you to resolve every FAIL,
which says nothing about a check that passed *before* these edits and no longer does. Compare the
closing `agent-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or
closing `factory-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or
a SUGGESTION that has become a FAIL, is damage this flow caused and is in scope for it. Only the
improve flow can make that comparison; the create flow has no prior state to compare against.

View File

@@ -12,9 +12,9 @@ a Copilot marker — the file is vendor-neutral.
## Frontmatter
The permitted keys are the `apm-agent-allowlist` section of `agent-audit`'s
`references/field-inventory.md`. Read them from there as data — that section is the single source
of truth, `agent-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
The permitted keys are the `apm-agent-allowlist` section of `factory-audit`'s
`references/agent-field-inventory.md`. Read them from there as data — that section is the single source
of truth, `factory-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
the roster, here or in a template or in script output, goes stale one step further out than the
list itself.
@@ -60,7 +60,7 @@ when research sources informed the agent, with slugs matching H2 headings in the
Follow the Body section of `references/contract.md`: role instruction, one job, and delegation
to installed skills instead of transcribed procedure.
## Before invoking `agent-audit`
## Before invoking `factory-audit`
- [ ] `name` kebab-case, matching the filename stem, unique in scope
- [ ] `description` written to `references/contract.md`

View File

@@ -92,7 +92,7 @@ Both formats truncate a body past **30,000 characters** silently.
Copilot has no `permissionMode`, `maxTurns`, `isolation`, `memory`, `effort`, `hooks` or
`mcpServers`. Never let those cross over from the Claude Code file.
## Before invoking `agent-audit`
## Before invoking `factory-audit`
Both files:

View File

@@ -15,7 +15,7 @@ All scripts in this skill must follow these rules:
- **Idempotent** — "create if not exists" per file. The scaffold script skips any file that already exists; agents may safely re-run it.
- **Meaningful exit codes** — `0` success, `1` invalid arguments or precondition failure. Document in `--help`.
- **Self-contained** — no external package installs at runtime. The script uses only bash builtins and POSIX tools (`sed`, `mkdir`, `cat`).
- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, which `agent-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure.
- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md`, which `factory-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure.
## Template variables

View File

@@ -25,8 +25,8 @@ Arguments:
its frontmatter verbatim to every target with no
per-target field integrator, so the permitted
field set is narrow — see the apm-agent-allowlist
section of agent-audit's
references/field-inventory.md and ADR-0016)
section of factory-audit's
references/agent-field-inventory.md and ADR-0016)
→ creates <package-root>/sources.md (if absent)
project scope : no type:-bearing apm.yml found; root is a
project directory
@@ -109,7 +109,7 @@ is_apm_package_manifest() {
# --- Walk-up package-root detection ---
#
# Mirrors agent-audit's validate.sh scope walk-up, with apm.yml + type: swapped
# Mirrors factory-audit's validate.sh scope walk-up, with apm.yml + type: swapped
# in for the old plugin.json marker. Starting at ROOT, walk upward:
# - an apm.yml with a top-level `type:` field marks an APM package root
# (plugin/APM scope) — stop and return it.
@@ -266,13 +266,13 @@ fi
# A roster restated in terminal output goes stale one step further out than the list
# itself: the old "(name, description, model, body only)" hint outlived ADR-0016's
# 2026-08-14 amendment, which added disallowedTools to the permitted set. Point at the
# scaffolded file's own comments for what to fill, and at agent-audit's validate.sh —
# which reads the allowlist from field-inventory.md as data — for what is permitted.
AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../agent-audit/scripts" 2>/dev/null && pwd || true)"
# scaffolded file's own comments for what to fill, and at factory-audit's validate.sh —
# which reads the allowlist from agent-field-inventory.md as data — for what is permitted.
AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../factory-audit/scripts" 2>/dev/null && pwd || true)"
if [[ -n "$AUDIT_SCRIPTS" && -f "$AUDIT_SCRIPTS/validate.sh" ]]; then
VALIDATE_HINT="$AUDIT_SCRIPTS/validate.sh"
else
VALIDATE_HINT="agent-audit's scripts/validate.sh"
VALIDATE_HINT="factory-audit's scripts/validate.sh"
fi
if [[ "$created_any" == false ]]; then
@@ -290,7 +290,7 @@ else
echo " 2. Populate $SOURCES_DIR/sources.md with research sources, or delete it" >&2
echo " 3. Validate: $VALIDATE_HINT $APM_FILE" >&2
echo " It checks the frontmatter against the apm-agent-allowlist section of" >&2
echo " agent-audit's references/field-inventory.md, the authoritative field list." >&2
echo " factory-audit's references/agent-field-inventory.md, the authoritative field list." >&2
else
echo " 1. Fill in $CC_FILE — replace every FILL IN: placeholder. Optional fields are" >&2
echo " scaffolded there as commented blocks; uncomment the ones that apply." >&2

View File

@@ -77,14 +77,14 @@ teardown() {
assert_failure
}
# The permitted set is read from the same data agent-audit's validate.sh reads --
# the apm-agent-allowlist section of agent-audit's field-inventory.md -- rather than
# The permitted set is read from the same data factory-audit's validate.sh reads --
# the apm-agent-allowlist section of factory-audit's agent-field-inventory.md -- rather than
# restated here. A hardcoded copy drifts: this assertion listed four fields and went
# on passing after ADR-0016's amendment added disallowedTools, and would have
# rejected a scaffolded agent that legitimately carried it.
@test "plugin/APM scope: frontmatter carries only allowlisted fields" {
inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md"
[ -f "$inventory" ] || fail "field-inventory.md not found at $inventory"
inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md"
[ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory"
allowlist="$(awk '
/^## apm-agent-allowlist$/ { insection = 1; next }
insection && /^##/ { exit }
@@ -99,7 +99,7 @@ teardown() {
keys="$(grep -oE '^[a-zA-Z][a-zA-Z0-9_-]*:' <<< "$fm" | sed 's/:$//' | sort -u)"
for key in $keys; do
if ! grep -qw "$key" <<< "$allowlist"; then
fail "frontmatter key '$key' is not in field-inventory.md's apm-agent-allowlist ($allowlist)"
fail "frontmatter key '$key' is not in agent-field-inventory.md's apm-agent-allowlist ($allowlist)"
fi
done
}
@@ -111,8 +111,8 @@ teardown() {
# must name every one of them, so a partial restatement -- the only shape that can go
# stale silently -- fails. Naming none, the current design, passes.
@test "plugin/APM scope: next-steps guidance does not partially restate the allowlist" {
inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md"
[ -f "$inventory" ] || fail "field-inventory.md not found at $inventory"
inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md"
[ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory"
allowlist="$(awk '
/^## apm-agent-allowlist$/ { insection = 1; next }
insection && /^##/ { exit }
@@ -134,7 +134,7 @@ teardown() {
fi
done
if [ -n "$named" ] && [ -n "$missing" ]; then
fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at field-inventory.md instead of restating it."
fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at agent-field-inventory.md instead of restating it."
fi
}

View File

@@ -0,0 +1,77 @@
---
name: factory-audit
description: >
Use when the user wants a skill directory or agent definition audited,
including "is this ready to ship", or after hand-editing one outside its
author skill. Not applying skill fixes -> skill-author. Not applying agent
fixes -> agent-author.
allowed-tools: Bash Read
metadata:
version: "1.0.1"
category: factory
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
## Gotchas
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
- A file carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
## Step 0 — Dispatch
Resolve the flow from the target path **before running anything**. The two flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches.
| Target | Flow | Read |
|---|---|---|
| A directory containing `SKILL.md` | skill | `references/skill-flow.md` |
| A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` |
| A file named `*.agent.md` | agent | `references/agent-flow.md` |
| A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` |
| Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — |
Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4.
On the last row, stop: run no validator and tell the user the two accepted shapes — a skill directory (or its `SKILL.md`), or an agent file (`*.agent.md`, or a `.md` directly under an `agents/` directory). Guessing a flow audits the path against the wrong spec.
The scripts re-detect the flow from the path. If `validate.sh` reports on the other artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line.
## Step 4 — Report
Open with the coverage line for the flow you took, naming every dimension checked.
Skill flow:
```text
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
```
Agent flow:
```text
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
```
On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check.
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
Each finding:
```text
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
```
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow. Do not apply fixes — report and propose only.

View File

@@ -1,5 +1,8 @@
StylesPath = styles
[**/SKILL.md]
BasedOnStyles = Kyberforge
[**/agents/*.md]
BasedOnStyles = Kyberforge

View File

@@ -30,7 +30,7 @@ prompt of a fresh context that has nothing else in it. The rationale for the 900
not transfer, so:
- **Never report an agent body as too long on a word count.** There is no number to cite.
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate.bats` pins its absence with a
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate-agent.bats` pins its absence with a
body far past 900 words that must still pass, and adding one would contradict the ADR.
- The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which
`validate.sh` already reports as a SUGGESTION because content past it is silently truncated.
@@ -100,6 +100,6 @@ frontmatter block that still contains one.
## Where the criteria live
Every FAIL and SUGGESTION criterion for these dimensions is in `references/finding-criteria.md`,
Every FAIL and SUGGESTION criterion for these dimensions is in `references/agent-finding-criteria.md`,
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
puts the body, delegation or comment-discipline dimension in play.

View File

@@ -91,6 +91,6 @@ description: >
## Where the criteria live
Every FAIL and SUGGESTION criterion for this dimension is in `references/finding-criteria.md`,
Every FAIL and SUGGESTION criterion for this dimension is in `references/agent-finding-criteria.md`,
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
puts the description dimension in play.

View File

@@ -20,7 +20,7 @@ Two rules on using it:
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
to read; it does not settle a close call on its own.
## description — `references/description-quality.md`
## description — `references/agent-description-quality.md`
Flag as FAIL if:
@@ -49,7 +49,7 @@ Flag as FAIL if:
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0 of
`references/description-quality.md`.
`references/agent-description-quality.md`.
Flag as SUGGESTION if:
@@ -68,7 +68,7 @@ What is left to judgment is semantic and the script cannot reach it: whether a t
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
files manually") should have named one.
## body, delegation and comment-discipline — `references/body-and-delegation.md`
## body, delegation and comment-discipline — `references/agent-body-and-delegation.md`
Flag as FAIL if:

View File

@@ -0,0 +1,64 @@
---
source_keys:
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
# Agent Flow
Steps 1 to 3 for an agent definition — the target Step 0 matched as a `*.agent.md` file, or as a
`.md` file whose immediate parent directory is `agents/`. Work them in order, then return to
`SKILL.md` Step 4 to report.
## Gotchas
- An agent takes the skill description gates (250 characters SUGGESTION, 400 FAIL) and **no body word gate at all**. Its body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer and no number exists to cite. Judge an over-long agent body through the delegation check.
- **Plugin/APM scope only:** provider safety means survival of a verbatim copy to every target, not Claude-Code-versus-Copilot field leakage — `references/agent-scope-plugin-apm.md` carries the contract.
## Step 1 — Deterministic checks
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
```bash
bash scripts/validate.sh <agent-file>
bash scripts/validate-provenance.sh <agent-file>
bash scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
```
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, a `scripts/lib-*.sh` library or `references/agent-field-inventory.md` missing — read `references/agent-validation-scripts.md`; what these scripts measure is not reproducible by reading.
`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same tiers: **1** is real findings, **2** is never ran — report that as `### Structure` unverified, never as a failure.
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
| Rule | Dimension |
|---|---|
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
## Step 2 — Read the agent and load its scope contract
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
| Scope | Read |
|---|---|
| plugin/APM | `references/agent-scope-plugin-apm.md` |
| project, user | `references/agent-scope-project-user.md` |
## Step 3 — Qualitative audit
Read `references/agent-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
| Dimension | Rubric |
|---|---|
| description | `references/agent-description-quality.md` |
| body, delegation, comment-discipline | `references/agent-body-and-delegation.md` |
Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding.
Then return to `SKILL.md` Step 4.

View File

@@ -18,13 +18,13 @@ cannot carry either (ADR-0016). That single fact drives everything below.
## Frontmatter allowlist
The permitted keys are the `apm-agent-allowlist` section of `references/field-inventory.md`. Read
The permitted keys are the `apm-agent-allowlist` section of `references/agent-field-inventory.md`. Read
them from there. Do not recite the list in a finding, do not work from memory, and do not trust any
restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it
has changed before, and `validate.sh` parses that same section at load time, so a recitation is a
copy that can disagree with the check the agent just ran.
`field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
`agent-field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
is not. Read that note before arguing with a finding about it.
## Dimension routing
@@ -55,4 +55,4 @@ known upstream schema limitation (ADR-0016), not an authoring mistake, and the f
give the author visibility into the gap rather than to imply the schema can be made to close it.
Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it.
A denylist-shaped restriction is the available half of that — see `field-inventory.md`.
A denylist-shaped restriction is the available half of that — see `agent-field-inventory.md`.

View File

@@ -30,7 +30,7 @@ the finding, not presented as a platform spec failure.
| everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure |
The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of
`references/field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
`references/agent-field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
those same sections, so any restatement is a copy that can disagree with the check (ADR-0009).
## Field and naming rules that differ by provider
@@ -42,7 +42,7 @@ those same sections, so any restatement is a copy that can disagree with the che
- `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent
unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags
it. The Copilot equivalent is `disable-model-invocation`, which changes the description contract
entirely — see `references/description-quality.md`, Step 0.
entirely — see `references/agent-description-quality.md`, Step 0.
## Pair consistency
@@ -54,6 +54,6 @@ Check that:
- The two files describe the **same job**. Divergent capability claims across the pair mean one
half was edited and the other was not, which is the defect this dimension exists to catch.
- Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is
the Step 0 case in `references/description-quality.md`, not a pair-consistency finding.
the Step 0 case in `references/agent-description-quality.md`, not a pair-consistency finding.
Keep `pair-consistency` in the Step 4 coverage line at these scopes.

View File

@@ -50,18 +50,18 @@ kebab-case; no `FILL IN:` placeholders in the description or body; the descripti
characters measured on the folded YAML value.
**Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter;
no frontmatter key outside the `apm-agent-allowlist` section of `references/field-inventory.md` —
no frontmatter key outside the `apm-agent-allowlist` section of `references/agent-field-inventory.md` —
open that file, do not work from memory.
**Project/user scope:** the counterpart file exists; `name` matches the filename stem in the
Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in
the Copilot file and none from `copilot-only-fields` in the CC file, both read from
`references/field-inventory.md`.
`references/agent-field-inventory.md`.
## Script-specific failures
- **`Error: field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
`references/field-inventory.md` at load time and refuses to run without it, rather than falling
- **`Error: agent-field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
`references/agent-field-inventory.md` at load time and refuses to run without it, rather than falling
back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not
work around it.
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full

View File

@@ -205,5 +205,5 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
```
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
which Step 3 loads on every run.

View File

@@ -83,5 +83,5 @@ description: >
(`data-model` is illustrative. In a real description the target has to resolve.)
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
which Step 3 loads on every run.

View File

@@ -15,8 +15,11 @@ Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification per
directories; this house does not, because an unlisted directory is content no auditor and no host
knows to look at. Flag any other directory as a FAIL.
- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`,
`test_*.sh`) there are a FAIL — they belong in `tests/`.
- `scripts/` holds only executable code an agent can run, and the sourced libraries those entry
points load. A `lib-*.sh` that is never invoked on its own belongs here beside the entry point
that sources it — it is executable code, not documentation, so do not flag it for failing to run
standalone. Test files (`.bats`, `*_test.*`, `test_*.sh`) there are a FAIL — they belong in
`tests/`.
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
- An optional directory that exists must hold real content, not an unfilled placeholder README.
@@ -38,9 +41,9 @@ Resolve before flagging, twice over:
surrounding prose presents it as the form to copy.
**Referring to another skill's file.** There is one sanctioned spelling, and it is possessive:
`skill-audit's references/validation-scripts.md`. Write the skill by name and let the reader
`skill-author's references/contract.md`. Write the skill by name and let the reader
resolve it — do not spell the repo path. The full path is the thing this section forbids, and
`references/validation-scripts.md` on its own is a hard ERROR from the gate, which
`references/contract.md` on its own is a hard ERROR from the gate, which
requires an unqualified `references/` pointer to exist in the skill's OWN directory. The
possessive form is the only spelling both rules accept; the gate recognises it and skips the
on-disk check. Flag any other spelling of a cross-skill reference.
@@ -65,5 +68,5 @@ The skill has to agree with itself. Two checks:
- Placeholder READMEs inside `scripts/`, `tests/` and `assets/` say the same thing about each
directory that `SKILL.md` does.
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
which Step 3 loads on every run.

View File

@@ -19,7 +19,7 @@ Two rules on using it:
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
to read; it does not settle a close call on its own.
## description — `references/description-quality.md`
## description — `references/skill-description-quality.md`
Flag as FAIL if:
@@ -43,7 +43,7 @@ Flag as FAIL if:
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
judgment.
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of
`references/description-quality.md`.
`references/skill-description-quality.md`.
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
of the 400-character house ceiling above.
@@ -62,7 +62,7 @@ left to judgment here is semantic and the script cannot reach it: whether a targ
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
files manually") should have named one.
## body-discipline — `references/body-discipline.md`
## body-discipline — `references/skill-body-discipline.md`
Flag as FAIL if:
@@ -84,7 +84,7 @@ Flag as SUGGESTION if:
- Gotchas are correct but placed late in the body rather than near the top
- Content that only one branch reaches is inlined where a `references/` file would serve
## patterns — `references/patterns.md`
## patterns — `references/skill-patterns.md`
Flag as FAIL if:
@@ -101,7 +101,7 @@ Flag as SUGGESTION if:
- An output template is present but permissive where the consumer needs it exact
- A conditional reference names a trigger that is real but broader than the branch it guards
## file-structure and internal-consistency — `references/file-structure.md`
## file-structure and internal-consistency — `references/skill-file-structure.md`
Flag as FAIL if:
@@ -117,7 +117,7 @@ Flag as SUGGESTION if:
- An optional directory exists but holds only a placeholder README
## formatting and scripts — `references/formatting-and-scripts.md`
## formatting and scripts — `references/skill-formatting-and-scripts.md`
Flag as FAIL if:

View File

@@ -0,0 +1,62 @@
---
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
---
# Skill Flow
Steps 1 to 3 for a skill directory — the target Step 0 matched as a directory containing
`SKILL.md`, or as a `SKILL.md` file, in which case `<skill-dir>` below is its parent directory.
Work them in order, then return to `SKILL.md` Step 4 to report.
## Gotchas
- A skill takes two independent length families, and it can sit inside one while failing the other — so report them separately. The 500-line / 2,770-word pair counts the **whole file** for spec conformance. The 250/400-character and 600/900-word pair is the house context budget, and its word half counts the **body only**.
## Step 1 — Deterministic checks
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
```bash
bash scripts/validate.sh <skill-dir>
bash scripts/validate-provenance.sh <skill-dir>
bash scripts/vale-wrap.sh <skill-dir>/SKILL.md
```
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation. It exits **0** when no check failed, **1** on real findings, and **2** when it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr. Report an exit 2 as `### Structure` unverified, quoting that reason, never as a failure or a pass.
Read `references/skill-validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback.
`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/skill-validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass.
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
| Rule | Dimension |
|---|---|
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
| `Kyberforge.PaddingPhrase` | patterns |
## Step 2 — Read the whole skill
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
## Step 3 — Qualitative audit
Read `references/skill-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
| Dimension | Rubric |
|---|---|
| description | `references/skill-description-quality.md` |
| body-discipline | `references/skill-body-discipline.md` |
| patterns | `references/skill-patterns.md` |
| file-structure, internal-consistency | `references/skill-file-structure.md` |
| formatting, scripts | `references/skill-formatting-and-scripts.md` |
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
Then return to `SKILL.md` Step 4.

View File

@@ -44,5 +44,5 @@ follow from that:
undocumented one is a coin flip.
- **`--dry-run` present for destructive operations.**
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
which Step 3 loads on every run.

View File

@@ -10,7 +10,7 @@ Upstream source: agentskills.io — best-practices (instruction patterns), speci
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
*content* is judged in `references/body-discipline.md`.
*content* is judged in `references/skill-body-discipline.md`.
## The constructs and when each is right
@@ -50,5 +50,5 @@ forms are judgment.
`references/` when only one dispatch branch produces that output. A template inlined for a branch
most invocations never take is body-discipline padding.
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
which Step 3 loads on every run.

View File

@@ -0,0 +1,153 @@
---
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
- context7-websites-code-claude
- claude-code-plugins-docs
- claude-code-subagents-docs
- context7-github-en-copilot
- github-custom-agents-configuration
---
# Sources
<!-- agentskills.io/llms.txt was used for initial source discovery and is not listed below; it contributed no skill file content directly. -->
## agentskills-home
- **URL:** https://agentskills.io/home.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
- **Contributing files:** SKILL.md, references/skill-flow.md
- **Status:** `extracted`
## agentskills-spec
- **URL:** https://agentskills.io/specification.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-description-quality.md, references/skill-patterns.md, references/skill-file-structure.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
- **Status:** `extracted`
## agentskills-best-practices
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-patterns.md, references/skill-finding-criteria.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-description-quality.md, references/skill-finding-criteria.md
- **Status:** `extracted`
## agentskills-evaluating-skills
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
- **Contributing files:** (none — eval workflow not directly informing audit dimensions)
- **Status:** `extracted`
## agentskills-using-scripts
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
- **Status:** `extracted`
## agentskills-quickstart
- **URL:** https://agentskills.io/skill-creation/quickstart.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
- **Contributing files:** (none — creation guide not directly informing audit criteria)
- **Status:** `extracted`
## context7-websites-code-claude
- **URL:** context7:/websites/code_claude
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
- **Status:** `extracted`
## claude-code-plugins-docs
- **URL:** https://code.claude.com/docs/en/plugins
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-validation-scripts.md
- **Status:** `extracted`
## claude-code-subagents-docs
- **URL:** https://code.claude.com/docs/en/sub-agents
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
- **Status:** `extracted`
## context7-github-en-copilot
- **URL:** context7:/websites/github_en_copilot
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
- **Status:** `extracted`
## github-custom-agents-configuration
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
- **Status:** `extracted`
## github-cli-plugin-reference
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-creating
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-finding-installing
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow
- **Contributing files:** (none)
- **Status:** `extracted`
## github-plugins-marketplace
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands
- **Contributing files:** (none)
- **Status:** `extracted`
## github-sdk-custom-agents
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
- **Contributing files:** (none)
- **Status:** `extracted`

View File

@@ -1,127 +1,54 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate.sh <skill-dir>
Validate a skill directory against the agentskills.io specification.
Arguments:
skill-dir Path to the skill directory containing SKILL.md.
Exit codes:
0 All checks passed (may include SUGGESTIONs)
1 One or more checks failed
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: skill-dir is required." >&2
echo "" >&2
usage >&2
exit 1
fi
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
# be measured after YAML folding is resolved, and the hand-rolled reader that
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
# boundary — same description, two verdicts, depending on which reader ran.
# Refusing to start is the only honest option; the repo's jq / apm / vale
# dependencies are declared the same way.
# Check the interpreter separately from the library: `python3 -c` fails the same
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
# dependency sends the reader to install the wrong thing.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
exit 1
fi
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
echo "Error: PyYAML is required but is not importable by python3." >&2
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
import glob
import yaml
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
# default to ASCII, and this script's own message text carries em dashes (the
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
# every check has already run, which loses the whole report and (here) flips a
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
# resolver block below pins the reads; this pins the writes.
# lib-boundary-resolver.sh — SOURCED, never executed.
#
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
# validate.sh copies print findings, skill-size-check.sh has its own top-level
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
# byte-identity across all three.
for _stream in (sys.stdout, sys.stderr):
try:
_stream.reconfigure(encoding='utf-8')
except AttributeError: # pragma: no cover — Python < 3.7
pass
skill_dir = os.path.abspath(sys.argv[1])
skill_md = os.path.join(skill_dir, "SKILL.md")
if not os.path.isfile(skill_md):
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
sys.exit(1)
failed = False
suggestions = []
def ok(msg):
print(f"PASS {msg}")
def fail(msg):
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
# scripts in the ADR-0020 family now agree: findings that fail the run go to
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
# Both repo callers capture `2>&1`, so nothing a human reads moves.
global failed
print(f"FAIL {msg}", file=sys.stderr)
failed = True
def suggest(msg):
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
# rather than another silently-ignored warning (ADR-0013).
suggestions.append(msg)
def info(msg):
# A check that DECLINED to run says so out loud, rather than passing
# silently. Silence is what let a whole gate family go missing unnoticed.
print(f"INFO {msg}")
# The ADR-0020 shared boundary resolver, as ONE copy for this skill. Both of
# validate.sh's modes compose it into the Python program they run, so the
# skill-mode and agent-mode check suites resolve boundary targets through the
# same code rather than through two copies that can drift apart.
#
# The resolver is Python, and bash cannot source Python, so the block is held
# in a shell variable filled from a QUOTED here-doc: nothing inside it is
# expanded, substituted or rewritten, and the text between the two markers
# below is therefore byte-identical to the copy in scripts/skill-size-check.sh
# that tests/test-adr0020-contract.sh hashes. The markers stay on lines of
# their own, at column 0, exactly once each, so `sed -n '/^BEGIN$/,/^END$/p'`
# extracts the same span here as it does from the scripts the test already
# reads. Edit one copy, then paste it over the others.
#
# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`.
# This file is sourced by validate.sh before the mode-specific python3/PyYAML
# preflight runs, so a `cat` here made coreutils a hard dependency ahead of
# python3: on a PATH with neither, the script exited 127 naming `cat` instead of
# reaching the preflight that names python3. `read -r -d ''` reads to a NUL that
# never arrives and so returns non-zero at EOF — hence the `|| true` — and it
# keeps the last line's newline, which the joining newline in the caller would
# otherwise double — hence the single strip after the delimiter. It removes
# exactly ONE newline, never a run: blank lines at the end of a chunk are part
# of the program text the entry script reassembles, and stripping every
# trailing newline deleted them. The here-doc itself is unchanged: still
# QUOTED, still byte-identical between its markers.
#
# Self-containment (agentskills.io, skill-author/references/deployment-modes.md)
# binds BETWEEN skills, not within one: a cache-installed plugin copies each
# skill's own directory whole, so a sibling file in this same scripts/ directory
# travels with the skill and is always readable. That is why this is sourced
# here and duplicated across skill boundaries elsewhere.
#
# Consumed by: validate.sh (both modes), via $KYBERFORGE_RESOLVER_PY.
# shellcheck shell=bash
# shellcheck disable=SC2034
IFS='' read -r -d '' KYBERFORGE_RESOLVER_PY <<'KYBERFORGE_ADR0020_RESOLVER_PY' || true
# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER =====
# ONE resolver, embedded VERBATIM in three scripts:
# ONE resolver, embedded VERBATIM in two scripts (ADR-0025 retired the third):
# scripts/skill-size-check.sh
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh
# The block between these markers must stay byte-identical in all three. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file all
# three can share (same constraint that forces the ADR-0020 constants to be
# duplicated). Edit one copy, then paste it over the other two.
# plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh
# The block between these markers must stay byte-identical in both. It is copied
# rather than imported because a cache-installed plugin's scripts cannot read
# files outside their own plugin directory, and this repo-root hook resolves via
# .pre-commit-hooks.yaml, where entry[0] is the only token pre-commit rewrites --
# so no single file is reachable by both (the same constraint that duplicates the
# ADR-0020 constants). Edit one copy, then paste it over the other.
#
# Requires: glob, os, re, yaml (imported by the host script; PyYAML is a hard
# dependency, preflighted in bash before the interpreter starts).
@@ -155,9 +82,9 @@ def read_text(path):
# The set of names a boundary clause may resolve against is derived from an
# AUTHORING ROOT found by walking up FROM THE TARGET FILE. It is NEVER derived
# from this script's own location: deriving it from ${BASH_SOURCE} leaked
# holocron's 39-skill universe into every consumer repo that ran this hook
# through pre-commit, so a consumer skill routing to `skill-audit` resolved
# against a plugin it had never installed.
# holocron's whole skill universe into every consumer repo that ran this hook
# through pre-commit, so a consumer skill routing to a holocron skill such as
# `factory-audit` resolved against a plugin it had never installed.
#
# An authoring root is the nearest ancestor holding plugins/*/.apm/skills/ or
# plugins/*/.apm/agents/ (a plugin monorepo), falling back to the nearest
@@ -417,8 +344,8 @@ def known_targets(start_dir):
# written yet, and any new process chain re-arms it. Unexercised is not the
# same as unnecessary, and the branch it guards is still load-bearing: the
# bare-arrow rule is the sole extractor for three real targets in
# kyberforge's audit skills (agent-audit -> agent-author, agent-audit ->
# skill-audit, skill-audit -> skill-author), all written unbackticked.
# kyberforge (factory-audit -> skill-author, factory-audit ->
# agent-author, apm-orchestrate -> apm-install), all written unbackticked.
# * A backticked hyphenated token counts only inside a boundary sentence.
# Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard
# FAIL with no escape hatch. Gating it costs nothing (measured over this
@@ -1051,7 +978,7 @@ def hand_invoked(fm_text):
#
# Both read a FENCE-MASKED copy of the body. Scanning the raw body made a
# ```-fenced example a hard ERROR — and the skills most likely to carry one are
# skill-author and skill-audit, which DOCUMENT the references/ convention — and
# skill-author and factory-audit, which DOCUMENT the references/ convention — and
# let a `## Gotchas` heading inside a fenced block stand in for the real
# section. Masking preserves every byte offset (content becomes spaces,
# newlines stay), so a span found in the mask slices the original.
@@ -1076,12 +1003,12 @@ REFERENCE_POINTER = re.compile(
REFERENCE_PAST = re.compile(
r'\b(?:removed|deleted|renamed|superseded|replaced|obsolete|deprecated'
r'|former|formerly|gone|no longer|used to)\b', re.I)
# A pointer QUALIFIED by another skill's name — "skill-audit's
# references/validation-scripts.md" — names a file that is deliberately NOT in
# A pointer QUALIFIED by another skill's name — "factory-audit's
# references/skill-validation-scripts.md" — names a file that is deliberately NOT in
# this skill's directory. Requiring it on the local disk left NO legal spelling
# for a cross-skill reference at all: the only alternative, a full repo path
# (`plugins/kyberforge/.apm/skills/skill-audit/references/...`), is itself a
# FAIL under skill-audit's own file-structure rubric, because a path that climbs
# (`plugins/kyberforge/.apm/skills/factory-audit/references/...`), is itself a
# FAIL under factory-audit's own file-structure rubric, because a path climbing
# out of the skill directory stops resolving once the plugin is cache-installed.
# The possessive form is the sanctioned spelling, and it is skipped here. It is
# not checked further — this function has no way to locate another skill's
@@ -1173,505 +1100,5 @@ def missing_reference_pointers(body, skill_dir):
missing.add('references/' + match.group(1))
return sorted(missing)
# ===== END ADR-0020 SHARED BOUNDARY RESOLVER =====
# A leading BOM is stripped before anything is parsed or counted. It changes
# neither count below — it is not a line separator and str.split() does not
# treat it as whitespace — but it did defeat the frontmatter match.
try:
content = strip_bom(read_text(skill_md))
except EncodingError as exc:
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
f"hard failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
# --- Parse frontmatter ---
fm_match = FRONTMATTER_RE.match(content)
if not fm_match:
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
"spaces after either marker and CRLF endings are all tolerated). Nothing "
"downstream can be measured, so this is a hard failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
fm = fm_match.group(1)
body_start = fm_match.end()
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
# by `description: ...` captured the NEXT KEY as the name and reported a
# mismatch instead of an absence. Same class of bug as the `description:` one
# the shared resolver's description_value() docstring records.
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
name = name_m.group(1).strip('"\'') if name_m else ""
# Extract description — the VALUE, with YAML folding resolved. Most of this
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
# carry indentation and newlines that are not part of the value: every length
# measurement below is wrong unless the scalar is folded first.
try:
desc = description_value(fm)
except FrontmatterError as exc:
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or a
# description of the wrong type. Do not prefix a diagnosis here; the last
# one named a syntax error for two failures that have none.
fail(f"{exc}. Nothing downstream can be measured, so this is a hard "
f"failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
dir_name = os.path.basename(skill_dir)
# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation:
# true` takes the skill out of the model-visible listing entirely, so the
# trigger/capability/boundary rules and the 250-character routing target do not
# apply to it — the audit's own references/description-quality.md Step 0 says
# so, and until this line existed no check here knew the field existed. What the
# flag does NOT lift: the body word budget and the 400-character description
# ceiling. See the shared resolver's hand_invoked().
by_hand = hand_invoked(fm)
# --- Checks ---
# name present
if name:
ok(f"name present: '{name}'")
else:
fail("name field is missing or empty")
# name matches directory
if name and dir_name:
if name == dir_name:
ok(f"name '{name}' matches directory '{dir_name}'")
else:
fail(f"name '{name}' does not match directory '{dir_name}'")
# name length
if name:
if len(name) <= 64:
ok(f"name length {len(name)} chars (limit: 64)")
else:
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
# name format
if name:
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
ok("name format valid (kebab-case)")
else:
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
# description present
if desc:
ok("description present")
else:
fail("description field is missing or empty")
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
# sits underneath it rather than replacing it.
if desc:
dlen = len(desc)
if dlen <= 1024:
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
else:
fail(f"description length {dlen} chars — exceeds 1024-character limit")
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
# but not backtick-quoted references like `FILL IN:` used in instructions.
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
# description contains unfilled placeholder
if desc and PLACEHOLDER_RE.search(desc):
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
else:
if desc:
ok("description has no unfilled placeholders")
# --- ADR-0022: metadata.version is mandatory -------------------------------
# FAIL, not SUGGESTION, and the tier is set by the gate rather than by taste.
# `.pre-commit-config.yaml`'s `skill-size-check` hook REJECTS a SKILL.md with
# no `metadata.version`, and rejects a value that is not three-part semver.
# skill-author's Step 4 says to run this audit and "resolve every FAIL", so any
# tier below FAIL lets that step report done on a skill the commit gate then
# refuses — the same audit-disagrees-with-the-gate failure the MAX_LINES note
# below warns about, arrived at from the other direction. Verified before this
# check existed: a SKILL.md with no `metadata:` block at all reported "All
# checks passed".
#
# The rule is DUPLICATED from that hook for the same cache-isolation reason as
# every other constant here — an installed plugin's scripts cannot read the
# repo-root config. Keep the two in step: this check must accept exactly what
# the hook accepts.
SEMVER_RE = re.compile(r'^\d+\.\d+\.\d+$')
try:
fm_data = yaml.safe_load(fm)
except Exception:
# Unreachable in practice: description_value() above parses the same text
# and hard-exits on a YAML error, so anything arriving here already parsed.
fm_data = None
metadata_block = fm_data.get('metadata') if isinstance(fm_data, dict) else None
if not isinstance(metadata_block, dict) or metadata_block.get('version') is None:
fail("frontmatter has no metadata.version — ADR-0022 makes it mandatory for "
"every skill, and the skill-size-check pre-commit hook rejects the file "
"without it. Add `metadata:` / ` version: \"1.0.0\"` (new skills start "
"at \"0.1.0\")")
else:
version_value = metadata_block['version']
# NOT str()-coerced blind: `version: 1.0` is a YAML float, and its "1.0"
# spelling is exactly the two-part value the hook rejects — coercing and
# then matching keeps this check and the hook agreeing on that case.
version_text = version_value if isinstance(version_value, str) else str(version_value)
version_text = version_text.strip()
if SEMVER_RE.match(version_text):
ok(f"metadata.version present: '{version_text}' (ADR-0022)")
else:
fail(f"metadata.version '{version_text}' is not three-part semver — the "
f"skill-size-check pre-commit hook rejects it. Use MAJOR.MINOR.PATCH, "
f"e.g. \"1.0.0\"")
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
# outside the plugin directory once the plugin is cache-installed, so there is
# no single source to share. Keep the two in sync by hand: if they drift, this
# audit will report a skill ready to ship that the commit hook then rejects.
MAX_LINES = 500
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
# for the full measurement.
MAX_WORDS = 2770
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
# agree, so drift fails CI instead of shipping an audit that disagrees with the
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
# description constants; per ADR-0020 agents take the description gates and
# deliberately take NO body word gate, because an agent body becomes the system
# prompt of a fresh context rather than competing with a live conversation.
#
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
# quality gate. Likewise the 1024-character description limit above is the
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
# underneath it.
DESC_SUGGEST_CHARS = 250
DESC_MAX_CHARS = 400
BODY_SUGGEST_WORDS = 600
BODY_MAX_WORDS = 900
line_count = len(content.splitlines())
if line_count <= MAX_LINES:
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
else:
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
# str.split() with no argument splits on runs of whitespace, matching the
# `wc -w` the hook uses, and counts the whole file including frontmatter.
word_count = len(content.split())
if word_count <= MAX_WORDS:
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
else:
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
body = content[body_start:]
# --- ADR-0020: description budget -----------------------------------------
if desc:
dlen = len(desc)
if dlen > DESC_MAX_CHARS:
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
f"and a boundary clause; move capability enumeration, output-format detail, "
f"composition notes and implementation detail to the body or README.md")
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
f"what moves the corpus average; the FAIL tier only stops outliers")
elif by_hand:
ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character "
f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)")
else:
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
# --- ADR-0020: body budget -------------------------------------------------
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
# This is a different measurement from MAX_WORDS above, which counts the whole
# file including frontmatter as a spec-conformance backstop. Both are reported.
body_word_count = len(body.split())
if body_word_count > BODY_MAX_WORDS:
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
f"and rationale prose to references/ behind an explicit "
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
f"common to every branch, each flow gets its own self-contained references/ file")
elif body_word_count > BODY_SUGGEST_WORDS:
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
else:
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
# --- Reference pointers must exist -----------------------------------------
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
# on disk is a hard break, and until this check existed nothing in the
# gate/audit/vale stack noticed it — all three exited 0.
missing_refs = missing_reference_pointers(body, skill_dir)
for ref in missing_refs:
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
if not missing_refs:
ok("all referenced references/ files exist")
# --- Gotchas discipline -----------------------------------------------------
# SUGGESTION on both counts: the measurement is deterministic, but whether a
# given gotcha earns its place in the body is the auditor's judgment.
gotchas = gotcha_stats(body)
if gotchas is not None:
gotcha_entries, gotcha_words = gotchas
if gotcha_entries > GOTCHA_MAX_ENTRIES:
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
f"missing references/ file or a design problem written up as a warning")
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
f"parts to references/ and keep the section for live traps")
# --- ADR-0020: boundary clause present -------------------------------------
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
# this particular skill warrants a boundary clause is judgment. Both accepted
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
#
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not
# parse" are different findings, and reporting the first for the second sends
# the author hunting for a problem that is not there (issue #110).
#
# Skipped entirely for a hand-invoked skill — the contract gives it one plain
# sentence with no boundary clause, so the finding would be wrong and its remedy
# names a router that cannot see the skill (issue #108).
if desc and by_hand:
ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger "
"rules do not apply; audited as one plain human-facing sentence")
elif desc:
status = boundary_clause_status(desc)
if status == 'present':
ok("description has a boundary clause")
elif status == 'absent':
suggest("description has no boundary clause — add the prose form (\"Do not use "
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
"so the router knows where NOT to send this skill")
else:
suggest("description has an arrow boundary clause (\"Not X -> y\") from which no "
"target could be read, so the dangling-target check did not run on it — "
"the clause is PRESENT and unparsed, not missing. Most often the target is "
"a single word, which is deliberately not matchable bare because "
"`research`, `triage` and `forge` are all ordinary English: write it as "
"`name` or /name")
# One arrow, one target. A second name after the same arrow is resolved by
# nothing and reported by nothing, so the clause claims coverage it does not
# have and this script printed "1 of 1 boundary target(s) resolve" on a
# clause naming two (issue #107).
for first, second in multi_target_arrow_clauses(desc):
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
f"'{second}') and only the first is resolved — the second is checked by "
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
f"Not Y -> {second}.\"")
# --- ADR-0020: resolvable boundary targets ---------------------------------
# The resolution universe comes from the SKILL's own location: the authoring
# root above it (every sibling plugin in the monorepo), its own apm package, and
# the packages that package declares in apm.yml dependencies.apm. It is never
# derived from this script's own path, and — when an authoring root exists — it
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
# run `apm install` return the same verdict. See the shared resolver's header.
if desc:
routing_targets = boundary_targets(desc)
known = known_targets(skill_dir) if routing_targets else set()
if routing_targets and not known:
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
f"determined for this path (no authoring root above it, no apm package "
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
elif routing_targets:
# blocking vs reported: a target only earns a FAIL when it is written in
# route notation or its own sentence corroborates it by naming another
# target that resolves. See the shared resolver's CORROBORATION note.
unresolved, soft = unresolved_targets(desc, known)
for target in unresolved:
fail(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
f"target sends the router nowhere")
for target in soft:
suggest(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
f"else in that sentence resolves, so it is equally likely to be a tool, a "
f"file format or an English compound. If it IS a route, write it as "
f"`/{target}` or `-> {target}` and it will be checked properly")
if not unresolved:
# Counts the targets that ACTUALLY resolve, not every target found:
# a confirm-only target (one used attributively — see the resolver's
# ATTRIBUTIVE USE note) is exempt from the failure above, so
# reporting it as resolved would be a false claim.
resolved = [t for t in routing_targets if normalize_target(t) in known]
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
f"{', '.join(resolved) if resolved else '(none)'}")
# Body unfilled placeholders
fill_matches = PLACEHOLDER_RE.findall(body)
if fill_matches:
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
else:
ok("SKILL.md body has no unfilled placeholders")
# Interactive prompt heuristic.
#
# A line-initial `read` only blocks an agent when its stdin is the terminal.
# These forms never touch a TTY and are ordinary data plumbing, so flagging
# them is a false positive — one that has already cost two authors a
# contorted rewrite of working source:
#
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
# read -r X <<EOF here-doc
# read -r line < "$file" redirect from a file
# printf '%s' "$v" | piped stdin — the pipe ends the
# read -r X PREVIOUS line, not this one
#
# So a `read` is reported only when it has neither a stdin redirection on its
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
# `read -p "..." X` and a bare `read` still fail, which is the case the check
# exists for.
def stdin_redirected(line, prev_line):
# Quoted spans are stripped first so a `<` inside a prompt string is not
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
# must still fail.
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
return '<' in unquoted or prev_line.rstrip().endswith('|')
# A here-doc body is DATA, not command position. Every script in this corpus
# carries a `usage() { cat <<EOF ... EOF; }`, and prose wrapped inside one puts
# ordinary English at the start of a line — "read is reported as an INFO ..."
# in this skill's own validate-provenance.sh, which made skill-audit hard-FAIL
# on its own script. Reflowing that one sentence would have cleared the finding
# and left the cause: every future usage text is one wrap away from the same
# false positive, and the remedy an author reaches for is contorting working
# source, which the note above records has already happened twice.
#
# Detection is deliberately conservative in the direction that matters. A
# here-doc body is skipped only when its terminator is actually found further
# down the file; an opener with no terminator — the shape a stray `<<` inside a
# string would produce — is ignored rather than allowed to swallow the tail,
# because swallowing the tail is a false NEGATIVE and this check exists to fail
# closed. `<<<` here-strings open nothing and are excluded by the lookbehind.
HEREDOC_START_RE = re.compile(r'(?<!<)<<-?\s*(["\']?)([A-Za-z_][A-Za-z0-9_]*)\1')
def heredoc_delimiter(line):
"""The here-doc terminator this line opens, or None."""
m = HEREDOC_START_RE.search(line)
return m.group(2) if m else None
def heredoc_body_indices(lines):
"""Line indices that are here-doc BODY (plus its terminator), not code."""
skip = set()
i, n = 0, len(lines)
while i < n:
stripped = lines[i].strip()
delim = None if stripped.startswith('#') else heredoc_delimiter(lines[i])
if delim:
# `<<-` allows an indented terminator, so compare stripped.
for j in range(i + 1, n):
if lines[j].strip() == delim:
skip.update(range(i + 1, j + 1))
i = j
break
i += 1
return skip
# The here-doc exemption applies to the `read` heuristic ONLY, and the
# asymmetry is the point. `read` is an ordinary English verb, so any prose a
# script prints is one line-wrap away from opening with it. `input(` is not a
# word — a line beginning `input(` inside a here-doc is an embedded Python
# program pausing for a keypress, which is exactly what this check is for, and
# these scripts embed Python in a here-doc as a matter of course. Exempting the
# whole body would have disarmed the check across every script in the corpus.
def interactive_reads(source):
hits = []
prev_line = ''
lines = source.splitlines()
in_heredoc = heredoc_body_indices(lines)
for idx, line in enumerate(lines):
stripped = line.strip()
if idx in in_heredoc:
if re.match(r'input\(', stripped):
hits.append(stripped)
continue
if re.match(r'read(\s|$)', stripped):
if not stdin_redirected(line, prev_line):
hits.append(stripped)
elif re.match(r'input\(', stripped):
hits.append(stripped)
# Blank lines and comments cannot carry the pipe that feeds a
# following `read`, so they never displace the previous line.
if stripped and not stripped.startswith('#'):
prev_line = line
return hits
# Scripts checks
scripts_dir = os.path.join(skill_dir, "scripts")
if os.path.isdir(scripts_dir):
scripts = [f for f in os.listdir(scripts_dir)
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
for fname in scripts:
fpath = os.path.join(scripts_dir, fname)
try:
sc = read_text(fpath)
except EncodingError as exc:
# The executable-bit check below still runs — one unreadable byte
# must not silently drop a second, independent check.
sc = None
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
f"interactive prompts")
interactive = interactive_reads(sc) if sc is not None else []
if interactive:
fail(f"scripts/{fname}: may use interactive input "
f"(read/input from a terminal detected): {interactive[0]}")
elif sc is not None:
ok(f"scripts/{fname}: no interactive prompts detected")
# Executable bit
if os.access(fpath, os.X_OK):
ok(f"scripts/{fname}: is executable")
else:
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
# Summary
print()
for s in suggestions:
print(f"SUGGESTION {s}")
if suggestions:
print()
if not failed:
if suggestions:
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
# SUGGESTION never changes the exit code — only a FAIL does.
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
else:
print("All checks passed.")
sys.exit(0)
else:
print("One or more checks failed.")
sys.exit(1)
PYTHON
KYBERFORGE_ADR0020_RESOLVER_PY
KYBERFORGE_RESOLVER_PY="${KYBERFORGE_RESOLVER_PY%$'\n'}"

View File

@@ -0,0 +1,683 @@
#!/usr/bin/env bash
# lib-checks-agent.sh — SOURCED, never executed.
#
# agent-audit's structural check suite: everything in its validate.sh that is
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
# resolver's markers. validate.sh reassembles
#
# $KYBERFORGE_AGENT_PREAMBLE_PY
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
# $KYBERFORGE_AGENT_BODY_PY
#
# in that order — the order the resolver block sat in the original file — and
# feeds the result to python3 with the agent file as argv[1] and this scripts/
# directory as argv[2]. argv[2] is what the preamble resolves the frontmatter
# allowlist against, and the body's detect_scope() walk-up and provider
# detection are unchanged from agent-audit's copy.
#
# The dimension vocabulary and the tiers here are agent-audit's and are
# deliberately NOT reconciled with lib-checks-skill.sh's. Agents take the
# ADR-0020 description gates and no body word gate at all; adding one would
# contradict the ADR.
#
# Consumed by: validate.sh, agent mode.
# shellcheck shell=bash
# shellcheck disable=SC2034
kyberforge_agent_preflight() {
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
# be measured after YAML folding is resolved, and the hand-rolled reader that
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
# boundary — same description, two verdicts, depending on which reader ran.
# Refusing to start is the only honest option; the repo's jq / apm / vale
# dependencies are declared the same way.
# Check the interpreter separately from the library: `python3 -c` fails the same
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
# dependency sends the reader to install the wrong thing.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
# lib-provenance-*.sh has always exited 2 here; this matches it.
exit 2
fi
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
echo "Error: PyYAML is required but is not importable by python3." >&2
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
exit 2
fi
}
IFS='' read -r -d '' KYBERFORGE_AGENT_PREAMBLE_PY <<'KYBERFORGE_AGENT_PREAMBLE' || true
import sys
import os
import re
import glob
import yaml
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
# default to ASCII, and this script's own message text carries em dashes (the
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
# every check has already run, which loses the whole report and (here) flips a
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
# resolver block below pins the reads; this pins the writes.
#
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
# validate.sh copies print findings, skill-size-check.sh has its own top-level
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
# byte-identity across all three.
for _stream in (sys.stdout, sys.stderr):
try:
_stream.reconfigure(encoding='utf-8')
except AttributeError: # pragma: no cover — Python < 3.7
pass
agent_file = os.path.abspath(sys.argv[1])
script_dir = sys.argv[2]
fname = os.path.basename(agent_file)
# --- Detect provider (check .agent.md before .md) ---
if fname.endswith('.agent.md'):
provider = 'copilot'
name_stem = fname[:-len('.agent.md')]
elif fname.endswith('.md'):
provider = 'claude-code'
name_stem = fname[:-len('.md')]
else:
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
sys.exit(2)
# --- Load agent-field-inventory.md ---
# The merged skill holds one references/ directory for both modes, so every
# flow-specific file is name-prefixed and the agent half of the inventory is
# agent-field-inventory.md (ADR-0025). There is no fallback to the pre-merge
# `field-inventory.md` spelling: agent-audit no longer exists, so a file at that
# name would be a stray, and silently reading it would mean auditing against an
# inventory this skill does not ship.
inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'agent-field-inventory.md'))
if not os.path.isfile(inv_path):
print(f"Error: agent-field-inventory.md not found at {inv_path}", file=sys.stderr)
sys.exit(2)
# Encoding is pinned to UTF-8 rather than inherited from the locale: under
# LC_ALL=C the inherited default is ASCII, and this file legitimately carries
# non-ASCII prose. read_text() in the shared resolver block below does the same
# thing for every other file; this one is read before that block is defined.
try:
with open(inv_path, encoding='utf-8') as f:
inv_content = f.read()
except UnicodeDecodeError as exc:
print(f"Error: agent-field-inventory.md at {inv_path} is not valid UTF-8 "
f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.",
file=sys.stderr)
sys.exit(2)
def parse_section_tokens(content, section_name):
lines = content.splitlines()
for i, line in enumerate(lines):
if line.strip() == f'## {section_name}':
for j in range(i + 1, len(lines)):
stripped = lines[j].strip()
if stripped and not stripped.startswith('#') and not stripped.startswith('---'):
return set(stripped.split())
return set()
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
# Tools the runtime withholds from subagents regardless of the tools field
SUBAGENT_UNAVAILABLE_TOOLS = {
'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers',
}
# Copilot body length limit (chars) — content beyond this is silently truncated
COPILOT_BODY_LIMIT = 30000
# ADR-0020 description budget. An agent's name + description is preloaded into
# every session exactly like a skill's, so agents take the SAME description
# gates. These two constants are DUPLICATED in three places:
# scripts/skill-size-check.sh, lib-checks-skill.sh beside this file, and here.
# The repo-root hook's copy cannot be shared with this skill — a cache-installed
# plugin's scripts cannot read files outside their own plugin directory, and the
# hook cannot reach inside the plugin. The two copies INSIDE this skill could be
# shared (ADR-0025: two files in one skill may source a third), and are not only
# because each mode library is a verbatim lift of the pre-merge suite whose
# constants sit in its Python preamble; hoisting them is a separate change.
# tests/test-skill-size-check.sh asserts all three agree, so drift fails CI
# rather than silently diverging.
#
# Agents deliberately take NO body word gate, and adding one here would
# contradict ADR-0020: a skill body is loaded into the caller's context and
# competes with the live conversation, while an agent body becomes the system
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
# not transfer. Agent body length falls out of the delegation rule instead.
DESC_SUGGEST_CHARS = 250
DESC_MAX_CHARS = 400
# --- Helpers (shared by every scope) ---
failed = False
suggestions = []
def fail(msg):
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
# scripts in the ADR-0020 family now agree: findings that fail the run go to
# stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo
# callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture
# `2>&1`, so nothing a human reads moves.
global failed
failed = True
print(f"FAIL {msg}", file=sys.stderr)
def suggest(msg):
suggestions.append(msg)
def info(msg):
# A check that DECLINED to run says so out loud, rather than passing
# silently. Silence is what let a whole gate family go missing unnoticed.
print(f"INFO {msg}")
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
KYBERFORGE_AGENT_PREAMBLE
KYBERFORGE_AGENT_PREAMBLE_PY="${KYBERFORGE_AGENT_PREAMBLE_PY%$'\n'}"
IFS='' read -r -d '' KYBERFORGE_AGENT_BODY_PY <<'KYBERFORGE_AGENT_BODY' || true
def parse_frontmatter(content):
m = FRONTMATTER_RE.match(strip_bom(content))
if not m:
return None, content
return m.group(1), strip_bom(content)[m.end():]
def extract_field(fm, field):
"""The raw text after `field:` ON ITS OWN LINE, or None.
The character class is `[^\\S\\r\\n]`, never `\\s`: under re.MULTILINE a
`\\s*` after the colon crosses the newline, so `description:` with no value
followed by `model: sonnet` captured `model: sonnet` as the description.
That made the value look present, skipped the "missing or empty" failure,
and then every ADR-0020 gate early-returned on the genuinely empty folded
value — a valueless description exited 0 with zero output on a BLOCKING
pre-push gate. This function is now used only for fields with no folding
semantics (name, tools); description goes through description_value(), the
shared resolver's YAML reader, which is the only thing that can see through
`>`, `null`, `''` and a quoted `"description"` key alike.
"""
m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE)
return m.group(1).strip() if m else None
def get_frontmatter_keys(fm):
keys = set()
for line in fm.splitlines():
m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line)
if m:
keys.add(m.group(1))
return keys
def agent_description(fm, local_fname):
"""The folded description VALUE, or None if it could not be read."""
try:
return description_value(fm)
except FrontmatterError as exc:
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or
# a description of the wrong type. Do not prefix a diagnosis here; the
# last one named a syntax error for two failures that have none.
fail(f"{exc} — the ADR-0020 description and boundary-target gates could "
f"not run — {local_fname}")
return None
def check_description_budget(value, local_fname, by_hand=False):
"""ADR-0020 description gates — identical for every scope.
`by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent
carrying `disable-model-invocation: true` is absent from the model-visible
listing, so the 250-character SUGGESTION — a routing-quality budget — has
no listing to apply to. The 400-character ceiling is unaffected.
"""
if not value:
return
dlen = len(value)
if dlen > DESC_MAX_CHARS:
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
f"and a boundary clause; move capability enumeration, output-format detail, "
f"composition notes and implementation detail to the body — {local_fname}")
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
f"what moves the corpus average; the FAIL tier only stops outliers "
f"— {local_fname}")
def check_boundary(value, fpath, local_fname, by_hand=False):
"""ADR-0020 boundary clause + resolvable boundary targets.
agent-author's SKILL.md states that an agent's boundary targets must
resolve, but until this ran no script checked it — the contract was
documented and unenforced. The resolution universe is derived from the
AGENT FILE's own location (the authoring root above it, its own apm
package, and that package's declared apm dependencies), never from this
script's path, and — when an authoring root exists — never from a deployed
.claude/ tree, so a fresh clone and a machine that has run `apm install`
return the same verdict.
"""
if not value:
return
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
# this particular agent warrants a boundary clause is judgment. All four
# agents in this corpus currently lack one.
#
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could
# not parse" are different findings (issue #110). And a hand-invoked agent is
# exempt from the clause altogether (issue #108) — the boundary-target
# resolution below still runs, because a target it DOES name should still
# resolve.
status = boundary_clause_status(value) if not by_hand else 'present'
if status == 'absent':
suggest(f"description has no boundary clause — add the prose form (\"Do not use "
f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
f"so the router knows where NOT to send this agent — {local_fname}")
elif status == 'unparsed':
suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no "
f"target could be read, so the dangling-target check did not run on it — "
f"the clause is PRESENT and unparsed, not missing. Most often the target "
f"is a single word, which is deliberately not matchable bare: write it as "
f"`name` or /name — {local_fname}")
if not by_hand:
# One arrow, one target: a second name after the same arrow is resolved
# by nothing and reported by nothing (issue #107).
for first, second in multi_target_arrow_clauses(value):
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
f"'{second}') and only the first is resolved — the second is checked by "
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
f"Not Y -> {second}.\" — {local_fname}")
targets = boundary_targets(value)
if not targets:
return
known = known_targets(os.path.dirname(os.path.abspath(fpath)))
if not known:
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
f"determined for this path (no authoring root above it, no apm package "
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}")
return
# blocking vs reported: a target only earns a FAIL when it is written in
# route notation or its own sentence corroborates it by naming another target
# that resolves. See the shared resolver's CORROBORATION note.
blocking, reported = unresolved_targets(value, known)
for target in blocking:
fail(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
f"target sends the router nowhere — {local_fname}")
for target in reported:
suggest(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
f"else in that sentence resolves, so it is equally likely to be a tool, a "
f"file format or an English compound. If it IS a route, write it as "
f"`/{target}` or `-> {target}` and it will be checked properly — "
f"{local_fname}")
def extract_tools_list(fm):
"""Tool names from the `tools` field — inline scalar OR YAML block sequence.
Read off the PARSED mapping, never off extract_field(). That function's
capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:`
written as a block sequence — the shape Copilot agent files use — captured
nothing at all and the subagent-unavailable-tool check silently stopped
firing on exactly the files it was written for. Both spellings are legal
YAML, so both are read here.
"""
try:
data = yaml.safe_load(fm)
except Exception:
# Not this function's failure to report: the frontmatter's validity is
# decided (and failed) by agent_description() on the same text.
return set()
if not isinstance(data, dict):
return set()
val = data.get('tools')
if isinstance(val, list):
items = [str(item).strip() for item in val]
elif isinstance(val, str):
items = re.split(r'[\s,]+', val.strip())
else:
return set()
return {item for item in items if item}
def is_copilot_cloud_ide(fpath):
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
# --- Detect scope ---
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
# exactly one of the four package content types. Group 1 captures an optional
# opening quote; \1 requires the same character (or nothing) to close it, so
# "skill" and '"skill"' both match but a mismatched quote doesn't. The value
# must then be followed by whitespace or end-of-line — not just a non-word
# character — so a malformed value like `prompts-only` is correctly rejected
# instead of false-matching on the `prompts` prefix.
APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
def find_apm_package_root(apm_yml_path):
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
manifest, not a type:-less marketplace-only apm.yml)."""
# errors='replace', not a hard failure: this only asks whether a `type:`
# line exists, and a stray undecodable byte elsewhere in someone else's
# apm.yml must not abort scope detection.
with open(apm_yml_path, encoding='utf-8', errors='replace') as f:
for line in f:
if APM_TYPE_RE.match(line):
return True
return False
def detect_scope(start_dir):
home = os.path.expanduser('~')
original_start = os.path.abspath(start_dir)
# Agent files conventionally live exactly two path segments below their
# scope root — <root>/.claude/agents, <root>/.github/agents,
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
# recovers the same root new-agent.sh would have been invoked with to
# produce this exact file, independent of how far the walk below has to
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
# `root` vs `current` distinction even though validate.sh is handed a
# file's directory, not the scope root itself.
#
# That arithmetic is only trustworthy when the path actually has this
# shape: parent directory literally named "agents", grandparent one of
# the four known scope-dir names. A hand-placed or otherwise
# non-conventional agent file (never produced by new-agent.sh) has no
# such guarantee — blindly trusting two-segments-up there could point at
# an unrelated ancestor. conventional_shape gates every use of
# conventional_root below; when it's false, the walked-to `current`
# directory is used instead, the same fallback this function used before
# conventional_root existed.
scope_dir_name = os.path.basename(os.path.dirname(original_start))
conventional_shape = (
os.path.basename(original_start) == 'agents'
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
)
conventional_root = os.path.dirname(os.path.dirname(original_start))
current = original_start
while True:
# The filesystem root is never a candidate, the same guard the shared
# resolver's walk-up loops carry. Without it a file under a marker-less
# temp directory walked all the way to `/` and returned it as the scope
# root, which then reported `counterpart file not found:
# /.claude/agents/<name>.md` — a path that names someone else's machine,
# not the user's project. When the walk runs out, the agent file's own
# directory (or its conventional root) is the honest answer.
if _is_fs_root(current):
return 'project', conventional_root if conventional_shape else original_start
apm_yml = os.path.join(current, 'apm.yml')
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
return 'plugin', current
# $HOME is the user-scope boundary — checked before the .git test
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
# can't shadow user scope by being its own .git repo. 'user' scope
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
# conventional two-segments-below-root depth (i.e. $HOME IS that
# root, matching the real ~/.claude/agents or ~/.copilot/agents
# shape). Any other walk-up into $HOME — a marker-less directory
# nested deeper than that convention — resolves to project scope
# instead: a stray directory under $HOME can't be silently
# redirected into the shared global ~/.claude or ~/.copilot agent
# directories.
if current == home:
if original_start == home or (conventional_shape and conventional_root == home):
return 'user', home
return 'project', conventional_root if conventional_shape else current
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
# in a git worktree — exists() covers both. Returns conventional_root,
# not current: new-agent.sh's project-scope file placement always
# uses its `$ROOT` argument directly, never the walked-up `.git`
# location, so a <root> one or more levels below the repo's .git
# (a subdirectory of a larger git-tracked tree — explicitly a
# supported case per new-agent.sh's usage text) must resolve to the
# same root new-agent.sh actually wrote to, not to the .git dir —
# unless the path lacks the conventional shape, in which case that
# arithmetic isn't trustworthy and current is used instead.
if os.path.exists(os.path.join(current, '.git')):
return 'project', conventional_root if conventional_shape else current
parent = os.path.dirname(current)
if parent == current:
return 'project', conventional_root if conventional_shape else current
current = parent
agent_dir = os.path.dirname(agent_file)
scope, scope_root = detect_scope(agent_dir)
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
def check_apm_agent_file(fpath, allowlist, stem):
local_fname = os.path.basename(fpath)
try:
content = read_text(fpath)
except EncodingError as exc:
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
f"failure, not a skip — {local_fname}")
return
except OSError as exc:
# A path that cannot be opened gets a FAIL line naming it, not a bare
# FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes
# this path for an agent file deleted from the worktree but still
# tracked in the index — a real, expected state, and the caller needs to
# be told which file, not handed an interpreter stack.
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
f"be measured, so this is a hard failure, not a skip — {local_fname}")
return
fm, body = parse_frontmatter(content)
if fm is None:
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
f"after either marker and CRLF endings are all tolerated). Nothing could be "
f"measured, so this is a hard failure, not a skip — {local_fname}")
return
# The apm-agent.md template embeds its authoring guidance as HTML
# comments inside the frontmatter block (so they render invisible in a
# Markdown preview but stay visible in the raw file). get_frontmatter_keys
# silently ignores any line that isn't a `key:` match, so a comment left
# behind at ship time would otherwise pass unnoticed — yet apm compile
# copies this frontmatter verbatim to both harnesses, and `<!-- -->` is
# not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016).
if re.search(r'<!--|-->', fm):
fail(f"frontmatter still contains template HTML comments (<!-- ... -->) "
f"— delete them before shipping — {local_fname}")
# Allowlist: the permitted keys are data, read at load time from
# references/agent-field-inventory.md's `## apm-agent-allowlist` section — do not
# restate them here, or this comment goes stale the next time that line
# changes. apm compile verbatim-copies frontmatter to every target, so a key
# outside the list is unsafe on at least one harness (ADR-0016). Note the
# list admits denylist-shaped restrictions (disallowedTools) but never
# allowlist-shaped ones (tools), whose value shape differs per harness.
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key not in allowlist:
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
f"({', '.join(sorted(allowlist))}) — {local_fname}")
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
name_val = extract_field(fm, 'name')
if not name_val:
fail(f"name field is missing or empty — {local_fname}")
else:
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
if name_val != stem:
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
# description — required, non-empty, no placeholder
# Presence is decided on the FOLDED value, never on a line regex. Deciding
# it on extract_field's raw capture is what let `description:` with no value
# pass this gate in total silence: the capture picked up the next key, so
# "missing or empty" never fired, and every ADR-0020 check below then
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
folded = agent_description(fm, local_fname)
if folded is None:
pass # frontmatter is not valid YAML — agent_description already failed
elif not folded:
fail(f"description field is missing or empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(folded):
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
by_hand = hand_invoked(fm)
check_description_budget(folded, local_fname, by_hand)
check_boundary(folded, fpath, local_fname, by_hand)
# body — required, non-empty, no placeholder; same Copilot truncation risk
# applies since this file compiles verbatim into a real Copilot file downstream.
if not body.strip():
fail(f"system prompt body is empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(body):
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
if len(body) > COPILOT_BODY_LIMIT:
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
f"content beyond the limit is silently truncated by the Copilot runtime "
f"once apm compile emits it downstream — {local_fname}")
if scope == 'plugin':
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
for s in suggestions:
print(f"SUGGESTION {s}")
sys.exit(1 if failed else 0)
# --- Project/user scope: unchanged CC/Copilot pair validation ---
# --- Derive counterpart path ---
if scope == 'project':
if provider == 'claude-code':
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
else: # user
home = os.path.expanduser('~')
if provider == 'claude-code':
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
counterpart_provider = 'copilot'
else:
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
counterpart_provider = 'claude-code'
def check_file(fpath, file_provider):
local_fname = os.path.basename(fpath)
try:
content = read_text(fpath)
except EncodingError as exc:
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
f"failure, not a skip — {local_fname}")
return
except OSError as exc:
# Same reason as check_apm_agent_file's: a diagnostic naming the path
# beats a FileNotFoundError traceback. The counterpart is pre-checked at
# the bottom of this script, but agent_file itself never was.
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
f"be measured, so this is a hard failure, not a skip — {local_fname}")
return
fm, body = parse_frontmatter(content)
if fm is None:
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
f"after either marker and CRLF endings are all tolerated). Nothing could be "
f"measured, so this is a hard failure, not a skip — {local_fname}")
return
# name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents
cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath))
name_val = extract_field(fm, 'name')
if not cloud_ide:
if not name_val:
fail(f"name field is missing or empty — {local_fname}")
else:
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
# Stem check applies to Copilot CLI only; CC docs say filename need not match name
if file_provider == 'copilot':
stem = local_fname[:-len('.agent.md')]
if name_val != stem:
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
# cloud/IDE: name is optional, but if present it must be valid
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
# description
# Presence is decided on the FOLDED value, never on a line regex. Deciding
# it on extract_field's raw capture is what let `description:` with no value
# pass this gate in total silence: the capture picked up the next key, so
# "missing or empty" never fired, and every ADR-0020 check below then
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
folded = agent_description(fm, local_fname)
if folded is None:
pass # frontmatter is not valid YAML — agent_description already failed
elif not folded:
fail(f"description field is missing or empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(folded):
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
by_hand = hand_invoked(fm)
check_description_budget(folded, local_fname, by_hand)
check_boundary(folded, fpath, local_fname, by_hand)
# body
if not body.strip():
fail(f"system prompt body is empty — {local_fname}")
else:
if PLACEHOLDER_RE.search(body):
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
# Copilot body length limit
if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT:
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}")
# CC-only fields in Copilot file
if file_provider == 'copilot':
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key in cc_only_fields:
fail(f"CC-only field '{key}' present in Copilot file — {local_fname}")
# Copilot-only fields in CC file
if file_provider == 'claude-code':
fm_keys = get_frontmatter_keys(fm)
for key in sorted(fm_keys):
if key in copilot_only_fields:
fail(f"Copilot-only field '{key}' present in CC file — {local_fname}")
# Subagent-unavailable tools listed in tools field
tools = extract_tools_list(fm)
unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS
for tool in sorted(unavailable):
suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}")
# --- Check counterpart exists ---
if not os.path.isfile(counterpart):
fail(f"counterpart file not found: {counterpart}")
sys.exit(1)
# --- Check both files ---
check_file(agent_file, provider)
check_file(counterpart, counterpart_provider)
for s in suggestions:
print(f"SUGGESTION {s}")
sys.exit(1 if failed else 0)
KYBERFORGE_AGENT_BODY
KYBERFORGE_AGENT_BODY_PY="${KYBERFORGE_AGENT_BODY_PY%$'\n'}"

View File

@@ -0,0 +1,621 @@
#!/usr/bin/env bash
# lib-checks-skill.sh — SOURCED, never executed.
#
# skill-audit's structural check suite: everything in its validate.sh that is
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
# resolver's markers. validate.sh reassembles
#
# $KYBERFORGE_SKILL_PREAMBLE_PY
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
# $KYBERFORGE_SKILL_BODY_PY
#
# in that order — the order the resolver block sat in the original file — and
# feeds the result to python3, so every check runs against the same names it
# always did.
#
# The dimension vocabulary, the message wording and the PASS/FAIL/SUGGESTION/
# INFO tiers here are skill-audit's and are deliberately NOT reconciled with
# lib-checks-agent.sh's. The two suites disagree on purpose: a skill body is
# loaded into the caller's context, an agent body becomes the system prompt of
# a fresh one, so ADR-0020 gives skills a body word budget and agents none.
#
# Consumed by: validate.sh, skill mode.
# shellcheck shell=bash
# shellcheck disable=SC2034
kyberforge_skill_preflight() {
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
# be measured after YAML folding is resolved, and the hand-rolled reader that
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
# boundary — same description, two verdicts, depending on which reader ran.
# Refusing to start is the only honest option; the repo's jq / apm / vale
# dependencies are declared the same way.
# Check the interpreter separately from the library: `python3 -c` fails the same
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
# dependency sends the reader to install the wrong thing.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
# lib-provenance-*.sh has always exited 2 here; this matches it.
exit 2
fi
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
echo "Error: PyYAML is required but is not importable by python3." >&2
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
exit 2
fi
}
IFS='' read -r -d '' KYBERFORGE_SKILL_PREAMBLE_PY <<'KYBERFORGE_SKILL_PREAMBLE' || true
import sys
import os
import re
import glob
import yaml
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
# default to ASCII, and this script's own message text carries em dashes (the
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
# every check has already run, which loses the whole report and (here) flips a
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
# resolver block below pins the reads; this pins the writes.
#
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
# validate.sh copies print findings, skill-size-check.sh has its own top-level
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
# byte-identity across all three.
for _stream in (sys.stdout, sys.stderr):
try:
_stream.reconfigure(encoding='utf-8')
except AttributeError: # pragma: no cover — Python < 3.7
pass
skill_dir = os.path.abspath(sys.argv[1])
skill_md = os.path.join(skill_dir, "SKILL.md")
if not os.path.isfile(skill_md):
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
sys.exit(1)
failed = False
suggestions = []
def ok(msg):
print(f"PASS {msg}")
def fail(msg):
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
# scripts in the ADR-0020 family now agree: findings that fail the run go to
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
# Both repo callers capture `2>&1`, so nothing a human reads moves.
global failed
print(f"FAIL {msg}", file=sys.stderr)
failed = True
def suggest(msg):
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
# factory-audit's SKILL.md Step 4 report counts them into its
# `PASS (N suggestions)` result line, which is what makes the ADR-0020 SUGGESTION tier visible
# rather than another silently-ignored warning (ADR-0013).
suggestions.append(msg)
def info(msg):
# A check that DECLINED to run says so out loud, rather than passing
# silently. Silence is what let a whole gate family go missing unnoticed.
print(f"INFO {msg}")
KYBERFORGE_SKILL_PREAMBLE
KYBERFORGE_SKILL_PREAMBLE_PY="${KYBERFORGE_SKILL_PREAMBLE_PY%$'\n'}"
IFS='' read -r -d '' KYBERFORGE_SKILL_BODY_PY <<'KYBERFORGE_SKILL_BODY' || true
# A leading BOM is stripped before anything is parsed or counted. It changes
# neither count below — it is not a line separator and str.split() does not
# treat it as whitespace — but it did defeat the frontmatter match.
try:
content = strip_bom(read_text(skill_md))
except EncodingError as exc:
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
f"hard failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
# --- Parse frontmatter ---
fm_match = FRONTMATTER_RE.match(content)
if not fm_match:
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
"spaces after either marker and CRLF endings are all tolerated). Nothing "
"downstream can be measured, so this is a hard failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
fm = fm_match.group(1)
body_start = fm_match.end()
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
# by `description: ...` captured the NEXT KEY as the name and reported a
# mismatch instead of an absence. Same class of bug as the `description:` one
# the shared resolver's description_value() docstring records.
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
name = name_m.group(1).strip('"\'') if name_m else ""
# Extract description — the VALUE, with YAML folding resolved. Most of this
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
# carry indentation and newlines that are not part of the value: every length
# measurement below is wrong unless the scalar is folded first.
try:
desc = description_value(fm)
except FrontmatterError as exc:
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or a
# description of the wrong type. Do not prefix a diagnosis here; the last
# one named a syntax error for two failures that have none.
fail(f"{exc}. Nothing downstream can be measured, so this is a hard "
f"failure, not a skip")
print("One or more checks failed.")
sys.exit(1)
dir_name = os.path.basename(skill_dir)
# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation:
# true` takes the skill out of the model-visible listing entirely, so the
# trigger/capability/boundary rules and the 250-character routing target do not
# apply to it — the audit's own references/skill-description-quality.md Step 0 says
# so, and until this line existed no check here knew the field existed. What the
# flag does NOT lift: the body word budget and the 400-character description
# ceiling. See the shared resolver's hand_invoked().
by_hand = hand_invoked(fm)
# --- Checks ---
# name present
if name:
ok(f"name present: '{name}'")
else:
fail("name field is missing or empty")
# name matches directory
if name and dir_name:
if name == dir_name:
ok(f"name '{name}' matches directory '{dir_name}'")
else:
fail(f"name '{name}' does not match directory '{dir_name}'")
# name length
if name:
if len(name) <= 64:
ok(f"name length {len(name)} chars (limit: 64)")
else:
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
# name format
if name:
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
ok("name format valid (kebab-case)")
else:
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
# description present
if desc:
ok("description present")
else:
fail("description field is missing or empty")
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
# sits underneath it rather than replacing it.
if desc:
dlen = len(desc)
if dlen <= 1024:
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
else:
fail(f"description length {dlen} chars — exceeds 1024-character limit")
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
# but not backtick-quoted references like `FILL IN:` used in instructions.
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
# description contains unfilled placeholder
if desc and PLACEHOLDER_RE.search(desc):
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
else:
if desc:
ok("description has no unfilled placeholders")
# --- ADR-0022: metadata.version is mandatory -------------------------------
# FAIL, not SUGGESTION, and the tier is set by the gate rather than by taste.
# `.pre-commit-config.yaml`'s `skill-size-check` hook REJECTS a SKILL.md with
# no `metadata.version`, and rejects a value that is not three-part semver.
# skill-author's Step 4 says to run this audit and "resolve every FAIL", so any
# tier below FAIL lets that step report done on a skill the commit gate then
# refuses — the same audit-disagrees-with-the-gate failure the MAX_LINES note
# below warns about, arrived at from the other direction. Verified before this
# check existed: a SKILL.md with no `metadata:` block at all reported "All
# checks passed".
#
# The rule is DUPLICATED from that hook for the same cache-isolation reason as
# every other constant here — an installed plugin's scripts cannot read the
# repo-root config. Keep the two in step: this check must accept exactly what
# the hook accepts.
SEMVER_RE = re.compile(r'^\d+\.\d+\.\d+$')
try:
fm_data = yaml.safe_load(fm)
except Exception:
# Unreachable in practice: description_value() above parses the same text
# and hard-exits on a YAML error, so anything arriving here already parsed.
fm_data = None
metadata_block = fm_data.get('metadata') if isinstance(fm_data, dict) else None
if not isinstance(metadata_block, dict) or metadata_block.get('version') is None:
fail("frontmatter has no metadata.version — ADR-0022 makes it mandatory for "
"every skill, and the skill-size-check pre-commit hook rejects the file "
"without it. Add `metadata:` / ` version: \"1.0.0\"` (new skills start "
"at \"0.1.0\")")
else:
version_value = metadata_block['version']
# NOT str()-coerced blind: `version: 1.0` is a YAML float, and its "1.0"
# spelling is exactly the two-part value the hook rejects — coercing and
# then matching keeps this check and the hook agreeing on that case.
version_text = version_value if isinstance(version_value, str) else str(version_value)
version_text = version_text.strip()
if SEMVER_RE.match(version_text):
ok(f"metadata.version present: '{version_text}' (ADR-0022)")
else:
fail(f"metadata.version '{version_text}' is not three-part semver — the "
f"skill-size-check pre-commit hook rejects it. Use MAJOR.MINOR.PATCH, "
f"e.g. \"1.0.0\"")
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
# outside the plugin directory once the plugin is cache-installed, so there is
# no single source to share. Keep the two in sync by hand: if they drift, this
# audit will report a skill ready to ship that the commit hook then rejects.
MAX_LINES = 500
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
# for the full measurement.
MAX_WORDS = 2770
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
# agree, so drift fails CI instead of shipping an audit that disagrees with the
# commit hook. lib-checks-agent.sh, beside this file, holds a third copy of the
# two description constants; per ADR-0020 agents take the description gates and
# deliberately take NO body word gate, because an agent body becomes the system
# prompt of a fresh context rather than competing with a live conversation.
#
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
# quality gate. Likewise the 1024-character description limit above is the
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
# underneath it.
DESC_SUGGEST_CHARS = 250
DESC_MAX_CHARS = 400
BODY_SUGGEST_WORDS = 600
BODY_MAX_WORDS = 900
line_count = len(content.splitlines())
if line_count <= MAX_LINES:
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
else:
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
# str.split() with no argument splits on runs of whitespace, matching the
# `wc -w` the hook uses, and counts the whole file including frontmatter.
word_count = len(content.split())
if word_count <= MAX_WORDS:
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
else:
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
body = content[body_start:]
# --- ADR-0020: description budget -----------------------------------------
if desc:
dlen = len(desc)
if dlen > DESC_MAX_CHARS:
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
f"and a boundary clause; move capability enumeration, output-format detail, "
f"composition notes and implementation detail to the body or README.md")
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
f"what moves the corpus average; the FAIL tier only stops outliers")
elif by_hand:
ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character "
f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)")
else:
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
# --- ADR-0020: body budget -------------------------------------------------
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
# This is a different measurement from MAX_WORDS above, which counts the whole
# file including frontmatter as a spec-conformance backstop. Both are reported.
body_word_count = len(body.split())
if body_word_count > BODY_MAX_WORDS:
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
f"and rationale prose to references/ behind an explicit "
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
f"common to every branch, each flow gets its own self-contained references/ file")
elif body_word_count > BODY_SUGGEST_WORDS:
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
else:
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
# --- Reference pointers must exist -----------------------------------------
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
# on disk is a hard break, and until this check existed nothing in the
# gate/audit/vale stack noticed it — all three exited 0.
missing_refs = missing_reference_pointers(body, skill_dir)
for ref in missing_refs:
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
if not missing_refs:
ok("all referenced references/ files exist")
# --- Gotchas discipline -----------------------------------------------------
# SUGGESTION on both counts: the measurement is deterministic, but whether a
# given gotcha earns its place in the body is the auditor's judgment.
gotchas = gotcha_stats(body)
if gotchas is not None:
gotcha_entries, gotcha_words = gotchas
if gotcha_entries > GOTCHA_MAX_ENTRIES:
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
f"missing references/ file or a design problem written up as a warning")
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
f"parts to references/ and keep the section for live traps")
# --- ADR-0020: boundary clause present -------------------------------------
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
# this particular skill warrants a boundary clause is judgment. Both accepted
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
#
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not
# parse" are different findings, and reporting the first for the second sends
# the author hunting for a problem that is not there (issue #110).
#
# Skipped entirely for a hand-invoked skill — the contract gives it one plain
# sentence with no boundary clause, so the finding would be wrong and its remedy
# names a router that cannot see the skill (issue #108).
if desc and by_hand:
ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger "
"rules do not apply; audited as one plain human-facing sentence")
elif desc:
status = boundary_clause_status(desc)
if status == 'present':
ok("description has a boundary clause")
elif status == 'absent':
suggest("description has no boundary clause — add the prose form (\"Do not use "
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
"so the router knows where NOT to send this skill")
else:
suggest("description has an arrow boundary clause (\"Not X -> y\") from which no "
"target could be read, so the dangling-target check did not run on it — "
"the clause is PRESENT and unparsed, not missing. Most often the target is "
"a single word, which is deliberately not matchable bare because "
"`research`, `triage` and `forge` are all ordinary English: write it as "
"`name` or /name")
# One arrow, one target. A second name after the same arrow is resolved by
# nothing and reported by nothing, so the clause claims coverage it does not
# have and this script printed "1 of 1 boundary target(s) resolve" on a
# clause naming two (issue #107).
for first, second in multi_target_arrow_clauses(desc):
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
f"'{second}') and only the first is resolved — the second is checked by "
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
f"Not Y -> {second}.\"")
# --- ADR-0020: resolvable boundary targets ---------------------------------
# The resolution universe comes from the SKILL's own location: the authoring
# root above it (every sibling plugin in the monorepo), its own apm package, and
# the packages that package declares in apm.yml dependencies.apm. It is never
# derived from this script's own path, and — when an authoring root exists — it
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
# run `apm install` return the same verdict. See the shared resolver's header.
if desc:
routing_targets = boundary_targets(desc)
known = known_targets(skill_dir) if routing_targets else set()
if routing_targets and not known:
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
f"determined for this path (no authoring root above it, no apm package "
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
elif routing_targets:
# blocking vs reported: a target only earns a FAIL when it is written in
# route notation or its own sentence corroborates it by naming another
# target that resolves. See the shared resolver's CORROBORATION note.
unresolved, soft = unresolved_targets(desc, known)
for target in unresolved:
fail(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
f"target sends the router nowhere")
for target in soft:
suggest(f"description routes to '{target}', which resolves to no skill or agent "
f"in this monorepo, in this package, or in a package it declares in "
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
f"else in that sentence resolves, so it is equally likely to be a tool, a "
f"file format or an English compound. If it IS a route, write it as "
f"`/{target}` or `-> {target}` and it will be checked properly")
if not unresolved:
# Counts the targets that ACTUALLY resolve, not every target found:
# a confirm-only target (one used attributively — see the resolver's
# ATTRIBUTIVE USE note) is exempt from the failure above, so
# reporting it as resolved would be a false claim.
resolved = [t for t in routing_targets if normalize_target(t) in known]
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
f"{', '.join(resolved) if resolved else '(none)'}")
# Body unfilled placeholders
fill_matches = PLACEHOLDER_RE.findall(body)
if fill_matches:
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
else:
ok("SKILL.md body has no unfilled placeholders")
# Interactive prompt heuristic.
#
# A line-initial `read` only blocks an agent when its stdin is the terminal.
# These forms never touch a TTY and are ordinary data plumbing, so flagging
# them is a false positive — one that has already cost two authors a
# contorted rewrite of working source:
#
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
# read -r X <<EOF here-doc
# read -r line < "$file" redirect from a file
# printf '%s' "$v" | piped stdin — the pipe ends the
# read -r X PREVIOUS line, not this one
#
# So a `read` is reported only when it has neither a stdin redirection on its
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
# `read -p "..." X` and a bare `read` still fail, which is the case the check
# exists for.
def stdin_redirected(line, prev_line):
# Quoted spans are stripped first so a `<` inside a prompt string is not
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
# must still fail.
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
return '<' in unquoted or prev_line.rstrip().endswith('|')
# A here-doc body is DATA, not command position. Every script in this corpus
# carries a `usage() { cat <<EOF ... EOF; }`, and prose wrapped inside one puts
# ordinary English at the start of a line — "read is reported as an INFO ..."
# in this skill's own validate-provenance.sh, which made the skill audit
# hard-FAIL on its own script. Reflowing that one sentence would have cleared the finding
# and left the cause: every future usage text is one wrap away from the same
# false positive, and the remedy an author reaches for is contorting working
# source, which the note above records has already happened twice.
#
# Detection is deliberately conservative in the direction that matters. A
# here-doc body is skipped only when its terminator is actually found further
# down the file; an opener with no terminator — the shape a stray `<<` inside a
# string would produce — is ignored rather than allowed to swallow the tail,
# because swallowing the tail is a false NEGATIVE and this check exists to fail
# closed. `<<<` here-strings open nothing and are excluded by the lookbehind.
HEREDOC_START_RE = re.compile(r'(?<!<)<<-?\s*(["\']?)([A-Za-z_][A-Za-z0-9_]*)\1')
def heredoc_delimiter(line):
"""The here-doc terminator this line opens, or None."""
m = HEREDOC_START_RE.search(line)
return m.group(2) if m else None
def heredoc_body_indices(lines):
"""Line indices that are here-doc BODY (plus its terminator), not code."""
skip = set()
i, n = 0, len(lines)
while i < n:
stripped = lines[i].strip()
delim = None if stripped.startswith('#') else heredoc_delimiter(lines[i])
if delim:
# `<<-` allows an indented terminator, so compare stripped.
for j in range(i + 1, n):
if lines[j].strip() == delim:
skip.update(range(i + 1, j + 1))
i = j
break
i += 1
return skip
# The here-doc exemption applies to the `read` heuristic ONLY, and the
# asymmetry is the point. `read` is an ordinary English verb, so any prose a
# script prints is one line-wrap away from opening with it. `input(` is not a
# word — a line beginning `input(` inside a here-doc is an embedded Python
# program pausing for a keypress, which is exactly what this check is for, and
# these scripts embed Python in a here-doc as a matter of course. Exempting the
# whole body would have disarmed the check across every script in the corpus.
def interactive_reads(source):
hits = []
prev_line = ''
lines = source.splitlines()
in_heredoc = heredoc_body_indices(lines)
for idx, line in enumerate(lines):
stripped = line.strip()
if idx in in_heredoc:
if re.match(r'input\(', stripped):
hits.append(stripped)
continue
if re.match(r'read(\s|$)', stripped):
if not stdin_redirected(line, prev_line):
hits.append(stripped)
elif re.match(r'input\(', stripped):
hits.append(stripped)
# Blank lines and comments cannot carry the pipe that feeds a
# following `read`, so they never displace the previous line.
if stripped and not stripped.startswith('#'):
prev_line = line
return hits
# Scripts checks
scripts_dir = os.path.join(skill_dir, "scripts")
if os.path.isdir(scripts_dir):
scripts = [f for f in os.listdir(scripts_dir)
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
for fname in scripts:
fpath = os.path.join(scripts_dir, fname)
try:
sc = read_text(fpath)
except EncodingError as exc:
# The executable-bit check below still runs — one unreadable byte
# must not silently drop a second, independent check.
sc = None
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
f"interactive prompts")
interactive = interactive_reads(sc) if sc is not None else []
if interactive:
fail(f"scripts/{fname}: may use interactive input "
f"(read/input from a terminal detected): {interactive[0]}")
elif sc is not None:
ok(f"scripts/{fname}: no interactive prompts detected")
# Executable bit
if os.access(fpath, os.X_OK):
ok(f"scripts/{fname}: is executable")
else:
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
# Summary
print()
for s in suggestions:
print(f"SUGGESTION {s}")
if suggestions:
print()
if not failed:
if suggestions:
# Feeds SKILL.md Step 4's `PASS (N suggestions)` result line. A
# SUGGESTION never changes the exit code — only a FAIL does.
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
else:
print("All checks passed.")
sys.exit(0)
else:
print("One or more checks failed.")
sys.exit(1)
KYBERFORGE_SKILL_BODY
KYBERFORGE_SKILL_BODY_PY="${KYBERFORGE_SKILL_BODY_PY%$'\n'}"

View File

@@ -0,0 +1,134 @@
#!/usr/bin/env bash
# lib-contributing-files.sh — SOURCED, never executed.
#
# The shared Contributing-files parser, as ONE copy for this skill. Both of
# validate-provenance.sh's modes compose it into the Python program they run.
# Before the merge it was embedded twice. The two copies drifted once
# (484357a) into different spellings of the bullet loop — behaviourally
# identical, but unchecked while a docstring asserted they matched — and were
# re-unified at 598a7c3, so they were byte-identical when ADR-0025 merged them.
# tests/test-adr0020-contract.sh pins this file as the sole authority so that
# a second copy cannot reappear.
#
# Held in a shell variable filled from a QUOTED here-doc for the same reason as
# lib-boundary-resolver.sh's block: nothing inside is expanded, so the text
# between the two markers below stays byte-identical to the copies the contract
# test reads, and the markers stay on lines of their own, at column 0, exactly
# once each, so the same sed range extracts the same span.
#
# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`.
# validate-provenance.sh sources this file before either mode's python3
# preflight, so a `cat` here made coreutils a hard dependency ahead of python3:
# on a PATH with neither, the script exited 127 naming `cat` instead of reaching
# the preflight that names python3. `read -r -d ''` reads to a NUL that never
# arrives and so returns non-zero at EOF — hence the `|| true` — and it keeps the
# last line's newline, which the joining newline in the caller would otherwise
# double — hence the single strip after the delimiter. It removes exactly ONE
# newline, never a run: blank lines at the end of a chunk are program text, and
# stripping every trailing newline deleted them. The here-doc itself is
# unchanged.
#
# Consumed by: validate-provenance.sh (both modes), via
# $KYBERFORGE_CONTRIBUTING_FILES_PY.
# shellcheck shell=bash
# shellcheck disable=SC2034
IFS='' read -r -d '' KYBERFORGE_CONTRIBUTING_FILES_PY <<'KYBERFORGE_CONTRIBUTING_FILES' || true
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, and since ADR-0025 exactly one copy of it: this file, sourced by
# validate-provenance.sh for both the skill and the agent flow. It used to be
# embedded verbatim in skill-audit's and agent-audit's separate
# validate-provenance.sh copies, because a cache-installed plugin's scripts
# cannot read files outside their own plugin directory and no single file was
# reachable by both skills. Merging those skills removed that constraint: two
# files in ONE skill directory can source a third. The two copies had drifted
# once before (cosmetically, and re-unified before the merge) while a docstring
# claimed they had not, which is why tests/test-adr0020-contract.sh now pins
# this file as the SOLE authority — that it exists, that validate-provenance.sh
# sources it, and that nothing anywhere has re-inlined the parser. Do not paste
# this block into a caller.
#
# Before the contract test pinned it, the agent-side copy's docstring merely
# ASSERTED the two copies were "behaviourally identical" and nothing checked
# it — which is how the two diverged spellings of the bullet loop went
# unnoticed at 484357a.
#
# Requires: re (imported by the host script).
def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content.
Both authored forms are accepted, because both are in use across the
corpus and only recognising the first silently skipped the contributing-
file checks on every sources.md written the other way:
- **Contributing files:** SKILL.md, references/a.md
**Contributing files:**
- SKILL.md (what this source contributed)
- references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped.
Note the bullet form's notes may themselves contain commas, so the list
is built per bullet rather than by splitting the joined value.
The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
"""
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
def strip_note(entry):
# "references/a.md (why)" -> "references/a.md"
return re.sub(r'\s*\(.*$', '', entry).strip()
# Inline form: value on the same line, comma-separated, no notes.
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if cf_m:
value = cf_m.group(1).strip()
if value.startswith("(none"):
return []
return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m:
return None
files = []
for line in block[cf_m.end():].splitlines():
line = line.strip()
if not line:
if files:
break
continue
if not line.startswith("- "):
break
entry = line[2:].strip()
if entry.startswith("(none"):
return []
entry = strip_note(entry)
if entry:
files.append(entry)
return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
KYBERFORGE_CONTRIBUTING_FILES
KYBERFORGE_CONTRIBUTING_FILES_PY="${KYBERFORGE_CONTRIBUTING_FILES_PY%$'\n'}"

View File

@@ -1,7 +1,32 @@
#!/usr/bin/env bash
set -euo pipefail
# lib-provenance-agent.sh — SOURCED, never executed.
#
# agent-audit's provenance suite: its validate-provenance.sh, minus the shared
# Contributing-files parser (lib-contributing-files.sh holds the one copy) and
# minus the --help dispatch that validate-provenance.sh now owns. The bash
# argument handling, the preconditions and every exit code are lifted verbatim,
# except the extension check, which the dispatcher made unreachable (see
# kyberforge_prov_agent_run).
#
# The two provenance modes have DIFFERENT exit contracts and they are NOT
# unified. Agent mode prints NOTHING on a clean run, and exits 0 silently when
# the scope walk-up finds no type:-bearing apm.yml above the agent file — that
# is a verdict about a real file ("this agent is user or project scope, so
# plugin-scope provenance does not apply"), not a rejected input, and
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it. Skill mode
# (lib-provenance-skill.sh) has no such verdict and instead treats exit 0 with
# output as INFO-only findings. Neither contract may be spelled with the
# other's codes.
#
# Agent mode also has no check 9, so it takes no --base-ref flag: a --base-ref
# passed alongside an agent target is an extra argument and is rejected with
# exit 2, exactly as before the merge.
#
# Consumed by: validate-provenance.sh, agent mode.
# shellcheck shell=bash
# shellcheck disable=SC2034
usage() {
kyberforge_prov_agent_usage() {
cat <<EOF
Usage: validate-provenance.sh <agent-file>
@@ -52,91 +77,98 @@ Checks performed:
4 Contributing files back-reference the parent slug in their source_keys
5 Research doc field present and not placeholder
This script has no counterpart to skill-audit's checks 6, 7 and 8 (Research
Agent mode has no counterpart to skill mode's checks 6, 7 and 8 (Research
doc field / upstream forward / upstream reverse are numbered 6, 7, 8 there and
5 here): an agent at plugin scope is a single file with a plugin-root
sources.md, so there is no references/ tree to walk and no upstream research
source index to cross-check. parse_status() and the sources.md-basename gate
that those checks need exist only in the skill-audit copy.
that those checks need exist only in lib-provenance-skill.sh.
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
# Usage and environment problems exit 2, findings exit 1. See the usage text
# above for why the two must not share a code, and for why "not plugin scope"
# is neither of them. This is a deliberate divergence from validate.sh, which
# has no 2 tier for content: validate.sh always prints PASS lines, so a usage
# error there is visibly not a findings report. This script prints NOTHING on a
# clean run, so exit 1 plus empty stdout was the only signal a caller got
# either way.
if [[ $# -lt 1 ]]; then
echo "Error: agent-file is required." >&2
echo "" >&2
usage >&2
exit 2
fi
# Extra positional arguments were silently dropped, so a typo'd flag or a second
# path looked like it had been honoured.
if [[ $# -gt 1 ]]; then
echo "Error: expected exactly one argument, got $#: $*" >&2
echo "" >&2
usage >&2
exit 2
fi
# python3 is a HARD dependency. Without this preflight a missing interpreter
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
# caller maps to anything, from a message that names this script's line number
# rather than the missing dependency.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
exit 2
fi
# A path that does not exist, or exists but is not a regular file, used to reach
# the Python body, get os.path.dirname()'d into some ancestor directory and then
# either report a silent exit 0 (no package above it) or — worse — audit a
# DIFFERENT agent's package while naming the typo'd path. A typo'd target was
# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a
# nonexistent path for exactly this reason.
#
# This is decided from the argument alone, before any walk-up runs, so it cannot
# collide with the not-plugin-scope exit 0: that verdict is only ever reached by
# a file that got past here.
if [[ ! -e "$1" ]]; then
echo "Error: no such file: $1" >&2
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
echo " Fix: pass the path of the agent file to validate." >&2
exit 2
fi
if [[ ! -f "$1" ]]; then
echo "Error: not a regular file: $1" >&2
echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2
echo " Fix: pass the agent file itself — .apm/agents/<name>.agent.md — not its parent directory." >&2
exit 2
fi
# The extension check used to live inside the Python body. It stays exit 2 and
# keeps its wording; it moves up here so that every "this argument is not
# auditable" verdict is reached in one place, before the interpreter starts and
# before the scope walk-up can turn a bad argument into a silent exit 0.
case "$1" in
*.agent.md | *.md) ;;
*)
echo "Error: unrecognized extension '$(basename "$1")' — expected .md or .agent.md" >&2
kyberforge_prov_agent_run() {
# Usage and environment problems exit 2, findings exit 1. See the usage text
# above for why the two must not share a code, and for why "not plugin scope"
# is neither of them. This is a deliberate divergence from validate.sh, which
# has no 2 tier for content: validate.sh always prints PASS lines, so a usage
# error there is visibly not a findings report. This script prints NOTHING on a
# clean run, so exit 1 plus empty stdout was the only signal a caller got
# either way.
if [[ $# -lt 1 ]]; then
echo "Error: agent-file is required." >&2
echo "" >&2
kyberforge_prov_agent_usage >&2
exit 2
;;
esac
fi
python3 -u - "$1" <<'PYTHON'
# Extra positional arguments were silently dropped, so a typo'd flag or a second
# path looked like it had been honoured.
if [[ $# -gt 1 ]]; then
echo "Error: expected exactly one argument, got $#: $*" >&2
echo "" >&2
kyberforge_prov_agent_usage >&2
exit 2
fi
# python3 is a HARD dependency. Without this preflight a missing interpreter
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
# caller maps to anything, from a message that names this script's line number
# rather than the missing dependency.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
exit 2
fi
# A path that does not exist, or exists but is not a regular file, used to reach
# the Python body, get os.path.dirname()'d into some ancestor directory and then
# either report a silent exit 0 (no package above it) or — worse — audit a
# DIFFERENT agent's package while naming the typo'd path. A typo'd target was
# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a
# nonexistent path for exactly this reason.
#
# This is decided from the argument alone, before any walk-up runs, so it cannot
# collide with the not-plugin-scope exit 0: that verdict is only ever reached by
# a file that got past here.
if [[ ! -e "$1" ]]; then
echo "Error: no such file: $1" >&2
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
echo " Fix: pass the path of the agent file to validate." >&2
exit 2
fi
if [[ ! -f "$1" ]]; then
echo "Error: not a regular file: $1" >&2
echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2
echo " Fix: pass the agent file itself — .apm/agents/<name>.agent.md — not its parent directory." >&2
exit 2
fi
# No extension check here. The pre-merge script carried one ("unrecognized
# extension — expected .md or .agent.md"), but validate-provenance.sh only
# dispatches a *.agent.md, or a *.md directly under an agents/ directory, to
# this function, and the argument-count check above guarantees $1 IS that
# target — so the check could never fire. The "no such file" and "not a
# regular file" checks stay: a nonexistent x.agent.md and a FIFO named
# x.agent.md both pass the dispatcher and both still reach them.
# The Python program, reassembled in the order the parser block sat in before
# the merge: preamble, shared parser, body.
local prog="$KYBERFORGE_PROV_AGENT_PREAMBLE_PY
$KYBERFORGE_CONTRIBUTING_FILES_PY
$KYBERFORGE_PROV_AGENT_BODY_PY"
local rc=0
python3 -u - "$1" <<< "$prog" || rc=$?
# The findings code travels in KYBERFORGE_PROV_RC and this function returns 0,
# so the caller can invoke it UNTESTED. See lib-provenance-skill.sh for why:
# testing a function's status disables errexit for its whole body.
KYBERFORGE_PROV_RC="$rc"
return 0
}
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_PREAMBLE_PY <<'KYBERFORGE_PROV_AGENT_PREAMBLE' || true
import sys
import os
import re
@@ -291,99 +323,10 @@ def parse_source_keys(fm):
def parse_h2_slugs(content):
return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
KYBERFORGE_PROV_AGENT_PREAMBLE
KYBERFORGE_PROV_AGENT_PREAMBLE_PY="${KYBERFORGE_PROV_AGENT_PREAMBLE_PY%$'\n'}"
def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content.
Both authored forms are accepted, because both are in use across the
corpus and only recognising the first silently skipped the contributing-
file checks on every sources.md written the other way:
- **Contributing files:** SKILL.md, references/a.md
**Contributing files:**
- SKILL.md (what this source contributed)
- references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped.
Note the bullet form's notes may themselves contain commas, so the list
is built per bullet rather than by splitting the joined value.
The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
"""
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
def strip_note(entry):
# "references/a.md (why)" -> "references/a.md"
return re.sub(r'\s*\(.*$', '', entry).strip()
# Inline form: value on the same line, comma-separated, no notes.
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if cf_m:
value = cf_m.group(1).strip()
if value.startswith("(none"):
return []
return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m:
return None
files = []
for line in block[cf_m.end():].splitlines():
line = line.strip()
if not line:
if files:
break
continue
if not line.startswith("- "):
break
entry = line[2:].strip()
if entry.startswith("(none"):
return []
entry = strip_note(entry)
if entry:
files.append(entry)
return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_BODY_PY <<'KYBERFORGE_PROV_AGENT_BODY' || true
def parse_research_docs(content, slug):
"""Every Research doc value under a given slug H2, in document order.
@@ -629,4 +572,5 @@ for slug in unique_slugs:
print_findings()
sys.exit(1 if has_fail else 0)
PYTHON
KYBERFORGE_PROV_AGENT_BODY
KYBERFORGE_PROV_AGENT_BODY_PY="${KYBERFORGE_PROV_AGENT_BODY_PY%$'\n'}"

View File

@@ -1,7 +1,30 @@
#!/usr/bin/env bash
set -euo pipefail
# lib-provenance-skill.sh — SOURCED, never executed.
#
# skill-audit's provenance suite: its validate-provenance.sh, minus the shared
# Contributing-files parser (lib-contributing-files.sh holds the one copy) and
# minus the --help dispatch that validate-provenance.sh now owns. The bash
# argument handling, the preconditions and every exit code are lifted verbatim.
#
# The two provenance modes have DIFFERENT exit contracts and they are NOT
# unified. Skill mode exits 0 with output whenever the only findings are INFO
# — a check that could not run, announced rather than skipped silently — so a
# caller must read exit 0 plus output as INFO-only findings, not as noise.
# Agent mode (lib-provenance-agent.sh) prints nothing at all on a clean run and
# additionally exits 0 SILENTLY when the scope walk-up finds no plugin package.
# Skill mode has no such verdict: it has already hard-failed (exit 2) on a
# directory that is not a skill before the interpreter starts.
#
# Skill mode also owns the --base-ref=<ref> flag (check 9) and the
# VALIDATE_PROVENANCE_BASE_REF environment variable. Agent mode has no check 9
# and takes no flags, so a --base-ref passed to an agent target is still an
# extra argument and is still rejected, exactly as before the merge.
#
# Consumed by: validate-provenance.sh, skill mode.
# shellcheck shell=bash
# shellcheck disable=SC2034
usage() {
kyberforge_prov_skill_usage() {
cat <<EOF
Usage: validate-provenance.sh <skill-dir> [--base-ref=<ref>]
@@ -64,102 +87,121 @@ Checks performed:
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
kyberforge_prov_skill_run() {
local arg
# --base-ref=<ref> is the only recognised flag, for check 9's base-ref
# override. It is pulled out before the positional-count checks below so it
# never counts against them — a caller passing it alongside skill-dir sees
# the same argument-count behaviour as one who does not pass it at all, and a
# genuinely extra positional argument is still rejected.
#
# BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to
# the empty string, and deliberately NOT declared `local`: `local X` with no
# value still leaves X unset, but declaring it at all would scope it away from
# a future caller that wants to inspect it. `--base-ref=` (given, but empty)
# and "no flag at all" are different instructions — the first says "use the
# default resolution, ignoring the environment", the second says "fall back to
# the environment" — and an empty-string initialiser collapsed them:
# `${BASE_REF_OVERRIDE:-$ENV}` treats an empty flag value as absent, so the
# environment variable won and the usage text's "the flag wins if both are
# given" was false for exactly that spelling.
declare -a POSITIONAL_ARGS=()
for arg in "$@"; do
case "$arg" in
--base-ref=*)
BASE_REF_OVERRIDE="${arg#--base-ref=}"
;;
*)
POSITIONAL_ARGS+=("$arg")
;;
esac
done
# --base-ref=<ref> is the only recognised flag, for check 9's base-ref
# override. It is pulled out before the positional-count checks below so it
# never counts against them — a caller passing it alongside skill-dir sees
# the same argument-count behaviour as one who does not pass it at all, and a
# genuinely extra positional argument is still rejected.
#
# BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to
# the empty string. `--base-ref=` (given, but empty) and "no flag at all" are
# different instructions — the first says "use the default resolution, ignoring
# the environment", the second says "fall back to the environment" — and an
# empty-string initialiser collapsed them: `${BASE_REF_OVERRIDE:-$ENV}` treats
# an empty flag value as absent, so the environment variable won and the usage
# text's "the flag wins if both are given" was false for exactly that spelling.
declare -a POSITIONAL_ARGS=()
for arg in "$@"; do
case "$arg" in
--base-ref=*)
BASE_REF_OVERRIDE="${arg#--base-ref=}"
;;
*)
POSITIONAL_ARGS+=("$arg")
;;
esac
done
# Usage and environment problems exit 2, findings exit 1. See the usage text
# above for why the two must not share a code. This is a deliberate divergence
# from validate.sh, which has no 2 tier: validate.sh always prints PASS lines,
# so a usage error there is visibly not a findings report. This script prints
# NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a
# caller got either way.
if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then
echo "Error: skill-dir is required." >&2
echo "" >&2
kyberforge_prov_skill_usage >&2
exit 2
fi
# Usage and environment problems exit 2, findings exit 1. See the usage text
# above for why the two must not share a code. This is a deliberate divergence
# from validate.sh, which has no 2 tier: validate.sh always prints PASS lines,
# so a usage error there is visibly not a findings report. This script prints
# NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a
# caller got either way.
if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then
echo "Error: skill-dir is required." >&2
echo "" >&2
usage >&2
exit 2
fi
# Extra positional arguments were silently dropped, so a typo'd flag or a second
# path looked like it had been honoured.
if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then
echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2
echo "" >&2
kyberforge_prov_skill_usage >&2
exit 2
fi
# Extra positional arguments were silently dropped, so a typo'd flag or a second
# path looked like it had been honoured.
if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then
echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2
echo "" >&2
usage >&2
exit 2
fi
local SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}"
SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}"
# The flag wins over the environment variable whenever the flag was GIVEN —
# `+x` tests for presence, not for a non-empty value, which is the distinction
# `:-` could not make. An empty result either way tells the Python body to fall
# back to `git merge-base HEAD origin/main`.
local BASE_REF
if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then
BASE_REF="$BASE_REF_OVERRIDE"
else
BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}"
fi
# The flag wins over the environment variable whenever the flag was GIVEN —
# `+x` tests for presence, not for a non-empty value, which is the distinction
# `:-` could not make. An empty result either way tells the Python body to fall
# back to `git merge-base HEAD origin/main`.
if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then
BASE_REF="$BASE_REF_OVERRIDE"
else
BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}"
fi
# python3 is a HARD dependency. Without this preflight a missing interpreter
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
# caller maps to anything, from a message that names this script's line number
# rather than the missing dependency.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
exit 2
fi
# python3 is a HARD dependency. Without this preflight a missing interpreter
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
# caller maps to anything, from a message that names this script's line number
# rather than the missing dependency.
if ! command -v python3 > /dev/null 2>&1; then
echo "Error: python3 is required but was not found on PATH." >&2
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
exit 2
fi
# A path that is not a directory, or a directory that is not a skill, used to
# reach the Python body, find no sources.md and no source_keys, take the
# "nothing to validate" early exit and report exit 0 with no output — which
# references/skill-validation-scripts.md explicitly told the auditor to read as a
# pass. A typo'd target was therefore indistinguishable from a clean skill.
# vale-wrap.sh hard-errors on a nonexistent path for exactly this reason.
if [[ ! -d "$SKILL_DIR_ARG" ]]; then
echo "Error: not a directory: $SKILL_DIR_ARG" >&2
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
echo " Fix: pass the path of the skill directory to validate." >&2
exit 2
fi
# A path that is not a directory, or a directory that is not a skill, used to
# reach the Python body, find no sources.md and no source_keys, take the
# "nothing to validate" early exit and report exit 0 with no output — which
# references/validation-scripts.md explicitly told the auditor to read as a
# pass. A typo'd target was therefore indistinguishable from a clean skill.
# vale-wrap.sh hard-errors on a nonexistent path for exactly this reason.
if [[ ! -d "$SKILL_DIR_ARG" ]]; then
echo "Error: not a directory: $SKILL_DIR_ARG" >&2
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
echo " Fix: pass the path of the skill directory to validate." >&2
exit 2
fi
if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then
echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2
echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2
echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2
exit 2
fi
if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then
echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2
echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2
echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2
exit 2
fi
# The Python program, reassembled in the order the parser block sat in before
# the merge: preamble, shared parser, body.
local prog="$KYBERFORGE_PROV_SKILL_PREAMBLE_PY
$KYBERFORGE_CONTRIBUTING_FILES_PY
$KYBERFORGE_PROV_SKILL_BODY_PY"
python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<'PYTHON'
local rc=0
python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<< "$prog" || rc=$?
# The findings code travels in KYBERFORGE_PROV_RC and this function returns 0,
# so the caller can invoke it UNTESTED. Testing a function's status (`f || RC=$?`)
# disables errexit for its entire body, which would leave every command above
# unguarded -- and no subshell or `set -e` inside can re-arm it once the call
# sits in a condition context. Error paths above use `exit`, which is unaffected
# either way; this keeps errexit armed for anything added later.
KYBERFORGE_PROV_RC="$rc"
return 0
}
IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_PREAMBLE_PY <<'KYBERFORGE_PROV_SKILL_PREAMBLE' || true
import sys
import os
import re
@@ -297,99 +339,10 @@ def parse_h2_slugs(content):
"""Return list of H2 heading values from a markdown file."""
return re.findall(r'^## (.+)$', content, re.MULTILINE)
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
# ONE parser, embedded VERBATIM in two scripts:
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
# The block between these markers must stay byte-identical in both. It is
# copied rather than imported because a cache-installed plugin's scripts cannot
# read files outside their own plugin directory, so there is no single file both
# can share — the same constraint that forces the ADR-0020 boundary resolver to
# be duplicated across three scripts. Edit one copy, then paste it over the
# other.
#
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
# it did, the agent-audit copy's docstring merely ASSERTED the two were
# "behaviourally identical" and nothing checked it — which is how the two
# already-diverged spellings of the bullet loop went unnoticed.
#
# Requires: re (imported by the host script).
KYBERFORGE_PROV_SKILL_PREAMBLE
KYBERFORGE_PROV_SKILL_PREAMBLE_PY="${KYBERFORGE_PROV_SKILL_PREAMBLE_PY%$'\n'}"
def parse_contributing_files(content, slug):
"""Find the Contributing files for a given slug H2 in content.
Both authored forms are accepted, because both are in use across the
corpus and only recognising the first silently skipped the contributing-
file checks on every sources.md written the other way:
- **Contributing files:** SKILL.md, references/a.md
**Contributing files:**
- SKILL.md (what this source contributed)
- references/a.md (what this source contributed)
Returns a list of paths with any trailing parenthetical note stripped.
Note the bullet form's notes may themselves contain commas, so the list
is built per bullet rather than by splitting the joined value.
The three return values are NOT interchangeable, and callers depend on
the distinction:
[path, ...] the entry names contributing files
[] the entry EXPLICITLY records "(none)"
None the entry says nothing this parser can read
Only an explicit "(none)" yields []. A "Contributing files:" heading
followed by a numbered list, by `*` bullets, or by prose parses nothing
and returns None, never [] — a caller reads [] as a deliberate "no
contributing files" record and SKIPS its check on that basis, so a parse
failure returning [] would silently disable the check instead of leaving
the unreadable entry exposed to it.
"""
pattern = re.compile(
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
re.MULTILINE | re.DOTALL
)
m = pattern.search(content)
if not m:
return None
block = m.group(1)
def strip_note(entry):
# "references/a.md (why)" -> "references/a.md"
return re.sub(r'\s*\(.*$', '', entry).strip()
# Inline form: value on the same line, comma-separated, no notes.
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
if cf_m:
value = cf_m.group(1).strip()
if value.startswith("(none"):
return []
return [p for p in (strip_note(x) for x in value.split(","))
if p] or None
# Bullet form: heading on its own line, one file per following bullet.
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
if not cf_m:
return None
files = []
for line in block[cf_m.end():].splitlines():
line = line.strip()
if not line:
if files:
break
continue
if not line.startswith("- "):
break
entry = line[2:].strip()
if entry.startswith("(none"):
return []
entry = strip_note(entry)
if entry:
files.append(entry)
return files or None
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_BODY_PY <<'KYBERFORGE_PROV_SKILL_BODY' || true
def parse_research_docs(content, slug):
"""Every Research doc value under a given slug H2, in document order.
@@ -1195,4 +1148,5 @@ else:
print_findings()
sys.exit(1 if has_fail else 0)
PYTHON
KYBERFORGE_PROV_SKILL_BODY
KYBERFORGE_PROV_SKILL_BODY_PY="${KYBERFORGE_PROV_SKILL_BODY_PY%$'\n'}"

View File

@@ -189,7 +189,16 @@ for arg in "$@"; do
done
if [[ "$config_given" == false ]]; then
vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini")
# `scripts/../assets/vale` begins with neither `/` nor `.`, so `cd` consults
# CDPATH for it — and when a CDPATH entry supplies the directory, `cd` PRINTS
# the directory it chose. A bare `$(cd ... && pwd)` therefore captured TWO
# lines, and the chosen directory could be an unrelated tree entirely: with
# CDPATH=/tmp/decoy and /tmp/decoy/scripts present, this resolved to
# /tmp/decoy/assets/vale and vale died on a two-line --config path. CDPATH is
# cleared for the one command, `--` ends option parsing for a directory named
# like a flag, and stdout is discarded so only `pwd` is captured. Same fix as
# validate.sh and validate-provenance.sh apply to their SCRIPT_DIR.
vale_args+=(--config "$(CDPATH='' cd -- "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" > /dev/null && pwd)/.vale.ini")
fi
if [[ ${#path_args[@]} -eq 0 ]]; then
@@ -506,7 +515,7 @@ for arg in ${path_args[@]+"${path_args[@]}"}; do
while IFS= read -r -d '' rel; do
mkdir -p "$dest/$(dirname "$rel")"
cp "$arg/$rel" "$dest/$rel"
done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0)
done < <(CDPATH='' cd -- "$arg" && find -L . -name .git -prune -o -type f -print0)
while IFS= read -r -d '' md; do
flatten "$md" "$md"
done < <(find "$dest" -type f -name '*.md' -print0)

View File

@@ -0,0 +1,324 @@
#!/usr/bin/env bash
set -euo pipefail
# The ONE entry point for provenance validation. It auto-detects whether the
# target is a skill directory or an agent definition file — the same rule
# validate.sh uses — and runs the matching suite from lib-provenance-skill.sh or
# lib-provenance-agent.sh. The Contributing-files parser both suites need is
# sourced once, from lib-contributing-files.sh, instead of being embedded twice.
#
# The two suites have DIFFERENT exit contracts, and merging the entry point does
# not merge those:
#
# skill mode exits 0 with output when the only findings are INFO — a check
# that could not run, announced rather than skipped silently. A
# caller must read exit 0 plus output as INFO-only findings.
# agent mode prints nothing at all on a clean run, and exits 0 SILENTLY when
# the scope walk-up finds no plugin package above the agent file.
# That is a verdict about a real file, not a rejected input;
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it.
#
# Exit 2 means the argument is not auditable at all — missing, doubled, the
# wrong shape, or an environment problem. It is never a finding.
# --- Path splitting, with bash builtins only -------------------------------
# dirname and basename are EXTERNAL commands, and every call below happens
# before the mode's python3 preflight. Using them put coreutils ahead of python3
# in the dependency order: on a PATH carrying neither, this script died at exit
# 127 naming `dirname` (and, through the sourced libraries, `cat`) instead of
# reaching the preflight that names python3 — the exact failure
# tests/test-adr0020-contract.sh assertion 2 exists to prevent. The pre-merge
# validate-provenance.sh was one self-contained file that reached its preflight
# on builtins alone; these two functions, plus the `read`-based loaders in the
# sourced libraries, restore that property. `cd` and `pwd` are builtins and may
# stay.
#
# They reproduce dirname/basename semantics for the shapes this script sees:
# trailing slashes are stripped, a path with no slash yields "." / itself, and
# "/" yields "/".
_kf_dirname() {
local _p="$1"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ "$_p" == "/" ]]; then
printf '%s' "/"
return 0
fi
if [[ "$_p" != */* ]]; then
printf '%s' "."
return 0
fi
_p="${_p%/*}"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ -z "$_p" ]]; then
_p="/"
fi
printf '%s' "$_p"
}
_kf_basename() {
local _p="$1"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ "$_p" == "/" ]]; then
printf '%s' "/"
return 0
fi
printf '%s' "${_p##*/}"
}
# --- The target's parent directory NAME, resolved -------------------------
# The agent rule tests the NAME of the target's parent directory. Reading that
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
# directory was refused as matching neither shape, by an error message naming
# that exact shape as valid. The pre-merge agent validator had no path-shape
# gate and worked from any working directory.
#
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
# still addressed as agents/, which is what the literal test always honoured.
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
# that cannot be entered — a typo'd path — falls back to the literal name, so
# the neither-shape error still fires for it.
_kf_parent_name() {
local _dir _resolved
_dir="$(_kf_dirname "$1")"
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
_kf_basename "$_resolved"
else
_kf_basename "$_dir"
fi
}
# --- This script's own directory, and the libraries beside it -------------
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
# CDPATH exported and the relative invocation the flow references prescribe
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
# and could resolve through CDPATH to an unrelated directory and source a
# same-named file from there. CDPATH is cleared for the one command, `--` ends
# option parsing for a directory named like a flag, and stdout is discarded so
# only `pwd` is captured.
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
exit 2
fi
# A sourced library that is missing or unreadable used to kill the script under
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
# partial install, or a copy or symlink of this one file taken out of scripts/,
# was therefore reported as a failing audit. Checked explicitly instead, and
# exit 2, which the same references read as "it never ran".
_kf_require_lib() {
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
exit 2
fi
}
# Each mode's own usage text lives in that mode's library, verbatim, so usage()
# needs the libraries — but `--help` must not. Sourcing them unconditionally at
# the top made a missing lib-*.sh turn `--help` into exit 2, so the one command
# that explains how to use the script was the one command a partial install
# could not answer. validate.sh's usage() is self-contained and always works;
# this restores the same property without copying the per-mode text down here
# and letting it drift from the libraries that own it. When a library is gone,
# the shared half of the usage still prints and the mode's half says why it
# cannot.
#
# The two call sites below are spelled out rather than folded into one helper
# taking the library as a parameter: a parameterized `.` is a non-constant
# source, which is SC1090 at warning severity — the level .pre-commit-config.yaml
# runs shellcheck at — and the only way to silence it, a `source=/dev/null`
# directive, is a directive that resolves to nothing, which
# tests/test-vale-wrap.sh part C rejects outright because a non-resolving
# directive silently disarms that file's array-seeding exemption. Two literal
# sources with two real directives cost a few lines and keep both gates honest.
_kf_lib_readable() {
[[ -f "$SCRIPT_DIR/$1" && -r "$SCRIPT_DIR/$1" ]]
}
_kf_usage_lib_missing() {
echo "(This mode's usage lives in $1, which is missing or unreadable in"
echo "$SCRIPT_DIR. Reinstall the factory-audit skill to restore it. Note that"
echo "an audit cannot run in this state either — it would exit 2.)"
}
usage() {
cat <<EOF
Usage: validate-provenance.sh <skill-dir> [--base-ref=<ref>]
validate-provenance.sh <agent-file>
Validate that a skill's or an agent's sources provenance chain is complete and
internally consistent. The mode is detected from the target:
skill mode the target is a directory (a skill directory contains SKILL.md),
or the target IS a SKILL.md file.
agent mode the target is a *.agent.md file, or a *.md file whose parent
directory is named 'agents' (.apm/agents, .claude/agents,
.github/agents, .copilot/agents).
The two modes have different checks, different exit contracts and different
flags — --base-ref belongs to skill mode's check 9 and agent mode has no
check 9 — so each mode's own usage follows below, verbatim.
Exit codes:
0 All checks passed (or nothing to validate; in agent mode, also "not plugin
scope")
1 One or more checks failed
2 Usage error, the target matches neither a skill directory nor an agent
file this script can read, or a lib-*.sh beside this script is missing or
unreadable
An exit code of 2 is NOT a finding. SKILL.md tells the auditor to surface a
non-zero exit as findings, so a usage error leaving exit 1 with nothing on
stdout was indistinguishable from a clean-but-failing run. Environment and
argument problems exit 2; only real findings exit 1.
=== skill mode ===
EOF
if _kf_lib_readable lib-provenance-skill.sh; then
# shellcheck source=lib-provenance-skill.sh
. "$SCRIPT_DIR/lib-provenance-skill.sh"
kyberforge_prov_skill_usage
else
_kf_usage_lib_missing lib-provenance-skill.sh
fi
cat <<EOF
=== agent mode ===
EOF
if _kf_lib_readable lib-provenance-agent.sh; then
# shellcheck source=lib-provenance-agent.sh
. "$SCRIPT_DIR/lib-provenance-agent.sh"
kyberforge_prov_agent_usage
else
_kf_usage_lib_missing lib-provenance-agent.sh
fi
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: a skill directory or an agent file is required." >&2
echo "" >&2
usage >&2
exit 2
fi
# Sourced only once an audit is actually going to be attempted. It sat at the
# top of the file until `--help` on a partial install exited 2 instead of
# printing usage; see the comment above _kf_lib_readable for the whole story.
# usage() loads the two provenance libraries on its own when it needs them, so
# nothing here is reached by the --help path.
_kf_require_lib lib-contributing-files.sh
# shellcheck source=lib-contributing-files.sh
. "$SCRIPT_DIR/lib-contributing-files.sh"
_kf_require_lib lib-provenance-skill.sh
# shellcheck source=lib-provenance-skill.sh
. "$SCRIPT_DIR/lib-provenance-skill.sh"
_kf_require_lib lib-provenance-agent.sh
# shellcheck source=lib-provenance-agent.sh
. "$SCRIPT_DIR/lib-provenance-agent.sh"
# --- Detect the mode -------------------------------------------------------
# The first non-flag argument decides the mode. Only the mode is decided here:
# the argument COUNT, the flag rules and every precondition belong to the mode's
# own suite and are applied there, unchanged, over the original "$@". So a
# --base-ref handed to an agent target is still an extra argument and is still
# rejected, and a second positional is still rejected by whichever mode it
# reaches.
# _saw_positional is tracked separately because an EMPTY positional and NO
# positional are different mistakes with different fixes, and `-z "$TARGET"`
# alone cannot tell them apart: `validate-provenance.sh ""` — an unquoted shell
# variable that expanded to nothing, the usual way this happens — was reported
# as "only flags were given", which is false and sends the reader looking for a
# flag they did not type instead of at the variable that came up empty.
TARGET=""
_saw_positional=false
for _arg in "$@"; do
case "$_arg" in
--base-ref=*) ;;
*) TARGET="$_arg"; _saw_positional=true; break ;;
esac
done
if [[ "$_saw_positional" == false ]]; then
echo "Error: a skill directory or an agent file is required." >&2
echo " Why: only flags were given, so there is no target to detect a mode from." >&2
echo " Fix: pass the skill directory, or the agent file, as a positional argument." >&2
exit 2
fi
if [[ -z "$TARGET" ]]; then
echo "Error: the target argument is an empty string." >&2
echo " Why: a positional argument was passed, but it is empty, so there is no path to detect a mode from — usually an unquoted or unset shell variable expanding to nothing at the call site, not a missing argument." >&2
echo " Fix: check the variable that supplies the target, and pass the skill directory, or the agent file, as a non-empty positional argument." >&2
exit 2
fi
TARGET_BASE="$(_kf_basename "$TARGET")"
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
if [[ -d "$TARGET" ]]; then
if [[ -f "$TARGET/SKILL.md" ]]; then
MODE=skill
else
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would run the wrong provenance checks." >&2
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
exit 2
fi
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
MODE=skill
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
MODE=agent
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
MODE=agent
else
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would report a silent pass on a typo'd target, which is the failure both suites' preconditions exist to prevent." >&2
echo " Fix: pass one of those two shapes." >&2
exit 2
fi
# In skill mode a SKILL.md target names its directory. The token is replaced in
# place rather than assumed to be $1, because --base-ref may precede it; the
# suite's own preconditions then apply to that directory, exactly as before the
# merge.
if [[ "$MODE" == skill && "$TARGET_BASE" == "SKILL.md" && ! -d "$TARGET" ]]; then
declare -a _rewritten=()
_replaced=false
for _arg in "$@"; do
if [[ "$_replaced" == false && "$_arg" == "$TARGET" ]]; then
_rewritten+=("$(_kf_dirname "$TARGET")")
_replaced=true
else
_rewritten+=("$_arg")
fi
done
# Guarded expansion: bash 3.2 under `set -u` aborts on "${arr[@]}" when the
# array is empty, and the loop above cannot prove non-emptiness to a static
# scan. tests/test-vale-wrap.sh enforces bash-3.2 portability across this tree.
set -- ${_rewritten[@]+"${_rewritten[@]}"}
fi
# Called UNTESTED, on purpose: `f || RC=$?` would disable errexit for the whole
# function body. Each run function stashes its findings code in
# KYBERFORGE_PROV_RC and returns 0; its error paths exit directly.
KYBERFORGE_PROV_RC=0
case "$MODE" in
skill) kyberforge_prov_skill_run "$@" ;;
agent) kyberforge_prov_agent_run "$@" ;;
esac
RC="$KYBERFORGE_PROV_RC"
exit "$RC"

View File

@@ -0,0 +1,255 @@
#!/usr/bin/env bash
set -euo pipefail
# The ONE entry point for structural validation. It auto-detects whether the
# target is a skill directory or an agent definition file and runs the matching
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh
# and are unchanged from the skill-audit / agent-audit scripts they came from.
# The ADR-0020 boundary resolver both of them need is sourced once, from
# lib-boundary-resolver.sh, instead of being embedded twice.
#
# Detection never guesses. A target that matches neither shape is a hard exit 2
# naming the mismatch, because the alternative — picking a mode and letting the
# suite fail on its own terms — reports a skill-shaped finding about an agent
# file, or the reverse, and sends the reader after the wrong problem.
# --- Path splitting, with bash builtins only -------------------------------
# dirname and basename are EXTERNAL commands, and every call below happens
# before the mode-specific python3/PyYAML preflight. Using them put coreutils
# ahead of python3 in the dependency order: on a PATH carrying neither, this
# script died at exit 127 naming `dirname` instead of reaching the preflight
# that names python3 — the exact failure tests/test-adr0020-contract.sh
# assertion 2 exists to prevent ("the two are checked separately so the message
# names the thing to install rather than the wrong one"). The pre-merge
# validate.sh was one self-contained file that reached its preflight on builtins
# alone; these two functions restore that property. `cd` and `pwd` are builtins
# and may stay.
#
# They reproduce dirname/basename semantics for the shapes this script sees:
# trailing slashes are stripped, a path with no slash yields "." / itself, and
# "/" yields "/".
_kf_dirname() {
local _p="$1"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ "$_p" == "/" ]]; then
printf '%s' "/"
return 0
fi
if [[ "$_p" != */* ]]; then
printf '%s' "."
return 0
fi
_p="${_p%/*}"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ -z "$_p" ]]; then
_p="/"
fi
printf '%s' "$_p"
}
_kf_basename() {
local _p="$1"
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
if [[ "$_p" == "/" ]]; then
printf '%s' "/"
return 0
fi
printf '%s' "${_p##*/}"
}
# --- The target's parent directory NAME, resolved -------------------------
# The agent rule tests the NAME of the target's parent directory. Reading that
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
# directory was refused as matching neither shape, by an error message naming
# that exact shape as valid. The pre-merge agent validator had no path-shape
# gate and worked from any working directory.
#
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
# still addressed as agents/, which is what the literal test always honoured.
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
# that cannot be entered — a typo'd path — falls back to the literal name, so
# the neither-shape error still fires for it.
_kf_parent_name() {
local _dir _resolved
_dir="$(_kf_dirname "$1")"
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
_kf_basename "$_resolved"
else
_kf_basename "$_dir"
fi
}
# --- This script's own directory, and the libraries beside it -------------
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
# CDPATH exported and the relative invocation the flow references prescribe
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
# and could resolve through CDPATH to an unrelated directory and source a
# same-named file from there. CDPATH is cleared for the one command, `--` ends
# option parsing for a directory named like a flag, and stdout is discarded so
# only `pwd` is captured.
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
exit 2
fi
# A sourced library that is missing or unreadable used to kill the script under
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
# partial install, or a copy or symlink of this one file taken out of scripts/,
# was therefore reported as a failing audit. Checked explicitly instead, and
# exit 2, which the same references read as "it never ran".
_kf_require_lib() {
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
exit 2
fi
}
usage() {
cat <<EOF
Usage: validate.sh <skill-dir | agent-file>
Validate a skill directory against the agentskills.io specification, or an agent
definition file against the agent definition spec. The mode is detected from the
target:
skill mode the target is a directory (a skill directory contains SKILL.md),
or the target IS a SKILL.md file.
agent mode the target is a *.agent.md file, or a *.md file whose parent
directory is named 'agents' (.apm/agents, .claude/agents,
.github/agents, .copilot/agents).
Skill mode audits the directory named by <skill-dir>.
Agent mode: at plugin/APM scope, <agent-file> is a single vendor-neutral
.apm/agents/<name>.agent.md file with no counterpart. Its frontmatter allowlist
is not restated here: it is read at load time from the apm-agent-allowlist
section of references/agent-field-inventory.md, which is the authoritative list.
At project or user scope, <agent-file> is either half of a Claude Code .md /
Copilot .agent.md pair.
Arguments:
skill-dir Path to the skill directory containing SKILL.md.
agent-file Path to the agent file (or either half of a project/user-scope pair).
Exit codes:
0 All checks passed (may include SUGGESTIONs)
1 One or more checks failed
2 Nothing was audited (no argument, the target matches neither shape, the
target does not exist, an unrecognized file extension, a missing
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
beside this script)
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: a skill directory or an agent file is required." >&2
echo "" >&2
usage >&2
exit 2
fi
TARGET="$1"
# --- The target has to be there --------------------------------------------
# Only the directory branch below stats the target; the *.agent.md, the
# agents/-parent and the SKILL.md branches classify on NAME alone, so a typo'd
# path matching one of those shapes was handed to python3 and came back as a
# FAIL at exit 1 — the findings tier, for a target that was never there to have
# findings about. The tiers are: 2 nothing is at this path so no check ran, 1
# something is there and it is broken.
#
# This runs on the TYPED path, before the SKILL.md -> parent-directory rewrite
# further down: rewritten first, a missing `docs/SKILL.md` would be tested as
# `docs`, which exists, and the guard would miss it.
#
# `-L` deliberately rescues what `-e` rejects. A dangling symlink and a symlink
# loop are both FALSE to -e but TRUE to -L, and neither belongs here: something
# IS at that path, it just cannot be opened, and "exists but unreadable" is a
# real finding the agent suite's check_file reports as a FAIL naming the file.
# Catching them here would replace that FAIL with a false "does not exist".
if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then
echo "Error: '$TARGET' does not exist." >&2
echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2
exit 2
fi
# --- Detect the mode -------------------------------------------------------
# Pure path and stat inspection, no interpreter and no external command needed,
# so it runs before the python3/PyYAML preflight — which is mode-specific,
# because each suite names the gates it would otherwise skip.
TARGET_BASE="$(_kf_basename "$TARGET")"
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
if [[ -d "$TARGET" ]]; then
if [[ -f "$TARGET/SKILL.md" ]]; then
MODE=skill
else
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
exit 2
fi
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
MODE=skill
TARGET="$(_kf_dirname "$TARGET")"
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
MODE=agent
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
MODE=agent
else
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2
echo " Fix: pass one of those two shapes." >&2
exit 2
fi
# --- Run the matching suite ------------------------------------------------
# Each suite is reassembled in the order the resolver block sat in before the
# merge — preamble, resolver, body — so every check runs against exactly the
# names and the order it always did.
RC=0
case "$MODE" in
skill)
_kf_require_lib lib-boundary-resolver.sh
# shellcheck source=lib-boundary-resolver.sh
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
_kf_require_lib lib-checks-skill.sh
# shellcheck source=lib-checks-skill.sh
. "$SCRIPT_DIR/lib-checks-skill.sh"
kyberforge_skill_preflight
PROG="$KYBERFORGE_SKILL_PREAMBLE_PY
$KYBERFORGE_RESOLVER_PY
$KYBERFORGE_SKILL_BODY_PY"
python3 -u - "$TARGET" <<< "$PROG" || RC=$?
;;
agent)
_kf_require_lib lib-boundary-resolver.sh
# shellcheck source=lib-boundary-resolver.sh
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
_kf_require_lib lib-checks-agent.sh
# shellcheck source=lib-checks-agent.sh
. "$SCRIPT_DIR/lib-checks-agent.sh"
kyberforge_agent_preflight
PROG="$KYBERFORGE_AGENT_PREAMBLE_PY
$KYBERFORGE_RESOLVER_PY
$KYBERFORGE_AGENT_BODY_PY"
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
;;
esac
exit "$RC"

View File

@@ -0,0 +1,94 @@
# tests/
Test files for scripts bundled with this skill.
## When to add tests
Add tests here when the skill has scripts in `scripts/` that are complex enough
to break silently — validators, parsers, generators, anything with branching
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
belongs here, not in `scripts/`.
## Dependencies
Tests require [bats-support](https://github.com/bats-core/bats-support) and
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
helpers from the repo root's `tests/test_helper/`.
From the repo root:
```bash
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
```
Run all tests for this skill (from the repo root):
```bash
bats plugins/kyberforge/.apm/skills/factory-audit/tests/
```
## Files
| File | Purpose |
|------|---------|
| `validate-skill.bats` | `scripts/validate.sh` against skill directories |
| `validate-agent.bats` | `scripts/validate.sh` against agent files |
| `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories |
| `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files |
## Two scripts, four suites
`factory-audit` merges what were two skills — `skill-audit` and `agent-audit` —
each of which shipped its own `validate.sh` and `validate-provenance.sh`. The
merged skill has **one** of each. Every suite here invokes one of those two
scripts; the four files are two scripts × two artifact types, not four scripts.
`validate-skill.bats` and `validate-agent.bats` run the same
`scripts/validate.sh` and differ only in the fixtures they point it at. The two
provenance suites stand in the same relation to `scripts/validate-provenance.sh`.
Do not add a third script path here on the assumption that a differently named
suite must mean a differently named script.
### Auto-detection is pinned across the pair
Each entry point decides for itself what it was handed. ADR-0025 states the
rule: a directory containing `SKILL.md` takes the skill flow; an `.agent.md`
file, or a file under a directory named `agents/`, takes the agent flow.
Anything else is rejected rather than guessed at. That behaviour is new with the
merge — before it, each script was hard-wired to one artifact type and nothing
about classification could be wrong — so it is asserted from both sides rather
than in one place:
- the skill-side suites pin the skill-directory classification and the
neither-shape rejection,
- the agent-side suites pin the two agent rules *separately* — `.agent.md` in a
directory that is not `agents/`, and a plain `.md` under `.apm/agents/` — so
that a detector implementing only one of them cannot pass both. Plus a control
asserting an agent file never picks up a skill-only gate.
Both skill-side suites additionally pin the `SKILL.md` **file** path, not just
the directory: a pre-commit `files:` hook matches files, so every hook-driven
invocation hands over a `SKILL.md` path. Each entry point resolves it to the
directory, and the suites assert the two spellings produce the identical
verdict rather than merely that the file spelling survives.
A misclassification is silent and total — the wrong rubric runs end to end and
reports the artifact clean against gates that never applied to it — and no other
fixture in these suites would notice, because every other fixture is already the
shape its own suite expects.
### The two provenance exit contracts are different on purpose
`scripts/validate-provenance.sh` does **not** behave identically in its two
modes, and the difference is deliberate:
| Mode | Successful run |
|------|----------------|
| skill | may exit 0 **with** output — INFO findings print, status stays 0 |
| agent | exits 0 and prints **nothing** |
Each half is asserted from its own side, by the `exit contract:` test in each
provenance suite. Both are asserted on purpose: a merge that collapsed one
contract into the other would still satisfy whichever side was left unasserted,
so a single-sided pin would go green on exactly the defect it exists to catch.

View File

@@ -131,7 +131,22 @@ EOF
}
@test "user scope: agent file directly in \$HOME (start dir IS exactly \$HOME, no walk-up) resolves to user scope" {
local fake_home="$TMPDIR/fakehome-direct"
# The fake $HOME is named `agents` on purpose, and it is a fixture detail
# forced by the merged entry point rather than anything the scope resolver
# cares about. The case under test needs the agent file's own directory to BE
# $HOME (detect_scope's `original_start == home` branch), AND the input has
# to be the Claude Code half — a plain .md — because that is the only half
# whose counterpart path differs between user scope ($HOME/.copilot/agents)
# and project scope ($HOME/.github/agents). With a *.agent.md input the two
# scopes derive the identical counterpart and the test would pass without
# discriminating. Post-merge, a plain .md only classifies as an agent when
# its parent directory is named `agents`, so the two requirements can only be
# satisfied together by a $HOME whose basename is `agents`.
#
# detect_scope's conventional_shape stays FALSE here (the grandparent is not
# one of .claude/.github/.copilot/.apm), so this still exercises the
# no-walk-up branch and not the two-segments-up arithmetic.
local fake_home="$TMPDIR/fakehome-direct/agents"
mkdir -p "$fake_home" "$fake_home/.copilot/agents"
cat > "$fake_home/my-agent.md" <<EOF
---
@@ -267,13 +282,22 @@ EOF
@test "project scope: a non-conventional path (agent file not directly under a literal 'agents' dir) falls back to the nearest .git boundary instead of two-segments-up arithmetic" {
local outer="$TMPDIR/outer-repo"
local pkg="$outer/pkgA"
mkdir -p "$pkg/.git" "$pkg/.github/agents" "$pkg/extra"
mkdir -p "$pkg/.git" "$pkg/.claude/agents" "$pkg/extra"
# Misplaced file: sits two path segments below $outer (pkgA/extra), which
# matches the conventional_root arithmetic by coincidence, but its
# immediate parent dir is "extra", not "agents" — conventional_shape is
# false, so the fix must fall back to the nearest .git boundary (pkgA),
# not trust $outer.
cat > "$pkg/extra/my-agent.md" <<EOF
#
# The *.agent.md suffix is a post-merge fixture requirement, not part of what
# is being measured: the merged entry point classifies a plain .md as an
# agent only when its parent directory is named `agents`, and the whole point
# of this fixture is that the parent is `extra`. The suffix is what keeps the
# file classifiable while leaving conventional_shape false. It flips which
# half of the pair is the input, so the counterpart below is the Claude Code
# half rather than the Copilot one; the scope arithmetic under test is
# untouched by that.
cat > "$pkg/extra/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -283,9 +307,9 @@ You are a test agent. When invoked, do the thing.
EOF
# Counterpart at the nearest-.git root (pkgA), not at $outer — if the
# arithmetic were trusted here, validate.sh would look for a counterpart
# at $outer/.github/agents/my-agent.agent.md, which doesn't exist, and
# at $outer/.claude/agents/my-agent.md, which doesn't exist, and
# false-FAIL.
cat > "$pkg/.github/agents/my-agent.agent.md" <<EOF
cat > "$pkg/.claude/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
@@ -293,7 +317,7 @@ description: A valid agent description.
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$pkg/extra/my-agent.md"
run bash "$SCRIPT" "$pkg/extra/my-agent.agent.md"
assert_success
refute_output --partial "counterpart file not found"
}
@@ -873,21 +897,42 @@ make_tools_pair() {
}
# ---------------------------------------------------------------------------
# A file that cannot be read
# A file that is not there, and a file that cannot be read
# ---------------------------------------------------------------------------
# scripts/check-apm-agents-valid.sh derives its expected agent-file set from
# `git ls-files`, so it hands this script paths that are tracked but absent from
# the worktree — a real and expected state, not a corner case. That used to exit
# 1 with a bare FileNotFoundError traceback and no FAIL line at all: non-zero, so
# the gate blocked, but with an interpreter stack instead of a diagnostic naming
# the file. Both scope paths are covered because they are separate call sites
# (check_apm_agent_file and check_file) and each needed its own handler.
# These are two different tiers and the split is the point.
#
# `is-a-dir.agent.md` is a DIRECTORY rather than a chmod 000 file on purpose:
# these tests run as root in CI, where mode bits do not deny anything and a
# permissions fixture would be silently readable and prove nothing.
# NOT THERE is exit 2: nothing is at the path, so no check ran. The entry
# point's existence guard catches it before python3 is invoked, because the
# *.agent.md and agents/-parent branches classify on NAME alone and a typo'd
# path used to sail through them into python and come back as a FAIL at exit 1
# — the FINDINGS tier, for a target that was never there to have findings
# about. (Before that it was worse still: a bare FileNotFoundError traceback
# and no FAIL line at all.) Both scope paths are covered because they were
# separate call sites, check_apm_agent_file and check_file, and each needed its
# own handler; the guard now short-circuits both, and the traceback assertions
# below are kept to pin that the diagnostic is still a sentence and not a stack.
#
# CANNOT BE READ is exit 1: something IS at the path and opening it fails, which
# is a real finding, and it must still reach check_file's `except OSError` arm.
# That is why the guard tests `! -e && ! -L` rather than `! -e` alone — a
# dangling symlink and a symlink loop are both false to -e and true to -L. The
# unreadable/nonexistent pairs below sit together so that distinction cannot be
# collapsed by accident.
#
# The unreadable fixture is a SYMLINK LOOP rather than a chmod 000 file: these
# tests run as root in CI, where mode bits do not deny anything and a permissions
# fixture would be silently readable and prove nothing.
#
# It used to be a DIRECTORY named `is-a-dir.agent.md`, which served the same
# purpose pre-merge. That no longer reaches the handler: the merged entry point
# classifies on shape first, and a directory is a skill-shaped target whatever it
# is named, so it is rejected at exit 2 before python3 is invoked. A symlink loop
# is a FILE by the detector's reckoning, classifies as an agent on its name, and
# still raises OSError on open — so it exercises the same `except OSError` arm.
# The directory-named-like-an-agent case did not go away with the fixture; it is
# pinned in its own test immediately below.
@test "a nonexistent plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
@test "a nonexistent plugin/APM-scope agent file exits 2 naming the path, never the findings tier" {
local root="$TMPDIR/pkg"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
@@ -896,15 +941,45 @@ version: 0.1.0
type: skill
EOF
run bash "$SCRIPT" "$root/.apm/agents/absent.agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "absent.agent.md"
# The literal 2, not assert_failure: 1 would pass a bare assert_failure, and
# 1 is exactly the wrong answer here — it is the tier that says the file was
# audited and has findings.
[ "$status" -eq 2 ]
assert_output --partial "Error: '$root/.apm/agents/absent.agent.md' does not exist."
# Kept from when this path reached python3 and returned a stack. Trivially
# true now that the guard short-circuits before the interpreter, and kept
# anyway: the assertion is "a reader gets a sentence, not a traceback", and
# that intent outlives the mechanism that currently satisfies it.
refute_output --partial "Traceback"
refute_output --partial "FileNotFoundError"
}
@test "an unreadable plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
local root="$TMPDIR/pkg-loop"
mkdir -p "$root/.apm/agents"
ln -s "unreadable.agent.md" "$root/.apm/agents/unreadable.agent.md"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
run bash "$SCRIPT" "$root/.apm/agents/unreadable.agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "unreadable.agent.md"
refute_output --partial "Traceback"
refute_output --partial "OSError"
}
@test "a DIRECTORY named like an agent file is rejected by shape, not audited as one" {
# The other half of the case the fixture above used to cover, kept as its own
# test so nothing is lost in the swap. A directory cannot be an agent target
# under ADR-0025's rule at all, so it never reaches check_apm_agent_file's
# OSError arm — it is refused at classification time. Exit 2 (not auditable)
# rather than 1 (audited and failing), and still a message naming the path:
# the failure mode this guards against is a silent exit 0 or a bare non-zero
# with an interpreter stack for a diagnostic.
local root="$TMPDIR/pkg-dir"
mkdir -p "$root/.apm/agents/is-a-dir.agent.md"
cat > "$root/apm.yml" <<EOF
@@ -913,18 +988,19 @@ version: 0.1.0
type: skill
EOF
run bash "$SCRIPT" "$root/.apm/agents/is-a-dir.agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "is-a-dir.agent.md"
[ "$status" -eq 2 ]
assert_output --partial "Error: '$root/.apm/agents/is-a-dir.agent.md' is a directory with no SKILL.md in it."
refute_output --partial "Traceback"
refute_output --partial "IsADirectoryError"
}
@test "a nonexistent project-scope agent file gets a FAIL naming the path, not a traceback" {
# The counterpart is pre-checked before either file is opened, so this
# exercises the OTHER call site: the counterpart exists, the named file does
# not, and check_file is what has to report it.
@test "a nonexistent project-scope agent file exits 2 naming the path, never the findings tier" {
# The counterpart exists and the named file does not. Before the guard this
# exercised the OTHER call site — check_file, reached after the counterpart
# pre-check — and the fact that the counterpart is present is what made it a
# distinct case. The guard is upstream of scope detection entirely, so both
# scopes now stop at the same place; the fixture is kept so a guard that ever
# regresses to a scope-specific check is caught on both paths.
local root="$TMPDIR/proj-missing"
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
@@ -936,10 +1012,11 @@ description: A valid agent description. Do not use for anything else.
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
assert_failure
assert_output --partial "FAIL"
assert_output --partial "could not be read"
assert_output --partial "my-agent.md"
# The literal 2, not assert_failure — see the plugin-scope twin above.
[ "$status" -eq 2 ]
assert_output --partial "Error: '$root/.claude/agents/my-agent.md' does not exist."
# Trivially true now that the guard short-circuits before python3; kept
# because the intent — a sentence, not an interpreter stack — is unchanged.
refute_output --partial "Traceback"
refute_output --partial "FileNotFoundError"
}
@@ -963,3 +1040,180 @@ EOF
refute_output --partial "UnicodeEncodeError"
refute_output --partial "Traceback"
}
# ---------------------------------------------------------------------------
# Auto-detection — the merged entry point classifies its own target
#
# NEW with the factory-audit merge, and new behaviour rather than a ported
# case: scripts/validate.sh is now ONE entry point for both artifact types and
# works out from the target which rubric to run. This file is the AGENT half of
# that contract; validate-skill.bats holds the skill half and the
# neither-shape rejection. Same script in all four, different fixtures.
#
# ADR-0025 states the rule: "A directory containing SKILL.md takes the skill
# flow; an .agent.md file or a file under agents/ takes the agent flow." That is
# TWO independent rules on the agent side, and each is pinned on its own below,
# because either one alone would make the other look like it worked:
#
# 1. the filename ends in .agent.md, wherever it sits
# 2. the file sits under a directory named agents/, whatever it is called
# (.apm/agents/, .claude/agents/, .github/agents/ — the same shape the
# exported Vale hook matches with `(^|/)agents/[^/]+\.md$`)
#
# A misclassification here is silent and total — the skill rubric would run
# against an agent file, so ADR-0022's metadata.version and ADR-0020's 900-word
# body ceiling would both be applied to an artifact that ADR-0020 explicitly
# exempts from the latter (see the 1,500-word body test above, which AGENTS.md
# cites as the pin for that asymmetry).
# ---------------------------------------------------------------------------
@test "auto-detect: a *.agent.md file is audited in AGENT mode wherever it sits" {
# Rule 1 in isolation. The directory is deliberately named `defs/`, not
# `agents/`, so rule 2 cannot reach this fixture and the extension is the
# only thing that can classify it. Put it under agents/ and both rules match,
# and a detector implementing only rule 2 would still pass this test.
local root="$TMPDIR/pkg-by-extension"
mkdir -p "$root/defs"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/defs/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description. Do not use for anything else.
target: cli
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/defs/my-agent.agent.md"
# 'target' is a Copilot-only field, and the vendor-field allowlist that
# rejects it exists only in the agent rubric. The finding is therefore proof
# of WHICH rubric ran, not just that something went wrong. Exit 1 exactly:
# a bare assert_failure also accepts exit 2, which is a refusal to audit, and
# "target" is too common a word to tell a findings report from a refusal.
assert_failure 1
assert_output --partial "target"
}
@test "auto-detect: a file under .apm/agents/ is audited in AGENT mode even without the .agent.md suffix" {
# Rule 2 in isolation. `.apm/agents/` is the vendor-neutral authoring
# directory, so anything sitting in it is an agent definition whatever it is
# called — and the plain `.md` suffix here is deliberate, so that rule 1
# cannot rescue the fixture and rule 2 is the only thing being measured.
local root="$TMPDIR/pkg-by-location"
mkdir -p "$root/.apm/agents"
cat > "$root/apm.yml" <<EOF
name: test-package
version: 0.1.0
type: skill
EOF
cat > "$root/.apm/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description. Do not use for anything else.
tools: Read Edit
---
You are a test agent. When invoked, do the thing.
EOF
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
# 'tools' sits outside the plugin/APM allowlist — again agent-rubric-only
# vocabulary, so the finding names the rubric that produced it. Exit 1
# exactly, for the same reason as the case above: exit 2 is a refusal, not a
# findings report, and "tools" alone cannot tell the two apart.
assert_failure 1
assert_output --partial "tools"
}
@test "auto-detect: an agent file never picks up a skill-only gate" {
# The control for both cases above. Without it, either is satisfied by a
# detector that runs BOTH rubrics and unions the findings — which would
# report every agent file in the corpus as missing a metadata.version it is
# not required to have.
local root="$TMPDIR/pkg-clean-detect"
make_apm_agent "$root" "my-agent"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
assert_success
refute_output --partial "metadata.version"
refute_output --partial "900-word"
refute_output --partial "SKILL.md"
}
# ---------------------------------------------------------------------------
# Entry-point robustness: how the target and the libraries are located
# ---------------------------------------------------------------------------
# Three defects in the merged entry point's own plumbing, none of which a valid
# absolute-path target exercises:
#
# - The agents/-parent rule read the parent's NAME off the argument text, so a
# bare `my-agent.md` typed from inside the agents/ directory had a parent of
# "." and was refused as matching neither shape — by a message naming that
# exact shape as valid.
# - A missing lib-*.sh killed the script under `set -e` with exit 1, the tier
# the flow references relay verbatim as real findings.
# - SCRIPT_DIR was resolved with a bare `cd`, which under an exported CDPATH
# prints the directory it chose and can choose a DIFFERENT one — so a
# relative invocation could source same-named libraries from elsewhere.
# ---------------------------------------------------------------------------
@test "entry point: a bare filename typed from inside an agents/ directory is audited, not refused" {
# The plain .md suffix is deliberate: *.agent.md is accepted on its name
# alone, so only the agents/-parent rule is measured here.
local root="$TMPDIR/pkg-bare"
make_apm_agent "$root" "my-agent"
mv "$root/.apm/agents/my-agent.agent.md" "$root/.apm/agents/my-agent.md"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
local abs_status="$status"
# The literal 0, not `-ne 2`. The three invocations below are only compared
# against this one, so an anchor of "anything but 2" left the whole test
# green under a regression that made all three exit 1 — every verdict here
# was self-referential. make_apm_agent builds a CLEAN agent, so the
# absolute-path baseline is 0 and nothing weaker belongs here. Its provenance
# twin in validate-provenance-agent.bats pins the same literal.
[ "$abs_status" -eq 0 ]
cd "$root/.apm/agents"
run bash "$SCRIPT" my-agent.md
refute_output --partial "matches neither"
[ "$status" -eq "$abs_status" ]
run bash "$SCRIPT" ./my-agent.md
refute_output --partial "matches neither"
[ "$status" -eq "$abs_status" ]
}
@test "entry point: a missing mode library exits 2 naming it, never exit 1" {
# Exit 1 is the findings tier. A validate.sh separated from its libraries is
# an install problem, and has to land in the "never ran" tier instead.
local root="$TMPDIR/pkg-lone"
make_apm_agent "$root" "my-agent"
local lone="$TMPDIR/lone-scripts"
cp -R "$(dirname "$SCRIPT")" "$lone"
rm "$lone/lib-checks-agent.sh"
run bash "$lone/validate.sh" "$root/.apm/agents/my-agent.agent.md"
[ "$status" -eq 2 ]
assert_output --partial "required library '$lone/lib-checks-agent.sh' is missing or unreadable"
refute_output --partial "No such file or directory"
}
@test "entry point: an exported CDPATH cannot redirect which libraries are sourced" {
# A decoy scripts/ directory reachable through CDPATH, holding libraries that
# announce themselves and abort. The relative invocation `bash
# scripts/validate.sh` is the one the flow references prescribe, and it is the
# one CDPATH applies to.
local root="$TMPDIR/pkg-cdpath"
make_apm_agent "$root" "my-agent"
local decoy="$TMPDIR/decoy"
mkdir -p "$decoy/scripts"
local lib
for lib in lib-boundary-resolver.sh lib-checks-skill.sh lib-checks-agent.sh; do
printf 'echo DECOY-SOURCED; exit 99\n' > "$decoy/scripts/$lib"
done
cd "$(dirname "$SCRIPT")/.."
run env CDPATH="$decoy" bash scripts/validate.sh "$root/.apm/agents/my-agent.agent.md"
refute_output --partial "DECOY-SOURCED"
assert_success
}

View File

@@ -586,12 +586,41 @@ EOF
# pins it as exit 0 with empty output. Every exit-2 gate is therefore decided
# from the ARGUMENT ALONE, before the walk-up runs, so the two can never
# collide. The two tests at the end of this block assert both halves.
#
# PORT NOTE RESOLVED (factory-audit merge): two cases in this block asserted
# wording that the skill suite asserted differently for the same class of input
# — "agent-file is required" against its "skill-dir is required", and "no such
# file" against its "not a directory". Each pre-merge script knew what shape it
# was owed; the merged one classifies before it complains. How each pair landed:
#
# * no argument — NEITHER old wording survives. One entry point takes both
# shapes, so it names both: "a skill directory or an agent file is
# required." Both suites now assert that one sentence.
# * nonexistent path — "no such file" SURVIVES here and did not move. A
# nonexistent *.agent.md still classifies as an agent on its name alone, so
# agent mode's own precondition is what rejects it. The skill suite's "not a
# directory" is the wording that moved, because a nonexistent path with no
# agent-shaped name classifies as neither.
#
# Two further cases in this block did move, and neither moved to the other
# suite's wording: a directory handed to this script is now "is a directory with
# no SKILL.md in it" rather than "not a regular file", and an unrecognized
# extension is "matches neither a skill directory nor an agent file" rather than
# "unrecognized extension". Both are classification verdicts now, reached before
# either mode's preconditions run.
#
# The invariant every case in this block pins is unchanged and still pinned at
# every site: exit 2 with an explanatory message on stderr, never a silent exit
# 0 or a bare 1 with no output.
# ---------------------------------------------------------------------------
@test "exit 2: no arguments is a usage error, not a finding" {
run bash "$SCRIPT"
[ "$status" -eq 2 ]
assert_output --partial "agent-file is required"
assert_output --partial "Error: a skill directory or an agent file is required."
# The usage block has to follow the error, or "required" names no shape the
# caller can act on.
assert_output --partial "Usage: validate-provenance.sh"
}
@test "exit 2: a second positional argument is rejected instead of silently dropped" {
@@ -610,20 +639,29 @@ EOF
}
@test "exit 2: a directory is not an agent file" {
# Post-merge this is a CLASSIFICATION rejection, not agent mode's old
# "not a regular file" precondition: a directory is half of a skill target,
# so the detector reaches for SKILL.md, does not find one, and says so. The
# thing being pinned is the same — a directory handed to a script that
# audits agent FILES exits 2 with a message, never a silent 0.
local root="$TMPDIR/package"
make_package "$root"
run bash "$SCRIPT" "$root/.apm/agents"
[ "$status" -eq 2 ]
assert_output --partial "not a regular file"
assert_output --partial "Error: '$root/.apm/agents' is a directory with no SKILL.md in it."
}
@test "exit 2: an unrecognized extension is rejected before the walk-up runs" {
# Also a classification rejection now. .txt under .apm/agents/ is neither
# shape: the agents/-parent rule only admits .md, and this is not *.agent.md.
# Rejected on the name alone, so the walk-up still never runs — which is what
# keeps this case from colliding with the silent not-plugin-scope exit 0.
local root="$TMPDIR/package"
make_package "$root"
echo "not an agent" > "$root/.apm/agents/my-agent.txt"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.txt"
[ "$status" -eq 2 ]
assert_output --partial "unrecognized extension"
assert_output --partial "Error: '$root/.apm/agents/my-agent.txt' matches neither a skill directory nor an agent file."
}
@test "exit 2: a PATH with no python3 names the missing dependency instead of exiting 127" {
@@ -632,6 +670,22 @@ EOF
make_clean_agent "$root"
local emptybin="$TMPDIR/emptybin"
mkdir -p "$emptybin"
# dirname and basename are deliberately ABSENT, and that absence is load-
# bearing. The invariant: nothing external is needed to reach the python3
# preflight. The entry point resolves SCRIPT_DIR and classifies the target
# with bash builtins (_kf_dirname, _kf_basename, _kf_parent_name) and the
# mode libraries build their Python bodies with `read` heredocs, so a
# genuinely empty PATH reaches the preflight too. Widening this list to keep
# a test green would silently retire that guarantee: the script would die at
# 127 naming `dirname` instead of the dependency it actually needs, the
# failure tests/test-adr0020-contract.sh assertion 2 exists to prevent. cat
# and sed stay so the stub matches the skill suite's and is a DENY OF python3
# ALONE rather than a test of "no PATH at all"; bash is not among them
# because `env -i` below invokes it by absolute path.
local cmd
for cmd in cat sed; do
ln -s "$(command -v "$cmd")" "$emptybin/$cmd"
done
local bash_bin
bash_bin="$(command -v bash)"
run env -i PATH="$emptybin" HOME="$HOME" "$bash_bin" "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
@@ -907,3 +961,187 @@ EOF
count="$(printf '%s\n' "$output" | grep -c "^FAIL File is not valid UTF-8" || true)"
[ "$count" -eq 1 ]
}
# ---------------------------------------------------------------------------
# Auto-detection — the merged provenance entry point classifies its own target
#
# NEW with the factory-audit merge, and new behaviour rather than a ported
# case: scripts/validate-provenance.sh is now ONE entry point for both artifact
# types and works out from the target which rubric to run. This file is the
# AGENT half of that contract; validate-provenance-skill.bats holds the skill
# half and the neither-shape rejection. Same script in all four suites.
#
# ADR-0025 states the rule: "A directory containing SKILL.md takes the skill
# flow; an .agent.md file or a file under agents/ takes the agent flow." That is
# TWO independent rules on the agent side, and each is pinned on its own below,
# because either one alone would make the other look like it worked:
#
# 1. the filename ends in .agent.md, wherever it sits
# 2. the file sits under a directory named agents/, whatever it is called
# (.apm/agents/, .claude/agents/, .github/agents/ — the same shape the
# exported Vale hook matches with `(^|/)agents/[^/]+\.md$`)
#
# The skill rubric rejects a FILE outright ("not a directory"), so a
# misclassified agent file does not produce a wrong finding — it produces a
# usage error about a file that was perfectly well-formed, which is why these
# assert on a real finding rather than merely on a non-crash.
# ---------------------------------------------------------------------------
@test "auto-detect: a *.agent.md file is checked in AGENT provenance mode wherever it sits" {
# Rule 1 in isolation. The directory is deliberately named `defs/`, not
# `agents/`, so rule 2 cannot reach this fixture and the extension is the
# only thing that can classify it.
local root="$TMPDIR/package"
make_package "$root"
mkdir -p "$root/defs"
cat > "$root/defs/my-agent.agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- ghost-source
---
You are a test agent.
EOF
make_sources_md "$root" "my-source" "(none)"
run bash "$SCRIPT" "$root/defs/my-agent.agent.md"
assert_failure
assert_output --partial "source_keys slug 'ghost-source' not found in sources.md"
# Check 9 is skill-only. It must not appear here at any tier — its presence
# would mean the skill rubric had been run against an agent file.
refute_output --partial "Check 9"
}
@test "auto-detect: a file under .apm/agents/ is checked in AGENT provenance mode even without the .agent.md suffix" {
# Rule 2 in isolation. `.apm/agents/` is the vendor-neutral authoring
# directory, so anything sitting in it is an agent definition whatever it is
# called — and the plain `.md` suffix here is deliberate, so that rule 1
# cannot rescue the fixture and rule 2 is the only thing being measured.
local root="$TMPDIR/package"
make_package "$root"
cat > "$root/.apm/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
source_keys:
- ghost-source
---
You are a test agent.
EOF
make_sources_md "$root" "my-source" "(none)"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
assert_failure
assert_output --partial "source_keys slug 'ghost-source' not found in sources.md"
refute_output --partial "Check 9"
}
@test "--base-ref is SKILL-mode only: handing it an agent target is an error, not a silent no-op" {
# --base-ref belongs to skill mode's check 9, and agent mode has no check 9
# at any tier. The merged entry point does NOT own the flag rules — it skips
# --base-ref only far enough to find the positional that decides the mode,
# then hands the original "$@" to the mode's own suite. So in agent mode the
# flag is still just a second argument and is still rejected as one. Silently
# accepting and ignoring it is the failure worth pinning: a caller who passed
# --base-ref believes a diff ran, and nothing would tell them otherwise.
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md" --base-ref=HEAD
[ "$status" -eq 2 ]
assert_output --partial "expected exactly one argument"
}
@test "--base-ref before an agent target is rejected too, not swallowed by the mode detector" {
# The detector skips leading --base-ref= tokens to find the positional. That
# skip must not launder the flag out of the argument list: the suite still
# sees two arguments and still rejects them. Asserted separately from the
# trailing-flag case because the skip loop only runs on this ordering, so a
# detector that consumed the flag instead of stepping over it would pass the
# test above and fail only here.
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
run bash "$SCRIPT" --base-ref=HEAD "$root/.apm/agents/my-agent.agent.md"
[ "$status" -eq 2 ]
assert_output --partial "expected exactly one argument"
}
@test "exit contract: AGENT mode prints NOTHING on a successful run" {
# The other half of the contract pinned in validate-provenance-skill.bats:
# skill mode is allowed to exit 0 with INFO findings on stdout; agent mode
# says nothing at all when it passes.
#
# Both halves are asserted explicitly and on purpose. A merge that collapsed
# one contract into the other would still satisfy whichever side was left
# unasserted, so a single-sided pin would go green on exactly the defect it
# was written to catch. assert_output "" is an equality assertion, not a
# --partial one, so any added banner, mode line or progress note trips it.
local root="$TMPDIR/package"
make_package "$root"
make_agent_with_source_keys "$root"
make_sources_md "$root"
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
[ "$status" -eq 0 ]
assert_output ""
}
# ---------------------------------------------------------------------------
# Entry-point robustness: how the target and the libraries are located
# ---------------------------------------------------------------------------
# The same three plumbing defects validate-agent.bats pins for validate.sh — a
# bare filename inside agents/ refused as neither shape, a missing lib-*.sh
# reported as exit 1 findings, and a CDPATH-redirected SCRIPT_DIR — pinned here
# for the provenance entry point, which resolves both the same way.
# ---------------------------------------------------------------------------
@test "entry point: a bare filename typed from inside an agents/ directory is audited, not refused" {
local root="$TMPDIR/package"
make_package "$root"
cat > "$root/.apm/agents/my-agent.md" <<EOF
---
name: my-agent
description: A valid agent description.
---
You are a test agent.
EOF
cd "$root/.apm/agents"
run bash "$SCRIPT" my-agent.md
[ "$status" -eq 0 ]
assert_output ""
run bash "$SCRIPT" ./my-agent.md
[ "$status" -eq 0 ]
assert_output ""
}
@test "entry point: a missing library exits 2 naming it, never exit 1" {
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
local lone="$TMPDIR/lone-scripts"
cp -R "$(dirname "$SCRIPT")" "$lone"
rm "$lone/lib-contributing-files.sh"
run bash "$lone/validate-provenance.sh" "$root/.apm/agents/my-agent.agent.md"
[ "$status" -eq 2 ]
assert_output --partial "required library '$lone/lib-contributing-files.sh' is missing or unreadable"
refute_output --partial "No such file or directory"
}
@test "entry point: an exported CDPATH cannot redirect which libraries are sourced" {
local root="$TMPDIR/package"
make_package "$root"
make_clean_agent "$root"
local decoy="$TMPDIR/decoy"
mkdir -p "$decoy/scripts"
local lib
for lib in lib-contributing-files.sh lib-provenance-skill.sh lib-provenance-agent.sh; do
printf 'echo DECOY-SOURCED; exit 99\n' > "$decoy/scripts/$lib"
done
cd "$(dirname "$SCRIPT")/.."
run env CDPATH="$decoy" bash scripts/validate-provenance.sh "$root/.apm/agents/my-agent.agent.md"
[ "$status" -eq 0 ]
assert_output ""
}

View File

@@ -1291,19 +1291,44 @@ EOF
# ---------------------------------------------------------------------------
# Cycle 21 — G1: a bad target is a hard error, not a silent pass
#
# PORT NOTE RESOLVED (factory-audit merge): this block used to assert
# "not a directory" for the nonexistent target while the agent suite asserted
# "no such file" for its own, because each pre-merge script knew what shape it
# was owed. The merged entry point classifies before it complains, so neither
# old wording survives at THIS site and the two cases are no longer the same
# case:
#
# * a path that exists as nothing this script recognises (and a path that does
# not exist at all, when its name is not *.agent.md and its parent is not
# agents/) is "matches neither a skill directory nor an agent file";
# * a path that IS a directory but holds no SKILL.md gets its own, more
# specific line naming the missing SKILL.md.
#
# The agent suite's "no such file" wording did NOT move — a nonexistent
# *.agent.md path still classifies as an agent and is rejected by agent mode's
# own precondition — so that side stays verbatim over there. Both wordings are
# now asserted exactly rather than partially-matched, because the invariant both
# sides pin is exit 2 with a message that names the path, never a silent exit 0
# and never a bare code with no output.
# ---------------------------------------------------------------------------
@test "G1: a nonexistent directory is a hard error (exit 2), not a silent exit 0" {
run bash "$SCRIPT" "$TMPDIR/does-not-exist"
[ "$status" -eq 2 ]
assert_output --partial "not a directory"
assert_output --partial "Error: '$TMPDIR/does-not-exist' matches neither a skill directory nor an agent file."
}
@test "G1: a directory with no SKILL.md is a hard error (exit 2), not a silent exit 0" {
# Distinct from the case above on purpose: a directory IS half of a skill
# target, so the detector can say something sharper than "neither shape" and
# does. The exit code is unchanged from the pre-merge test, which already
# pinned 2; only the wording moved, from the suite's own "not a skill
# directory" precondition to the dispatcher's classification message.
mkdir -p "$TMPDIR/not-a-skill/references"
run bash "$SCRIPT" "$TMPDIR/not-a-skill"
[ "$status" -eq 2 ]
assert_output --partial "not a skill directory"
assert_output --partial "Error: '$TMPDIR/not-a-skill' is a directory with no SKILL.md in it."
}
# ---------------------------------------------------------------------------
@@ -1496,12 +1521,27 @@ EOF
# ---------------------------------------------------------------------------
# Cycle 27 — G8: usage and environment errors exit 2, never 1
#
# PORT NOTE RESOLVED (factory-audit merge): the two suites were written against
# two separate scripts and disagreed about the no-argument wording — this file
# asserted "skill-dir is required", the agent suite "agent-file is required".
# Neither survives. One entry point takes both target shapes, so its no-argument
# message names both, and both suites now assert that one sentence. Exit 2 and a
# message (not a silent exit 1, and not a bare code with no output) is the
# invariant both sides were really pinning, and it is still pinned from both.
#
# Note the asymmetry with scripts/validate.sh, which exits 1 on no argument:
# only validate-provenance.sh carries the exit-2 usage tier, and this test is
# what holds it there.
# ---------------------------------------------------------------------------
@test "G8: a missing argument exits 2, not 1" {
run bash "$SCRIPT"
[ "$status" -eq 2 ]
assert_output --partial "skill-dir is required"
assert_output --partial "Error: a skill directory or an agent file is required."
# The usage block has to follow the error, or "required" names no shape the
# caller can act on.
assert_output --partial "Usage: validate-provenance.sh"
}
@test "G8: an extra positional argument is rejected, not silently ignored" {
@@ -1517,6 +1557,18 @@ EOF
make_clean_skill "$skill"
local stub="$TMPDIR/emptybin"
mkdir -p "$stub"
# dirname and basename are deliberately ABSENT, and that absence is load-
# bearing. The invariant: nothing external is needed to reach the python3
# preflight. The entry point resolves SCRIPT_DIR and classifies the target
# with _kf_dirname/_kf_basename/_kf_parent_name — pure-bash replacements
# that exist for precisely this reason — and the mode libraries build their
# Python bodies with `read` heredocs. Widening this list to keep a test
# green would silently retire that guarantee: on a PATH with neither
# coreutils nor python3 the script would die at 127 naming `dirname`
# instead of naming the dependency it actually needs, which is the failure
# tests/test-adr0020-contract.sh assertion 2 exists to prevent. bash, cat
# and sed stay so the stub is a DENY OF python3 ALONE rather than a test of
# "no PATH at all" — the diagnostic under test here is the python3 one.
for cmd in bash cat sed; do
ln -s "$(command -v "$cmd")" "$stub/$cmd"
done
@@ -1900,3 +1952,111 @@ EOF
assert_output --partial "Check 9 skipped — no base ref could be resolved"
assert_output --partial "not-a-real-ref"
}
# ---------------------------------------------------------------------------
# Auto-detection — the merged provenance entry point classifies its own target
#
# NEW with the factory-audit merge, and new behaviour rather than a ported
# case: scripts/validate-provenance.sh is now ONE entry point for both artifact
# types and works out from the target which rubric to run. A DIRECTORY holding
# SKILL.md is a skill; a FILE named *.agent.md, or sitting under .apm/agents/,
# is an agent.
#
# The two rubrics are not near-copies of one another — the skill side has a
# check 9 the agent side has none of, and reads sources.md from the skill's own
# references/ rather than from the package root — so a misclassification is not
# a near miss. It runs a set of checks that cannot apply and skips the set that
# can, at exit 0.
#
# This file is the SKILL half plus the neither-shape rejection;
# validate-provenance-agent.bats holds the agent half. Same script in both.
# ---------------------------------------------------------------------------
@test "auto-detect: a directory holding SKILL.md is checked in SKILL provenance mode" {
local skill="$TMPDIR/my-skill"
make_skill_with_source_keys "$skill"
make_sources_md "$skill"
# No commit_as_base, so check 9 has no repo root and announces the skip.
run bash "$SCRIPT" "$skill"
assert_success
# Check 9 exists only on the skill side — the agent rubric has no check 9 at
# any tier — so naming it is positive proof the SKILL rubric ran, rather
# than merely that nothing crashed.
assert_output --partial "Check 9 skipped — no repo root above the skill directory"
}
@test "auto-detect: a SKILL.md FILE path is checked in SKILL provenance mode, not rejected" {
# NEW with the merge and additive: pre-merge, handing the SKILL.md itself to
# skill-audit's validate-provenance.sh hit the "not a directory" precondition
# and died. It matters because pre-commit `files:` hooks match FILES — the
# exported kyberforge-vale-audit-skill hook's regex is (^|/)SKILL\.md$ — so
# every hook-driven invocation hands over a SKILL.md path, never its
# directory. The entry point rewrites the token to the directory in place.
local skill="$TMPDIR/my-skill"
make_skill_with_source_keys "$skill"
make_sources_md "$skill"
run bash "$SCRIPT" "$skill/SKILL.md"
assert_success
# Same positive proof as the directory case: check 9 is skill-only.
assert_output --partial "Check 9 skipped — no repo root above the skill directory"
}
@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" {
# The rewrite must be transparent, not merely non-fatal. If the two spellings
# of the same target could disagree, a pre-commit run and a hand run would
# report differently on one skill and neither would be obviously wrong.
local skill="$TMPDIR/my-skill"
make_skill_with_source_keys "$skill"
make_sources_md "$skill"
run bash "$SCRIPT" "$skill"
local dir_status="$status"
local dir_output="$output"
run bash "$SCRIPT" "$skill/SKILL.md"
[ "$status" -eq "$dir_status" ]
[ "$output" = "$dir_output" ]
# Guard against the comparison being satisfied by two empty runs: this
# fixture has a check-9 INFO to print, so silence here means neither
# spelling ran the rubric.
refute_output ""
}
@test "exit contract: SKILL mode exits 0 WITH output when the only findings are INFO" {
# The two modes' exit contracts are DIFFERENT and the merge must not quietly
# unify them. Skill mode is allowed to be chatty on a passing run: INFO
# findings print and the status stays 0. Agent mode's opposite half — a
# passing run prints nothing at all — is pinned from its own side in
# validate-provenance-agent.bats.
#
# Both halves are asserted explicitly and on purpose. A merge that collapsed
# one contract into the other would still satisfy whichever side was left
# unasserted, so a single-sided pin would go green on exactly the defect it
# was written to catch.
local skill="$TMPDIR/my-skill"
make_skill_with_source_keys "$skill"
make_sources_md "$skill"
run bash "$SCRIPT" "$skill"
[ "$status" -eq 0 ]
refute_output ""
assert_output --partial "INFO"
refute_output --partial "FAIL"
}
@test "auto-detect: a provenance target that is neither a skill directory nor an agent file FAILs, naming what it was handed" {
# The detector must not guess. Guessing here is worse than in validate.sh:
# this script's whole not-in-scope path is a SILENT exit 0, so a wrong guess
# followed by "provenance does not apply at this scope" is indistinguishable
# from a clean pass — which is the exact confusion the exit-2 tier above was
# created to end.
#
# A plain .txt file is neither shape under any reading of the contract: not
# a directory holding SKILL.md, not *.agent.md, not under .apm/agents/.
local dir="$TMPDIR/neither"
mkdir -p "$dir"
echo "not an artifact of either kind" > "$dir/notes.txt"
run bash "$SCRIPT" "$dir/notes.txt"
assert_failure
refute_output ""
assert_output --partial "$dir/notes.txt"
}

View File

@@ -728,7 +728,7 @@ PY
# A skill carrying `disable-model-invocation: true` is absent from the
# model-visible listing entirely: not preloaded, and the Skill tool refuses to
# call it. Its description is never matched against user intent, so
# references/description-quality.md Step 0 gives it ONE plain human-facing
# references/skill-description-quality.md Step 0 gives it ONE plain human-facing
# sentence — no trigger list, no boundary clause — and calls a
# missing-boundary-clause finding on such a skill "a wrong finding, not a strict
# one". Until this ran, nothing here knew the field existed, so the audit
@@ -920,3 +920,110 @@ EOF
refute_output --partial "UnicodeEncodeError"
refute_output --partial "Traceback"
}
# ---------------------------------------------------------------------------
# Auto-detection — the merged entry point classifies its own target
#
# NEW with the factory-audit merge, and new behaviour rather than a ported
# case: scripts/validate.sh is now ONE entry point for both artifact types and
# works out from the target which rubric to run. A DIRECTORY holding SKILL.md is
# a skill; a FILE named *.agent.md, or sitting under .apm/agents/, is an agent.
#
# Before the merge each script was hard-wired to one type, so there was nothing
# here that could be wrong. Now a misclassification is silent and total: the
# wrong rubric runs end to end and reports the artifact clean against gates that
# never applied to it, while every gate that did apply goes unrun. Nothing else
# in this suite would notice, because every other fixture is a skill directory
# and would be classified correctly even by a detector that always guessed
# "skill".
#
# The agent half of the same contract is pinned from the other side, in
# validate-agent.bats — same script, same detector, agent-shaped fixtures.
# ---------------------------------------------------------------------------
@test "auto-detect: a directory holding SKILL.md is audited in SKILL mode" {
local skill="$TMPDIR/my-skill"
make_valid_skill "$skill"
run bash "$SCRIPT" "$skill"
assert_success
# ADR-0022's metadata.version is mandatory for skills and has no agent
# analogue whatsoever, so this line is positive proof the SKILL rubric ran —
# not merely that the run survived. A bare assert_success would be satisfied
# by a detector that classified this as an agent and found nothing to say.
assert_output --partial "metadata.version present"
# 'counterpart' is agent-mode vocabulary (the CC/Copilot pair check). A skill
# directory must never reach a check that has a concept of a counterpart.
refute_output --partial "counterpart"
}
@test "auto-detect: a SKILL.md FILE path is audited in SKILL mode, not rejected" {
# NEW with the merge and additive rather than ported: pre-merge, handing the
# SKILL.md itself to skill-audit's validate.sh hit the directory precondition
# and gave a useless exit 1. It matters because pre-commit `files:` hooks
# match FILES — the exported kyberforge-vale-audit-skill hook's regex is
# (^|/)SKILL\.md$ — so every hook-driven invocation hands over a SKILL.md
# path, never the directory above it. The entry point resolves the file to
# its directory before dispatching.
local skill="$TMPDIR/my-skill"
make_valid_skill "$skill"
run bash "$SCRIPT" "$skill/SKILL.md"
assert_success
# Same positive proof as the directory case: metadata.version is skill-only
# and has no agent analogue, so this line says the SKILL rubric ran.
assert_output --partial "metadata.version present"
refute_output --partial "counterpart"
}
@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" {
# The resolution must be transparent, not merely non-fatal. If the two
# spellings of one target could disagree, a pre-commit run and a hand run
# would report differently on the same skill and neither would look wrong.
# The name check is the sharp end: it compares `name` against the DIRECTORY
# basename, so a target left as the file would compare against "SKILL.md".
local skill="$TMPDIR/my-skill"
make_valid_skill "$skill"
run bash "$SCRIPT" "$skill"
local dir_status="$status"
local dir_output="$output"
run bash "$SCRIPT" "$skill/SKILL.md"
[ "$status" -eq "$dir_status" ]
[ "$output" = "$dir_output" ]
assert_output --partial "name 'my-skill' matches directory 'my-skill'"
}
@test "auto-detect: a target that is neither a skill directory nor an agent file FAILs, naming what it was handed" {
# The detector must not guess. Falling back to either rubric on an
# unclassifiable target yields a verdict about rules that were never meant
# to apply, and exiting 0 publishes that verdict as a pass — the worst of
# the three possible outcomes, because it is the silent one.
#
# A plain .txt file is neither shape under any reading of the contract: not
# a directory holding SKILL.md, not *.agent.md, not under .apm/agents/. The
# directory flavour of the same mismatch — a directory with no SKILL.md — is
# pinned separately by "fails when SKILL.md is missing" above.
local dir="$TMPDIR/neither"
mkdir -p "$dir"
echo "not an artifact of either kind" > "$dir/notes.txt"
run bash "$SCRIPT" "$dir/notes.txt"
assert_failure
# Non-zero is necessary but not sufficient: a non-zero exit with nothing on
# stdout is indistinguishable from a clean-but-failing run, which is the
# confusion the exit-2 tier in the provenance suites was created to end.
refute_output ""
assert_output --partial "$dir/notes.txt"
}
@test "entry point: a missing resolver library in skill mode exits 2 naming it, never exit 1" {
# validate-agent.bats pins this for agent mode; each mode sources its own
# libraries, so each mode's guard is pinned separately.
make_valid_skill "$TMPDIR/my-skill"
local lone="$TMPDIR/lone-scripts"
cp -R "$(dirname "$SCRIPT")" "$lone"
rm "$lone/lib-boundary-resolver.sh"
run bash "$lone/validate.sh" "$TMPDIR/my-skill"
[ "$status" -eq 2 ]
assert_output --partial "required library '$lone/lib-boundary-resolver.sh' is missing or unreadable"
refute_output --partial "No such file or directory"
}

View File

@@ -18,7 +18,7 @@ metadata:
## Gotchas
- forge is an optional guided entry point, not a gate — `skill-author`, `skill-audit`, `agent-author`, `agent-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one.
- forge is an optional guided entry point, not a gate — `skill-author`, `agent-author`, `factory-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one.
- Claude Code's skill-level `context: fork` frontmatter field and the `/fork` subagent command are opposites despite the shared word: `context: fork` isolates (fresh context, no parent access), while `/fork` inherits the full conversation. The route reference each classification loads spends that distinction: `references/author-routes.md` chooses between the two, `references/apm-routes.md` rules the fork out.
## Step 1 — Grill the intent

View File

@@ -7,8 +7,8 @@ source_keys:
Reached from `SKILL.md` Step 2 when the classified artifact is a skill or an agent/subagent
definition. Route a skill to `skill-author` and an agent to `agent-author`. The two branches
differ on one axis only — which audit skill verifies the result — and everything below applies to
both.
differ on the author skill only — both verify the result with `factory-audit`, which detects the
artifact type itself — and everything below applies to both.
## Choose fork or inline
@@ -26,8 +26,8 @@ Fall back to an **inline invocation** — same conversation, no subagent — whe
## Two-tier verification
Both author skills already close out with their own inline audit, in the same context as the
authoring work: `skill-author` runs `/skill-audit`, `agent-author` invokes
`agent-audit`. That is tier one, and forge does not change it.
authoring work: `skill-author` and `agent-author` each invoke `factory-audit` on what they wrote.
That is tier one, and forge does not change it.
Tier two belongs to forge. Once the author skill's run has finished, spin up a separate
**clean-context subagent** — fresh, not forked, no inherited context — to independently re-run the

View File

@@ -1,87 +0,0 @@
---
name: skill-audit
description: >
Use when the user wants a skill directory audited against the agentskills.io
spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or
after hand-editing a skill outside skill-author. Not applying fixes ->
skill-author.
allowed-tools: Bash Read
metadata:
version: "1.0.2"
category: factory
source_keys:
- agentskills-home
- agentskills-spec
- agentskills-best-practices
- agentskills-optimizing-descriptions
- agentskills-using-scripts
---
## Gotchas
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately.
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
## Step 1 — Deterministic checks
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
```bash
bash scripts/validate.sh <skill-dir>
bash scripts/validate-provenance.sh <skill-dir>
bash scripts/vale-wrap.sh <skill-dir>/SKILL.md
```
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation.
Read `references/validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback.
`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — a usage or environment error, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass.
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
| Rule | Dimension |
|---|---|
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
| `Kyberforge.PaddingPhrase` | patterns |
## Step 2 — Read the whole skill
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
## Step 3 — Qualitative audit
Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
| Dimension | Rubric |
|---|---|
| description | `references/description-quality.md` |
| body-discipline | `references/body-discipline.md` |
| patterns | `references/patterns.md` |
| file-structure, internal-consistency | `references/file-structure.md` |
| formatting, scripts | `references/formatting-and-scripts.md` |
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
## Step 4 — Report
Open with a coverage line naming every dimension checked:
```text
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
```
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
Each finding:
```text
FAIL/SUGGESTION <finding> — file:line
Why: <why this is a problem>
Fix: <exact change — quote before/after where applicable>
```
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.

View File

@@ -1,4 +0,0 @@
StylesPath = styles
[**/SKILL.md]
BasedOnStyles = Kyberforge

View File

@@ -1,13 +0,0 @@
extends: existence
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
level: error
scope: text.frontmatter.description
ignorecase: true
tokens:
- cross-cutting
- shared (skill|agent)
- human-facing
- entry[- ]point
- composes
- rather than duplicating
- replaces the (old|former|previous)

View File

@@ -1,7 +0,0 @@
extends: existence
message: "Description opens with '%s' — use an imperative 'Use when...' opener instead"
level: error
scope: text.frontmatter.description
ignorecase: true
raw:
- '^This\b'

View File

@@ -1,7 +0,0 @@
extends: existence
message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead"
level: error
scope: text
ignorecase: true
raw:
- 'see references?/? for (more )?(info|information|details)\b'

View File

@@ -1,7 +0,0 @@
extends: existence
message: "Don't start a sentence with '%s' — name the subject directly"
level: error
scope: sentence
ignorecase: false
raw:
- '^There\s(is|are)\b'

View File

@@ -1,10 +0,0 @@
extends: existence
message: "Vague capability wording: '%s' — state the capability precisely instead"
level: error
scope: text.frontmatter.description
ignorecase: true
tokens:
- helps with
- utilize
- assists with
- used for

View File

@@ -1,59 +0,0 @@
# Sources
<!-- agentskills.io/llms.txt was used for initial source discovery and is not listed below; it contributed no skill file content directly. -->
## agentskills-home
- **URL:** https://agentskills.io/home.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
- **Contributing files:** SKILL.md
- **Status:** `extracted`
## agentskills-spec
- **URL:** https://agentskills.io/specification.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
- **Status:** `extracted`
## agentskills-best-practices
- **URL:** https://agentskills.io/skill-creation/best-practices.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md, references/finding-criteria.md
- **Status:** `extracted`
## agentskills-optimizing-descriptions
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
- **Contributing files:** SKILL.md, references/description-quality.md, references/finding-criteria.md
- **Status:** `extracted`
## agentskills-evaluating-skills
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
- **Contributing files:** (none — eval workflow not directly informing audit dimensions)
- **Status:** `extracted`
## agentskills-using-scripts
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
- **Status:** `extracted`
## agentskills-quickstart
- **URL:** https://agentskills.io/skill-creation/quickstart.md
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
- **Contributing files:** (none — creation guide not directly informing audit criteria)
- **Status:** `extracted`

View File

@@ -1,526 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# Works around a Vale limitation: the `text.frontmatter.description` NLP scope
# silently stops matching once the `description:` value spans 2+ physical lines
# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block
# scalar (the style used by most skills/agents in this repo), a plain scalar
# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped
# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed
# value keeps exactly the line breaks the source has, and vale matches it fine
# (verified against vale 3.15.2), so literal blocks are deliberately left alone.
# This script flattens an affected description to a one-line scalar in a scratch
# copy — or, for the rare value no inline scalar can spell out verbatim, to a
# `|-` literal block with a single content line, which vale matches just as well
# (padding with blank lines so every other line number is unchanged), then
# runs the real `vale` binary against the copies. Drop-in replacement for calling
# `vale` directly: same args, same exit code, bar the two documented divergences
# below.
#
# "Same args" means relative paths — path arguments and the values of the
# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against
# the caller's current directory, exactly as bare `vale` resolves them. The flag
# values are rewritten to absolute form because the run ends up `cd`'d into the
# scratch mirror, where a relative one would no longer resolve. (An earlier
# version resolved path arguments against the repo root, an invented convention
# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse,
# silently dropped file arguments that didn't happen to resolve from the repo
# root — skipping the flattening this script exists for.)
#
# Divergence 1: with no `--config` at all, this script's own sibling
# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit
# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in
# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and
# hard-fail (E100) for every external consumer. The manifest therefore passes the
# script alone, and an explicit `--config` from any other caller still wins.
#
# Divergence 2: a path-shaped argument that does not exist is a hard error
# (exit 2). Bare vale drops it, falls back to reading stdin, and prints
# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable
# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather
# than clean, and `in stdin` does not match that guard, so the silent form would
# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way
# to keep that guard honest. Linting prose piped on stdin is therefore
# unsupported here — it already was, since the no-path handoff closes stdin so
# vale can't block on a pipe that will never carry content.
#
# Vale prints each path exactly as it was handed to it, so the scratch tree
# mirrors the caller's absolute cwd: a relative path argument is passed through
# verbatim and resolves to its flattened copy, keeping the report byte-identical
# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that
# property. Only an absolute path outside the cwd is rewritten to its scratch
# copy and so reports a scratch path — unavoidable, since a file can only be
# read from where it actually is.
cwd="$(pwd -P)"
# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 —
# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"`
# on an empty array as an unbound variable under `set -u`. No expansion site is
# reachable while empty on today's control flow, so this is insurance against a
# later edit breaking that invariant, not a live fix.
vale_args=()
path_args=()
pending_flag=""
config_given=false
# `--output` takes either one of vale's built-in style names or a template file
# path. Only the file form needs absolutizing, and the built-in names have to be
# excluded by name *before* the existence test below: a file or directory
# literally called `line` in the caller's cwd would otherwise rewrite the
# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template]
# Runtime error`) where bare vale just uses the built-in. `--path` has no such
# names — it is always a path — so the check is keyed on the flag too.
is_builtin_output() {
case "$2" in
line|JSON|CLI) [[ "$1" == "--output" ]] ;;
*) false ;;
esac
}
# Absolutizes a `--config` value against the caller's cwd. Shared by both
# argument forms below — separated (`--config X`) and joined (`--config=X`)
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
# one place instead of being duplicated per form.
abs_config_value() {
if [[ "$1" == /* ]]; then
printf '%s' "$1"
else
printf '%s' "$cwd/$1"
fi
}
for arg in "$@"; do
if [[ -n "$pending_flag" ]]; then
# Value of a separated two-argv flag. It is never a lint target, however
# file-like it looks. The run ends up `cd`'d into the scratch mirror, so a
# value naming a file has to be absolutized here or it stops resolving.
case "$pending_flag" in
--config)
# Always a path, and required to exist.
vale_args+=("$(abs_config_value "$arg")")
;;
--output|--path)
# See `is_builtin_output` above for why the built-in `--output` names
# are excluded first. Anything that names nothing is passed through and
# left for vale to interpret.
if is_builtin_output "$pending_flag" "$arg"; then
vale_args+=("$arg")
elif [[ "$arg" != /* && -e "$arg" ]]; then
vale_args+=("$cwd/$arg")
else
vale_args+=("$arg")
fi
;;
*)
vale_args+=("$arg")
;;
esac
pending_flag=""
continue
fi
case "$arg" in
--config)
vale_args+=("$arg")
pending_flag="$arg"
config_given=true
continue
;;
--config=*)
vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
config_given=true
continue
;;
# Same cwd-relative resolution for the `--flag=value` spelling of the two
# other path-valued flags.
--output=*|--path=*)
flag_val="${arg#*=}"
if is_builtin_output "${arg%%=*}" "$flag_val"; then
vale_args+=("$arg")
elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then
vale_args+=("${arg%%=*}=$cwd/$flag_val")
else
vale_args+=("$arg")
fi
continue
;;
# Vale's remaining value-taking flags, per `vale --help` (3.x). In the
# separated two-argv form the value must not be classified as a lint target
# — `--output tmpl.tmpl` names a real template file, and treating it as
# input both lints the template and reorders argv so vale sees
# `--output --no-wrap`. The `--flag=value` form needs no entry here: it
# starts with `-` and falls through to vale untouched. A value flag added by
# some future vale release is simply absent from this list and lands back on
# today's behaviour, so this list going stale is never worse than not having
# it.
--ext|--filter|--glob|--minAlertLevel|--output|--path)
vale_args+=("$arg")
pending_flag="$arg"
continue
;;
# Vale's subcommands are bare words that name no file, so they would trip
# the not-found error below. A lint target literally named `sync` (no
# extension, no slash) is misread as the subcommand — accepted, because the
# alternative is failing every `vale-wrap.sh ls-config`.
ls-config|ls-dirs|ls-metrics|ls-vars|sync)
vale_args+=("$arg")
continue
;;
esac
if [[ "$arg" == -* ]]; then
vale_args+=("$arg")
continue
fi
# Everything left is a lint target: `vale [options] [input...]` has no third
# kind of argument. See divergence 2 above for why a missing one is fatal here.
if [[ ! -e "$arg" ]]; then
echo "vale-wrap.sh: no such file or directory: $arg" >&2
exit 2
fi
# An absolute path inside the caller's cwd is relativized so the report cites
# a path that resolves against the real tree. Left absolute, it would be
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
# path to a file that is deleted on exit, which reads as a bug in any report
# quoting it. Absolute paths outside the cwd have no relative form and keep
# the scratch-path behaviour documented above.
if [[ "$arg" == "$cwd"/* ]]; then
path_args+=("${arg#"$cwd"/}")
else
path_args+=("$arg")
fi
done
if [[ "$config_given" == false ]]; then
vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini")
fi
if [[ ${#path_args[@]} -eq 0 ]]; then
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
# block waiting on a pipe that will never carry content.
exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null
fi
# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing)
# is a GNU extension the BSD realpath on macOS doesn't have — and every dest
# below is a path that doesn't exist yet. python3 is already a hard dependency.
abspath() {
python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1"
}
flatten() {
# Two call shapes: `flatten src dest` (dest already resolved and inside the
# scratch tree — the per-markdown-file calls in the directory branch below)
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
# branch further down) additionally resolves `raw_dest` the way a separate
# `abspath` call used to, applies the same sandbox-escape guard, and prints
# the resolved path — folding two python3 spawns per file into one.
python3 - "$@" <<'PYTHON'
import os
import re
import sys
src, dest_input = sys.argv[1], sys.argv[2]
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
if tmpdir is None:
dest = dest_input
else:
dest = os.path.abspath(dest_input)
if not dest.startswith(tmpdir + os.sep):
print(
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
f"land outside {tmpdir}",
file=sys.stderr,
)
sys.exit(2)
os.makedirs(os.path.dirname(dest), exist_ok=True)
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
# a byte-for-byte round trip instead of aborting the whole run on a decode error.
with open(src, encoding='utf-8', errors='surrogateescape') as fh:
content = fh.read()
# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\<newline>` is handled
# separately in unescape_double because it also swallows the next indentation.
DQ_ESCAPES = {
'0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n',
'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"',
'/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028',
'P': '\u2029',
}
# First characters that make a plain (unquoted) scalar mean something other than
# text: YAML's c-indicator set.
PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`'
def unescape_double(text):
"""Decode a double-quoted YAML scalar's body to the string YAML parses."""
out = []
i = 0
while i < len(text):
char = text[i]
if char != '\\':
out.append(char)
i += 1
continue
i += 1
if i >= len(text):
break
esc = text[i]
if esc == '\n':
i += 1
while i < len(text) and text[i] in ' \t':
i += 1
continue
if esc in 'xuU':
width = {'x': 2, 'u': 4, 'U': 8}[esc]
digits = text[i + 1:i + 1 + width]
if len(digits) == width:
try:
out.append(chr(int(digits, 16)))
except ValueError:
pass
else:
i += 1 + width
continue
out.append(DQ_ESCAPES.get(esc, esc))
i += 1
return ''.join(out)
def close_quote(text, quote):
"""Index of the closing `quote` in `text`, which starts just past the
opening one. None while the scalar is still unterminated."""
i = 0
while i < len(text):
char = text[i]
if quote == '"' and char == '\\':
i += 2
continue
if char == quote:
if quote == "'" and text[i + 1:i + 2] == "'":
i += 2
continue
return i
i += 1
return None
def continuation_lines(rest):
"""Yield the physical lines of `rest` that continue the value started on the
`description:` line. Indentation-based and blank-line-tolerant, per YAML:
a blank line (any amount of whitespace) always stays inside; the indent is
set by the first content line; the value ends at the first line indented
less than that, at any line flush with the key (that is the next mapping
key, not a continuation), or at EOF."""
indent = None
for line in rest.splitlines(keepends=True):
text = line.rstrip('\n')
if text.strip() == '':
yield line
continue
line_indent = len(text) - len(text.lstrip(' \t'))
if line_indent == 0:
return
if indent is None:
indent = line_indent
elif line_indent < indent:
return
yield line
def emit(value):
"""Render `value` as a YAML scalar whose source text spells the value out
verbatim. Vale locates the description by matching the parsed value back
against the source, so a scalar carrying any escape — `''` in a
single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the
whole `text.frontmatter.description` scope vanish, the same failure this
script exists to work around. Verbatim forms only, therefore, tried in
descending order of fidelity. The first three occupy one physical line; the
`|-` fallback occupies two, which the caller accounts for when padding."""
if (value
and value[0] not in PLAIN_UNSAFE_FIRST
and ': ' not in value
and not value.endswith(':')
and ' #' not in value):
return value # plain: nothing needs escaping at all
if "'" not in value:
return "'" + value + "'" # single-quoted: only `'` would escape
if '"' not in value and '\\' not in value:
return '"' + value + '"' # double-quoted: only `"`/`\` would
# Last resort: the value needs quoting AND holds an apostrophe AND a double
# quote or backslash, so no *inline* scalar can carry it verbatim. A `|-`
# literal block can — a block scalar's body has no escape syntax at all, so
# `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches
# the description scope against it (the header above says the same of the
# `|` blocks this script deliberately leaves alone; verified against vale
# 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed
# value is exactly `value` with no trailing newline.
return '|-\n ' + value
fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL)
if fm_match:
fm = fm_match.group(2)
header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE)
else:
header_m = None
if header_m:
head_start = header_m.start()
value_start = header_m.end()
header_end = fm.find('\n', value_start)
header_end = len(fm) if header_end == -1 else header_end
first = fm[value_start:header_end]
body_start = header_end + 1
indicator = first.rstrip()
block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator)
if block_m and block_m.group(1) == '|':
kind = None # literal blocks keep their line breaks; vale is fine
elif block_m:
kind = 'block' # folded (`>`): the value starts on the next line
elif indicator == '':
kind = 'block' # bare `description:`: a plain scalar on later lines
elif first[:1] == '"':
kind = 'double'
elif first[:1] == "'":
kind = 'single'
elif first[:1] in '#&*!':
kind = None # comment, anchor, alias or tag — not a plain scalar
else:
kind = 'plain'
text = ''
value_end = value_start
value_lines = 0
if kind in ('block', 'plain'):
body = ''.join(continuation_lines(fm[body_start:]))
value_end = body_start + len(body)
if kind == 'block':
text = body
value_lines = body.count('\n')
else:
text = fm[value_start:value_end]
value_lines = 1 + body.count('\n')
if ' #' in text or text.lstrip().startswith('#'):
# A `#` opens a comment inside a plain scalar. Folding it in
# would lint text YAML never treats as part of the value, so
# leave the file alone rather than lint the wrong string.
kind = None
elif kind in ('double', 'single'):
quote = '"' if kind == 'double' else "'"
inner_start = value_start + 1
acc = fm[inner_start:body_start]
idx = close_quote(acc, quote)
lines = continuation_lines(fm[body_start:])
while idx is None:
try:
acc += next(lines)
except StopIteration:
break
idx = close_quote(acc, quote)
if idx is None:
kind = None # unterminated quote: invalid YAML, leave it to vale
else:
inner = acc[:idx]
value_end = inner_start + idx + 1
text = unescape_double(inner) if quote == '"' else inner.replace("''", "'")
value_lines = 1 + inner.count('\n')
flat = re.sub(r'\s+', ' ', text).strip()
if kind and flat and value_lines >= 2:
# `value_end` can land mid-line, just past a closing quote, so extend to
# the end of that physical line and carry whatever follows (a trailing
# comment) across unchanged.
if value_end > 0 and fm[value_end - 1] == '\n':
span_end = value_end
trailer = ''
else:
newline = fm.find('\n', value_end)
span_end = len(fm) if newline == -1 else newline + 1
trailer = fm[value_end:span_end].rstrip('\n')
scalar = emit(flat)
# A trailing comment carried across from the original line stays on the
# `description:` line itself: after a block scalar's `|-` header it is
# still a comment, but inside the block body it would become part of the
# value.
head, newline_sep, block_body = scalar.partition('\n')
# The replacement displaces the whole span, so the blank-line pad makes
# up the difference between the lines it displaced and the lines it
# occupies — every later line number is unchanged. That is one line for
# the three inline forms and two for the `|-` block; the span itself is
# at least two lines here (`value_lines >= 2` is a precondition), so the
# pad count never goes negative.
pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n'))
new_fm = (fm[:head_start] + 'description: ' + head + trailer
+ newline_sep + block_body + '\n' + pad + fm[span_end:])
content = (fm_match.group(1) + new_fm + fm_match.group(3)
+ content[fm_match.end():])
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
fh.write(content)
if tmpdir is not None:
print(dest)
PYTHON
}
tmpdir="$(cd "$(mktemp -d)" && pwd -P)"
trap 'rm -rf "$tmpdir"' EXIT
# Mirror of the caller's cwd inside the scratch tree; relative path arguments
# are resolved from here.
mirror="$tmpdir$cwd"
mkdir -p "$mirror"
argv_paths=()
for arg in ${path_args[@]+"${path_args[@]}"}; do
if [[ "$arg" == /* ]]; then
raw_dest="$tmpdir$arg"
else
raw_dest="$mirror/$arg"
fi
if [[ -d "$arg" ]]; then
dest="$(abspath "$raw_dest")"
# A path argument with enough leading `..` to climb past the mirror root would
# write outside the scratch dir. The real filesystem clamps such a path at
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
case "$dest" in
"$tmpdir"/*) ;;
*)
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
exit 2
;;
esac
mkdir -p "$(dirname "$dest")"
# A directory is mirrored whole — vale applies its own format filtering to
# the tree, so any file dropped here would be silently unlinted — and then
# every markdown file in the copy is flattened in place. `.git` is pruned:
# vale never lints it and copying it can dwarf the rest of the tree.
# `find -L` follows symlinks because vale does: it lints both a symlinked
# file and a file under a symlinked directory, and a bare `-type f` walk
# would report "0 files" where bare vale reports one. (A symlink loop makes
# `find` warn on stderr and carry on, which is also what vale does.) The
# second walk needs no `-L`: the mirror is all real files by construction.
mkdir -p "$dest"
while IFS= read -r -d '' rel; do
mkdir -p "$dest/$(dirname "$rel")"
cp "$arg/$rel" "$dest/$rel"
done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0)
while IFS= read -r -d '' md; do
flatten "$md" "$md"
done < <(find "$dest" -type f -name '*.md' -print0)
else
# `abspath` + `flatten` folded into one python3 process — see the comment
# atop `flatten` above.
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
fi
if [[ "$arg" == /* ]]; then
argv_paths+=("$dest")
else
argv_paths+=("$arg")
fi
done
cd "$mirror"
vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"}

View File

@@ -1,29 +0,0 @@
# tests/
Test files for scripts bundled with this skill.
## Dependencies
Tests require [bats-support](https://github.com/bats-core/bats-support) and
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
helpers from the repo root's `tests/test_helper/`.
From the repo root:
```bash
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
```
Run all tests for this skill (from the repo root):
```bash
bats plugins/kyberforge/.apm/skills/skill-audit/tests/
```
## Files
| File | Purpose |
|------|---------|
| `validate.bats` | Bats test suite for `scripts/validate.sh` |
| `validate-provenance.bats` | Bats test suite for `scripts/validate-provenance.sh` |

View File

@@ -3,7 +3,7 @@ name: skill-author
description: >
Use when the user wants to create a new skill from scratch, or apply audit
findings, grill output, eval results, or inline feedback to an existing one.
Not read-only review -> `skill-audit`. Not agent files -> `agent-author`.
Not read-only review -> `factory-audit`. Not agent files -> `agent-author`.
allowed-tools: Bash Read Write Edit
metadata:
version: "1.0.2"
@@ -21,7 +21,7 @@ metadata:
## Gotchas
- The word gates are two measurements, not two tiers of one rule: the 2,770-word / 500-line spec backstop counts the whole file, Step 3's gate the body alone. Never unify them.
- Never spawn a subagent to audit or recheck your own work — run `/skill-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft.
- Never spawn a subagent to audit or recheck your own work — run `/factory-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft.
- Do not create new scripts unless a signal explicitly calls for it. Writing one from scratch requires out-of-scope transcript analysis — flag the opportunity as a suggestion instead.
## Step 1 — Dispatch
@@ -32,7 +32,7 @@ metadata:
| Directory exists, at least one improvement signal present | Improve | `references/improve.md` |
| Directory exists, no signals | Stop and ask | — |
Signals: grill output, `/skill-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply.
Signals: grill output, `/factory-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply.
Read only the reference matching the resolved flow — each is self-contained. If the target sits inside a git worktree, capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it.
@@ -47,7 +47,7 @@ Decide before writing any description: model-invoked or hand-invoked?
Before writing or editing a description, or restructuring a body, read `references/contract.md` — the banned-content list, boundary form, include/exclude rubric and body patterns.
Gates `/skill-audit` enforces in both flows:
Gates `/factory-audit` enforces in both flows:
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <skill-name>` whose target resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only.
- **Body** — decision procedure only: ordered steps, branches, gates, and which reference to load when. 600 words SUGGESTION, 900 FAIL, body only. At two or more mutually exclusive flows a dispatch table is mandatory and each flow gets its own self-contained `references/` file.
@@ -55,7 +55,7 @@ Gates `/skill-audit` enforces in both flows:
## Step 4 — Validate and close
Run `/skill-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists.
Run `/factory-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists.
Bump `metadata.version`: the **minor** version on create (new skills start at `0.1.0`) and the **patch** version on improve.

View File

@@ -7,7 +7,7 @@ source_keys:
# The description and body contract
House contract. Every rule here is enforced by `/skill-audit` —
House contract. Every rule here is enforced by `/factory-audit` —
`scripts/validate.sh` for the counts and the boundary targets, the bundled Vale styles for the
prose patterns, and its reference files for the judgment calls.
@@ -146,7 +146,7 @@ row's target exists on disk; each row pairs exactly one target with a condition
evaluate from the request, never a literal slash invocation; one line after the table names the
matched file as the only one to read; and the gates every branch needs sit in the body, not inside
one flow's file. That last one is the property the `git-commits` v0.1.2 failure turned on, and it is
the one a dispatch split is most likely to break. `skill-audit`'s `references/body-discipline.md`
the one a dispatch split is most likely to break. `factory-audit`'s `references/skill-body-discipline.md`
carries the audit-side form of the same exemption; the two lists are the same four properties, and
an edit to either belongs in both.

View File

@@ -28,8 +28,8 @@ Before touching the filesystem, verify you have:
If any are missing, stop and ask the user before proceeding.
**Requires `/skill-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the
**Requires `/factory-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
the kyberforge plugin and are co-installed. If `/factory-audit` is unavailable, stop and ask the
user to install the kyberforge plugin before continuing.
## Package-intent gate
@@ -175,7 +175,7 @@ If a research `sources.md` is present in the conversation context:
`- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the
plugin-level research sources file this entry was drawn from (e.g.
`plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every
entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
entry — it makes the provenance chain explicit and is validated by `/factory-audit`.
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of
sources that informed it.
5. For each file in `references/` that was informed by research sources, add `source_keys`

View File

@@ -44,7 +44,7 @@ Use `${CLAUDE_PLUGIN_ROOT}` only in hook commands and `.mcp.json` configs — no
## Standalone mode
Deployed directly to `~/.agents/skills/<name>/`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/skill-audit`) work if the called skill is also installed.
Deployed directly to `~/.agents/skills/<name>/`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/factory-audit`) work if the called skill is also installed.
## Cross-tool portability

View File

@@ -16,12 +16,12 @@ Confirm the skill directory path exists and that at least one improvement signal
conversation or a referenced file.
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
existing signals to a skill. For a blind review without signals, use `/factory-audit` instead."
Signals can come from anywhere in the conversation or referenced files:
- Grill session output (most common predecessor in the factory sequence)
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
- `/factory-audit` findings (PASS/FAIL/SUGGESTION punch list)
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
- Session context describing what went wrong

View File

@@ -186,4 +186,4 @@ echo " 3. Add docs to references/ if needed (or delete the directory)" >&2
echo " 4. Add resources to assets/ if needed (or delete the directory)" >&2
echo " 5. Add tests to tests/ if the skill has scripts (or delete the directory)" >&2
echo " 6. Populate references/sources.md with research sources, or delete it" >&2
echo " 7. Validate: run /skill-audit on $TARGET" >&2
echo " 7. Validate: run /factory-audit on $TARGET" >&2

View File

@@ -60,9 +60,9 @@ teardown() {
assert [ -d "$DEST/my-tool-2" ]
}
@test "next-steps output references /skill-audit not validate.sh" {
@test "next-steps output references /factory-audit not validate.sh" {
run bash "$SCRIPT" my-tool "$DEST"
assert_output --partial "/skill-audit"
assert_output --partial "/factory-audit"
refute_output --partial "validate.sh"
}

Some files were not shown because too many files have changed in this diff Show More