From 620f20b0fd5a38ff99e1ca7279b6a6057dba93b2 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Tue, 15 Sep 2026 18:39:43 +0000 Subject: [PATCH] refactor(kyberforge)!: merge skill-audit and agent-audit into factory-audit MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Why The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser (93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line test suite. Sync-by-gate had already failed once: at 484357a the two parser copies drifted into different spellings of the bullet loop while a docstring asserted they were identical. That drift was behaviour-neutral and was re-unified by hand at 598a7c3, so the copies were identical at merge time — but nothing had caught it, and the next drift need not be neutral. Implementation Notes Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching across skill directories, which is why two separate skills needed embedded copies; two files inside ONE skill may source a third. That is the whole reason the merge removes duplication rather than relocating it. The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path before any validation, and its table mirrors validate.sh's detection exactly: a directory holding SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory (agent); anything else stops without running a validator. Steps 1-3 live in references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the other artifact type, the body restarts at Step 0. Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that matches a live sibling reads as ownership rather than membership. The description carries one arrow per boundary target, because ADR-0020 resolves only the first target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated "audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16% of the body: no size SUGGESTIONs. The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both. tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources the resolver and that no third copy exists. The entry scripts classify the target from its resolved parent directory, so a bare agent filename typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is missing, rather than dying with exit 1, the tier the flows relay as real findings. The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's status (`f || RC=$?`) disables errexit for its entire body, and no subshell or `set -e` inside can re-arm it once the call sits in a condition context (measured, both spellings). Their error paths use `exit`, which is unaffected either way; this keeps errexit armed for anything added later. Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's dev environment is root -- so the guard could never fire where it exists to fire. A read attempt is also the stricter question, catching EIO. This is the reasoning scripts/check-vale-style-sync.sh carried before this commit deleted it; the hazard did not go with it. All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared, the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two remaining bare cd calls take absolute paths, which CDPATH is never consulted for. Impact BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to 2.0.0 (catalog 0.4.7). Check logic is unchanged: differential runs of the old and new validators across every skill and agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename. One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change, deliberately (full table in ADR-0025): - a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is by path shape, so a shape-matching path that is simply absent used to reach the validator and come back as a FAIL against a file that never existed; - an unshaped target exits 2 under the generic "matches neither" message, and a directory with no SKILL.md under a third, distinct one -- three exit-2 messages, not one; - a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding about the artifact rather than a usage error; - a SKILL.md file path is audited as its skill directory instead of refused; - a .md agent outside an agents/ directory is refused rather than audited; - a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2. validate-provenance.sh already exited 2 for the last two; validate.sh now matches it. .pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both files: regexes are unchanged; only entry: and description: moved. scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21), tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite. Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8. ADR: 0025 BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by factory-audit, which auto-detects whether it was handed a skill directory or an agent file. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD --- .claude-plugin/marketplace.json | 4 +- .pre-commit-config.yaml | 46 +- .pre-commit-hooks.yaml | 16 +- CONTEXT.md | 18 +- LESSONS.md | 4 +- README.md | 4 +- SIMPLIFICATION-AUDIT.md | 28 +- apm.yml | 8 +- .../0004-skill-audit-info-finding-level.md | 5 + ...0008-agent-audit-single-file-invocation.md | 6 + ...9-agent-audit-field-inventory-reference.md | 5 + ...nt-sources-relocated-outside-agents-dir.md | 3 + .../0012-agentsmd-tooling-in-core-plugin.md | 2 +- ...013-vale-harness-scope-and-rule-sources.md | 3 +- ...14-vale-prefilter-ships-from-the-plugin.md | 26 +- ...m-replaces-plugin-marketplace-authoring.md | 3 +- ...rimitive-drops-provider-specific-fields.md | 3 +- ...po-consumes-its-own-plugins-through-apm.md | 4 +- ...l-description-and-body-context-contract.md | 49 +- ...in-descriptions-state-a-domain-boundary.md | 3 +- ...nd-agent-audit-merge-into-factory-audit.md | 376 ++++ docs/spec/architecture.md | 2 +- docs/spec/gates.md | 237 ++- .../.apm/skills/agent-audit/SKILL.md | 90 - .../skills/agent-audit/references/sources.md | 90 - .../.apm/skills/agent-audit/scripts/README.md | 47 - .../skills/agent-audit/scripts/validate.sh | 1738 ----------------- .../.apm/skills/agent-audit/tests/README.md | 33 - .../.apm/skills/agent-author/SKILL.md | 10 +- .../.apm/skills/agent-author/assets/README.md | 2 +- .../assets/templates/apm-agent.md | 8 +- .../assets/templates/claude-code.md | 6 +- .../templates/copilot.agent.md.template | 6 +- .../agent-author/references/contract.md | 6 +- .../skills/agent-author/references/create.md | 2 +- .../references/deployment-modes.md | 2 +- .../skills/agent-author/references/improve.md | 8 +- .../agent-author/references/plugin-scope.md | 8 +- .../references/project-user-scope.md | 2 +- .../skills/agent-author/references/scripts.md | 2 +- .../skills/agent-author/scripts/new-agent.sh | 16 +- .../skills/agent-author/tests/new-agent.bats | 16 +- .../.apm/skills/factory-audit/SKILL.md | 77 + .../assets/vale/.vale.ini | 3 + .../styles/Kyberforge/CompositionNote.yml | 0 .../styles/Kyberforge/DescriptionOpener.yml | 0 .../vale/styles/Kyberforge/PaddingPhrase.yml | 0 .../Kyberforge/SentenceOpenerThereIs.yml | 0 .../vale/styles/Kyberforge/VagueWording.yml | 0 .../KyberforgeCopilot/ProactivePhrase.yml | 0 .../references/agent-body-and-delegation.md} | 4 +- .../references/agent-description-quality.md} | 2 +- .../references/agent-field-inventory.md} | 0 .../references/agent-finding-criteria.md} | 6 +- .../factory-audit/references/agent-flow.md | 64 + .../references/agent-scope-plugin-apm.md} | 6 +- .../references/agent-scope-project-user.md} | 6 +- .../references/agent-validation-scripts.md} | 8 +- .../references/skill-body-discipline.md} | 2 +- .../references/skill-description-quality.md} | 2 +- .../references/skill-file-structure.md} | 13 +- .../references/skill-finding-criteria.md} | 12 +- .../factory-audit/references/skill-flow.md | 62 + .../skill-formatting-and-scripts.md} | 2 +- .../references/skill-patterns.md} | 4 +- .../references/skill-validation-scripts.md} | 0 .../factory-audit/references/sources.md | 153 ++ .../scripts/lib-boundary-resolver.sh} | 689 +------ .../factory-audit/scripts/lib-checks-agent.sh | 683 +++++++ .../factory-audit/scripts/lib-checks-skill.sh | 621 ++++++ .../scripts/lib-contributing-files.sh | 134 ++ .../scripts/lib-provenance-agent.sh} | 286 ++- .../scripts/lib-provenance-skill.sh} | 318 ++- .../scripts/vale-wrap.sh | 13 +- .../scripts/validate-provenance.sh | 324 +++ .../skills/factory-audit/scripts/validate.sh | 255 +++ .../.apm/skills/factory-audit/tests/README.md | 94 + .../tests/validate-agent.bats} | 322 ++- .../tests/validate-provenance-agent.bats} | 244 ++- .../tests/validate-provenance-skill.bats} | 166 +- .../tests/validate-skill.bats} | 109 +- plugins/kyberforge/.apm/skills/forge/SKILL.md | 2 +- .../skills/forge/references/author-routes.md | 8 +- .../.apm/skills/skill-audit/SKILL.md | 87 - .../skills/skill-audit/assets/vale/.vale.ini | 4 - .../styles/Kyberforge/CompositionNote.yml | 13 - .../styles/Kyberforge/DescriptionOpener.yml | 7 - .../vale/styles/Kyberforge/PaddingPhrase.yml | 7 - .../Kyberforge/SentenceOpenerThereIs.yml | 7 - .../vale/styles/Kyberforge/VagueWording.yml | 10 - .../skills/skill-audit/references/sources.md | 59 - .../skills/skill-audit/scripts/vale-wrap.sh | 526 ----- .../.apm/skills/skill-audit/tests/README.md | 29 - .../.apm/skills/skill-author/SKILL.md | 10 +- .../skill-author/references/contract.md | 4 +- .../skills/skill-author/references/create.md | 6 +- .../references/deployment-modes.md | 2 +- .../skills/skill-author/references/improve.md | 4 +- .../skills/skill-author/scripts/new-skill.sh | 2 +- .../skills/skill-author/tests/new-skill.bats | 4 +- plugins/kyberforge/README.md | 3 +- plugins/kyberforge/apm.yml | 2 +- plugins/kyberforge/docs/README.md | 2 +- scripts/check-apm-agents-valid.sh | 22 +- scripts/check-scope-walkup-sync.sh | 48 +- scripts/check-vale-style-sync.sh | 413 ---- scripts/skill-size-check.sh | 65 +- scripts/sync-vale-styles.sh | 21 - tests/run-tests.sh | 12 +- tests/test-adr0020-body-checks.sh | 4 +- tests/test-adr0020-contract.sh | 452 ++++- tests/test-adr0020-differential.sh | 18 +- tests/test-adr0020-frontmatter.sh | 38 +- tests/test-check-apm-agents-valid.sh | 38 +- tests/test-check-scope-walkup-sync.sh | 35 +- tests/test-check-vale-style-sync.sh | 797 -------- tests/test-skill-size-check.sh | 48 +- tests/test-vale-hooks-consumer.sh | 16 +- tests/test-vale-wrap.sh | 1339 ++++++++++++- 119 files changed, 6308 insertions(+), 5487 deletions(-) create mode 100644 docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md delete mode 100644 plugins/kyberforge/.apm/skills/agent-audit/SKILL.md delete mode 100644 plugins/kyberforge/.apm/skills/agent-audit/references/sources.md delete mode 100644 plugins/kyberforge/.apm/skills/agent-audit/scripts/README.md delete mode 100755 plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh delete mode 100644 plugins/kyberforge/.apm/skills/agent-audit/tests/README.md create mode 100644 plugins/kyberforge/.apm/skills/factory-audit/SKILL.md rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/.vale.ini (75%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/Kyberforge/CompositionNote.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/Kyberforge/DescriptionOpener.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/Kyberforge/PaddingPhrase.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/Kyberforge/VagueWording.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml (100%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/body-and-delegation.md => factory-audit/references/agent-body-and-delegation.md} (98%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/description-quality.md => factory-audit/references/agent-description-quality.md} (99%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/field-inventory.md => factory-audit/references/agent-field-inventory.md} (100%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/finding-criteria.md => factory-audit/references/agent-finding-criteria.md} (96%) create mode 100644 plugins/kyberforge/.apm/skills/factory-audit/references/agent-flow.md rename plugins/kyberforge/.apm/skills/{agent-audit/references/scope-plugin-apm.md => factory-audit/references/agent-scope-plugin-apm.md} (94%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/scope-project-user.md => factory-audit/references/agent-scope-project-user.md} (90%) rename plugins/kyberforge/.apm/skills/{agent-audit/references/validation-scripts.md => factory-audit/references/agent-validation-scripts.md} (93%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/body-discipline.md => factory-audit/references/skill-body-discipline.md} (99%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/description-quality.md => factory-audit/references/skill-description-quality.md} (99%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/file-structure.md => factory-audit/references/skill-file-structure.md} (85%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/finding-criteria.md => factory-audit/references/skill-finding-criteria.md} (94%) create mode 100644 plugins/kyberforge/.apm/skills/factory-audit/references/skill-flow.md rename plugins/kyberforge/.apm/skills/{skill-audit/references/formatting-and-scripts.md => factory-audit/references/skill-formatting-and-scripts.md} (98%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/patterns.md => factory-audit/references/skill-patterns.md} (96%) rename plugins/kyberforge/.apm/skills/{skill-audit/references/validation-scripts.md => factory-audit/references/skill-validation-scripts.md} (100%) create mode 100644 plugins/kyberforge/.apm/skills/factory-audit/references/sources.md rename plugins/kyberforge/.apm/skills/{skill-audit/scripts/validate.sh => factory-audit/scripts/lib-boundary-resolver.sh} (63%) create mode 100755 plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh create mode 100755 plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh create mode 100755 plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-contributing-files.sh rename plugins/kyberforge/.apm/skills/{agent-audit/scripts/validate-provenance.sh => factory-audit/scripts/lib-provenance-agent.sh} (73%) rename plugins/kyberforge/.apm/skills/{skill-audit/scripts/validate-provenance.sh => factory-audit/scripts/lib-provenance-skill.sh} (86%) rename plugins/kyberforge/.apm/skills/{agent-audit => factory-audit}/scripts/vale-wrap.sh (96%) create mode 100755 plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh create mode 100755 plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh create mode 100644 plugins/kyberforge/.apm/skills/factory-audit/tests/README.md rename plugins/kyberforge/.apm/skills/{agent-audit/tests/validate.bats => factory-audit/tests/validate-agent.bats} (67%) rename plugins/kyberforge/.apm/skills/{agent-audit/tests/validate-provenance.bats => factory-audit/tests/validate-provenance-agent.bats} (72%) rename plugins/kyberforge/.apm/skills/{skill-audit/tests/validate-provenance.bats => factory-audit/tests/validate-provenance-skill.bats} (87%) rename plugins/kyberforge/.apm/skills/{skill-audit/tests/validate.bats => factory-audit/tests/validate-skill.bats} (86%) mode change 100755 => 100644 delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/SKILL.md delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/CompositionNote.yml delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/VagueWording.yml delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/references/sources.md delete mode 100755 plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh delete mode 100644 plugins/kyberforge/.apm/skills/skill-audit/tests/README.md delete mode 100755 scripts/check-vale-style-sync.sh delete mode 100755 scripts/sync-vale-styles.sh delete mode 100755 tests/test-check-vale-style-sync.sh diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index c8211b6..f9aa711 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -1,7 +1,7 @@ { "name": "holocron", "description": "AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows.", - "version": "0.4.6", + "version": "0.4.7", "owner": { "name": "Defame1297", "email": "defame1297@rkdr.net", @@ -11,7 +11,7 @@ { "name": "kyberforge", "description": "Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace.", - "version": "1.6.2", + "version": "2.0.0", "category": "Developer Tools", "source": "./plugins/kyberforge" }, diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 55185a2..2d58a43 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -138,7 +138,7 @@ repos: - id: check-apm-agents-valid name: Validate real APM agent files - description: Run agent-audit's validate.sh over every plugins/*/.apm/agents/*.agent.md file in this repo -- the artifacts it governs, not fixtures + description: Run factory-audit's validate.sh over every plugins/*/.apm/agents/*.agent.md file in this repo -- the artifacts it governs, not fixtures entry: bash scripts/check-apm-agents-valid.sh language: system stages: [pre-push] @@ -163,24 +163,16 @@ repos: pass_filenames: false always_run: true - - id: check-vale-style-sync - name: Check Vale style copies are in sync - description: Diff skill-audit's Vale copy against agent-audit's canonical copy - entry: bash scripts/check-vale-style-sync.sh - language: system - stages: [pre-push] - pass_filenames: false - always_run: true - # verbose so the DOWNGRADED run is audible. This hook can pass while - # having verified strictly less than its name claims: - # CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 skips all six glob probes - # and says so on a `passed (text-level only, vale unavailable)` line. - # pre-commit prints nothing at all for a passing hook, so without this - # the opt-out reinstated exactly the silent vacuous pass the script was - # written to kill, one level up -- the run showed a bare `Passed` and - # the documented instruction to read that summary line was impossible to - # follow in the one situation the opt-out exists for. The script's clean - # output is a single line, so this costs one line per push. + # check-vale-style-sync was removed by ADR-0025. Only 6 of its 17 + # assertions diffed skill-audit's Vale copy against agent-audit's; the + # merge into factory-audit leaves one copy, so those are moot. The other + # 11 moved into tests/test-vale-wrap.sh (case 0, cases 28-31, its + # Vale-absent skip, and case 33 for cross-manifest files: agreement), + # which run-tests runs here at + # pre-push, so do not re-add the hook to restore coverage. Do not + # confuse its removal with check-scope-walkup-sync below, which survives: + # that one cross-checks four hand-ported walk-up implementations, only two + # of which lived in the audit pair. - id: check-scope-walkup-sync name: Check scope walk-up implementations agree @@ -236,7 +228,7 @@ repos: files: '^plugins/[^/]+/\.apm/(skills/.*\.md|agents/.*\.agent\.md)$' # README.md is excluded on purpose, not by oversight. A skill-directory # README is consumer-facing prose that no agent ever loads, and the - # `git clone` lines in the seven tests/README.md files are setup + # `git clone` lines in the six tests/README.md files are setup # instructions for a third party who has no rtk installed. Prefixing # those would be actively wrong -- see ADR-0023's consumer section. exclude: '(^|/)README\.md$' @@ -245,17 +237,23 @@ repos: - id: vale-audit-prefilter-skill stages: ['pre-commit'] name: Vale audit prefilter (SKILL.md) - description: Run Vale against SKILL.md files as a deterministic prefilter for skill-audit, via skill-audit's own bundled copy - entry: plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh + description: Run Vale against SKILL.md files as a deterministic prefilter for factory-audit's skill flow, via factory-audit's own bundled copy + entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh language: script files: '^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$' pass_filenames: true + # Two hook IDs pointing at ONE vale-wrap.sh is deliberate, not leftover + # duplication. ADR-0014 measured a single hook entry silently scanning 0 + # files of the other type, and ADR-0025 carried that finding across the + # merge: the two `files:` regexes are what keep the SKILL.md scope and the + # agent-file scope independently addressable. The script self-locates its + # config via ${BASH_SOURCE[0]}, so one copy serves both. - id: vale-audit-prefilter-agent stages: ['pre-commit'] name: Vale audit prefilter (agent files) - description: Run Vale against agent markdown files as a deterministic prefilter for agent-audit, via agent-audit's own bundled copy - entry: plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh + description: Run Vale against agent markdown files as a deterministic prefilter for factory-audit's agent flow, via factory-audit's own bundled copy + entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh language: script files: '^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$' pass_filenames: true diff --git a/.pre-commit-hooks.yaml b/.pre-commit-hooks.yaml index 9031b65..298b187 100644 --- a/.pre-commit-hooks.yaml +++ b/.pre-commit-hooks.yaml @@ -1,14 +1,22 @@ +# PUBLISHED CONTRACT. External repos consume these IDs with `rev: `, so an +# ID or a `files:` regex here may not change without breaking them on upgrade. +# ADR-0025 merged skill-audit and agent-audit into factory-audit and re-pointed +# both `entry:` paths at its single vale-wrap.sh; both IDs and both regexes are +# unchanged, deliberately. Collapsing them into one was considered and rejected: +# it breaks every consumer pinning kyberforge-vale-audit-agent, and it re-creates +# ADR-0014's measured failure where one hook against one config silently scanned +# 0 files of the other type. Two IDs are what keep both file scopes addressable. - id: kyberforge-vale-audit-skill name: Kyberforge Vale prose audit (SKILL.md) - description: Deterministic prose-pattern prefilter for kyberforge's skill-audit, via its own bundled Vale config/styles - entry: plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh + description: Deterministic prose-pattern prefilter for kyberforge's factory-audit skill flow, via its own bundled Vale config/styles + entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh language: script files: '(^|/)SKILL\.md$' - id: kyberforge-vale-audit-agent name: Kyberforge Vale prose audit (agent files) - description: Deterministic prose-pattern prefilter for kyberforge's agent-audit, via its own bundled Vale config/styles - entry: plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh + description: Deterministic prose-pattern prefilter for kyberforge's factory-audit agent flow, via its own bundled Vale config/styles + entry: plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh language: script files: '(^|/)agents/[^/]+\.md$|\.agent\.md$' diff --git a/CONTEXT.md b/CONTEXT.md index 60b411f..58075d2 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -77,7 +77,7 @@ _Avoid_: the marketplace, upstream **Provenance chain**: The three-stage traceability record linking a skill back to its research inputs: `/research` produces topic docs and a `sources.md`; the author skill records which sources informed which files in -`references/sources.md` and `source_keys` frontmatter; `skill-audit` validates the chain is complete +`references/sources.md` and `source_keys` frontmatter; `factory-audit` validates the chain is complete and internally consistent. _Avoid_: sources, citations, attribution @@ -122,7 +122,7 @@ _Avoid_: repo root, project root **Near-miss**: A query that shares keywords with this skill but needs a different one — and, by extension, the sibling that would wrongly answer it; boundary clauses exist to exclude genuine near-misses rather -than to enumerate siblings. Detail: `skill-audit/references/description-quality.md`. +than to enumerate siblings. Detail: `factory-audit/references/skill-description-quality.md`. _Avoid_: overlap, similar skill **Issue**: @@ -130,6 +130,12 @@ The cross-provider term for a tracked unit of work. Gitea is this repo's canonic (ADR-0007), but skills say "linked issue" generically rather than naming a provider. _Avoid_: ticket, card, task +**Family prefix**: +The shared first segment of a skill name (`git-`, `gitea-`, `apm-`, `agentsmd-`) marking a group of +siblings. No bare skill name is a **Family prefix** of another: `forge` exists, so no skill is named +`forge-*`, because a prefix that matches a live sibling reads as ownership rather than membership. +_Avoid_: namespace, category + ## Relationships - An **apm package** bundles one or more **Skills** and agents; a **Plugin marketplace** lists @@ -140,7 +146,7 @@ _Avoid_: ticket, card, task type to the matching author skill — an already-specified fix (file, line, and change known) calls that author skill directly, because each routing hop re-derives instructions from a shorter brief and has been observed to drop hard constraints handed down the chain. -- A **Skill** built on research carries a **Provenance chain**; `skill-audit` fails it when broken. +- A **Skill** built on research carries a **Provenance chain**; `factory-audit` fails it when broken. - **LESSONS.md** feeds the standing files: three or more entries on one pattern graduate the pattern into the relevant standing document. @@ -175,6 +181,8 @@ _Avoid_: ticket, card, task the compounds keep the word and are not being renamed. - "context" means both the model's live token window and the bounded domain this file describes — resolved: unqualified "context" in this repo means the token window. -- "audit" was used for both an author skill's inline closeout and `forge`'s independent +- "audit" was used for both an author skill's inline closeout and the audit skill's independent clean-context recheck — resolved: these are two distinct layers, kept separate precisely because - an audit running in the same context as the work it checks shares that work's blind spots. + an audit running in the same context as the work it checks shares that work's blind spots. The + recheck belongs to `factory-audit`, not to `forge`, which routes only to the author + skills and never to an audit. diff --git a/LESSONS.md b/LESSONS.md index e8c114c..265097a 100644 --- a/LESSONS.md +++ b/LESSONS.md @@ -54,7 +54,7 @@ Skills sharing a resource (e.g. `validate.sh`) via a `shared/` directory and rel ## 2026-06-22 — Qualitative rubrics should be grounded in upstream spec docs, not in-repo usage -`skill-audit`'s description and body-discipline rubrics were derived from `skill-write`'s own conventions — circular, so drift in one silently propagated to the other. Fix: extract condensed reference files directly from the upstream spec (agentskills.io) into the audit skill, so the rubric is independent of in-repo convention drift. +`skill-audit`'s (now `factory-audit`'s skill flow, per ADR-0025: `references/skill-description-quality.md` and `references/skill-body-discipline.md`) description and body-discipline rubrics were derived from `skill-write`'s own conventions — circular, so drift in one silently propagated to the other. Fix: extract condensed reference files directly from the upstream spec (agentskills.io) into the audit skill, so the rubric is independent of in-repo convention drift. ## 2026-06-22 — Test files in scripts/ are dev tooling; document them in README as non-spec @@ -62,7 +62,7 @@ The agentskills.io spec defines `scripts/` for bundled executables, not test inf ## 2026-06-27 — Clean-context audit catches what biased forks miss -A fresh-context skill-audit caught two FAILs (an incomplete README table, invalid cache paths) that the implementing fork's own audit missed — the fork that built the artifact knows what was intended and fills gaps silently. Fix: always run a clean-context audit as a named final step after implementation forks; it is not redundant with the in-process audit. +A fresh-context skill-audit (now `factory-audit`, per ADR-0025) caught two FAILs (an incomplete README table, invalid cache paths) that the implementing fork's own audit missed — the fork that built the artifact knows what was intended and fills gaps silently. Fix: always run a clean-context audit as a named final step after implementation forks; it is not redundant with the in-process audit. ## 2026-06-27 — Parallel forks on the same file produce conflicts requiring a third fork to reconcile diff --git a/README.md b/README.md index 14491b7..348aa9e 100644 --- a/README.md +++ b/README.md @@ -32,13 +32,13 @@ Install all of these before setting up. Each one is a hard dependency of a git h | --- | --- | --- | | `apm` CLI | Two pre-push hooks shell out to it (`apm-audit-ci` and `apm-pack-check-clean`) | The `apm-install` skill, or `curl -sSL https://aka.ms/apm-unix \| sh`. Verify with `apm --version` | | `python3` + PyYAML | Required by `scripts/skill-size-check.sh` (the `skill-size-check` pre-commit hook), which reads folded YAML frontmatter | `python3` is usually present — pre-commit is itself a Python application. `pip install pyyaml` if the hook reports PyYAML missing | -| `vale` | Required by the `vale-audit-prefilter-skill` / `-agent` pre-commit hooks and the `check-vale-style-sync` pre-push hook | `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or https://vale.sh/docs/vale-cli/installation/ | +| `vale` | Required by the `vale-audit-prefilter-skill` / `-agent` pre-commit hooks, and by the `test-vale-wrap.sh` / `test-vale-hooks-consumer.sh` suites that `run-tests --strict` runs at pre-push | `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or https://vale.sh/docs/vale-cli/installation/ | | `claude` CLI | Required by the `validate-marketplace` pre-push hook | Claude Code | Two notes worth reading before you skip one: - **PyYAML is a hard requirement, not an optional accelerator.** The hand-rolled fallback frontmatter reader was removed deliberately: a reader that mis-parses an unfamiliar scalar shape reports a clean pass on a file it never measured. -- **No `vale sync` is needed.** The `Kyberforge` styles are committed under `plugins/kyberforge/.apm/skills/{skill-audit,agent-audit}/assets/vale/styles/`, not downloaded packages (ADR-0014). +- **No `vale sync` is needed.** The `Kyberforge` styles are committed under `plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/`, not downloaded packages (ADR-0014, ADR-0025). ## Setup diff --git a/SIMPLIFICATION-AUDIT.md b/SIMPLIFICATION-AUDIT.md index 40cd506..056ede5 100644 --- a/SIMPLIFICATION-AUDIT.md +++ b/SIMPLIFICATION-AUDIT.md @@ -76,13 +76,15 @@ Where the 276 s goes (each suite run alone, sequential): | ~~`test-sync-plugin-content.sh`~~ deleted | ~~83 s~~ | 14 temp trees, 2 `git init`, repeated `apm pack` | | all 351 bats tests (10 files, kyberforge and core validators) | 64 s | mostly `validate.sh` / `validate-provenance.sh` fixtures | | `test-adr0020-differential.sh` | 29 s | 12 assertions; re-runs two validators over the live corpus and a fixture tree | -| `test-check-vale-style-sync.sh` | 25 s | guards a byte-identical copy | +| ~~`test-check-vale-style-sync.sh`~~ deleted | ~~25 s~~ | guards a byte-identical copy | | `test-vale-wrap.sh` | 14 s | | | `test-adr0020-frontmatter.sh` + `-targets.sh` | 25 s | | | Remaining 20 suites | 36 s | 12 of them run in under 2 s each | Five suites account for 215 s of 276 s. Three of those five (sync-plugin-content, vale-style-sync, adr0020-differential) test tooling that findings 2, 7, and 14 propose to delete or shrink, so the fastest path to a quick pre-push is removing the duplication those tests guard rather than optimising the tests. +> **Also struck (2026-09-15):** `test-check-vale-style-sync.sh` (25 s) went with the `check-vale-style-sync` hook in finding 14's merge (ADR-0025). Measured at HEAD: `tests/` holds **19** `test-*.sh` suites and `tests/run-tests.sh` reports `19 passed, 0 skipped, 0 failed`. Same basis as the note below — arithmetic on the 2026-09-10 baseline minus the struck rows, not a fresh timing run. + > **Done (2026-09-14):** see commit `718c79a` on `docs/simplification-audit`. The three struck-through rows are gone: `test-sync-plugin-content.sh` (83 s, 1,289 lines, 92 cases), `check-plugin-content-sync` (4.5 s) and `validate-plugins` (4.9 s). Expected, not re-measured: roughly 92 s comes off every push (83 + 4.5 + 4.9 = 92.4 s) (~83 s of it out of `run-tests`, which loses its single slowest suite), on the arithmetic of the 2026-09-10 figures alone. The remaining rows have not been re-timed since, so treat every number in this section as the 2026-09-10 baseline minus those three, not as a fresh measurement. ## 3. Enforcement layer: hooks, tests, scripts @@ -95,7 +97,7 @@ This is the area you named as hardest to understand and slowest. Root cause: mos > **Corrected and closed (2026-09-14, at `a6434e0`):** two things above went stale within hours of being written, and the finding was never given a marker. > > - **"Keep the two `claude plugin validate` hooks"** is void. `718c79a` (ADR-0024) deleted `validate-plugins` — the ADR's own reasoning is that `claude plugin validate` reads manifests only and could never detect the empty-content defect it was credited with guarding, and with the per-plugin manifests gone it has nothing left to read. Only **`validate-marketplace`** survives, over the one manifest this repo still ships (`.claude-plugin/marketplace.json`). Of the six hooks this finding named, three now exist: `validate-marketplace`, `apm-pack-check-clean`, `apm-audit-ci`. Verified against `.pre-commit-config.yaml`: 27 `- id:` entries, 9 with `stages: [pre-push]`, no `validate-plugins` entry. - > - **The gates.md figures above ("13→11 self-authored / 15→13 total") were correct for `0dffff3` and are no longer current.** `718c79a` removed two more pre-push hooks after that commit, and `docs/spec/gates.md:24` now reads **11 reported / 9 self-authored**. Read the count from that file, not from this note. + > - **The gates.md figures above ("13→11 self-authored / 15→13 total") were correct for `0dffff3` and are no longer current.** `718c79a` removed two more pre-push hooks after that commit, and `docs/spec/gates.md:24` read **11 reported / 9 self-authored** when this note was written; finding 14's merge has since removed `check-vale-style-sync`, and it now reads **10 reported / 8 self-authored**. Read the count from that file, not from this note. > > Marked `[x]`: all three of this finding's decisions are resolved — `check-manifests` deleted (`e647f14`), `apm-audit-ci` kept on the grill above, `apm-marketplace-check` removed (`0dffff3`). @@ -104,6 +106,7 @@ This is the area you named as hardest to understand and slowest. Root cause: mos - `check-vale-style-sync`: 413 lines + 798 test lines guarding a byte-identical 526-line `vale-wrap.sh` and style directory copied between skill-audit and agent-audit. About 350 of its lines run Vale glob probes against the hook file patterns. Disappears if the two audit skills merge (finding 14); the probes belong in `test-vale-wrap.sh`. - `check-scope-walkup-sync`: 365 lines cross-checking four independent ports of the same package-root walk-up. Disappears if the ports share one script or the skills merge. > **Grilled, held (2026-09-14):** both of the above are gated on findings 14/15 (merging skill-audit+agent-audit and skill-author+agent-author), deliberately held for a separate session rather than decided here. Correction for that session: the audit's §8 grouping is wrong — these merges don't need ADR-0012 revisited (that ADR governs the unrelated `core` plugin's three `agentsmd-*` skills). The actual constraint is ADR-0014 (no-cross-skill file sharing on plugin cache-install), and merging sidesteps it rather than requiring it be reversed. The open question for that session is a design one — a shared skill's `description` carrying both skill- and agent-audit trigger phrases — not an ADR supersession. ADR-0012 revisit is needed only for finding 24. + > **Settled (2026-09-15) — split verdict, and the first bullet held in full.** Finding 14 landed as `factory-audit` (ADR-0025). **`check-vale-style-sync` is deleted**, hook, script and test, exactly as the first bullet predicted — and its probes **were** rehomed into `test-vale-wrap.sh`, as cases 28-30 (case 31 carries the override allowlist), so both halves of that bullet are closed. `docs/spec/gates.md` records the rehoming, not an open gap. *(Updated later on 2026-09-15.)* The one assertion this note used to call still uncovered — cross-manifest *agreement* between `.pre-commit-hooks.yaml`'s and `.pre-commit-config.yaml`'s `files:` regexes — is now ported as case 33, which pairs the hooks by `id:`. Case 32 covers the separate zero-match question. It was a real gap while it lasted: narrowing the local skill hook to `^plugins/kyberforge/` left 6 of 38 skills prefiltered and the suite green. `bash tests/test-vale-wrap.sh` now reports `61 passed, 0 failed` (it was 56 before cases 0 and 33 and the Part B mutation self-tests). The bullet's "about 350 of its lines run Vale glob probes" overstates the probe half: at `a5962ba` the script is **413 lines**, of which the `.vale.ini` coverage section is **332** (`67..398`) and the machinery that actually invokes vale against a probe path is **204** (`195..398`). The balance of that section is `StylesPath`, `BasedOnStyles` and per-rule-override greps — text assertions, not probes. (Its test file is **797** lines, as the note above says, not the 798 the bullet carries.) **`check-scope-walkup-sync` stays**, and the second bullet's "or the skills merge" is wrong: two of its four walk-up ports are in the *author* skills (`new-agent.sh`, `new-skill.sh`), which this merge does not touch, and the audit-side pair is Python against the author-side pair's Bash, so the gate can never degrade into a text diff. Full reasoning in §10's 2026-09-15 note. Finding 15 would not remove it either. - [x] ~~`check-marketplace-mirror-sync`: guards `.github/plugin/marketplace.json`. The script header calls it Copilot's legacy convention path and says Copilot also accepts the Claude path; the vendored Copilot docs list it as primary. Verify against current Copilot CLI before deleting hook, script, test, and mirror file.~~ > **Grilled and done (2026-09-14):** verified against GitHub's current Copilot CLI plugin docs (not the vendored copy, which risked drift). Copilot CLI's marketplace discovery checks paths in order — `marketplace.json`, `.plugin/marketplace.json`, `.github/plugin/marketplace.json`, `.claude-plugin/marketplace.json` — falling through to whichever exists first. `.claude-plugin/marketplace.json` (apm's own `claude` output) already satisfies that chain's last step, so the dedicated `.github/plugin/marketplace.json` mirror bought Copilot users its *preferred* discovery path rather than a required one. Decided against reopening ADR-0018 (native install for both Claude Code and Copilot CLI stays supported) to justify this — the deletion holds either way, since Copilot's own fallback covers it. Deleted `.github/plugin/marketplace.json`, `scripts/sync-marketplace-mirror.sh` (81 lines), `tests/test-sync-marketplace-mirror.sh` (304 lines), and the `check-marketplace-mirror-sync` pre-push hook; removed the dangling references to the deleted script in `scripts/sync-plugin-content.sh` and `tests/test-sync-plugin-content.sh` (both had comments citing its reasoning by name), and updated `docs/spec/architecture.md`'s description of the marketplace-manifest compile step. `tests/test-sync-plugin-content.sh` (92 cases) still passes in full. > @@ -135,6 +138,8 @@ This is the area you named as hardest to understand and slowest. Root cause: mos > **Corrected (2026-09-14):** two of the six named targets no longer exist — `validate-plugins` and `check-plugin-content-sync` were deleted in commit `718c79a` (finding 7, ADR-0024). Actual state today: **9 repo-authored pre-push hooks** — `run-tests`, `check-executables-allow-sync`, `apm-audit-ci`, `check-apm-agents-valid`, `apm-pack-check-clean`, `check-vale-style-sync`, `check-scope-walkup-sync`, `check-release-needed`, `validate-marketplace` — plus the 2 pre-commit `meta` hooks that also run at this stage, so 11 are reported at pre-push. `validate-marketplace` was kept: the root `marketplace:` block in `apm.yml` and the root `.claude-plugin/marketplace.json` stay, because apm's own marketplace consumers read that same catalogue and `@holocron` short names depend on it. (That manifest is the only tracked file under `.claude-plugin/` — `git ls-files .claude-plugin` returns it alone. The sibling `.claude-plugin/plugin.json` is a local `apm pack` byproduct, has never been tracked on any branch, and is ignored at `.gitignore:59`; it was not "kept", because it was never there.) +> **Superseded count (2026-09-15):** finding 14 deleted `check-vale-style-sync` with the merge into `factory-audit` (ADR-0025), so pre-push is now **8 repo-authored hooks** (10 reported). The dated note above is the state on 2026-09-14; see finding 14's note for the correction. + Pre-commit stays roughly as is minus `skill-frontmatter`, and minus `check-ast` once finding 9 removes the only `.py` files. ~~Tests 26 files to about 10 (12,400 to about 5,000 lines).~~ Keep bats and its three submodules; the 351 bats tests ship inside plugins and are the right tool there. Do not port the bash suites to bats; delete them instead. > **Re-measured (2026-09-14, at `a6434e0`):** the tests target was stated against the 2026-09-10 baseline and both its numbers are stale. `tests/` now holds **20 `test-*.sh` suites totalling 9,123 lines** (plus the two runners, 490). Six suites have gone since the baseline: `test-check-manifests.sh` (`e647f14`), `test-skill-frontmatter.sh` (`c8a7c9e`), `test-governance-layer.sh` and `test-instructions-and-docs.sh` (`5f9f2b3`), `test-sync-marketplace-mirror.sh` (`0dffff3`), `test-sync-plugin-content.sh` (`718c79a`). Restated on the same basis the target is **20 files to about 10, 9,123 to about 5,000 lines** — the file half of the target is now the closer half, and finding 9's `check-ast` clause is moot anyway, since finding 9 is not proceeding. @@ -156,7 +161,7 @@ The shared pattern: per-skill `README.md` files no model reads, a `docs/research > > Corrected figures: **46 `sources.md` files / 1,752 lines** in three distinct classes — 29 skill `references/sources.md` (1,217 lines), 13 research indexes (435), 4 plugin-root files (100, ADR-0010). The finding does **not** double-count: it states two disjoint classes additively ("32 plugin and skill `sources.md` files (about 1,300 lines) **plus** 9 research indexes"), and that plugin-and-skill subtotal is really **33 files / 1,317 lines**, matching its "about 1,300" exactly — had the 32 swept in the research indexes the figure would have been ~1,750. Its real errors there are an off-by-one (32 should be 33) and an omission: it missed the 4 vendored example indexes under `kyberforge/docs/research/examples/skill-write/`, so 9 should be 13. Carriers of `source_keys` in YAML frontmatter: **196** — 168 at column 0 and 28 nested two spaces under `metadata:` — so the finding's 216 is closer to the truth than it looks. (219 files merely *mention* the string. A naive `^[[:space:]]*source_keys:` grep returns 200, but 4 of those are heredoc or fixture text rather than frontmatter: both `validate-provenance.bats` copies, `scripts/check-scope-walkup-sync.sh`, and a fenced example in `plugins/bin/.apm/skills/research/references/file-format.md`.) Checks: **16 across the two copies** (skill-audit 0–9, agent-audit 0–5), not ten. Validator line counts (1,198 / 632) and 125 bats tests are exact. > - > **"Touches every skill's frontmatter" is roughly right.** **28 of the 39 real skills carry `source_keys` in frontmatter**, nested under `metadata:` — see `plugins/git/.apm/skills/git-commits/SKILL.md:10-17`, where `metadata:` → `source_keys:` carries four slugs. (44 tracked files match `*SKILL.md`; subtract `skill-author/assets/templates/SKILL.md` and the 4 vendored under `kyberforge/docs/research/examples/skill-write/`, leaving 39 real skills.) The 11 without it are exactly the `plugins/bin/` skills. Check 2 in the skill-audit validator (SKILL.md `source_keys` → slug in `sources.md`) is correspondingly **live**, not dead code: `parse_source_keys()` at `plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh:242-270` handles both spellings explicitly — the metadata-nested branch at `:257`, the top-level branch at `:260`, and a docstring that says "handles metadata.source_keys and top-level" — check 2 at `:766` runs against all 28 carrier skills, every one of which has a `references/sources.md`, and bats pins it at `validate-provenance.bats:222` ("FAIL: source_keys slug in SKILL.md not present as H2 in sources.md") and `:1313` (a BOM must not silently disable check 2). The imbalance the finding names is real and **worse** than claimed: 4,641 validator+bats lines against 1,752 of metadata, a 2.6:1 ratio. + > **"Touches every skill's frontmatter" is roughly right.** **28 of the 39 real skills carry `source_keys` in frontmatter**, nested under `metadata:` — see `plugins/git/.apm/skills/git-commits/SKILL.md:10-17`, where `metadata:` → `source_keys:` carries four slugs. (44 tracked files match `*SKILL.md`; subtract `skill-author/assets/templates/SKILL.md` and the 4 vendored under `kyberforge/docs/research/examples/skill-write/`, leaving 39 real skills.) The 11 without it are exactly the `plugins/bin/` skills. Check 2 in the skill-side validator (SKILL.md `source_keys` → slug in `sources.md`) is correspondingly **live**, not dead code: `parse_source_keys()` at `plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh:277-305` handles both spellings explicitly — the metadata-nested branch at `:292`, the top-level branch at `:295`, and a docstring that says "handles metadata.source_keys and top-level" — check 2 at `:712` runs against all 28 carrier skills, every one of which has a `references/sources.md`, and bats pins it at `plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats:222` ("FAIL: source_keys slug in SKILL.md not present as H2 in sources.md") and `:1337` (a BOM must not silently disable check 2). (Paths and line numbers re-derived at HEAD: ADR-0025's merge moved this code out of `skill-audit/scripts/validate-provenance.sh` into the shared skill-side library, so the figures this note carried at `062ca47` — `:242-270`, `:257`, `:260`, `:766`, `:1313` — no longer resolve.) The imbalance the finding names is real and **worse** than claimed: 4,641 validator+bats lines against 1,752 of metadata, a 2.6:1 ratio. > > **Omitted entirely: the chain has a producer.** `plugins/bin/.apm/skills/research/` *specifies* the `sources.md` + `source_keys:` output format, and `plugins/bin/evals/research/research/eval.yaml` carries three criteria asserting it. **This is the blocking decision: does `research` keep emitting `sources.md`?** If yes, the chain is not dropped — only unenforced, and the finding collapses to "delete the validators." If no, the research skill's output contract and its evals need redesigning. > @@ -172,7 +177,14 @@ The shared pattern: per-skill `README.md` files no model reads, a `docs/research ### 4.2 kyberforge (290 files, 44,568 lines incl. mirror; the 7 SKILL.md bodies are 333 lines, under 1%) -14. **Merge `skill-audit` + `agent-audit` into one `audit` skill (removes about 3,300 lines and two pre-push hooks).** `vale-wrap.sh` is byte-identical in both; five Vale rules byte-identical (agent-audit carries one extra, so it is the superset); `validate.sh` shares a 1,061-line boundary-target resolver block that diffs as zero lines; SKILL.md steps 1, 3, 4 and the gotchas are the same text. Each copy is hard-wired to one mode, so the merged script needs a path switch. The duplication exists because a plugin-cache install copies only each skill's own files (the rule ADR-0014 follows), so a script cannot be shared across skills; merging the skills is the only way to remove the copy. Effort M. +14. [x] ~~**Merge `skill-audit` + `agent-audit` into one `audit` skill (removes about 3,300 lines and two pre-push hooks).** `vale-wrap.sh` is byte-identical in both; five Vale rules byte-identical (agent-audit carries one extra, so it is the superset); `validate.sh` shares a 1,061-line boundary-target resolver block that diffs as zero lines; SKILL.md steps 1, 3, 4 and the gotchas are the same text. Each copy is hard-wired to one mode, so the merged script needs a path switch. The duplication exists because a plugin-cache install copies only each skill's own files (the rule ADR-0014 follows), so a script cannot be shared across skills; merging the skills is the only way to remove the copy. Effort M.~~ + > **Done (2026-09-15), with three of its claims corrected.** Merged into **`factory-audit`**, not `audit` — the name states the domain (the artifact factory's own output) rather than the verb. See `docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md`. The repo goes from 39 skills to 38. Entry scripts are `scripts/validate.sh`, `scripts/validate-provenance.sh` and `scripts/vale-wrap.sh`. Only the first two auto-detect the artifact type they were handed and dispatch to a per-type library. `vale-wrap.sh` does not and never did: it is byte-identical to both pre-merge copies (`diff` clean against each at `a5962ba`) and names neither `SKILL.md` nor `.agent.md` anywhere in its 526 lines. Its scoping comes from outside it — the `.vale.ini` glob sections and the `files:` regexes of the two hooks that call it. + > + > - **Yield: 2,934 lines and ONE pre-push hook, not ~3,300 and two.** The hook is `check-vale-style-sync`, deleted with its script (413 lines) and `tests/test-check-vale-style-sync.sh` (797). **They did *not* exist only to diff the two now-merged Vale copies** — an earlier revision of this bullet said so and it was wrong, as this document's own ":104" measurement already implied. The script has **17** assertion sites (13 `err` calls and 4 hard-fail exits; ADR-0025 maps each one). Only **6** are genuinely moot: two diffed the copies and four guarded the script's ability to locate them. **10** were rehomed into `tests/test-vale-wrap.sh`: case 0 (config loads), cases 28–30 (glob probes, style loading, Copilot scoping), case 31 (override allowlist) and the suite-level exit 77. **1**, the cross-manifest `files:` drift check, is ported as case 33, pairing hooks by `id:` since both now share one `entry:`. *(Corrected later on 2026-09-15.)* An earlier revision of this bullet said 18 / 6 / 11 / 1. It called the cross-manifest check knowingly dropped and "seven of them stronger". None of that survives a recount. Two text greps became behavioural Vale probes, not seven, and case 32 alone never covered the narrowing that case 33 now catches. `check-scope-walkup-sync` **survives**; see the §10 correction below for why. Pre-push goes 9 repo-authored hooks to 8 (11 reported to 10). The rest of the saving is the second embedded resolver (1,061), the second `vale-wrap.sh` (526), the second `assets/vale/styles/Kyberforge/` copy (44), and the Contributing-files parser embedded in both `validate-provenance.sh` copies (93). **413 + 797 + 1,061 + 526 + 44 + 93 = 2,934**, which is the headline. An earlier revision of this bullet listed 413 + 797 + 526 + 48 + 1,061 = 2,845: it dropped the 93-line parser outright, and its **48** for the Vale copy is the five byte-identical style rules (13 + 7 + 7 + 7 + 10 = **44**) plus skill-audit's 4-line `.vale.ini`. ADR-0025 counts **44** on purpose — the two `.vale.ini` files were deliberately *not* identical (agent-audit's carried the extra `[**/*.agent.md]` section and the `KyberforgeCopilot` style), so that file is a deleted file rather than a removed duplicate, and folding it in would make the headline 2,938. All six figures measured at `a5962ba`. + > - **"Each copy is hard-wired to one mode" was false**, and it is the claim that made this look like a bigger win than it is. The two `validate.sh` files are not one script parameterised per mode: outside the shared 1,061-line resolver they hold **1,293 lines between them** (616 skill-side, 677 agent-side) and share **91** of those. That 91 is ADR-0025's figure and it is exactly reproducible: strip the marked resolver block from each copy at `a5962ba` (`115..1175` skill-side, `189..1249` agent-side, 1,061 lines each), then take the size of the intersection of the two *distinct raw line sets* — 510 distinct lines skill-side, 530 agent-side, 91 in common. An earlier revision of this bullet said "about 115", which matches no counting rule that has been reproduced: dropping blank lines gives 90 and dropping comments as well gives 64. The merged validator dispatches on artifact type over two largely independent bodies of checks; it does not collapse them. + > - **The §8 blocker was a non-issue.** The design question held open there — whether one `description` could carry both skills' trigger sets without breaching the ADR-0020 ceiling — was answered against the **400-character FAIL**, which the merged description clears. Read the number from the shipped file, not from a draft. *(Corrected later on 2026-09-15.)* The shipped description is **241 characters**, inside the 250-character SUGGESTION target, and `bash scripts/skill-size-check.sh plugins/kyberforge/.apm/skills/factory-audit/SKILL.md` prints nothing for it. A first cut shipped at **319** and accepted the SUGGESTION as the cost of carrying both artifact types' trigger phrases. That reasoning was wrong. The quoted phrases (`audit this skill`, `review my SKILL.md`, `audit this agent`, `review my agent file`) restated the "skill directory or agent definition audited" trigger in a second register, which ADR-0020 makes a FAIL. Removing them, and keeping both boundary arrows, gives 241. An earlier "241" in this document and ADR-0025's "240" came from a hypothetical single-arrow draft that was never reproduced. That today's figure is also 241 is a coincidence, not a confirmation of it. The real ceiling was the other one: a single body covering both artifact types ran past the **900-word body FAIL**. Solved the way ADR-0020 prescribes — a dispatch body that routes to per-type references, with the 16 per-type reference files namespaced `skill-*` and `agent-*` (plus the shared `sources.md`). + > + > **Finding 18 was deliberately kept out of scope.** Its re-scoped remainder is prose trimming inside these same files and would have made the merge diff unreviewable; it stays open against `factory-audit`'s files. 15. **Merge `skill-author` + `agent-author` likewise.** `contract.md` shares most of its Description section; `new-skill.sh` and `new-agent.sh` implement the same package-root walk-up with different mode names; step 1 dispatch tables and step 3 gates are near-identical. Keep the agent scope logic (plugin vs project/user) as its own reference. Effort M. @@ -489,6 +501,14 @@ The recurring failure mode is worth naming, because it has now produced six wron **Where the real remaining opportunity is:** finding 14 (merge `skill-audit` + `agent-audit`) at **−1,587 lines with zero coverage loss**, which is also where finding 16's savings actually live. Its blocker is the design question in §8 — one `description` carrying both skills' trigger phrases — not an ADR supersession. +> **Executed, and one knock-on claim corrected (2026-09-15).** Finding 14 landed as `factory-audit` (ADR-0025); yield **2,934 lines and one pre-push hook**, and the §8 blocker turned out to be a non-issue. The body was the binding ceiling, not the description, which ships at 241 characters, under the 250 target, once a duplicated trigger register was removed. See finding 14's own note for the corrections. +> +> **The merge does not unblock `check-scope-walkup-sync`, and nothing in this audit should be read as saying it does.** §3's finding 2 bullet says that gate "disappears if the ports share one script or the skills merge"; the second half of that is wrong, and the first is unreachable. The gate cross-checks **four** independent `$HOME`/`.git`/`apm.yml` walk-up ports, and only two of them are in the audit pair (`validate.sh`'s `detect_scope`, `validate-provenance.sh`'s `find_plugin_root`). The other two — `new-agent.sh`'s and `new-skill.sh`'s `find_package_root` — live in the **author** skills, which finding 15 has not merged and which could not be merged into the audit skill in any case. Four ports go to four ports. +> +> It cannot degrade into a text diff either, which is the shape that would let it be deleted rather than merely shrunk: the two audit-side ports are **Python** (`def detect_scope`, `def find_plugin_root`, inside heredocs) and the two author-side ports are **Bash** functions. Byte-comparing them is not an option at any point on this path, so the behavioural fixture cross-check is the only available form of the gate. It survives finding 15 too. +> +> `check-vale-style-sync` was the only one of the two "keep two copies in sync" gates that finding 14 could remove, which is why the yield is one hook and not two. + ### Two defects to fix independently of any finding - **~~A live bug in always-on context.~~ Fixed (2026-09-15).** The deployed `core/instructions/governance.md` cited `docs/HUMANS.md`, which does not exist — the file is `docs/wiki/HUMANS.md`. Five occurrences across three files (`governance.md:82`, which was self-inconsistent against its own correct line 73; `CONTROLS.md:5,101,106`; `ai-constitution.md:238`), in a file `@`-imported into every session in every project. All five now point at `docs/wiki/HUMANS.md`. Note the deployed copy under `~/.claude/` no longer matches the repo until `scripts/install.sh` re-runs. diff --git a/apm.yml b/apm.yml index e1f00f0..fdc69ea 100644 --- a/apm.yml +++ b/apm.yml @@ -1,5 +1,5 @@ name: holocron -version: 0.4.6 +version: 0.4.7 description: AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows. license: MIT @@ -42,7 +42,7 @@ dependencies: # after a kyberforge release, check this first. executables: allow: - kyberforge#1.6.2: + kyberforge#2.0.0: hooks: true bin: true @@ -52,7 +52,7 @@ marketplace: # top-level apm.yml description:/version: above are NOT inherited into the # compiled output despite being used elsewhere (e.g. by `apm audit`). description: AI development skills for Claude Code and GitHub Copilot CLI — factory, design, implement, review, and cross-cutting workflows. - version: 0.4.6 + version: 0.4.7 owner: name: Defame1297 email: defame1297@rkdr.net @@ -77,7 +77,7 @@ marketplace: - name: kyberforge description: Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace. source: ./plugins/kyberforge - version: 1.6.2 + version: 2.0.0 category: Developer Tools - name: bin diff --git a/docs/adr/0004-skill-audit-info-finding-level.md b/docs/adr/0004-skill-audit-info-finding-level.md index 23cf925..031dc8b 100644 --- a/docs/adr/0004-skill-audit-info-finding-level.md +++ b/docs/adr/0004-skill-audit-info-finding-level.md @@ -1,5 +1,10 @@ # Add INFO as a third finding level in skill-audit reports +**Skill renamed per ADR-0025 (2026-09-15):** `skill-audit` and `agent-audit` merged into +`factory-audit`, which dispatches to a skill flow and an agent flow at Step 0. Read `skill-audit` +below as `factory-audit`'s skill flow. The decision itself is unchanged — ADR-0025 carried every +audit criterion, tier and finding level across as-is. + `skill-audit` shipped with two finding levels: FAIL (blocks shipping) and SUGGESTION (optional improvement). Provenance validation introduced observations that are worth surfacing but not actionable: a `references/*.md` file with no diff --git a/docs/adr/0008-agent-audit-single-file-invocation.md b/docs/adr/0008-agent-audit-single-file-invocation.md index 3eca37f..d0c7002 100644 --- a/docs/adr/0008-agent-audit-single-file-invocation.md +++ b/docs/adr/0008-agent-audit-single-file-invocation.md @@ -1,5 +1,11 @@ # agent-audit takes a single file path and derives the counterpart by scope detection +**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into +`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow. The single-file +invocation contract this ADR sets survives the merge intact — `factory-audit` dispatches at Step 0 +on the target path, and an `*.agent.md` or a path under `.apm/agents/` takes the agent flow, so the +caller still names one file and the script still derives the rest. + `agent-audit` validates agent definition file pairs (Claude Code `.md` + Copilot `.agent.md`). The skill accepts a path to either file and derives the counterpart using scope detection rather than requiring the caller to name both files or supply a root directory. ## Considered options diff --git a/docs/adr/0009-agent-audit-field-inventory-reference.md b/docs/adr/0009-agent-audit-field-inventory-reference.md index bc7c3e0..47f54c8 100644 --- a/docs/adr/0009-agent-audit-field-inventory-reference.md +++ b/docs/adr/0009-agent-audit-field-inventory-reference.md @@ -1,5 +1,10 @@ # agent-audit reads field lists from a reference file, not hardcoded script arrays +**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into +`factory-audit`. Read `agent-audit` below as `factory-audit`'s agent flow; the reference file this +ADR is about is now `factory-audit/references/agent-field-inventory.md`. The decision is unchanged — +the field lists still live in a reference file read at runtime, not in script arrays. + `agent-audit`'s `validate.sh` checks for Claude Code-only fields in Copilot files and silently-ignored fields in plugin agents. Rather than hardcoding those field lists in the script, the script reads `references/field-inventory.md` at runtime. This keeps field list diff --git a/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md b/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md index edbde4d..7635dca 100644 --- a/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md +++ b/docs/adr/0010-agent-sources-relocated-outside-agents-dir.md @@ -15,6 +15,9 @@ ADR's own conclusion is unaffected by that move: the provenance file still belon auto-scans, and `.apm/agents/` is, if anything, further removed from plugin-root than the old flat `agents/` directory was, so the reasoning below still holds. References below to `/agents/` describe the pre-APM layout in effect when this decision was made. +**Skill renamed per ADR-0025 (2026-09-15):** `agent-audit` merged with `skill-audit` into +`factory-audit`; read the `agent-audit` references below as `factory-audit`'s agent flow, whose +`validate-provenance.sh` still resolves `/sources.md` exactly as this ADR decided. **Scope boundary (per ADR-0016):** this path change is plugin scope only. Project scope (`.claude/agents/` + `.github/agents/`) and user scope (`~/.claude/agents/` + `~/.copilot/agents/`) are unaffected — they are not APM packages and keep the dual-file diff --git a/docs/adr/0012-agentsmd-tooling-in-core-plugin.md b/docs/adr/0012-agentsmd-tooling-in-core-plugin.md index eac8b90..dad0775 100644 --- a/docs/adr/0012-agentsmd-tooling-in-core-plugin.md +++ b/docs/adr/0012-agentsmd-tooling-in-core-plugin.md @@ -6,7 +6,7 @@ Three skills in the `core` plugin (`core`'s first active skills): -- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern. When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition. +- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern (**skill renamed per ADR-0025, 2026-09-15:** `skill-audit` is now `factory-audit`'s skill flow; the author-then-audit pattern is unchanged). When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition. - **`agentsmd-audit`** — a single combined pass checking three mandatory baselines against `AGENTS.md` only: secrets/credentials (governance.md hard prohibition), structural completeness (common-sections checklist from the agents.md spec), and accuracy/drift (do referenced commands/paths resolve against the repo). Never inspects provider adapter files. - **`provider-adapter-author`** — detects and converts a provider-specific instruction file into a thin adapter that imports `AGENTS.md` (mirroring this repo's own two-tier `CLAUDE.md` pattern). Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`) rather than a separate paired audit skill, since the check (import present, no duplicated headings, size threshold) is mechanical. diff --git a/docs/adr/0013-vale-harness-scope-and-rule-sources.md b/docs/adr/0013-vale-harness-scope-and-rule-sources.md index 1171785..18862cd 100644 --- a/docs/adr/0013-vale-harness-scope-and-rule-sources.md +++ b/docs/adr/0013-vale-harness-scope-and-rule-sources.md @@ -109,7 +109,8 @@ every rule to `level: error` is what actually implements this decision. 1.81 tokens per word, so a worst-case `SKILL.md` at the ceiling still lands under 5,000 tokens — `wc -w` is not BPE tokenization). Either one exceeded fails the hook. Both are inclusive: a file at exactly 500 lines or exactly 2,770 words passes, and only one past a ceiling - fails. `skill-audit/scripts/validate.sh` enforces the same pair on the same inclusive terms, so + fails. `skill-audit/scripts/validate.sh` (now `factory-audit/scripts/validate.sh`, see ADR-0025) + enforces the same pair on the same inclusive terms, so the audit and the commit hook cannot disagree about whether a given `SKILL.md` is over size. - `styles/KyberforgeTrial/` and `.vale.trial.ini` were deliberately not created — noted here so a future reader doesn't wonder if a trial tier was forgotten. diff --git a/docs/adr/0014-vale-prefilter-ships-from-the-plugin.md b/docs/adr/0014-vale-prefilter-ships-from-the-plugin.md index bda5050..2b5871a 100644 --- a/docs/adr/0014-vale-prefilter-ships-from-the-plugin.md +++ b/docs/adr/0014-vale-prefilter-ships-from-the-plugin.md @@ -5,6 +5,28 @@ out of the repo root was deliberately deferred there, not fixed. ADR-0013's othe (rule scope, `level: error` model, `SentenceOpenerThereIs`/`VagueQualifier` trial outcomes) is unaffected and remains in force. +**Amended by ADR-0025 (2026-09-15).** The reasoning below is not reversed; its *precondition* is +gone. The two skill-scoped Vale copies this ADR mandates — `agent-audit/assets/vale/` (canonical) +and `skill-audit/assets/vale/` (subset) — existed because the no-cross-skill-sharing rule made it +impossible for one audit skill to read the other's config. ADR-0025 merges the two skills into +`factory-audit`, so there is no boundary left to duplicate across: there is now **one** copy, at +`plugins/kyberforge/.apm/skills/factory-audit/assets/vale/`, carrying both styles and the +single-file `.vale.ini` — `[**/SKILL.md]`, `[**/agents/*.md]`, `[**/*.agent.md]` — that this ADR's +"One hook per file-scope" section had split in two. `scripts/check-vale-style-sync.sh`, decided on +below and wired at pre-push, is deleted with the copy it diffed. Nothing it asserted about the +config was lost. Its six-row glob-coverage probe table is now `tests/test-vale-wrap.sh` cases 28-30, +run against the merged config. Case 31 carries across the per-rule override allowlist, and case 0 +carries across the "config loads" guards. Its **cross-manifest `files:` drift check** is ported as +case 33. The original keyed each hook's record on `entry:`, which stopped working once both vale +hooks shared one entry, so the port pairs the hooks by `id:` instead. Of the script's 17 assertion +sites, 6 compared the two copies and are moot, 10 are rehomed and 1 is ported. ADR-0025 gives the +per-assertion mapping; read the "six" here as probe *rows*, not as a share of those 17. +What does **not** change: the two exported hook IDs, `kyberforge-vale-audit-skill` and +`kyberforge-vale-audit-agent`, keep their IDs and their `files:` regexes — external consumers pin +them by name — and the argument-free `entry:` contract is untouched. Read the two-copy table, the +sync-check paragraph, and the `tests/test-vale-wrap.sh` Consequences bullet below as the state this +ADR established, not as current layout. + `skill-audit`/`agent-audit`'s Step 1 called `"$(git rev-parse --show-toplevel)/scripts/vale-wrap.sh" --config "$(git rev-parse --show-toplevel)/.vale.ini"` — which resolves to whichever repo the skill happens to be running in. Inside `ai-development` @@ -128,7 +150,9 @@ doesn't wonder if it was overlooked. `.pre-commit-config.yaml` stays byte-identical to the shipped manifest on those `entry:` lines so the local gate keeps exercising the same resolution path a consumer does. - `tests/test-vale-wrap.sh` now exercises skill-audit's copy specifically — its fixtures are all - `SKILL.md`-shaped, and only skill-audit's `.vale.ini` has the matching glob section. + `SKILL.md`-shaped, and only skill-audit's `.vale.ini` has the matching glob section. (State as of + this ADR. Since ADR-0025 there is one `vale-wrap.sh` and one `.vale.ini` under `factory-audit/`, + and that suite exercises all three glob sections of the merged config — see cases 28-30.) - The first `vX.Y.Z` tag is cut once this change and its tests pass, giving external `.pre-commit-hooks.yaml` consumers something to pin. - **Cutting the tag is not left to memory.** `scripts/check-release-needed.sh`, wired at diff --git a/docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md b/docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md index 48e6d42..6062ee8 100644 --- a/docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md +++ b/docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md @@ -123,7 +123,8 @@ correction) sorted what they document into three buckets: - ADR-0016 (a narrower decision discovered while designing issue #89) turned out to gate how issue #90 had to re-author plugin-scope agents: `.apm/agents/*.agent.md` compiles verbatim to both Claude and Copilot, so those files carry only the fields in the `apm-agent-allowlist` section - of `plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (as amended + of `plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (now + `factory-audit/references/agent-field-inventory.md`, see ADR-0025) (as amended 2026-08-14: `name`/`description`/`model`/`source_keys`/`disallowedTools`) — existing dual-file `.md`+`.agent.md` pairs could not be raw-moved, only re-authored. - Two follow-up issues tracked the remaining work: #89 (`skill-author`/`agent-author` routing diff --git a/docs/adr/0016-apm-agent-primitive-drops-provider-specific-fields.md b/docs/adr/0016-apm-agent-primitive-drops-provider-specific-fields.md index 46ce8c8..9a91f31 100644 --- a/docs/adr/0016-apm-agent-primitive-drops-provider-specific-fields.md +++ b/docs/adr/0016-apm-agent-primitive-drops-provider-specific-fields.md @@ -143,7 +143,8 @@ below is narrowed accordingly. Enforcement follows the decision: `agent-audit`'s plugin-scope validator reads its allowlist as data from the `apm-agent-allowlist` section of -`plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md`, and that line now reads +`plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md` (now +`factory-audit/references/agent-field-inventory.md`, see ADR-0025), and that line now reads `name description model source_keys disallowedTools`. `disallowedTools` also stays in that file's `claude-code-only-fields` list, which is not a contradiction — that list governs whether a field may cross the CC/Copilot boundary in a real project/user-scope *pair*, a different question from diff --git a/docs/adr/0018-repo-consumes-its-own-plugins-through-apm.md b/docs/adr/0018-repo-consumes-its-own-plugins-through-apm.md index 1edc752..6537b90 100644 --- a/docs/adr/0018-repo-consumes-its-own-plugins-through-apm.md +++ b/docs/adr/0018-repo-consumes-its-own-plugins-through-apm.md @@ -65,7 +65,9 @@ Three sub-decisions inside that: ## Consequences **Skills gain an unnamespaced name.** apm deploys plain project skills, so `git:git-commits` also -answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit`. This is not configurable — +answers to `git-commits` and `kyberforge:skill-audit` to `skill-audit` (**skill renamed per ADR-0025, +2026-09-15:** that skill is now `factory-audit`, so the live example is `kyberforge:factory-audit` to +`factory-audit`; the rule is unchanged). This is not configurable — a project skill has no plugin to prefix. `AGENTS.md` and `CONTEXT.md` are updated to name the bare form, which is what apm deploys and the only form a repo consuming holocron through apm gets. diff --git a/docs/adr/0020-skill-description-and-body-context-contract.md b/docs/adr/0020-skill-description-and-body-context-contract.md index f103b2e..59701fd 100644 --- a/docs/adr/0020-skill-description-and-body-context-contract.md +++ b/docs/adr/0020-skill-description-and-body-context-contract.md @@ -8,6 +8,23 @@ gates that hold them. **Status: accepted (2026-08-14).** +**Amended by ADR-0025 (2026-09-15).** The contract, the tiers and every verdict rule below stand +unchanged. What moved is the **number and location of the scripts that carry them**. This ADR names +three: `scripts/skill-size-check.sh`, `skill-audit/scripts/validate.sh` and +`agent-audit/scripts/validate.sh` — "all three validators" (Decision), "all three scripts" +(Enforcement table footnote), "`scripts/skill-size-check.sh` and its two mirrored copies" +(the `_add()` amendment). ADR-0025 merged the two audit skills, so there are now **two**: the root +`scripts/skill-size-check.sh`, which still embeds the 1,061-line block between `BEGIN`/`END ADR-0020 +SHARED BOUNDARY RESOLVER` markers, and one plugin copy — extracted out of the merged validator into +`plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh` and sourced by +`factory-audit`'s `validate.sh` and `validate-provenance.sh` rather than pasted into each. The +Enforcement table's "constants mirrored in `skill-audit/scripts/validate.sh` and +`agent-audit/scripts/validate.sh`" is one path now, `factory-audit/scripts/validate.sh`, which +auto-detects the artifact type; the skills/agents columns are unaffected, since the merged validator +applies the body tiers on the skill path only. The two copies must still stay byte-identical — a +plugin script cannot source the root one, which is why a second copy exists at all. Read every +"three" below as the count at the time of writing. + ## Context Every `file:line` citation in this ADR is against the base commit the decision was taken on, @@ -189,7 +206,8 @@ Agents take the same description gates — they are preloaded identically — an A skill body is loaded into the caller's context, competing with the live conversation; an agent body becomes the system prompt of a fresh context. The rationale for the 900-word FAIL does not transfer. -That exemption is expressed in `agent-audit/scripts/validate.sh`, which has no body constant, and in +That exemption is expressed in `agent-audit/scripts/validate.sh` (now `factory-audit`'s +auto-detecting `validate.sh` on its agent path, see ADR-0025), which has no body constant, and in the `files:` pattern of the `skill-size-check` pre-commit hook, which is `SKILL.md`-only. It is *not* expressed in `scripts/skill-size-check.sh` itself, which measures whatever path it is handed — running it directly over `plugins/*/.apm/agents/*.agent.md` exits 1 with 900-word body FAILs on @@ -201,7 +219,8 @@ file pattern, not by the script knowing the difference. Anyone widening that pat would silently enforce a gate this ADR declines to set. A plugin-scope agent is a single file with no sibling `references/` directory, so it cannot disclose -to itself — it can only delegate to skills. `agent-audit` therefore gains a **delegation check**: an +to itself — it can only delegate to skills. `agent-audit` (now `factory-audit`'s agent flow, see +ADR-0025) therefore gains a **delegation check**: an agent body that restates a procedure owned by a skill it can invoke is a FAIL, with the fix being "invoke `` instead". Length falls out of delegation rather than being gated directly. @@ -232,18 +251,20 @@ type of input they take should be **one skill with a dispatch table**. This catc one-or-two-file agent pair, per ADR-0005 and ADR-0016) and their overlap is in the improve flow rather than the core job. -**DEFERRED — not implemented in the change that carries this ADR. Tracked as issue #101.** Both -skills still exist separately, and this change made the split deeper rather than shallower: retrofit +**DEFERRED when this ADR was written — not implemented in the change that carries it. Tracked as +issue #101. IMPLEMENTED by ADR-0025 (2026-09-15), which merged the pair into `factory-audit` with a +Step 0 dispatch and closed the deferral.** At the time of writing both +skills still existed separately, and this change made the split deeper rather than shallower: retrofit to the dispatch pattern took `skill-audit` from 3 reference files to 7 and `agent-audit` from 4 to 8, and their two same-named `references/description-quality.md` files now differ on 100 of ~120 lines after normalising `skill`/`agent`, where before they were closer. It has kept deepening since: the #99 retrofit added `finding-criteria.md` to `skill-audit`, drawing it level with `agent-audit`. Both figures move with the next retrofit, so measure rather than quote — -`ls plugins/kyberforge/.apm/skills//references/ | grep -c '\.md$'`. The merge stays the +`ls plugins/kyberforge/.apm/skills/factory-audit/references/ | grep -c '\.md$'`. The merge stayed the decision; it reopens ADR-0008 (agent-audit's single-file invocation contract) and touches every call -site in `skill-author`, `agent-author` and `forge`, which is why it is its own change and not a rider -on this one. Recorded here rather than dropped, so the gap between the rule and the tree is deliberate -and dated instead of discovered later. +site in `skill-author`, `agent-author` and `forge`, which is why it was its own change and not a rider +on this one. Recorded here rather than dropped, so the gap between the rule and the tree was deliberate +and dated instead of discovered later — and ADR-0025 is where it was closed. ### Enforcement and rollout @@ -266,7 +287,7 @@ which tier each rule is in, because the failure this ADR is most exposed to is a | description opener, composition notes in a description | skills, agents | prose pattern | `plugins/kyberforge/.apm/skills/*/assets/vale/styles/Kyberforge/` | | a Gotcha paraphrasing a body step | skills | **auditor judgment** | `references/body-discipline.md` | | dispatch at two or more mutually exclusive flows | skills | **auditor judgment** | `references/body-discipline.md` | -| delegation: an agent body restating a skill's procedure | agents | **auditor judgment** | `agent-audit` | +| delegation: an agent body restating a skill's procedure | agents | **auditor judgment** | `agent-audit` (now `factory-audit`'s agent flow, see ADR-0025) | | capability enumeration, restatement, trigger quality | skills, agents | **auditor judgment** | `references/description-quality.md` | The rows in bold are stated as FAILs in the Decision above and are FAILs an *auditor* issues. None of @@ -399,16 +420,18 @@ over the same 39 files now reports 0 errors, 0 warnings and 0 suggestions, so independent of `skill-size-check`, so a new description can reintroduce it; `skill-size-check` does not cover the Vale half, and no `references/` file is linted by anything (`docs/spec/gates.md` has both causes, issue #117 tracks them). Re-derive rather than quote —* -`bash plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh plugins/*/.apm/skills/*/SKILL.md`. +`bash plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh plugins/*/.apm/skills/*/SKILL.md` +*(path re-pointed by ADR-0025; the `skill-audit` copy this ADR originally named no longer exists).* **A ceiling does not produce an average.** If every author writes to the 400-character FAIL, the preload lands at 39 × 400 = 15,600 chars — a 33% cut off 23,427, not the ~50% intended. Writing to the 250-character SUGGESTION instead lands at 9,750, a 58% cut. The halving depends entirely on the 250-character SUGGESTION tier being visible and respected. That tier works here in a way it does not -elsewhere in this repo: `skill-audit` already reports `PASS (N suggestions)` as a first-class +elsewhere in this repo: `skill-audit` (now `factory-audit`'s skill flow, see ADR-0025) already +reports `PASS (N suggestions)` as a first-class outcome. This is explicitly **not** the failure ADR-0013 records — Vale warnings are invisible -because vale's exit code keys on `error` alone, but these gates live in `validate.sh` and -`skill-audit`, where a SUGGESTION reaches the report. Realistic landing is somewhere in that 33-58% +because vale's exit code keys on `error` alone, but these gates live in `validate.sh` and the +audit skill itself, where a SUGGESTION reaches the report. Realistic landing is somewhere in that 33-58% band, not a guaranteed 50%. **A word gate cannot detect the defect it is standing in for.** `git-commits` carries twelve Gotchas diff --git a/docs/adr/0021-plugin-descriptions-state-a-domain-boundary.md b/docs/adr/0021-plugin-descriptions-state-a-domain-boundary.md index 17f32d2..3fd00ad 100644 --- a/docs/adr/0021-plugin-descriptions-state-a-domain-boundary.md +++ b/docs/adr/0021-plugin-descriptions-state-a-domain-boundary.md @@ -83,7 +83,8 @@ unnamed in `git`'s corrected description, though `65bac15`'s own commit message **Nothing checks any of this.** `scripts/check-manifests.sh` does not contain the string `description`. The three ADR-0020 validators (`scripts/skill-size-check.sh` and skill-audit's and -agent-audit's `validate.sh`) gate on SKILL.md and agent frontmatter; they do open `apm.yml`, but only +agent-audit's `validate.sh` — two since ADR-0025 merged the audit pair into `factory-audit`, whose +single auto-detecting `validate.sh` carries both) gate on SKILL.md and agent frontmatter; they do open `apm.yml`, but only to read `dependencies.apm` when resolving the boundary-target universe — none of them reads the `description:` key, and their hook globs match `SKILL.md` and `*.agent.md` only. `apm audit --ci`, `apm pack --check-clean` and `scripts/sync-plugin-content.sh --check --all` all compare compiled diff --git a/docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md b/docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md new file mode 100644 index 0000000..d2e51d7 --- /dev/null +++ b/docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md @@ -0,0 +1,376 @@ +# `skill-audit` and `agent-audit` merge into one `factory-audit` with a Step 0 dispatch + +**Status: accepted (2026-09-15).** Implements ADR-0020's "Merging siblings" rule, which named this +exact pair, scoped itself to them, and then deferred the work as issue #101. The deferral is closed +here. `skill-author` and `agent-author` stay separate — ADR-0020 excluded the author pair +deliberately, and nothing in this change touches that exclusion. + +## Context + +Every figure below was measured against the worktree on 2026-09-15. Re-derive rather than quote; the +commands are given where a number is load-bearing. + +The two skills duplicate content because they cannot share a file. +`plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md`, sourced from the +agentskills.io spec, states the constraint for APM package mode: file references inside +`.apm/skills//` must not reach outside that skill's own directory, and the spec defines no +cross-skill sharing mechanism. apm deploys skills flat into `.claude/skills//` with no plugin +tier above them, so there is no directory a second skill could read from. ADR-0024 re-confirmed this +after deleting the native install path, specifically to stop the constraint being re-litigated as a +Claude-Code artifact. It is not one. It binds under the only install path that survives. + +What that constraint costs, between these two skills: + +| Duplicated artifact | Lines | Verification | +|---|---|---| +| ADR-0020 boundary resolver, embedded in both `validate.sh` copies | 1,061 | marker block `115..1175` (skill) and `189..1249` (agent); `tests/test-adr0020-contract.sh` assertion 1 hashes them | +| `scripts/vale-wrap.sh` | 526 | `diff -q` clean | +| `assets/vale/styles/Kyberforge/`, five rules | 44 | `diff -r` clean | +| Contributing-files parser, embedded in both `validate-provenance.sh` copies | 93 | marker block `300..392` (skill) and `294..386` (agent); `diff -q` clean on the extracted blocks | +| **Removable by merging** | **1,724** | | + +On top of that, `scripts/check-vale-style-sync.sh` (413 lines) and +`tests/test-check-vale-style-sync.sh` (797 lines) go with the merge. That is **not because the whole +gate was a copy diff**; it was not, and saying so would overstate the case for deleting it. The +script has **17 assertion sites**: 13 `err` calls and 4 hard-fail exits. Its closing +`exit 1` only reports the `err` count, so it is not an assertion. Count them with +`git show 61b0b9c^:scripts/check-vale-style-sync.sh`. An earlier revision of this ADR said 18. No +reproducible counting rule gives 18, and it is corrected here. + +| Class | Old line | What it asserted | Now | +|---|---|---|---| +| **Moot (6)** | 19 | `REPO_ROOT` is a directory | nothing to guard; no script | +| | 42 | the `.apm/` paths are not stale | no copies to locate | +| | 52, 56 | neither copy is missing | one copy | +| | 60 | the two `vale-wrap.sh` copies are identical | one copy | +| | 64 | the two `styles/Kyberforge/` copies are identical | one copy | +| **Rehomed (10)** | 95, 113 | `.vale.ini` exists and is readable | case 0 | +| | 122 | `StylesPath = styles` is set | case 0 | +| | 127 | some section's `BasedOnStyles` names `Kyberforge` | case 28 (Part B proves it fails) | +| | 174 | no Kyberforge rule is overridden below a bare `YES`/`error` | case 31 | +| | 191 | `KyberforgeCopilot` ships and is loaded | case 30 | +| | 308 | `vale` is installed | suite-level: exit 77, which `run-tests --strict` fails | +| | 341 | every probe path matches some vale hook's `files:` regex | case 28 | +| | 347 | every probe path raises a Kyberforge alert under the config | cases 28 and 29 | +| | 397 | at least one probe row was checked | case 28's section floor | +| **Ported (1)** | 343 | local and published `files:` regexes agree per probe | case 33 | + +Six are moot. Two diffed the copies, and four guarded the script's own ability to locate them: a +real `REPO_ROOT`, non-stale `.apm/` paths, and both copies present. With one copy and no script +there is nothing left to diff or locate. The other ten read `.vale.ini`, the style directory and +the hook manifests on their own terms, so they are **rehomed rather than retired**, into `tests/test-vale-wrap.sh`, which already owns the +wrapper's behaviour against this config. + +Two rehomed checks got stronger, because a text grep became a behavioural Vale probe: + +- line 127: case 28 Part B drops `Kyberforge` from a copy and requires vale to report the style as + not loaded; +- line 191: case 30 requires the Copilot rule to fire on `.agent.md` and nowhere else, and Part B + proves both an unload and a leak fail. + +The rest moved at equal strength. Case 31 is the same grep as before. An earlier revision claimed +"7 of 11 stronger"; that claim is withdrawn. + +**The cross-manifest check is ported, not dropped.** It extracts each vale hook's `files:` regex +from `.pre-commit-hooks.yaml` (the external-facing manifest) and from `.pre-commit-config.yaml` +(this repo's own copy of the same hooks) *independently*. It then asserts that a probe path in scope +of one is in scope of the other. That catches this repo narrowing its local hook without narrowing +the published one, or the reverse. + +The original selected each hook's record by matching `entry:` against the owning skill's +`scripts/vale-wrap.sh` path. After the merge both vale hooks point at the same entry, so that +selector can no longer tell them apart. Case 33 pairs the hooks by `id:` instead, from an explicit +table: `kyberforge-vale-audit-skill` ↔ `vale-audit-prefilter-skill`, and +`kyberforge-vale-audit-agent` ↔ `vale-audit-prefilter-agent`. It carries the original six probe rows +unchanged. It also fails by name on a missing hook id, and on a class with no shared probe. + +An earlier revision of this ADR shipped *without* that port and called the gap half-closed by case +32. It was not. Narrowing `vale-audit-prefilter-skill` from `^plugins/[^/]+/...` to +`^plugins/kyberforge/...` still matches tracked files of the right class. That clears case 32 while +silently dropping every other plugin's skills from this repo's prefilter, and it was measured +leaving the whole suite green. Case 33's Part B now makes exactly that mutation, the agent-hook +equivalent and a renamed hook id, and requires each to fail. + +Case 32 stays, for the separate zero-match question: each local hook must still select at least one +tracked file, and only files of its own artifact class. + +**Line count.** Merging removes the 1,724 duplicated lines above. Deleting the two sync-gate files +(413 + 797 = 1,210) removes more, for **2,934 lines** in total, plus one pre-push hook, +`check-vale-style-sync`, formerly at `.pre-commit-config.yaml:166`. Two smaller deletions are not in +that figure: + +- `scripts/sync-vale-styles.sh` (21 lines), the helper that regenerated skill-audit's copy from + agent-audit's, now has nothing to sync. +- `agent-audit/scripts/README.md` (47 lines) has no successor. Nothing referenced it, and the only + README `references/skill-file-structure.md` mandates is `tests/README.md`, which survives. + +The duplication is not symmetrical across the whole tree, and the asymmetry is what shapes the +decision. Outside the shared resolver the two `validate.sh` copies total 1,293 lines (616 skill, 677 +agent) and have **91 distinct lines** in common. The two validators are not one script with a mode +flag; they are two genuinely different scripts that happen to embed one identical block. + +The bodies are the binding constraint on the merge. `skill-audit`'s body is 724 words and +`agent-audit`'s is 808 — 1,532 together against `BODY_MAX_WORDS = 900`. Only 211 words are common +to both (47 byte-identical body lines). A merged body that simply concatenated the two flows would +fail the gate its own plugin enforces by a factor of 1.7, and there is no trimming route to 900: +1,321 of the 1,532 words are flow-specific. + +Both skills already carry `category: factory` in their metadata, and both carry five +`source_keys` — ten in total, disjoint, because they audit against different specs. + +## Decision + +**The two skills become one, named `factory-audit`.** + +**1. The name.** `factory` is what both already declare as their `category`, so the merged skill is +named for the thing it audits rather than for the two input types it now dispatches between. Two +alternatives were live and both are rejected below for naming reasons rather than substance: +`audit` collides with the unrelated `agentsmd-audit`, and `forge-audit` makes a bare skill name a +family prefix of a live sibling — `forge` exists at +`plugins/kyberforge/.apm/skills/forge/`. No bare skill name may be a family prefix of another. + +**2. `SKILL.md` becomes a dispatch body.** Steps 1-3 move out to `references/skill-flow.md` and +`references/agent-flow.md`. The body carries the Gotchas that apply to both branches, the dispatch +table, and Step 4 — Report, which is shared. This is ADR-0020's own rule ("Dispatch is mandatory at +two or more mutually exclusive flows") applied to the file that defines it, and the word arithmetic +above is why it is mandatory here rather than stylistic. + +**Dispatch happens at Step 0, keyed on the target path, before Step 1 runs.** The table accepts +exactly the shapes `scripts/validate.sh` detects: + +- A directory containing `SKILL.md`, or a `SKILL.md` file (its parent directory is audited), takes + the skill flow. +- A `*.agent.md` file, or a `.md` file whose *immediate* parent directory is `agents/`, takes the + agent flow. +- Anything else stops, runs no validator, and names the two accepted shapes. + +Putting the dispatch after any deterministic check would mean running the wrong validator first and +reading its output as a finding. An earlier revision of the body carried a two-row table with no +fallback row. It could not route a `SKILL.md` file path, a trigger its own description advertised. +Its agent row ("a path under `.apm/agents/`… or an agent markdown file") was both wider than the +script and circular. + +**3. One entry point per script, auto-detecting, with the mode-specific half sourced.** + +- `scripts/validate.sh` detects the target type itself, then sources `scripts/lib-boundary-resolver.sh` + and one of `scripts/lib-checks-skill.sh` / `scripts/lib-checks-agent.sh`. +- `scripts/validate-provenance.sh` does the same, sourcing `scripts/lib-contributing-files.sh` and + one of `scripts/lib-provenance-skill.sh` / `scripts/lib-provenance-agent.sh`. + +Two things justify this shape. First, **self-containment binds between skills, not within one.** The +resolver had to be embedded verbatim in three copies because three skill directories cannot read +each other's files; two files inside one skill directory have no such problem. Sourcing is available +the moment the directory boundary between them disappears. Second, **a single auto-detecting entry +point makes a Step 0 misdispatch detectable.** The script re-detects the flow from the target, so +even after a misdispatch it runs the right checks and its finding tiers are correct. That alone does +not make the misdispatch self-correcting, and an earlier revision of this ADR wrongly said it did. +The flow file drives Steps 2-4, so a misdispatched audit would still apply the wrong Step 3 rubrics, +print the wrong coverage line and recommend the wrong author skill. The body closes that gap with an +explicit guard under the Step 0 table: if `validate.sh` reports on the other artifact type than the +row taken, discard the run and restart at Step 0. + +**4. Reference files are prefixed by flow, with one exception.** Every flow-specific file becomes +`skill-*` or `agent-*` — `skill-description-quality.md`, `agent-description-quality.md`, +`skill-finding-criteria.md`, `agent-finding-criteria.md`, and so on. The exception is `sources.md`, +which stays singular and carries all ten `source_keys`, because the skill-side provenance check +hard-codes `os.path.join(skill_dir, "references", "sources.md")` (pre-merge +`skill-audit/scripts/validate-provenance.sh:180`, now `scripts/lib-provenance-skill.sh:215`). A per-flow sources file would mean +changing the provenance contract to get a cosmetic gain. + +**5. Both exported Vale hook IDs survive unchanged.** `.pre-commit-hooks.yaml` keeps +`kyberforge-vale-audit-skill` and `kyberforge-vale-audit-agent`, keeps both `files:` regexes +(`(^|/)SKILL\.md$` and `(^|/)agents/[^/]+\.md$|\.agent\.md$`), and re-points both `entry:` lines at +the one surviving `vale-wrap.sh`. Nothing in the published hook-repo contract changes: an external +consumer's `.pre-commit-config.yaml` keeps working byte-for-byte across the merge. Two IDs pointing +at one script is not a redundancy — it is what keeps the two `files:` scopes addressable +independently, which is exactly ADR-0014's "one hook per file-scope" finding. + +**6. `tests/test-adr0020-contract.sh` changes in three ways, and the third is a conversion, not a +deletion.** Assertion 1 drops from three resolver copies to two: the merged `factory-audit` holds +one, and `scripts/skill-size-check.sh` keeps its embedded copy. A new assertion 1a gives the +resolver the same protection 1b already gave the parser. It asserts that `validate.sh` sources +`lib-boundary-resolver.sh` in both mode branches, and that the resolver's BEGIN marker and +`def _authoring_root(` appear in exactly those two files and nowhere else. A byte-identity hash alone +would miss a third pasted copy, or an entry point that quietly stopped sourcing the library. Sourcing the resolver from the +plugin tree into `skill-size-check.sh` was considered and refuted — that script is a repo-root hook +consumed through `.pre-commit-hooks.yaml`, where `entry[0]` is the only token pre-commit rewrites, +so it cannot reach a file inside the plugin at a path any consumer has. Assertion 1b is **converted**: +it stops pinning that two `validate-provenance.sh` copies of the Contributing-files parser are +byte-identical, and starts pinning that `lib-contributing-files.sh` is a single sourced copy that has +not been re-inlined into either mode library. The claim it protects is the same one — the parser has +exactly one authority — stated against the new structure. The drift history behind it is smaller than +an earlier revision of this ADR implied. `484357a` (2026-08-30) added the bullet-form parser to both +copies with two different spellings of the loop: a temporary `rest` in skill-audit and an inline +slice in agent-audit. The two were behaviourally identical. `598a7c3` (2026-09-01) unified the +spellings and added the `SHARED CONTRIBUTING-FILES PARSER` markers that 1b hashed. From then until +the merge's parent the two marker blocks were byte-identical (`md5 0857272d…` both). So the parser +never *parsed* differently. What the gate never covered was the prose around the block, and a +docstring there asserted identity the loop did not have. One sourced library removes the question. + +**7. Two things this change does not do.** `skill-author` and `agent-author` are **not** merged +here. That remains an open finding and it is unmeasured; ADR-0020 excluded the pair on the grounds +that they emit genuinely different artifacts, and nothing measured in this session revisits that. +And **no audit criterion changes.** Every check, tier, threshold, regex and branch is carried across +as-is. The Python payloads reassembled from the new libraries differ from the pre-merge heredocs only +in comments. The one exception is three lines naming `references/agent-field-inventory.md`, a +byte-identical rename of `field-inventory.md`. Byte-level differential runs over every live skill +directory and agent file matched stdout, stderr and exit code. + +**The entry points are not behaviour-neutral, and an earlier revision of this ADR said they were.** +Those differential runs used valid targets only, so they could not see that the new detection layer +changed what happens to *invalid* ones. Every change below is deliberate: + +| Input | Pre-merge | Now | +|---|---|---| +| a missing path, a directory with no `SKILL.md`, a non-agent `.md` (e.g. `README.md`) | exit 1, or a mode-specific exit-2 message | **exit 2** with one generic "matches neither" Error/Why/Fix. Exit 2 is the never-ran tier, so the flow files report the section as unverified and quote the reason. | +| a `SKILL.md` file path | exit 1 or 2 (`…/SKILL.md/SKILL.md not found`, "not a directory") | **accepted**; its parent directory is audited | +| an agent `.md` *not* under an `agents/` directory (e.g. `~/drafts/my-agent.md`) | audited | **refused, exit 2**. Detection never guesses. No tracked file in this repo is affected. | +| a bare or `./`-relative agent filename, run from inside its `agents/` directory | audited | audited. The parent directory's name is read from the real path, not the typed string. | +| a `lib-*.sh` missing or unreadable, or the script directory unresolvable | did not apply (single file) | **exit 2** with Error/Why/Fix, never a raw bash error at exit 1, which is the real-findings tier | +| `CDPATH` exported | did not apply (no `cd`) | no effect. `SCRIPT_DIR` resolves with `CDPATH=''` and `cd -- … >/dev/null`. | +| no argument | `Error: skill-dir is required.` / `agent-file is required.` | one combined message and usage block; exit code unchanged (1 from `validate.sh`, 2 from `validate-provenance.sh`) | + +A single `validate.sh` copied or symlinked out of its `scripts/` directory still does not work, +because its libraries are not beside it. It now fails at exit 2 and says so. + +## Considered options + +**Keep two skills and rely on the byte-identity contract test alone (rejected).** This is the status +quo: `tests/test-adr0020-contract.sh` already hashes the resolver across copies, and +`check-vale-style-sync.sh` already diffs the Vale halves at pre-push. Only 6 of its 17 assertion +sites exist because there are two copies. The other 11 do other work, and are rehomed or ported above rather than being an +argument for the status quo. On the duplication itself it polices drift rather than removing the thing that drifts, +and it pays 2,934 lines plus a pre-push hook to do so. It also leaves +the router carrying a mutually-excluding near-miss pair whose two descriptions each spend a boundary +clause pointing at the other — a routing cost the merge removes for free. ADR-0020 already weighed +this option for this pair and chose merging; nothing measured since changes the balance. + +**One monolithic dispatching `validate.sh` (rejected).** Dropping one resolver copy from the +concatenation of the two current files gives roughly 2,354 lines in a single script. It is the +straightforward reading of "merge the scripts", and it is wrong on the evidence: the two validators +share only 91 distinct lines outside the resolver, so a monolith would be two near-disjoint +implementations behind one `if`, with every future edit to either half requiring a reader to hold +both in context. Sourcing per-mode libraries gets the same single entry point and keeps the halves +readable apart. + +**Genuinely merging the three colliding reference files into two-section files (rejected).** +`description-quality.md`, `finding-criteria.md` and `validation-scripts.md` exist under both skills +today, and folding each into one file with a skill section and an agent section is the tidier-looking +outcome. It defeats the dispatch. The entire point of moving Steps 1-3 into `references/` is that an +invocation loads one flow's content and not the other's; a two-section reference file re-inflates +per-invocation context to the full 1,532-word span the body ceiling forced out. ADR-0020 measured +these same files at 100 of ~120 differing lines after normalising `skill`/`agent`, so the merged file +would also be mostly disjoint text under one heading. + +**Naming it `audit` (rejected).** Shortest available name and an accurate one. It collides with +`agentsmd-audit`, which audits a repo's `AGENTS.md` and has nothing to do with the factory. A bare +`audit` alongside it reads as the general case of a skill it is unrelated to, which is precisely the +routing confusion a merge is supposed to reduce. + +**Naming it `forge-audit` (rejected).** It matches the plugin and reads well. `forge` is a live skill +in the same plugin, so `forge-audit` makes one bare skill name a prefix of another — a router asked +to distinguish `forge` from `forge-audit` is being asked to disambiguate on a suffix, and a user +typing `forge` gets an ambiguity that does not exist today. + +**Collapsing the two exported Vale hook IDs into one (rejected).** With a single `vale-wrap.sh` and a +single `.vale.ini`, one hook ID looks sufficient. It is a breaking change to a published hook-repo +contract: any external repo pinning `kyberforge-vale-audit-agent` breaks on upgrade, for no gain. +It also re-creates ADR-0014's measured failure in a new place — that ADR confirmed empirically that a +single hook entry pointed at one config silently scanned 0 files of the other type. Two IDs cost two +manifest stanzas and keep both file scopes explicit. + +## Consequences + +**The single-file `.vale.ini` comes back, and this does not reverse ADR-0014.** ADR-0014 split one +root config into two skill-scoped copies because two skills each needed their own, and no +plugin-level shared directory exists to hold one. Its reasoning is untouched; the merge removes the +condition that reasoning operated on. One skill needs one config, so the union is written back into +one file. **The union is behaviour-neutral and this was checked rather than assumed:** skill-audit's +config has a single `[**/SKILL.md]` section, agent-audit's has `[**/agents/*.md]` and +`[**/*.agent.md]`, and no file in the corpus matches more than one of the three. Where an overlap is +constructible at all (`agents/SKILL.md`), both matching sections assign `BasedOnStyles = Kyberforge`, +so even then no verdict moves. `KyberforgeCopilot` stays scoped to `[**/*.agent.md]` exactly as it is +now, which is what keeps the merged config from widening Copilot-specific rules onto `SKILL.md`. + +**`scripts/check-scope-walkup-sync.sh` survives, and confusing it with `check-vale-style-sync.sh` is +the obvious mistake here.** The two look like the same kind of gate and are not. The walk-up checker +covers **four** independent ports of the scope walk-up, and only two of them live in the audit pair: +the other two are `agent-author/scripts/new-agent.sh` and `skill-author/scripts/new-skill.sh`, which +this change does not touch. They are also Bash where the audit pair's are Python, so as its own header +records, it can never become a text diff — it asserts behavioural agreement across a fixture matrix +instead. Merging two of four ports leaves three ports and the same job. + +**Roughly 71 files carry inbound references to the two skill names and must be re-pointed.** +Derived as `git grep -l -E "skill-audit|agent-audit" | wc -l` — it includes ADRs, `LESSONS.md`, +`docs/spec/gates.md`, both author skills' routing targets, `forge`'s dispatch, the test suite and the +two manifests. Boundary clauses naming `skill-audit` or `agent-audit` are the sharp end: ADR-0020's +resolvable-target check is a blocking ERROR on a dangling route, so a missed rename fails the push +rather than degrading quietly. Historical references inside ADRs describing the pre-merge state stay +as they are; the resolver reads boundary clauses in descriptions, not ADR prose. + +**The dispatch body carries only the gotchas common to both flows, and ships with no SUGGESTION.** An +earlier revision of this change shipped the Gotchas section at **229 of 548 body words, 42%**, +against `GOTCHA_MAX_BODY_FRACTION = 0.25`. It accepted that as standing output, arguing that moving +a gotcha to `references/` meant an extra file read on every invocation. That argument was wrong for +the two gotchas that were over budget, because neither was shared: + +- the `Agent flow, plugin/APM scope only` provider-safety bullet names its one branch in its own + text; +- the 112-word body-word-gate bullet was two separate pre-merge gotchas welded together, a skill half + and an agent half. + +A dispatch body is the dispatch table *plus the gates common to every branch* (CONTEXT.md; the +skill-flow rubric `references/skill-body-discipline.md`). Keeping a single-branch gotcha in it +contradicts that definition. Moving it into its flow file costs no read either, because the body +already loads exactly one flow file on every invocation by construction. + +So the skill half now sits under `## Gotchas` in `references/skill-flow.md`. The agent half and the +provider-safety bullet sit under `## Gotchas` in `references/agent-flow.md`. The body keeps three +gotchas: the no-narration rule, the `disable-model-invocation` exemption and the Vale `0 files` +trap. Measured with `scripts/skill-size-check.sh` thresholds zeroed to force the figures out, the +section is now **91 of 555 body words, 16%**. + +**Every invocation now reads one extra `references/` file.** The dispatch body names the flow file +and the agent loads it, where today Steps 1-3 arrive with the body. This is the cost the progressive- +disclosure trade always carries, and it is paid against a saving: an invocation loads the dispatch +body plus one flow instead of a body that would have to carry both. It is also the reason the +two-section reference file was rejected above. + +**The original audit's figures for this finding were wrong in three ways, and each is worth naming +so the correction is not re-derived from scratch later.** + +- It claimed roughly 3,300 duplicated lines and **two** pre-push hooks. The measured removal is + **2,934 lines and one hook**. The second hook it counted was `check-scope-walkup-sync`, which + survives for the reason above. +- It claimed the two validators were one script hard-wired per mode. They are not. Outside the shared + resolver they total 1,293 lines with 91 distinct lines in common. That error matters because it is + what made the monolithic `validate.sh` look like the obvious implementation. +- It named the **merged `description`** as the blocker on merging. It is not. Merging deletes + description content rather than accumulating it: the `Not a skill directory -> skill-audit` clause + loses its referent, and the `"is this ready to ship"` trigger was duplicated verbatim across both. + The two descriptions it replaces measure **239** (skill-audit) and **250** (agent-audit) at + `61b0b9c^`. The description this skill ships measures **241**, inside the 250 SUGGESTION target. + It carries one arrow per boundary target (`Not applying skill fixes -> skill-author. Not applying + agent fixes -> agent-author.`), because ADR-0020 resolves only the first target after an arrow, so + a one-arrow form would leave `agent-author` checked by nothing. The real blocker was the body: 1,532 + words against `BODY_MAX_WORDS = 900`, with only 211 words shared. Diagnosing the description would + have produced a merge with a concatenated body that failed its own plugin's gate. + + **Correction to an earlier revision of this bullet.** It shipped the description at **319** + characters and accepted the SUGGESTION. It said the excess paid for the second arrow and for + "both flows' artifact-specific trigger phrases carried in full". Only the arrow was worth it. + The trigger phrases stated one trigger twice in two registers: "a skill directory or agent + definition audited", then quoted `audit this skill`, `review my SKILL.md`, `audit this agent` and + `review my agent file`. ADR-0020 makes that a FAIL ("Stating the same trigger twice in two + registers is a FAIL"), so it was not a cost of merging. Dropping the quoted duplicates, and keeping + the one indirect trigger that omits the domain word (`is this ready to ship`), gives 241 with both + arrows kept. The same revision's "240 characters" figure for a hypothetical single-arrow merge was + never reproduced, and is withdrawn rather than re-derived. + +**`factory-audit` shipped at `metadata.version: "1.0.0"`, not ADR-0022's `0.1.0` for a new skill.** +It is a new directory, but not a new skill in the sense ADR-0022's starting version encodes: it +carries every check, rubric and reference of two skills that were both already at `1.0.0`, and +resetting to `0.1.0` would signal an immaturity that the merged content does not have. The fixes +above to Step 0, the gotchas and the description are an improve pass, so under `skill-author`'s +patch-bump rule it is now **`1.0.1`**. The plugin itself goes from `1.6.2` to **`2.0.0`**, because +removing two invocable skills breaks anyone calling them by name. diff --git a/docs/spec/architecture.md b/docs/spec/architecture.md index 453b294..6ef6245 100644 --- a/docs/spec/architecture.md +++ b/docs/spec/architecture.md @@ -79,7 +79,7 @@ This repo also has a `CLAUDE.md` at its root — the Claude Code entry point for ## Reference conventions -The stated convention is that files referencing other files declare those references explicitly: the referencing file carries the forward reference (the content index in `core/AGENTS.md`, `references:` in frontmatter), the referenced file carries a `when:` field describing when it is loaded, and divergence between the two signals staleness. It is aspirational, not a description of the repo today — no file under `core/instructions/` carries frontmatter at all, `when:` appears in exactly one of the 39 `SKILL.md` sources under `plugins/*/.apm/skills/`, and the reference scanner script meant to derive the reverse map ("what files reference this file?") does not exist; `docs/notes/skill-implementation-workflow.md` still lists it as unbuilt work. Treat it as intent for instruction files, skills, and workflow documents, not as a rule the repo enforces. +The stated convention is that files referencing other files declare those references explicitly: the referencing file carries the forward reference (the content index in `core/AGENTS.md`, `references:` in frontmatter), the referenced file carries a `when:` field describing when it is loaded, and divergence between the two signals staleness. It is aspirational, not a description of the repo today — no file under `core/instructions/` carries frontmatter at all, `when:` appears in exactly one of the 38 `SKILL.md` sources under `plugins/*/.apm/skills/`, and the reference scanner script meant to derive the reverse map ("what files reference this file?") does not exist; `docs/notes/skill-implementation-workflow.md` still lists it as unbuilt work. Treat it as intent for instruction files, skills, and workflow documents, not as a rule the repo enforces. ## Provider model diff --git a/docs/spec/gates.md b/docs/spec/gates.md index fdeb5a8..1e69e2a 100644 --- a/docs/spec/gates.md +++ b/docs/spec/gates.md @@ -21,24 +21,24 @@ Install hooks via `pc-run`, wiring **all three stages**. This repo's `.pre-commi `default_install_hook_types`, so a plain install silently skips `commit-msg` (Conventional Commits) and `pre-push` (everything below). -The pre-push command reports **11** hooks, not 9. The extra two are pre-commit's own `meta` hooks, +The pre-push command reports **10** hooks, not 8. The extra two are pre-commit's own `meta` hooks, `check-hooks-apply` and `check-useless-excludes`: they declare no `stages:`, so they run at every stage including this one. Both are declared in this repo's `.pre-commit-config.yaml` like everything -else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Nine +else — what separates them is `repo: meta` (pre-commit's own built-ins) from `repo: local`. Eight is the count of hooks this repo authors itself. -**The caveat: one of those 9 is a silent no-op under that invocation.** +**The caveat: one of those 8 is a silent no-op under that invocation.** `check-release-needed` exits 0 immediately unless `PRE_COMMIT_REMOTE_BRANCH` equals `refs/heads/main`, and pre-commit exports that variable only from the real pre-push git hook during an actual `git push`. Running the stage by hand — or from a CI runner — therefore reports it `Passed` having checked nothing. That is by design for feature branches — pushing WIP must not be blocked on cutting a premature tag — but it means `--hook-stage pre-push --all-files` is a full -rehearsal of 8 hooks and a skip of the ninth. The script's own header records the same gap for +rehearsal of 7 hooks and a skip of the eighth. The script's own header records the same gap for a PR merged through Gitea's merge button, where no local push happens at all. ## The pre-push gate -Nine hooks, grouped below by what they guard rather than by the order `.pre-commit-config.yaml` declares them in. +Eight hooks, grouped below by what they guard rather than by the order `.pre-commit-config.yaml` declares them in. **Core checks** @@ -50,7 +50,6 @@ Nine hooks, grouped below by what they guard rather than by the order `.pre-comm | Hook | Guards | |---|---| -| `check-vale-style-sync` | skill-audit's Vale copy matches agent-audit's canonical copy, plus six glob-coverage probes (see [Vale](#vale)) | | `check-scope-walkup-sync` | `validate.sh`, `validate-provenance.sh`, `new-agent.sh` and `new-skill.sh`'s four independent `$HOME`/`.git`/`apm.yml` walk-up ports still agree behaviorally | | `check-executables-allow-sync` | root `apm.yml`'s `executables.allow` key names kyberforge's actual version (see [apm gates](#apm-gates)) | @@ -61,7 +60,7 @@ drift in generated text. | Hook | Guards | |---|---| -| `check-apm-agents-valid` | runs agent-audit's `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md` (see [Agent files](#agent-files-take-the-description-gates-not-the-body-gate)) | +| `check-apm-agents-valid` | runs `factory-audit`'s `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md` (see [Agent files](#agent-files-take-the-description-gates-not-the-body-gate)) | **apm's own gates** @@ -204,9 +203,9 @@ gets promoted. ### Target resolution walk Resolution walks up **from the file being checked** — never from the script's own location. Deriving -it from `${BASH_SOURCE}` leaked holocron's 39-skill universe into every consumer repo running the -hook through pre-commit, so a consumer skill routing to `skill-audit` resolved against a plugin it -had never installed. +it from `${BASH_SOURCE}` leaked holocron's own skill universe into every consumer repo running the +hook through pre-commit, so a consumer skill routing to a holocron skill (`skill-audit` at the time, +now `factory-audit`) resolved against a plugin it had never installed. The walk finds an **authoring root**: the nearest ancestor holding `plugins/*/.apm/skills` or `plugins/*/.apm/agents`, falling back to the nearest ancestor holding `.git`. **Two passes, not one @@ -354,14 +353,19 @@ findings. ### Duplicated constants -`skill-audit`'s `validate.sh` holds a second copy of the four ADR-0020 constants -(`DESC_SUGGEST_CHARS` / `DESC_MAX_CHARS` / `BODY_SUGGEST_WORDS` / `BODY_MAX_WORDS`), and -`agent-audit`'s `validate.sh` holds a third copy of the two description constants. They are copied -rather than imported because a cache-installed plugin's scripts cannot read files outside their own -plugin directory. `tests/test-skill-size-check.sh` asserts the copies agree, so drift fails CI rather -than silently letting an audit bless a skill the commit hook then rejects. The shared boundary -resolver block is embedded verbatim in all three scripts between `BEGIN`/`END ADR-0020 SHARED -BOUNDARY RESOLVER` markers and must stay byte-identical. +`factory-audit`'s `validate.sh` holds a second copy of the four ADR-0020 constants +(`DESC_SUGGEST_CHARS` / `DESC_MAX_CHARS` / `BODY_SUGGEST_WORDS` / `BODY_MAX_WORDS`) — the two +description constants apply to both artifact types it handles, the two body constants only to +skills. They are copied rather than imported because a cache-installed plugin's scripts cannot read +files outside their own plugin directory. `tests/test-skill-size-check.sh` asserts the copies agree, +so drift fails CI rather than silently letting an audit bless a skill the commit hook then rejects. + +**The shared boundary resolver is now two copies, not three** (ADR-0025). `scripts/skill-size-check.sh` +still carries it embedded between `BEGIN`/`END ADR-0020 SHARED BOUNDARY RESOLVER` markers; the two +plugin copies that used to sit inside `skill-audit`'s and `agent-audit`'s `validate.sh` collapsed +into the single `factory-audit/scripts/lib-boundary-resolver.sh`, sourced by that skill's scripts. +The two remaining copies must still stay byte-identical — a plugin script cannot source the root +one, which is the constraint that forces a copy to exist at all. ### `python3` and PyYAML are hard requirements @@ -387,7 +391,7 @@ fold. ## Agent files take the description gates, not the body gate -`check-apm-agents-valid` runs agent-audit's `validate.sh` over every real +`check-apm-agents-valid` runs `factory-audit`'s `validate.sh` over every real `plugins/*/.apm/agents/*.agent.md`. It derives its expected file set from `git ls-files` — the pattern `tests/run-bats.sh` established — so an agent file deleted from the worktree but still tracked fails the run, and **discovering zero agent files is an error, not a pass**. An untracked *new* agent file @@ -399,12 +403,13 @@ against synthetic `mktemp` fixtures — it had never run against the agent files how ADR-0016 could be amended to bless a `disallowedTools` frontmatter field while `validate.sh`'s allowlist still rejected it: spec and enforcer disagreed and every gate stayed green. -Agents take the ADR-0020 **description** gates (agent-audit's `validate.sh` holds its own copy of +Agents take the ADR-0020 **description** gates (`factory-audit`'s `validate.sh` holds its own copy of those two constants) and, deliberately, **no body word gate**. A skill body is loaded into the caller's context and competes with the live conversation; an agent body becomes the system prompt of a *fresh* context. The rationale for the 900-word FAIL does not transfer. A bats test pins that -absence in agent-audit's validator — adding a body gate there contradicts the ADR rather than fixing -an inconsistency. +absence for the agent path of `factory-audit`'s validator — adding a body gate there contradicts the +ADR rather than fixing an inconsistency. The merge did not change this: the validator auto-detects +the target type, and the body gate applies on the skill path only. **Be precise about the scope of that guarantee: it holds for the *validator*, not for the shared script.** `scripts/skill-size-check.sh` applies its body gate to whatever path it is handed, and @@ -424,7 +429,7 @@ knows the difference; doing so silently enforces a gate ADR-0020 declines to set ## Current retrofit status The ADR-0020 gates ship hot, with no baseline file — a shrinking baseline was considered and -rejected. The corpus is currently clean on both: 0 of 39 descriptions/bodies exceed their FAIL tier, +rejected. The corpus is currently clean on both: 0 of 38 descriptions/bodies exceed their FAIL tier, 0 dangling targets, 0 `Kyberforge.CompositionNote` (Vale) errors. History: issue #99. Nothing is grandfathered — a new skill, or an edit that crosses a FAIL tier, is blocked on first @@ -503,7 +508,7 @@ boundary, and a stricter form would only move the same trust to a different stri in list items, not fences. Those are clause-1 sites the gate cannot see, because it cannot distinguish them from clause-2 mentions in the same list. - **`README.md`, excluded by pattern.** A skill-directory README is consumer-facing prose no agent - loads, and the `git clone https://github.com/bats-core/…` lines in the seven `tests/README.md` + loads, and the `git clone https://github.com/bats-core/…` lines in the six `tests/README.md` files are setup instructions for a third party who has no `rtk`. Prefixing those would be actively wrong, not merely noisy — see ADR-0023's consumer section. - **Quoting.** The line splitter breaks on `;`, `|`, `&&`, `||`, `$(` and backticks without tracking @@ -526,29 +531,92 @@ it was written for. Install the `vale` binary — `brew install vale` (macOS), `snap install vale` (Linux), `choco install vale` (Windows), or see . No `vale sync` is needed: the `Kyberforge` styles are **committed** under -`plugins/kyberforge/.apm/skills/{skill-audit,agent-audit}/assets/vale/styles/`, not downloaded -packages (ADR-0014). +`plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/`, not downloaded packages +(ADR-0014). -### Two copies, one canonical +### One copy, one config -Wiring Vale as a deterministic prefilter for `skill-audit`/`agent-audit`'s Description dimension -(motivation: issue #84) is repo-specific, not part of the generic `lint` plugin, so it does not live -in `plugins/lint/` — and per ADR-0014 it no longer lives at the repo root either. It lives **twice**, -one copy per skill, both under `plugins/kyberforge/.apm/skills/`: +Wiring Vale as a deterministic prefilter for `factory-audit`'s Description dimension (motivation: +issue #84) is repo-specific, not part of the generic `lint` plugin, so it does not live in +`plugins/lint/` — and per ADR-0014 it no longer lives at the repo root either. It lives **once**, +under `plugins/kyberforge/.apm/skills/factory-audit/assets/vale/`, carrying both the `Kyberforge` +and `KyberforgeCopilot` styles and a single `.vale.ini` with all three glob sections: +`[**/SKILL.md]`, `[**/agents/*.md]`, `[**/*.agent.md]`. -| Copy | Styles | `.vale.ini` sections | -|---|---|---| -| `agent-audit/assets/vale/` — **canonical** | `Kyberforge`, `KyberforgeCopilot` | `[**/agents/*.md]`, `[**/*.agent.md]` | -| `skill-audit/assets/vale/` — smaller duplicate | `Kyberforge` | `[**/SKILL.md]` | +ADR-0014 split this into two skill-scoped copies because a plugin's cache-install copies only each +skill's own files and `skill-audit` could not reach across the skill boundary into `agent-audit`'s +copy. Merging the two audit skills removed the boundary, so the copy went with it and the single-file +`.vale.ini` ADR-0014 split apart is restored (ADR-0025). `KyberforgeCopilot` stays scoped to +`.agent.md` files alone, for the Copilot-only "`Use proactively` has no effect" check. -Duplicated rather than shared because a plugin's cache-install copies only each skill's own files — -there is no cross-skill sharing to point at. `check-vale-style-sync` at pre-push is what keeps them -from drifting; `KyberforgeCopilot` is the one deliberate inequality, being scoped only to `.agent.md` -files for the Copilot-only "`Use proactively` has no effect" check. +With one copy there is nothing left to diff, so the `check-vale-style-sync` pre-push hook, +`scripts/check-vale-style-sync.sh` and `tests/test-check-vale-style-sync.sh` are deleted — one hook +off the push gate. **Read what went with it, not just what became vacuous.** The script had 17 +assertion sites. ADR-0025 maps each one; the short version follows. + +**Genuinely moot (6):** + +- the `vale-wrap.sh` diff and the `styles/Kyberforge/` diff, which compared two copies that are now + one; +- the four hard-fail guards that located those copies (`REPO_ROOT` is a directory, the `.apm/` paths + are not stale, neither copy is missing). + +Its `StylesPath` and `BasedOnStyles` checks were **not** diffs. They were per-file greps of each +`.vale.ini`, so they survive: case 0 below checks that the config loads, and case 28 checks that the +`Kyberforge` style is actually loaded. + +**Rehomed or ported (11).** The largest group is the **six-row glob-coverage probe table**, which +invoked `vale --config` on one representative path per file shape. It was the only assertion +anywhere that catches a `.vale.ini` glob typo (`[**/SKILL.md]` → `[**/SKILLS.md]`), the failure mode +where every other check stays clean while Vale lints zero files. One config does not make that +impossible: a typo in any one of the three sections still 0-file-skips that shape. + +**Case 0** runs before any Vale-dependent case and needs no Vale binary. It asserts that the shipped +`.vale.ini` exists and is readable, sets a `StylesPath` that resolves to a directory, and names only +styles that ship. A config that cannot load used to surface as nine generic "vale printed no summary +line" failures across cases 28–31. It now fails once, names the cause, and holds the Vale-dependent +cases back. + +The probes now live in `tests/test-vale-wrap.sh` (cases 28–30), rehomed against the merged config: +one representative path per file shape, each asserted to produce a Vale scan of more than zero files +*and* a Kyberforge alert (case 28). Case 28 also checks that each probe path is in scope of a +published vale hook, and that every `.vale.ini` section has a probe row. Its Part B drops +`Kyberforge` from each section's `BasedOnStyles` in a copy and requires that section's probes to +fail as "style not loaded". Case 29 is a mutation case: it typos each section in a copy of the +assets and requires that section's isolating probes to drop to zero. Case 30 asserts that +`KyberforgeCopilot` reaches `.agent.md` files alone. Its Part B requires both an unload (dropped from +`[**/*.agent.md]`) and a leak (added to `[**/SKILL.md]`) to fail. Case **31** is the third class that went with the +script and is not a glob probe at all: the per-rule override allowlist, which pins every Kyberforge +rule at a blocking bare `YES`/`error`. It is not redundant with the probes above — those key on +`Kyberforge.VagueWording` and `KyberforgeCopilot.ProactivePhrase`, so the other four rules +(`DescriptionOpener`, `PaddingPhrase`, `SentenceOpenerThereIs`, `CompositionNote`) can each be +overridden out of `error` underneath a passing probe. That gap is closed. + +Two cases cover the hook manifests. + +**Case 33** is the original's cross-manifest `files:` drift check, ported. It extracts each vale +hook's `files:` regex from `.pre-commit-hooks.yaml` and from `.pre-commit-config.yaml` +*independently*, compares them per hook and never as a union, and asserts that each shared probe path +is in scope of both or neither. The original selected each hook's record by matching `entry:` +against the owning skill's `vale-wrap.sh` path. After the merge both hook IDs share one `entry:`, so +the port pairs them by `id:` from an explicit table: `kyberforge-vale-audit-skill` ↔ +`vale-audit-prefilter-skill`, and `kyberforge-vale-audit-agent` ↔ `vale-audit-prefilter-agent`. A +missing hook id or a class with no shared probe fails by name. Part B requires three mutations to +fail: the skill hook narrowed to one plugin, the agent hook narrowed the same way, and a renamed +local hook id. + +This was briefly a real hole. Narrowing `vale-audit-prefilter-skill` from `^plugins/[^/]+/...` to +`^plugins/kyberforge/...` left 6 of 38 skills prefiltered, and the whole suite green, before case 33 +existed. + +**Case 32** covers the separate zero-match question on the local manifest alone. Each +`.pre-commit-config.yaml` vale hook's `files:` regex must still match at least one tracked file, and +every path it matches must be in that hook's own artifact class. A hook narrowed to zero files never +runs, and pre-commit reports no error. ### What Vale owns, and what stays LLM judgment -Eleven rule files across the two copies, six distinct rules: +Six rule files, six distinct rules: | Rule | Vale scope | Bans | From | |---|---|---|---| @@ -591,7 +659,7 @@ analogue here — Vale has no tier to make audible. ### External consumers: the root `.pre-commit-hooks.yaml` -The root `.pre-commit-hooks.yaml` exposes both Vale copies (`kyberforge-vale-audit-skill`, +The root `.pre-commit-hooks.yaml` exposes two Vale hook IDs (`kyberforge-vale-audit-skill`, `kyberforge-vale-audit-agent`) plus `kyberforge-skill-size-check`, so any external repo can enforce the same rules with `repo: , rev: ` in its own `.pre-commit-config.yaml`. pre-commit clones the pinned rev into its own cache, independent of whether Claude Code or the @@ -599,8 +667,14 @@ pre-commit clones the pinned rev into its own cache, independent of whether Clau --all-files`. `skill-size-check` has no external asset dependency, so it needed no relocation under ADR-0014 — only exposure. +**The two IDs survive the merge even though they now point at the same wrapper.** Both +`kyberforge-vale-audit-skill` and `kyberforge-vale-audit-agent` keep their IDs and their `files:` +regexes, because an external repo pins them by name in its own `.pre-commit-config.yaml` and +collapsing them to one would break every such consumer silently. What changed is only the `entry:` +target: both now name `factory-audit/scripts/vale-wrap.sh`. + This repo's own `vale-audit-prefilter-skill` / `-agent` hooks consume the **identical** -plugin-bundled copies via `repo: local`. Deliberately not a third root copy, and deliberately **not a +plugin-bundled copy via `repo: local`. Deliberately not a second root copy, and deliberately **not a pinned self-reference** — a pinned self-reference would lint working-tree edits against the last tagged release rather than against the change being made. @@ -619,18 +693,22 @@ vendored research-corpus `SKILL.md` files match neither pattern (see for `skill-size-check`), so prose findings surface only when you edit a file this repo actually authors. Without the binary the hooks fail with a bare "command not found" and no install pointer. -**Two hooks, not one combined hook.** Both manifests split the prefilter in two precisely because a -single hook can point at only one copy, and that copy would silently 0-file-skip the other file -shape (see [A 0-file Vale run is NOT RUN](#a-0-file-vale-run-is-not-run)). +**Two hooks, not one combined hook — for a different reason than ADR-0014 gave.** The original +reason was mechanical: with a config per skill, a single hook could point at only one copy and would +silently 0-file-skip the other file shape (see +[A 0-file Vale run is NOT RUN](#a-0-file-vale-run-is-not-run)). One `.vale.ini` carrying all three +sections removes that constraint. The split stays anyway because the two IDs are an exported +contract external consumers pin by name, and because the `files:` regexes still have to differ — +each hook hands Vale only the file shape it is scoped to. ### The `.vale.ini` globs do no scoping -Each `.vale.ini`'s section globs are **path-agnostic** — `[**/SKILL.md]` for skill-audit's copy, -`[**/agents/*.md]` and `[**/*.agent.md]` for agent-audit's — and constrain filename *shape*, not +The `.vale.ini`'s section globs are **path-agnostic** — `[**/SKILL.md]`, `[**/agents/*.md]` and +`[**/*.agent.md]` — and constrain filename *shape*, not location: Vale's `*` crosses `/`. A `SKILL.md` outside `plugins/` (a project-scope `.claude/skills/foo/SKILL.md`, say) still matches `[**/SKILL.md]` and gets linted normally. -All scoping therefore comes from the pre-commit hook's own `files:` regex and from the audit skills +All scoping therefore comes from the pre-commit hook's own `files:` regex and from `factory-audit` passing one explicit file per invocation. The two manifests scope **differently on purpose**: | Manifest | `-skill` | `-agent` | @@ -638,8 +716,10 @@ passing one explicit file per invocation. The two manifests scope **differently | `.pre-commit-config.yaml` (pins this repo's layout) | `^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$` | `^plugins/[^/]+/\.apm/agents/[^/]+\.agent\.md$` | | `.pre-commit-hooks.yaml` (layout-agnostic for consumers) | `(^\|/)SKILL\.md$` | `(^\|/)agents/[^/]+\.md$\|\.agent\.md$` | -Narrowing a `.vale.ini` glob to a `plugins/`-shaped path to "tighten" it breaks the consumer case, -and `check-vale-style-sync`'s probe set is built to catch exactly that. +Narrowing a `.vale.ini` glob to a `plugins/`-shaped path to "tighten" it breaks the consumer case. +`check-vale-style-sync`'s probe set was built to catch exactly that; it moved to +`tests/test-vale-wrap.sh` with the hook's deletion, and two of the six probes exist specifically to +pin this location independence — see [One copy, one config](#one-copy-one-config). ### The blind spot: `references/` is unlinted, for two independent reasons @@ -647,16 +727,19 @@ Every `references/*.md` file in the corpus is outside the prose gate. Count them `git ls-files | grep -cE '^plugins/[^/]+/\.apm/skills/[^/]+/references/.*\.md$'` rather than reading a figure here; it moves with every retrofit. This is the gap that matters most, because the context contract's own remedy for an over-long body is to move prose **into** `references/` — the gate pushes -text across its own boundary and then stops watching it. +text across its own boundary and then stops watching it. `factory-audit` is the live example. Its +dispatch body keeps only the gotchas common to both flows, and the flow-specific gotchas live under +`## Gotchas` in `references/skill-flow.md` and `references/agent-flow.md` (ADR-0025). Handing both +flow files to `vale-wrap.sh` prints `0 errors … in 0 files` and exits 0. **Closing either cause alone changes nothing.** There are two, and they are independent: | Cause | Where | Effect on a `references/` file | |---|---|---| -| the `Kyberforge` style is scoped `[**/SKILL.md]` | `skill-audit/assets/vale/.vale.ini` | matches no section, so Vale lints 0 files and exits 0 | +| the `Kyberforge` style is scoped `[**/SKILL.md]` | `factory-audit/assets/vale/.vale.ini` | matches no section, so Vale lints 0 files and exits 0 | | the hook's `files:` regex is `^plugins/[^/]+/\.apm/skills/[^/]+/SKILL\.md$` | `vale-audit-prefilter-skill` in `.pre-commit-config.yaml` | the file is never handed to Vale at all | -Verified both ways. Handing skill-audit's `vale-wrap.sh` a reference file directly — bypassing +Verified both ways. Handing `factory-audit`'s `vale-wrap.sh` a reference file directly — bypassing pre-commit entirely, so only the style scope is in play — prints `0 errors … in 0 files` and exits 0, where the same wrapper on a `SKILL.md` reports `in 1 file`. And the hook's `files:` regex, applied to `git ls-files`, selects only the skill-directory `SKILL.md` files scoped at the top of this page, so @@ -672,8 +755,8 @@ The consumer manifest is a third axis and does not rescue this either: `.pre-com ### `vale-wrap.sh`, never bare `vale` -Both audit skills' Step 1 and both pre-commit hooks call **each copy's own** -`scripts/vale-wrap.sh`, not `vale`. It works around a confirmed **Vale 3.15.2** limitation: +`factory-audit`'s Step 1 and both pre-commit hooks call +`factory-audit/scripts/vale-wrap.sh`, not `vale`. It works around a confirmed **Vale 3.15.2** limitation: `text.frontmatter.description` silently stops matching on most — not all — multi-line descriptions. Verified by reproduction on a deliberately-bad fixture, not assumed: @@ -718,8 +801,9 @@ alongside it would resolve against the cwd instead, yielding `E100 Runtime error and exit 2 — which both skills' fallback misreads as "vale unavailable" and silently downgrades to full LLM judgment. -`tests/test-vale-wrap.sh` regression-tests this against **skill-audit's** copy specifically: its -fixtures are all `SKILL.md`-shaped, and only skill-audit's `.vale.ini` carries that glob section. +`tests/test-vale-wrap.sh` regression-tests this against `factory-audit`'s copy — the only one left. +Its fixtures are all `SKILL.md`-shaped, and that copy's `.vale.ini` carries the matching glob section +along with the two agent ones. ### A 0-file Vale run is NOT RUN @@ -736,25 +820,18 @@ clean. ### Pre-push -`vale` is a **pre-push** dependency too, not only pre-commit. `check-vale-style-sync` runs **six -glob-coverage probes** by invoking `vale --config` — one representative path per file shape the -prefilter is supposed to cover. They are the only assertions in the script that catch a `.vale.ini` -glob typo (`[**/SKILL.md]` → `[**/SKILLS.md]`), the failure mode where every text-level check stays -clean while vale lints zero files. As a warning this self-disabled on exactly that mutation and -exited 0, and since pre-commit swallows a passing hook's output the stderr line was never seen — the -hook reported `Passed`. Missing `vale` is therefore a hard failure here. +`vale` is still a **pre-push** dependency, but no longer through a hook of its own. +`check-vale-style-sync` — the hook that ran the six glob probes, and whose +`CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1` opt-out downgraded them audibly rather than skipping the +hook — is deleted with the second Vale copy (ADR-0025). The six glob probes survive it inside +`test-vale-wrap.sh`, so `run-tests --strict` is now the gate that runs them. That is also what keeps +`vale` a pre-push requirement: `test-vale-hooks-consumer.sh` exits 77 without the binary, and so does +`test-vale-wrap.sh` once its static cases pass, and a skip fails the push. -The opt-out is `CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1`, and **it is not `SKIP=`**: the hook -still runs and still asserts everything verifiable from file text, but the six probes do not, and its -summary says so explicitly — - -``` -Vale style sync check passed (text-level only, vale unavailable): … 0 glob probe(s) verified. -``` - -Use it only on a machine that genuinely cannot install `vale`, and read that line as "the glob axis -was not checked", not as a pass. The hook is `verbose: true` for exactly that reason — its clean -output is a single line, so it costs one line per push. +`test-vale-wrap.sh` without Vale skips only its Vale-dependent cases, not the whole suite. The cases +that are plain greps and awk over the config and the two hook manifests still run: case 0, 16, 26, +27, the static halves of 28, 31 Parts A and B, 32 and 33. A static failure exits 1, because a real +defect is not a setup error. Only an all-static-pass run exits 77. ### Mentioning banned phrasing without tripping the rule @@ -780,9 +857,10 @@ run. The pre-push hook invokes the same script as `--strict` (`RUN_TESTS_STRICT= where a skip **does** fail the push: at pre-push a skip means one of the documented dependencies is absent on this machine, so the gate would otherwise report success having run fewer suites than it appears to. Without `--strict` the gate once went green having verified 15 of 17 suites on a -vale-less PATH, with the skip list swallowed. Without vale, three suites skip — -`test-check-vale-style-sync.sh`, `test-vale-hooks-consumer.sh`, `test-vale-wrap.sh` — and the strict -failure names each one and what to install. +vale-less PATH, with the skip list swallowed. Without vale, two suites skip — +`test-vale-hooks-consumer.sh` and `test-vale-wrap.sh` — and the strict failure names each one and +what to install. (It was three until `test-check-vale-style-sync.sh` was deleted with its hook; see +[One copy, one config](#one-copy-one-config).) `tests/run-bats.sh` derives the set of `.bats` files it expects from `git ls-files`, so a `.bats` file deleted from the worktree but still tracked in the index fails the run rather than silently @@ -903,6 +981,9 @@ this remote before any network call. - `docs/adr/0015-apm-replaces-plugin-marketplace-authoring.md`, `docs/adr/0014-vale-prefilter-ships-from-the-plugin.md` — apm-generated manifests, committed Vale styles +- `docs/adr/0025-skill-audit-and-agent-audit-merge-into-factory-audit.md` — the audit-pair merge that + collapsed the two Vale copies to one, removed the `check-vale-style-sync` hook, and took the shared + boundary resolver from three copies to two. It amends ADR-0014 and ADR-0020 on those points - `docs/spec/architecture.md` — directory structure, install pipeline, what is generated and what is hand-authored - `.pre-commit-config.yaml` — the hooks themselves, with inline rationale comments diff --git a/plugins/kyberforge/.apm/skills/agent-audit/SKILL.md b/plugins/kyberforge/.apm/skills/agent-audit/SKILL.md deleted file mode 100644 index 91d3e5d..0000000 --- a/plugins/kyberforge/.apm/skills/agent-audit/SKILL.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -name: agent-audit -description: > - Use when the user wants an agent definition audited — "audit this agent", - "review my agent file", "is this ready to ship" — or after hand-editing an - agent outside agent-author. Not applying fixes -> agent-author. Not a skill - directory -> skill-audit. -allowed-tools: Bash Read -metadata: - version: "1.0.2" - category: factory - source_keys: - - context7-websites-code-claude - - claude-code-plugins-docs - - claude-code-subagents-docs - - context7-github-en-copilot - - github-custom-agents-configuration ---- - -## Gotchas - -- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here. -- Agents take the same 250/400-character description gates as skills and **no body word gate at all** — an agent body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer. Judge an over-long agent body through the delegation check, never by word count. -- At plugin/APM scope the agent is a single vendor-neutral file by design, so provider safety stops meaning Claude-Code-versus-Copilot field leakage there. -- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered. - -## Step 1 — Deterministic checks - -Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly: - -```bash -bash scripts/validate.sh -bash scripts/validate-provenance.sh -bash scripts/vale-wrap.sh [] -``` - -`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes. - -If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, `references/field-inventory.md` missing — read `references/validation-scripts.md`; what these scripts measure is not reproducible by reading. - -`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — a bad argument or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same 2 tier. - -`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment: - -| Rule | Dimension | -|---|---| -| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description | -| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body | - -## Step 2 — Read the agent and load its scope contract - -Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong: - -| Scope | Read | -|---|---| -| plugin/APM | `references/scope-plugin-apm.md` | -| project, user | `references/scope-project-user.md` | - -## Step 3 — Qualitative audit - -Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call. - -| Dimension | Rubric | -|---|---| -| description | `references/description-quality.md` | -| body, delegation, comment-discipline | `references/body-and-delegation.md` | - -Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding. - -## Step 4 — Report - -Open with a coverage line naming every dimension checked. At project/user scope: - -```text -Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance -``` - -At plugin/APM scope, drop `pair-consistency` — there is no pair to check. - -Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed. - -Each finding: - -```text -FAIL/SUGGESTION — file:line - Why: - Fix: -``` - -Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run agent-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/sources.md b/plugins/kyberforge/.apm/skills/agent-audit/references/sources.md deleted file mode 100644 index 7fcca6f..0000000 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/sources.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -source_keys: - - context7-websites-code-claude - - claude-code-plugins-docs - - claude-code-subagents-docs - - context7-github-en-copilot - - github-custom-agents-configuration ---- - -# Sources - -## context7-websites-code-claude - -- **URL:** context7:/websites/code_claude -- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md -- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format -- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md -- **Status:** `extracted` - -## claude-code-plugins-docs - -- **URL:** https://code.claude.com/docs/en/plugins -- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md -- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission -- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/validation-scripts.md -- **Status:** `extracted` - -## claude-code-subagents-docs - -- **URL:** https://code.claude.com/docs/en/sub-agents -- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md -- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations -- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md -- **Status:** `extracted` - -## context7-github-en-copilot - -- **URL:** context7:/websites/github_en_copilot -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace -- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md -- **Status:** `extracted` - -## github-custom-agents-configuration - -- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy -- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md -- **Status:** `extracted` - -## github-cli-plugin-reference - -- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config -- **Contributing files:** (none) -- **Status:** `extracted` - -## github-plugins-creating - -- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle -- **Contributing files:** (none) -- **Status:** `extracted` - -## github-plugins-finding-installing - -- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow -- **Contributing files:** (none) -- **Status:** `extracted` - -## github-plugins-marketplace - -- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands -- **Contributing files:** (none) -- **Status:** `extracted` - -## github-sdk-custom-agents - -- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents -- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md -- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling -- **Contributing files:** (none) -- **Status:** `extracted` diff --git a/plugins/kyberforge/.apm/skills/agent-audit/scripts/README.md b/plugins/kyberforge/.apm/skills/agent-audit/scripts/README.md deleted file mode 100644 index f71640e..0000000 --- a/plugins/kyberforge/.apm/skills/agent-audit/scripts/README.md +++ /dev/null @@ -1,47 +0,0 @@ -# scripts/ - -Executable code bundled with this skill. Agents run scripts in this directory -to perform repeatable operations rather than reinventing the logic each run. - -## When to add a script - -Add a script when agents independently reinvent the same logic across runs — -building the same parser, chart, or validation routine from scratch each time. -Bundle it here once, tested and reliable. - -## Script requirements (agentskills.io) - -Scripts must be designed for non-interactive, agentic execution: - -- **No interactive prompts** — agents run in non-interactive shells. - Accept all input via flags, env vars, or stdin. A script that blocks on - TTY input hangs indefinitely. -- **Expose `--help`** — this is how agents learn your script's interface. - Keep the output concise; it enters the agent's context window. -- **Structured output** — write data (JSON, CSV, TSV) to stdout. - Write progress, warnings, and diagnostics to stderr. -- **Idempotent** — prefer "create if not exists" over "create and fail on - duplicate". Agents may retry on failure. -- **Meaningful exit codes** — `0` for success, non-zero for failure. - Use distinct codes for different failure types; document them in `--help`. -- **Dry-run support** — add `--dry-run` for destructive operations. - -## Self-contained scripts - -Bundle dependencies inline so the agent can run the script with a single command. - -Python (PEP 723 + uv): -```python -# /// script -# dependencies = ["requests>=2.31,<3"] -# requires-python = ">=3.11" -# /// -import requests -``` -```bash -uv run scripts/my-script.py -``` - -## If no scripts are needed - -Delete this README and the `scripts/` directory entirely. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh b/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh deleted file mode 100755 index 738ab4e..0000000 --- a/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh +++ /dev/null @@ -1,1738 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -usage() { - cat < - -Validate an agent definition file against the agent definition spec. - -At plugin/APM scope, is a single vendor-neutral -.apm/agents/.agent.md file with no counterpart. Its frontmatter allowlist -is not restated here: it is read at load time from the apm-agent-allowlist -section of references/field-inventory.md, which is the authoritative list. -At project or user scope, is either half of a Claude Code .md / -Copilot .agent.md pair. - -Arguments: - agent-file Path to the agent file (or either half of a project/user-scope pair). - -Exit codes: - 0 All checks passed (may include SUGGESTIONs) - 1 One or more checks failed - 2 Script error (unrecognized file extension or missing field-inventory.md) -EOF -} - -if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then - usage - exit 0 -fi - -if [[ $# -lt 1 ]]; then - echo "Error: agent-file is required." >&2 - echo "" >&2 - usage >&2 - exit 1 -fi - -# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to -# be measured after YAML folding is resolved, and the hand-rolled reader that -# used to stand in for PyYAML disagreed with it across the 400-character FAIL -# boundary — same description, two verdicts, depending on which reader ran. -# Refusing to start is the only honest option; the repo's jq / apm / vale -# dependencies are declared the same way. -# Check the interpreter separately from the library: `python3 -c` fails the same -# way whether python3 is missing or PyYAML is, and reporting the wrong missing -# dependency sends the reader to install the wrong thing. -if ! command -v python3 > /dev/null 2>&1; then - echo "Error: python3 is required but was not found on PATH." >&2 - echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2 - echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 - exit 1 -fi - -if ! python3 -c 'import yaml' > /dev/null 2>&1; then - echo "Error: PyYAML is required but is not importable by python3." >&2 - echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2 - echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2 - exit 1 -fi - -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - -python3 -u - "$1" "$SCRIPT_DIR" <<'PYTHON' -import sys -import os -import re -import glob - -import yaml - -# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams -# default to ASCII, and this script's own message text carries em dashes (the -# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash -# from the read to the write — a UnicodeEncodeError raised while PRINTING, after -# every check has already run, which loses the whole report and (here) flips a -# clean exit 0 into a traceback and an exit 1. read_text() in the shared -# resolver block below pins the reads; this pins the writes. -# -# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two -# validate.sh copies print findings, skill-size-check.sh has its own top-level -# equivalent, and tests/test-adr0020-contract.sh hashes that block for -# byte-identity across all three. -for _stream in (sys.stdout, sys.stderr): - try: - _stream.reconfigure(encoding='utf-8') - except AttributeError: # pragma: no cover — Python < 3.7 - pass - -agent_file = os.path.abspath(sys.argv[1]) -script_dir = sys.argv[2] - -fname = os.path.basename(agent_file) - -# --- Detect provider (check .agent.md before .md) --- -if fname.endswith('.agent.md'): - provider = 'copilot' - name_stem = fname[:-len('.agent.md')] -elif fname.endswith('.md'): - provider = 'claude-code' - name_stem = fname[:-len('.md')] -else: - print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr) - sys.exit(2) - -# --- Load field-inventory.md --- -inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'field-inventory.md')) -if not os.path.isfile(inv_path): - print(f"Error: field-inventory.md not found at {inv_path}", file=sys.stderr) - sys.exit(2) - -# Encoding is pinned to UTF-8 rather than inherited from the locale: under -# LC_ALL=C the inherited default is ASCII, and this file legitimately carries -# non-ASCII prose. read_text() in the shared resolver block below does the same -# thing for every other file; this one is read before that block is defined. -try: - with open(inv_path, encoding='utf-8') as f: - inv_content = f.read() -except UnicodeDecodeError as exc: - print(f"Error: field-inventory.md at {inv_path} is not valid UTF-8 " - f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.", - file=sys.stderr) - sys.exit(2) - -def parse_section_tokens(content, section_name): - lines = content.splitlines() - for i, line in enumerate(lines): - if line.strip() == f'## {section_name}': - for j in range(i + 1, len(lines)): - stripped = lines[j].strip() - if stripped and not stripped.startswith('#') and not stripped.startswith('---'): - return set(stripped.split()) - return set() - -cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields') -copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields') -apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist') - -# Tools the runtime withholds from subagents regardless of the tools field -SUBAGENT_UNAVAILABLE_TOOLS = { - 'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers', -} - -# Copilot body length limit (chars) — content beyond this is silently truncated -COPILOT_BODY_LIMIT = 30000 - -# ADR-0020 description budget. An agent's name + description is preloaded into -# every session exactly like a skill's, so agents take the SAME description -# gates. These two constants are DUPLICATED from scripts/skill-size-check.sh -# and skill-audit/scripts/validate.sh rather than shared from one file: a -# cache-installed plugin's scripts cannot read files outside their own plugin -# directory, so there is no single source to share (same rationale as -# vale-wrap.sh's per-plugin duplication). tests/test-skill-size-check.sh -# asserts all copies agree, so drift fails CI rather than silently diverging. -# -# Agents deliberately take NO body word gate, and adding one here would -# contradict ADR-0020: a skill body is loaded into the caller's context and -# competes with the live conversation, while an agent body becomes the system -# prompt of a fresh context. The rationale for the 900-word skill ceiling does -# not transfer. Agent body length falls out of the delegation rule instead. -DESC_SUGGEST_CHARS = 250 -DESC_MAX_CHARS = 400 - -# --- Helpers (shared by every scope) --- -failed = False -suggestions = [] - -def fail(msg): - # stderr, matching scripts/skill-size-check.sh's ERROR routing. All three - # scripts in the ADR-0020 family now agree: findings that fail the run go to - # stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo - # callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture - # `2>&1`, so nothing a human reads moves. - global failed - failed = True - print(f"FAIL {msg}", file=sys.stderr) - -def suggest(msg): - suggestions.append(msg) - -def info(msg): - # A check that DECLINED to run says so out loud, rather than passing - # silently. Silence is what let a whole gate family go missing unnoticed. - print(f"INFO {msg}") - -PLACEHOLDER_RE = re.compile(r'(?/plugins/*/ — sibling plugins resolve, -# which is what a monorepo means, -# 2. the target's own apm package, -# 3. the packages that package DECLARES in apm.yml dependencies.apm. -# Deployed .claude/ and .agents/ trees are deliberately NOT consulted when the -# root came from the plugins/ probe. They are `apm install` output, gitignored, -# and present only on a machine that has run it: four cross-plugin targets in -# this repo (gitea-branches -> git-branches, gitea-branches -> git-history, -# gitea-issues -> git-branches, gitea-workflow -> git-workflow) resolved through -# .claude/skills/ alone, so the same commit measured 2 dangling targets on a -# developer machine and 6 on a fresh clone. A gate shipping hot with no baseline -# cannot give two answers. -# -# Deployed trees ARE used when no plugin monorepo was found — whether the walk -# landed on a bare .git ancestor or on nothing at all. That is the consumer -# case: the file being checked lives in or beside a deployed tree, inside an -# ordinary git repo, with no monorepo to read. The two cases are told apart by -# which probe matched, never by how many names a root contributed; see -# known_targets(). - - -def _is_fs_root(path): - return os.path.dirname(path) == path - - -def _collect_package(pkg_dir, names): - """Add every skill/agent name a package directory exposes, any layout.""" - # glob.escape() the DIRECTORY only. A checkout path containing `[`, `]`, - # `*` or `?` — a worktree named `feature[2]`, say — otherwise turns the - # whole pattern into a character class that matches nothing, and the - # resolver degrades to the "DID NOT RUN" INFO with rc=0 across every file - # in the tree. The wildcards in `sub` are the intended ones and stay raw. - safe_dir = glob.escape(pkg_dir) - for sub in ('.apm/skills/*/', 'skills/*/'): - for path in glob.glob(os.path.join(safe_dir, sub)): - # A directory is a skill only if it HOLDS a SKILL.md. An empty - # leftover — a deleted skill whose directory survived, a scaffolding - # stub, an editor's stray mkdir — is untracked by git, so it exists - # on the machine that made it and nowhere else. Counting it made a - # boundary target resolve locally and dangle in a fresh clone: the - # same install-dependence the deployed-tree rule above exists to - # remove, arriving through a different door. - if os.path.isfile(os.path.join(path, 'SKILL.md')): - names.add(os.path.basename(path.rstrip('/')).lower()) - for sub in ('.apm/agents/*.md', 'agents/*.md'): - for path in glob.glob(os.path.join(safe_dir, sub)): - # The same rule one directory over, which until now had no - # counterpart here at all: the skills branch above tests for a - # SKILL.md, the agents branch took every glob hit on trust. A - # DIRECTORY named `ghost-agent.md` matches `*.md` and glob does not - # tell the two apart, so a leftover of that shape resolved a routing - # target on the machine holding it and dangled everywhere else — - # identical install-dependence, arriving through the one door - # nobody guarded. - if not os.path.isfile(path): - continue - base = os.path.basename(path) - if base.endswith('.agent.md'): - base = base[:-len('.agent.md')] - else: - base = base[:-len('.md')] - names.add(base.lower()) - - -def _apm_package_root(start_dir): - """Nearest ancestor that is an apm package root (apm.yml or .apm/). - - The filesystem root is never a candidate: a stray /.apm/skills/ — a - scaffolding test's leftover, say, and one really does exist on at least one - machine here — would otherwise become the package root of every path on it. - Capped at ten levels so a pathological path can't become a filesystem - crawl; that covers every real layout by a wide margin. - """ - current = os.path.abspath(start_dir) - for _ in range(10): - if _is_fs_root(current): - return None - if (os.path.isfile(os.path.join(current, 'apm.yml')) - or os.path.isdir(os.path.join(current, '.apm'))): - return current - current = os.path.dirname(current) - return None - - -def _authoring_root(start_dir): - """Nearest ancestor that is a plugin monorepo, else the nearest .git tree. - - Returns (root, matched_plugins_probe). The flag reports WHICH probe - matched: True for the plugins/*/.apm/{skills,agents} glob, False for the - .git fallback and for no match at all. known_targets() needs that - distinction — only a real plugins/ root makes the deployed trees - redundant, and a name-count delta cannot tell the two apart. - - Two passes, not one interleaved walk: a nested .git (a submodule, a - worktree of a sub-package) must not win over a real plugins/ root further - up. Both passes stop before the filesystem root for the same reason - _apm_package_root does. - """ - probes = ( - lambda d: bool(glob.glob(os.path.join(glob.escape(d), 'plugins', '*', '.apm', 'skills')) - or glob.glob(os.path.join(glob.escape(d), 'plugins', '*', '.apm', 'agents'))), - lambda d: os.path.exists(os.path.join(d, '.git'))) - for index, probe in enumerate(probes): - current = os.path.abspath(start_dir) - for _ in range(12): - if _is_fs_root(current): - break - if probe(current): - return current, index == 0 - current = os.path.dirname(current) - return None, False - - -def _collect_authoring_root(root, names): - """Every plugin in the monorepo contributes its names.""" - for pkg in glob.glob(os.path.join(glob.escape(root), 'plugins', '*')): - if os.path.isdir(pkg): - _collect_package(pkg, names) - - -def _declared_dependency_dirs(pkg_dir): - """Directories of the apm packages pkg_dir's manifest DECLARES. - - Reads dependencies.apm and resolves each entry to a directory on disk: - a monorepo-relative `path:` (against the package root and the nearest - ancestor manifest, which is the monorepo root) or an installed - apm_modules//. Entries that resolve to nothing are skipped — an - undeployed dependency contributes no names rather than an error. - """ - manifest = os.path.join(pkg_dir, 'apm.yml') - if not os.path.isfile(manifest): - return [] - try: - data = yaml.safe_load(read_text(manifest)) or {} - except Exception: - return [] - if not isinstance(data, dict): - return [] - deps = data.get('dependencies') - deps = deps.get('apm') if isinstance(deps, dict) else None - if not isinstance(deps, list): - return [] - - roots = [pkg_dir] - ancestor = os.path.dirname(os.path.abspath(pkg_dir)) - for _ in range(10): - if _is_fs_root(ancestor): - break - if os.path.isfile(os.path.join(ancestor, 'apm.yml')): - roots.append(ancestor) - break - ancestor = os.path.dirname(ancestor) - - found = [] - for entry in deps: - candidates = [] - if isinstance(entry, dict): - rel = entry.get('path') - name = entry.get('name') - if not name and rel: - name = os.path.basename(str(rel).rstrip('/')) - if rel: - candidates.extend(os.path.join(r, str(rel)) for r in roots) - if name: - candidates.append(os.path.join(pkg_dir, 'apm_modules', str(name))) - elif isinstance(entry, str): - name = re.split(r'[#@]', entry)[0].strip().rstrip('/').split('/')[-1] - if name: - candidates.append(os.path.join(pkg_dir, 'apm_modules', name)) - candidates.extend(os.path.join(r, 'plugins', name) for r in roots) - for candidate in candidates: - if os.path.isdir(candidate): - found.append(candidate) - return found - - -def _deployed_roots(start_dir): - """.claude/ and .agents/ trees above start_dir — what a host really sees. - - Consulted ONLY when no plugin monorepo root was found; see the header. The - filesystem root is skipped for the same reason _apm_package_root skips it: - a stray /.claude/skills/ must not join every path's universe. - """ - found = [] - current = os.path.abspath(start_dir) - for _ in range(10): - if _is_fs_root(current): - break - for name in ('.claude', '.agents'): - base = os.path.join(current, name) - if os.path.isdir(base): - found.append(base) - current = os.path.dirname(current) - return found - - -def known_targets(start_dir): - """Every skill/agent name a boundary clause in start_dir may name.""" - names = set() - start = os.path.abspath(start_dir) - - # Siblings: a cache-installed plugin and a deployed .claude/skills/ tree - # both put peers one level up, with no plugins/ directory above them. The - # grandparent is guarded against the filesystem root exactly like the two - # walk-up loops above — for a start dir of /skills/ the grandparent is - # `/`, and collecting there picks up this machine's stray /.apm/skills/. - parent = os.path.dirname(start) - grandparent = os.path.dirname(parent) - if (os.path.basename(parent) in ('skills', 'agents') - and os.path.isdir(parent) and not _is_fs_root(grandparent)): - _collect_package(grandparent, names) - - package = _apm_package_root(start) - if package: - _collect_package(package, names) - for dep_dir in _declared_dependency_dirs(package): - _collect_package(dep_dir, names) - - # A .git ancestor is an authoring root only if it actually holds plugins. - # _authoring_root() falls back to the nearest .git, so it is truthy in ANY - # git repo; without the distinction that fallback wins in every consumer - # checkout, _collect_authoring_root() contributes nothing, and the deployed - # branch below is dead code in the exact case it exists for. So condition - # on WHICH probe matched, which _authoring_root() reports directly. A - # name-count delta looks equivalent and is not: _collect_authoring_root() - # re-collects the checked file's own plugin, whose names the blocks above - # already added, so a one-plugin monorepo shows a delta of zero and would - # wrongly reach for the deployed trees — including the user's global - # ~/.claude/skills, making the verdict depend on what happens to be - # installed (ADR-0020 lines 118-127). - root, root_has_plugins = _authoring_root(start) - if root: - _collect_authoring_root(root, names) - if not root_has_plugins: - for base in _deployed_roots(start): - _collect_package(base, names) - return names - - -# --- Extraction ----------------------------------------------------------- -# False positives are the design constraint here, not recall. The rules: -# * A BARE target must be hyphenated AND sit in a boundary sentence (one -# carrying "do not"/"instead"/"rather than"/"not for"). Without the second -# condition, pc-run's "run pre-commit hooks" reads as a route to a -# non-existent `pre-commit` skill. -# * A BARE arrow target counts only in ADR-0020's compressed boundary form, -# `Not -> `. The example that motivated it is gone: -# diagnose's process chain "fix -> regression-test", which without the -# gate read as a route to a non-existent `regression-test` skill, was cut -# when issue #99 retrofitted that description. So the gate is currently -# UNEXERCISED — gating and not gating produce the same verdict corpus-wide. -# Keep it anyway. It is a false-positive guard against prose no one has -# written yet, and any new process chain re-arms it. Unexercised is not the -# same as unnecessary, and the branch it guards is still load-bearing: the -# bare-arrow rule is the sole extractor for three real targets in -# kyberforge's audit skills (agent-audit -> agent-author, agent-audit -> -# skill-audit, skill-audit -> skill-author), all written unbackticked. -# * A backticked hyphenated token counts only inside a boundary sentence. -# Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard -# FAIL with no escape hatch. Gating it costs nothing (measured over this -# corpus: 54 targets before and after); DELETING it costs 7 real targets -# across three gitea skills, so it is gated, not removed. -# * SINGLE-WORD targets are deliberately NOT matchable bare — `research`, -# `triage`, `forge`, `prototype` and `tdd` are all real skill names and all -# ordinary English, so a bare-word rule would flag most of the corpus. A -# single-word target must be written `` `forge` `` or /forge to be seen. -# That is a known recall limitation, accepted over the false positives. -# Tool names (Read/Write/Edit) are excluded by the lowercase-only pattern; MCP -# tool names (issue_write) by its rejection of underscores; file names by its -# rejection of dots and slashes. -# -# ATTRIBUTIVE USE. The boundary-sentence gate above does NOT solve the -# `pre-commit` false positive, and the comment that claimed it did was wrong: -# "instead", "rather than", "do not" and "not for" are exactly the words a -# boundary clause uses, so the gate is open precisely where the risk is. All of -# these were hard dangling FAILs with no suppression: -# Use pre-commit hooks instead of ad-hoc scripts. -# Invoke the pull-request template instead of writing one by hand. -# Use conventional-commits formatting rather than free-form messages. -# Composes label-resolution logic instead of duplicating it. -# Do not use for X — run the `pre-push` hooks instead. -# What separates every one of them from a real route is grammar, not marking: -# the hyphenated token is a compound MODIFIER of the noun that follows it -# ("pre-commit hooks", "pull-request template"), where a route target is -# terminal — followed by punctuation, a conjunction, or a boundary word. So a -# target whose next token is an ordinary lowercase noun is CONFIRM-ONLY: it -# still resolves and still counts as a route when the name exists, but it can -# never raise a dangling error on its own. -# -# This is deliberately NOT the simpler "only marked targets may dangle" rule, -# which would have been wrong here: BOTH live true positives in this corpus are -# BARE — research's "(use neuledge-context)" and gitea-issues' "Composes -# gitea-labels-\n milestones", where the `>` fold yields "gitea-labels- -# milestones" and the trailing hyphen is what keeps it terminal. Marking is a -# poor proxy, so the follower token is the signal, and it is applied to -# backticked targets too. -# -# TERMINAL IS NOT ENOUGH — IN-SENTENCE CORROBORATION. The follower test clears -# `pre-push` in the example above only because that example happens to be -# followed by the noun "hooks". Move the same token into terminal position and -# it was a hard FAIL again, with no suppression mechanism anywhere in this gate: -# Do not use for running hooks — run `pre-commit` instead. -# Do not use for the commit message — see `commit-msg`. -# Do not use for type errors — run `type-check` first. -# Instead, use `semantic-release`. -# Do not use for the old flow — use the clean-up instead. -# Do not run end-to-end, run unit-tests. -# Every one of those is grammatically identical to a genuinely broken route: -# "route verb + hyphenated name + terminal" is also exactly how prose cites a -# tool, a hook, a file format or an English compound. Nothing local separates -# them, and the skills most exposed are the ones this contract sends authors -# back to rewrite first — pc-run, pc-author, vale-run, vale-config and the apm-* -# family are all ABOUT hyphenated tools. -# -# So the confidence to BLOCK a commit comes from the sentence, not the token: a -# prose-form target may raise a hard error only when its own sentence names at -# least one OTHER target that RESOLVES. A routing sentence proves itself by -# routing somewhere real; a lone unresolvable name proves nothing. That is not a -# rule fitted to the fixtures — it is the shape of both live true positives, -# which sit beside `write-docs` and `gitea-labels-milestones` respectively, and -# it changes this corpus's verdict by exactly nothing. -# -# An uncorroborated unresolvable target is NOT discarded: every caller reports -# it at its SUGGESTION tier, naming the target. The finding stays visible on -# every run; only the power to block a commit is withdrawn, which is the part -# that had no escape hatch. -# -# EXPLICIT ROUTE NOTATION is exempt from corroboration and always blocks: -# ADR-0020's compressed arrow (`Not -> `) and Claude Code's -# invocation form (`/`). Neither is ever how English cites a tool — nobody -# writes `-> pre-commit` or `/pre-commit` to mean the hook — so there is no -# ambiguity to resolve, and an author who wants a route checked unconditionally -# has two ways to say so. -# -# BOTH FORMS ARE SWEPT FOR ON THEIR OWN, and that is a repair of the promise -# above rather than a widening of it. Until the sweeps existed, notation was -# only ever seen as the OBJECT OF A ROUTE VERB (`use -# /name`) or as the tail of a `not ... ->` clause with no `;` or sentence end in -# between. Every one of these therefore exited 0 in total silence — no ERROR, no -# SUGGESTION, not even the target's name: -# Do not use for Y — /no-such-skill instead. -# Do not use for Y; /no-such-skill handles that. -# Do not use for Y (/no-such-skill covers it). -# Do not use for Y — that is /no-such-skill's job. -# Do not use for Y — defer to /no-such-skill. -# Do not use for Y — /no-such-skill. -# Do not use for Y; -> no-such-skill covers it. -# For W, /no-such-skill is the right entry point. -# The target was never EXTRACTED, so the notation-first rule in _add() had -# nothing to apply itself to and the "always blocks" promise was false for the -# ordinary way an author writes the thing. The SUGGESTION tier made it worse -# than a gap: its printed remedy tells the author to "write it as `/name` or -# `-> name` and it will be checked properly", and taking that advice turned a -# visible SUGGESTION into silence — the gate teaching the one edit that blinds -# it. -# -# THE TWO SWEEPS ARE GATED DIFFERENTLY, and the asymmetry is the whole point. -# `/name` is Claude Code's invocation syntax and nothing else — no English -# sentence contains one by accident — so the ADR-0020 amendment and -# docs/spec/gates.md both promise it blocks UNCONDITIONALLY, for any name. So -# NOTATION_SLASH is swept over every sentence, boundary marker or not. Gating it -# on BOUNDARY_MARKER made that promise false for the last sentence of -# Do not use for Z — use /real-skill instead. -# For W, /no-such-skill is the right entry point. -# which exited 0 in total silence: the boundary clause is one sentence up, so -# the sweep never looked at the sentence carrying the broken route. Extraction is -# per-sentence by design (corroboration is scoped to one sentence), which is -# exactly what made the gap invisible. -# -# NOTATION_ARROW stays gated on BOUNDARY_MARKER, and so does the backtick sweep. -# Neither form is unambiguous: `-> name` is also how a process chain is written -# ("reproduce -> minimise -> regression-test") and a code span is how a tool, a -# file and a skill are all cited. Ungating either would fire on prose that -# carries no routing intent at all — the false-positive class this whole -# extractor is tuned against. -# -# BOTH `/name` PATTERNS REFUSE A TOKEN THAT IS PART OF A PATH: a following `/`, -# or a `.` followed by a non-space, means `references/foo.md`, `docs/a/b.md` or -# `https://x/y`, not a route. A sentence's closing `.` is not followed by a -# non-space, so `— /no-such-skill.` still counts. -# -# THAT GUARD IS WRITTEN `(?![\w-])` AND NOT `\b`, because `\b` is not a guard at -# all here: it holds after a hyphen, so when the trailing lookahead rejected the -# full segment the engine simply backtracked to a shorter hyphen-terminated -# prefix and reported THAT as a route. Every one of these was a hard blocking -# ERROR naming a skill nobody had written: -# the config lives at /opt-tools/bin/thing. -> 'opt' -# see /api-docs/v2.md for the schema. -> 'api' AND 'api-docs' -# the file /no-such-skill.md documents it. -> 'no-such' -# `(?![\w-])` forbids the shortened prefix outright, so the whole segment is -# rejected as the path it is. MARKED_TARGET carries the same guard: it had no -# trailing lookahead whatsoever, so `see /api-docs/v2.md` raised the second of -# the two errors above through the route-verb path rather than the sweep. -# -# NAMESPACE: `plugin:skill` is live in this repo (native user-scope installs -# still resolve `gitea:gitea-prs`), so the patterns admit an optional -# `:` prefix and normalize_target() strips it before resolution. -NS = r"(?:[a-z0-9]+(?:-[a-z0-9]+)*:)?" -NAME_ANY = NS + r"[a-z0-9]+(?:-[a-z0-9]+)*" -NAME_HYPH = NS + r"[a-z0-9]+(?:-[a-z0-9]+)+" -ROUTE_VERB = (r"(?:use|uses|using|run|runs|invoke|invokes|invoking|try|see" - r"|that'?s|compose|composes|call|calls" - r"|routes?\s+to|delegates?\s+to|prefers?|switch(?:es)?\s+to" - r"|hands?\s+off\s+to)") -MARKED_TARGET = (r"(?:`/?(%s)`|(?|→)\s*%s" % MARKED_TARGET, re.I) -# The two EXPLICIT ROUTE NOTATION sweeps. NOTATION_SLASH runs over EVERY -# sentence; NOTATION_ARROW is scoped to a boundary sentence by its caller (see -# the asymmetry note in the header). NOTATION_SLASH is deliberately not a reuse -# of MARKED_TARGET's `/name` alternative: that one only ever runs behind a route -# verb or an arrow, and it may match a namespaced or path-adjacent token in -# positions this free-standing sweep must refuse. -# NOTATION_ARROW is ARROW_BOUNDARY minus its leading `\bnot\b%s*?`, which is -# what made `Do not use for Y; -> no-such-skill covers it.` invisible: -# CLAUSE_BODY cannot cross the `;`, so the clause's own punctuation disarmed the -# check. Dropping that prefix costs the one false positive the bare-arrow bullet -# above names — a process chain ending in a hyphenated word, `Instead, reproduce -# -> minimise -> regression-test.` — and costs it only in a sentence that already -# carries a BOUNDARY_MARKER. That exposure is neither new nor larger: the same -# chain written `Do not use for X — reproduce -> regression-test.` was already a -# hard ERROR under ARROW_BOUNDARY, so this changes which boundary words reach the -# arrow, not whether prose can. An author who means the chain and not a route -# writes it in its own sentence, where neither pattern looks. -NOTATION_SLASH = re.compile( - r"(?|→)\s*(%s)\b" % NAME_HYPH, re.I) -# CLAUSE_BODY is what may sit between `Not` and the arrow, and it is NOT -# `[^.;]`. That class cannot cross a `.`, so every boundary clause naming a -# DOTTED FILENAME between the two — `.pre-commit-config.yaml`, `AGENTS.md`, -# `.vale.ini` — was invisible to both patterns below, and the two resulting -# failures were different sizes (issue #110): -# * with a BACKTICKED target the clause was MISDIAGNOSED. The backtick sweep -# still extracted the target, so the route was checked, but the gate -# reported "no boundary clause" on a clause that was present and working. -# Three authors in two retrofit waves reworded a correct clause to satisfy -# the regex, one of them stripping the very filename that discriminates the -# skill from its neighbour. -# * with a BARE target the clause was UNCHECKED. ARROW_BOUNDARY is the only -# extractor for a bare arrow target, so `Not AGENTS.md -> no-such-skill` -# produced no target, no dangling report and no missing-clause SUGGESTION. -# Silence, not noise — the worse of the two failure modes. -# A dot inside a filename is followed by a non-space; a sentence-ending dot is -# followed by whitespace or by end of string. So the class admits a `.` only -# when the next character is not whitespace, which crosses `AGENTS.md` and -# still stops at a real sentence end. -CLAUSE_BODY = r"(?:[^.;]|\.(?=\S))" -ARROW_BOUNDARY = re.compile( - r"\bnot\b%s*?(?:->|→)\s*(%s)\b" % (CLAUSE_BODY, NAME_HYPH), re.I) -BACKTICK = re.compile(r"`(%s)`" % NAME_HYPH, re.I) -# A boundary clause takes two shapes and BOTH count: the prose markers, and -# ADR-0020's compressed arrow form `Not -> `. -BOUNDARY_MARKER = re.compile(r"\b(?:do\s+not|instead|rather\s+than|not\s+for)\b", re.I) -BOUNDARY_ARROW = re.compile(r"\bnot\b%s*?(?:->|→)" % CLAUSE_BODY, re.I) -# Sentence boundaries decide the CORROBORATION scope above, so getting one wrong -# is not cosmetic — it moves a target between SUGGESTION and blocking ERROR. Two -# shapes common in these descriptions defeat the naive "period, space, capital" -# rule, in OPPOSITE directions: -# OVER-SPLIT. `e.g. "set up the manifest"` ends no sentence, but the quote -# looks like one starting. The clause is cut in half, the corroborating -# target lands on the far side of the cut, and a genuinely dangling target -# silently demotes to SUGGESTION — the gate takes a measurement and then -# throws it away, which is the vacuous-green shape this file exists to stop. -# UNDER-SPLIT. A real sentence opening with a code span or a lowercase skill -# name ("... Composes it. `gitea-prs` also uses it.") is not seen as a start -# at all, so two sentences merge and a resolving target vouches for an -# unresolvable one it never stood beside — a hard FAIL with no escape hatch, -# which is exactly the failure the corroboration rule was added to prevent. -# Both are closed here: the five abbreviations that actually occur in routing -# prose are excluded as sentence ends, and the opener class admits a backtick or -# a lowercase letter. Verified zero-delta on the current corpus (37 ERROR / 58 -# SUGGESTION / 2 dangling before and after) — this protects the descriptions -# issue #99 is about to rewrite, not the ones already measured. -# re.I here too, and NOT as a tidy-up: this was the one pattern in the file -# built without it, contradicting the uniformity note on CONT_*/ARROW_* above. -# Without the flag `E.g.` and `I.e.` — the sentence-initial spellings, which is -# where an abbreviation most often lands — matched none of the lookbehinds, so -# the clause split at the abbreviation, the corroborating target was stranded on -# the far side of the cut, and a genuinely dangling target silently demoted from -# blocking ERROR to SUGGESTION. That is the OVER-SPLIT failure described -# directly above, still live for exactly the capitalised half of the input. -SENTENCE_SPLIT = re.compile( - u'(? name` (ADR-0020's compressed boundary - form, passed in by the caller that matched the arrow). A backticked name - does NOT qualify — a code span is how a tool, a file and a skill are all - cited, so it carries no intent the follower test hasn't already read. - """ - return arrow or (start > 0 and text[start - 1] == '/') - - -def _add(out, text, name, start, end, strict=None, arrow=False): - """Record one target as (name, may_dangle, notation). - - NOTATION IS DECIDED FIRST, and when it is set the follower test is skipped. - The header above promises that route notation "always blocks", and for the - `/name` form that was false: `-> name` reached this function with - strict=True from its two call sites, but `/name` did not, so it fell to - _terminal() and a follower outside FOLLOWER_OK set may_dangle=False. The - target then reached unresolved_targets() unblockable — and, before the - companion fix there, unreported as well. `... use /no-such-skill - afterwards.` exited 0 in total silence, on the one form ADR-0020 offers an - author who wants a route checked unconditionally. - """ - if not name: - return - notation = _notation(text, start, arrow) - if strict is None and notation: - strict = True - out.append((name, - _terminal(text, end) if strict is None else strict, - notation)) - - -def _scan(text, route_re, cont_re, out): - for match in route_re.finditer(text): - name, start, end = _first(match) - if not name: - continue - _add(out, text, name, start, end) - # "use git-history or git-branches instead" / "use gitea-issues / - # gitea-prs" — keep consuming conjoined targets after the first. - pos = match.end() - while True: - cont = cont_re.match(text, pos) - if not cont: - break - _add(out, text, *_first(cont)) - pos = cont.end() - - -def _extract_sentence(sentence): - """[(name, may_dangle, notation)] for the routing targets in ONE sentence. - - Kept separate from _extract() because corroboration is scoped to a single - sentence: a target's evidence is what stands beside it, not what the rest of - the description happens to mention. - """ - out = [] - boundary = bool(BOUNDARY_MARKER.search(sentence)) - _scan(sentence, - ROUTE_ANY if boundary else ROUTE_MARKED, - CONT_ANY if boundary else CONT_MARKED, - out) - for match in ARROW_MARKED.finditer(sentence): - # `-> name` and `-> /name` are route notation, not prose: nothing - # reads as a compound modifier after an arrow, so no follower test. - _add(out, sentence, *_first(match), strict=True, arrow=True) - for match in ARROW_BOUNDARY.finditer(sentence): - _add(out, sentence, match.group(1), match.start(1), match.end(1), - strict=True, arrow=True) - # `/name` wherever it sits, in ANY sentence — not only where a route verb or - # an arrow happens to precede it, and NOT only inside a boundary sentence. - # See the EXPLICIT ROUTE NOTATION note in the header for the eight phrasings - # this recovers and for why silence was the failure mode. The sweep takes no - # follower test: _add() reads the notation first and marks it. - for match in NOTATION_SLASH.finditer(sentence): - _add(out, sentence, match.group(1), match.start(1), match.end(1)) - if boundary: - # The arrow and backtick forms are ambiguous in ordinary prose, so they - # stay scoped to a sentence that carries a boundary marker. - for match in NOTATION_ARROW.finditer(sentence): - _add(out, sentence, match.group(1), match.start(1), match.end(1), - strict=True, arrow=True) - for match in BACKTICK.finditer(sentence): - _add(out, sentence, match.group(1), match.start(1), match.end(1)) - return out - - -def _extract(description): - """[(name, may_dangle, notation)] for every routing target.""" - out = [] - for sentence in SENTENCE_SPLIT.split(description): - out.extend(_extract_sentence(sentence)) - return out - - -def boundary_targets(description): - """Every routing target, for reporting and for confirming a route.""" - return sorted({name for name, _, _ in _extract(description)}) - - -def _arrow_targets(description): - """Names extracted from ARROW notation specifically. - - Kept apart from boundary_targets() because the arrow form is the one shape - that ALWAYS names a target: ADR-0020's `Not -> `. A clause - written that way from which nothing could be extracted is a parse failure - that deserves its own message, and telling it apart needs the arrow targets - alone rather than every target in the description. - """ - out = [] - for sentence in SENTENCE_SPLIT.split(description): - for match in ARROW_MARKED.finditer(sentence): - name, _, _ = _first(match) - if name: - out.append(name) - for match in ARROW_BOUNDARY.finditer(sentence): - out.append(match.group(1)) - return out - - -def boundary_clause_status(description): - """'absent', 'unparsed' or 'present' — three outcomes, not two. - - Issue #110's standing request: the gate must distinguish "no boundary - clause" from "boundary clause I could not parse". Reporting the first for - the second sends the author hunting for a problem that is not there, and - three of them reworded a correct clause to satisfy a regex instead. - - 'unparsed' is the narrow, certain case: an ADR-0020 arrow clause was - detected and NO target came out of it. The arrow form always names one, so - zero targets means the name is written in a shape the extractor cannot see - — a single-word bare target (`Not X -> forge`, which has to be written - `` `forge` `` or `/forge`) is the live example, since single-word names are - deliberately not matchable bare. - - A PROSE clause yielding no target is NOT reported: "Do not use for anything - else" is a complete and legitimate boundary clause that names nowhere to go. - """ - if BOUNDARY_ARROW.search(description) and not _arrow_targets(description): - return 'unparsed' - if has_boundary_clause(description): - return 'present' - return 'absent' - - -def multi_target_arrow_clauses(description): - """[(first, second)] for arrow clauses naming more than one target. - - Issue #107: only the FIRST target after an arrow is resolved. The - conjunction continuation (CONT_*) is wired to the prose route verbs and - never to arrows, so `Not X -> a or b` resolved `a`, left `b` neither - resolved nor reported, and then printed "1 of 1 boundary target(s) resolve" - on a clause naming two — a gate under-reporting its own coverage, which is - the one failure mode ADR-0020 says a gate must not have. - - The clause is REJECTED rather than the arrow scan extended. Extending it - would widen the resolver's deliberately conservative false-positive tuning - across every arrow in the corpus; rejecting costs nothing and makes the - one-arrow-per-target convention — already what every retrofitted gitea - skill does in practice — explicit instead of folkloric. The caller emits a - SUGGESTION telling the author to split. - """ - hits = [] - for sentence in SENTENCE_SPLIT.split(description): - matches = (list(ARROW_MARKED.finditer(sentence)) - + list(ARROW_BOUNDARY.finditer(sentence))) - for match in matches: - first, _, _ = _first(match) - if not first: - continue - cont = CONT_ANY.match(sentence, match.end()) - if not cont: - continue - second, _, _ = _first(cont) - if second: - hits.append((first, second)) - return hits - - -def unresolved_targets(description, known): - """Targets resolving to nothing, split into (blocking, reported). - - `blocking` earns a hard error; `reported` is SUGGESTION tier — named on - every run, never fatal. Three conditions gate the promotion, and all of them - are documented at length in the ATTRIBUTIVE USE and CORROBORATION notes - above: - - 1. the target must be terminal, not a compound modifier ("pre-commit - hooks" is prose about a tool, not a route), - 2. it must be written in route notation (`/name`, `-> name`), OR - 3. its own sentence must name another target that DOES resolve. - - Everything else is reported and left alone. `known` is the resolved - universe from known_targets(); passing an empty set is not meaningful — - callers check for that first and decline out loud instead. - - A NON-TERMINAL target is reported, never dropped. FOLLOWER_OK is a closed - whitelist of maybe eighty words, so the follower rule says "this token is - outside a list I keep" and not "this is prose" — and the old `continue` - turned that into invisibility at every tier. The gate then failed OPEN on - its own unfamiliarity: any target followed by a word nobody thought to - enumerate was neither blocked nor mentioned, so the check that did not run - said nothing about not running. The follower rule may withdraw the power to - BLOCK a commit — that is what it was added for, and the ATTRIBUTIVE USE note - above is the argument for it — but it may not withdraw visibility, which is - the same rule the corroboration tier already follows. - """ - blocking, reported = set(), set() - for sentence in SENTENCE_SPLIT.split(description): - found = _extract_sentence(sentence) - resolved = {normalize_target(name) for name, _, _ in found - if normalize_target(name) in known} - for name, may_dangle, notation in found: - key = normalize_target(name) - if key in known: - continue - if not may_dangle: - reported.add(name) - continue - if notation or (resolved - {key}): - blocking.add(name) - else: - reported.add(name) - return sorted(blocking), sorted(reported - blocking) - -# --- Frontmatter ---------------------------------------------------------- -# Tolerant on the way in, HARD-FAILING on the way out. A UTF-8 BOM, a leading -# blank line, trailing whitespace after either `---`, or CRLF line endings all -# defeated the old `^---\n(.*?)\n---`, and the miss was SILENT: every ADR-0020 -# check was skipped and the file reported green (measured: a 550-character -# description with a 1,000-word body exited 0 behind a BOM). A file that cannot -# be measured must never report green, so every caller of these two ERRORs on a -# miss instead of moving on. -# -# The CLOSING marker is anchored at column 0 — deliberately NOT `[ \t]*---`. -# YAML block-scalar content must be indented deeper than its key, so an -# indented `---` inside a folded description is CONTENT; letting it close the -# frontmatter truncated the description mid-value and silently reclassified the -# rest as body, which is a vacuous green in both directions at once. Leading -# whitespace is still tolerated on the OPENING marker, where no such content -# can exist. -FRONTMATTER_RE = re.compile( - r'^[ \t\r\n]*---[ \t]*\r?\n(.*?)\r?\n---[ \t]*(?:\r?\n|\Z)', re.DOTALL) - - -def strip_bom(text): - return text[1:] if text.startswith(u'') else text - - -class FrontmatterError(Exception): - pass - - -def description_value(fm_text): - """The description VALUE, with YAML folding resolved. - - PyYAML is a HARD requirement, preflighted in bash. The hand-rolled fallback - this replaced diverged from a real parser across the FAIL boundary — one - corpus description measured 270 characters parsed and 412 unparsed, and a - quoted `"description"` key or an explicit `description: null` returned empty - from it, silently skipping the description AND routing checks. A gate that - disagrees with itself depending on which reader ran is worse than no gate. - - This is the ONLY reader any of the three scripts may use to decide whether a - description is present. A line regex cannot: `description:` with no value - followed by `model: sonnet` lets `\\s*` cross the newline and captures the - NEXT key, which reads as a non-empty description, skips the "missing or - empty" failure, and then early-returns out of every ADR-0020 gate on the - genuinely empty folded value. That combination exited 0 with zero output on - a BLOCKING pre-push gate. - """ - try: - data = yaml.safe_load(fm_text) - except Exception as exc: - # Every FrontmatterError message is a COMPLETE clause, never a detail a - # caller wraps in one. Callers used to prefix a hard-coded "frontmatter - # is not valid YAML (...)", which is true only of this branch: the two - # type failures below come from frontmatter that parsed fine, and - # telling their author the YAML is invalid sends them hunting for a - # syntax error that is not there — on a blocking gate with no baseline. - raise FrontmatterError('frontmatter is not valid YAML (%s)' - % re.sub(r'\s+', ' ', str(exc)).strip()) - if not isinstance(data, dict): - raise FrontmatterError('frontmatter is not a YAML mapping') - value = data.get('description') - if value is None: - return '' - if not isinstance(value, str): - # NOT str()-coerced. `description: true` became the 4-character "True" - # and sailed through the 400-character gate; a list or mapping was - # measured as its Python repr. Neither is a description a host can - # preload, so this is a parse failure, reported as one. - raise FrontmatterError( - 'description is a %s, not a string' % type(value).__name__) - return re.sub(r'\s+', ' ', value).strip() - - -def hand_invoked(fm_text): - """True when the frontmatter marks this file as reached only by hand. - - `disable-model-invocation: true` removes a skill from the model-visible - listing entirely — it is not preloaded, and the Skill tool refuses to call - it — so its description is never matched against user intent. ADR-0020 and - skill-author's contract give such a skill ONE plain human-facing sentence: - no trigger list, no boundary clause. No validator knew the field existed - (issue #108), so the boundary-clause SUGGESTION fired on exactly the shape - the contract mandates, and its remedy — "add a boundary clause so the router - knows where NOT to send this skill" — was addressed to a router that cannot - see the skill at all. An author who followed the advice made the file worse. - - Only the ROUTING rules are lifted. The body word budget still applies: the - body is loaded on invocation like any other, and competes with the caller's - live conversation the same way. So does the 400-character description FAIL — - a hand-invoked description is not preloaded, but it is still the one line - the user reads when choosing from the `/` menu, and the ceiling is the - outlier stop rather than the style target. - - A parse failure returns False rather than raising. This is a MODIFIER on - other checks, not a check of its own: the frontmatter's validity is decided, - and failed, by description_value() on the same text, and raising a second - exception here would report one broken file twice with two different - diagnoses. - """ - try: - data = yaml.safe_load(fm_text) - except Exception: - return False - if not isinstance(data, dict): - return False - value = data.get('disable-model-invocation') - if isinstance(value, str): - # PyYAML already resolves the unquoted YAML 1.1 booleans, so this only - # catches a QUOTED "true" — which a host reads as truthy and which no - # gate should treat as opting back in to the routing rules. - return value.strip().lower() in ('true', 'yes', 'on') - return value is True - - -# --- Body-shape checks (skills only; agents have no references/ dir) ------- -# Deterministic and countable, so they are enforced here. Whether a given -# gotcha is WARRANTED is semantic and stays the auditor's judgment, which is why -# both gotcha checks are SUGGESTION tier. A missing reference file is not a -# style opinion — it is a broken pointer — so that one is ERROR tier. -# -# Both read a FENCE-MASKED copy of the body. Scanning the raw body made a -# ```-fenced example a hard ERROR — and the skills most likely to carry one are -# skill-author and skill-audit, which DOCUMENT the references/ convention — and -# let a `## Gotchas` heading inside a fenced block stand in for the real -# section. Masking preserves every byte offset (content becomes spaces, -# newlines stay), so a span found in the mask slices the original. -GOTCHA_MAX_ENTRIES = 5 -GOTCHA_MAX_BODY_FRACTION = 0.25 -# The heading has to BE "Gotchas", not merely contain the word: `## Gotcha -# handling` and `## Why gotchas matter` are prose sections, and treating one as -# the Gotchas section measured a span that was never a gotcha list. -GOTCHA_HEADING = re.compile(r'^(#{1,6})[ \t]+(?:[^\n]*?[ \t])?gotchas?[ \t]*:?[ \t]*$', - re.I | re.M) -# Column 0 only. `^[ \t]{0,3}` counted a two-space-indented CHILD bullet as a -# top-level entry, so a five-entry section with sub-bullets reported nine. -GOTCHA_ENTRY = re.compile(r'^(?:[-*+]|\d+[.)])[ \t]+', re.M) -FENCE_OPEN = re.compile(r'^[ \t]{0,3}(`{3,}|~{3,})') -REFERENCE_POINTER = re.compile( - r'(?= len(fence) - and not stripped.strip()[len(marker):].strip()): - fence = None - # An UNCLOSED fence has no cost-free answer, only a choice of which way to - # be wrong. Masking to end-of-body blanks the rest of the body, silently - # disabling the ERROR-tier references/ check and the gotcha counts. - # Returning the raw text instead exposes the unclosed example's own - # content, so a fenced example naming a nonexistent references/ file - # becomes a hard ERROR it would not have been had the fence been closed — - # confirmed, not hypothetical. The loud-false-positive direction is the one - # chosen: this script's rule is that a file it cannot measure must never - # report green, and masking-onward is exactly that failure. Both outcomes - # need an already-malformed file, and the false positive costs one fence. - if fence is not None: - return text - return ''.join(out) - - -def gotcha_stats(body): - """(entry count, section word count) for the first Gotchas section, or None. - - The section runs to the next heading at the same level or shallower. - Entries are top-level list items; a section written as subheadings instead - of a list counts those. Headings and entries are read from the fence mask; - the word count is taken from the original slice, because fenced lines are - real body words and the fraction is measured against the whole body. - """ - masked = mask_fenced(body) - match = GOTCHA_HEADING.search(masked) - if not match: - return None - level = len(match.group(1)) - rest = masked[match.end():] - nxt = re.search(r'^#{1,%d}[ \t]+' % level, rest, re.M) - end = match.end() + (nxt.start() if nxt else len(rest)) - section = masked[match.end():end] - entries = len(GOTCHA_ENTRY.findall(section)) - if entries == 0 and level < 6: - entries = len(re.findall(r'^#{%d,6}[ \t]+' % (level + 1), section, re.M)) - return entries, len(body[match.end():end].split()) - - -def missing_reference_pointers(body, skill_dir): - """references/.md named in the body but absent from disk.""" - masked = mask_fenced(body) - missing = set() - for match in REFERENCE_POINTER.finditer(masked): - start = masked.rfind('\n', 0, match.start()) + 1 - end = masked.find('\n', match.end()) - if end < 0: - end = len(masked) - # The pointer's OWN SPAN is excised before the sweep. Run over the - # whole line, the past-tense test matched the very path it was judging, - # so a file exempted itself by its NAME: `references/deprecated-api.md`, - # `references/removed-flags.md` and `references/gone.md` produced no - # ERROR at all, while `references/missing.md` — an identical break — - # errored. The exemption is about what the SENTENCE says about the - # pointer, never about what the pointer is called. - line = masked[start:match.start()] + masked[match.end():end] - if REFERENCE_PAST.search(line): - continue - if REFERENCE_QUALIFIER.search(masked[start:match.start()]): - continue - if not os.path.isfile(os.path.join(skill_dir, 'references', match.group(1))): - missing.add('references/' + match.group(1)) - return sorted(missing) -# ===== END ADR-0020 SHARED BOUNDARY RESOLVER ===== - - -def parse_frontmatter(content): - m = FRONTMATTER_RE.match(strip_bom(content)) - if not m: - return None, content - return m.group(1), strip_bom(content)[m.end():] - -def extract_field(fm, field): - """The raw text after `field:` ON ITS OWN LINE, or None. - - The character class is `[^\\S\\r\\n]`, never `\\s`: under re.MULTILINE a - `\\s*` after the colon crosses the newline, so `description:` with no value - followed by `model: sonnet` captured `model: sonnet` as the description. - That made the value look present, skipped the "missing or empty" failure, - and then every ADR-0020 gate early-returned on the genuinely empty folded - value — a valueless description exited 0 with zero output on a BLOCKING - pre-push gate. This function is now used only for fields with no folding - semantics (name, tools); description goes through description_value(), the - shared resolver's YAML reader, which is the only thing that can see through - `>`, `null`, `''` and a quoted `"description"` key alike. - """ - m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE) - return m.group(1).strip() if m else None - -def get_frontmatter_keys(fm): - keys = set() - for line in fm.splitlines(): - m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line) - if m: - keys.add(m.group(1)) - return keys - -def agent_description(fm, local_fname): - """The folded description VALUE, or None if it could not be read.""" - try: - return description_value(fm) - except FrontmatterError as exc: - # `exc` carries the whole clause — invalid YAML, a non-mapping block, or - # a description of the wrong type. Do not prefix a diagnosis here; the - # last one named a syntax error for two failures that have none. - fail(f"{exc} — the ADR-0020 description and boundary-target gates could " - f"not run — {local_fname}") - return None - -def check_description_budget(value, local_fname, by_hand=False): - """ADR-0020 description gates — identical for every scope. - - `by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent - carrying `disable-model-invocation: true` is absent from the model-visible - listing, so the 250-character SUGGESTION — a routing-quality budget — has - no listing to apply to. The 400-character ceiling is unaffected. - """ - if not value: - return - dlen = len(value) - if dlen > DESC_MAX_CHARS: - fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character " - f"ADR-0020 ceiling. It is preloaded into every session whether or not the " - f"agent is invoked. Keep a trigger clause, at most one capability clause, " - f"and a boundary clause; move capability enumeration, output-format detail, " - f"composition notes and implementation detail to the body — {local_fname}") - elif dlen > DESC_SUGGEST_CHARS and not by_hand: - suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character " - f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is " - f"what moves the corpus average; the FAIL tier only stops outliers " - f"— {local_fname}") - -def check_boundary(value, fpath, local_fname, by_hand=False): - """ADR-0020 boundary clause + resolvable boundary targets. - - agent-author's SKILL.md states that an agent's boundary targets must - resolve, but until this ran no script checked it — the contract was - documented and unenforced. The resolution universe is derived from the - AGENT FILE's own location (the authoring root above it, its own apm - package, and that package's declared apm dependencies), never from this - script's path, and — when an authoring root exists — never from a deployed - .claude/ tree, so a fresh clone and a machine that has run `apm install` - return the same verdict. - """ - if not value: - return - # SUGGESTION, not FAIL: detecting the absence is deterministic, but whether - # this particular agent warrants a boundary clause is judgment. All four - # agents in this corpus currently lack one. - # - # THREE outcomes, not two: "no boundary clause" and "boundary clause I could - # not parse" are different findings (issue #110). And a hand-invoked agent is - # exempt from the clause altogether (issue #108) — the boundary-target - # resolution below still runs, because a target it DOES name should still - # resolve. - status = boundary_clause_status(value) if not by_hand else 'present' - if status == 'absent': - suggest(f"description has no boundary clause — add the prose form (\"Do not use " - f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") " - f"so the router knows where NOT to send this agent — {local_fname}") - elif status == 'unparsed': - suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no " - f"target could be read, so the dangling-target check did not run on it — " - f"the clause is PRESENT and unparsed, not missing. Most often the target " - f"is a single word, which is deliberately not matchable bare: write it as " - f"`name` or /name — {local_fname}") - if not by_hand: - # One arrow, one target: a second name after the same arrow is resolved - # by nothing and reported by nothing (issue #107). - for first, second in multi_target_arrow_clauses(value): - suggest(f"an arrow boundary clause names more than one target ('{first}', then " - f"'{second}') and only the first is resolved — the second is checked by " - f"nothing. Split it into one arrow per target: \"Not X -> {first}. " - f"Not Y -> {second}.\" — {local_fname}") - targets = boundary_targets(value) - if not targets: - return - known = known_targets(os.path.dirname(os.path.abspath(fpath))) - if not known: - info(f"boundary-target resolution DID NOT RUN — no skill universe could be " - f"determined for this path (no authoring root above it, no apm package " - f"root, no declared apm dependencies, no deployed .claude/ or .agents/ " - f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}") - return - # blocking vs reported: a target only earns a FAIL when it is written in - # route notation or its own sentence corroborates it by naming another target - # that resolves. See the shared resolver's CORROBORATION note. - blocking, reported = unresolved_targets(value, known) - for target in blocking: - fail(f"description routes to '{target}', which resolves to no skill or agent " - f"in this monorepo, in this package, or in a package it declares in " - f"apm.yml dependencies.apm — a boundary clause naming a non-existent " - f"target sends the router nowhere — {local_fname}") - for target in reported: - suggest(f"description routes to '{target}', which resolves to no skill or agent " - f"in this monorepo, in this package, or in a package it declares in " - f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing " - f"else in that sentence resolves, so it is equally likely to be a tool, a " - f"file format or an English compound. If it IS a route, write it as " - f"`/{target}` or `-> {target}` and it will be checked properly — " - f"{local_fname}") - -def extract_tools_list(fm): - """Tool names from the `tools` field — inline scalar OR YAML block sequence. - - Read off the PARSED mapping, never off extract_field(). That function's - capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:` - written as a block sequence — the shape Copilot agent files use — captured - nothing at all and the subagent-unavailable-tool check silently stopped - firing on exactly the files it was written for. Both spellings are legal - YAML, so both are read here. - """ - try: - data = yaml.safe_load(fm) - except Exception: - # Not this function's failure to report: the frontmatter's validity is - # decided (and failed) by agent_description() on the same text. - return set() - if not isinstance(data, dict): - return set() - val = data.get('tools') - if isinstance(val, list): - items = [str(item).strip() for item in val] - elif isinstance(val, str): - items = re.split(r'[\s,]+', val.strip()) - else: - return set() - return {item for item in items if item} - -def is_copilot_cloud_ide(fpath): - """True if the file is a cloud/IDE Copilot agent (name is optional for these).""" - return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/') - -# --- Detect scope --- -# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is -# exactly one of the four package content types. Group 1 captures an optional -# opening quote; \1 requires the same character (or nothing) to close it, so -# "skill" and '"skill"' both match but a mismatched quote doesn't. The value -# must then be followed by whitespace or end-of-line — not just a non-word -# character — so a malformed value like `prompts-only` is correctly rejected -# instead of false-matching on the `prompts` prefix. -APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)") - -def find_apm_package_root(apm_yml_path): - """Return True if apm_yml_path has a top-level type: line (i.e. is a package - manifest, not a type:-less marketplace-only apm.yml).""" - # errors='replace', not a hard failure: this only asks whether a `type:` - # line exists, and a stray undecodable byte elsewhere in someone else's - # apm.yml must not abort scope detection. - with open(apm_yml_path, encoding='utf-8', errors='replace') as f: - for line in f: - if APM_TYPE_RE.match(line): - return True - return False - -def detect_scope(start_dir): - home = os.path.expanduser('~') - original_start = os.path.abspath(start_dir) - # Agent files conventionally live exactly two path segments below their - # scope root — /.claude/agents, /.github/agents, - # /.copilot/agents, or /.apm/agents (see new-agent.sh's - # CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments - # recovers the same root new-agent.sh would have been invoked with to - # produce this exact file, independent of how far the walk below has to - # travel to find (or fail to find) a marker — mirrors new-agent.sh's - # `root` vs `current` distinction even though validate.sh is handed a - # file's directory, not the scope root itself. - # - # That arithmetic is only trustworthy when the path actually has this - # shape: parent directory literally named "agents", grandparent one of - # the four known scope-dir names. A hand-placed or otherwise - # non-conventional agent file (never produced by new-agent.sh) has no - # such guarantee — blindly trusting two-segments-up there could point at - # an unrelated ancestor. conventional_shape gates every use of - # conventional_root below; when it's false, the walked-to `current` - # directory is used instead, the same fallback this function used before - # conventional_root existed. - scope_dir_name = os.path.basename(os.path.dirname(original_start)) - conventional_shape = ( - os.path.basename(original_start) == 'agents' - and scope_dir_name in ('.claude', '.github', '.copilot', '.apm') - ) - conventional_root = os.path.dirname(os.path.dirname(original_start)) - current = original_start - while True: - # The filesystem root is never a candidate, the same guard the shared - # resolver's walk-up loops carry. Without it a file under a marker-less - # temp directory walked all the way to `/` and returned it as the scope - # root, which then reported `counterpart file not found: - # /.claude/agents/.md` — a path that names someone else's machine, - # not the user's project. When the walk runs out, the agent file's own - # directory (or its conventional root) is the honest answer. - if _is_fs_root(current): - return 'project', conventional_root if conventional_shape else original_start - apm_yml = os.path.join(current, 'apm.yml') - if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml): - return 'plugin', current - # $HOME is the user-scope boundary — checked before the .git test - # below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) - # can't shadow user scope by being its own .git repo. 'user' scope - # requires EITHER start_dir to BE $HOME itself (no walk-up — the - # new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the - # conventional two-segments-below-root depth (i.e. $HOME IS that - # root, matching the real ~/.claude/agents or ~/.copilot/agents - # shape). Any other walk-up into $HOME — a marker-less directory - # nested deeper than that convention — resolves to project scope - # instead: a stray directory under $HOME can't be silently - # redirected into the shared global ~/.claude or ~/.copilot agent - # directories. - if current == home: - if original_start == home or (conventional_shape and conventional_root == home): - return 'user', home - return 'project', conventional_root if conventional_shape else current - # .git is a directory in a normal checkout but a file (`gitdir: ...`) - # in a git worktree — exists() covers both. Returns conventional_root, - # not current: new-agent.sh's project-scope file placement always - # uses its `$ROOT` argument directly, never the walked-up `.git` - # location, so a one or more levels below the repo's .git - # (a subdirectory of a larger git-tracked tree — explicitly a - # supported case per new-agent.sh's usage text) must resolve to the - # same root new-agent.sh actually wrote to, not to the .git dir — - # unless the path lacks the conventional shape, in which case that - # arithmetic isn't trustworthy and current is used instead. - if os.path.exists(os.path.join(current, '.git')): - return 'project', conventional_root if conventional_shape else current - parent = os.path.dirname(current) - if parent == current: - return 'project', conventional_root if conventional_shape else current - current = parent - -agent_dir = os.path.dirname(agent_file) -scope, scope_root = detect_scope(agent_dir) - -# --- Plugin/APM scope: single vendor-neutral file, no counterpart --- -def check_apm_agent_file(fpath, allowlist, stem): - local_fname = os.path.basename(fpath) - try: - content = read_text(fpath) - except EncodingError as exc: - fail(f"file is {exc}. Nothing could be measured, so this is a hard " - f"failure, not a skip — {local_fname}") - return - except OSError as exc: - # A path that cannot be opened gets a FAIL line naming it, not a bare - # FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes - # this path for an agent file deleted from the worktree but still - # tracked in the index — a real, expected state, and the caller needs to - # be told which file, not handed an interpreter stack. - fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could " - f"be measured, so this is a hard failure, not a skip — {local_fname}") - return - - fm, body = parse_frontmatter(content) - if fm is None: - fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, " - f"then a closing `---` line (a BOM, leading blank lines, trailing spaces " - f"after either marker and CRLF endings are all tolerated). Nothing could be " - f"measured, so this is a hard failure, not a skip — {local_fname}") - return - - # The apm-agent.md template embeds its authoring guidance as HTML - # comments inside the frontmatter block (so they render invisible in a - # Markdown preview but stay visible in the raw file). get_frontmatter_keys - # silently ignores any line that isn't a `key:` match, so a comment left - # behind at ship time would otherwise pass unnoticed — yet apm compile - # copies this frontmatter verbatim to both harnesses, and `` is - # not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016). - if re.search(r'', fm): - fail(f"frontmatter still contains template HTML comments () " - f"— delete them before shipping — {local_fname}") - - # Allowlist: the permitted keys are data, read at load time from - # references/field-inventory.md's `## apm-agent-allowlist` section — do not - # restate them here, or this comment goes stale the next time that line - # changes. apm compile verbatim-copies frontmatter to every target, so a key - # outside the list is unsafe on at least one harness (ADR-0016). Note the - # list admits denylist-shaped restrictions (disallowedTools) but never - # allowlist-shaped ones (tools), whose value shape differs per harness. - fm_keys = get_frontmatter_keys(fm) - for key in sorted(fm_keys): - if key not in allowlist: - fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist " - f"({', '.join(sorted(allowlist))}) — {local_fname}") - - # name — required, kebab-case, must match filename stem (file is .agent.md) - name_val = extract_field(fm, 'name') - if not name_val: - fail(f"name field is missing or empty — {local_fname}") - else: - if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): - fail(f"name '{name_val}' is not kebab-case — {local_fname}") - if name_val != stem: - fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}") - - # description — required, non-empty, no placeholder - # Presence is decided on the FOLDED value, never on a line regex. Deciding - # it on extract_field's raw capture is what let `description:` with no value - # pass this gate in total silence: the capture picked up the next key, so - # "missing or empty" never fired, and every ADR-0020 check below then - # early-returned on the empty folded value. Exit 0, zero output, no gate run. - folded = agent_description(fm, local_fname) - if folded is None: - pass # frontmatter is not valid YAML — agent_description already failed - elif not folded: - fail(f"description field is missing or empty — {local_fname}") - else: - if PLACEHOLDER_RE.search(folded): - fail(f"description contains unfilled FILL IN: placeholder — {local_fname}") - by_hand = hand_invoked(fm) - check_description_budget(folded, local_fname, by_hand) - check_boundary(folded, fpath, local_fname, by_hand) - - # body — required, non-empty, no placeholder; same Copilot truncation risk - # applies since this file compiles verbatim into a real Copilot file downstream. - if not body.strip(): - fail(f"system prompt body is empty — {local_fname}") - else: - if PLACEHOLDER_RE.search(body): - fail(f"body contains unfilled FILL IN: placeholder — {local_fname}") - if len(body) > COPILOT_BODY_LIMIT: - suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — " - f"content beyond the limit is silently truncated by the Copilot runtime " - f"once apm compile emits it downstream — {local_fname}") - -if scope == 'plugin': - check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem) - for s in suggestions: - print(f"SUGGESTION {s}") - sys.exit(1 if failed else 0) - -# --- Project/user scope: unchanged CC/Copilot pair validation --- - -# --- Derive counterpart path --- -if scope == 'project': - if provider == 'claude-code': - counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md') - counterpart_provider = 'copilot' - else: - counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md') - counterpart_provider = 'claude-code' -else: # user - home = os.path.expanduser('~') - if provider == 'claude-code': - counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md') - counterpart_provider = 'copilot' - else: - counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md') - counterpart_provider = 'claude-code' - -def check_file(fpath, file_provider): - local_fname = os.path.basename(fpath) - try: - content = read_text(fpath) - except EncodingError as exc: - fail(f"file is {exc}. Nothing could be measured, so this is a hard " - f"failure, not a skip — {local_fname}") - return - except OSError as exc: - # Same reason as check_apm_agent_file's: a diagnostic naming the path - # beats a FileNotFoundError traceback. The counterpart is pre-checked at - # the bottom of this script, but agent_file itself never was. - fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could " - f"be measured, so this is a hard failure, not a skip — {local_fname}") - return - - fm, body = parse_frontmatter(content) - if fm is None: - fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, " - f"then a closing `---` line (a BOM, leading blank lines, trailing spaces " - f"after either marker and CRLF endings are all tolerated). Nothing could be " - f"measured, so this is a hard failure, not a skip — {local_fname}") - return - - # name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents - cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath)) - name_val = extract_field(fm, 'name') - if not cloud_ide: - if not name_val: - fail(f"name field is missing or empty — {local_fname}") - else: - if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): - fail(f"name '{name_val}' is not kebab-case — {local_fname}") - # Stem check applies to Copilot CLI only; CC docs say filename need not match name - if file_provider == 'copilot': - stem = local_fname[:-len('.agent.md')] - if name_val != stem: - fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}") - elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): - # cloud/IDE: name is optional, but if present it must be valid - fail(f"name '{name_val}' is not kebab-case — {local_fname}") - - # description - # Presence is decided on the FOLDED value, never on a line regex. Deciding - # it on extract_field's raw capture is what let `description:` with no value - # pass this gate in total silence: the capture picked up the next key, so - # "missing or empty" never fired, and every ADR-0020 check below then - # early-returned on the empty folded value. Exit 0, zero output, no gate run. - folded = agent_description(fm, local_fname) - if folded is None: - pass # frontmatter is not valid YAML — agent_description already failed - elif not folded: - fail(f"description field is missing or empty — {local_fname}") - else: - if PLACEHOLDER_RE.search(folded): - fail(f"description contains unfilled FILL IN: placeholder — {local_fname}") - by_hand = hand_invoked(fm) - check_description_budget(folded, local_fname, by_hand) - check_boundary(folded, fpath, local_fname, by_hand) - - # body - if not body.strip(): - fail(f"system prompt body is empty — {local_fname}") - else: - if PLACEHOLDER_RE.search(body): - fail(f"body contains unfilled FILL IN: placeholder — {local_fname}") - # Copilot body length limit - if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT: - suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}") - - # CC-only fields in Copilot file - if file_provider == 'copilot': - fm_keys = get_frontmatter_keys(fm) - for key in sorted(fm_keys): - if key in cc_only_fields: - fail(f"CC-only field '{key}' present in Copilot file — {local_fname}") - - # Copilot-only fields in CC file - if file_provider == 'claude-code': - fm_keys = get_frontmatter_keys(fm) - for key in sorted(fm_keys): - if key in copilot_only_fields: - fail(f"Copilot-only field '{key}' present in CC file — {local_fname}") - - # Subagent-unavailable tools listed in tools field - tools = extract_tools_list(fm) - unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS - for tool in sorted(unavailable): - suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}") - -# --- Check counterpart exists --- -if not os.path.isfile(counterpart): - fail(f"counterpart file not found: {counterpart}") - sys.exit(1) - -# --- Check both files --- -check_file(agent_file, provider) -check_file(counterpart, counterpart_provider) - -for s in suggestions: - print(f"SUGGESTION {s}") - -sys.exit(1 if failed else 0) -PYTHON diff --git a/plugins/kyberforge/.apm/skills/agent-audit/tests/README.md b/plugins/kyberforge/.apm/skills/agent-audit/tests/README.md deleted file mode 100644 index bf9245b..0000000 --- a/plugins/kyberforge/.apm/skills/agent-audit/tests/README.md +++ /dev/null @@ -1,33 +0,0 @@ -# tests/ - -Test files for scripts bundled with this skill. - -## When to add tests - -Add tests here when the skill has scripts in `scripts/` that are complex enough -to break silently — validators, parsers, generators, anything with branching -logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`) -belongs here, not in `scripts/`. - -## Dependencies - -Tests require [bats-support](https://github.com/bats-core/bats-support) and -[bats-assert](https://github.com/bats-core/bats-assert). The test files load -helpers from the repo root's `tests/test_helper/`. - -From the repo root: - -```bash -git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support -git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert -``` - -Run all tests for this skill (from the repo root): - -```bash -bats /agent-audit/tests/ -``` - -## If no tests are needed - -Delete this README and the `tests/` directory entirely. diff --git a/plugins/kyberforge/.apm/skills/agent-author/SKILL.md b/plugins/kyberforge/.apm/skills/agent-author/SKILL.md index af51352..b4898d5 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/SKILL.md +++ b/plugins/kyberforge/.apm/skills/agent-author/SKILL.md @@ -3,7 +3,7 @@ name: agent-author description: > Use when the user wants to create a new agent definition file from scratch, or apply grill findings, audit findings, or inline feedback to an existing one. - Not read-only review -> `agent-audit`. Not skills -> `skill-author`. + Not read-only review -> `factory-audit`. Not skills -> `skill-author`. allowed-tools: Bash Read Write Edit metadata: version: "1.0.2" @@ -18,7 +18,7 @@ metadata: - At plugin/APM scope `tools` and every Claude-only field are omitted entirely, not merely ignored: `apm compile` copies frontmatter verbatim to both harnesses, so fencing a read-only agent with `tools:` is wrong on one of them. `disallowedTools` is the one restriction that survives (ADR-0016). - That fence is partial. It denies only the tools it names, never `Bash`, which a plugin-scope agent inherits — a shell redirect still writes. State the read-only boundary in the body too. -- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is an `agent-audit` FAIL. +- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is a `factory-audit` FAIL. - Duplicate `name` values in one scope: Claude Code discards one silently. Verify uniqueness before shipping. ## Step 1 — Dispatch @@ -29,7 +29,7 @@ metadata: | A file exists, at least one improvement signal present | Improve | `references/improve.md` | | A file exists, no signals | Stop and ask | — | -Signals: grill output, `agent-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?" +Signals: grill output, `factory-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?" Read only the reference for the resolved flow. Capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it. @@ -48,7 +48,7 @@ Read only the file for the resolved scope; the other describes fields this run c Before writing or editing a `description`, or restructuring a body, read `references/contract.md` — the three-part shape, banned content, the delegation rule and the body pattern. -Gates `agent-audit` enforces at every scope: +Gates `factory-audit` enforces at every scope: - **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not -> ` that resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only: an agent's `name` and `description` is preloaded into every session exactly as a skill's is. - **Body** — no word gate, and a delegation check in its place: name the skill to invoke rather than restating what it does. @@ -58,7 +58,7 @@ At every scope, five tools reach no subagent whatever `tools` says — `AskUserQ ## Step 4 — Validate and close -Invoke `agent-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those. +Invoke `factory-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those. At plugin/APM scope bump the resolved package's `apm.yml` `version` — **minor** on create, **patch** on improve — because consumers compare it to detect updates. Project and user scope have no manifest. diff --git a/plugins/kyberforge/.apm/skills/agent-author/assets/README.md b/plugins/kyberforge/.apm/skills/agent-author/assets/README.md index 658ad7f..18edbf4 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/assets/README.md +++ b/plugins/kyberforge/.apm/skills/agent-author/assets/README.md @@ -7,4 +7,4 @@ All three scaffold the `description` in the three-part ADR-0020 shape — a `Use - **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders. - **`copilot.agent.md.template`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`). -- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the authoritative list, read from there as data by `agent-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment). +- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md` — the authoritative list, read from there as data by `factory-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment). diff --git a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/apm-agent.md b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/apm-agent.md index 03783ce..0e69ea1 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/apm-agent.md +++ b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/apm-agent.md @@ -6,8 +6,8 @@ anything, so a harness-specific value is wrong on at least one of them. This template does not restate the permitted-field list. The authoritative - list is the `apm-agent-allowlist` section of agent-audit's - references/field-inventory.md, which agent-audit's validate.sh reads from + list is the `apm-agent-allowlist` section of factory-audit's + references/agent-field-inventory.md, which factory-audit's validate.sh reads from there as data — a list copied into a template goes stale one step further out than the list itself. Every field scaffolded below is on it; before adding any other field, check that section. @@ -37,7 +37,7 @@ description: FILL IN: Use when . Not - deleted. Never write "Use proactively" here. It steers the Claude Code runtime and does nothing anywhere else, and this file compiles to a Copilot `.agent.md` too, where - agent-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL. + factory-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL. The phrase is CC-only; at this scope, a precise trigger clause does that job. Example: "Use when a diff needs checking for injected credentials before it merges. Not prose or style linting -> `lint-runner`." --> @@ -69,7 +69,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action. ## Inputs diff --git a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/claude-code.md b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/claude-code.md index 15a4175..8c09d60 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/claude-code.md +++ b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/claude-code.md @@ -18,7 +18,7 @@ description: FILL IN: Use when . Not - "Use proactively" is valid HERE and only here: it steers the Claude Code runtime to offer this agent unprompted. Add it only if that is what you want. If you add it, leave it OUT of the Copilot half of the pair — the phrase does nothing there - and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The + and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The pair must describe the same job; it does not have to be byte-identical. Example: "Use when a diff needs checking for injected credentials before it merges. Not prose or style linting -> `lint-runner`." --> @@ -31,7 +31,7 @@ description: FILL IN: Use when . Not - Omit Agent entirely to prevent this agent from spawning subagents. Never available to subagents regardless of tools field: AskUserQuestion, EnterPlanMode, ExitPlanMode, ScheduleWakeup, WaitForMcpServers - Listing any of them is a finding: agent-audit enforces the flat rule. --> + Listing any of them is a finding: factory-audit enforces the flat rule. --> ## Inputs diff --git a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/copilot.agent.md.template b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/copilot.agent.md.template index 131aab0..7921371 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/assets/templates/copilot.agent.md.template +++ b/plugins/kyberforge/.apm/skills/agent-author/assets/templates/copilot.agent.md.template @@ -20,8 +20,8 @@ description: FILL IN: Use when . Not - 250 characters is the target, 400 the hard ceiling (ADR-0020). Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was deleted. Never write "Use proactively" here. It steers the Claude Code runtime and does nothing - in Copilot, and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. - Otherwise keep the wording matched to the Claude Code half of the pair: agent-audit + in Copilot, and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. + Otherwise keep the wording matched to the Claude Code half of the pair: factory-audit checks that both halves describe the same job, not that they are byte-identical, so dropping the CC-only phrase here is not a pair-consistency finding. Example: "Use when a diff needs checking for injected credentials before it @@ -58,7 +58,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action. ## Inputs diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/contract.md b/plugins/kyberforge/.apm/skills/agent-author/references/contract.md index d5ee27e..ac9c93e 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/contract.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/contract.md @@ -7,7 +7,7 @@ source_keys: # The agent description and body contract House contract. The counts and the boundary targets are enforced by -`agent-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the +`factory-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the judgment calls by its reference files. ## Why the budget exists @@ -54,7 +54,7 @@ appear depends on the file: | Vendor-neutral `.apm/agents/.agent.md` (plugin/APM scope) | **Never.** Same Vale rule, same hard FAIL — the file matches the `**/*.agent.md` glob, and it compiles to a real Copilot agent downstream. | A pair whose Claude Code half carries the phrase and whose Copilot half omits it is correct, not -inconsistent: `agent-audit` checks that both halves describe the same job, not that they match +inconsistent: `factory-audit` checks that both halves describe the same job, not that they match word for word. Indirect triggers ("even if the user doesn't say X") take a similar conditional at every scope: @@ -126,7 +126,7 @@ One job per agent. An agent covering two jobs gets delegated to for the wrong on **Delegation discipline replaces the word gate.** A plugin/APM agent is a single file with no sibling `references/` directory: it cannot disclose progressively to itself, so its only way to stay short is to *invoke* rather than *restate*. A body that transcribes a procedure a skill it -can invoke already owns is an `agent-audit` FAIL, and the fix is one line — "invoke ``". +can invoke already owns is a `factory-audit` FAIL, and the fix is one line — "invoke ``". - Restating: "To commit, check the message against Conventional Commits: type, scope, description; header under 100 chars; …" diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/create.md b/plugins/kyberforge/.apm/skills/agent-author/references/create.md index 369a36e..758f5dc 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/create.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/create.md @@ -22,7 +22,7 @@ Before touching the filesystem, confirm you have: If any are missing, stop and ask before proceeding. -`agent-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and +`factory-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and is co-installed with this skill; if it is unavailable, stop and ask the user to install kyberforge before continuing. diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/deployment-modes.md b/plugins/kyberforge/.apm/skills/agent-author/references/deployment-modes.md index d067eda..e742241 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/deployment-modes.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/deployment-modes.md @@ -27,7 +27,7 @@ When the same agent `name` appears at multiple scopes, **user scope wins over pr Field rules are per scope and live with the scope: `references/plugin-scope.md` for the single vendor-neutral file, `references/project-user-scope.md` for the Claude Code / Copilot pair. Read one, not both. The short version is that plugin/APM frontmatter is an allowlist read from -`agent-audit`'s `references/field-inventory.md`, narrow because `apm compile` copies frontmatter +`factory-audit`'s `references/agent-field-inventory.md`, narrow because `apm compile` copies frontmatter verbatim to every target (ADR-0016), while project and user scope carry the full per-provider field sets. diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/improve.md b/plugins/kyberforge/.apm/skills/agent-author/references/improve.md index 2a30537..f3e9813 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/improve.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/improve.md @@ -14,9 +14,9 @@ Confirm the agent file (or, at project and user scope, the pair) exists and that improvement signal is present in the conversation or in a referenced file. If no signals are present, stop: "This skill applies existing signals to an agent. For a blind -review, run `agent-audit` instead." +review, run `factory-audit` instead." -`agent-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if +`factory-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if it is unavailable, stop and ask the user to install the kyberforge plugin before continuing. **Partial pair — project and user scope only.** If one provider file exists and the other does @@ -53,7 +53,7 @@ Edit whichever file the signals point to. scoped to the cases you have seen overfits and performs worse on new input. **Delegate rather than grow.** An agent body has no word ceiling, but a body that restates a -procedure a skill it can invoke already owns is an `agent-audit` FAIL. When a signal reports a +procedure a skill it can invoke already owns is a `factory-audit` FAIL. When a signal reports a missing procedure, check first whether an installed skill owns it and name that skill instead of transcribing it. See `references/contract.md`. @@ -81,7 +81,7 @@ the matching `sources.md` entry — the create flow's Step 3 has the rules. **Check for regressions before handing back.** `SKILL.md` Step 4 tells you to resolve every FAIL, which says nothing about a check that passed *before* these edits and no longer does. Compare the -closing `agent-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or +closing `factory-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or a SUGGESTION that has become a FAIL, is damage this flow caused and is in scope for it. Only the improve flow can make that comparison; the create flow has no prior state to compare against. diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/plugin-scope.md b/plugins/kyberforge/.apm/skills/agent-author/references/plugin-scope.md index 3d71359..bb10788 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/plugin-scope.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/plugin-scope.md @@ -12,9 +12,9 @@ a Copilot marker — the file is vendor-neutral. ## Frontmatter -The permitted keys are the `apm-agent-allowlist` section of `agent-audit`'s -`references/field-inventory.md`. Read them from there as data — that section is the single source -of truth, `agent-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of +The permitted keys are the `apm-agent-allowlist` section of `factory-audit`'s +`references/agent-field-inventory.md`. Read them from there as data — that section is the single source +of truth, `factory-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of the roster, here or in a template or in script output, goes stale one step further out than the list itself. @@ -60,7 +60,7 @@ when research sources informed the agent, with slugs matching H2 headings in the Follow the Body section of `references/contract.md`: role instruction, one job, and delegation to installed skills instead of transcribed procedure. -## Before invoking `agent-audit` +## Before invoking `factory-audit` - [ ] `name` kebab-case, matching the filename stem, unique in scope - [ ] `description` written to `references/contract.md` diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/project-user-scope.md b/plugins/kyberforge/.apm/skills/agent-author/references/project-user-scope.md index 0e63019..a47a1bc 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/project-user-scope.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/project-user-scope.md @@ -92,7 +92,7 @@ Both formats truncate a body past **30,000 characters** silently. Copilot has no `permissionMode`, `maxTurns`, `isolation`, `memory`, `effort`, `hooks` or `mcpServers`. Never let those cross over from the Claude Code file. -## Before invoking `agent-audit` +## Before invoking `factory-audit` Both files: diff --git a/plugins/kyberforge/.apm/skills/agent-author/references/scripts.md b/plugins/kyberforge/.apm/skills/agent-author/references/scripts.md index 42c4493..c3eb6ea 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/references/scripts.md +++ b/plugins/kyberforge/.apm/skills/agent-author/references/scripts.md @@ -15,7 +15,7 @@ All scripts in this skill must follow these rules: - **Idempotent** — "create if not exists" per file. The scaffold script skips any file that already exists; agents may safely re-run it. - **Meaningful exit codes** — `0` success, `1` invalid arguments or precondition failure. Document in `--help`. - **Self-contained** — no external package installs at runtime. The script uses only bash builtins and POSIX tools (`sed`, `mkdir`, `cat`). -- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, which `agent-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure. +- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md`, which `factory-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure. ## Template variables diff --git a/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh b/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh index bb1559c..83b6a2e 100755 --- a/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh +++ b/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh @@ -25,8 +25,8 @@ Arguments: its frontmatter verbatim to every target with no per-target field integrator, so the permitted field set is narrow — see the apm-agent-allowlist - section of agent-audit's - references/field-inventory.md and ADR-0016) + section of factory-audit's + references/agent-field-inventory.md and ADR-0016) → creates /sources.md (if absent) project scope : no type:-bearing apm.yml found; root is a project directory @@ -109,7 +109,7 @@ is_apm_package_manifest() { # --- Walk-up package-root detection --- # -# Mirrors agent-audit's validate.sh scope walk-up, with apm.yml + type: swapped +# Mirrors factory-audit's validate.sh scope walk-up, with apm.yml + type: swapped # in for the old plugin.json marker. Starting at ROOT, walk upward: # - an apm.yml with a top-level `type:` field marks an APM package root # (plugin/APM scope) — stop and return it. @@ -266,13 +266,13 @@ fi # A roster restated in terminal output goes stale one step further out than the list # itself: the old "(name, description, model, body only)" hint outlived ADR-0016's # 2026-08-14 amendment, which added disallowedTools to the permitted set. Point at the -# scaffolded file's own comments for what to fill, and at agent-audit's validate.sh — -# which reads the allowlist from field-inventory.md as data — for what is permitted. -AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../agent-audit/scripts" 2>/dev/null && pwd || true)" +# scaffolded file's own comments for what to fill, and at factory-audit's validate.sh — +# which reads the allowlist from agent-field-inventory.md as data — for what is permitted. +AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../factory-audit/scripts" 2>/dev/null && pwd || true)" if [[ -n "$AUDIT_SCRIPTS" && -f "$AUDIT_SCRIPTS/validate.sh" ]]; then VALIDATE_HINT="$AUDIT_SCRIPTS/validate.sh" else - VALIDATE_HINT="agent-audit's scripts/validate.sh" + VALIDATE_HINT="factory-audit's scripts/validate.sh" fi if [[ "$created_any" == false ]]; then @@ -290,7 +290,7 @@ else echo " 2. Populate $SOURCES_DIR/sources.md with research sources, or delete it" >&2 echo " 3. Validate: $VALIDATE_HINT $APM_FILE" >&2 echo " It checks the frontmatter against the apm-agent-allowlist section of" >&2 - echo " agent-audit's references/field-inventory.md, the authoritative field list." >&2 + echo " factory-audit's references/agent-field-inventory.md, the authoritative field list." >&2 else echo " 1. Fill in $CC_FILE — replace every FILL IN: placeholder. Optional fields are" >&2 echo " scaffolded there as commented blocks; uncomment the ones that apply." >&2 diff --git a/plugins/kyberforge/.apm/skills/agent-author/tests/new-agent.bats b/plugins/kyberforge/.apm/skills/agent-author/tests/new-agent.bats index 5904546..e4cc4de 100644 --- a/plugins/kyberforge/.apm/skills/agent-author/tests/new-agent.bats +++ b/plugins/kyberforge/.apm/skills/agent-author/tests/new-agent.bats @@ -77,14 +77,14 @@ teardown() { assert_failure } -# The permitted set is read from the same data agent-audit's validate.sh reads -- -# the apm-agent-allowlist section of agent-audit's field-inventory.md -- rather than +# The permitted set is read from the same data factory-audit's validate.sh reads -- +# the apm-agent-allowlist section of factory-audit's agent-field-inventory.md -- rather than # restated here. A hardcoded copy drifts: this assertion listed four fields and went # on passing after ADR-0016's amendment added disallowedTools, and would have # rejected a scaffolded agent that legitimately carried it. @test "plugin/APM scope: frontmatter carries only allowlisted fields" { - inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md" - [ -f "$inventory" ] || fail "field-inventory.md not found at $inventory" + inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md" + [ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory" allowlist="$(awk ' /^## apm-agent-allowlist$/ { insection = 1; next } insection && /^##/ { exit } @@ -99,7 +99,7 @@ teardown() { keys="$(grep -oE '^[a-zA-Z][a-zA-Z0-9_-]*:' <<< "$fm" | sed 's/:$//' | sort -u)" for key in $keys; do if ! grep -qw "$key" <<< "$allowlist"; then - fail "frontmatter key '$key' is not in field-inventory.md's apm-agent-allowlist ($allowlist)" + fail "frontmatter key '$key' is not in agent-field-inventory.md's apm-agent-allowlist ($allowlist)" fi done } @@ -111,8 +111,8 @@ teardown() { # must name every one of them, so a partial restatement -- the only shape that can go # stale silently -- fails. Naming none, the current design, passes. @test "plugin/APM scope: next-steps guidance does not partially restate the allowlist" { - inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md" - [ -f "$inventory" ] || fail "field-inventory.md not found at $inventory" + inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md" + [ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory" allowlist="$(awk ' /^## apm-agent-allowlist$/ { insection = 1; next } insection && /^##/ { exit } @@ -134,7 +134,7 @@ teardown() { fi done if [ -n "$named" ] && [ -n "$missing" ]; then - fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at field-inventory.md instead of restating it." + fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at agent-field-inventory.md instead of restating it." fi } diff --git a/plugins/kyberforge/.apm/skills/factory-audit/SKILL.md b/plugins/kyberforge/.apm/skills/factory-audit/SKILL.md new file mode 100644 index 0000000..144cbd7 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/SKILL.md @@ -0,0 +1,77 @@ +--- +name: factory-audit +description: > + Use when the user wants a skill directory or agent definition audited, + including "is this ready to ship", or after hand-editing one outside its + author skill. Not applying skill fixes -> skill-author. Not applying agent + fixes -> agent-author. +allowed-tools: Bash Read +metadata: + version: "1.0.1" + category: factory + source_keys: + - agentskills-home + - agentskills-spec + - agentskills-best-practices + - agentskills-optimizing-descriptions + - agentskills-using-scripts + - context7-websites-code-claude + - claude-code-plugins-docs + - claude-code-subagents-docs + - context7-github-en-copilot + - github-custom-agents-configuration +--- + +## Gotchas + +- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here. +- A file carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead. +- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered. + +## Step 0 — Dispatch + +Resolve the flow from the target path **before running anything**. The two flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches. + +| Target | Flow | Read | +|---|---|---| +| A directory containing `SKILL.md` | skill | `references/skill-flow.md` | +| A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` | +| A file named `*.agent.md` | agent | `references/agent-flow.md` | +| A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` | +| Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — | + +Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4. + +On the last row, stop: run no validator and tell the user the two accepted shapes — a skill directory (or its `SKILL.md`), or an agent file (`*.agent.md`, or a `.md` directly under an `agents/` directory). Guessing a flow audits the path against the wrong spec. + +The scripts re-detect the flow from the path. If `validate.sh` reports on the other artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line. + +## Step 4 — Report + +Open with the coverage line for the flow you took, naming every dimension checked. + +Skill flow: + +```text +Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance +``` + +Agent flow: + +```text +Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance +``` + +On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check. + +Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed. + +Each finding: + +```text +FAIL/SUGGESTION — file:line + Why: + Fix: +``` + +Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow. Do not apply fixes — report and propose only. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/.vale.ini similarity index 75% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/.vale.ini index a93c7c8..f4a9999 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini +++ b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/.vale.ini @@ -1,5 +1,8 @@ StylesPath = styles +[**/SKILL.md] +BasedOnStyles = Kyberforge + [**/agents/*.md] BasedOnStyles = Kyberforge diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/CompositionNote.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/CompositionNote.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/CompositionNote.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/CompositionNote.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/VagueWording.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/VagueWording.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/VagueWording.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/Kyberforge/VagueWording.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml b/plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml rename to plugins/kyberforge/.apm/skills/factory-audit/assets/vale/styles/KyberforgeCopilot/ProactivePhrase.yml diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/body-and-delegation.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-body-and-delegation.md similarity index 98% rename from plugins/kyberforge/.apm/skills/agent-audit/references/body-and-delegation.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-body-and-delegation.md index c3d32d6..98077f1 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/body-and-delegation.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-body-and-delegation.md @@ -30,7 +30,7 @@ prompt of a fresh context that has nothing else in it. The rationale for the 900 not transfer, so: - **Never report an agent body as too long on a word count.** There is no number to cite. -- **Never add such a gate to `scripts/validate.sh`.** `tests/validate.bats` pins its absence with a +- **Never add such a gate to `scripts/validate.sh`.** `tests/validate-agent.bats` pins its absence with a body far past 900 words that must still pass, and adding one would contradict the ADR. - The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which `validate.sh` already reports as a SUGGESTION because content past it is silently truncated. @@ -100,6 +100,6 @@ frontmatter block that still contains one. ## Where the criteria live -Every FAIL and SUGGESTION criterion for these dimensions is in `references/finding-criteria.md`, +Every FAIL and SUGGESTION criterion for these dimensions is in `references/agent-finding-criteria.md`, which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file puts the body, delegation or comment-discipline dimension in play. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/description-quality.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-description-quality.md similarity index 99% rename from plugins/kyberforge/.apm/skills/agent-audit/references/description-quality.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-description-quality.md index 6382d84..99bfe6f 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/description-quality.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-description-quality.md @@ -91,6 +91,6 @@ description: > ## Where the criteria live -Every FAIL and SUGGESTION criterion for this dimension is in `references/finding-criteria.md`, +Every FAIL and SUGGESTION criterion for this dimension is in `references/agent-finding-criteria.md`, which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file puts the description dimension in play. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-field-inventory.md similarity index 100% rename from plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-field-inventory.md diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/finding-criteria.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-finding-criteria.md similarity index 96% rename from plugins/kyberforge/.apm/skills/agent-audit/references/finding-criteria.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-finding-criteria.md index 6c9c5e0..62fcbf1 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/finding-criteria.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-finding-criteria.md @@ -20,7 +20,7 @@ Two rules on using it: that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics to read; it does not settle a close call on its own. -## description — `references/description-quality.md` +## description — `references/agent-description-quality.md` Flag as FAIL if: @@ -49,7 +49,7 @@ Flag as FAIL if: `KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour. - **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0 of - `references/description-quality.md`. + `references/agent-description-quality.md`. Flag as SUGGESTION if: @@ -68,7 +68,7 @@ What is left to judgment is semantic and the script cannot reach it: whether a t resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the files manually") should have named one. -## body, delegation and comment-discipline — `references/body-and-delegation.md` +## body, delegation and comment-discipline — `references/agent-body-and-delegation.md` Flag as FAIL if: diff --git a/plugins/kyberforge/.apm/skills/factory-audit/references/agent-flow.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-flow.md new file mode 100644 index 0000000..6e96831 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-flow.md @@ -0,0 +1,64 @@ +--- +source_keys: + - context7-websites-code-claude + - claude-code-plugins-docs + - claude-code-subagents-docs + - context7-github-en-copilot + - github-custom-agents-configuration +--- + +# Agent Flow + +Steps 1 to 3 for an agent definition — the target Step 0 matched as a `*.agent.md` file, or as a +`.md` file whose immediate parent directory is `agents/`. Work them in order, then return to +`SKILL.md` Step 4 to report. + +## Gotchas + +- An agent takes the skill description gates (250 characters SUGGESTION, 400 FAIL) and **no body word gate at all**. Its body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer and no number exists to cite. Judge an over-long agent body through the delegation check. +- **Plugin/APM scope only:** provider safety means survival of a verbatim copy to every target, not Claude-Code-versus-Copilot field leakage — `references/agent-scope-plugin-apm.md` carries the contract. + +## Step 1 — Deterministic checks + +Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly: + +```bash +bash scripts/validate.sh +bash scripts/validate-provenance.sh +bash scripts/vale-wrap.sh [] +``` + +`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes. + +If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, a `scripts/lib-*.sh` library or `references/agent-field-inventory.md` missing — read `references/agent-validation-scripts.md`; what these scripts measure is not reproducible by reading. + +`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same tiers: **1** is real findings, **2** is never ran — report that as `### Structure` unverified, never as a failure. + +`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment: + +| Rule | Dimension | +|---|---| +| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description | +| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body | + +## Step 2 — Read the agent and load its scope contract + +Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong: + +| Scope | Read | +|---|---| +| plugin/APM | `references/agent-scope-plugin-apm.md` | +| project, user | `references/agent-scope-project-user.md` | + +## Step 3 — Qualitative audit + +Read `references/agent-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call. + +| Dimension | Rubric | +|---|---| +| description | `references/agent-description-quality.md` | +| body, delegation, comment-discipline | `references/agent-body-and-delegation.md` | + +Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding. + +Then return to `SKILL.md` Step 4. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/scope-plugin-apm.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-plugin-apm.md similarity index 94% rename from plugins/kyberforge/.apm/skills/agent-audit/references/scope-plugin-apm.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-plugin-apm.md index 54a65e1..acf7842 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/scope-plugin-apm.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-plugin-apm.md @@ -18,13 +18,13 @@ cannot carry either (ADR-0016). That single fact drives everything below. ## Frontmatter allowlist -The permitted keys are the `apm-agent-allowlist` section of `references/field-inventory.md`. Read +The permitted keys are the `apm-agent-allowlist` section of `references/agent-field-inventory.md`. Read them from there. Do not recite the list in a finding, do not work from memory, and do not trust any restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it has changed before, and `validate.sh` parses that same section at load time, so a recitation is a copy that can disagree with the check the agent just ran. -`field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one +`agent-field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one is not. Read that note before arguing with a finding about it. ## Dimension routing @@ -55,4 +55,4 @@ known upstream schema limitation (ADR-0016), not an authoring mistake, and the f give the author visibility into the gap rather than to imply the schema can be made to close it. Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it. -A denylist-shaped restriction is the available half of that — see `field-inventory.md`. +A denylist-shaped restriction is the available half of that — see `agent-field-inventory.md`. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/scope-project-user.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-project-user.md similarity index 90% rename from plugins/kyberforge/.apm/skills/agent-audit/references/scope-project-user.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-project-user.md index 642c128..934d3cc 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/scope-project-user.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-scope-project-user.md @@ -30,7 +30,7 @@ the finding, not presented as a platform spec failure. | everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure | The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of -`references/field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses +`references/agent-field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses those same sections, so any restatement is a copy that can disagree with the check (ADR-0009). ## Field and naming rules that differ by provider @@ -42,7 +42,7 @@ those same sections, so any restatement is a copy that can disagree with the che - `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags it. The Copilot equivalent is `disable-model-invocation`, which changes the description contract - entirely — see `references/description-quality.md`, Step 0. + entirely — see `references/agent-description-quality.md`, Step 0. ## Pair consistency @@ -54,6 +54,6 @@ Check that: - The two files describe the **same job**. Divergent capability claims across the pair mean one half was edited and the other was not, which is the defect this dimension exists to catch. - Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is - the Step 0 case in `references/description-quality.md`, not a pair-consistency finding. + the Step 0 case in `references/agent-description-quality.md`, not a pair-consistency finding. Keep `pair-consistency` in the Step 4 coverage line at these scopes. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/references/validation-scripts.md b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-validation-scripts.md similarity index 93% rename from plugins/kyberforge/.apm/skills/agent-audit/references/validation-scripts.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/agent-validation-scripts.md index 55e949c..8fa304a 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/references/validation-scripts.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/agent-validation-scripts.md @@ -50,18 +50,18 @@ kebab-case; no `FILL IN:` placeholders in the description or body; the descripti characters measured on the folded YAML value. **Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter; -no frontmatter key outside the `apm-agent-allowlist` section of `references/field-inventory.md` — +no frontmatter key outside the `apm-agent-allowlist` section of `references/agent-field-inventory.md` — open that file, do not work from memory. **Project/user scope:** the counterpart file exists; `name` matches the filename stem in the Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in the Copilot file and none from `copilot-only-fields` in the CC file, both read from -`references/field-inventory.md`. +`references/agent-field-inventory.md`. ## Script-specific failures -- **`Error: field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from - `references/field-inventory.md` at load time and refuses to run without it, rather than falling +- **`Error: agent-field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from + `references/agent-field-inventory.md` at load time and refuses to run without it, rather than falling back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not work around it. - **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/body-discipline.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-body-discipline.md similarity index 99% rename from plugins/kyberforge/.apm/skills/skill-audit/references/body-discipline.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-body-discipline.md index b071ca2..b74e1b5 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/body-discipline.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-body-discipline.md @@ -205,5 +205,5 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image... Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead. ``` -The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`, +The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`, which Step 3 loads on every run. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/description-quality.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-description-quality.md similarity index 99% rename from plugins/kyberforge/.apm/skills/skill-audit/references/description-quality.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-description-quality.md index 08b5a7c..6ff1a38 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/description-quality.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-description-quality.md @@ -83,5 +83,5 @@ description: > (`data-model` is illustrative. In a real description the target has to resolve.) -The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`, +The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`, which Step 3 loads on every run. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/file-structure.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md similarity index 85% rename from plugins/kyberforge/.apm/skills/skill-audit/references/file-structure.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md index aaaeb84..99bb81d 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/file-structure.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-file-structure.md @@ -15,8 +15,11 @@ Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification per directories; this house does not, because an unlisted directory is content no auditor and no host knows to look at. Flag any other directory as a FAIL. -- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`, - `test_*.sh`) there are a FAIL — they belong in `tests/`. +- `scripts/` holds only executable code an agent can run, and the sourced libraries those entry + points load. A `lib-*.sh` that is never invoked on its own belongs here beside the entry point + that sources it — it is executable code, not documentation, so do not flag it for failing to run + standalone. Test files (`.bats`, `*_test.*`, `test_*.sh`) there are a FAIL — they belong in + `tests/`. - No non-spec files at the skill root: no `META.md`, no stray config outside the four directories. - An optional directory that exists must hold real content, not an unfilled placeholder README. @@ -38,9 +41,9 @@ Resolve before flagging, twice over: surrounding prose presents it as the form to copy. **Referring to another skill's file.** There is one sanctioned spelling, and it is possessive: -`skill-audit's references/validation-scripts.md`. Write the skill by name and let the reader +`skill-author's references/contract.md`. Write the skill by name and let the reader resolve it — do not spell the repo path. The full path is the thing this section forbids, and -`references/validation-scripts.md` on its own is a hard ERROR from the gate, which +`references/contract.md` on its own is a hard ERROR from the gate, which requires an unqualified `references/` pointer to exist in the skill's OWN directory. The possessive form is the only spelling both rules accept; the gate recognises it and skips the on-disk check. Flag any other spelling of a cross-skill reference. @@ -65,5 +68,5 @@ The skill has to agree with itself. Two checks: - Placeholder READMEs inside `scripts/`, `tests/` and `assets/` say the same thing about each directory that `SKILL.md` does. -The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`, +The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`, which Step 3 loads on every run. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/finding-criteria.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-finding-criteria.md similarity index 94% rename from plugins/kyberforge/.apm/skills/skill-audit/references/finding-criteria.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-finding-criteria.md index 955fd1f..7ee4907 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/finding-criteria.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-finding-criteria.md @@ -19,7 +19,7 @@ Two rules on using it: that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics to read; it does not settle a close call on its own. -## description — `references/description-quality.md` +## description — `references/skill-description-quality.md` Flag as FAIL if: @@ -43,7 +43,7 @@ Flag as FAIL if: available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is judgment. - **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of - `references/description-quality.md`. + `references/skill-description-quality.md`. - **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent of the 400-character house ceiling above. @@ -62,7 +62,7 @@ left to judgment here is semantic and the script cannot reach it: whether a targ resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the files manually") should have named one. -## body-discipline — `references/body-discipline.md` +## body-discipline — `references/skill-body-discipline.md` Flag as FAIL if: @@ -84,7 +84,7 @@ Flag as SUGGESTION if: - Gotchas are correct but placed late in the body rather than near the top - Content that only one branch reaches is inlined where a `references/` file would serve -## patterns — `references/patterns.md` +## patterns — `references/skill-patterns.md` Flag as FAIL if: @@ -101,7 +101,7 @@ Flag as SUGGESTION if: - An output template is present but permissive where the consumer needs it exact - A conditional reference names a trigger that is real but broader than the branch it guards -## file-structure and internal-consistency — `references/file-structure.md` +## file-structure and internal-consistency — `references/skill-file-structure.md` Flag as FAIL if: @@ -117,7 +117,7 @@ Flag as SUGGESTION if: - An optional directory exists but holds only a placeholder README -## formatting and scripts — `references/formatting-and-scripts.md` +## formatting and scripts — `references/skill-formatting-and-scripts.md` Flag as FAIL if: diff --git a/plugins/kyberforge/.apm/skills/factory-audit/references/skill-flow.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-flow.md new file mode 100644 index 0000000..f3619b5 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-flow.md @@ -0,0 +1,62 @@ +--- +source_keys: + - agentskills-home + - agentskills-spec + - agentskills-best-practices + - agentskills-optimizing-descriptions + - agentskills-using-scripts +--- + +# Skill Flow + +Steps 1 to 3 for a skill directory — the target Step 0 matched as a directory containing +`SKILL.md`, or as a `SKILL.md` file, in which case `` below is its parent directory. +Work them in order, then return to `SKILL.md` Step 4 to report. + +## Gotchas + +- A skill takes two independent length families, and it can sit inside one while failing the other — so report them separately. The 500-line / 2,770-word pair counts the **whole file** for spec conformance. The 250/400-character and 600/900-word pair is the house context budget, and its word half counts the **body only**. + +## Step 1 — Deterministic checks + +Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly: + +```bash +bash scripts/validate.sh +bash scripts/validate-provenance.sh +bash scripts/vale-wrap.sh /SKILL.md +``` + +`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation. It exits **0** when no check failed, **1** on real findings, and **2** when it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr. Report an exit 2 as `### Structure` unverified, quoting that reason, never as a failure or a pass. + +Read `references/skill-validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback. + +`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/skill-validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass. + +`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment: + +| Rule | Dimension | +|---|---| +| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description | +| `Kyberforge.SentenceOpenerThereIs` | body-discipline | +| `Kyberforge.PaddingPhrase` | patterns | + +## Step 2 — Read the whole skill + +Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture. + +## Step 3 — Qualitative audit + +Read `references/skill-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call. + +| Dimension | Rubric | +|---|---| +| description | `references/skill-description-quality.md` | +| body-discipline | `references/skill-body-discipline.md` | +| patterns | `references/skill-patterns.md` | +| file-structure, internal-consistency | `references/skill-file-structure.md` | +| formatting, scripts | `references/skill-formatting-and-scripts.md` | + +Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding. + +Then return to `SKILL.md` Step 4. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/formatting-and-scripts.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-formatting-and-scripts.md similarity index 98% rename from plugins/kyberforge/.apm/skills/skill-audit/references/formatting-and-scripts.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-formatting-and-scripts.md index 0b9d23e..c0ecc92 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/formatting-and-scripts.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-formatting-and-scripts.md @@ -44,5 +44,5 @@ follow from that: undocumented one is a coin flip. - **`--dry-run` present for destructive operations.** -The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`, +The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`, which Step 3 loads on every run. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/patterns.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-patterns.md similarity index 96% rename from plugins/kyberforge/.apm/skills/skill-audit/references/patterns.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-patterns.md index bf8337d..281be5e 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/patterns.md +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-patterns.md @@ -10,7 +10,7 @@ Upstream source: agentskills.io — best-practices (instruction patterns), speci Read this when judging the **patterns** dimension: whether each instruction construct a skill uses is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's -*content* is judged in `references/body-discipline.md`. +*content* is judged in `references/skill-body-discipline.md`. ## The constructs and when each is right @@ -50,5 +50,5 @@ forms are judgment. `references/` when only one dispatch branch produces that output. A template inlined for a branch most invocations never take is body-discipline padding. -The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`, +The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`, which Step 3 loads on every run. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/validation-scripts.md b/plugins/kyberforge/.apm/skills/factory-audit/references/skill-validation-scripts.md similarity index 100% rename from plugins/kyberforge/.apm/skills/skill-audit/references/validation-scripts.md rename to plugins/kyberforge/.apm/skills/factory-audit/references/skill-validation-scripts.md diff --git a/plugins/kyberforge/.apm/skills/factory-audit/references/sources.md b/plugins/kyberforge/.apm/skills/factory-audit/references/sources.md new file mode 100644 index 0000000..4f470d7 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/references/sources.md @@ -0,0 +1,153 @@ +--- +source_keys: + - agentskills-home + - agentskills-spec + - agentskills-best-practices + - agentskills-optimizing-descriptions + - agentskills-using-scripts + - context7-websites-code-claude + - claude-code-plugins-docs + - claude-code-subagents-docs + - context7-github-en-copilot + - github-custom-agents-configuration +--- + +# Sources + + + +## agentskills-home + +- **URL:** https://agentskills.io/home.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools +- **Contributing files:** SKILL.md, references/skill-flow.md +- **Status:** `extracted` + +## agentskills-spec + +- **URL:** https://agentskills.io/specification.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation +- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-description-quality.md, references/skill-patterns.md, references/skill-file-structure.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md +- **Status:** `extracted` + +## agentskills-best-practices + +- **URL:** https://agentskills.io/skill-creation/best-practices.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops) +- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-patterns.md, references/skill-finding-criteria.md +- **Status:** `extracted` + +## agentskills-optimizing-descriptions + +- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop +- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-description-quality.md, references/skill-finding-criteria.md +- **Status:** `extracted` + +## agentskills-evaluating-skills + +- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop +- **Contributing files:** (none — eval workflow not directly informing audit dimensions) +- **Status:** `extracted` + +## agentskills-using-scripts + +- **URL:** https://agentskills.io/skill-creation/using-scripts.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency) +- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md +- **Status:** `extracted` + +## agentskills-quickstart + +- **URL:** https://agentskills.io/skill-creation/quickstart.md +- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md +- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice +- **Contributing files:** (none — creation guide not directly informing audit criteria) +- **Status:** `extracted` + +## context7-websites-code-claude + +- **URL:** context7:/websites/code_claude +- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md +- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format +- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md +- **Status:** `extracted` + +## claude-code-plugins-docs + +- **URL:** https://code.claude.com/docs/en/plugins +- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md +- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission +- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-validation-scripts.md +- **Status:** `extracted` + +## claude-code-subagents-docs + +- **URL:** https://code.claude.com/docs/en/sub-agents +- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md +- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations +- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md +- **Status:** `extracted` + +## context7-github-en-copilot + +- **URL:** context7:/websites/github_en_copilot +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace +- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md +- **Status:** `extracted` + +## github-custom-agents-configuration + +- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy +- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md +- **Status:** `extracted` + +## github-cli-plugin-reference + +- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config +- **Contributing files:** (none) +- **Status:** `extracted` + +## github-plugins-creating + +- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle +- **Contributing files:** (none) +- **Status:** `extracted` + +## github-plugins-finding-installing + +- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow +- **Contributing files:** (none) +- **Status:** `extracted` + +## github-plugins-marketplace + +- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands +- **Contributing files:** (none) +- **Status:** `extracted` + +## github-sdk-custom-agents + +- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents +- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md +- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling +- **Contributing files:** (none) +- **Status:** `extracted` diff --git a/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh similarity index 63% rename from plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh rename to plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh index 65a0790..24aaebf 100755 --- a/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh @@ -1,127 +1,54 @@ #!/usr/bin/env bash -set -euo pipefail - -usage() { - cat < - -Validate a skill directory against the agentskills.io specification. - -Arguments: - skill-dir Path to the skill directory containing SKILL.md. - -Exit codes: - 0 All checks passed (may include SUGGESTIONs) - 1 One or more checks failed -EOF -} - -if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then - usage - exit 0 -fi - -if [[ $# -lt 1 ]]; then - echo "Error: skill-dir is required." >&2 - echo "" >&2 - usage >&2 - exit 1 -fi - -# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to -# be measured after YAML folding is resolved, and the hand-rolled reader that -# used to stand in for PyYAML disagreed with it across the 400-character FAIL -# boundary — same description, two verdicts, depending on which reader ran. -# Refusing to start is the only honest option; the repo's jq / apm / vale -# dependencies are declared the same way. -# Check the interpreter separately from the library: `python3 -c` fails the same -# way whether python3 is missing or PyYAML is, and reporting the wrong missing -# dependency sends the reader to install the wrong thing. -if ! command -v python3 > /dev/null 2>&1; then - echo "Error: python3 is required but was not found on PATH." >&2 - echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2 - echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 - exit 1 -fi - -if ! python3 -c 'import yaml' > /dev/null 2>&1; then - echo "Error: PyYAML is required but is not importable by python3." >&2 - echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2 - echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2 - exit 1 -fi - -python3 -u - "$1" <<'PYTHON' -import sys -import os -import re -import glob - -import yaml - -# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams -# default to ASCII, and this script's own message text carries em dashes (the -# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash -# from the read to the write — a UnicodeEncodeError raised while PRINTING, after -# every check has already run, which loses the whole report and (here) flips a -# clean exit 0 into a traceback and an exit 1. read_text() in the shared -# resolver block below pins the reads; this pins the writes. +# lib-boundary-resolver.sh — SOURCED, never executed. # -# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two -# validate.sh copies print findings, skill-size-check.sh has its own top-level -# equivalent, and tests/test-adr0020-contract.sh hashes that block for -# byte-identity across all three. -for _stream in (sys.stdout, sys.stderr): - try: - _stream.reconfigure(encoding='utf-8') - except AttributeError: # pragma: no cover — Python < 3.7 - pass - -skill_dir = os.path.abspath(sys.argv[1]) -skill_md = os.path.join(skill_dir, "SKILL.md") - -if not os.path.isfile(skill_md): - print(f"Error: '{skill_md}' not found.", file=sys.stderr) - sys.exit(1) - -failed = False -suggestions = [] - -def ok(msg): - print(f"PASS {msg}") - -def fail(msg): - # stderr, matching scripts/skill-size-check.sh's ERROR routing. All three - # scripts in the ADR-0020 family now agree: findings that fail the run go to - # stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout. - # Both repo callers capture `2>&1`, so nothing a human reads moves. - global failed - print(f"FAIL {msg}", file=sys.stderr) - failed = True - -def suggest(msg): - # SUGGESTIONs are printed after every check and NEVER touch the exit code. - # skill-audit's Step 4 report counts them into its `PASS (N suggestions)` - # result line, which is what makes the ADR-0020 SUGGESTION tier visible - # rather than another silently-ignored warning (ADR-0013). - suggestions.append(msg) - -def info(msg): - # A check that DECLINED to run says so out loud, rather than passing - # silently. Silence is what let a whole gate family go missing unnoticed. - print(f"INFO {msg}") - +# The ADR-0020 shared boundary resolver, as ONE copy for this skill. Both of +# validate.sh's modes compose it into the Python program they run, so the +# skill-mode and agent-mode check suites resolve boundary targets through the +# same code rather than through two copies that can drift apart. +# +# The resolver is Python, and bash cannot source Python, so the block is held +# in a shell variable filled from a QUOTED here-doc: nothing inside it is +# expanded, substituted or rewritten, and the text between the two markers +# below is therefore byte-identical to the copy in scripts/skill-size-check.sh +# that tests/test-adr0020-contract.sh hashes. The markers stay on lines of +# their own, at column 0, exactly once each, so `sed -n '/^BEGIN$/,/^END$/p'` +# extracts the same span here as it does from the scripts the test already +# reads. Edit one copy, then paste it over the others. +# +# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`. +# This file is sourced by validate.sh before the mode-specific python3/PyYAML +# preflight runs, so a `cat` here made coreutils a hard dependency ahead of +# python3: on a PATH with neither, the script exited 127 naming `cat` instead of +# reaching the preflight that names python3. `read -r -d ''` reads to a NUL that +# never arrives and so returns non-zero at EOF — hence the `|| true` — and it +# keeps the last line's newline, which the joining newline in the caller would +# otherwise double — hence the single strip after the delimiter. It removes +# exactly ONE newline, never a run: blank lines at the end of a chunk are part +# of the program text the entry script reassembles, and stripping every +# trailing newline deleted them. The here-doc itself is unchanged: still +# QUOTED, still byte-identical between its markers. +# +# Self-containment (agentskills.io, skill-author/references/deployment-modes.md) +# binds BETWEEN skills, not within one: a cache-installed plugin copies each +# skill's own directory whole, so a sibling file in this same scripts/ directory +# travels with the skill and is always readable. That is why this is sourced +# here and duplicated across skill boundaries elsewhere. +# +# Consumed by: validate.sh (both modes), via $KYBERFORGE_RESOLVER_PY. +# shellcheck shell=bash +# shellcheck disable=SC2034 +IFS='' read -r -d '' KYBERFORGE_RESOLVER_PY <<'KYBERFORGE_ADR0020_RESOLVER_PY' || true # ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER ===== -# ONE resolver, embedded VERBATIM in three scripts: +# ONE resolver, embedded VERBATIM in two scripts (ADR-0025 retired the third): # scripts/skill-size-check.sh -# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh -# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh -# The block between these markers must stay byte-identical in all three. It is -# copied rather than imported because a cache-installed plugin's scripts cannot -# read files outside their own plugin directory, so there is no single file all -# three can share (same constraint that forces the ADR-0020 constants to be -# duplicated). Edit one copy, then paste it over the other two. +# plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh +# The block between these markers must stay byte-identical in both. It is copied +# rather than imported because a cache-installed plugin's scripts cannot read +# files outside their own plugin directory, and this repo-root hook resolves via +# .pre-commit-hooks.yaml, where entry[0] is the only token pre-commit rewrites -- +# so no single file is reachable by both (the same constraint that duplicates the +# ADR-0020 constants). Edit one copy, then paste it over the other. # # Requires: glob, os, re, yaml (imported by the host script; PyYAML is a hard # dependency, preflighted in bash before the interpreter starts). @@ -155,9 +82,9 @@ def read_text(path): # The set of names a boundary clause may resolve against is derived from an # AUTHORING ROOT found by walking up FROM THE TARGET FILE. It is NEVER derived # from this script's own location: deriving it from ${BASH_SOURCE} leaked -# holocron's 39-skill universe into every consumer repo that ran this hook -# through pre-commit, so a consumer skill routing to `skill-audit` resolved -# against a plugin it had never installed. +# holocron's whole skill universe into every consumer repo that ran this hook +# through pre-commit, so a consumer skill routing to a holocron skill such as +# `factory-audit` resolved against a plugin it had never installed. # # An authoring root is the nearest ancestor holding plugins/*/.apm/skills/ or # plugins/*/.apm/agents/ (a plugin monorepo), falling back to the nearest @@ -417,8 +344,8 @@ def known_targets(start_dir): # written yet, and any new process chain re-arms it. Unexercised is not the # same as unnecessary, and the branch it guards is still load-bearing: the # bare-arrow rule is the sole extractor for three real targets in -# kyberforge's audit skills (agent-audit -> agent-author, agent-audit -> -# skill-audit, skill-audit -> skill-author), all written unbackticked. +# kyberforge (factory-audit -> skill-author, factory-audit -> +# agent-author, apm-orchestrate -> apm-install), all written unbackticked. # * A backticked hyphenated token counts only inside a boundary sentence. # Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard # FAIL with no escape hatch. Gating it costs nothing (measured over this @@ -1051,7 +978,7 @@ def hand_invoked(fm_text): # # Both read a FENCE-MASKED copy of the body. Scanning the raw body made a # ```-fenced example a hard ERROR — and the skills most likely to carry one are -# skill-author and skill-audit, which DOCUMENT the references/ convention — and +# skill-author and factory-audit, which DOCUMENT the references/ convention — and # let a `## Gotchas` heading inside a fenced block stand in for the real # section. Masking preserves every byte offset (content becomes spaces, # newlines stay), so a span found in the mask slices the original. @@ -1076,12 +1003,12 @@ REFERENCE_POINTER = re.compile( REFERENCE_PAST = re.compile( r'\b(?:removed|deleted|renamed|superseded|replaced|obsolete|deprecated' r'|former|formerly|gone|no longer|used to)\b', re.I) -# A pointer QUALIFIED by another skill's name — "skill-audit's -# references/validation-scripts.md" — names a file that is deliberately NOT in +# A pointer QUALIFIED by another skill's name — "factory-audit's +# references/skill-validation-scripts.md" — names a file that is deliberately NOT in # this skill's directory. Requiring it on the local disk left NO legal spelling # for a cross-skill reference at all: the only alternative, a full repo path -# (`plugins/kyberforge/.apm/skills/skill-audit/references/...`), is itself a -# FAIL under skill-audit's own file-structure rubric, because a path that climbs +# (`plugins/kyberforge/.apm/skills/factory-audit/references/...`), is itself a +# FAIL under factory-audit's own file-structure rubric, because a path climbing # out of the skill directory stops resolving once the plugin is cache-installed. # The possessive form is the sanctioned spelling, and it is skipped here. It is # not checked further — this function has no way to locate another skill's @@ -1173,505 +1100,5 @@ def missing_reference_pointers(body, skill_dir): missing.add('references/' + match.group(1)) return sorted(missing) # ===== END ADR-0020 SHARED BOUNDARY RESOLVER ===== - - -# A leading BOM is stripped before anything is parsed or counted. It changes -# neither count below — it is not a line separator and str.split() does not -# treat it as whitespace — but it did defeat the frontmatter match. -try: - content = strip_bom(read_text(skill_md)) -except EncodingError as exc: - fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a " - f"hard failure, not a skip") - print("One or more checks failed.") - sys.exit(1) - -# --- Parse frontmatter --- -fm_match = FRONTMATTER_RE.match(content) -if not fm_match: - fail("No parseable YAML frontmatter block found. Expected a `---` line, the " - "fields, then a closing `---` line (a BOM, leading blank lines, trailing " - "spaces after either marker and CRLF endings are all tolerated). Nothing " - "downstream can be measured, so this is a hard failure, not a skip") - print("One or more checks failed.") - sys.exit(1) - -fm = fm_match.group(1) -body_start = fm_match.end() - -# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE -# a `\s*` after the colon crosses the newline, so a valueless `name:` followed -# by `description: ...` captured the NEXT KEY as the name and reported a -# mismatch instead of an absence. Same class of bug as the `description:` one -# the shared resolver's description_value() docstring records. -name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE) -name = name_m.group(1).strip('"\'') if name_m else "" - -# Extract description — the VALUE, with YAML folding resolved. Most of this -# corpus writes descriptions as `>`-folded block scalars, so the raw lines -# carry indentation and newlines that are not part of the value: every length -# measurement below is wrong unless the scalar is folded first. -try: - desc = description_value(fm) -except FrontmatterError as exc: - # `exc` carries the whole clause — invalid YAML, a non-mapping block, or a - # description of the wrong type. Do not prefix a diagnosis here; the last - # one named a syntax error for two failures that have none. - fail(f"{exc}. Nothing downstream can be measured, so this is a hard " - f"failure, not a skip") - print("One or more checks failed.") - sys.exit(1) - -dir_name = os.path.basename(skill_dir) - -# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation: -# true` takes the skill out of the model-visible listing entirely, so the -# trigger/capability/boundary rules and the 250-character routing target do not -# apply to it — the audit's own references/description-quality.md Step 0 says -# so, and until this line existed no check here knew the field existed. What the -# flag does NOT lift: the body word budget and the 400-character description -# ceiling. See the shared resolver's hand_invoked(). -by_hand = hand_invoked(fm) - -# --- Checks --- - -# name present -if name: - ok(f"name present: '{name}'") -else: - fail("name field is missing or empty") - -# name matches directory -if name and dir_name: - if name == dir_name: - ok(f"name '{name}' matches directory '{dir_name}'") - else: - fail(f"name '{name}' does not match directory '{dir_name}'") - -# name length -if name: - if len(name) <= 64: - ok(f"name length {len(name)} chars (limit: 64)") - else: - fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit") - -# name format -if name: - if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name): - ok("name format valid (kebab-case)") - else: - fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens") - -# description present -if desc: - ok("description present") -else: - fail("description field is missing or empty") - -# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020: -# 1024 is the specification's hard limit, and the ADR-0020 budget gate below -# sits underneath it rather than replacing it. -if desc: - dlen = len(desc) - if dlen <= 1024: - ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)") - else: - fail(f"description length {dlen} chars — exceeds 1024-character limit") - -# Unfilled placeholder detection — matches FILL IN: followed by actual content, -# but not backtick-quoted references like `FILL IN:` used in instructions. -PLACEHOLDER_RE = re.compile(r'(? DESC_MAX_CHARS: - fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character " - f"ADR-0020 ceiling. It is preloaded into every session whether or not the " - f"skill is invoked. Keep a trigger clause, at most one capability clause, " - f"and a boundary clause; move capability enumeration, output-format detail, " - f"composition notes and implementation detail to the body or README.md") - elif dlen > DESC_SUGGEST_CHARS and not by_hand: - suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character " - f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is " - f"what moves the corpus average; the FAIL tier only stops outliers") - elif by_hand: - ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character " - f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)") - else: - ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})") - -# --- ADR-0020: body budget ------------------------------------------------- -# Counts the BODY ONLY — everything after the closing --- of the frontmatter. -# This is a different measurement from MAX_WORDS above, which counts the whole -# file including frontmatter as a spec-conformance backstop. Both are reported. -body_word_count = len(body.split()) -if body_word_count > BODY_MAX_WORDS: - fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word " - f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file " - f"limit above). Move lookup tables, spec restatements, output schemas, templates " - f"and rationale prose to references/ behind an explicit " - f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive " - f"flows, dispatch is mandatory: the body carries the dispatch table and the gates " - f"common to every branch, each flow gets its own self-contained references/ file") -elif body_word_count > BODY_SUGGEST_WORDS: - suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word " - f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})") -else: - ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})") - -# --- Reference pointers must exist ----------------------------------------- -# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not -# on disk is a hard break, and until this check existed nothing in the -# gate/audit/vale stack noticed it — all three exited 0. -missing_refs = missing_reference_pointers(body, skill_dir) -for ref in missing_refs: - fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch " - f"table or \"read X\" trigger naming a missing file sends the agent nowhere") -if not missing_refs: - ok("all referenced references/ files exist") - -# --- Gotchas discipline ----------------------------------------------------- -# SUGGESTION on both counts: the measurement is deterministic, but whether a -# given gotcha earns its place in the body is the auditor's judgment. -gotchas = gotcha_stats(body) -if gotchas is not None: - gotcha_entries, gotcha_words = gotchas - if gotcha_entries > GOTCHA_MAX_ENTRIES: - suggest(f"Gotchas section has {gotcha_entries} entries — over the " - f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a " - f"missing references/ file or a design problem written up as a warning") - if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION: - suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words " - f"({round(100.0 * gotcha_words / body_word_count)}%) — over the " - f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable " - f"parts to references/ and keep the section for live traps") - -# --- ADR-0020: boundary clause present ------------------------------------- -# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether -# this particular skill warrants a boundary clause is judgment. Both accepted -# shapes count — the prose markers and the compressed `Not -> `. -# -# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not -# parse" are different findings, and reporting the first for the second sends -# the author hunting for a problem that is not there (issue #110). -# -# Skipped entirely for a hand-invoked skill — the contract gives it one plain -# sentence with no boundary clause, so the finding would be wrong and its remedy -# names a router that cannot see the skill (issue #108). -if desc and by_hand: - ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger " - "rules do not apply; audited as one plain human-facing sentence") -elif desc: - status = boundary_clause_status(desc) - if status == 'present': - ok("description has a boundary clause") - elif status == 'absent': - suggest("description has no boundary clause — add the prose form (\"Do not use " - "for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") " - "so the router knows where NOT to send this skill") - else: - suggest("description has an arrow boundary clause (\"Not X -> y\") from which no " - "target could be read, so the dangling-target check did not run on it — " - "the clause is PRESENT and unparsed, not missing. Most often the target is " - "a single word, which is deliberately not matchable bare because " - "`research`, `triage` and `forge` are all ordinary English: write it as " - "`name` or /name") - # One arrow, one target. A second name after the same arrow is resolved by - # nothing and reported by nothing, so the clause claims coverage it does not - # have and this script printed "1 of 1 boundary target(s) resolve" on a - # clause naming two (issue #107). - for first, second in multi_target_arrow_clauses(desc): - suggest(f"an arrow boundary clause names more than one target ('{first}', then " - f"'{second}') and only the first is resolved — the second is checked by " - f"nothing. Split it into one arrow per target: \"Not X -> {first}. " - f"Not Y -> {second}.\"") - -# --- ADR-0020: resolvable boundary targets --------------------------------- -# The resolution universe comes from the SKILL's own location: the authoring -# root above it (every sibling plugin in the monorepo), its own apm package, and -# the packages that package declares in apm.yml dependencies.apm. It is never -# derived from this script's own path, and — when an authoring root exists — it -# never reads a deployed .claude/ tree, so a fresh clone and a machine that has -# run `apm install` return the same verdict. See the shared resolver's header. -if desc: - routing_targets = boundary_targets(desc) - known = known_targets(skill_dir) if routing_targets else set() - if routing_targets and not known: - info(f"boundary-target resolution DID NOT RUN — no skill universe could be " - f"determined for this path (no authoring root above it, no apm package " - f"root, no declared apm dependencies, no deployed .claude/ or .agents/ " - f"tree). Unchecked target(s): {', '.join(routing_targets)}") - elif routing_targets: - # blocking vs reported: a target only earns a FAIL when it is written in - # route notation or its own sentence corroborates it by naming another - # target that resolves. See the shared resolver's CORROBORATION note. - unresolved, soft = unresolved_targets(desc, known) - for target in unresolved: - fail(f"description routes to '{target}', which resolves to no skill or agent " - f"in this monorepo, in this package, or in a package it declares in " - f"apm.yml dependencies.apm — a boundary clause naming a non-existent " - f"target sends the router nowhere") - for target in soft: - suggest(f"description routes to '{target}', which resolves to no skill or agent " - f"in this monorepo, in this package, or in a package it declares in " - f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing " - f"else in that sentence resolves, so it is equally likely to be a tool, a " - f"file format or an English compound. If it IS a route, write it as " - f"`/{target}` or `-> {target}` and it will be checked properly") - if not unresolved: - # Counts the targets that ACTUALLY resolve, not every target found: - # a confirm-only target (one used attributively — see the resolver's - # ATTRIBUTIVE USE note) is exempt from the failure above, so - # reporting it as resolved would be a false claim. - resolved = [t for t in routing_targets if normalize_target(t) in known] - ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: " - f"{', '.join(resolved) if resolved else '(none)'}") - -# Body unfilled placeholders -fill_matches = PLACEHOLDER_RE.findall(body) -if fill_matches: - fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)") -else: - ok("SKILL.md body has no unfilled placeholders") - -# Interactive prompt heuristic. -# -# A line-initial `read` only blocks an agent when its stdin is the terminal. -# These forms never touch a TTY and are ordinary data plumbing, so flagging -# them is a false positive — one that has already cost two authors a -# contorted rewrite of working source: -# -# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string -# read -r X <: " X` is interactive and - # must still fail. - unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line) - return '<' in unquoted or prev_line.rstrip().endswith('|') - -# A here-doc body is DATA, not command position. Every script in this corpus -# carries a `usage() { cat < /dev/null 2>&1; then + echo "Error: python3 is required but was not found on PATH." >&2 + echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2 + echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 + # Exit 2, the never-ran tier: no check ran, so this is not a findings result. + # lib-provenance-*.sh has always exited 2 here; this matches it. + exit 2 + fi + + if ! python3 -c 'import yaml' > /dev/null 2>&1; then + echo "Error: PyYAML is required but is not importable by python3." >&2 + echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2 + echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2 + # Exit 2, the never-ran tier: a missing hard dependency is not a findings result. + exit 2 + fi +} + +IFS='' read -r -d '' KYBERFORGE_AGENT_PREAMBLE_PY <<'KYBERFORGE_AGENT_PREAMBLE' || true +import sys +import os +import re +import glob + +import yaml + +# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams +# default to ASCII, and this script's own message text carries em dashes (the +# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash +# from the read to the write — a UnicodeEncodeError raised while PRINTING, after +# every check has already run, which loses the whole report and (here) flips a +# clean exit 0 into a traceback and an exit 1. read_text() in the shared +# resolver block below pins the reads; this pins the writes. +# +# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two +# validate.sh copies print findings, skill-size-check.sh has its own top-level +# equivalent, and tests/test-adr0020-contract.sh hashes that block for +# byte-identity across all three. +for _stream in (sys.stdout, sys.stderr): + try: + _stream.reconfigure(encoding='utf-8') + except AttributeError: # pragma: no cover — Python < 3.7 + pass + +agent_file = os.path.abspath(sys.argv[1]) +script_dir = sys.argv[2] + +fname = os.path.basename(agent_file) + +# --- Detect provider (check .agent.md before .md) --- +if fname.endswith('.agent.md'): + provider = 'copilot' + name_stem = fname[:-len('.agent.md')] +elif fname.endswith('.md'): + provider = 'claude-code' + name_stem = fname[:-len('.md')] +else: + print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr) + sys.exit(2) + +# --- Load agent-field-inventory.md --- +# The merged skill holds one references/ directory for both modes, so every +# flow-specific file is name-prefixed and the agent half of the inventory is +# agent-field-inventory.md (ADR-0025). There is no fallback to the pre-merge +# `field-inventory.md` spelling: agent-audit no longer exists, so a file at that +# name would be a stray, and silently reading it would mean auditing against an +# inventory this skill does not ship. +inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'agent-field-inventory.md')) +if not os.path.isfile(inv_path): + print(f"Error: agent-field-inventory.md not found at {inv_path}", file=sys.stderr) + sys.exit(2) + +# Encoding is pinned to UTF-8 rather than inherited from the locale: under +# LC_ALL=C the inherited default is ASCII, and this file legitimately carries +# non-ASCII prose. read_text() in the shared resolver block below does the same +# thing for every other file; this one is read before that block is defined. +try: + with open(inv_path, encoding='utf-8') as f: + inv_content = f.read() +except UnicodeDecodeError as exc: + print(f"Error: agent-field-inventory.md at {inv_path} is not valid UTF-8 " + f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.", + file=sys.stderr) + sys.exit(2) + +def parse_section_tokens(content, section_name): + lines = content.splitlines() + for i, line in enumerate(lines): + if line.strip() == f'## {section_name}': + for j in range(i + 1, len(lines)): + stripped = lines[j].strip() + if stripped and not stripped.startswith('#') and not stripped.startswith('---'): + return set(stripped.split()) + return set() + +cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields') +copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields') +apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist') + +# Tools the runtime withholds from subagents regardless of the tools field +SUBAGENT_UNAVAILABLE_TOOLS = { + 'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers', +} + +# Copilot body length limit (chars) — content beyond this is silently truncated +COPILOT_BODY_LIMIT = 30000 + +# ADR-0020 description budget. An agent's name + description is preloaded into +# every session exactly like a skill's, so agents take the SAME description +# gates. These two constants are DUPLICATED in three places: +# scripts/skill-size-check.sh, lib-checks-skill.sh beside this file, and here. +# The repo-root hook's copy cannot be shared with this skill — a cache-installed +# plugin's scripts cannot read files outside their own plugin directory, and the +# hook cannot reach inside the plugin. The two copies INSIDE this skill could be +# shared (ADR-0025: two files in one skill may source a third), and are not only +# because each mode library is a verbatim lift of the pre-merge suite whose +# constants sit in its Python preamble; hoisting them is a separate change. +# tests/test-skill-size-check.sh asserts all three agree, so drift fails CI +# rather than silently diverging. +# +# Agents deliberately take NO body word gate, and adding one here would +# contradict ADR-0020: a skill body is loaded into the caller's context and +# competes with the live conversation, while an agent body becomes the system +# prompt of a fresh context. The rationale for the 900-word skill ceiling does +# not transfer. Agent body length falls out of the delegation rule instead. +DESC_SUGGEST_CHARS = 250 +DESC_MAX_CHARS = 400 + +# --- Helpers (shared by every scope) --- +failed = False +suggestions = [] + +def fail(msg): + # stderr, matching scripts/skill-size-check.sh's ERROR routing. All three + # scripts in the ADR-0020 family now agree: findings that fail the run go to + # stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo + # callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture + # `2>&1`, so nothing a human reads moves. + global failed + failed = True + print(f"FAIL {msg}", file=sys.stderr) + +def suggest(msg): + suggestions.append(msg) + +def info(msg): + # A check that DECLINED to run says so out loud, rather than passing + # silently. Silence is what let a whole gate family go missing unnoticed. + print(f"INFO {msg}") + +PLACEHOLDER_RE = re.compile(r'(?`, `null`, `''` and a quoted `"description"` key alike. + """ + m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE) + return m.group(1).strip() if m else None + +def get_frontmatter_keys(fm): + keys = set() + for line in fm.splitlines(): + m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line) + if m: + keys.add(m.group(1)) + return keys + +def agent_description(fm, local_fname): + """The folded description VALUE, or None if it could not be read.""" + try: + return description_value(fm) + except FrontmatterError as exc: + # `exc` carries the whole clause — invalid YAML, a non-mapping block, or + # a description of the wrong type. Do not prefix a diagnosis here; the + # last one named a syntax error for two failures that have none. + fail(f"{exc} — the ADR-0020 description and boundary-target gates could " + f"not run — {local_fname}") + return None + +def check_description_budget(value, local_fname, by_hand=False): + """ADR-0020 description gates — identical for every scope. + + `by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent + carrying `disable-model-invocation: true` is absent from the model-visible + listing, so the 250-character SUGGESTION — a routing-quality budget — has + no listing to apply to. The 400-character ceiling is unaffected. + """ + if not value: + return + dlen = len(value) + if dlen > DESC_MAX_CHARS: + fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character " + f"ADR-0020 ceiling. It is preloaded into every session whether or not the " + f"agent is invoked. Keep a trigger clause, at most one capability clause, " + f"and a boundary clause; move capability enumeration, output-format detail, " + f"composition notes and implementation detail to the body — {local_fname}") + elif dlen > DESC_SUGGEST_CHARS and not by_hand: + suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character " + f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is " + f"what moves the corpus average; the FAIL tier only stops outliers " + f"— {local_fname}") + +def check_boundary(value, fpath, local_fname, by_hand=False): + """ADR-0020 boundary clause + resolvable boundary targets. + + agent-author's SKILL.md states that an agent's boundary targets must + resolve, but until this ran no script checked it — the contract was + documented and unenforced. The resolution universe is derived from the + AGENT FILE's own location (the authoring root above it, its own apm + package, and that package's declared apm dependencies), never from this + script's path, and — when an authoring root exists — never from a deployed + .claude/ tree, so a fresh clone and a machine that has run `apm install` + return the same verdict. + """ + if not value: + return + # SUGGESTION, not FAIL: detecting the absence is deterministic, but whether + # this particular agent warrants a boundary clause is judgment. All four + # agents in this corpus currently lack one. + # + # THREE outcomes, not two: "no boundary clause" and "boundary clause I could + # not parse" are different findings (issue #110). And a hand-invoked agent is + # exempt from the clause altogether (issue #108) — the boundary-target + # resolution below still runs, because a target it DOES name should still + # resolve. + status = boundary_clause_status(value) if not by_hand else 'present' + if status == 'absent': + suggest(f"description has no boundary clause — add the prose form (\"Do not use " + f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") " + f"so the router knows where NOT to send this agent — {local_fname}") + elif status == 'unparsed': + suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no " + f"target could be read, so the dangling-target check did not run on it — " + f"the clause is PRESENT and unparsed, not missing. Most often the target " + f"is a single word, which is deliberately not matchable bare: write it as " + f"`name` or /name — {local_fname}") + if not by_hand: + # One arrow, one target: a second name after the same arrow is resolved + # by nothing and reported by nothing (issue #107). + for first, second in multi_target_arrow_clauses(value): + suggest(f"an arrow boundary clause names more than one target ('{first}', then " + f"'{second}') and only the first is resolved — the second is checked by " + f"nothing. Split it into one arrow per target: \"Not X -> {first}. " + f"Not Y -> {second}.\" — {local_fname}") + targets = boundary_targets(value) + if not targets: + return + known = known_targets(os.path.dirname(os.path.abspath(fpath))) + if not known: + info(f"boundary-target resolution DID NOT RUN — no skill universe could be " + f"determined for this path (no authoring root above it, no apm package " + f"root, no declared apm dependencies, no deployed .claude/ or .agents/ " + f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}") + return + # blocking vs reported: a target only earns a FAIL when it is written in + # route notation or its own sentence corroborates it by naming another target + # that resolves. See the shared resolver's CORROBORATION note. + blocking, reported = unresolved_targets(value, known) + for target in blocking: + fail(f"description routes to '{target}', which resolves to no skill or agent " + f"in this monorepo, in this package, or in a package it declares in " + f"apm.yml dependencies.apm — a boundary clause naming a non-existent " + f"target sends the router nowhere — {local_fname}") + for target in reported: + suggest(f"description routes to '{target}', which resolves to no skill or agent " + f"in this monorepo, in this package, or in a package it declares in " + f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing " + f"else in that sentence resolves, so it is equally likely to be a tool, a " + f"file format or an English compound. If it IS a route, write it as " + f"`/{target}` or `-> {target}` and it will be checked properly — " + f"{local_fname}") + +def extract_tools_list(fm): + """Tool names from the `tools` field — inline scalar OR YAML block sequence. + + Read off the PARSED mapping, never off extract_field(). That function's + capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:` + written as a block sequence — the shape Copilot agent files use — captured + nothing at all and the subagent-unavailable-tool check silently stopped + firing on exactly the files it was written for. Both spellings are legal + YAML, so both are read here. + """ + try: + data = yaml.safe_load(fm) + except Exception: + # Not this function's failure to report: the frontmatter's validity is + # decided (and failed) by agent_description() on the same text. + return set() + if not isinstance(data, dict): + return set() + val = data.get('tools') + if isinstance(val, list): + items = [str(item).strip() for item in val] + elif isinstance(val, str): + items = re.split(r'[\s,]+', val.strip()) + else: + return set() + return {item for item in items if item} + +def is_copilot_cloud_ide(fpath): + """True if the file is a cloud/IDE Copilot agent (name is optional for these).""" + return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/') + +# --- Detect scope --- +# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is +# exactly one of the four package content types. Group 1 captures an optional +# opening quote; \1 requires the same character (or nothing) to close it, so +# "skill" and '"skill"' both match but a mismatched quote doesn't. The value +# must then be followed by whitespace or end-of-line — not just a non-word +# character — so a malformed value like `prompts-only` is correctly rejected +# instead of false-matching on the `prompts` prefix. +APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)") + +def find_apm_package_root(apm_yml_path): + """Return True if apm_yml_path has a top-level type: line (i.e. is a package + manifest, not a type:-less marketplace-only apm.yml).""" + # errors='replace', not a hard failure: this only asks whether a `type:` + # line exists, and a stray undecodable byte elsewhere in someone else's + # apm.yml must not abort scope detection. + with open(apm_yml_path, encoding='utf-8', errors='replace') as f: + for line in f: + if APM_TYPE_RE.match(line): + return True + return False + +def detect_scope(start_dir): + home = os.path.expanduser('~') + original_start = os.path.abspath(start_dir) + # Agent files conventionally live exactly two path segments below their + # scope root — /.claude/agents, /.github/agents, + # /.copilot/agents, or /.apm/agents (see new-agent.sh's + # CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments + # recovers the same root new-agent.sh would have been invoked with to + # produce this exact file, independent of how far the walk below has to + # travel to find (or fail to find) a marker — mirrors new-agent.sh's + # `root` vs `current` distinction even though validate.sh is handed a + # file's directory, not the scope root itself. + # + # That arithmetic is only trustworthy when the path actually has this + # shape: parent directory literally named "agents", grandparent one of + # the four known scope-dir names. A hand-placed or otherwise + # non-conventional agent file (never produced by new-agent.sh) has no + # such guarantee — blindly trusting two-segments-up there could point at + # an unrelated ancestor. conventional_shape gates every use of + # conventional_root below; when it's false, the walked-to `current` + # directory is used instead, the same fallback this function used before + # conventional_root existed. + scope_dir_name = os.path.basename(os.path.dirname(original_start)) + conventional_shape = ( + os.path.basename(original_start) == 'agents' + and scope_dir_name in ('.claude', '.github', '.copilot', '.apm') + ) + conventional_root = os.path.dirname(os.path.dirname(original_start)) + current = original_start + while True: + # The filesystem root is never a candidate, the same guard the shared + # resolver's walk-up loops carry. Without it a file under a marker-less + # temp directory walked all the way to `/` and returned it as the scope + # root, which then reported `counterpart file not found: + # /.claude/agents/.md` — a path that names someone else's machine, + # not the user's project. When the walk runs out, the agent file's own + # directory (or its conventional root) is the honest answer. + if _is_fs_root(current): + return 'project', conventional_root if conventional_shape else original_start + apm_yml = os.path.join(current, 'apm.yml') + if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml): + return 'plugin', current + # $HOME is the user-scope boundary — checked before the .git test + # below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) + # can't shadow user scope by being its own .git repo. 'user' scope + # requires EITHER start_dir to BE $HOME itself (no walk-up — the + # new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the + # conventional two-segments-below-root depth (i.e. $HOME IS that + # root, matching the real ~/.claude/agents or ~/.copilot/agents + # shape). Any other walk-up into $HOME — a marker-less directory + # nested deeper than that convention — resolves to project scope + # instead: a stray directory under $HOME can't be silently + # redirected into the shared global ~/.claude or ~/.copilot agent + # directories. + if current == home: + if original_start == home or (conventional_shape and conventional_root == home): + return 'user', home + return 'project', conventional_root if conventional_shape else current + # .git is a directory in a normal checkout but a file (`gitdir: ...`) + # in a git worktree — exists() covers both. Returns conventional_root, + # not current: new-agent.sh's project-scope file placement always + # uses its `$ROOT` argument directly, never the walked-up `.git` + # location, so a one or more levels below the repo's .git + # (a subdirectory of a larger git-tracked tree — explicitly a + # supported case per new-agent.sh's usage text) must resolve to the + # same root new-agent.sh actually wrote to, not to the .git dir — + # unless the path lacks the conventional shape, in which case that + # arithmetic isn't trustworthy and current is used instead. + if os.path.exists(os.path.join(current, '.git')): + return 'project', conventional_root if conventional_shape else current + parent = os.path.dirname(current) + if parent == current: + return 'project', conventional_root if conventional_shape else current + current = parent + +agent_dir = os.path.dirname(agent_file) +scope, scope_root = detect_scope(agent_dir) + +# --- Plugin/APM scope: single vendor-neutral file, no counterpart --- +def check_apm_agent_file(fpath, allowlist, stem): + local_fname = os.path.basename(fpath) + try: + content = read_text(fpath) + except EncodingError as exc: + fail(f"file is {exc}. Nothing could be measured, so this is a hard " + f"failure, not a skip — {local_fname}") + return + except OSError as exc: + # A path that cannot be opened gets a FAIL line naming it, not a bare + # FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes + # this path for an agent file deleted from the worktree but still + # tracked in the index — a real, expected state, and the caller needs to + # be told which file, not handed an interpreter stack. + fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could " + f"be measured, so this is a hard failure, not a skip — {local_fname}") + return + + fm, body = parse_frontmatter(content) + if fm is None: + fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, " + f"then a closing `---` line (a BOM, leading blank lines, trailing spaces " + f"after either marker and CRLF endings are all tolerated). Nothing could be " + f"measured, so this is a hard failure, not a skip — {local_fname}") + return + + # The apm-agent.md template embeds its authoring guidance as HTML + # comments inside the frontmatter block (so they render invisible in a + # Markdown preview but stay visible in the raw file). get_frontmatter_keys + # silently ignores any line that isn't a `key:` match, so a comment left + # behind at ship time would otherwise pass unnoticed — yet apm compile + # copies this frontmatter verbatim to both harnesses, and `` is + # not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016). + if re.search(r'', fm): + fail(f"frontmatter still contains template HTML comments () " + f"— delete them before shipping — {local_fname}") + + # Allowlist: the permitted keys are data, read at load time from + # references/agent-field-inventory.md's `## apm-agent-allowlist` section — do not + # restate them here, or this comment goes stale the next time that line + # changes. apm compile verbatim-copies frontmatter to every target, so a key + # outside the list is unsafe on at least one harness (ADR-0016). Note the + # list admits denylist-shaped restrictions (disallowedTools) but never + # allowlist-shaped ones (tools), whose value shape differs per harness. + fm_keys = get_frontmatter_keys(fm) + for key in sorted(fm_keys): + if key not in allowlist: + fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist " + f"({', '.join(sorted(allowlist))}) — {local_fname}") + + # name — required, kebab-case, must match filename stem (file is .agent.md) + name_val = extract_field(fm, 'name') + if not name_val: + fail(f"name field is missing or empty — {local_fname}") + else: + if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): + fail(f"name '{name_val}' is not kebab-case — {local_fname}") + if name_val != stem: + fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}") + + # description — required, non-empty, no placeholder + # Presence is decided on the FOLDED value, never on a line regex. Deciding + # it on extract_field's raw capture is what let `description:` with no value + # pass this gate in total silence: the capture picked up the next key, so + # "missing or empty" never fired, and every ADR-0020 check below then + # early-returned on the empty folded value. Exit 0, zero output, no gate run. + folded = agent_description(fm, local_fname) + if folded is None: + pass # frontmatter is not valid YAML — agent_description already failed + elif not folded: + fail(f"description field is missing or empty — {local_fname}") + else: + if PLACEHOLDER_RE.search(folded): + fail(f"description contains unfilled FILL IN: placeholder — {local_fname}") + by_hand = hand_invoked(fm) + check_description_budget(folded, local_fname, by_hand) + check_boundary(folded, fpath, local_fname, by_hand) + + # body — required, non-empty, no placeholder; same Copilot truncation risk + # applies since this file compiles verbatim into a real Copilot file downstream. + if not body.strip(): + fail(f"system prompt body is empty — {local_fname}") + else: + if PLACEHOLDER_RE.search(body): + fail(f"body contains unfilled FILL IN: placeholder — {local_fname}") + if len(body) > COPILOT_BODY_LIMIT: + suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — " + f"content beyond the limit is silently truncated by the Copilot runtime " + f"once apm compile emits it downstream — {local_fname}") + +if scope == 'plugin': + check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem) + for s in suggestions: + print(f"SUGGESTION {s}") + sys.exit(1 if failed else 0) + +# --- Project/user scope: unchanged CC/Copilot pair validation --- + +# --- Derive counterpart path --- +if scope == 'project': + if provider == 'claude-code': + counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md') + counterpart_provider = 'copilot' + else: + counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md') + counterpart_provider = 'claude-code' +else: # user + home = os.path.expanduser('~') + if provider == 'claude-code': + counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md') + counterpart_provider = 'copilot' + else: + counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md') + counterpart_provider = 'claude-code' + +def check_file(fpath, file_provider): + local_fname = os.path.basename(fpath) + try: + content = read_text(fpath) + except EncodingError as exc: + fail(f"file is {exc}. Nothing could be measured, so this is a hard " + f"failure, not a skip — {local_fname}") + return + except OSError as exc: + # Same reason as check_apm_agent_file's: a diagnostic naming the path + # beats a FileNotFoundError traceback. The counterpart is pre-checked at + # the bottom of this script, but agent_file itself never was. + fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could " + f"be measured, so this is a hard failure, not a skip — {local_fname}") + return + + fm, body = parse_frontmatter(content) + if fm is None: + fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, " + f"then a closing `---` line (a BOM, leading blank lines, trailing spaces " + f"after either marker and CRLF endings are all tolerated). Nothing could be " + f"measured, so this is a hard failure, not a skip — {local_fname}") + return + + # name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents + cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath)) + name_val = extract_field(fm, 'name') + if not cloud_ide: + if not name_val: + fail(f"name field is missing or empty — {local_fname}") + else: + if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): + fail(f"name '{name_val}' is not kebab-case — {local_fname}") + # Stem check applies to Copilot CLI only; CC docs say filename need not match name + if file_provider == 'copilot': + stem = local_fname[:-len('.agent.md')] + if name_val != stem: + fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}") + elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val): + # cloud/IDE: name is optional, but if present it must be valid + fail(f"name '{name_val}' is not kebab-case — {local_fname}") + + # description + # Presence is decided on the FOLDED value, never on a line regex. Deciding + # it on extract_field's raw capture is what let `description:` with no value + # pass this gate in total silence: the capture picked up the next key, so + # "missing or empty" never fired, and every ADR-0020 check below then + # early-returned on the empty folded value. Exit 0, zero output, no gate run. + folded = agent_description(fm, local_fname) + if folded is None: + pass # frontmatter is not valid YAML — agent_description already failed + elif not folded: + fail(f"description field is missing or empty — {local_fname}") + else: + if PLACEHOLDER_RE.search(folded): + fail(f"description contains unfilled FILL IN: placeholder — {local_fname}") + by_hand = hand_invoked(fm) + check_description_budget(folded, local_fname, by_hand) + check_boundary(folded, fpath, local_fname, by_hand) + + # body + if not body.strip(): + fail(f"system prompt body is empty — {local_fname}") + else: + if PLACEHOLDER_RE.search(body): + fail(f"body contains unfilled FILL IN: placeholder — {local_fname}") + # Copilot body length limit + if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT: + suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}") + + # CC-only fields in Copilot file + if file_provider == 'copilot': + fm_keys = get_frontmatter_keys(fm) + for key in sorted(fm_keys): + if key in cc_only_fields: + fail(f"CC-only field '{key}' present in Copilot file — {local_fname}") + + # Copilot-only fields in CC file + if file_provider == 'claude-code': + fm_keys = get_frontmatter_keys(fm) + for key in sorted(fm_keys): + if key in copilot_only_fields: + fail(f"Copilot-only field '{key}' present in CC file — {local_fname}") + + # Subagent-unavailable tools listed in tools field + tools = extract_tools_list(fm) + unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS + for tool in sorted(unavailable): + suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}") + +# --- Check counterpart exists --- +if not os.path.isfile(counterpart): + fail(f"counterpart file not found: {counterpart}") + sys.exit(1) + +# --- Check both files --- +check_file(agent_file, provider) +check_file(counterpart, counterpart_provider) + +for s in suggestions: + print(f"SUGGESTION {s}") + +sys.exit(1 if failed else 0) +KYBERFORGE_AGENT_BODY +KYBERFORGE_AGENT_BODY_PY="${KYBERFORGE_AGENT_BODY_PY%$'\n'}" diff --git a/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh new file mode 100755 index 0000000..8585a01 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh @@ -0,0 +1,621 @@ +#!/usr/bin/env bash +# lib-checks-skill.sh — SOURCED, never executed. +# +# skill-audit's structural check suite: everything in its validate.sh that is +# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the +# resolver's markers. validate.sh reassembles +# +# $KYBERFORGE_SKILL_PREAMBLE_PY +# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh) +# $KYBERFORGE_SKILL_BODY_PY +# +# in that order — the order the resolver block sat in the original file — and +# feeds the result to python3, so every check runs against the same names it +# always did. +# +# The dimension vocabulary, the message wording and the PASS/FAIL/SUGGESTION/ +# INFO tiers here are skill-audit's and are deliberately NOT reconciled with +# lib-checks-agent.sh's. The two suites disagree on purpose: a skill body is +# loaded into the caller's context, an agent body becomes the system prompt of +# a fresh one, so ADR-0020 gives skills a body word budget and agents none. +# +# Consumed by: validate.sh, skill mode. +# shellcheck shell=bash +# shellcheck disable=SC2034 + +kyberforge_skill_preflight() { + # PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to + # be measured after YAML folding is resolved, and the hand-rolled reader that + # used to stand in for PyYAML disagreed with it across the 400-character FAIL + # boundary — same description, two verdicts, depending on which reader ran. + # Refusing to start is the only honest option; the repo's jq / apm / vale + # dependencies are declared the same way. + # Check the interpreter separately from the library: `python3 -c` fails the same + # way whether python3 is missing or PyYAML is, and reporting the wrong missing + # dependency sends the reader to install the wrong thing. + if ! command -v python3 > /dev/null 2>&1; then + echo "Error: python3 is required but was not found on PATH." >&2 + echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2 + echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 + # Exit 2, the never-ran tier: no check ran, so this is not a findings result. + # lib-provenance-*.sh has always exited 2 here; this matches it. + exit 2 + fi + + if ! python3 -c 'import yaml' > /dev/null 2>&1; then + echo "Error: PyYAML is required but is not importable by python3." >&2 + echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2 + echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2 + # Exit 2, the never-ran tier: a missing hard dependency is not a findings result. + exit 2 + fi +} + +IFS='' read -r -d '' KYBERFORGE_SKILL_PREAMBLE_PY <<'KYBERFORGE_SKILL_PREAMBLE' || true +import sys +import os +import re +import glob + +import yaml + +# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams +# default to ASCII, and this script's own message text carries em dashes (the +# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash +# from the read to the write — a UnicodeEncodeError raised while PRINTING, after +# every check has already run, which loses the whole report and (here) flips a +# clean exit 0 into a traceback and an exit 1. read_text() in the shared +# resolver block below pins the reads; this pins the writes. +# +# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two +# validate.sh copies print findings, skill-size-check.sh has its own top-level +# equivalent, and tests/test-adr0020-contract.sh hashes that block for +# byte-identity across all three. +for _stream in (sys.stdout, sys.stderr): + try: + _stream.reconfigure(encoding='utf-8') + except AttributeError: # pragma: no cover — Python < 3.7 + pass + +skill_dir = os.path.abspath(sys.argv[1]) +skill_md = os.path.join(skill_dir, "SKILL.md") + +if not os.path.isfile(skill_md): + print(f"Error: '{skill_md}' not found.", file=sys.stderr) + sys.exit(1) + +failed = False +suggestions = [] + +def ok(msg): + print(f"PASS {msg}") + +def fail(msg): + # stderr, matching scripts/skill-size-check.sh's ERROR routing. All three + # scripts in the ADR-0020 family now agree: findings that fail the run go to + # stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout. + # Both repo callers capture `2>&1`, so nothing a human reads moves. + global failed + print(f"FAIL {msg}", file=sys.stderr) + failed = True + +def suggest(msg): + # SUGGESTIONs are printed after every check and NEVER touch the exit code. + # factory-audit's SKILL.md Step 4 report counts them into its + # `PASS (N suggestions)` result line, which is what makes the ADR-0020 SUGGESTION tier visible + # rather than another silently-ignored warning (ADR-0013). + suggestions.append(msg) + +def info(msg): + # A check that DECLINED to run says so out loud, rather than passing + # silently. Silence is what let a whole gate family go missing unnoticed. + print(f"INFO {msg}") + + +KYBERFORGE_SKILL_PREAMBLE +KYBERFORGE_SKILL_PREAMBLE_PY="${KYBERFORGE_SKILL_PREAMBLE_PY%$'\n'}" + +IFS='' read -r -d '' KYBERFORGE_SKILL_BODY_PY <<'KYBERFORGE_SKILL_BODY' || true + + +# A leading BOM is stripped before anything is parsed or counted. It changes +# neither count below — it is not a line separator and str.split() does not +# treat it as whitespace — but it did defeat the frontmatter match. +try: + content = strip_bom(read_text(skill_md)) +except EncodingError as exc: + fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a " + f"hard failure, not a skip") + print("One or more checks failed.") + sys.exit(1) + +# --- Parse frontmatter --- +fm_match = FRONTMATTER_RE.match(content) +if not fm_match: + fail("No parseable YAML frontmatter block found. Expected a `---` line, the " + "fields, then a closing `---` line (a BOM, leading blank lines, trailing " + "spaces after either marker and CRLF endings are all tolerated). Nothing " + "downstream can be measured, so this is a hard failure, not a skip") + print("One or more checks failed.") + sys.exit(1) + +fm = fm_match.group(1) +body_start = fm_match.end() + +# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE +# a `\s*` after the colon crosses the newline, so a valueless `name:` followed +# by `description: ...` captured the NEXT KEY as the name and reported a +# mismatch instead of an absence. Same class of bug as the `description:` one +# the shared resolver's description_value() docstring records. +name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE) +name = name_m.group(1).strip('"\'') if name_m else "" + +# Extract description — the VALUE, with YAML folding resolved. Most of this +# corpus writes descriptions as `>`-folded block scalars, so the raw lines +# carry indentation and newlines that are not part of the value: every length +# measurement below is wrong unless the scalar is folded first. +try: + desc = description_value(fm) +except FrontmatterError as exc: + # `exc` carries the whole clause — invalid YAML, a non-mapping block, or a + # description of the wrong type. Do not prefix a diagnosis here; the last + # one named a syntax error for two failures that have none. + fail(f"{exc}. Nothing downstream can be measured, so this is a hard " + f"failure, not a skip") + print("One or more checks failed.") + sys.exit(1) + +dir_name = os.path.basename(skill_dir) + +# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation: +# true` takes the skill out of the model-visible listing entirely, so the +# trigger/capability/boundary rules and the 250-character routing target do not +# apply to it — the audit's own references/skill-description-quality.md Step 0 says +# so, and until this line existed no check here knew the field existed. What the +# flag does NOT lift: the body word budget and the 400-character description +# ceiling. See the shared resolver's hand_invoked(). +by_hand = hand_invoked(fm) + +# --- Checks --- + +# name present +if name: + ok(f"name present: '{name}'") +else: + fail("name field is missing or empty") + +# name matches directory +if name and dir_name: + if name == dir_name: + ok(f"name '{name}' matches directory '{dir_name}'") + else: + fail(f"name '{name}' does not match directory '{dir_name}'") + +# name length +if name: + if len(name) <= 64: + ok(f"name length {len(name)} chars (limit: 64)") + else: + fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit") + +# name format +if name: + if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name): + ok("name format valid (kebab-case)") + else: + fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens") + +# description present +if desc: + ok("description present") +else: + fail("description field is missing or empty") + +# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020: +# 1024 is the specification's hard limit, and the ADR-0020 budget gate below +# sits underneath it rather than replacing it. +if desc: + dlen = len(desc) + if dlen <= 1024: + ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)") + else: + fail(f"description length {dlen} chars — exceeds 1024-character limit") + +# Unfilled placeholder detection — matches FILL IN: followed by actual content, +# but not backtick-quoted references like `FILL IN:` used in instructions. +PLACEHOLDER_RE = re.compile(r'(? DESC_MAX_CHARS: + fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character " + f"ADR-0020 ceiling. It is preloaded into every session whether or not the " + f"skill is invoked. Keep a trigger clause, at most one capability clause, " + f"and a boundary clause; move capability enumeration, output-format detail, " + f"composition notes and implementation detail to the body or README.md") + elif dlen > DESC_SUGGEST_CHARS and not by_hand: + suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character " + f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is " + f"what moves the corpus average; the FAIL tier only stops outliers") + elif by_hand: + ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character " + f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)") + else: + ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})") + +# --- ADR-0020: body budget ------------------------------------------------- +# Counts the BODY ONLY — everything after the closing --- of the frontmatter. +# This is a different measurement from MAX_WORDS above, which counts the whole +# file including frontmatter as a spec-conformance backstop. Both are reported. +body_word_count = len(body.split()) +if body_word_count > BODY_MAX_WORDS: + fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word " + f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file " + f"limit above). Move lookup tables, spec restatements, output schemas, templates " + f"and rationale prose to references/ behind an explicit " + f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive " + f"flows, dispatch is mandatory: the body carries the dispatch table and the gates " + f"common to every branch, each flow gets its own self-contained references/ file") +elif body_word_count > BODY_SUGGEST_WORDS: + suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word " + f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})") +else: + ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})") + +# --- Reference pointers must exist ----------------------------------------- +# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not +# on disk is a hard break, and until this check existed nothing in the +# gate/audit/vale stack noticed it — all three exited 0. +missing_refs = missing_reference_pointers(body, skill_dir) +for ref in missing_refs: + fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch " + f"table or \"read X\" trigger naming a missing file sends the agent nowhere") +if not missing_refs: + ok("all referenced references/ files exist") + +# --- Gotchas discipline ----------------------------------------------------- +# SUGGESTION on both counts: the measurement is deterministic, but whether a +# given gotcha earns its place in the body is the auditor's judgment. +gotchas = gotcha_stats(body) +if gotchas is not None: + gotcha_entries, gotcha_words = gotchas + if gotcha_entries > GOTCHA_MAX_ENTRIES: + suggest(f"Gotchas section has {gotcha_entries} entries — over the " + f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a " + f"missing references/ file or a design problem written up as a warning") + if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION: + suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words " + f"({round(100.0 * gotcha_words / body_word_count)}%) — over the " + f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable " + f"parts to references/ and keep the section for live traps") + +# --- ADR-0020: boundary clause present ------------------------------------- +# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether +# this particular skill warrants a boundary clause is judgment. Both accepted +# shapes count — the prose markers and the compressed `Not -> `. +# +# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not +# parse" are different findings, and reporting the first for the second sends +# the author hunting for a problem that is not there (issue #110). +# +# Skipped entirely for a hand-invoked skill — the contract gives it one plain +# sentence with no boundary clause, so the finding would be wrong and its remedy +# names a router that cannot see the skill (issue #108). +if desc and by_hand: + ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger " + "rules do not apply; audited as one plain human-facing sentence") +elif desc: + status = boundary_clause_status(desc) + if status == 'present': + ok("description has a boundary clause") + elif status == 'absent': + suggest("description has no boundary clause — add the prose form (\"Do not use " + "for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") " + "so the router knows where NOT to send this skill") + else: + suggest("description has an arrow boundary clause (\"Not X -> y\") from which no " + "target could be read, so the dangling-target check did not run on it — " + "the clause is PRESENT and unparsed, not missing. Most often the target is " + "a single word, which is deliberately not matchable bare because " + "`research`, `triage` and `forge` are all ordinary English: write it as " + "`name` or /name") + # One arrow, one target. A second name after the same arrow is resolved by + # nothing and reported by nothing, so the clause claims coverage it does not + # have and this script printed "1 of 1 boundary target(s) resolve" on a + # clause naming two (issue #107). + for first, second in multi_target_arrow_clauses(desc): + suggest(f"an arrow boundary clause names more than one target ('{first}', then " + f"'{second}') and only the first is resolved — the second is checked by " + f"nothing. Split it into one arrow per target: \"Not X -> {first}. " + f"Not Y -> {second}.\"") + +# --- ADR-0020: resolvable boundary targets --------------------------------- +# The resolution universe comes from the SKILL's own location: the authoring +# root above it (every sibling plugin in the monorepo), its own apm package, and +# the packages that package declares in apm.yml dependencies.apm. It is never +# derived from this script's own path, and — when an authoring root exists — it +# never reads a deployed .claude/ tree, so a fresh clone and a machine that has +# run `apm install` return the same verdict. See the shared resolver's header. +if desc: + routing_targets = boundary_targets(desc) + known = known_targets(skill_dir) if routing_targets else set() + if routing_targets and not known: + info(f"boundary-target resolution DID NOT RUN — no skill universe could be " + f"determined for this path (no authoring root above it, no apm package " + f"root, no declared apm dependencies, no deployed .claude/ or .agents/ " + f"tree). Unchecked target(s): {', '.join(routing_targets)}") + elif routing_targets: + # blocking vs reported: a target only earns a FAIL when it is written in + # route notation or its own sentence corroborates it by naming another + # target that resolves. See the shared resolver's CORROBORATION note. + unresolved, soft = unresolved_targets(desc, known) + for target in unresolved: + fail(f"description routes to '{target}', which resolves to no skill or agent " + f"in this monorepo, in this package, or in a package it declares in " + f"apm.yml dependencies.apm — a boundary clause naming a non-existent " + f"target sends the router nowhere") + for target in soft: + suggest(f"description routes to '{target}', which resolves to no skill or agent " + f"in this monorepo, in this package, or in a package it declares in " + f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing " + f"else in that sentence resolves, so it is equally likely to be a tool, a " + f"file format or an English compound. If it IS a route, write it as " + f"`/{target}` or `-> {target}` and it will be checked properly") + if not unresolved: + # Counts the targets that ACTUALLY resolve, not every target found: + # a confirm-only target (one used attributively — see the resolver's + # ATTRIBUTIVE USE note) is exempt from the failure above, so + # reporting it as resolved would be a false claim. + resolved = [t for t in routing_targets if normalize_target(t) in known] + ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: " + f"{', '.join(resolved) if resolved else '(none)'}") + +# Body unfilled placeholders +fill_matches = PLACEHOLDER_RE.findall(body) +if fill_matches: + fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)") +else: + ok("SKILL.md body has no unfilled placeholders") + +# Interactive prompt heuristic. +# +# A line-initial `read` only blocks an agent when its stdin is the terminal. +# These forms never touch a TTY and are ordinary data plumbing, so flagging +# them is a false positive — one that has already cost two authors a +# contorted rewrite of working source: +# +# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string +# read -r X <: " X` is interactive and + # must still fail. + unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line) + return '<' in unquoted or prev_line.rstrip().endswith('|') + +# A here-doc body is DATA, not command position. Every script in this corpus +# carries a `usage() { cat < "references/a.md" + return re.sub(r'\s*\(.*$', '', entry).strip() + + # Inline form: value on the same line, comma-separated, no notes. + cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE) + if cf_m: + value = cf_m.group(1).strip() + if value.startswith("(none"): + return [] + return [p for p in (strip_note(x) for x in value.split(",")) + if p] or None + + # Bullet form: heading on its own line, one file per following bullet. + cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) + if not cf_m: + return None + files = [] + for line in block[cf_m.end():].splitlines(): + line = line.strip() + if not line: + if files: + break + continue + if not line.startswith("- "): + break + entry = line[2:].strip() + if entry.startswith("(none"): + return [] + entry = strip_note(entry) + if entry: + files.append(entry) + return files or None +# ===== END SHARED CONTRIBUTING-FILES PARSER ===== +KYBERFORGE_CONTRIBUTING_FILES +KYBERFORGE_CONTRIBUTING_FILES_PY="${KYBERFORGE_CONTRIBUTING_FILES_PY%$'\n'}" diff --git a/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh similarity index 73% rename from plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh rename to plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh index 1dcc003..f9649e3 100755 --- a/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh @@ -1,7 +1,32 @@ #!/usr/bin/env bash -set -euo pipefail +# lib-provenance-agent.sh — SOURCED, never executed. +# +# agent-audit's provenance suite: its validate-provenance.sh, minus the shared +# Contributing-files parser (lib-contributing-files.sh holds the one copy) and +# minus the --help dispatch that validate-provenance.sh now owns. The bash +# argument handling, the preconditions and every exit code are lifted verbatim, +# except the extension check, which the dispatcher made unreachable (see +# kyberforge_prov_agent_run). +# +# The two provenance modes have DIFFERENT exit contracts and they are NOT +# unified. Agent mode prints NOTHING on a clean run, and exits 0 silently when +# the scope walk-up finds no type:-bearing apm.yml above the agent file — that +# is a verdict about a real file ("this agent is user or project scope, so +# plugin-scope provenance does not apply"), not a rejected input, and +# scripts/check-scope-walkup-sync.sh's fixture 6 pins it. Skill mode +# (lib-provenance-skill.sh) has no such verdict and instead treats exit 0 with +# output as INFO-only findings. Neither contract may be spelled with the +# other's codes. +# +# Agent mode also has no check 9, so it takes no --base-ref flag: a --base-ref +# passed alongside an agent target is an extra argument and is rejected with +# exit 2, exactly as before the merge. +# +# Consumed by: validate-provenance.sh, agent mode. +# shellcheck shell=bash +# shellcheck disable=SC2034 -usage() { +kyberforge_prov_agent_usage() { cat < @@ -52,91 +77,98 @@ Checks performed: 4 Contributing files back-reference the parent slug in their source_keys 5 Research doc field present and not placeholder -This script has no counterpart to skill-audit's checks 6, 7 and 8 (Research +Agent mode has no counterpart to skill mode's checks 6, 7 and 8 (Research doc field / upstream forward / upstream reverse are numbered 6, 7, 8 there and 5 here): an agent at plugin scope is a single file with a plugin-root sources.md, so there is no references/ tree to walk and no upstream research source index to cross-check. parse_status() and the sources.md-basename gate -that those checks need exist only in the skill-audit copy. +that those checks need exist only in lib-provenance-skill.sh. EOF } -if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then - usage - exit 0 -fi - -# Usage and environment problems exit 2, findings exit 1. See the usage text -# above for why the two must not share a code, and for why "not plugin scope" -# is neither of them. This is a deliberate divergence from validate.sh, which -# has no 2 tier for content: validate.sh always prints PASS lines, so a usage -# error there is visibly not a findings report. This script prints NOTHING on a -# clean run, so exit 1 plus empty stdout was the only signal a caller got -# either way. -if [[ $# -lt 1 ]]; then - echo "Error: agent-file is required." >&2 - echo "" >&2 - usage >&2 - exit 2 -fi - -# Extra positional arguments were silently dropped, so a typo'd flag or a second -# path looked like it had been honoured. -if [[ $# -gt 1 ]]; then - echo "Error: expected exactly one argument, got $#: $*" >&2 - echo "" >&2 - usage >&2 - exit 2 -fi - -# python3 is a HARD dependency. Without this preflight a missing interpreter -# produced 'line NN: python3: command not found' and exit 127 — an exit code no -# caller maps to anything, from a message that names this script's line number -# rather than the missing dependency. -if ! command -v python3 > /dev/null 2>&1; then - echo "Error: python3 is required but was not found on PATH." >&2 - echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2 - echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 - exit 2 -fi - -# A path that does not exist, or exists but is not a regular file, used to reach -# the Python body, get os.path.dirname()'d into some ancestor directory and then -# either report a silent exit 0 (no package above it) or — worse — audit a -# DIFFERENT agent's package while naming the typo'd path. A typo'd target was -# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a -# nonexistent path for exactly this reason. -# -# This is decided from the argument alone, before any walk-up runs, so it cannot -# collide with the not-plugin-scope exit 0: that verdict is only ever reached by -# a file that got past here. -if [[ ! -e "$1" ]]; then - echo "Error: no such file: $1" >&2 - echo " Why: a nonexistent target would otherwise report a silent pass." >&2 - echo " Fix: pass the path of the agent file to validate." >&2 - exit 2 -fi - -if [[ ! -f "$1" ]]; then - echo "Error: not a regular file: $1" >&2 - echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2 - echo " Fix: pass the agent file itself — .apm/agents/.agent.md — not its parent directory." >&2 - exit 2 -fi - -# The extension check used to live inside the Python body. It stays exit 2 and -# keeps its wording; it moves up here so that every "this argument is not -# auditable" verdict is reached in one place, before the interpreter starts and -# before the scope walk-up can turn a bad argument into a silent exit 0. -case "$1" in - *.agent.md | *.md) ;; - *) - echo "Error: unrecognized extension '$(basename "$1")' — expected .md or .agent.md" >&2 +kyberforge_prov_agent_run() { + # Usage and environment problems exit 2, findings exit 1. See the usage text + # above for why the two must not share a code, and for why "not plugin scope" + # is neither of them. This is a deliberate divergence from validate.sh, which + # has no 2 tier for content: validate.sh always prints PASS lines, so a usage + # error there is visibly not a findings report. This script prints NOTHING on a + # clean run, so exit 1 plus empty stdout was the only signal a caller got + # either way. + if [[ $# -lt 1 ]]; then + echo "Error: agent-file is required." >&2 + echo "" >&2 + kyberforge_prov_agent_usage >&2 exit 2 - ;; -esac + fi -python3 -u - "$1" <<'PYTHON' + # Extra positional arguments were silently dropped, so a typo'd flag or a second + # path looked like it had been honoured. + if [[ $# -gt 1 ]]; then + echo "Error: expected exactly one argument, got $#: $*" >&2 + echo "" >&2 + kyberforge_prov_agent_usage >&2 + exit 2 + fi + + # python3 is a HARD dependency. Without this preflight a missing interpreter + # produced 'line NN: python3: command not found' and exit 127 — an exit code no + # caller maps to anything, from a message that names this script's line number + # rather than the missing dependency. + if ! command -v python3 > /dev/null 2>&1; then + echo "Error: python3 is required but was not found on PATH." >&2 + echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2 + echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 + exit 2 + fi + + # A path that does not exist, or exists but is not a regular file, used to reach + # the Python body, get os.path.dirname()'d into some ancestor directory and then + # either report a silent exit 0 (no package above it) or — worse — audit a + # DIFFERENT agent's package while naming the typo'd path. A typo'd target was + # indistinguishable from a clean agent. vale-wrap.sh hard-errors on a + # nonexistent path for exactly this reason. + # + # This is decided from the argument alone, before any walk-up runs, so it cannot + # collide with the not-plugin-scope exit 0: that verdict is only ever reached by + # a file that got past here. + if [[ ! -e "$1" ]]; then + echo "Error: no such file: $1" >&2 + echo " Why: a nonexistent target would otherwise report a silent pass." >&2 + echo " Fix: pass the path of the agent file to validate." >&2 + exit 2 + fi + + if [[ ! -f "$1" ]]; then + echo "Error: not a regular file: $1" >&2 + echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2 + echo " Fix: pass the agent file itself — .apm/agents/.agent.md — not its parent directory." >&2 + exit 2 + fi + + # No extension check here. The pre-merge script carried one ("unrecognized + # extension — expected .md or .agent.md"), but validate-provenance.sh only + # dispatches a *.agent.md, or a *.md directly under an agents/ directory, to + # this function, and the argument-count check above guarantees $1 IS that + # target — so the check could never fire. The "no such file" and "not a + # regular file" checks stay: a nonexistent x.agent.md and a FIFO named + # x.agent.md both pass the dispatcher and both still reach them. + + # The Python program, reassembled in the order the parser block sat in before + # the merge: preamble, shared parser, body. + local prog="$KYBERFORGE_PROV_AGENT_PREAMBLE_PY +$KYBERFORGE_CONTRIBUTING_FILES_PY +$KYBERFORGE_PROV_AGENT_BODY_PY" + + local rc=0 + python3 -u - "$1" <<< "$prog" || rc=$? + # The findings code travels in KYBERFORGE_PROV_RC and this function returns 0, + # so the caller can invoke it UNTESTED. See lib-provenance-skill.sh for why: + # testing a function's status disables errexit for its whole body. + KYBERFORGE_PROV_RC="$rc" + return 0 +} + +IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_PREAMBLE_PY <<'KYBERFORGE_PROV_AGENT_PREAMBLE' || true import sys import os import re @@ -291,99 +323,10 @@ def parse_source_keys(fm): def parse_h2_slugs(content): return re.findall(r'^## (.+)$', content, re.MULTILINE) -# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ===== -# ONE parser, embedded VERBATIM in two scripts: -# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh -# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh -# The block between these markers must stay byte-identical in both. It is -# copied rather than imported because a cache-installed plugin's scripts cannot -# read files outside their own plugin directory, so there is no single file both -# can share — the same constraint that forces the ADR-0020 boundary resolver to -# be duplicated across three scripts. Edit one copy, then paste it over the -# other. -# -# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before -# it did, the agent-audit copy's docstring merely ASSERTED the two were -# "behaviourally identical" and nothing checked it — which is how the two -# already-diverged spellings of the bullet loop went unnoticed. -# -# Requires: re (imported by the host script). +KYBERFORGE_PROV_AGENT_PREAMBLE +KYBERFORGE_PROV_AGENT_PREAMBLE_PY="${KYBERFORGE_PROV_AGENT_PREAMBLE_PY%$'\n'}" - -def parse_contributing_files(content, slug): - """Find the Contributing files for a given slug H2 in content. - - Both authored forms are accepted, because both are in use across the - corpus and only recognising the first silently skipped the contributing- - file checks on every sources.md written the other way: - - - **Contributing files:** SKILL.md, references/a.md - - **Contributing files:** - - SKILL.md (what this source contributed) - - references/a.md (what this source contributed) - - Returns a list of paths with any trailing parenthetical note stripped. - Note the bullet form's notes may themselves contain commas, so the list - is built per bullet rather than by splitting the joined value. - - The three return values are NOT interchangeable, and callers depend on - the distinction: - - [path, ...] the entry names contributing files - [] the entry EXPLICITLY records "(none)" - None the entry says nothing this parser can read - - Only an explicit "(none)" yields []. A "Contributing files:" heading - followed by a numbered list, by `*` bullets, or by prose parses nothing - and returns None, never [] — a caller reads [] as a deliberate "no - contributing files" record and SKIPS its check on that basis, so a parse - failure returning [] would silently disable the check instead of leaving - the unreadable entry exposed to it. - """ - pattern = re.compile( - r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', - re.MULTILINE | re.DOTALL - ) - m = pattern.search(content) - if not m: - return None - block = m.group(1) - - def strip_note(entry): - # "references/a.md (why)" -> "references/a.md" - return re.sub(r'\s*\(.*$', '', entry).strip() - - # Inline form: value on the same line, comma-separated, no notes. - cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE) - if cf_m: - value = cf_m.group(1).strip() - if value.startswith("(none"): - return [] - return [p for p in (strip_note(x) for x in value.split(",")) - if p] or None - - # Bullet form: heading on its own line, one file per following bullet. - cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) - if not cf_m: - return None - files = [] - for line in block[cf_m.end():].splitlines(): - line = line.strip() - if not line: - if files: - break - continue - if not line.startswith("- "): - break - entry = line[2:].strip() - if entry.startswith("(none"): - return [] - entry = strip_note(entry) - if entry: - files.append(entry) - return files or None -# ===== END SHARED CONTRIBUTING-FILES PARSER ===== +IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_BODY_PY <<'KYBERFORGE_PROV_AGENT_BODY' || true def parse_research_docs(content, slug): """Every Research doc value under a given slug H2, in document order. @@ -629,4 +572,5 @@ for slug in unique_slugs: print_findings() sys.exit(1 if has_fail else 0) -PYTHON +KYBERFORGE_PROV_AGENT_BODY +KYBERFORGE_PROV_AGENT_BODY_PY="${KYBERFORGE_PROV_AGENT_BODY_PY%$'\n'}" diff --git a/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh similarity index 86% rename from plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh rename to plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh index 4b720b4..f9fc8a1 100755 --- a/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh @@ -1,7 +1,30 @@ #!/usr/bin/env bash -set -euo pipefail +# lib-provenance-skill.sh — SOURCED, never executed. +# +# skill-audit's provenance suite: its validate-provenance.sh, minus the shared +# Contributing-files parser (lib-contributing-files.sh holds the one copy) and +# minus the --help dispatch that validate-provenance.sh now owns. The bash +# argument handling, the preconditions and every exit code are lifted verbatim. +# +# The two provenance modes have DIFFERENT exit contracts and they are NOT +# unified. Skill mode exits 0 with output whenever the only findings are INFO +# — a check that could not run, announced rather than skipped silently — so a +# caller must read exit 0 plus output as INFO-only findings, not as noise. +# Agent mode (lib-provenance-agent.sh) prints nothing at all on a clean run and +# additionally exits 0 SILENTLY when the scope walk-up finds no plugin package. +# Skill mode has no such verdict: it has already hard-failed (exit 2) on a +# directory that is not a skill before the interpreter starts. +# +# Skill mode also owns the --base-ref= flag (check 9) and the +# VALIDATE_PROVENANCE_BASE_REF environment variable. Agent mode has no check 9 +# and takes no flags, so a --base-ref passed to an agent target is still an +# extra argument and is still rejected, exactly as before the merge. +# +# Consumed by: validate-provenance.sh, skill mode. +# shellcheck shell=bash +# shellcheck disable=SC2034 -usage() { +kyberforge_prov_skill_usage() { cat < [--base-ref=] @@ -64,102 +87,121 @@ Checks performed: EOF } -if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then - usage - exit 0 -fi +kyberforge_prov_skill_run() { + local arg + # --base-ref= is the only recognised flag, for check 9's base-ref + # override. It is pulled out before the positional-count checks below so it + # never counts against them — a caller passing it alongside skill-dir sees + # the same argument-count behaviour as one who does not pass it at all, and a + # genuinely extra positional argument is still rejected. + # + # BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to + # the empty string, and deliberately NOT declared `local`: `local X` with no + # value still leaves X unset, but declaring it at all would scope it away from + # a future caller that wants to inspect it. `--base-ref=` (given, but empty) + # and "no flag at all" are different instructions — the first says "use the + # default resolution, ignoring the environment", the second says "fall back to + # the environment" — and an empty-string initialiser collapsed them: + # `${BASE_REF_OVERRIDE:-$ENV}` treats an empty flag value as absent, so the + # environment variable won and the usage text's "the flag wins if both are + # given" was false for exactly that spelling. + declare -a POSITIONAL_ARGS=() + for arg in "$@"; do + case "$arg" in + --base-ref=*) + BASE_REF_OVERRIDE="${arg#--base-ref=}" + ;; + *) + POSITIONAL_ARGS+=("$arg") + ;; + esac + done -# --base-ref= is the only recognised flag, for check 9's base-ref -# override. It is pulled out before the positional-count checks below so it -# never counts against them — a caller passing it alongside skill-dir sees -# the same argument-count behaviour as one who does not pass it at all, and a -# genuinely extra positional argument is still rejected. -# -# BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to -# the empty string. `--base-ref=` (given, but empty) and "no flag at all" are -# different instructions — the first says "use the default resolution, ignoring -# the environment", the second says "fall back to the environment" — and an -# empty-string initialiser collapsed them: `${BASE_REF_OVERRIDE:-$ENV}` treats -# an empty flag value as absent, so the environment variable won and the usage -# text's "the flag wins if both are given" was false for exactly that spelling. -declare -a POSITIONAL_ARGS=() -for arg in "$@"; do - case "$arg" in - --base-ref=*) - BASE_REF_OVERRIDE="${arg#--base-ref=}" - ;; - *) - POSITIONAL_ARGS+=("$arg") - ;; - esac -done + # Usage and environment problems exit 2, findings exit 1. See the usage text + # above for why the two must not share a code. This is a deliberate divergence + # from validate.sh, which has no 2 tier: validate.sh always prints PASS lines, + # so a usage error there is visibly not a findings report. This script prints + # NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a + # caller got either way. + if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then + echo "Error: skill-dir is required." >&2 + echo "" >&2 + kyberforge_prov_skill_usage >&2 + exit 2 + fi -# Usage and environment problems exit 2, findings exit 1. See the usage text -# above for why the two must not share a code. This is a deliberate divergence -# from validate.sh, which has no 2 tier: validate.sh always prints PASS lines, -# so a usage error there is visibly not a findings report. This script prints -# NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a -# caller got either way. -if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then - echo "Error: skill-dir is required." >&2 - echo "" >&2 - usage >&2 - exit 2 -fi + # Extra positional arguments were silently dropped, so a typo'd flag or a second + # path looked like it had been honoured. + if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then + echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2 + echo "" >&2 + kyberforge_prov_skill_usage >&2 + exit 2 + fi -# Extra positional arguments were silently dropped, so a typo'd flag or a second -# path looked like it had been honoured. -if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then - echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2 - echo "" >&2 - usage >&2 - exit 2 -fi + local SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}" -SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}" + # The flag wins over the environment variable whenever the flag was GIVEN — + # `+x` tests for presence, not for a non-empty value, which is the distinction + # `:-` could not make. An empty result either way tells the Python body to fall + # back to `git merge-base HEAD origin/main`. + local BASE_REF + if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then + BASE_REF="$BASE_REF_OVERRIDE" + else + BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}" + fi -# The flag wins over the environment variable whenever the flag was GIVEN — -# `+x` tests for presence, not for a non-empty value, which is the distinction -# `:-` could not make. An empty result either way tells the Python body to fall -# back to `git merge-base HEAD origin/main`. -if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then - BASE_REF="$BASE_REF_OVERRIDE" -else - BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}" -fi + # python3 is a HARD dependency. Without this preflight a missing interpreter + # produced 'line NN: python3: command not found' and exit 127 — an exit code no + # caller maps to anything, from a message that names this script's line number + # rather than the missing dependency. + if ! command -v python3 > /dev/null 2>&1; then + echo "Error: python3 is required but was not found on PATH." >&2 + echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2 + echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 + exit 2 + fi -# python3 is a HARD dependency. Without this preflight a missing interpreter -# produced 'line NN: python3: command not found' and exit 127 — an exit code no -# caller maps to anything, from a message that names this script's line number -# rather than the missing dependency. -if ! command -v python3 > /dev/null 2>&1; then - echo "Error: python3 is required but was not found on PATH." >&2 - echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2 - echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2 - exit 2 -fi + # A path that is not a directory, or a directory that is not a skill, used to + # reach the Python body, find no sources.md and no source_keys, take the + # "nothing to validate" early exit and report exit 0 with no output — which + # references/skill-validation-scripts.md explicitly told the auditor to read as a + # pass. A typo'd target was therefore indistinguishable from a clean skill. + # vale-wrap.sh hard-errors on a nonexistent path for exactly this reason. + if [[ ! -d "$SKILL_DIR_ARG" ]]; then + echo "Error: not a directory: $SKILL_DIR_ARG" >&2 + echo " Why: a nonexistent target would otherwise report a silent pass." >&2 + echo " Fix: pass the path of the skill directory to validate." >&2 + exit 2 + fi -# A path that is not a directory, or a directory that is not a skill, used to -# reach the Python body, find no sources.md and no source_keys, take the -# "nothing to validate" early exit and report exit 0 with no output — which -# references/validation-scripts.md explicitly told the auditor to read as a -# pass. A typo'd target was therefore indistinguishable from a clean skill. -# vale-wrap.sh hard-errors on a nonexistent path for exactly this reason. -if [[ ! -d "$SKILL_DIR_ARG" ]]; then - echo "Error: not a directory: $SKILL_DIR_ARG" >&2 - echo " Why: a nonexistent target would otherwise report a silent pass." >&2 - echo " Fix: pass the path of the skill directory to validate." >&2 - exit 2 -fi + if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then + echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2 + echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2 + echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2 + exit 2 + fi -if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then - echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2 - echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2 - echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2 - exit 2 -fi + # The Python program, reassembled in the order the parser block sat in before + # the merge: preamble, shared parser, body. + local prog="$KYBERFORGE_PROV_SKILL_PREAMBLE_PY +$KYBERFORGE_CONTRIBUTING_FILES_PY +$KYBERFORGE_PROV_SKILL_BODY_PY" -python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<'PYTHON' + local rc=0 + python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<< "$prog" || rc=$? + # The findings code travels in KYBERFORGE_PROV_RC and this function returns 0, + # so the caller can invoke it UNTESTED. Testing a function's status (`f || RC=$?`) + # disables errexit for its entire body, which would leave every command above + # unguarded -- and no subshell or `set -e` inside can re-arm it once the call + # sits in a condition context. Error paths above use `exit`, which is unaffected + # either way; this keeps errexit armed for anything added later. + KYBERFORGE_PROV_RC="$rc" + return 0 +} + +IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_PREAMBLE_PY <<'KYBERFORGE_PROV_SKILL_PREAMBLE' || true import sys import os import re @@ -297,99 +339,10 @@ def parse_h2_slugs(content): """Return list of H2 heading values from a markdown file.""" return re.findall(r'^## (.+)$', content, re.MULTILINE) -# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER ===== -# ONE parser, embedded VERBATIM in two scripts: -# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh -# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh -# The block between these markers must stay byte-identical in both. It is -# copied rather than imported because a cache-installed plugin's scripts cannot -# read files outside their own plugin directory, so there is no single file both -# can share — the same constraint that forces the ADR-0020 boundary resolver to -# be duplicated across three scripts. Edit one copy, then paste it over the -# other. -# -# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before -# it did, the agent-audit copy's docstring merely ASSERTED the two were -# "behaviourally identical" and nothing checked it — which is how the two -# already-diverged spellings of the bullet loop went unnoticed. -# -# Requires: re (imported by the host script). +KYBERFORGE_PROV_SKILL_PREAMBLE +KYBERFORGE_PROV_SKILL_PREAMBLE_PY="${KYBERFORGE_PROV_SKILL_PREAMBLE_PY%$'\n'}" - -def parse_contributing_files(content, slug): - """Find the Contributing files for a given slug H2 in content. - - Both authored forms are accepted, because both are in use across the - corpus and only recognising the first silently skipped the contributing- - file checks on every sources.md written the other way: - - - **Contributing files:** SKILL.md, references/a.md - - **Contributing files:** - - SKILL.md (what this source contributed) - - references/a.md (what this source contributed) - - Returns a list of paths with any trailing parenthetical note stripped. - Note the bullet form's notes may themselves contain commas, so the list - is built per bullet rather than by splitting the joined value. - - The three return values are NOT interchangeable, and callers depend on - the distinction: - - [path, ...] the entry names contributing files - [] the entry EXPLICITLY records "(none)" - None the entry says nothing this parser can read - - Only an explicit "(none)" yields []. A "Contributing files:" heading - followed by a numbered list, by `*` bullets, or by prose parses nothing - and returns None, never [] — a caller reads [] as a deliberate "no - contributing files" record and SKIPS its check on that basis, so a parse - failure returning [] would silently disable the check instead of leaving - the unreadable entry exposed to it. - """ - pattern = re.compile( - r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)', - re.MULTILINE | re.DOTALL - ) - m = pattern.search(content) - if not m: - return None - block = m.group(1) - - def strip_note(entry): - # "references/a.md (why)" -> "references/a.md" - return re.sub(r'\s*\(.*$', '', entry).strip() - - # Inline form: value on the same line, comma-separated, no notes. - cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE) - if cf_m: - value = cf_m.group(1).strip() - if value.startswith("(none"): - return [] - return [p for p in (strip_note(x) for x in value.split(",")) - if p] or None - - # Bullet form: heading on its own line, one file per following bullet. - cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE) - if not cf_m: - return None - files = [] - for line in block[cf_m.end():].splitlines(): - line = line.strip() - if not line: - if files: - break - continue - if not line.startswith("- "): - break - entry = line[2:].strip() - if entry.startswith("(none"): - return [] - entry = strip_note(entry) - if entry: - files.append(entry) - return files or None -# ===== END SHARED CONTRIBUTING-FILES PARSER ===== +IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_BODY_PY <<'KYBERFORGE_PROV_SKILL_BODY' || true def parse_research_docs(content, slug): """Every Research doc value under a given slug H2, in document order. @@ -1195,4 +1148,5 @@ else: print_findings() sys.exit(1 if has_fail else 0) -PYTHON +KYBERFORGE_PROV_SKILL_BODY +KYBERFORGE_PROV_SKILL_BODY_PY="${KYBERFORGE_PROV_SKILL_BODY_PY%$'\n'}" diff --git a/plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh similarity index 96% rename from plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh rename to plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh index 862f44c..dde287c 100755 --- a/plugins/kyberforge/.apm/skills/agent-audit/scripts/vale-wrap.sh +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh @@ -189,7 +189,16 @@ for arg in "$@"; do done if [[ "$config_given" == false ]]; then - vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini") + # `scripts/../assets/vale` begins with neither `/` nor `.`, so `cd` consults + # CDPATH for it — and when a CDPATH entry supplies the directory, `cd` PRINTS + # the directory it chose. A bare `$(cd ... && pwd)` therefore captured TWO + # lines, and the chosen directory could be an unrelated tree entirely: with + # CDPATH=/tmp/decoy and /tmp/decoy/scripts present, this resolved to + # /tmp/decoy/assets/vale and vale died on a two-line --config path. CDPATH is + # cleared for the one command, `--` ends option parsing for a directory named + # like a flag, and stdout is discarded so only `pwd` is captured. Same fix as + # validate.sh and validate-provenance.sh apply to their SCRIPT_DIR. + vale_args+=(--config "$(CDPATH='' cd -- "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" > /dev/null && pwd)/.vale.ini") fi if [[ ${#path_args[@]} -eq 0 ]]; then @@ -506,7 +515,7 @@ for arg in ${path_args[@]+"${path_args[@]}"}; do while IFS= read -r -d '' rel; do mkdir -p "$dest/$(dirname "$rel")" cp "$arg/$rel" "$dest/$rel" - done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0) + done < <(CDPATH='' cd -- "$arg" && find -L . -name .git -prune -o -type f -print0) while IFS= read -r -d '' md; do flatten "$md" "$md" done < <(find "$dest" -type f -name '*.md' -print0) diff --git a/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh new file mode 100755 index 0000000..d926ce0 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh @@ -0,0 +1,324 @@ +#!/usr/bin/env bash +set -euo pipefail + +# The ONE entry point for provenance validation. It auto-detects whether the +# target is a skill directory or an agent definition file — the same rule +# validate.sh uses — and runs the matching suite from lib-provenance-skill.sh or +# lib-provenance-agent.sh. The Contributing-files parser both suites need is +# sourced once, from lib-contributing-files.sh, instead of being embedded twice. +# +# The two suites have DIFFERENT exit contracts, and merging the entry point does +# not merge those: +# +# skill mode exits 0 with output when the only findings are INFO — a check +# that could not run, announced rather than skipped silently. A +# caller must read exit 0 plus output as INFO-only findings. +# agent mode prints nothing at all on a clean run, and exits 0 SILENTLY when +# the scope walk-up finds no plugin package above the agent file. +# That is a verdict about a real file, not a rejected input; +# scripts/check-scope-walkup-sync.sh's fixture 6 pins it. +# +# Exit 2 means the argument is not auditable at all — missing, doubled, the +# wrong shape, or an environment problem. It is never a finding. + +# --- Path splitting, with bash builtins only ------------------------------- +# dirname and basename are EXTERNAL commands, and every call below happens +# before the mode's python3 preflight. Using them put coreutils ahead of python3 +# in the dependency order: on a PATH carrying neither, this script died at exit +# 127 naming `dirname` (and, through the sourced libraries, `cat`) instead of +# reaching the preflight that names python3 — the exact failure +# tests/test-adr0020-contract.sh assertion 2 exists to prevent. The pre-merge +# validate-provenance.sh was one self-contained file that reached its preflight +# on builtins alone; these two functions, plus the `read`-based loaders in the +# sourced libraries, restore that property. `cd` and `pwd` are builtins and may +# stay. +# +# They reproduce dirname/basename semantics for the shapes this script sees: +# trailing slashes are stripped, a path with no slash yields "." / itself, and +# "/" yields "/". +_kf_dirname() { + local _p="$1" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ "$_p" == "/" ]]; then + printf '%s' "/" + return 0 + fi + if [[ "$_p" != */* ]]; then + printf '%s' "." + return 0 + fi + _p="${_p%/*}" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ -z "$_p" ]]; then + _p="/" + fi + printf '%s' "$_p" +} + +_kf_basename() { + local _p="$1" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ "$_p" == "/" ]]; then + printf '%s' "/" + return 0 + fi + printf '%s' "${_p##*/}" +} + +# --- The target's parent directory NAME, resolved ------------------------- +# The agent rule tests the NAME of the target's parent directory. Reading that +# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` — +# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/ +# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/ +# directory was refused as matching neither shape, by an error message naming +# that exact shape as valid. The pre-merge agent validator had no path-shape +# gate and worked from any working directory. +# +# So the parent is resolved with the `cd` and `pwd` builtins in a subshell — +# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not +# `pwd -P`): an agents/ directory reached through a symlink named agents/ is +# still addressed as agents/, which is what the literal test always honoured. +# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent +# that cannot be entered — a typo'd path — falls back to the literal name, so +# the neither-shape error still fires for it. +_kf_parent_name() { + local _dir _resolved + _dir="$(_kf_dirname "$1")" + if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then + _kf_basename "$_resolved" + else + _kf_basename "$_dir" + fi +} + +# --- This script's own directory, and the libraries beside it ------------- +# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with +# CDPATH exported and the relative invocation the flow references prescribe +# (`bash scripts/.sh`), a bare `$(cd ... && pwd)` captured two lines — +# and could resolve through CDPATH to an unrelated directory and source a +# same-named file from there. CDPATH is cleared for the one command, `--` ends +# option parsing for a directory named like a flag, and stdout is discarded so +# only `pwd` is captured. +if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then + echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2 + echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2 + echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2 + exit 2 +fi + +# A sourced library that is missing or unreadable used to kill the script under +# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the +# flow references tell the auditor to surface verbatim as REAL FINDINGS. A +# partial install, or a copy or symlink of this one file taken out of scripts/, +# was therefore reported as a failing audit. Checked explicitly instead, and +# exit 2, which the same references read as "it never ran". +_kf_require_lib() { + if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then + echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2 + echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2 + echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2 + exit 2 + fi +} + +# Each mode's own usage text lives in that mode's library, verbatim, so usage() +# needs the libraries — but `--help` must not. Sourcing them unconditionally at +# the top made a missing lib-*.sh turn `--help` into exit 2, so the one command +# that explains how to use the script was the one command a partial install +# could not answer. validate.sh's usage() is self-contained and always works; +# this restores the same property without copying the per-mode text down here +# and letting it drift from the libraries that own it. When a library is gone, +# the shared half of the usage still prints and the mode's half says why it +# cannot. +# +# The two call sites below are spelled out rather than folded into one helper +# taking the library as a parameter: a parameterized `.` is a non-constant +# source, which is SC1090 at warning severity — the level .pre-commit-config.yaml +# runs shellcheck at — and the only way to silence it, a `source=/dev/null` +# directive, is a directive that resolves to nothing, which +# tests/test-vale-wrap.sh part C rejects outright because a non-resolving +# directive silently disarms that file's array-seeding exemption. Two literal +# sources with two real directives cost a few lines and keep both gates honest. +_kf_lib_readable() { + [[ -f "$SCRIPT_DIR/$1" && -r "$SCRIPT_DIR/$1" ]] +} + +_kf_usage_lib_missing() { + echo "(This mode's usage lives in $1, which is missing or unreadable in" + echo "$SCRIPT_DIR. Reinstall the factory-audit skill to restore it. Note that" + echo "an audit cannot run in this state either — it would exit 2.)" +} + +usage() { + cat < [--base-ref=] + validate-provenance.sh + +Validate that a skill's or an agent's sources provenance chain is complete and +internally consistent. The mode is detected from the target: + + skill mode the target is a directory (a skill directory contains SKILL.md), + or the target IS a SKILL.md file. + agent mode the target is a *.agent.md file, or a *.md file whose parent + directory is named 'agents' (.apm/agents, .claude/agents, + .github/agents, .copilot/agents). + +The two modes have different checks, different exit contracts and different +flags — --base-ref belongs to skill mode's check 9 and agent mode has no +check 9 — so each mode's own usage follows below, verbatim. + +Exit codes: + 0 All checks passed (or nothing to validate; in agent mode, also "not plugin + scope") + 1 One or more checks failed + 2 Usage error, the target matches neither a skill directory nor an agent + file this script can read, or a lib-*.sh beside this script is missing or + unreadable + +An exit code of 2 is NOT a finding. SKILL.md tells the auditor to surface a +non-zero exit as findings, so a usage error leaving exit 1 with nothing on +stdout was indistinguishable from a clean-but-failing run. Environment and +argument problems exit 2; only real findings exit 1. + +=== skill mode === +EOF + if _kf_lib_readable lib-provenance-skill.sh; then + # shellcheck source=lib-provenance-skill.sh + . "$SCRIPT_DIR/lib-provenance-skill.sh" + kyberforge_prov_skill_usage + else + _kf_usage_lib_missing lib-provenance-skill.sh + fi + cat <&2 + echo "" >&2 + usage >&2 + exit 2 +fi + +# Sourced only once an audit is actually going to be attempted. It sat at the +# top of the file until `--help` on a partial install exited 2 instead of +# printing usage; see the comment above _kf_lib_readable for the whole story. +# usage() loads the two provenance libraries on its own when it needs them, so +# nothing here is reached by the --help path. +_kf_require_lib lib-contributing-files.sh +# shellcheck source=lib-contributing-files.sh +. "$SCRIPT_DIR/lib-contributing-files.sh" +_kf_require_lib lib-provenance-skill.sh +# shellcheck source=lib-provenance-skill.sh +. "$SCRIPT_DIR/lib-provenance-skill.sh" +_kf_require_lib lib-provenance-agent.sh +# shellcheck source=lib-provenance-agent.sh +. "$SCRIPT_DIR/lib-provenance-agent.sh" + +# --- Detect the mode ------------------------------------------------------- +# The first non-flag argument decides the mode. Only the mode is decided here: +# the argument COUNT, the flag rules and every precondition belong to the mode's +# own suite and are applied there, unchanged, over the original "$@". So a +# --base-ref handed to an agent target is still an extra argument and is still +# rejected, and a second positional is still rejected by whichever mode it +# reaches. +# _saw_positional is tracked separately because an EMPTY positional and NO +# positional are different mistakes with different fixes, and `-z "$TARGET"` +# alone cannot tell them apart: `validate-provenance.sh ""` — an unquoted shell +# variable that expanded to nothing, the usual way this happens — was reported +# as "only flags were given", which is false and sends the reader looking for a +# flag they did not type instead of at the variable that came up empty. +TARGET="" +_saw_positional=false +for _arg in "$@"; do + case "$_arg" in + --base-ref=*) ;; + *) TARGET="$_arg"; _saw_positional=true; break ;; + esac +done + +if [[ "$_saw_positional" == false ]]; then + echo "Error: a skill directory or an agent file is required." >&2 + echo " Why: only flags were given, so there is no target to detect a mode from." >&2 + echo " Fix: pass the skill directory, or the agent file, as a positional argument." >&2 + exit 2 +fi + +if [[ -z "$TARGET" ]]; then + echo "Error: the target argument is an empty string." >&2 + echo " Why: a positional argument was passed, but it is empty, so there is no path to detect a mode from — usually an unquoted or unset shell variable expanding to nothing at the call site, not a missing argument." >&2 + echo " Fix: check the variable that supplies the target, and pass the skill directory, or the agent file, as a non-empty positional argument." >&2 + exit 2 +fi + +TARGET_BASE="$(_kf_basename "$TARGET")" +TARGET_PARENT="$(_kf_parent_name "$TARGET")" + +if [[ -d "$TARGET" ]]; then + if [[ -f "$TARGET/SKILL.md" ]]; then + MODE=skill + else + echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2 + echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would run the wrong provenance checks." >&2 + echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (.agent.md, or a .md file under an agents/ directory)." >&2 + exit 2 + fi +elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then + MODE=skill +elif [[ "$TARGET_BASE" == *.agent.md ]]; then + MODE=agent +elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then + MODE=agent +else + echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2 + echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a .agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would report a silent pass on a typo'd target, which is the failure both suites' preconditions exist to prevent." >&2 + echo " Fix: pass one of those two shapes." >&2 + exit 2 +fi + +# In skill mode a SKILL.md target names its directory. The token is replaced in +# place rather than assumed to be $1, because --base-ref may precede it; the +# suite's own preconditions then apply to that directory, exactly as before the +# merge. +if [[ "$MODE" == skill && "$TARGET_BASE" == "SKILL.md" && ! -d "$TARGET" ]]; then + declare -a _rewritten=() + _replaced=false + for _arg in "$@"; do + if [[ "$_replaced" == false && "$_arg" == "$TARGET" ]]; then + _rewritten+=("$(_kf_dirname "$TARGET")") + _replaced=true + else + _rewritten+=("$_arg") + fi + done + # Guarded expansion: bash 3.2 under `set -u` aborts on "${arr[@]}" when the + # array is empty, and the loop above cannot prove non-emptiness to a static + # scan. tests/test-vale-wrap.sh enforces bash-3.2 portability across this tree. + set -- ${_rewritten[@]+"${_rewritten[@]}"} +fi + +# Called UNTESTED, on purpose: `f || RC=$?` would disable errexit for the whole +# function body. Each run function stashes its findings code in +# KYBERFORGE_PROV_RC and returns 0; its error paths exit directly. +KYBERFORGE_PROV_RC=0 +case "$MODE" in + skill) kyberforge_prov_skill_run "$@" ;; + agent) kyberforge_prov_agent_run "$@" ;; +esac +RC="$KYBERFORGE_PROV_RC" + +exit "$RC" diff --git a/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh b/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh new file mode 100755 index 0000000..43bb8b4 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh @@ -0,0 +1,255 @@ +#!/usr/bin/env bash +set -euo pipefail + +# The ONE entry point for structural validation. It auto-detects whether the +# target is a skill directory or an agent definition file and runs the matching +# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh +# and are unchanged from the skill-audit / agent-audit scripts they came from. +# The ADR-0020 boundary resolver both of them need is sourced once, from +# lib-boundary-resolver.sh, instead of being embedded twice. +# +# Detection never guesses. A target that matches neither shape is a hard exit 2 +# naming the mismatch, because the alternative — picking a mode and letting the +# suite fail on its own terms — reports a skill-shaped finding about an agent +# file, or the reverse, and sends the reader after the wrong problem. + +# --- Path splitting, with bash builtins only ------------------------------- +# dirname and basename are EXTERNAL commands, and every call below happens +# before the mode-specific python3/PyYAML preflight. Using them put coreutils +# ahead of python3 in the dependency order: on a PATH carrying neither, this +# script died at exit 127 naming `dirname` instead of reaching the preflight +# that names python3 — the exact failure tests/test-adr0020-contract.sh +# assertion 2 exists to prevent ("the two are checked separately so the message +# names the thing to install rather than the wrong one"). The pre-merge +# validate.sh was one self-contained file that reached its preflight on builtins +# alone; these two functions restore that property. `cd` and `pwd` are builtins +# and may stay. +# +# They reproduce dirname/basename semantics for the shapes this script sees: +# trailing slashes are stripped, a path with no slash yields "." / itself, and +# "/" yields "/". +_kf_dirname() { + local _p="$1" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ "$_p" == "/" ]]; then + printf '%s' "/" + return 0 + fi + if [[ "$_p" != */* ]]; then + printf '%s' "." + return 0 + fi + _p="${_p%/*}" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ -z "$_p" ]]; then + _p="/" + fi + printf '%s' "$_p" +} + +_kf_basename() { + local _p="$1" + while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done + if [[ "$_p" == "/" ]]; then + printf '%s' "/" + return 0 + fi + printf '%s' "${_p##*/}" +} + +# --- The target's parent directory NAME, resolved ------------------------- +# The agent rule tests the NAME of the target's parent directory. Reading that +# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` — +# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/ +# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/ +# directory was refused as matching neither shape, by an error message naming +# that exact shape as valid. The pre-merge agent validator had no path-shape +# gate and worked from any working directory. +# +# So the parent is resolved with the `cd` and `pwd` builtins in a subshell — +# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not +# `pwd -P`): an agents/ directory reached through a symlink named agents/ is +# still addressed as agents/, which is what the literal test always honoured. +# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent +# that cannot be entered — a typo'd path — falls back to the literal name, so +# the neither-shape error still fires for it. +_kf_parent_name() { + local _dir _resolved + _dir="$(_kf_dirname "$1")" + if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then + _kf_basename "$_resolved" + else + _kf_basename "$_dir" + fi +} + +# --- This script's own directory, and the libraries beside it ------------- +# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with +# CDPATH exported and the relative invocation the flow references prescribe +# (`bash scripts/.sh`), a bare `$(cd ... && pwd)` captured two lines — +# and could resolve through CDPATH to an unrelated directory and source a +# same-named file from there. CDPATH is cleared for the one command, `--` ends +# option parsing for a directory named like a flag, and stdout is discarded so +# only `pwd` is captured. +if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then + echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2 + echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2 + echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2 + exit 2 +fi + +# A sourced library that is missing or unreadable used to kill the script under +# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the +# flow references tell the auditor to surface verbatim as REAL FINDINGS. A +# partial install, or a copy or symlink of this one file taken out of scripts/, +# was therefore reported as a failing audit. Checked explicitly instead, and +# exit 2, which the same references read as "it never ran". +_kf_require_lib() { + if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then + echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2 + echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2 + echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2 + exit 2 + fi +} + +usage() { + cat < + +Validate a skill directory against the agentskills.io specification, or an agent +definition file against the agent definition spec. The mode is detected from the +target: + + skill mode the target is a directory (a skill directory contains SKILL.md), + or the target IS a SKILL.md file. + agent mode the target is a *.agent.md file, or a *.md file whose parent + directory is named 'agents' (.apm/agents, .claude/agents, + .github/agents, .copilot/agents). + +Skill mode audits the directory named by . + +Agent mode: at plugin/APM scope, is a single vendor-neutral +.apm/agents/.agent.md file with no counterpart. Its frontmatter allowlist +is not restated here: it is read at load time from the apm-agent-allowlist +section of references/agent-field-inventory.md, which is the authoritative list. +At project or user scope, is either half of a Claude Code .md / +Copilot .agent.md pair. + +Arguments: + skill-dir Path to the skill directory containing SKILL.md. + agent-file Path to the agent file (or either half of a project/user-scope pair). + +Exit codes: + 0 All checks passed (may include SUGGESTIONs) + 1 One or more checks failed + 2 Nothing was audited (no argument, the target matches neither shape, the + target does not exist, an unrecognized file extension, a missing + references/agent-field-inventory.md, or a missing or unreadable lib-*.sh + beside this script) +EOF +} + +if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then + usage + exit 0 +fi + +if [[ $# -lt 1 ]]; then + echo "Error: a skill directory or an agent file is required." >&2 + echo "" >&2 + usage >&2 + exit 2 +fi + +TARGET="$1" + +# --- The target has to be there -------------------------------------------- +# Only the directory branch below stats the target; the *.agent.md, the +# agents/-parent and the SKILL.md branches classify on NAME alone, so a typo'd +# path matching one of those shapes was handed to python3 and came back as a +# FAIL at exit 1 — the findings tier, for a target that was never there to have +# findings about. The tiers are: 2 nothing is at this path so no check ran, 1 +# something is there and it is broken. +# +# This runs on the TYPED path, before the SKILL.md -> parent-directory rewrite +# further down: rewritten first, a missing `docs/SKILL.md` would be tested as +# `docs`, which exists, and the guard would miss it. +# +# `-L` deliberately rescues what `-e` rejects. A dangling symlink and a symlink +# loop are both FALSE to -e but TRUE to -L, and neither belongs here: something +# IS at that path, it just cannot be opened, and "exists but unreadable" is a +# real finding the agent suite's check_file reports as a FAIL naming the file. +# Catching them here would replace that FAIL with a false "does not exist". +if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then + echo "Error: '$TARGET' does not exist." >&2 + echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2 + echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2 + exit 2 +fi + +# --- Detect the mode ------------------------------------------------------- +# Pure path and stat inspection, no interpreter and no external command needed, +# so it runs before the python3/PyYAML preflight — which is mode-specific, +# because each suite names the gates it would otherwise skip. +TARGET_BASE="$(_kf_basename "$TARGET")" +TARGET_PARENT="$(_kf_parent_name "$TARGET")" + +if [[ -d "$TARGET" ]]; then + if [[ -f "$TARGET/SKILL.md" ]]; then + MODE=skill + else + echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2 + echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2 + echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (.agent.md, or a .md file under an agents/ directory)." >&2 + exit 2 + fi +elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then + MODE=skill + TARGET="$(_kf_dirname "$TARGET")" +elif [[ "$TARGET_BASE" == *.agent.md ]]; then + MODE=agent +elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then + MODE=agent +else + echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2 + echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a .agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2 + echo " Fix: pass one of those two shapes." >&2 + exit 2 +fi + +# --- Run the matching suite ------------------------------------------------ +# Each suite is reassembled in the order the resolver block sat in before the +# merge — preamble, resolver, body — so every check runs against exactly the +# names and the order it always did. +RC=0 +case "$MODE" in + skill) + _kf_require_lib lib-boundary-resolver.sh + # shellcheck source=lib-boundary-resolver.sh + . "$SCRIPT_DIR/lib-boundary-resolver.sh" + _kf_require_lib lib-checks-skill.sh + # shellcheck source=lib-checks-skill.sh + . "$SCRIPT_DIR/lib-checks-skill.sh" + kyberforge_skill_preflight + PROG="$KYBERFORGE_SKILL_PREAMBLE_PY +$KYBERFORGE_RESOLVER_PY +$KYBERFORGE_SKILL_BODY_PY" + python3 -u - "$TARGET" <<< "$PROG" || RC=$? + ;; + agent) + _kf_require_lib lib-boundary-resolver.sh + # shellcheck source=lib-boundary-resolver.sh + . "$SCRIPT_DIR/lib-boundary-resolver.sh" + _kf_require_lib lib-checks-agent.sh + # shellcheck source=lib-checks-agent.sh + . "$SCRIPT_DIR/lib-checks-agent.sh" + kyberforge_agent_preflight + PROG="$KYBERFORGE_AGENT_PREAMBLE_PY +$KYBERFORGE_RESOLVER_PY +$KYBERFORGE_AGENT_BODY_PY" + python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$? + ;; +esac + +exit "$RC" diff --git a/plugins/kyberforge/.apm/skills/factory-audit/tests/README.md b/plugins/kyberforge/.apm/skills/factory-audit/tests/README.md new file mode 100644 index 0000000..2526d45 --- /dev/null +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/README.md @@ -0,0 +1,94 @@ +# tests/ + +Test files for scripts bundled with this skill. + +## When to add tests + +Add tests here when the skill has scripts in `scripts/` that are complex enough +to break silently — validators, parsers, generators, anything with branching +logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`) +belongs here, not in `scripts/`. + +## Dependencies + +Tests require [bats-support](https://github.com/bats-core/bats-support) and +[bats-assert](https://github.com/bats-core/bats-assert). The test files load +helpers from the repo root's `tests/test_helper/`. + +From the repo root: + +```bash +git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support +git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert +``` + +Run all tests for this skill (from the repo root): + +```bash +bats plugins/kyberforge/.apm/skills/factory-audit/tests/ +``` + +## Files + +| File | Purpose | +|------|---------| +| `validate-skill.bats` | `scripts/validate.sh` against skill directories | +| `validate-agent.bats` | `scripts/validate.sh` against agent files | +| `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories | +| `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files | + +## Two scripts, four suites + +`factory-audit` merges what were two skills — `skill-audit` and `agent-audit` — +each of which shipped its own `validate.sh` and `validate-provenance.sh`. The +merged skill has **one** of each. Every suite here invokes one of those two +scripts; the four files are two scripts × two artifact types, not four scripts. + +`validate-skill.bats` and `validate-agent.bats` run the same +`scripts/validate.sh` and differ only in the fixtures they point it at. The two +provenance suites stand in the same relation to `scripts/validate-provenance.sh`. +Do not add a third script path here on the assumption that a differently named +suite must mean a differently named script. + +### Auto-detection is pinned across the pair + +Each entry point decides for itself what it was handed. ADR-0025 states the +rule: a directory containing `SKILL.md` takes the skill flow; an `.agent.md` +file, or a file under a directory named `agents/`, takes the agent flow. +Anything else is rejected rather than guessed at. That behaviour is new with the +merge — before it, each script was hard-wired to one artifact type and nothing +about classification could be wrong — so it is asserted from both sides rather +than in one place: + +- the skill-side suites pin the skill-directory classification and the + neither-shape rejection, +- the agent-side suites pin the two agent rules *separately* — `.agent.md` in a + directory that is not `agents/`, and a plain `.md` under `.apm/agents/` — so + that a detector implementing only one of them cannot pass both. Plus a control + asserting an agent file never picks up a skill-only gate. + +Both skill-side suites additionally pin the `SKILL.md` **file** path, not just +the directory: a pre-commit `files:` hook matches files, so every hook-driven +invocation hands over a `SKILL.md` path. Each entry point resolves it to the +directory, and the suites assert the two spellings produce the identical +verdict rather than merely that the file spelling survives. + +A misclassification is silent and total — the wrong rubric runs end to end and +reports the artifact clean against gates that never applied to it — and no other +fixture in these suites would notice, because every other fixture is already the +shape its own suite expects. + +### The two provenance exit contracts are different on purpose + +`scripts/validate-provenance.sh` does **not** behave identically in its two +modes, and the difference is deliberate: + +| Mode | Successful run | +|------|----------------| +| skill | may exit 0 **with** output — INFO findings print, status stays 0 | +| agent | exits 0 and prints **nothing** | + +Each half is asserted from its own side, by the `exit contract:` test in each +provenance suite. Both are asserted on purpose: a merge that collapsed one +contract into the other would still satisfy whichever side was left unasserted, +so a single-sided pin would go green on exactly the defect it exists to catch. diff --git a/plugins/kyberforge/.apm/skills/agent-audit/tests/validate.bats b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-agent.bats similarity index 67% rename from plugins/kyberforge/.apm/skills/agent-audit/tests/validate.bats rename to plugins/kyberforge/.apm/skills/factory-audit/tests/validate-agent.bats index 9839841..2ec2594 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/tests/validate.bats +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-agent.bats @@ -131,7 +131,22 @@ EOF } @test "user scope: agent file directly in \$HOME (start dir IS exactly \$HOME, no walk-up) resolves to user scope" { - local fake_home="$TMPDIR/fakehome-direct" + # The fake $HOME is named `agents` on purpose, and it is a fixture detail + # forced by the merged entry point rather than anything the scope resolver + # cares about. The case under test needs the agent file's own directory to BE + # $HOME (detect_scope's `original_start == home` branch), AND the input has + # to be the Claude Code half — a plain .md — because that is the only half + # whose counterpart path differs between user scope ($HOME/.copilot/agents) + # and project scope ($HOME/.github/agents). With a *.agent.md input the two + # scopes derive the identical counterpart and the test would pass without + # discriminating. Post-merge, a plain .md only classifies as an agent when + # its parent directory is named `agents`, so the two requirements can only be + # satisfied together by a $HOME whose basename is `agents`. + # + # detect_scope's conventional_shape stays FALSE here (the grandparent is not + # one of .claude/.github/.copilot/.apm), so this still exercises the + # no-walk-up branch and not the two-segments-up arithmetic. + local fake_home="$TMPDIR/fakehome-direct/agents" mkdir -p "$fake_home" "$fake_home/.copilot/agents" cat > "$fake_home/my-agent.md" < "$pkg/extra/my-agent.md" < "$pkg/extra/my-agent.agent.md" < "$pkg/.github/agents/my-agent.agent.md" < "$pkg/.claude/agents/my-agent.md" < "$root/apm.yml" < "$root/apm.yml" < "$root/apm.yml" < "$root/.github/agents/my-agent.agent.md" < "$root/apm.yml" < "$root/defs/my-agent.agent.md" < "$root/apm.yml" < "$root/.apm/agents/my-agent.md" < "$decoy/scripts/$lib" + done + cd "$(dirname "$SCRIPT")/.." + run env CDPATH="$decoy" bash scripts/validate.sh "$root/.apm/agents/my-agent.agent.md" + refute_output --partial "DECOY-SOURCED" + assert_success +} diff --git a/plugins/kyberforge/.apm/skills/agent-audit/tests/validate-provenance.bats b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-agent.bats similarity index 72% rename from plugins/kyberforge/.apm/skills/agent-audit/tests/validate-provenance.bats rename to plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-agent.bats index 7ae7edf..2781469 100644 --- a/plugins/kyberforge/.apm/skills/agent-audit/tests/validate-provenance.bats +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-agent.bats @@ -586,12 +586,41 @@ EOF # pins it as exit 0 with empty output. Every exit-2 gate is therefore decided # from the ARGUMENT ALONE, before the walk-up runs, so the two can never # collide. The two tests at the end of this block assert both halves. +# +# PORT NOTE RESOLVED (factory-audit merge): two cases in this block asserted +# wording that the skill suite asserted differently for the same class of input +# — "agent-file is required" against its "skill-dir is required", and "no such +# file" against its "not a directory". Each pre-merge script knew what shape it +# was owed; the merged one classifies before it complains. How each pair landed: +# +# * no argument — NEITHER old wording survives. One entry point takes both +# shapes, so it names both: "a skill directory or an agent file is +# required." Both suites now assert that one sentence. +# * nonexistent path — "no such file" SURVIVES here and did not move. A +# nonexistent *.agent.md still classifies as an agent on its name alone, so +# agent mode's own precondition is what rejects it. The skill suite's "not a +# directory" is the wording that moved, because a nonexistent path with no +# agent-shaped name classifies as neither. +# +# Two further cases in this block did move, and neither moved to the other +# suite's wording: a directory handed to this script is now "is a directory with +# no SKILL.md in it" rather than "not a regular file", and an unrecognized +# extension is "matches neither a skill directory nor an agent file" rather than +# "unrecognized extension". Both are classification verdicts now, reached before +# either mode's preconditions run. +# +# The invariant every case in this block pins is unchanged and still pinned at +# every site: exit 2 with an explanatory message on stderr, never a silent exit +# 0 or a bare 1 with no output. # --------------------------------------------------------------------------- @test "exit 2: no arguments is a usage error, not a finding" { run bash "$SCRIPT" [ "$status" -eq 2 ] - assert_output --partial "agent-file is required" + assert_output --partial "Error: a skill directory or an agent file is required." + # The usage block has to follow the error, or "required" names no shape the + # caller can act on. + assert_output --partial "Usage: validate-provenance.sh" } @test "exit 2: a second positional argument is rejected instead of silently dropped" { @@ -610,20 +639,29 @@ EOF } @test "exit 2: a directory is not an agent file" { + # Post-merge this is a CLASSIFICATION rejection, not agent mode's old + # "not a regular file" precondition: a directory is half of a skill target, + # so the detector reaches for SKILL.md, does not find one, and says so. The + # thing being pinned is the same — a directory handed to a script that + # audits agent FILES exits 2 with a message, never a silent 0. local root="$TMPDIR/package" make_package "$root" run bash "$SCRIPT" "$root/.apm/agents" [ "$status" -eq 2 ] - assert_output --partial "not a regular file" + assert_output --partial "Error: '$root/.apm/agents' is a directory with no SKILL.md in it." } @test "exit 2: an unrecognized extension is rejected before the walk-up runs" { + # Also a classification rejection now. .txt under .apm/agents/ is neither + # shape: the agents/-parent rule only admits .md, and this is not *.agent.md. + # Rejected on the name alone, so the walk-up still never runs — which is what + # keeps this case from colliding with the silent not-plugin-scope exit 0. local root="$TMPDIR/package" make_package "$root" echo "not an agent" > "$root/.apm/agents/my-agent.txt" run bash "$SCRIPT" "$root/.apm/agents/my-agent.txt" [ "$status" -eq 2 ] - assert_output --partial "unrecognized extension" + assert_output --partial "Error: '$root/.apm/agents/my-agent.txt' matches neither a skill directory nor an agent file." } @test "exit 2: a PATH with no python3 names the missing dependency instead of exiting 127" { @@ -632,6 +670,22 @@ EOF make_clean_agent "$root" local emptybin="$TMPDIR/emptybin" mkdir -p "$emptybin" + # dirname and basename are deliberately ABSENT, and that absence is load- + # bearing. The invariant: nothing external is needed to reach the python3 + # preflight. The entry point resolves SCRIPT_DIR and classifies the target + # with bash builtins (_kf_dirname, _kf_basename, _kf_parent_name) and the + # mode libraries build their Python bodies with `read` heredocs, so a + # genuinely empty PATH reaches the preflight too. Widening this list to keep + # a test green would silently retire that guarantee: the script would die at + # 127 naming `dirname` instead of the dependency it actually needs, the + # failure tests/test-adr0020-contract.sh assertion 2 exists to prevent. cat + # and sed stay so the stub matches the skill suite's and is a DENY OF python3 + # ALONE rather than a test of "no PATH at all"; bash is not among them + # because `env -i` below invokes it by absolute path. + local cmd + for cmd in cat sed; do + ln -s "$(command -v "$cmd")" "$emptybin/$cmd" + done local bash_bin bash_bin="$(command -v bash)" run env -i PATH="$emptybin" HOME="$HOME" "$bash_bin" "$SCRIPT" "$root/.apm/agents/my-agent.agent.md" @@ -907,3 +961,187 @@ EOF count="$(printf '%s\n' "$output" | grep -c "^FAIL File is not valid UTF-8" || true)" [ "$count" -eq 1 ] } + +# --------------------------------------------------------------------------- +# Auto-detection — the merged provenance entry point classifies its own target +# +# NEW with the factory-audit merge, and new behaviour rather than a ported +# case: scripts/validate-provenance.sh is now ONE entry point for both artifact +# types and works out from the target which rubric to run. This file is the +# AGENT half of that contract; validate-provenance-skill.bats holds the skill +# half and the neither-shape rejection. Same script in all four suites. +# +# ADR-0025 states the rule: "A directory containing SKILL.md takes the skill +# flow; an .agent.md file or a file under agents/ takes the agent flow." That is +# TWO independent rules on the agent side, and each is pinned on its own below, +# because either one alone would make the other look like it worked: +# +# 1. the filename ends in .agent.md, wherever it sits +# 2. the file sits under a directory named agents/, whatever it is called +# (.apm/agents/, .claude/agents/, .github/agents/ — the same shape the +# exported Vale hook matches with `(^|/)agents/[^/]+\.md$`) +# +# The skill rubric rejects a FILE outright ("not a directory"), so a +# misclassified agent file does not produce a wrong finding — it produces a +# usage error about a file that was perfectly well-formed, which is why these +# assert on a real finding rather than merely on a non-crash. +# --------------------------------------------------------------------------- + +@test "auto-detect: a *.agent.md file is checked in AGENT provenance mode wherever it sits" { + # Rule 1 in isolation. The directory is deliberately named `defs/`, not + # `agents/`, so rule 2 cannot reach this fixture and the extension is the + # only thing that can classify it. + local root="$TMPDIR/package" + make_package "$root" + mkdir -p "$root/defs" + cat > "$root/defs/my-agent.agent.md" < "$root/.apm/agents/my-agent.md" < "$root/.apm/agents/my-agent.md" < "$decoy/scripts/$lib" + done + cd "$(dirname "$SCRIPT")/.." + run env CDPATH="$decoy" bash scripts/validate-provenance.sh "$root/.apm/agents/my-agent.agent.md" + [ "$status" -eq 0 ] + assert_output "" +} diff --git a/plugins/kyberforge/.apm/skills/skill-audit/tests/validate-provenance.bats b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats similarity index 87% rename from plugins/kyberforge/.apm/skills/skill-audit/tests/validate-provenance.bats rename to plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats index 3eae819..90b6a1e 100644 --- a/plugins/kyberforge/.apm/skills/skill-audit/tests/validate-provenance.bats +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-provenance-skill.bats @@ -1291,19 +1291,44 @@ EOF # --------------------------------------------------------------------------- # Cycle 21 — G1: a bad target is a hard error, not a silent pass +# +# PORT NOTE RESOLVED (factory-audit merge): this block used to assert +# "not a directory" for the nonexistent target while the agent suite asserted +# "no such file" for its own, because each pre-merge script knew what shape it +# was owed. The merged entry point classifies before it complains, so neither +# old wording survives at THIS site and the two cases are no longer the same +# case: +# +# * a path that exists as nothing this script recognises (and a path that does +# not exist at all, when its name is not *.agent.md and its parent is not +# agents/) is "matches neither a skill directory nor an agent file"; +# * a path that IS a directory but holds no SKILL.md gets its own, more +# specific line naming the missing SKILL.md. +# +# The agent suite's "no such file" wording did NOT move — a nonexistent +# *.agent.md path still classifies as an agent and is rejected by agent mode's +# own precondition — so that side stays verbatim over there. Both wordings are +# now asserted exactly rather than partially-matched, because the invariant both +# sides pin is exit 2 with a message that names the path, never a silent exit 0 +# and never a bare code with no output. # --------------------------------------------------------------------------- @test "G1: a nonexistent directory is a hard error (exit 2), not a silent exit 0" { run bash "$SCRIPT" "$TMPDIR/does-not-exist" [ "$status" -eq 2 ] - assert_output --partial "not a directory" + assert_output --partial "Error: '$TMPDIR/does-not-exist' matches neither a skill directory nor an agent file." } @test "G1: a directory with no SKILL.md is a hard error (exit 2), not a silent exit 0" { + # Distinct from the case above on purpose: a directory IS half of a skill + # target, so the detector can say something sharper than "neither shape" and + # does. The exit code is unchanged from the pre-merge test, which already + # pinned 2; only the wording moved, from the suite's own "not a skill + # directory" precondition to the dispatcher's classification message. mkdir -p "$TMPDIR/not-a-skill/references" run bash "$SCRIPT" "$TMPDIR/not-a-skill" [ "$status" -eq 2 ] - assert_output --partial "not a skill directory" + assert_output --partial "Error: '$TMPDIR/not-a-skill' is a directory with no SKILL.md in it." } # --------------------------------------------------------------------------- @@ -1496,12 +1521,27 @@ EOF # --------------------------------------------------------------------------- # Cycle 27 — G8: usage and environment errors exit 2, never 1 +# +# PORT NOTE RESOLVED (factory-audit merge): the two suites were written against +# two separate scripts and disagreed about the no-argument wording — this file +# asserted "skill-dir is required", the agent suite "agent-file is required". +# Neither survives. One entry point takes both target shapes, so its no-argument +# message names both, and both suites now assert that one sentence. Exit 2 and a +# message (not a silent exit 1, and not a bare code with no output) is the +# invariant both sides were really pinning, and it is still pinned from both. +# +# Note the asymmetry with scripts/validate.sh, which exits 1 on no argument: +# only validate-provenance.sh carries the exit-2 usage tier, and this test is +# what holds it there. # --------------------------------------------------------------------------- @test "G8: a missing argument exits 2, not 1" { run bash "$SCRIPT" [ "$status" -eq 2 ] - assert_output --partial "skill-dir is required" + assert_output --partial "Error: a skill directory or an agent file is required." + # The usage block has to follow the error, or "required" names no shape the + # caller can act on. + assert_output --partial "Usage: validate-provenance.sh" } @test "G8: an extra positional argument is rejected, not silently ignored" { @@ -1517,6 +1557,18 @@ EOF make_clean_skill "$skill" local stub="$TMPDIR/emptybin" mkdir -p "$stub" + # dirname and basename are deliberately ABSENT, and that absence is load- + # bearing. The invariant: nothing external is needed to reach the python3 + # preflight. The entry point resolves SCRIPT_DIR and classifies the target + # with _kf_dirname/_kf_basename/_kf_parent_name — pure-bash replacements + # that exist for precisely this reason — and the mode libraries build their + # Python bodies with `read` heredocs. Widening this list to keep a test + # green would silently retire that guarantee: on a PATH with neither + # coreutils nor python3 the script would die at 127 naming `dirname` + # instead of naming the dependency it actually needs, which is the failure + # tests/test-adr0020-contract.sh assertion 2 exists to prevent. bash, cat + # and sed stay so the stub is a DENY OF python3 ALONE rather than a test of + # "no PATH at all" — the diagnostic under test here is the python3 one. for cmd in bash cat sed; do ln -s "$(command -v "$cmd")" "$stub/$cmd" done @@ -1900,3 +1952,111 @@ EOF assert_output --partial "Check 9 skipped — no base ref could be resolved" assert_output --partial "not-a-real-ref" } + +# --------------------------------------------------------------------------- +# Auto-detection — the merged provenance entry point classifies its own target +# +# NEW with the factory-audit merge, and new behaviour rather than a ported +# case: scripts/validate-provenance.sh is now ONE entry point for both artifact +# types and works out from the target which rubric to run. A DIRECTORY holding +# SKILL.md is a skill; a FILE named *.agent.md, or sitting under .apm/agents/, +# is an agent. +# +# The two rubrics are not near-copies of one another — the skill side has a +# check 9 the agent side has none of, and reads sources.md from the skill's own +# references/ rather than from the package root — so a misclassification is not +# a near miss. It runs a set of checks that cannot apply and skips the set that +# can, at exit 0. +# +# This file is the SKILL half plus the neither-shape rejection; +# validate-provenance-agent.bats holds the agent half. Same script in both. +# --------------------------------------------------------------------------- + +@test "auto-detect: a directory holding SKILL.md is checked in SKILL provenance mode" { + local skill="$TMPDIR/my-skill" + make_skill_with_source_keys "$skill" + make_sources_md "$skill" + # No commit_as_base, so check 9 has no repo root and announces the skip. + run bash "$SCRIPT" "$skill" + assert_success + # Check 9 exists only on the skill side — the agent rubric has no check 9 at + # any tier — so naming it is positive proof the SKILL rubric ran, rather + # than merely that nothing crashed. + assert_output --partial "Check 9 skipped — no repo root above the skill directory" +} + +@test "auto-detect: a SKILL.md FILE path is checked in SKILL provenance mode, not rejected" { + # NEW with the merge and additive: pre-merge, handing the SKILL.md itself to + # skill-audit's validate-provenance.sh hit the "not a directory" precondition + # and died. It matters because pre-commit `files:` hooks match FILES — the + # exported kyberforge-vale-audit-skill hook's regex is (^|/)SKILL\.md$ — so + # every hook-driven invocation hands over a SKILL.md path, never its + # directory. The entry point rewrites the token to the directory in place. + local skill="$TMPDIR/my-skill" + make_skill_with_source_keys "$skill" + make_sources_md "$skill" + run bash "$SCRIPT" "$skill/SKILL.md" + assert_success + # Same positive proof as the directory case: check 9 is skill-only. + assert_output --partial "Check 9 skipped — no repo root above the skill directory" +} + +@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" { + # The rewrite must be transparent, not merely non-fatal. If the two spellings + # of the same target could disagree, a pre-commit run and a hand run would + # report differently on one skill and neither would be obviously wrong. + local skill="$TMPDIR/my-skill" + make_skill_with_source_keys "$skill" + make_sources_md "$skill" + + run bash "$SCRIPT" "$skill" + local dir_status="$status" + local dir_output="$output" + + run bash "$SCRIPT" "$skill/SKILL.md" + [ "$status" -eq "$dir_status" ] + [ "$output" = "$dir_output" ] + # Guard against the comparison being satisfied by two empty runs: this + # fixture has a check-9 INFO to print, so silence here means neither + # spelling ran the rubric. + refute_output "" +} + +@test "exit contract: SKILL mode exits 0 WITH output when the only findings are INFO" { + # The two modes' exit contracts are DIFFERENT and the merge must not quietly + # unify them. Skill mode is allowed to be chatty on a passing run: INFO + # findings print and the status stays 0. Agent mode's opposite half — a + # passing run prints nothing at all — is pinned from its own side in + # validate-provenance-agent.bats. + # + # Both halves are asserted explicitly and on purpose. A merge that collapsed + # one contract into the other would still satisfy whichever side was left + # unasserted, so a single-sided pin would go green on exactly the defect it + # was written to catch. + local skill="$TMPDIR/my-skill" + make_skill_with_source_keys "$skill" + make_sources_md "$skill" + run bash "$SCRIPT" "$skill" + [ "$status" -eq 0 ] + refute_output "" + assert_output --partial "INFO" + refute_output --partial "FAIL" +} + +@test "auto-detect: a provenance target that is neither a skill directory nor an agent file FAILs, naming what it was handed" { + # The detector must not guess. Guessing here is worse than in validate.sh: + # this script's whole not-in-scope path is a SILENT exit 0, so a wrong guess + # followed by "provenance does not apply at this scope" is indistinguishable + # from a clean pass — which is the exact confusion the exit-2 tier above was + # created to end. + # + # A plain .txt file is neither shape under any reading of the contract: not + # a directory holding SKILL.md, not *.agent.md, not under .apm/agents/. + local dir="$TMPDIR/neither" + mkdir -p "$dir" + echo "not an artifact of either kind" > "$dir/notes.txt" + run bash "$SCRIPT" "$dir/notes.txt" + assert_failure + refute_output "" + assert_output --partial "$dir/notes.txt" +} diff --git a/plugins/kyberforge/.apm/skills/skill-audit/tests/validate.bats b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-skill.bats old mode 100755 new mode 100644 similarity index 86% rename from plugins/kyberforge/.apm/skills/skill-audit/tests/validate.bats rename to plugins/kyberforge/.apm/skills/factory-audit/tests/validate-skill.bats index 6a8cb97..3a4d227 --- a/plugins/kyberforge/.apm/skills/skill-audit/tests/validate.bats +++ b/plugins/kyberforge/.apm/skills/factory-audit/tests/validate-skill.bats @@ -728,7 +728,7 @@ PY # A skill carrying `disable-model-invocation: true` is absent from the # model-visible listing entirely: not preloaded, and the Skill tool refuses to # call it. Its description is never matched against user intent, so -# references/description-quality.md Step 0 gives it ONE plain human-facing +# references/skill-description-quality.md Step 0 gives it ONE plain human-facing # sentence — no trigger list, no boundary clause — and calls a # missing-boundary-clause finding on such a skill "a wrong finding, not a strict # one". Until this ran, nothing here knew the field existed, so the audit @@ -920,3 +920,110 @@ EOF refute_output --partial "UnicodeEncodeError" refute_output --partial "Traceback" } + +# --------------------------------------------------------------------------- +# Auto-detection — the merged entry point classifies its own target +# +# NEW with the factory-audit merge, and new behaviour rather than a ported +# case: scripts/validate.sh is now ONE entry point for both artifact types and +# works out from the target which rubric to run. A DIRECTORY holding SKILL.md is +# a skill; a FILE named *.agent.md, or sitting under .apm/agents/, is an agent. +# +# Before the merge each script was hard-wired to one type, so there was nothing +# here that could be wrong. Now a misclassification is silent and total: the +# wrong rubric runs end to end and reports the artifact clean against gates that +# never applied to it, while every gate that did apply goes unrun. Nothing else +# in this suite would notice, because every other fixture is a skill directory +# and would be classified correctly even by a detector that always guessed +# "skill". +# +# The agent half of the same contract is pinned from the other side, in +# validate-agent.bats — same script, same detector, agent-shaped fixtures. +# --------------------------------------------------------------------------- + +@test "auto-detect: a directory holding SKILL.md is audited in SKILL mode" { + local skill="$TMPDIR/my-skill" + make_valid_skill "$skill" + run bash "$SCRIPT" "$skill" + assert_success + # ADR-0022's metadata.version is mandatory for skills and has no agent + # analogue whatsoever, so this line is positive proof the SKILL rubric ran — + # not merely that the run survived. A bare assert_success would be satisfied + # by a detector that classified this as an agent and found nothing to say. + assert_output --partial "metadata.version present" + # 'counterpart' is agent-mode vocabulary (the CC/Copilot pair check). A skill + # directory must never reach a check that has a concept of a counterpart. + refute_output --partial "counterpart" +} + +@test "auto-detect: a SKILL.md FILE path is audited in SKILL mode, not rejected" { + # NEW with the merge and additive rather than ported: pre-merge, handing the + # SKILL.md itself to skill-audit's validate.sh hit the directory precondition + # and gave a useless exit 1. It matters because pre-commit `files:` hooks + # match FILES — the exported kyberforge-vale-audit-skill hook's regex is + # (^|/)SKILL\.md$ — so every hook-driven invocation hands over a SKILL.md + # path, never the directory above it. The entry point resolves the file to + # its directory before dispatching. + local skill="$TMPDIR/my-skill" + make_valid_skill "$skill" + run bash "$SCRIPT" "$skill/SKILL.md" + assert_success + # Same positive proof as the directory case: metadata.version is skill-only + # and has no agent analogue, so this line says the SKILL rubric ran. + assert_output --partial "metadata.version present" + refute_output --partial "counterpart" +} + +@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" { + # The resolution must be transparent, not merely non-fatal. If the two + # spellings of one target could disagree, a pre-commit run and a hand run + # would report differently on the same skill and neither would look wrong. + # The name check is the sharp end: it compares `name` against the DIRECTORY + # basename, so a target left as the file would compare against "SKILL.md". + local skill="$TMPDIR/my-skill" + make_valid_skill "$skill" + + run bash "$SCRIPT" "$skill" + local dir_status="$status" + local dir_output="$output" + + run bash "$SCRIPT" "$skill/SKILL.md" + [ "$status" -eq "$dir_status" ] + [ "$output" = "$dir_output" ] + assert_output --partial "name 'my-skill' matches directory 'my-skill'" +} + +@test "auto-detect: a target that is neither a skill directory nor an agent file FAILs, naming what it was handed" { + # The detector must not guess. Falling back to either rubric on an + # unclassifiable target yields a verdict about rules that were never meant + # to apply, and exiting 0 publishes that verdict as a pass — the worst of + # the three possible outcomes, because it is the silent one. + # + # A plain .txt file is neither shape under any reading of the contract: not + # a directory holding SKILL.md, not *.agent.md, not under .apm/agents/. The + # directory flavour of the same mismatch — a directory with no SKILL.md — is + # pinned separately by "fails when SKILL.md is missing" above. + local dir="$TMPDIR/neither" + mkdir -p "$dir" + echo "not an artifact of either kind" > "$dir/notes.txt" + run bash "$SCRIPT" "$dir/notes.txt" + assert_failure + # Non-zero is necessary but not sufficient: a non-zero exit with nothing on + # stdout is indistinguishable from a clean-but-failing run, which is the + # confusion the exit-2 tier in the provenance suites was created to end. + refute_output "" + assert_output --partial "$dir/notes.txt" +} + +@test "entry point: a missing resolver library in skill mode exits 2 naming it, never exit 1" { + # validate-agent.bats pins this for agent mode; each mode sources its own + # libraries, so each mode's guard is pinned separately. + make_valid_skill "$TMPDIR/my-skill" + local lone="$TMPDIR/lone-scripts" + cp -R "$(dirname "$SCRIPT")" "$lone" + rm "$lone/lib-boundary-resolver.sh" + run bash "$lone/validate.sh" "$TMPDIR/my-skill" + [ "$status" -eq 2 ] + assert_output --partial "required library '$lone/lib-boundary-resolver.sh' is missing or unreadable" + refute_output --partial "No such file or directory" +} diff --git a/plugins/kyberforge/.apm/skills/forge/SKILL.md b/plugins/kyberforge/.apm/skills/forge/SKILL.md index a88377a..8bc3053 100644 --- a/plugins/kyberforge/.apm/skills/forge/SKILL.md +++ b/plugins/kyberforge/.apm/skills/forge/SKILL.md @@ -18,7 +18,7 @@ metadata: ## Gotchas -- forge is an optional guided entry point, not a gate — `skill-author`, `skill-audit`, `agent-author`, `agent-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one. +- forge is an optional guided entry point, not a gate — `skill-author`, `agent-author`, `factory-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one. - Claude Code's skill-level `context: fork` frontmatter field and the `/fork` subagent command are opposites despite the shared word: `context: fork` isolates (fresh context, no parent access), while `/fork` inherits the full conversation. The route reference each classification loads spends that distinction: `references/author-routes.md` chooses between the two, `references/apm-routes.md` rules the fork out. ## Step 1 — Grill the intent diff --git a/plugins/kyberforge/.apm/skills/forge/references/author-routes.md b/plugins/kyberforge/.apm/skills/forge/references/author-routes.md index 237debc..5314934 100644 --- a/plugins/kyberforge/.apm/skills/forge/references/author-routes.md +++ b/plugins/kyberforge/.apm/skills/forge/references/author-routes.md @@ -7,8 +7,8 @@ source_keys: Reached from `SKILL.md` Step 2 when the classified artifact is a skill or an agent/subagent definition. Route a skill to `skill-author` and an agent to `agent-author`. The two branches -differ on one axis only — which audit skill verifies the result — and everything below applies to -both. +differ on the author skill only — both verify the result with `factory-audit`, which detects the +artifact type itself — and everything below applies to both. ## Choose fork or inline @@ -26,8 +26,8 @@ Fall back to an **inline invocation** — same conversation, no subagent — whe ## Two-tier verification Both author skills already close out with their own inline audit, in the same context as the -authoring work: `skill-author` runs `/skill-audit`, `agent-author` invokes -`agent-audit`. That is tier one, and forge does not change it. +authoring work: `skill-author` and `agent-author` each invoke `factory-audit` on what they wrote. +That is tier one, and forge does not change it. Tier two belongs to forge. Once the author skill's run has finished, spin up a separate **clean-context subagent** — fresh, not forked, no inherited context — to independently re-run the diff --git a/plugins/kyberforge/.apm/skills/skill-audit/SKILL.md b/plugins/kyberforge/.apm/skills/skill-audit/SKILL.md deleted file mode 100644 index c6f0103..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/SKILL.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -name: skill-audit -description: > - Use when the user wants a skill directory audited against the agentskills.io - spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or - after hand-editing a skill outside skill-author. Not applying fixes -> - skill-author. -allowed-tools: Bash Read -metadata: - version: "1.0.2" - category: factory - source_keys: - - agentskills-home - - agentskills-spec - - agentskills-best-practices - - agentskills-optimizing-descriptions - - agentskills-using-scripts ---- - -## Gotchas - -- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here. -- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead. -- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately. -- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered. - -## Step 1 — Deterministic checks - -Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly: - -```bash -bash scripts/validate.sh -bash scripts/validate-provenance.sh -bash scripts/vale-wrap.sh /SKILL.md -``` - -`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation. - -Read `references/validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback. - -`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — a usage or environment error, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass. - -`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment: - -| Rule | Dimension | -|---|---| -| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description | -| `Kyberforge.SentenceOpenerThereIs` | body-discipline | -| `Kyberforge.PaddingPhrase` | patterns | - -## Step 2 — Read the whole skill - -Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture. - -## Step 3 — Qualitative audit - -Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call. - -| Dimension | Rubric | -|---|---| -| description | `references/description-quality.md` | -| body-discipline | `references/body-discipline.md` | -| patterns | `references/patterns.md` | -| file-structure, internal-consistency | `references/file-structure.md` | -| formatting, scripts | `references/formatting-and-scripts.md` | - -Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding. - -## Step 4 — Report - -Open with a coverage line naming every dimension checked: - -```text -Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance -``` - -Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed. - -Each finding: - -```text -FAIL/SUGGESTION — file:line - Why: - Fix: -``` - -Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only. diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini deleted file mode 100644 index b7ce2e5..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini +++ /dev/null @@ -1,4 +0,0 @@ -StylesPath = styles - -[**/SKILL.md] -BasedOnStyles = Kyberforge diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/CompositionNote.yml b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/CompositionNote.yml deleted file mode 100644 index 90ea015..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/CompositionNote.yml +++ /dev/null @@ -1,13 +0,0 @@ -extends: existence -message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md" -level: error -scope: text.frontmatter.description -ignorecase: true -tokens: - - cross-cutting - - shared (skill|agent) - - human-facing - - entry[- ]point - - composes - - rather than duplicating - - replaces the (old|former|previous) diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml deleted file mode 100644 index 1f41236..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/DescriptionOpener.yml +++ /dev/null @@ -1,7 +0,0 @@ -extends: existence -message: "Description opens with '%s' — use an imperative 'Use when...' opener instead" -level: error -scope: text.frontmatter.description -ignorecase: true -raw: - - '^This\b' diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml deleted file mode 100644 index 4c5f5ae..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/PaddingPhrase.yml +++ /dev/null @@ -1,7 +0,0 @@ -extends: existence -message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead" -level: error -scope: text -ignorecase: true -raw: - - 'see references?/? for (more )?(info|information|details)\b' diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml deleted file mode 100644 index c443bad..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/SentenceOpenerThereIs.yml +++ /dev/null @@ -1,7 +0,0 @@ -extends: existence -message: "Don't start a sentence with '%s' — name the subject directly" -level: error -scope: sentence -ignorecase: false -raw: - - '^There\s(is|are)\b' diff --git a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/VagueWording.yml b/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/VagueWording.yml deleted file mode 100644 index 45cb87f..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/styles/Kyberforge/VagueWording.yml +++ /dev/null @@ -1,10 +0,0 @@ -extends: existence -message: "Vague capability wording: '%s' — state the capability precisely instead" -level: error -scope: text.frontmatter.description -ignorecase: true -tokens: - - helps with - - utilize - - assists with - - used for diff --git a/plugins/kyberforge/.apm/skills/skill-audit/references/sources.md b/plugins/kyberforge/.apm/skills/skill-audit/references/sources.md deleted file mode 100644 index a01c5dd..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/references/sources.md +++ /dev/null @@ -1,59 +0,0 @@ -# Sources - - - -## agentskills-home - -- **URL:** https://agentskills.io/home.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools -- **Contributing files:** SKILL.md -- **Status:** `extracted` - -## agentskills-spec - -- **URL:** https://agentskills.io/specification.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation -- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md -- **Status:** `extracted` - -## agentskills-best-practices - -- **URL:** https://agentskills.io/skill-creation/best-practices.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops) -- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md, references/finding-criteria.md -- **Status:** `extracted` - -## agentskills-optimizing-descriptions - -- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop -- **Contributing files:** SKILL.md, references/description-quality.md, references/finding-criteria.md -- **Status:** `extracted` - -## agentskills-evaluating-skills - -- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop -- **Contributing files:** (none — eval workflow not directly informing audit dimensions) -- **Status:** `extracted` - -## agentskills-using-scripts - -- **URL:** https://agentskills.io/skill-creation/using-scripts.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency) -- **Contributing files:** SKILL.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md -- **Status:** `extracted` - -## agentskills-quickstart - -- **URL:** https://agentskills.io/skill-creation/quickstart.md -- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md -- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice -- **Contributing files:** (none — creation guide not directly informing audit criteria) -- **Status:** `extracted` diff --git a/plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh b/plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh deleted file mode 100755 index 862f44c..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh +++ /dev/null @@ -1,526 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Works around a Vale limitation: the `text.frontmatter.description` NLP scope -# silently stops matching once the `description:` value spans 2+ physical lines -# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block -# scalar (the style used by most skills/agents in this repo), a plain scalar -# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped -# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed -# value keeps exactly the line breaks the source has, and vale matches it fine -# (verified against vale 3.15.2), so literal blocks are deliberately left alone. -# This script flattens an affected description to a one-line scalar in a scratch -# copy — or, for the rare value no inline scalar can spell out verbatim, to a -# `|-` literal block with a single content line, which vale matches just as well -# (padding with blank lines so every other line number is unchanged), then -# runs the real `vale` binary against the copies. Drop-in replacement for calling -# `vale` directly: same args, same exit code, bar the two documented divergences -# below. -# -# "Same args" means relative paths — path arguments and the values of the -# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against -# the caller's current directory, exactly as bare `vale` resolves them. The flag -# values are rewritten to absolute form because the run ends up `cd`'d into the -# scratch mirror, where a relative one would no longer resolve. (An earlier -# version resolved path arguments against the repo root, an invented convention -# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse, -# silently dropped file arguments that didn't happen to resolve from the repo -# root — skipping the flattening this script exists for.) -# -# Divergence 1: with no `--config` at all, this script's own sibling -# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit -# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in -# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and -# hard-fail (E100) for every external consumer. The manifest therefore passes the -# script alone, and an explicit `--config` from any other caller still wins. -# -# Divergence 2: a path-shaped argument that does not exist is a hard error -# (exit 2). Bare vale drops it, falls back to reading stdin, and prints -# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable -# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather -# than clean, and `in stdin` does not match that guard, so the silent form would -# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way -# to keep that guard honest. Linting prose piped on stdin is therefore -# unsupported here — it already was, since the no-path handoff closes stdin so -# vale can't block on a pipe that will never carry content. -# -# Vale prints each path exactly as it was handed to it, so the scratch tree -# mirrors the caller's absolute cwd: a relative path argument is passed through -# verbatim and resolves to its flattened copy, keeping the report byte-identical -# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that -# property. Only an absolute path outside the cwd is rewritten to its scratch -# copy and so reports a scratch path — unavoidable, since a file can only be -# read from where it actually is. - -cwd="$(pwd -P)" - -# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 — -# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"` -# on an empty array as an unbound variable under `set -u`. No expansion site is -# reachable while empty on today's control flow, so this is insurance against a -# later edit breaking that invariant, not a live fix. -vale_args=() -path_args=() -pending_flag="" -config_given=false - -# `--output` takes either one of vale's built-in style names or a template file -# path. Only the file form needs absolutizing, and the built-in names have to be -# excluded by name *before* the existence test below: a file or directory -# literally called `line` in the caller's cwd would otherwise rewrite the -# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template] -# Runtime error`) where bare vale just uses the built-in. `--path` has no such -# names — it is always a path — so the check is keyed on the flag too. -is_builtin_output() { - case "$2" in - line|JSON|CLI) [[ "$1" == "--output" ]] ;; - *) false ;; - esac -} -# Absolutizes a `--config` value against the caller's cwd. Shared by both -# argument forms below — separated (`--config X`) and joined (`--config=X`) -# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly -# one place instead of being duplicated per form. -abs_config_value() { - if [[ "$1" == /* ]]; then - printf '%s' "$1" - else - printf '%s' "$cwd/$1" - fi -} -for arg in "$@"; do - if [[ -n "$pending_flag" ]]; then - # Value of a separated two-argv flag. It is never a lint target, however - # file-like it looks. The run ends up `cd`'d into the scratch mirror, so a - # value naming a file has to be absolutized here or it stops resolving. - case "$pending_flag" in - --config) - # Always a path, and required to exist. - vale_args+=("$(abs_config_value "$arg")") - ;; - --output|--path) - # See `is_builtin_output` above for why the built-in `--output` names - # are excluded first. Anything that names nothing is passed through and - # left for vale to interpret. - if is_builtin_output "$pending_flag" "$arg"; then - vale_args+=("$arg") - elif [[ "$arg" != /* && -e "$arg" ]]; then - vale_args+=("$cwd/$arg") - else - vale_args+=("$arg") - fi - ;; - *) - vale_args+=("$arg") - ;; - esac - pending_flag="" - continue - fi - case "$arg" in - --config) - vale_args+=("$arg") - pending_flag="$arg" - config_given=true - continue - ;; - --config=*) - vale_args+=("--config=$(abs_config_value "${arg#--config=}")") - config_given=true - continue - ;; - # Same cwd-relative resolution for the `--flag=value` spelling of the two - # other path-valued flags. - --output=*|--path=*) - flag_val="${arg#*=}" - if is_builtin_output "${arg%%=*}" "$flag_val"; then - vale_args+=("$arg") - elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then - vale_args+=("${arg%%=*}=$cwd/$flag_val") - else - vale_args+=("$arg") - fi - continue - ;; - # Vale's remaining value-taking flags, per `vale --help` (3.x). In the - # separated two-argv form the value must not be classified as a lint target - # — `--output tmpl.tmpl` names a real template file, and treating it as - # input both lints the template and reorders argv so vale sees - # `--output --no-wrap`. The `--flag=value` form needs no entry here: it - # starts with `-` and falls through to vale untouched. A value flag added by - # some future vale release is simply absent from this list and lands back on - # today's behaviour, so this list going stale is never worse than not having - # it. - --ext|--filter|--glob|--minAlertLevel|--output|--path) - vale_args+=("$arg") - pending_flag="$arg" - continue - ;; - # Vale's subcommands are bare words that name no file, so they would trip - # the not-found error below. A lint target literally named `sync` (no - # extension, no slash) is misread as the subcommand — accepted, because the - # alternative is failing every `vale-wrap.sh ls-config`. - ls-config|ls-dirs|ls-metrics|ls-vars|sync) - vale_args+=("$arg") - continue - ;; - esac - if [[ "$arg" == -* ]]; then - vale_args+=("$arg") - continue - fi - # Everything left is a lint target: `vale [options] [input...]` has no third - # kind of argument. See divergence 2 above for why a missing one is fatal here. - if [[ ! -e "$arg" ]]; then - echo "vale-wrap.sh: no such file or directory: $arg" >&2 - exit 2 - fi - # An absolute path inside the caller's cwd is relativized so the report cites - # a path that resolves against the real tree. Left absolute, it would be - # rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real - # path to a file that is deleted on exit, which reads as a bug in any report - # quoting it. Absolute paths outside the cwd have no relative form and keep - # the scratch-path behaviour documented above. - if [[ "$arg" == "$cwd"/* ]]; then - path_args+=("${arg#"$cwd"/}") - else - path_args+=("$arg") - fi -done - -if [[ "$config_given" == false ]]; then - vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini") -fi - -if [[ ${#path_args[@]} -eq 0 ]]; then - # Nothing to flatten. Hand off directly, with stdin closed so vale doesn't - # block waiting on a pipe that will never carry content. - exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null -fi - -# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing) -# is a GNU extension the BSD realpath on macOS doesn't have — and every dest -# below is a path that doesn't exist yet. python3 is already a hard dependency. -abspath() { - python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1" -} - -flatten() { - # Two call shapes: `flatten src dest` (dest already resolved and inside the - # scratch tree — the per-markdown-file calls in the directory branch below) - # writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file - # branch further down) additionally resolves `raw_dest` the way a separate - # `abspath` call used to, applies the same sandbox-escape guard, and prints - # the resolved path — folding two python3 spawns per file into one. - python3 - "$@" <<'PYTHON' -import os -import re -import sys - -src, dest_input = sys.argv[1], sys.argv[2] -tmpdir = sys.argv[3] if len(sys.argv) > 3 else None - -if tmpdir is None: - dest = dest_input -else: - dest = os.path.abspath(dest_input) - if not dest.startswith(tmpdir + os.sep): - print( - f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would " - f"land outside {tmpdir}", - file=sys.stderr, - ) - sys.exit(2) - os.makedirs(os.path.dirname(dest), exist_ok=True) - -# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument) -# a byte-for-byte round trip instead of aborting the whole run on a decode error. -with open(src, encoding='utf-8', errors='surrogateescape') as fh: - content = fh.read() - -# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\` is handled -# separately in unescape_double because it also swallows the next indentation. -DQ_ESCAPES = { - '0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n', - 'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"', - '/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028', - 'P': '\u2029', -} - -# First characters that make a plain (unquoted) scalar mean something other than -# text: YAML's c-indicator set. -PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`' - - -def unescape_double(text): - """Decode a double-quoted YAML scalar's body to the string YAML parses.""" - out = [] - i = 0 - while i < len(text): - char = text[i] - if char != '\\': - out.append(char) - i += 1 - continue - i += 1 - if i >= len(text): - break - esc = text[i] - if esc == '\n': - i += 1 - while i < len(text) and text[i] in ' \t': - i += 1 - continue - if esc in 'xuU': - width = {'x': 2, 'u': 4, 'U': 8}[esc] - digits = text[i + 1:i + 1 + width] - if len(digits) == width: - try: - out.append(chr(int(digits, 16))) - except ValueError: - pass - else: - i += 1 + width - continue - out.append(DQ_ESCAPES.get(esc, esc)) - i += 1 - return ''.join(out) - - -def close_quote(text, quote): - """Index of the closing `quote` in `text`, which starts just past the - opening one. None while the scalar is still unterminated.""" - i = 0 - while i < len(text): - char = text[i] - if quote == '"' and char == '\\': - i += 2 - continue - if char == quote: - if quote == "'" and text[i + 1:i + 2] == "'": - i += 2 - continue - return i - i += 1 - return None - - -def continuation_lines(rest): - """Yield the physical lines of `rest` that continue the value started on the - `description:` line. Indentation-based and blank-line-tolerant, per YAML: - a blank line (any amount of whitespace) always stays inside; the indent is - set by the first content line; the value ends at the first line indented - less than that, at any line flush with the key (that is the next mapping - key, not a continuation), or at EOF.""" - indent = None - for line in rest.splitlines(keepends=True): - text = line.rstrip('\n') - if text.strip() == '': - yield line - continue - line_indent = len(text) - len(text.lstrip(' \t')) - if line_indent == 0: - return - if indent is None: - indent = line_indent - elif line_indent < indent: - return - yield line - - -def emit(value): - """Render `value` as a YAML scalar whose source text spells the value out - verbatim. Vale locates the description by matching the parsed value back - against the source, so a scalar carrying any escape — `''` in a - single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the - whole `text.frontmatter.description` scope vanish, the same failure this - script exists to work around. Verbatim forms only, therefore, tried in - descending order of fidelity. The first three occupy one physical line; the - `|-` fallback occupies two, which the caller accounts for when padding.""" - if (value - and value[0] not in PLAIN_UNSAFE_FIRST - and ': ' not in value - and not value.endswith(':') - and ' #' not in value): - return value # plain: nothing needs escaping at all - if "'" not in value: - return "'" + value + "'" # single-quoted: only `'` would escape - if '"' not in value and '\\' not in value: - return '"' + value + '"' # double-quoted: only `"`/`\` would - # Last resort: the value needs quoting AND holds an apostrophe AND a double - # quote or backslash, so no *inline* scalar can carry it verbatim. A `|-` - # literal block can — a block scalar's body has no escape syntax at all, so - # `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches - # the description scope against it (the header above says the same of the - # `|` blocks this script deliberately leaves alone; verified against vale - # 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed - # value is exactly `value` with no trailing newline. - return '|-\n ' + value - - -fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL) -if fm_match: - fm = fm_match.group(2) - header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE) -else: - header_m = None - -if header_m: - head_start = header_m.start() - value_start = header_m.end() - header_end = fm.find('\n', value_start) - header_end = len(fm) if header_end == -1 else header_end - first = fm[value_start:header_end] - body_start = header_end + 1 - indicator = first.rstrip() - - block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator) - if block_m and block_m.group(1) == '|': - kind = None # literal blocks keep their line breaks; vale is fine - elif block_m: - kind = 'block' # folded (`>`): the value starts on the next line - elif indicator == '': - kind = 'block' # bare `description:`: a plain scalar on later lines - elif first[:1] == '"': - kind = 'double' - elif first[:1] == "'": - kind = 'single' - elif first[:1] in '#&*!': - kind = None # comment, anchor, alias or tag — not a plain scalar - else: - kind = 'plain' - - text = '' - value_end = value_start - value_lines = 0 - if kind in ('block', 'plain'): - body = ''.join(continuation_lines(fm[body_start:])) - value_end = body_start + len(body) - if kind == 'block': - text = body - value_lines = body.count('\n') - else: - text = fm[value_start:value_end] - value_lines = 1 + body.count('\n') - if ' #' in text or text.lstrip().startswith('#'): - # A `#` opens a comment inside a plain scalar. Folding it in - # would lint text YAML never treats as part of the value, so - # leave the file alone rather than lint the wrong string. - kind = None - elif kind in ('double', 'single'): - quote = '"' if kind == 'double' else "'" - inner_start = value_start + 1 - acc = fm[inner_start:body_start] - idx = close_quote(acc, quote) - lines = continuation_lines(fm[body_start:]) - while idx is None: - try: - acc += next(lines) - except StopIteration: - break - idx = close_quote(acc, quote) - if idx is None: - kind = None # unterminated quote: invalid YAML, leave it to vale - else: - inner = acc[:idx] - value_end = inner_start + idx + 1 - text = unescape_double(inner) if quote == '"' else inner.replace("''", "'") - value_lines = 1 + inner.count('\n') - - flat = re.sub(r'\s+', ' ', text).strip() - if kind and flat and value_lines >= 2: - # `value_end` can land mid-line, just past a closing quote, so extend to - # the end of that physical line and carry whatever follows (a trailing - # comment) across unchanged. - if value_end > 0 and fm[value_end - 1] == '\n': - span_end = value_end - trailer = '' - else: - newline = fm.find('\n', value_end) - span_end = len(fm) if newline == -1 else newline + 1 - trailer = fm[value_end:span_end].rstrip('\n') - scalar = emit(flat) - # A trailing comment carried across from the original line stays on the - # `description:` line itself: after a block scalar's `|-` header it is - # still a comment, but inside the block body it would become part of the - # value. - head, newline_sep, block_body = scalar.partition('\n') - # The replacement displaces the whole span, so the blank-line pad makes - # up the difference between the lines it displaced and the lines it - # occupies — every later line number is unchanged. That is one line for - # the three inline forms and two for the `|-` block; the span itself is - # at least two lines here (`value_lines >= 2` is a precondition), so the - # pad count never goes negative. - pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n')) - new_fm = (fm[:head_start] + 'description: ' + head + trailer - + newline_sep + block_body + '\n' + pad + fm[span_end:]) - content = (fm_match.group(1) + new_fm + fm_match.group(3) - + content[fm_match.end():]) - -with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh: - fh.write(content) - -if tmpdir is not None: - print(dest) -PYTHON -} - -tmpdir="$(cd "$(mktemp -d)" && pwd -P)" -trap 'rm -rf "$tmpdir"' EXIT - -# Mirror of the caller's cwd inside the scratch tree; relative path arguments -# are resolved from here. -mirror="$tmpdir$cwd" -mkdir -p "$mirror" - -argv_paths=() -for arg in ${path_args[@]+"${path_args[@]}"}; do - if [[ "$arg" == /* ]]; then - raw_dest="$tmpdir$arg" - else - raw_dest="$mirror/$arg" - fi - if [[ -d "$arg" ]]; then - dest="$(abspath "$raw_dest")" - # A path argument with enough leading `..` to climb past the mirror root would - # write outside the scratch dir. The real filesystem clamps such a path at - # `/`; the mirror can't, so refuse rather than scribble outside the sandbox. - case "$dest" in - "$tmpdir"/*) ;; - *) - echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2 - exit 2 - ;; - esac - mkdir -p "$(dirname "$dest")" - # A directory is mirrored whole — vale applies its own format filtering to - # the tree, so any file dropped here would be silently unlinted — and then - # every markdown file in the copy is flattened in place. `.git` is pruned: - # vale never lints it and copying it can dwarf the rest of the tree. - # `find -L` follows symlinks because vale does: it lints both a symlinked - # file and a file under a symlinked directory, and a bare `-type f` walk - # would report "0 files" where bare vale reports one. (A symlink loop makes - # `find` warn on stderr and carry on, which is also what vale does.) The - # second walk needs no `-L`: the mirror is all real files by construction. - mkdir -p "$dest" - while IFS= read -r -d '' rel; do - mkdir -p "$dest/$(dirname "$rel")" - cp "$arg/$rel" "$dest/$rel" - done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0) - while IFS= read -r -d '' md; do - flatten "$md" "$md" - done < <(find "$dest" -type f -name '*.md' -print0) - else - # `abspath` + `flatten` folded into one python3 process — see the comment - # atop `flatten` above. - dest="$(flatten "$arg" "$raw_dest" "$tmpdir")" - fi - if [[ "$arg" == /* ]]; then - argv_paths+=("$dest") - else - argv_paths+=("$arg") - fi -done - -cd "$mirror" -vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"} diff --git a/plugins/kyberforge/.apm/skills/skill-audit/tests/README.md b/plugins/kyberforge/.apm/skills/skill-audit/tests/README.md deleted file mode 100644 index 00f7db5..0000000 --- a/plugins/kyberforge/.apm/skills/skill-audit/tests/README.md +++ /dev/null @@ -1,29 +0,0 @@ -# tests/ - -Test files for scripts bundled with this skill. - -## Dependencies - -Tests require [bats-support](https://github.com/bats-core/bats-support) and -[bats-assert](https://github.com/bats-core/bats-assert). The test files load -helpers from the repo root's `tests/test_helper/`. - -From the repo root: - -```bash -git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support -git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert -``` - -Run all tests for this skill (from the repo root): - -```bash -bats plugins/kyberforge/.apm/skills/skill-audit/tests/ -``` - -## Files - -| File | Purpose | -|------|---------| -| `validate.bats` | Bats test suite for `scripts/validate.sh` | -| `validate-provenance.bats` | Bats test suite for `scripts/validate-provenance.sh` | diff --git a/plugins/kyberforge/.apm/skills/skill-author/SKILL.md b/plugins/kyberforge/.apm/skills/skill-author/SKILL.md index ccde99c..3564859 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/SKILL.md +++ b/plugins/kyberforge/.apm/skills/skill-author/SKILL.md @@ -3,7 +3,7 @@ name: skill-author description: > Use when the user wants to create a new skill from scratch, or apply audit findings, grill output, eval results, or inline feedback to an existing one. - Not read-only review -> `skill-audit`. Not agent files -> `agent-author`. + Not read-only review -> `factory-audit`. Not agent files -> `agent-author`. allowed-tools: Bash Read Write Edit metadata: version: "1.0.2" @@ -21,7 +21,7 @@ metadata: ## Gotchas - The word gates are two measurements, not two tiers of one rule: the 2,770-word / 500-line spec backstop counts the whole file, Step 3's gate the body alone. Never unify them. -- Never spawn a subagent to audit or recheck your own work — run `/skill-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft. +- Never spawn a subagent to audit or recheck your own work — run `/factory-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft. - Do not create new scripts unless a signal explicitly calls for it. Writing one from scratch requires out-of-scope transcript analysis — flag the opportunity as a suggestion instead. ## Step 1 — Dispatch @@ -32,7 +32,7 @@ metadata: | Directory exists, at least one improvement signal present | Improve | `references/improve.md` | | Directory exists, no signals | Stop and ask | — | -Signals: grill output, `/skill-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply. +Signals: grill output, `/factory-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply. Read only the reference matching the resolved flow — each is self-contained. If the target sits inside a git worktree, capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it. @@ -47,7 +47,7 @@ Decide before writing any description: model-invoked or hand-invoked? Before writing or editing a description, or restructuring a body, read `references/contract.md` — the banned-content list, boundary form, include/exclude rubric and body patterns. -Gates `/skill-audit` enforces in both flows: +Gates `/factory-audit` enforces in both flows: - **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not -> ` whose target resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only. - **Body** — decision procedure only: ordered steps, branches, gates, and which reference to load when. 600 words SUGGESTION, 900 FAIL, body only. At two or more mutually exclusive flows a dispatch table is mandatory and each flow gets its own self-contained `references/` file. @@ -55,7 +55,7 @@ Gates `/skill-audit` enforces in both flows: ## Step 4 — Validate and close -Run `/skill-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists. +Run `/factory-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists. Bump `metadata.version`: the **minor** version on create (new skills start at `0.1.0`) and the **patch** version on improve. diff --git a/plugins/kyberforge/.apm/skills/skill-author/references/contract.md b/plugins/kyberforge/.apm/skills/skill-author/references/contract.md index ec1d4c5..f4d03ea 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/references/contract.md +++ b/plugins/kyberforge/.apm/skills/skill-author/references/contract.md @@ -7,7 +7,7 @@ source_keys: # The description and body contract -House contract. Every rule here is enforced by `/skill-audit` — +House contract. Every rule here is enforced by `/factory-audit` — `scripts/validate.sh` for the counts and the boundary targets, the bundled Vale styles for the prose patterns, and its reference files for the judgment calls. @@ -146,7 +146,7 @@ row's target exists on disk; each row pairs exactly one target with a condition evaluate from the request, never a literal slash invocation; one line after the table names the matched file as the only one to read; and the gates every branch needs sit in the body, not inside one flow's file. That last one is the property the `git-commits` v0.1.2 failure turned on, and it is -the one a dispatch split is most likely to break. `skill-audit`'s `references/body-discipline.md` +the one a dispatch split is most likely to break. `factory-audit`'s `references/skill-body-discipline.md` carries the audit-side form of the same exemption; the two lists are the same four properties, and an edit to either belongs in both. diff --git a/plugins/kyberforge/.apm/skills/skill-author/references/create.md b/plugins/kyberforge/.apm/skills/skill-author/references/create.md index 202e50d..1a00348 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/references/create.md +++ b/plugins/kyberforge/.apm/skills/skill-author/references/create.md @@ -28,8 +28,8 @@ Before touching the filesystem, verify you have: If any are missing, stop and ask the user before proceeding. -**Requires `/skill-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in -the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the +**Requires `/factory-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in +the kyberforge plugin and are co-installed. If `/factory-audit` is unavailable, stop and ask the user to install the kyberforge plugin before continuing. ## Package-intent gate @@ -175,7 +175,7 @@ If a research `sources.md` is present in the conversation context: `- **Research doc:** ` where `` is the relative path from the repo root to the plugin-level research sources file this entry was drawn from (e.g. `plugins/myplugin/docs/research/docs//sources.md`). This field is required on every - entry — it makes the provenance chain explicit and is validated by `/skill-audit`. + entry — it makes the provenance chain explicit and is validated by `/factory-audit`. 4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of sources that informed it. 5. For each file in `references/` that was informed by research sources, add `source_keys` diff --git a/plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md b/plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md index ec22014..33884bd 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md +++ b/plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md @@ -44,7 +44,7 @@ Use `${CLAUDE_PLUGIN_ROOT}` only in hook commands and `.mcp.json` configs — no ## Standalone mode -Deployed directly to `~/.agents/skills//`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/skill-audit`) work if the called skill is also installed. +Deployed directly to `~/.agents/skills//`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/factory-audit`) work if the called skill is also installed. ## Cross-tool portability diff --git a/plugins/kyberforge/.apm/skills/skill-author/references/improve.md b/plugins/kyberforge/.apm/skills/skill-author/references/improve.md index 4df84d2..5d0d118 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/references/improve.md +++ b/plugins/kyberforge/.apm/skills/skill-author/references/improve.md @@ -16,12 +16,12 @@ Confirm the skill directory path exists and that at least one improvement signal conversation or a referenced file. If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies -existing signals to a skill. For a blind review without signals, use `/skill-audit` instead." +existing signals to a skill. For a blind review without signals, use `/factory-audit` instead." Signals can come from anywhere in the conversation or referenced files: - Grill session output (most common predecessor in the factory sequence) -- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list) +- `/factory-audit` findings (PASS/FAIL/SUGGESTION punch list) - Human feedback (feedback.json, inline in conversation, PR or issue comments) - Session context describing what went wrong diff --git a/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh b/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh index 2698f1d..1a39fe6 100755 --- a/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh +++ b/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh @@ -186,4 +186,4 @@ echo " 3. Add docs to references/ if needed (or delete the directory)" >&2 echo " 4. Add resources to assets/ if needed (or delete the directory)" >&2 echo " 5. Add tests to tests/ if the skill has scripts (or delete the directory)" >&2 echo " 6. Populate references/sources.md with research sources, or delete it" >&2 -echo " 7. Validate: run /skill-audit on $TARGET" >&2 +echo " 7. Validate: run /factory-audit on $TARGET" >&2 diff --git a/plugins/kyberforge/.apm/skills/skill-author/tests/new-skill.bats b/plugins/kyberforge/.apm/skills/skill-author/tests/new-skill.bats index 274c1f8..91d13d5 100644 --- a/plugins/kyberforge/.apm/skills/skill-author/tests/new-skill.bats +++ b/plugins/kyberforge/.apm/skills/skill-author/tests/new-skill.bats @@ -60,9 +60,9 @@ teardown() { assert [ -d "$DEST/my-tool-2" ] } -@test "next-steps output references /skill-audit not validate.sh" { +@test "next-steps output references /factory-audit not validate.sh" { run bash "$SCRIPT" my-tool "$DEST" - assert_output --partial "/skill-audit" + assert_output --partial "/factory-audit" refute_output --partial "validate.sh" } diff --git a/plugins/kyberforge/README.md b/plugins/kyberforge/README.md index 55858a0..f84fef9 100644 --- a/plugins/kyberforge/README.md +++ b/plugins/kyberforge/README.md @@ -41,9 +41,8 @@ Authoring source lives in `.apm/`; it is the only content source and the only th |---|---| | `forge` | Grill an unclassified "I want to add something" request, decide whether it's a skill, agent, plugin, or marketplace entry, then route to the matching author skill | | `skill-author` | Create or improve a skill from scratch, audit findings, or inline feedback | -| `skill-audit` | Audit a skill directory against the agentskills.io spec and produce a findings report | | `agent-author` | Author an agent definition file | -| `agent-audit` | Audit an agent definition across structure, provider safety, description and body quality, and provenance; produces a findings report | +| `factory-audit` | Audit a skill directory or an agent definition — structure, provider safety, description and body quality, and provenance; produces a findings report. Auto-detects which of the two it was handed (ADR-0025) | | `apm-install` | Install or upgrade the apm CLI and set up the agent runtimes it drives (Copilot CLI, Codex, Gemini, generic llm) | | `apm-workflow` | Author apm.yml, scaffold an apm package/marketplace, install dependencies, and compile/pack/publish/audit apm content | diff --git a/plugins/kyberforge/apm.yml b/plugins/kyberforge/apm.yml index f71b811..68e8902 100644 --- a/plugins/kyberforge/apm.yml +++ b/plugins/kyberforge/apm.yml @@ -1,5 +1,5 @@ name: kyberforge -version: 1.6.2 +version: 2.0.0 description: Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace. author: name: Defame1297 diff --git a/plugins/kyberforge/docs/README.md b/plugins/kyberforge/docs/README.md index 87da340..9cf2ea1 100644 --- a/plugins/kyberforge/docs/README.md +++ b/plugins/kyberforge/docs/README.md @@ -18,6 +18,6 @@ Upstream reference material gathered during skill authoring. Not shipped with th | `research/docs/claude-code-plugins/` | Claude Code plugin and marketplace manifests, plugin directory layout, agent definition format, `claude plugin validate` behaviour | | `research/docs/github-copilot-plugins/` | Copilot CLI plugin manifest and marketplace format, agent definition format, Copilot extensions and SDK | | `research/docs/microsoft-apm/` | apm CLI reference, `apm.yml` schema, primitive schemas (agent, prompt, instructions, hooks), monorepo repo shapes, marketplace/registries, packing and releasing | -| `research/examples/skill-write/` | Upstream skill examples reviewed when authoring skill-author and skill-audit | +| `research/examples/skill-write/` | Upstream skill examples reviewed when authoring `skill-author` and the audit skill, now `factory-audit` (ADR-0025) | AGENTS.md research moved to `plugins/core/docs/research/docs/agentsmd/` when the AGENTS.md skills landed in the `core` plugin (ADR-0012) — it is no longer part of kyberforge's provenance chain. diff --git a/scripts/check-apm-agents-valid.sh b/scripts/check-apm-agents-valid.sh index 3e6397f..5ef905f 100755 --- a/scripts/check-apm-agents-valid.sh +++ b/scripts/check-apm-agents-valid.sh @@ -1,8 +1,14 @@ #!/usr/bin/env bash set -euo pipefail -# Run agent-audit's validate.sh over every REAL plugin-scope agent file in this -# repo (plugins/*/.apm/agents/*.agent.md). +# Run factory-audit's validate.sh over every REAL plugin-scope agent file in +# this repo (plugins/*/.apm/agents/*.agent.md). +# +# Since ADR-0025 merged skill-audit and agent-audit into factory-audit, that +# validate.sh is a single auto-detecting entry point: it reads the target and +# sources the skill-mode or agent-mode check library itself. The invocation +# below is therefore unchanged apart from the path — no mode flag is passed, +# and passing one would be wrong. # # Why this exists: validate.sh was previously exercised only by # scripts/check-scope-walkup-sync.sh, and only against synthetic fixtures built @@ -31,14 +37,14 @@ if [[ ! -d "$REPO_ROOT" ]]; then fi REPO_ROOT="$(cd "$REPO_ROOT" && pwd)" -VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" +VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" # The validator's own absence is a hard failure, never a skip. If validate.sh # moves or is deleted, every assertion below evaporates and the hook would # otherwise exit 0 having validated nothing -- indistinguishable, from # pre-commit's silent-on-pass output, from a run where all four agents passed. if [[ ! -f "$VALIDATE" ]]; then - echo "APM agent validation failed: validator not found at $VALIDATE — this script's path has gone stale, so no agent file was checked. Update it to wherever agent-audit's validate.sh now lives." >&2 + echo "APM agent validation failed: validator not found at $VALIDATE — this script's path has gone stale, so no agent file was checked. Update it to wherever factory-audit's validate.sh now lives." >&2 exit 1 fi @@ -47,7 +53,7 @@ fi # reads like a validation failure rather than a missing dependency. Fail closed, # but say which it is. if ! command -v python3 >/dev/null 2>&1; then - echo "APM agent validation failed: python3 not found on PATH — agent-audit's validate.sh is a python3 program and cannot run. Install python3; this gate does not degrade to a pass." >&2 + echo "APM agent validation failed: python3 not found on PATH — factory-audit's validate.sh is a python3 program and cannot run. Install python3; this gate does not degrade to a pass." >&2 exit 1 fi @@ -147,15 +153,15 @@ for f in ${AGENT_FILES[@]+"${AGENT_FILES[@]}"}; do if [[ -n "$out" ]]; then printf '%s\n' "$out" | sed 's/^/ /' >&2 else - echo " (validate.sh produced no output — see its exit code above; 2 means script error, e.g. a missing references/field-inventory.md)" >&2 + echo " (validate.sh produced no output — see its exit code above; 2 means script error, e.g. a missing references/agent-field-inventory.md or lib-*.sh)" >&2 fi fi done if [[ "$FAIL" -ne 0 ]]; then echo "" >&2 - echo "APM agent validation failed: ${#FAILED_FILES[@]} of ${#AGENT_FILES[@]} agent file(s) did not pass agent-audit's validate.sh." >&2 + echo "APM agent validation failed: ${#FAILED_FILES[@]} of ${#AGENT_FILES[@]} agent file(s) did not pass factory-audit's validate.sh." >&2 exit 1 fi -echo "APM agent validation passed: ${#AGENT_FILES[@]} plugin-scope agent file(s) validated against agent-audit's validate.sh." +echo "APM agent validation passed: ${#AGENT_FILES[@]} plugin-scope agent file(s) validated against factory-audit's validate.sh." diff --git a/scripts/check-scope-walkup-sync.sh b/scripts/check-scope-walkup-sync.sh index c1502d5..f8fde49 100755 --- a/scripts/check-scope-walkup-sync.sh +++ b/scripts/check-scope-walkup-sync.sh @@ -5,19 +5,35 @@ set -euo pipefail # ports of "walk up from a directory looking for a scope-defining marker" living # in this repo: # -# - plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh (Python: detect_scope) -# - plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh (Python: find_plugin_root) -# - plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh (Bash: find_package_root) -# - plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh (Bash: find_package_root) +# - plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh (Python: detect_scope) +# - plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh (Python: find_plugin_root) +# - plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh (Bash: find_package_root) +# - plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh (Bash: find_package_root) # -# Per ADR-0014's no-cross-skill-path rule, these can't be consolidated into a -# shared file (each skill's cache-install copies only its own files), so unlike -# check-vale-style-sync.sh (which diffs literal file copies) this can't be a -# text diff — the four implementations are hand-ported, not copied. Instead -# this builds a matrix of fixture directory trees and asserts the *observable -# behavior* agrees: whatever new-agent.sh/new-skill.sh actually create on disk, -# validate.sh/validate-provenance.sh must classify the same way when pointed at -# the result. Run from repo root or pass REPO_ROOT as arg. +# THIS GATE SURVIVED ADR-0025 AND ITS JOB DID NOT SHRINK. The merge of +# skill-audit and agent-audit into factory-audit moved the first two ports into +# one directory; it did not merge the ports. Two of the four still live in +# agent-author and skill-author, which ADR-0020 deliberately left unmerged, so +# the walk-up still has four independent implementations across three skill +# directories and this is still the only thing comparing them. +# +# It also still cannot become a text diff, and the reason is not the one that +# retired check-vale-style-sync.sh alongside the merge. That gate diffed two +# literal copies of the same file, so collapsing them to one copy left it +# nothing to compare. These four are not copies of anything: they are hand- +# ported reimplementations of the same walk-up in two different languages — +# Python in factory-audit's two validators, Bash in the two authors' scaffold +# scripts. There is no byte sequence common to a Python function and a Bash +# function that agreeing on behavior would preserve. Sourcing cannot close the +# gap either: per ADR-0014's no-cross-skill-path rule a cache-installed skill +# copies only its own directory, which is exactly why factory-audit's two +# validators CAN now source a shared library from their own scripts/ while the +# author skills across the boundary still cannot. +# +# So this asserts behavioral agreement instead: it builds a matrix of fixture +# directory trees and checks that whatever new-agent.sh/new-skill.sh actually +# create on disk, validate.sh/validate-provenance.sh classify the same way when +# pointed at the result. Run from repo root or pass REPO_ROOT as arg. REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}" if [[ ! -d "$REPO_ROOT" ]]; then @@ -28,8 +44,8 @@ REPO_ROOT="$(cd "$REPO_ROOT" && pwd)" NEW_AGENT="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh" NEW_SKILL="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh" -VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" -VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh" +VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" +VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh" # Floor on the four hardcoded `plugins/kyberforge/.apm/...` paths above. A # missing target is only a legitimate no-op for a repo that has no kyberforge @@ -41,10 +57,10 @@ VALIDATE_PROVENANCE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scrip for f in "$NEW_AGENT" "$NEW_SKILL" "$VALIDATE" "$VALIDATE_PROVENANCE"; do if [[ ! -f "$f" ]]; then if [[ -d "$REPO_ROOT/plugins/kyberforge" ]]; then - echo "Scope walk-up sync check failed: $REPO_ROOT/plugins/kyberforge exists but $f does not — this script's .apm/ paths have gone stale, so none of the walk-up fixtures ran. Update them to wherever the agent-author/agent-audit/skill-author scripts now live." >&2 + echo "Scope walk-up sync check failed: $REPO_ROOT/plugins/kyberforge exists but $f does not — this script's .apm/ paths have gone stale, so none of the walk-up fixtures ran. Update them to wherever the factory-audit/agent-author/skill-author scripts now live." >&2 exit 1 fi - echo "Scope walk-up sync check: $f not found — kyberforge agent-author/agent-audit/skill-author skills not present, nothing to check." >&2 + echo "Scope walk-up sync check: $f not found — kyberforge factory-audit/agent-author/skill-author skills not present, nothing to check." >&2 exit 0 fi done diff --git a/scripts/check-vale-style-sync.sh b/scripts/check-vale-style-sync.sh deleted file mode 100755 index 0ce4d3d..0000000 --- a/scripts/check-vale-style-sync.sh +++ /dev/null @@ -1,413 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Kyberforge's Vale prefilter is duplicated into skill-audit and agent-audit's own -# scripts/assets (per plugins/kyberforge/.apm/skills/skill-author/references/deployment-modes.md's -# no-cross-skill-path rule: a plugin's cache-install copy only includes each skill's own files). -# agent-audit's copy is canonical — it's the superset (Kyberforge + KyberforgeCopilot) that the -# repo root's own pre-commit hook and .pre-commit-hooks.yaml both consume. This fails the build -# if skill-audit's copy has drifted from it, since nothing else would catch a rule fix landing in -# only one of the two. Run from repo root or pass REPO_ROOT as arg. - -REPO_ROOT="${1:-$(git rev-parse --show-toplevel 2>/dev/null || pwd)}" -# A nonexistent REPO_ROOT must fail loudly, not fall through to the "neither -# copy present" no-op below — that guard exists for a repo that legitimately -# has no kyberforge plugin installed, not for a typo'd or stale path, and a -# clean exit 0 here would read as "checked, in sync" when nothing ran at all. -if [[ ! -d "$REPO_ROOT" ]]; then - echo "Vale style sync check failed: REPO_ROOT '$REPO_ROOT' is not a directory." >&2 - exit 1 -fi -# Absolutized because the glob probe below `cd`s into a scratch tree, where a -# relative --config path would stop resolving. -REPO_ROOT="$(cd "$REPO_ROOT" && pwd)" -FAIL=0 - -err() { echo " FAIL: $1" >&2; FAIL=$((FAIL + 1)); } - -SKILL_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit" -AGENT_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit" - -# Floor on the hardcoded `plugins/kyberforge/.apm/...` paths above. Neither copy -# present is only a legitimate no-op for a repo that has no kyberforge plugin at -# all. If `plugins/kyberforge/` IS here and the `.apm/` targets under it are not, -# the paths in this script have gone stale — a rename or relocation of `.apm/` -# would otherwise turn every assertion below into a silent exit 0, which reads as -# "checked, in sync" exactly like the REPO_ROOT case above. That matters most for -# the change that introduced these paths: a path rewrite is precisely the edit -# this would survive unnoticed. -if [[ ! -d "$SKILL_AUDIT" && ! -d "$AGENT_AUDIT" ]]; then - if [[ -d "$REPO_ROOT/plugins/kyberforge" ]]; then - echo "Vale style sync check failed: $REPO_ROOT/plugins/kyberforge exists but neither $SKILL_AUDIT nor $AGENT_AUDIT does — this script's .apm/ paths have gone stale, so nothing was checked. Update them to wherever the audit skills now live." >&2 - exit 1 - fi - exit 0 -fi - -# Exactly one present is drift, not absence: the missing copy can't be in sync -# with the surviving one, and treating it as a no-op is how a deleted or -# renamed copy would slip through silently. -if [[ ! -d "$SKILL_AUDIT" ]]; then - echo "Vale style sync check failed: $AGENT_AUDIT exists but $SKILL_AUDIT does not — run scripts/sync-vale-styles.sh to regenerate skill-audit's copy." >&2 - exit 1 -fi -if [[ ! -d "$AGENT_AUDIT" ]]; then - echo "Vale style sync check failed: $SKILL_AUDIT exists but $AGENT_AUDIT does not — agent-audit holds the canonical copy, so restore it before syncing." >&2 - exit 1 -fi - -if ! diff -q "$SKILL_AUDIT/scripts/vale-wrap.sh" "$AGENT_AUDIT/scripts/vale-wrap.sh" >/dev/null 2>&1; then - err "scripts/vale-wrap.sh differs between skill-audit and agent-audit" -fi - -if ! diff -rq "$SKILL_AUDIT/assets/vale/styles/Kyberforge" "$AGENT_AUDIT/assets/vale/styles/Kyberforge" >/dev/null 2>&1; then - err "assets/vale/styles/Kyberforge differs between skill-audit and agent-audit" -fi - -# --- .vale.ini coverage ------------------------------------------------------ -# The two .vale.ini files are deliberately NOT identical — agent-audit's carries -# an extra [**/*.agent.md] section and the KyberforgeCopilot style — so they -# cannot be diffed like the styles above. Nothing else in the repo read them at -# all, and that is what let a one-character glob typo silently disable the -# prefilter for a whole file type: the hook still MATCHES the file via its -# `files:` regex, so pre-commit reports neither `Skipped` nor an error; vale -# lints zero files, prints `0 errors ... in 1 file` and exits 0, and the hook -# shows `Passed`. So check the parts that must hold in both, not equality. - -SKILL_INI="$SKILL_AUDIT/assets/vale/.vale.ini" -AGENT_INI="$AGENT_AUDIT/assets/vale/.vale.ini" - -# Counted, not assumed. The summary line at the bottom used to hardcode `2 -# .vale.ini file(s) checked` in both branches. That was true on any clean run -- -# a missing or unreadable file errs below and the script never reaches the -# summary -- but the line's whole purpose is to say what this run actually -# inspected, and a constant says what the author expected. Nothing asserted it, -# so it would have survived becoming false. -INIS_CHECKED=0 -for ini in "$SKILL_INI" "$AGENT_INI"; do - rel_ini="${ini#"$REPO_ROOT"/}" - # `-e`, not `-f`: a path that exists but is not a readable regular file (a - # directory sitting where the file should be, say) is not "missing", and - # reporting it as missing sends you looking for a deleted file. It belongs to - # the unreadable case below, which is the one that describes what actually - # went wrong. - if [[ ! -e "$ini" ]]; then - err "$rel_ini is missing — without it vale falls back to an upward config search and lints with whatever it finds" - continue - fi - # Present but unreadable is its own case: every assertion below is a grep, and - # grep exits 2 on a read error. The override capture swallows that into an - # empty result, which would read as "no findings" rather than "not checked", - # and the two greps above it report "has no StylesPath"/"names no Kyberforge" - # for a file that may well have both — a misdiagnosis, not a missed one. - # - # Decided by ACTUALLY READING the file, not by `[[ -r ]]`. `-r` is access(2), - # which answers "would the permission bits allow it" — and for uid 0 that is - # yes even on a mode-000 file (verified). This hook runs at pre-push, and this - # repo's dev environment is root, so an `[[ ! -r ]]` guard could never fire in - # the one place it exists to fire: it was untestable because it was dead. A - # read attempt is also the stricter question, catching EISDIR and EIO, which - # access(2) reports on neither. `cat`, not a bare `< "$ini"` redirect: opening - # a directory for reading succeeds, only the read fails. - if ! cat "$ini" >/dev/null 2>&1; then - err "$rel_ini exists but could not be read — none of its assertions could run, and an unreadable file cannot be distinguished from a clean one downstream" - continue - fi - # Counted here, past both `continue`s: the file exists and its bytes were - # readable, so every assertion below it really does run against it. - INIS_CHECKED=$((INIS_CHECKED + 1)) - # StylesPath is resolved relative to the .vale.ini, which is the only reason - # the bundled styles are found from a consuming repo's clone prefix. - if ! grep -Eq '^[[:space:]]*StylesPath[[:space:]]*=[[:space:]]*styles[[:space:]]*$' "$ini"; then - err "$rel_ini has no 'StylesPath = styles' — the bundled styles/ directory would not be found" - fi - # Matches `Kyberforge` as a whole name, so `KyberforgeCopilot` alone does not - # satisfy it. Avoids \b, which is a GNU grep extension. - if ! grep -Eq '^[[:space:]]*BasedOnStyles[[:space:]]*=.*Kyberforge([[:space:],]|$)' "$ini"; then - err "$rel_ini has no section whose BasedOnStyles names Kyberforge — every rule the audit prefilters on lives in that style" - fi - # Per-rule overrides are the third way to retire a rule without touching a - # style file or a glob. Per ADR-0013, every rule is `level: error` and every - # alert is a FAIL — there is no ignorable tier. Vale's exit - # code keys on `error` alerts alone, so any override that leaves a rule at - # anything other than `error` still lints the file, still exits 0, and still - # shows `Passed` in pre-commit. The glob probe below cannot backstop this: it - # keys on one `Kyberforge.VagueWording` alert, so DescriptionOpener, - # PaddingPhrase, SentenceOpenerThereIs and ProactivePhrase can each be retired - # underneath a passing probe. - # - # Asserted as an ALLOWLIST, not a blocklist of `NO|warning|suggestion`, because - # that is vale 3.15.2's own semantic: only the exact tokens `YES` and `error` - # keep a rule blocking. `warning`/`suggestion` downgrade it (alert still - # printed, exit 0 — invisible, since pre-commit swallows a passing hook's - # output); every other value — `NO`, `false`, `0`, `off`, `n`, empty, - # `garbage`, and lowercase `yes`, `true`, `1`, `on` — silences the rule - # outright. Lowercase `yes` is the trap a blocklist cannot cover: it reads as - # "enabled" to a human and disables the rule. Verified by enumerating the - # value space against vale 3.15.2. - # - # The allowlist demands a BARE `YES`/`error` with nothing after it, which also - # rejects `error # note` and `error ; note`. Vale itself strips those — a - # whitespace-preceded `#` or `;` comment is removed and the rule stays live — - # so rejecting them is deliberately stricter than vale, not a workaround for - # it. Uniformity is worth more here than the ability to annotate a line that - # should not exist: no shipped `.vale.ini` has any override line at all, and - # the failure mode is a loud false positive rather than a silent pass. The - # genuine hazard is the no-space form — `error# note` and `error; note` are - # NOT stripped and silence the rule outright — and a rule that demands a bare - # token catches those without having to reimplement vale's comment parsing. - # - # `[A-Za-z0-9_-]` on both halves of the name, not `[A-Za-z]`: a rule named - # `Kyberforge.Vague2` is genuinely silenced by `= NO` (verified: 1 error -> - # 0 errors), so an alpha-only class would let a digit-bearing rule name slip - # past the gate. All five current rule names are pure alpha, so this is - # forward cover, not a live hole. - bad_overrides="$( - grep -E '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=' "$ini" \ - | grep -Ev '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=[[:space:]]*(YES|error)[[:space:]]*$' \ - || true - )" - # No `grep -q` in that pipeline on purpose: `-q` exits on its first match, and - # under `set -o pipefail` the resulting SIGPIPE on the upstream grep would make - # the whole pipeline report 141 and read as "no findings". - if [[ -n "$bad_overrides" ]]; then - err "$rel_ini overrides a Kyberforge rule to something other than a bare YES or error (first: '${bad_overrides%%$'\n'*}') — every rule in this prefilter is level: error and every alert is a FAIL, and any other value downgrades or silences the rule while vale still exits 0. A trailing comment is rejected too: vale strips a spaced '# ...' but not 'error# ...', so this asks for the bare token rather than guessing which form you meant" - fi -done - -# KyberforgeCopilot is agent-audit's alone — ADR-0013 scopes it to `.agent.md` -# files only, for the Copilot-only 'Use proactively has no effect' check, and -# records that it must not be extended to `.md` files. The loop above -# deliberately asserts only `Kyberforge`, since -# skill-audit's copy legitimately has no Copilot style, so dropping -# `, KyberforgeCopilot` from agent-audit's `[**/*.agent.md]` section unloaded the -# whole style silently: no glob broke, the styles/ diff above stayed clean (the -# style directory is still shipped, just never loaded), the two .vale.ini files -# are deliberately unequal so no equality check applies, and the probe below -# still passed because it keys on a Kyberforge alert. Assert the style is loaded -# whenever it is shipped. -if [[ -d "$AGENT_AUDIT/assets/vale/styles/KyberforgeCopilot" && -f "$AGENT_INI" ]]; then - if ! grep -Eq '^[[:space:]]*BasedOnStyles[[:space:]]*=.*KyberforgeCopilot([[:space:],]|$)' "$AGENT_INI"; then - err "${AGENT_INI#"$REPO_ROOT"/} ships a styles/KyberforgeCopilot style but no section's BasedOnStyles names it — the style is never loaded, so its Copilot-only rules lint nothing" - fi -fi - -# Prints the `files:` regex of every hook, in ONE manifest ($2), whose entry -# is $1's vale-wrap.sh. Records are delimited by their `- id:` line, so the -# check does not depend on `entry:` preceding `files:` within a record. -# -# Deliberately kept per-manifest rather than unioned across both files: the -# validation loop below needs to know whether a probe path is in scope of -# .pre-commit-hooks.yaml (the canonical, external-facing manifest) and -# .pre-commit-config.yaml (this repo's own dev-time copy of the same hook) -# *independently*. A union here previously let a probe that matched only the -# older, looser .pre-commit-hooks.yaml pattern read as "in scope" even after -# .pre-commit-config.yaml's copy of the same hook had been narrowed away from -# it — silently masking exactly the kind of hook-rescoping drift this script -# exists to catch. -# -# Deliberately NOT memoized. The probe loop below calls this 12 times over the -# same two small manifests, which measures at 14ms against a ~870ms run (the six -# vale invocations are the wall clock). A previous memoization attempt was inert -# anyway: every call site is `x="$(hook_file_regexes ...)"`, a command -# substitution, so the cache writes landed in a subshell and the lookup never -# hit. Re-parsing is the honest, working version of a saving too small to buy. -hook_file_regexes() { - local skill="$1" manifest="$2" raw - if [[ -f "$manifest" ]]; then - awk -v skill="$skill" ' - function flush() { - if (entry ~ skill "/scripts/vale-wrap.sh" && files != "") print files - entry = ""; files = "" - } - /^[ \t]*-[ \t]*id:/ { flush() } - /^[ \t]*entry:/ { entry = $0 } - /^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) } - END { flush() } - ' "$manifest" | while IFS= read -r raw; do - # Strip the surrounding YAML quotes; the regex itself never carries them. - raw="${raw%\'}"; raw="${raw#\'}" - raw="${raw%\"}"; raw="${raw#\"}" - printf '%s\n' "$raw" - done - fi -} - -# True if $1 matches at least one newline-delimited regex in $2. -matches_any_regex() { - local rel="$1" regexes="$2" re - [[ -n "$regexes" ]] || return 1 - while IFS= read -r re; do - [[ -n "$re" ]] || continue - if printf '%s\n' "$rel" | grep -Eq "$re"; then - return 0 - fi - done < "$tmp/$rel" - out="$(cd "$tmp" && vale --config "$cfg" "$rel" 2>&1)" || rc=$? - rm -rf "$tmp" - if printf '%s\n' "$out" | grep -qF "Kyberforge.VagueWording"; then - VALE_PROBE_DIAG="" - return 0 - fi - # vale exits nonzero merely for *having* alerts, so rc alone proves nothing -- - # it is evidence only alongside the absent alert. - VALE_PROBE_DIAG="vale exited $rc; output: ${out:-}" - return 1 -} - -# Missing vale is a HARD FAILURE, not a warning. Six of this script's assertions -# — one glob probe per path below — are `vale --config` invocations, and they are -# the only ones that catch the defect the whole `.vale.ini coverage` section was -# written for: the one-character glob typo (`[**/SKILL.md]` -> `[**/SKILLS.md]`) -# that leaves every text-level assertion clean while vale lints zero files. As a -# warning this self-disabled on exactly that mutation and exited 0, and since -# pre-commit swallows a passing hook's output the stderr line was never seen — -# the pre-push hook reported `Passed`. That is the same "clean exit 0 reads as -# 'checked, in sync' when nothing ran" failure the REPO_ROOT guard at the top of -# this file already refuses to allow. -# -# The opt-out exists for a machine that genuinely cannot install vale, and it is -# an env var that has to be set on purpose — never mere absence of the binary. -# Setting it downgrades the run to text-level assertions only and says so. -VALE_AVAILABLE=true -if ! command -v vale >/dev/null 2>&1; then - VALE_AVAILABLE=false - if [[ "${CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE:-}" == "1" ]]; then - echo " WARNING: vale is not installed and CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 — .vale.ini glob coverage was NOT verified, only the text-level assertions ran. A clean result here does not mean the globs cover what their hooks lint." >&2 - else - err "vale is not installed, so none of the .vale.ini glob-coverage probes ran — a glob typo that silently lints zero files is invisible without them. Install it (https://vale.sh/docs/vale-cli/installation/), or set CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 to accept a text-only run" - fi -fi - -# One representative path per file shape the prefilter is supposed to cover, -# tagged with whether that shape is expected to be in scope of BOTH manifests -# ("shared") or only the external-facing .pre-commit-hooks.yaml ("hooks-only" -# — e.g. a Copilot .agent.md file living outside this repo's own plugins/.apm/ -# layout, which .pre-commit-config.yaml's repo-scoped regex has no reason to -# cover). Each probe is checked against the two manifests' `files:` regexes -# *separately*, not unioned: a path that goes stale because a hook was -# rescoped fails loudly here instead of quietly probing a shape nothing lints -# any more, and a "shared" path the two manifests disagree on fails loudly -# too — that disagreement is exactly how .pre-commit-config.yaml's regex can -# narrow out of sync with .pre-commit-hooks.yaml's without either manifest's -# own hook breaking (each still matches real files on its own), so nothing -# else would catch it. -PROBES_CHECKED=0 -while IFS='|' read -r skill rel scope; do - [[ -n "$skill" ]] || continue - dir="$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill" - ini="$dir/assets/vale/.vale.ini" - [[ -f "$ini" ]] || continue - PROBES_CHECKED=$((PROBES_CHECKED + 1)) - - hooks_regexes="$(hook_file_regexes "$skill" "$REPO_ROOT/.pre-commit-hooks.yaml")" - config_regexes="$(hook_file_regexes "$skill" "$REPO_ROOT/.pre-commit-config.yaml")" - in_hooks=false - matches_any_regex "$rel" "$hooks_regexes" && in_hooks=true - in_config=false - matches_any_regex "$rel" "$config_regexes" && in_config=true - - if [[ "$in_hooks" == false && "$in_config" == false ]]; then - err "$rel matches no 'files:' regex of any $skill hook — the probe path is stale, or the hook was rescoped away from a shape it still needs to lint" - elif [[ "$scope" == "shared" && "$in_hooks" != "$in_config" ]]; then - err "$rel is in scope of $skill's hook in .pre-commit-hooks.yaml but not .pre-commit-config.yaml (or vice versa: hooks=$in_hooks, config=$in_config) — the local and canonical 'files:' regexes have drifted out of sync for this hook" - fi - - if [[ "$VALE_AVAILABLE" == true ]] && ! vale_flags_path "$ini" "$rel"; then - err "$skill/assets/vale/.vale.ini raises no Kyberforge alert on $rel — its glob sections do not cover a path its own pre-commit hook is scoped to, so the hook passes that shape without linting it [$VALE_PROBE_DIAG]" - fi -# `demo.md` (bare, no `.agent.md` suffix) is `hooks-only` rather than -# `shared`: it exists only to exercise agent-audit's `[**/agents/*.md]` glob -# section in isolation from `[**/*.agent.md]` (test-check-vale-style-sync.sh's -# case 10), not because any real file under `.apm/agents/` still has that -# shape — per ADR-0016 every `.apm/agents/*` file is named `*.agent.md`, so -# `.pre-commit-config.yaml`'s regex correctly no longer matches it and that's -# not drift. `demo.agent.md` is the real, current shape and is `shared`. -# -# The two `.claude/`-prefixed probes carry the location-independence property: a -# `SKILL.md` outside `plugins/` (e.g. project-scope -# `.claude/skills/foo/SKILL.md`) still matches `[**/SKILL.md]` and gets linted -# normally — the globs constrain filename shape, not location. Every other -# probe here starts with `plugins/`, so narrowing a glob to a `plugins/`-shaped -# path (`[**/SKILL.md]` -> `[**/.apm/skills/*/SKILL.md]`) left all of them -# matching while the project-scope shape started linting as `0 errors ... in 0 -# files` — the exact "vale lints zero files, hook shows Passed" failure the -# comment at the top of this section describes. Both are `hooks-only`: only -# `.pre-commit-hooks.yaml` is layout-agnostic, and `.pre-commit-config.yaml` -# pinning this repo's own `plugins/**/.apm/` layout is by design, not drift. -done <<'EOF_PROBE' -skill-audit|plugins/demo/.apm/skills/demo/SKILL.md|shared -skill-audit|.claude/skills/demo/SKILL.md|hooks-only -agent-audit|plugins/demo/.apm/agents/demo.md|hooks-only -agent-audit|plugins/demo/.apm/agents/demo.agent.md|shared -agent-audit|.claude/agents/demo.md|hooks-only -agent-audit|copilot/demo.agent.md|hooks-only -EOF_PROBE - -# Second floor, on the probe TABLE rather than on the directory paths. Every row -# `continue`s when the `.vale.ini` of the skill its first column names is absent, -# so the table can verify nothing while FAIL stays 0. Two states do that, and no -# other assertion in this file sees either: -# -# * the `EOF_PROBE` heredoc gutted — a bad merge, a truncated edit, or a -# wholesale delete of the rows. The loop body never runs at all. -# * every row's skill column drifting away from the directory names on disk -# (`skill-audit|` -> `skill-auditX|`), which is what a skill rename plus a -# half-applied find/replace leaves behind. -# -# Both give a clean exit 0 from a section that checked nothing, which is why the -# guard is worth having. What it is NOT reachable by is a relocation of -# `assets/vale/`: PROBES_CHECKED only reaches 0 that way if BOTH `.vale.ini` -# files are gone, and the loop at the top of the `.vale.ini coverage` section -# errs on each of them first, so that state is already FAIL >= 2 and this guard -# is never the cause. The message therefore names the table, not the files — -# describing it as "every probe skill's .vale.ini is missing" misdiagnosed the -# one thing that can actually trigger it. -if [[ $PROBES_CHECKED -eq 0 ]]; then - err "no probe path was checked — the probe table is empty, or no row's first column names a skill directory under plugins/kyberforge/.apm/skills/ that has an assets/vale/.vale.ini, so the glob-coverage section verified nothing at all" -fi - -if [[ $FAIL -gt 0 ]]; then - echo "Vale style sync check failed: $FAIL error(s). For a drifted wrapper or style, agent-audit's copy is canonical — run scripts/sync-vale-styles.sh to regenerate skill-audit's copy, then commit both. A .vale.ini finding is not drift and sync-vale-styles.sh will not fix it: edit that file's own StylesPath, BasedOnStyles or glob sections." >&2 - exit 1 -fi - -# A clean run says what it actually inspected. Silence is what let the vacuous -# passes above look identical to real ones, and it is what made "did this script -# do any work against the real repo?" untestable from outside — the counts below -# are what tests/test-check-vale-style-sync.sh asserts a non-zero floor on. -if [[ "$VALE_AVAILABLE" == true ]]; then - echo "Vale style sync check passed: $INIS_CHECKED .vale.ini file(s) checked, $PROBES_CHECKED glob probe(s) verified with vale." -else - echo "Vale style sync check passed (text-level only, vale unavailable): $INIS_CHECKED .vale.ini file(s) checked, 0 glob probe(s) verified." -fi diff --git a/scripts/skill-size-check.sh b/scripts/skill-size-check.sh index 324cf78..24f1271 100755 --- a/scripts/skill-size-check.sh +++ b/scripts/skill-size-check.sh @@ -25,14 +25,14 @@ set -euo pipefail # # Both spec ceilings are inclusive: a file at exactly MAX_LINES or MAX_WORDS # passes, and only one past it fails. That matches -# skill-audit/scripts/validate.sh, which has always used `line_count <= 500` as +# factory-audit/scripts/lib-checks-skill.sh, which has always used `line_count <= 500` as # its pass condition — the two previously disagreed at exactly 500 lines, so a # SKILL.md could pass its own audit and still be blocked by the commit hook. # The ADR-0020 ceilings are inclusive the same way. # # Token counts aren't computed exactly here — a whitespace word count is used # as a proxy (Python's str.split(), the same primitive -# skill-audit/scripts/validate.sh applies to these two constants; `wc -w` +# factory-audit/scripts/lib-checks-skill.sh applies to these two constants; `wc -w` # disagrees with it on Unicode separators, which is why the awk pass that used # to live in the loop below is gone). # @@ -81,20 +81,21 @@ set -euo pipefail # description differently is worse than one reader that refuses to start. # These constants are intentionally duplicated in -# skill-audit/scripts/validate.sh (Python) rather than shared from one file: -# this script is a standalone bash pre-commit hook, that one is an in-skill -# Python validator invoked in a different context (same rationale as -# vale-wrap.sh's per-plugin duplication — see its own header comment). +# factory-audit/scripts/lib-checks-skill.sh (Python) rather than shared from one +# file: this script is a standalone bash pre-commit hook, that one is an +# in-skill Python check library invoked in a different context. # tests/test-skill-size-check.sh asserts both files agree on these values, so # drift between them fails CI rather than silently diverging. # # The ADR-0020 constants below are duplicated the same way and carry the same -# warning: skill-audit/scripts/validate.sh holds a second copy of +# warning: factory-audit/scripts/lib-checks-skill.sh holds a second copy of # DESC_SUGGEST_CHARS / DESC_MAX_CHARS / BODY_SUGGEST_WORDS / BODY_MAX_WORDS, -# and agent-audit/scripts/validate.sh holds a third copy of the two +# and factory-audit/scripts/lib-checks-agent.sh holds a third copy of the two # description constants (agents take the description gates and, per ADR-0020, -# deliberately take NO body word gate). If they drift, this audit reports a -# skill ready to ship that the commit hook then rejects. +# deliberately take NO body word gate). ADR-0025's merge put those two libraries +# in one directory but did NOT collapse the copies — the two flows gate +# different spans — so the drift risk is unchanged: if they diverge, the audit +# reports a skill ready to ship that the commit hook then rejects. MAX_LINES=500 MAX_WORDS=2770 @@ -146,7 +147,7 @@ fi # them as 0, and both ceilings passed in total silence — the one outcome # this script forbids itself. # * awk's NR/NF do not agree with the Python splitlines()/split() that -# skill-audit/scripts/validate.sh uses for the SAME two constants. +# factory-audit/scripts/lib-checks-skill.sh uses for the SAME two constants. # splitlines() also breaks on \x0b \x0c \x1c \x1d \x1e \x85 U+2028 U+2029 # and split() on every Unicode space, so a body padded with U+2028 read as # 6 lines here and 606 lines there — hook green, audit FAIL. @@ -225,16 +226,20 @@ def info(msg): print("INFO: %s" % msg) +# Anything inside the BEGIN/END markers below is hashed byte-for-byte against +# the plugin copy by tests/test-adr0020-contract.sh. Never edit the marked span +# in one file alone -- including its comments -- or that test fails. Change both +# copies in one commit, keeping the span's line count intact. # ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER ===== -# ONE resolver, embedded VERBATIM in three scripts: +# ONE resolver, embedded VERBATIM in two scripts (ADR-0025 retired the third): # scripts/skill-size-check.sh -# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh -# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh -# The block between these markers must stay byte-identical in all three. It is -# copied rather than imported because a cache-installed plugin's scripts cannot -# read files outside their own plugin directory, so there is no single file all -# three can share (same constraint that forces the ADR-0020 constants to be -# duplicated). Edit one copy, then paste it over the other two. +# plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh +# The block between these markers must stay byte-identical in both. It is copied +# rather than imported because a cache-installed plugin's scripts cannot read +# files outside their own plugin directory, and this repo-root hook resolves via +# .pre-commit-hooks.yaml, where entry[0] is the only token pre-commit rewrites -- +# so no single file is reachable by both (the same constraint that duplicates the +# ADR-0020 constants). Edit one copy, then paste it over the other. # # Requires: glob, os, re, yaml (imported by the host script; PyYAML is a hard # dependency, preflighted in bash before the interpreter starts). @@ -268,9 +273,9 @@ def read_text(path): # The set of names a boundary clause may resolve against is derived from an # AUTHORING ROOT found by walking up FROM THE TARGET FILE. It is NEVER derived # from this script's own location: deriving it from ${BASH_SOURCE} leaked -# holocron's 39-skill universe into every consumer repo that ran this hook -# through pre-commit, so a consumer skill routing to `skill-audit` resolved -# against a plugin it had never installed. +# holocron's whole skill universe into every consumer repo that ran this hook +# through pre-commit, so a consumer skill routing to a holocron skill such as +# `factory-audit` resolved against a plugin it had never installed. # # An authoring root is the nearest ancestor holding plugins/*/.apm/skills/ or # plugins/*/.apm/agents/ (a plugin monorepo), falling back to the nearest @@ -530,8 +535,8 @@ def known_targets(start_dir): # written yet, and any new process chain re-arms it. Unexercised is not the # same as unnecessary, and the branch it guards is still load-bearing: the # bare-arrow rule is the sole extractor for three real targets in -# kyberforge's audit skills (agent-audit -> agent-author, agent-audit -> -# skill-audit, skill-audit -> skill-author), all written unbackticked. +# kyberforge (factory-audit -> skill-author, factory-audit -> +# agent-author, apm-orchestrate -> apm-install), all written unbackticked. # * A backticked hyphenated token counts only inside a boundary sentence. # Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard # FAIL with no escape hatch. Gating it costs nothing (measured over this @@ -1164,7 +1169,7 @@ def hand_invoked(fm_text): # # Both read a FENCE-MASKED copy of the body. Scanning the raw body made a # ```-fenced example a hard ERROR — and the skills most likely to carry one are -# skill-author and skill-audit, which DOCUMENT the references/ convention — and +# skill-author and factory-audit, which DOCUMENT the references/ convention — and # let a `## Gotchas` heading inside a fenced block stand in for the real # section. Masking preserves every byte offset (content becomes spaces, # newlines stay), so a span found in the mask slices the original. @@ -1189,12 +1194,12 @@ REFERENCE_POINTER = re.compile( REFERENCE_PAST = re.compile( r'\b(?:removed|deleted|renamed|superseded|replaced|obsolete|deprecated' r'|former|formerly|gone|no longer|used to)\b', re.I) -# A pointer QUALIFIED by another skill's name — "skill-audit's -# references/validation-scripts.md" — names a file that is deliberately NOT in +# A pointer QUALIFIED by another skill's name — "factory-audit's +# references/skill-validation-scripts.md" — names a file that is deliberately NOT in # this skill's directory. Requiring it on the local disk left NO legal spelling # for a cross-skill reference at all: the only alternative, a full repo path -# (`plugins/kyberforge/.apm/skills/skill-audit/references/...`), is itself a -# FAIL under skill-audit's own file-structure rubric, because a path that climbs +# (`plugins/kyberforge/.apm/skills/factory-audit/references/...`), is itself a +# FAIL under factory-audit's own file-structure rubric, because a path climbing # out of the skill directory stops resolving once the plugin is cache-installed. # The possessive form is the sanctioned spelling, and it is skipped here. It is # not checked further — this function has no way to locate another skill's @@ -1318,7 +1323,7 @@ for path in files: continue # SPEC CONFORMANCE (family 1). Whole file, frontmatter included, counted - # with the SAME primitives skill-audit/scripts/validate.sh uses for these + # with the SAME primitives factory-audit/scripts/lib-checks-skill.sh uses for these # two constants — see the note in bash above for what the previous awk pass # got wrong. lines = len(raw.splitlines()) diff --git a/scripts/sync-vale-styles.sh b/scripts/sync-vale-styles.sh deleted file mode 100755 index ee98d69..0000000 --- a/scripts/sync-vale-styles.sh +++ /dev/null @@ -1,21 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# Regenerates skill-audit's Vale copy from agent-audit's canonical copy (see -# scripts/check-vale-style-sync.sh / ADR-0014). Both copies must exist on disk -# independently — a plugin's cache-install only copies each skill's own files, -# so a symlink or shared path would break at install time — but that doesn't -# mean the copy step has to be manual. Run this after editing agent-audit's -# vale-wrap.sh or styles/Kyberforge, review the diff, then commit both trees -# together. - -REPO_ROOT="${1:-$(git rev-parse --show-toplevel)}" -SKILL_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit" -AGENT_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit" - -cp "$AGENT_AUDIT/scripts/vale-wrap.sh" "$SKILL_AUDIT/scripts/vale-wrap.sh" -rm -rf "$SKILL_AUDIT/assets/vale/styles/Kyberforge" -cp -r "$AGENT_AUDIT/assets/vale/styles/Kyberforge" "$SKILL_AUDIT/assets/vale/styles/Kyberforge" - -echo "Synced skill-audit's vale-wrap.sh and styles/Kyberforge from agent-audit's canonical copy." -echo "Review the diff, then commit both directories together." diff --git a/tests/run-tests.sh b/tests/run-tests.sh index 9e560c6..ff93451 100755 --- a/tests/run-tests.sh +++ b/tests/run-tests.sh @@ -22,11 +22,13 @@ # green gate having verified 15 of the 17 suites that existed then. # Exactly the vacuous-pass class the rest of this file exists to close. # -# Deliberately its own switch, NOT folded into -# CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE. That one governs whether -# check-vale-style-sync may downgrade itself; this one governs whether the test -# dispatcher tolerates an unrunnable suite. They are separate decisions and one -# flag disarming both gates is how an opt-out quietly grows blast radius. +# Deliberately its own switch, scoped to this dispatcher alone: it governs +# whether an unrunnable suite is tolerated, nothing else. It was once kept +# separate from CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE, which governed whether +# check-vale-style-sync could downgrade itself; that gate is retired (ADR-0025 +# merged the two Vale copies it diffed), but the rule that retired it does not +# apply here. Keep any future vale-related opt-out separate too — one flag +# disarming several gates is how an opt-out quietly grows blast radius. # # TEST_DIR — override root to search for test-*.sh (default: REPO_ROOT); used by tests. set -euo pipefail diff --git a/tests/test-adr0020-body-checks.sh b/tests/test-adr0020-body-checks.sh index e9ae585..a95bbb3 100755 --- a/tests/test-adr0020-body-checks.sh +++ b/tests/test-adr0020-body-checks.sh @@ -15,7 +15,7 @@ # child bullets were counted as top-level entries, `## Gotcha handling` was read # as the Gotchas section, and a documented-then-removed references/ file became a # hard ERROR. The skills most likely to carry such an example are skill-author and -# skill-audit — the two that DOCUMENT these conventions — so a gate that fires on +# factory-audit — the two that DOCUMENT these conventions — so a gate that fires on # them is a gate nobody can turn on. # # Every case is a matched pair: the check fires just over its boundary, and stays @@ -198,7 +198,7 @@ expect "a Gotchas section at 26% of the body raises a SUGGESTION" \ # --------------------------------------------------------------------------- echo "" echo "--- a ## Gotchas heading inside a fenced block is not the Gotchas section ---" -# skill-author and skill-audit both document this convention by showing it. If a +# skill-author and factory-audit both document this convention by showing it. If a # fenced example counted, the two skills that define the rule would be the two # most likely to fail it. F_FENCED_HEADING="$(make_skill gotchas-fenced-heading "$CLEAN_DESC" < "$out" HASHES+=("$(md5sum < "$out" | cut -d' ' -f1)") LINECOUNTS+=("$(wc -l < "$out" | tr -d ' ')") done - if [[ "${HASHES[0]}" == "${HASHES[1]}" && "${HASHES[1]}" == "${HASHES[2]}" ]]; then - pass "all three copies hash to ${HASHES[0]} (${LINECOUNTS[0]} lines) — agreement by construction, not by coincidence" + if [[ "${HASHES[0]}" == "${HASHES[1]}" ]]; then + pass "both copies hash to ${HASHES[0]} (${LINECOUNTS[0]} lines) — agreement by construction, not by coincidence" else - fail "the shared resolver has DRIFTED: skill-size-check=${HASHES[0]} (${LINECOUNTS[0]} lines), skill-audit=${HASHES[1]} (${LINECOUNTS[1]} lines), agent-audit=${HASHES[2]} (${LINECOUNTS[2]} lines). Edit one copy, then paste it over the other two." + fail "the shared resolver has DRIFTED: skill-size-check=${HASHES[0]} (${LINECOUNTS[0]} lines), factory-audit/scripts/lib-boundary-resolver.sh=${HASHES[1]} (${LINECOUNTS[1]} lines). Edit one copy, then paste it over the other." fi - # A block that has been emptied out would hash equal in all three and pass the - # comparison above while enforcing nothing. The resolver is ~570 lines; 100 is - # a floor low enough never to need maintenance and high enough that a gutted - # block cannot sneak past. + # A block that has been emptied out would hash equal in both and pass the + # comparison above while enforcing nothing. The resolver is ~1,060 lines; 100 + # is a floor low enough never to need maintenance and high enough that a + # gutted block cannot sneak past. if [[ "${LINECOUNTS[0]}" -gt 100 ]]; then pass "the extracted block is ${LINECOUNTS[0]} lines — the comparison is over real content, not an empty span" else - fail "the extracted shared block is only ${LINECOUNTS[0]} lines — three identical empty spans would compare equal and assert nothing" + fail "the extracted shared block is only ${LINECOUNTS[0]} lines — two identical empty spans would compare equal and assert nothing" fi fi # --------------------------------------------------------------------------- -# 1b. The shared Contributing-files parser is byte-identical in both copies +# 1a. The resolver copies are the ONLY two, and validate.sh sources its one # --------------------------------------------------------------------------- -# Same defect class, one directory over. parse_contributing_files() is embedded -# in both validate-provenance.sh copies for the same reason the resolver is -# embedded three times, and until this assertion existed the agent-audit copy's +# Byte-identity between two named files says nothing about a THIRD copy, and +# nothing about whether factory-audit's copy is the one that runs. Assertion 1b +# pins both of those for the Contributing-files parser; the resolver is the same +# defect class and gets the same two checks: +# +# a. validate.sh actually SOURCES lib-boundary-resolver.sh, in BOTH mode +# branches — asserted inside each arm of `case "$MODE" in`, not by counting +# source lines file-wide, because a count cannot see a branch. A library +# that is identical, unique and never sourced is a copy that has quietly +# been replaced by an inline one — and the byte-identity check above would +# stay green over it. +# b. Nothing has re-inlined it. The BEGIN marker and a def unique to the +# resolver (`_authoring_root`) appear in exactly the two authorities — +# scripts/skill-size-check.sh and lib-boundary-resolver.sh — and nowhere +# else under the tree. A mode library that grows a "just this once" copy +# would otherwise escape assertion 1 entirely, because 1 hashes only the +# two files it names. +echo "" +echo "--- the ADR-0020 resolver has exactly two authorities, and validate.sh sources factory-audit's ---" + +# Deployed and vendored trees are generated copies, not authorities: .claude/ is +# apm install output, apm_modules/ is resolved dependencies, build/ is release +# artifacts. This file is excluded because it necessarily quotes what it +# searches for. Markdown is excluded because an authority is code that runs: +# ADR-0025 and gates.md quote these needles to document this very check, and a +# prose mention is not a re-inlined copy. +SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")" +tree_scan() { + grep -rlF --binary-files=without-match \ + --exclude-dir=.git --exclude-dir=build --exclude-dir=.claude \ + --exclude-dir=apm_modules --exclude-dir=node_modules --exclude='*.md' \ + -- "$1" "$REPO_ROOT" 2>/dev/null | grep -vFx "$SELF" | sort || true +} + +# (a) Sourced, once per mode branch — asserted PER BRANCH, not by counting. +# +# This used to be a file-wide `grep -c ... >= 2`, and a count cannot see a +# branch. Proven by mutation: moving the agent arm's source line out and +# duplicating the skill arm's leaves the count at 2 and the old assertion +# printed PASS while claiming "in both mode branches" — so a branch that lost +# its resolver gracefully (a defaulted variable, a `set +u` region, an optional +# resolver) stayed green and lying. The claim is per-branch, so the check is +# too: each arm of `case "$MODE" in` must source the resolver inside its own +# body. The mutation that fooled the old form is run against the new one below, +# because an assertion about branches that has never been shown to fail on a +# count-preserving branch edit is exactly the assertion that was here before. +RESOLVER_SOURCE_RE='^[[:space:]]*(\.|source)[[:space:]]+.*lib-boundary-resolver\.sh' + +# The body of one arm of the `case "$MODE" in` block: everything between the +# arm's label and its `;;`. Structural, not positional — the file is edited +# often and line numbers or a whole-file hash would pin the wrong thing. +mode_arm_body() { + awk -v arm="$2" ' + !incase && $0 ~ /^[[:space:]]*case[[:space:]]+"?\$MODE"?[[:space:]]+in[[:space:]]*$/ { incase = 1; next } + incase && $0 ~ /^[[:space:]]*esac([[:space:]]|$)/ { incase = 0; next } + incase && !inarm && $0 ~ "^[[:space:]]*\\(?" arm "\\)[[:space:]]*$" { inarm = 1; next } + inarm && $0 ~ /^[[:space:]]*;;[[:space:]]*$/ { inarm = 0; next } + inarm { print } + ' "$1" +} + +# Every mode arm sources the resolver exactly once, inside its own body. +# Returns 0/1 and leaves the reason in ARM_DETAIL, so the same function can be +# run against the real file and against the mutant below. +ARM_DETAIL="" +check_resolver_per_arm() { + local file="$1" arm body n rc=0 + ARM_DETAIL="" + for arm in skill agent; do + body="$(mode_arm_body "$file" "$arm")" + if [[ -z "$body" ]]; then + ARM_DETAIL+="the $arm) arm of the case \"\$MODE\" block was not found or is empty; " + rc=1 + continue + fi + n="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$body" || true)" + if [[ "$n" -eq 0 ]]; then + ARM_DETAIL+="the $arm) arm never sources lib-boundary-resolver.sh, so that mode runs some other resolver or none; " + rc=1 + elif [[ "$n" -ne 1 ]]; then + ARM_DETAIL+="the $arm) arm sources lib-boundary-resolver.sh $n times; " + rc=1 + fi + done + return $rc +} + +if [[ ! -f "$FACTORY_VALIDATE" ]]; then + fail "entry point not found: ${FACTORY_VALIDATE#"$REPO_ROOT/"}" +else + RESOLVER_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$FACTORY_VALIDATE" || true)" + if check_resolver_per_arm "$FACTORY_VALIDATE"; then + pass "${FACTORY_VALIDATE#"$REPO_ROOT/"}: the skill) and agent) arms of its case \"\$MODE\" block EACH source lib-boundary-resolver.sh inside their own body, exactly once" + else + fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} does not source lib-boundary-resolver.sh once per mode arm: ${ARM_DETAIL%; }" + fi + # Secondary, and deliberately not the verdict: with one source per arm proven + # above, a file-wide total of exactly 2 says there is no third source line + # sitting outside both arms. + if [[ "$RESOLVER_SOURCES" -eq 2 ]]; then + pass "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries exactly 2 resolver source lines file-wide — the two arm sources and nothing else" + else + fail "${FACTORY_VALIDATE#"$REPO_ROOT/"} carries $RESOLVER_SOURCES resolver source lines file-wide, expected the 2 that belong to the mode arms" + fi + + # Mutation self-test. The mutation is the one the retired count could not + # see: the agent arm's source line is removed and the skill arm's duplicated, + # so the FILE-WIDE COUNT IS UNCHANGED. Written into a copy; the real file is + # never touched. + MUT_DIR="$(mktemp -d "$TMPDIR_T/resolver-mutant.XXXXXX")" + MUT="$MUT_DIR/validate.sh" + cp "$FACTORY_VALIDATE" "$MUT" + if python3 - "$MUT" <<'PY' +import re +import sys + +path = sys.argv[1] +with open(path, encoding='utf-8') as fh: + lines = fh.read().split('\n') + +case_re = re.compile(r'^\s*case\s+"?\$MODE"?\s+in\s*$') +esac_re = re.compile(r'^\s*esac(\s|$)') +term_re = re.compile(r'^\s*;;\s*$') +src_re = re.compile(r'^\s*(\.|source)\s+.*lib-boundary-resolver\.sh') + +starts = [i for i, l in enumerate(lines) if case_re.match(l)] +assert len(starts) == 1, 'expected exactly one `case "$MODE" in`, found %d' % len(starts) +ci = starts[0] +ends = [i for i in range(ci + 1, len(lines)) if esac_re.match(lines[i])] +assert ends, 'the case "$MODE" block has no esac' +ei = ends[0] + + +def arm_sources(name): + for i in range(ci + 1, ei): + if re.match(r'^\s*\(?%s\)\s*$' % name, lines[i]): + for j in range(i + 1, ei): + if term_re.match(lines[j]): + return [k for k in range(i + 1, j) if src_re.match(lines[k])] + raise AssertionError('the %s) arm has no ;;' % name) + raise AssertionError('no %s) arm in the case "$MODE" block' % name) + + +skill = arm_sources('skill') +agent = arm_sources('agent') +assert len(skill) == 1 and len(agent) == 1, \ + 'expected one resolver source per arm before mutating, got skill=%d agent=%d' % (len(skill), len(agent)) + +out = list(lines) +del out[agent[0]] # the agent arm loses its resolver ... +out.insert(skill[0], lines[skill[0]]) # ... and the skill arm gains a duplicate +with open(path, 'w', encoding='utf-8') as fh: + fh.write('\n'.join(out)) +PY + then + MUT_SOURCES="$(grep -Ec "$RESOLVER_SOURCE_RE" "$MUT" || true)" + MUT_AGENT="$(grep -Ec "$RESOLVER_SOURCE_RE" <<< "$(mode_arm_body "$MUT" agent)" || true)" + # Guard the fixture before trusting its verdict: the mutation must have + # actually emptied the agent arm AND left the file-wide count where it was, + # or the case below proves nothing about the defect it stands for. + if [[ "$MUT_SOURCES" -eq "$RESOLVER_SOURCES" && "$MUT_AGENT" -eq 0 ]]; then + pass "fixture check: the mutant's agent arm sources no resolver while the file-wide count is still $MUT_SOURCES — the retired 'count >= 2' assertion would have passed it" + else + fail "the resolver mutation did not land as intended (file-wide $MUT_SOURCES vs $RESOLVER_SOURCES, agent arm $MUT_AGENT) — the case below would prove nothing" + fi + if check_resolver_per_arm "$MUT"; then + fail "the per-arm check PASSED a validate.sh whose agent arm has no resolver source — it is still counting, not reading branches" + else + pass "the per-arm check FAILS the count-preserving mutant (${ARM_DETAIL%; }) — it reads the branches, not a total" + fi + else + fail "could not build the resolver mutation fixture — validate.sh's case \"\$MODE\" structure is not the shape this self-test knows, so the per-arm check is unproven" + fi +fi + +# (b) Exactly the two authorities, for both spellings of a copy. +EXPECTED_RESOLVERS="$(printf '%s\n' "$HOOK" "$FACTORY_RESOLVER" | sort)" +check_resolver_authorities() { + local label="$1" needle="$2" + local found + found="$(tree_scan "$needle")" + if [[ "$found" == "$EXPECTED_RESOLVERS" ]]; then + pass "$label appears in exactly the two resolver authorities and nowhere else" + elif [[ -z "$found" ]]; then + fail "$label was found in NO file at all — the scan is looking for the wrong text" + else + fail "$label appears in an unexpected set of files, so the resolver has been re-inlined or lost: $(echo "$found" | tr '\n' ' ')— expected exactly ${HOOK#"$REPO_ROOT/"} and ${FACTORY_RESOLVER#"$REPO_ROOT/"}" + fi +} +check_resolver_authorities "the resolver's BEGIN marker" "$BEGIN_MARKER" +check_resolver_authorities "a 'def _authoring_root' definition" "def _authoring_root(" + +# --------------------------------------------------------------------------- +# 1b. The Contributing-files parser has exactly ONE authority, and it is sourced +# --------------------------------------------------------------------------- +# Same defect class, one directory over. parse_contributing_files() used to be +# embedded in both validate-provenance.sh copies for the same reason the resolver +# is embedded twice, and until this assertion existed the agent-audit copy's # docstring merely CLAIMED it was "kept behaviourally identical to skill-audit's -# copy" — an invariant nothing checked, and the two had already drifted into -# different spellings of the bullet loop. The parser decides whether checks 4, +# copy" — an invariant nothing checked, and the two did drift into different +# spellings of the bullet loop at 484357a. That drift happened to be +# behaviour-neutral and was re-unified by hand at 598a7c3; the next one need +# not be. The parser decides whether checks 4, # 5 and 8 run at all, so a one-sided edit disables a check in one script while # every other test stays green. +# +# ADR-0025 merged the two skills, so there is now ONE copy and nothing left to +# hash against. That does NOT retire the assertion: a byte-identity check over a +# single copy is vacuous, and deleting it outright would restore exactly the +# condition that allowed the original drift — a parser with no pinned authority. +# So the claim is CONVERTED, not dropped. It is the same claim ("the parser has +# exactly one authority") stated against the new structure: +# +# a. factory-audit/scripts/lib-contributing-files.sh exists and carries exactly +# one BEGIN/END marker pair, over a span of real content. +# b. validate-provenance.sh actually SOURCES it. A library nobody sources is a +# copy that has silently been replaced by an inline one somewhere else. +# c. Nothing has re-inlined it. No other file in the tree carries the marker +# pair, and no other file defines parse_contributing_files. This is the part +# that fails if the merge is ever partially reverted, or if a mode library +# grows its own "just this once" copy — which is precisely how the drift +# this assertion was written for got in. echo "" -echo "--- the shared Contributing-files parser is byte-identical in both validate-provenance.sh copies ---" +echo "--- the Contributing-files parser has exactly one authority, and validate-provenance.sh sources it ---" CF_BEGIN='# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====' CF_END='# ===== END SHARED CONTRIBUTING-FILES PARSER =====' -SKILL_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh" -AGENT_PROV="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh" +CF_LIB="$FACTORY_AUDIT/scripts/lib-contributing-files.sh" +FACTORY_PROV="$FACTORY_AUDIT/scripts/validate-provenance.sh" +# (a) One library, one well-formed marker pair, over real content. CF_MARKERS_OK=true -for f in "$SKILL_PROV" "$AGENT_PROV"; do - if [[ ! -f "$f" ]]; then - fail "script not found: $f" - CF_MARKERS_OK=false - continue - fi - b="$(grep -cFx "$CF_BEGIN" "$f" || true)" - e="$(grep -cFx "$CF_END" "$f" || true)" - if [[ "$b" == "1" && "$e" == "1" ]]; then - pass "${f#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker" - else - fail "${f#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1" - CF_MARKERS_OK=false - fi -done - -if ! $CF_MARKERS_OK; then - fail "skipping the parser byte-identity comparison — the marker pairs are not well-formed, so any extraction would measure the wrong span" +if [[ ! -f "$CF_LIB" ]]; then + fail "the single parser authority is missing: ${CF_LIB#"$REPO_ROOT/"}" + CF_MARKERS_OK=false else - CF_HASHES=() - CF_LINECOUNTS=() - for f in "$SKILL_PROV" "$AGENT_PROV"; do - out="$TMPDIR_T/cfblock-$(echo "$f" | md5sum | cut -c1-8).txt" - sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$f" > "$out" - CF_HASHES+=("$(md5sum < "$out" | cut -d' ' -f1)") - CF_LINECOUNTS+=("$(wc -l < "$out" | tr -d ' ')") - done - if [[ "${CF_HASHES[0]}" == "${CF_HASHES[1]}" ]]; then - pass "both copies hash to ${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines) — agreement by construction, not by coincidence" + b="$(grep -cFx "$CF_BEGIN" "$CF_LIB" || true)" + e="$(grep -cFx "$CF_END" "$CF_LIB" || true)" + if [[ "$b" == "1" && "$e" == "1" ]]; then + pass "${CF_LIB#"$REPO_ROOT/"} carries exactly one BEGIN and one END parser marker" else - fail "the shared Contributing-files parser has DRIFTED: skill-audit=${CF_HASHES[0]} (${CF_LINECOUNTS[0]} lines), agent-audit=${CF_HASHES[1]} (${CF_LINECOUNTS[1]} lines). Edit one copy, then paste it over the other." - fi - # Two identical EMPTY spans would hash equal and assert nothing, exactly as - # for the resolver above. The parser block is ~93 lines; 40 is a floor low - # enough never to need maintenance and high enough that a gutted block — or - # one reduced to its docstring — cannot sneak past. - if [[ "${CF_LINECOUNTS[0]}" -gt 40 ]]; then - pass "the extracted parser block is ${CF_LINECOUNTS[0]} lines — the comparison is over real content, not an empty span" - else - fail "the extracted parser block is only ${CF_LINECOUNTS[0]} lines — two identical empty spans would compare equal and assert nothing" + fail "${CF_LIB#"$REPO_ROOT/"} has $b BEGIN and $e END parser markers, expected 1 and 1" + CF_MARKERS_OK=false fi fi +if ! $CF_MARKERS_OK; then + fail "skipping the parser content check — the marker pair is not well-formed, so any extraction would measure the wrong span" +else + # A span gutted down to its docstring would still satisfy every structural + # check above while enforcing nothing, exactly as for the resolver. The parser + # block is ~93 lines; 40 is a floor low enough never to need maintenance and + # high enough that a gutted block cannot sneak past. + CF_LINECOUNT="$(sed -n "/^${CF_BEGIN}\$/,/^${CF_END}\$/p" "$CF_LIB" | wc -l | tr -d ' ')" + if [[ "$CF_LINECOUNT" -gt 40 ]]; then + pass "the extracted parser block is $CF_LINECOUNT lines — a real parser, not an empty or docstring-only span" + else + fail "the extracted parser block is only $CF_LINECOUNT lines — a gutted span asserts nothing" + fi +fi + +# (b) The one entry point sources it. Without this, (a) and (c) are satisfied by +# a library that is present, unique and entirely unused. +if [[ ! -f "$FACTORY_PROV" ]]; then + fail "entry point not found: ${FACTORY_PROV#"$REPO_ROOT/"}" +elif grep -Eq '^[[:space:]]*(\.|source)[[:space:]]+.*lib-contributing-files\.sh' "$FACTORY_PROV"; then + pass "${FACTORY_PROV#"$REPO_ROOT/"} sources lib-contributing-files.sh — the single copy is the one that actually runs" +else + fail "${FACTORY_PROV#"$REPO_ROOT/"} never sources lib-contributing-files.sh — the library is dead code and the parser that runs is some other copy" +fi + +# (c) Nobody re-inlined it. Both spellings are scanned: the marker pair (a +# copy-paste of the block) and a second `def parse_contributing_files` (a +# re-implementation that skipped the markers). tree_scan (defined in 1a) +# excludes the same generated trees and this file. +cf_scan() { tree_scan "$1"; } +check_sole_authority() { + local label="$1" needle="$2" + local found extra + found="$(cf_scan "$needle")" + extra="$(printf '%s\n' "$found" | grep -vFx "$CF_LIB" | grep -v '^$' || true)" + if [[ -z "$found" ]]; then + fail "$label was found in NO file at all — the parser authority has vanished, or the scan is looking for the wrong text" + elif [[ -n "$extra" ]]; then + fail "$label appears outside the single authority, so the parser has been re-inlined: $(echo "$extra" | tr '\n' ' ')— delete the copy and source lib-contributing-files.sh instead" + else + pass "$label appears only in ${CF_LIB#"$REPO_ROOT/"} — one authority, no re-inlined copies" + fi +} +check_sole_authority "the parser's BEGIN marker" "$CF_BEGIN" +check_sole_authority "a 'def parse_contributing_files' definition" "def parse_contributing_files(" + # --------------------------------------------------------------------------- -# 2. Both interpreter preflights, in all three scripts +# 2. Both interpreter preflights, in both scripts and both modes # --------------------------------------------------------------------------- # The two are checked separately on purpose: `python3 -c 'import yaml'` fails # identically whether python3 is missing or PyYAML is, and naming the wrong one # sends the reader to install the wrong thing. +# +# ADR-0025 merged the two validators into one auto-detecting entry point, but +# the preflight did NOT merge with them: validate.sh detects the mode first and +# then calls kyberforge_skill_preflight or kyberforge_agent_preflight from the +# mode library it sources. There are still two preflights, so both are still +# probed — once with a skill target and once with an agent target. Collapsing +# these to a single probe would leave one mode's preflight unpinned, and a mode +# whose preflight is gone reports a vacuous pass on a machine with no PyYAML. REAL_PYTHON="$(command -v python3)" # Absolute path, deliberately. The no-python3 fixture below replaces PATH # wholesale, so a bare `bash` (or `/usr/bin/env bash`) would be resolved against @@ -275,47 +544,48 @@ probe_preflight() { } echo "" -echo "--- a PATH with no python3 is a hard failure in all three scripts, naming python3 ---" +echo "--- a PATH with no python3 is a hard failure in both scripts and both modes, naming python3 ---" probe_preflight "scripts/skill-size-check.sh reports missing python3" \ nopython "python3 is required" \ "$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md" -probe_preflight "skill-audit/scripts/validate.sh reports missing python3" \ +probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing python3" \ nopython "python3 is required" \ - "$SKILL_VALIDATE" "$SUBJECT_SKILL_DIR" -probe_preflight "agent-audit/scripts/validate.sh reports missing python3" \ + "$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR" +probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing python3" \ nopython "python3 is required" \ - "$AGENT_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" + "$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" echo "" -echo "--- a python3 that cannot import yaml is a hard failure in all three scripts, naming PyYAML ---" +echo "--- a python3 that cannot import yaml is a hard failure in both scripts and both modes, naming PyYAML ---" probe_preflight "scripts/skill-size-check.sh reports missing PyYAML" \ noyaml "PyYAML is required" \ "$HOOK" "$SUBJECT_SKILL_DIR/SKILL.md" -probe_preflight "skill-audit/scripts/validate.sh reports missing PyYAML" \ +probe_preflight "factory-audit/scripts/validate.sh (skill mode) reports missing PyYAML" \ noyaml "PyYAML is required" \ - "$SKILL_VALIDATE" "$SUBJECT_SKILL_DIR" -probe_preflight "agent-audit/scripts/validate.sh reports missing PyYAML" \ + "$FACTORY_VALIDATE" "$SUBJECT_SKILL_DIR" +probe_preflight "factory-audit/scripts/validate.sh (agent mode) reports missing PyYAML" \ noyaml "PyYAML is required" \ - "$AGENT_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" + "$FACTORY_VALIDATE" "$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md" # The control. Without it, "fails when the dependency is missing" is satisfied by # a script that fails unconditionally, and the two cases above would be green on # a gate that never runs at all. echo "" echo "--- control: with both dependencies present the same subjects pass ---" -for probe in "$HOOK:$SUBJECT_SKILL_DIR/SKILL.md" \ - "$SKILL_VALIDATE:$SUBJECT_SKILL_DIR" \ - "$AGENT_VALIDATE:$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"; do - script="${probe%%:*}" - arg="${probe#*:}" +# The label is carried explicitly because the two validate.sh probes now name the +# same script and differ only in the mode its target selects. +for probe in "scripts/skill-size-check.sh|$HOOK|$SUBJECT_SKILL_DIR/SKILL.md" \ + "factory-audit/scripts/validate.sh (skill mode)|$FACTORY_VALIDATE|$SUBJECT_SKILL_DIR" \ + "factory-audit/scripts/validate.sh (agent mode)|$FACTORY_VALIDATE|$SUBJECT_AGENT_ROOT/.apm/agents/my-agent.agent.md"; do + IFS='|' read -r label script arg <<< "$probe" set +e ctl_out="$(bash "$script" "$arg" 2>&1)" ctl_rc=$? set -e if [[ $ctl_rc -eq 0 ]]; then - pass "${script#"$REPO_ROOT/"} exits 0 on a clean subject with python3 and PyYAML available" + pass "$label exits 0 on a clean subject with python3 and PyYAML available" else - fail "${script#"$REPO_ROOT/"} failed a clean subject (exit $ctl_rc): $ctl_out" + fail "$label failed a clean subject (exit $ctl_rc): $ctl_out" fi done diff --git a/tests/test-adr0020-differential.sh b/tests/test-adr0020-differential.sh index 9e11f7d..6373e92 100755 --- a/tests/test-adr0020-differential.sh +++ b/tests/test-adr0020-differential.sh @@ -1,7 +1,11 @@ #!/usr/bin/env bash # Differential test: scripts/skill-size-check.sh (the pre-commit hook) and -# skill-audit/scripts/validate.sh (the in-skill auditor) must reach the SAME -# ADR-0020 verdict on the same file. +# factory-audit/scripts/validate.sh in SKILL mode (the in-skill auditor) must +# reach the SAME ADR-0020 verdict on the same file. ADR-0025 merged skill-audit +# and agent-audit behind one auto-detecting entry point; every fixture here is a +# skill directory, so every invocation below runs the skill flow. The agent flow +# has no counterpart hook to differ from — there is no agent-file size gate in +# .pre-commit-hooks.yaml — so it is out of this suite's scope, not dropped from it. # # Why this exists as a separate suite. tests/test-skill-size-check.sh already # asserts the two agree on their CONSTANTS, and that assertion is necessary but @@ -11,7 +15,7 @@ # wording, which value gets measured, and which branch runs first are the others, # and none of them is visible to a constant check. # -# The consequence of divergence is specific and bad: skill-audit reports a skill +# The consequence of divergence is specific and bad: the auditor reports a skill # ready to ship and the commit hook then rejects it, or worse, the reverse. So the # comparison here is over VERDICTS on files, not over source text. # @@ -38,7 +42,7 @@ set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" HOOK="$REPO_ROOT/scripts/skill-size-check.sh" -SKILL_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh" +SKILL_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" TMPDIR_T="$(mktemp -d)" trap 'rm -rf "$TMPDIR_T"' EXIT @@ -338,7 +342,7 @@ def compare(label, skill_dir): if only_hook: problems.append('only the hook reported %s' % (only_hook,)) if only_audit: - problems.append('only skill-audit reported %s' % (only_audit,)) + problems.append('only the auditor reported %s' % (only_audit,)) # Exit codes are compared on the ADR-0020 axis only: an ERROR-tier ADR-0020 # finding must make BOTH scripts non-zero, and neither may be turned @@ -349,7 +353,7 @@ def compare(label, skill_dir): if hook_err and hook_rc == 0: problems.append('the hook reported an ADR-0020 ERROR but exited 0') if audit_err and audit_rc == 0: - problems.append('skill-audit reported an ADR-0020 FAIL but exited 0') + problems.append('the auditor reported an ADR-0020 FAIL but exited 0') # No escape hatch here any more. There used to be one — a # `_non_adr_hook_error()` helper that waved through a non-zero hook exit # explained by MAX_LINES / MAX_WORDS, on the grounds that those two were @@ -441,7 +445,7 @@ for name, token, expected in (('spec-lines-u2028', 'SPEC_LINES', '605'), _, a_out = run(['bash', validate, skill_dir]) want = ('ERROR', token, expected) missing = [who for who, v in (('the hook', verdict(h_out)), - ('skill-audit', verdict(a_out))) + ('the auditor', verdict(a_out))) if want not in v] if missing: bad('%s: %s did not report %s=%s. The two scripts must count with the ' diff --git a/tests/test-adr0020-frontmatter.sh b/tests/test-adr0020-frontmatter.sh index 07cf878..4129ba3 100755 --- a/tests/test-adr0020-frontmatter.sh +++ b/tests/test-adr0020-frontmatter.sh @@ -44,8 +44,12 @@ set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" HOOK="$REPO_ROOT/scripts/skill-size-check.sh" -SKILL_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh" -AGENT_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" +# ADR-0025 merged the two validators into one auto-detecting entry point. The two +# names are kept because the two MODES are what this suite probes, and each mode +# still needs its own target shape to reach: collapsing to a single invocation +# would leave one flow's checks unexercised. +SKILL_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" +AGENT_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" PASS=0 FAIL=0 @@ -174,9 +178,11 @@ PY # Builds all three subjects for one fixture kind and echoes nothing; the paths # are fixed by convention so the probes below can find them. # -# skill-audit takes a DIRECTORY (SKILL.md inside it, name matching the dir); -# agent-audit takes a FILE inside an apm package. The hook takes the SKILL.md -# directly, so it and skill-audit share one file. +# The auditor's skill mode takes a DIRECTORY (SKILL.md inside it, name matching +# the dir); its agent mode takes a FILE inside an apm package. The hook takes the +# SKILL.md directly, so it and skill mode share one file. The two shapes are also +# what selects the mode — validate.sh detects from the target — so the two probes +# below are the only way to reach both flows. build_subjects() { local kind="$1" base="$TMPDIR_T/$1" rm -rf "$base" @@ -195,21 +201,21 @@ EOF # assertion per script would let two of them drift apart while the suite stayed # green; the whole point of the shared resolver block is that they cannot. # -# A needle written `@skills:` is asserted for the hook and skill-audit but -# NOT for agent-audit. There is exactly one such needle in this file — the body +# A needle written `@skills:` is asserted for the hook and for skill mode +# but NOT for agent mode. There is exactly one such needle in this file — the body # word ceiling — and the exemption is the ADR, not a workaround: ADR-0020 gives # agents the description gates and deliberately NO body word gate, because an # agent body becomes the system prompt of a fresh context rather than competing -# with the caller's live conversation. Demanding a body finding from agent-audit -# would be demanding the ADR be contradicted. +# with the caller's live conversation. Demanding a body finding from the agent +# flow would be demanding the ADR be contradicted. probe_all() { local label="$1" kind="$2" shift 2 local base="$TMPDIR_T/$kind" local -a targets=( "hook|$HOOK|$base/skill/my-skill/SKILL.md" - "skill-audit|$SKILL_VALIDATE|$base/skill/my-skill" - "agent-audit|$AGENT_VALIDATE|$base/agent/.apm/agents/my-agent.agent.md" + "validate.sh skill mode|$SKILL_VALIDATE|$base/skill/my-skill" + "validate.sh agent mode|$AGENT_VALIDATE|$base/agent/.apm/agents/my-agent.agent.md" ) local problems="" for target in "${targets[@]}"; do @@ -230,7 +236,7 @@ probe_all() { # short form's exit status is the test's when it is false, and relying on # the &&-list exemption to keep that from aborting the run is a footgun # one edit away from biting. - if [[ "$who" == agent-audit ]]; then + if [[ "$who" == "validate.sh agent mode" ]]; then continue fi needle="${needle#@skills:}" @@ -397,8 +403,8 @@ for spec in "yaml-malformed|yes" "desc-list|no" "desc-mapping|no" "desc-bool|no" build_subjects "$kind" for target in \ "hook|$HOOK|$TMPDIR_T/$kind/skill/my-skill/SKILL.md" \ - "skill-audit|$SKILL_VALIDATE|$TMPDIR_T/$kind/skill/my-skill" \ - "agent-audit|$AGENT_VALIDATE|$TMPDIR_T/$kind/agent/.apm/agents/my-agent.agent.md" + "validate.sh skill mode|$SKILL_VALIDATE|$TMPDIR_T/$kind/skill/my-skill" \ + "validate.sh agent mode|$AGENT_VALIDATE|$TMPDIR_T/$kind/agent/.apm/agents/my-agent.agent.md" do who="${target%%|*}"; rest="${target#*|}" script="${rest%%|*}"; arg="${rest#*|}" @@ -430,9 +436,9 @@ SILENT_OUT="$(bash "$AGENT_VALIDATE" "$TMPDIR_T/desc-no-value/agent/.apm/agents/ SILENT_RC=$? set -e if [[ $SILENT_RC -ne 0 && -n "$SILENT_OUT" ]]; then - pass "agent-audit reports a valueless description rather than exiting 0 with zero output" + pass "validate.sh agent mode reports a valueless description rather than exiting 0 with zero output" else - fail "agent-audit exited $SILENT_RC with output '${SILENT_OUT:-}' — the original defect was exit 0 and total silence on a blocking pre-push gate" + fail "validate.sh agent mode exited $SILENT_RC with output '${SILENT_OUT:-}' — the original defect was exit 0 and total silence on a blocking pre-push gate" fi echo "" diff --git a/tests/test-check-apm-agents-valid.sh b/tests/test-check-apm-agents-valid.sh index 7c072b1..0739d53 100755 --- a/tests/test-check-apm-agents-valid.sh +++ b/tests/test-check-apm-agents-valid.sh @@ -1,8 +1,11 @@ #!/usr/bin/env bash set -euo pipefail -# Tests for scripts/check-apm-agents-valid.sh — the gate that runs agent-audit's -# validate.sh over the repo's REAL plugin-scope agent files. +# Tests for scripts/check-apm-agents-valid.sh — the gate that runs factory-audit's +# validate.sh over the repo's REAL plugin-scope agent files. Since ADR-0025 merged +# skill-audit and agent-audit, that validate.sh is one auto-detecting entry point; +# every target this gate hands it is an agent file, so every run below takes the +# agent branch and sources lib-checks-agent.sh. # # Case 1 runs against the real repo. Every other case runs against a synthetic # fixture, for the same reason scripts/check-scope-walkup-sync.sh's tests do: the @@ -26,7 +29,7 @@ fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); } # closed by design, which is correct behavior but makes every case here assert # the same missing-dependency message instead of what it is meant to assert. if ! command -v python3 >/dev/null 2>&1; then - echo "SKIP: python3 is not installed — agent-audit's validate.sh cannot run, so these cases would only re-assert the missing-dependency guard" + echo "SKIP: python3 is not installed — factory-audit's validate.sh cannot run, so these cases would only re-assert the missing-dependency guard" exit 77 fi @@ -40,8 +43,19 @@ trap cleanup EXIT RUN_TMP="$(mktemp -d)" FIXTURES+=("$RUN_TMP") -# Builds a minimal REPO_ROOT: agent-audit's validator and the field inventory it -# reads at load time, plus one plugin carrying a valid agent file. The plugin's +# Builds a minimal REPO_ROOT: factory-audit's validator and the field inventory it +# reads at load time, plus one plugin carrying a valid agent file. The validator +# is no longer self-contained — ADR-0025 replaced the embedded resolver with a +# sourced lib-boundary-resolver.sh and moved the agent checks into +# lib-checks-agent.sh — so the fixture copies the two libraries the agent branch +# sources as well. Copying validate.sh alone would make every case below fail on +# a missing source file rather than on what it is meant to assert. +# +# The inventory is `agent-field-inventory.md`: factory-audit prefixes every +# flow-specific reference file with skill-/agent-, and validate.sh reads this one +# by that name at load time. +# +# The plugin's # apm.yml needs a top-level `type:` line — that is the marker validate.sh's # walk-up uses to resolve plugin scope, and without it the fixture would resolve # to project scope and fail looking for a .github/agents counterpart. @@ -54,10 +68,14 @@ FIXTURES+=("$RUN_TMP") make_fixture() { local dir dir="$(cd "$(mktemp -d)" && pwd -P)" - local aa="$dir/plugins/kyberforge/.apm/skills/agent-audit" - mkdir -p "$aa/scripts" "$aa/references" "$dir/plugins/lint/.apm/agents" - cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" "$aa/scripts/" - cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md" "$aa/references/" + local fa="$dir/plugins/kyberforge/.apm/skills/factory-audit" + local src="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit" + mkdir -p "$fa/scripts" "$fa/references" "$dir/plugins/lint/.apm/agents" + cp "$src/scripts/validate.sh" \ + "$src/scripts/lib-boundary-resolver.sh" \ + "$src/scripts/lib-checks-agent.sh" \ + "$fa/scripts/" + cp "$src/references/agent-field-inventory.md" "$fa/references/" cat > "$dir/plugins/lint/apm.yml" <<'YAML' name: lint version: 0.0.1 @@ -248,7 +266,7 @@ echo "" echo "--- a missing validate.sh fails rather than validating nothing ---" FIX7="$(make_fixture)" FIXTURES+=("$FIX7") -rm -f "$FIX7/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" +rm -f "$FIX7/plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh" if bash "$SCRIPT" "$FIX7" > "$RUN_TMP/novalidator.out" 2>&1; then fail "a missing validate.sh exited 0 — the gate silently validated nothing" sed 's/^/ /' "$RUN_TMP/novalidator.out" diff --git a/tests/test-check-scope-walkup-sync.sh b/tests/test-check-scope-walkup-sync.sh index 711fbda..8b0d4a9 100755 --- a/tests/test-check-scope-walkup-sync.sh +++ b/tests/test-check-scope-walkup-sync.sh @@ -24,24 +24,32 @@ FIXTURES+=("$RUN_TMP") # paths) so the mutation cases below don't depend on — or risk mutating — the # real repo tree. Defined up here rather than beside its first mutation case # because case 2b's stale-.apm/ fixture is built from it too. +# +# Still FOUR ports, still four scripts. ADR-0025 merged two of them into +# factory-audit, which is a change of address, not of count: validate.sh and +# validate-provenance.sh are now factory-audit's, and the two authors' scaffold +# scripts are untouched. factory-audit's whole scripts/ directory is copied +# because those two are entry points now — each sources its resolver and its +# mode library at run time, and a fixture holding only the entry point would fail +# on a missing source file instead of on the scope-walk-up behaviour under test. make_minimal_repo_root() { local dir dir="$(mktemp -d)" local na="$dir/plugins/kyberforge/.apm/skills/agent-author/scripts" local ns="$dir/plugins/kyberforge/.apm/skills/skill-author/scripts" - local aa="$dir/plugins/kyberforge/.apm/skills/agent-audit/scripts" - mkdir -p "$na" "$ns" "$aa" + local fa="$dir/plugins/kyberforge/.apm/skills/factory-audit" + mkdir -p "$na" "$ns" "$fa" cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-author/scripts/new-agent.sh" "$na/" cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-author/scripts/new-skill.sh" "$ns/" - cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" "$aa/" - cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh" "$aa/" + cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts" "$fa/" # agent-author's templates are needed by new-agent.sh at runtime. cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-author/assets" "$dir/plugins/kyberforge/.apm/skills/agent-author/" cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-author/assets" "$dir/plugins/kyberforge/.apm/skills/skill-author/" - # validate.sh needs field-inventory.md - mkdir -p "$dir/plugins/kyberforge/.apm/skills/agent-audit/references" - cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/references/field-inventory.md" \ - "$dir/plugins/kyberforge/.apm/skills/agent-audit/references/" + # validate.sh's agent mode reads agent-field-inventory.md at load time — + # factory-audit prefixes every flow-specific reference file with skill-/agent-. + mkdir -p "$fa/references" + cp "$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/references/agent-field-inventory.md" \ + "$fa/references/" echo "$dir" } @@ -124,11 +132,16 @@ fi # --- 4. Regression guard: reintroducing the $HOME-collapse bug into # validate.sh's detect_scope must make the check fail. +# +# The mutation target is lib-checks-agent.sh, not validate.sh: ADR-0025 made +# validate.sh a mode-detecting entry point and moved the agent check suite — +# detect_scope with it — into the library it sources. The gate under test still +# runs validate.sh, so the fault injected here still reaches it. echo "" echo "--- exits 1 when validate.sh's detect_scope collapses back to the \$HOME-walk-up bug ---" FIXTURE_BUG="$(make_minimal_repo_root)" FIXTURES+=("$FIXTURE_BUG") -python3 - "$FIXTURE_BUG/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" <<'PYTHON' +python3 - "$FIXTURE_BUG/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh" <<'PYTHON' import re, sys path = sys.argv[1] with open(path) as f: @@ -173,7 +186,9 @@ echo "" echo "--- exits 1 when validate-provenance.sh's find_plugin_root loses its \$HOME boundary check ---" FIXTURE_BUG2="$(make_minimal_repo_root)" FIXTURES+=("$FIXTURE_BUG2") -python3 - "$FIXTURE_BUG2/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh" <<'PYTHON' +# Same relocation as case 4: the agent provenance suite, find_plugin_root +# included, now lives in the library validate-provenance.sh sources. +python3 - "$FIXTURE_BUG2/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh" <<'PYTHON' import re, sys path = sys.argv[1] with open(path) as f: diff --git a/tests/test-check-vale-style-sync.sh b/tests/test-check-vale-style-sync.sh deleted file mode 100755 index a7a6184..0000000 --- a/tests/test-check-vale-style-sync.sh +++ /dev/null @@ -1,797 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -SCRIPT="$REPO_ROOT/scripts/check-vale-style-sync.sh" -PASS=0 -FAIL=0 - -pass() { echo " PASS: $1"; PASS=$((PASS + 1)); } -fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); } - -# Same `exit 77` (automake convention; run-tests.sh renders it as SKIPPED) guard -# tests/test-vale-wrap.sh uses for a missing binary. Without it this suite reported 5 genuine failures on a machine with no -# vale, none of which were regressions. -# -# The suite SKIPPING while the script it tests HARD-FAILS is deliberate, not an -# inconsistency. The script is a pre-push gate whose exit 0 is a claim that the -# repo was verified, and six of its assertions are vale invocations — it must -# never make that claim on a machine where they could not run. This suite makes -# no claim about the repo; it claims the script behaves correctly, and most of -# its cases (every glob-coverage case, 10/10b/11) cannot be exercised at all -# without vale. Reporting those as FAIL would say "a regression landed" when the -# truth is "this machine is missing a dev dependency" — noise that competes with -# real failures. Note also that the vale-absent behavior is still fully covered -# here even so: the masking below constructs that condition deliberately on a -# machine that HAS vale, which is the only place it can be asserted against a -# known-good baseline. -if ! command -v vale &>/dev/null; then - echo "SKIP: vale is not installed — the glob-coverage cases cannot run (install it: https://vale.sh/docs/vale-cli/installation/)" - exit 77 -fi - -# One trap over a registry, rather than rebuilding the trap line per fixture: -# the guard is there because bash 3.2 treats "${arr[@]}" on an empty array as -# unbound under `set -u`. -FIXTURES=() -cleanup() { [[ ${#FIXTURES[@]} -eq 0 ]] || rm -rf "${FIXTURES[@]}"; } -trap cleanup EXIT - -# Helper: make a fixture repo with skill-audit/agent-audit's Vale copies, in sync by default. -# The wrapper is a stub — the script only diffs it — but the Vale assets and both -# pre-commit manifests are the repo's real ones, because the .vale.ini checks ask -# vale to apply those globs for real and cross-check them against the shipped -# hooks' `files:` regexes. A synthetic style or manifest would prove nothing, and -# copying the real ones keeps agent-audit's intentional KyberforgeCopilot -# divergence in the fixture instead of a sanitized stand-in for it. -make_fixture() { - local dir - dir="$(mktemp -d)" - local skill_audit="$dir/plugins/kyberforge/.apm/skills/skill-audit" - local agent_audit="$dir/plugins/kyberforge/.apm/skills/agent-audit" - mkdir -p "$skill_audit/scripts" "$agent_audit/scripts" - - echo '#!/usr/bin/env bash' > "$skill_audit/scripts/vale-wrap.sh" - echo 'echo wrap' >> "$skill_audit/scripts/vale-wrap.sh" - cp "$skill_audit/scripts/vale-wrap.sh" "$agent_audit/scripts/vale-wrap.sh" - - cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/assets" "$skill_audit/" - cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/assets" "$agent_audit/" - cp "$REPO_ROOT/.pre-commit-hooks.yaml" "$REPO_ROOT/.pre-commit-config.yaml" "$dir/" - - echo "$dir" -} - -# Helper: rewrite a glob section header in one copy's .vale.ini, leaving every -# other line — StylesPath, BasedOnStyles — intact. This is the shape of the -# typo the check exists to catch: the hook still matches the file via its -# `files:` regex, vale lints nothing, and pre-commit reports `Passed`. -break_glob() { - local ini="$1" old="$2" new="$3" - python3 - "$ini" "$old" "$new" <<'PYTHON' -import sys -path, old, new = sys.argv[1], sys.argv[2], sys.argv[3] -with open(path, encoding='utf-8') as fh: - content = fh.read() -assert old in content, f"{old} not found in {path}" -with open(path, 'w', encoding='utf-8') as fh: - fh.write(content.replace(old, new)) -PYTHON -} - -# Vale masking, hoisted so the text-only cases below can use it. Case 12 keeps -# its own independent construction and its own loud failure if masking breaks — -# it is what proves this mechanism works, so it is not refactored onto this. -# -# Why: a script run with vale on PATH performs six `vale` invocations (one per -# probe path), and they are the suite's entire wall clock. The cases that assert -# a text-level finding — StylesPath, BasedOnStyles, per-rule overrides — reach -# their verdict through `grep` alone and gain nothing from paying for the -# probes. Masking vale is not merely cheaper for them, it is STRICTER: with vale -# present a dropped StylesPath also breaks the probe, so such a case would still -# exit 1 with the assertion under test deleted. Without vale, only the assertion -# under test can produce the failure. -# -# run_check falls back to an unmasked run rather than skipping when masking is -# not safely available, so a machine where this cannot work loses speed, never -# coverage. The utility probe matters as much as the vale probe: PATH_NO_VALE -# deletes a whole PATH entry, and if that entry also carried grep/diff/awk/cat -# the script would fail for an unrelated reason and every negative case below -# would pass vacuously. -VALE_DIR="$(dirname "$(command -v vale 2>/dev/null || echo /nonexistent/vale)")" -PATH_NO_VALE="$(printf '%s' "$PATH" | tr ':' '\n' | grep -vxF "$VALE_DIR" | paste -sd: -)" -VALE_MASKED=false -if ! PATH="$PATH_NO_VALE" bash -c 'command -v vale' >/dev/null 2>&1 \ - && PATH="$PATH_NO_VALE" bash -c \ - 'command -v grep && command -v diff && command -v awk && command -v cat' >/dev/null 2>&1; then - VALE_MASKED=true -fi - -# Runs the check with vale masked off PATH when that is safe. For text-only -# assertions ONLY — never for a case whose verdict depends on a glob probe -# actually running. -# -# CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 is required now that the script -# treats a missing vale as a FAIL rather than a warning: without the opt-out -# every masked run exits 1 unconditionally and every negative case below would -# pass vacuously — the precise vacuity this whole round is closing. The opt-out -# restores what masking is for here: the text assertion under test becomes the -# only thing that can produce a non-zero exit. Case 12 asserts the un-opted-out -# masked run really does hard-fail, so this env var cannot quietly become the -# only path anyone exercises. -run_check_no_vale() { - if [[ "$VALE_MASKED" == true ]]; then - PATH="$PATH_NO_VALE" CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 bash "$SCRIPT" "$@" - else - bash "$SCRIPT" "$@" - fi -} - -# --- 1. Exits 0 when the two copies are in sync --- -echo "" -echo "--- exits 0 when skill-audit and agent-audit copies are in sync ---" -FIXTURE="$(make_fixture)" -FIXTURES+=("$FIXTURE") -if bash "$SCRIPT" "$FIXTURE" > /dev/null 2>&1; then - pass "exits 0 when copies are in sync" -else - fail "exited non-zero against in-sync copies" - bash "$SCRIPT" "$FIXTURE" 2>&1 | sed 's/^/ /' || true -fi - -# --- 2. Exits 1 when vale-wrap.sh differs between the two copies --- -echo "" -echo "--- exits 1 when vale-wrap.sh differs ---" -FIXTURE2="$(make_fixture)" -FIXTURES+=("$FIXTURE2") -echo 'echo different' >> "$FIXTURE2/plugins/kyberforge/.apm/skills/skill-audit/scripts/vale-wrap.sh" -if bash "$SCRIPT" "$FIXTURE2" > /dev/null 2>&1; then - fail "exited 0 when vale-wrap.sh copies differ — expected exit 1" -else - pass "exits non-zero when vale-wrap.sh copies differ" -fi - -# --- 3. Exits 1 when a style rule differs between the two copies --- -echo "" -echo "--- exits 1 when a Kyberforge style rule differs ---" -FIXTURE3="$(make_fixture)" -FIXTURES+=("$FIXTURE3") -echo ' - divergent token' >> "$FIXTURE3/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/VagueWording.yml" -if bash "$SCRIPT" "$FIXTURE3" > /dev/null 2>&1; then - fail "exited 0 when a style rule differs — expected exit 1" -else - pass "exits non-zero when a Kyberforge style rule differs between copies" -fi - -# --- 4. Exits 1 when a rule file exists in only one copy --- -echo "" -echo "--- exits 1 when a rule file is missing from one copy ---" -FIXTURE4="$(make_fixture)" -FIXTURES+=("$FIXTURE4") -cat > "$FIXTURE4/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/Kyberforge/Extra.yml" <<'EOF' -extends: existence -message: "Extra: '%s'" -level: error -tokens: - - divergent token -EOF -if bash "$SCRIPT" "$FIXTURE4" > /dev/null 2>&1; then - fail "exited 0 when a rule file exists in only one copy — expected exit 1" -else - pass "exits non-zero when a rule file is missing from one copy" -fi - -# --- 5. Exits 0 (no-op) ONLY when there is no kyberforge plugin at all --- -# The no-op is scoped to a repo that never installed kyberforge. Case 5c below -# is its counterpart and the one that matters: `plugins/kyberforge/` present but -# the `.apm/` targets under it absent is drift, not absence. -echo "" -echo "--- exits 0 when there is no plugins/kyberforge at all (no-op) ---" -FIXTURE5="$(mktemp -d)" -FIXTURES+=("$FIXTURE5") -if [[ -e "$FIXTURE5/plugins/kyberforge" ]]; then - fail "fixture 5 unexpectedly has a plugins/kyberforge, so it does not exercise the no-kyberforge no-op" -elif bash "$SCRIPT" "$FIXTURE5" > /dev/null 2>&1; then - pass "exits 0 as a no-op when the repo has no kyberforge plugin" -else - fail "exited non-zero when the repo simply has no kyberforge plugin" -fi - -# --- 5c. Exits 1, saying so, when plugins/kyberforge exists but its .apm/ -# targets do not --- -# This script hardcodes plugins/kyberforge/.apm/skills/{skill-audit,agent-audit} -# and had no floor under them: `mv plugins/kyberforge/.apm plugins/kyberforge/.apm2` -# made both directories absent, which fell into the no-op above and exited 0 — -# indistinguishable from a verified in-sync result, and swallowed by pre-commit -# as `Passed`. A path rewrite is exactly the edit that produces this, and it is -# what this PR did to these paths. -# -# Exit code alone proves little here (the script exits 1 for a dozen reasons), so -# assert the MESSAGE: deleting the floor leaves exit 0, but a floor that fired -# for the wrong reason would still be a bug this case must catch. -echo "" -echo "--- exits 1 and says so when plugins/kyberforge exists but .apm/ does not ---" -FIXTURE5C="$(make_fixture)" -FIXTURES+=("$FIXTURE5C") -mv "$FIXTURE5C/plugins/kyberforge/.apm" "$FIXTURE5C/plugins/kyberforge/.apm2" -STALE_OUT="" -STALE_RC=0 -STALE_OUT="$(bash "$SCRIPT" "$FIXTURE5C" 2>&1)" || STALE_RC=$? -if [[ $STALE_RC -eq 0 ]]; then - fail "exited 0 when plugins/kyberforge exists but its .apm/ targets are gone — expected exit 1" -elif ! printf '%s\n' "$STALE_OUT" | grep -q "\.apm/ paths have gone stale"; then - fail "failed for the wrong reason on a stale .apm/ path: $(printf '%s' "$STALE_OUT" | tr '\n' ' ')" -else - pass "exits non-zero and reports a stale .apm/ path when plugins/kyberforge exists without it" -fi - -# --- 5d/5d2. Exits 1, saying so, when the probe TABLE itself verifies nothing --- -# 5d used to relocate `assets/vale/` in both skills, on the belief that doing so -# skipped the whole probe table with FAIL still at 0. It does not. Run against -# the PRE-guard script that fixture already exited 1 with three errors: the -# `.vale.ini` loop errs on both missing files long before the probe loop, and -# PROBES_CHECKED can only reach 0 when both files are gone — which necessarily -# means FAIL >= 2. So it never exercised the guard as a cause, only checked that -# its message showed up beside unrelated failures. -# -# The guard is still worth having, but its real triggers live in the probe table, -# which is part of the script rather than the fixture — so these two cases mutate -# a COPY of the script and run that. Both assert `1 error(s)`, which is what makes -# them real: with the guard deleted each mutation exits 0, and with it present the -# guard is provably the only thing that failed the run. -assert_mutated() { - if diff -q "$SCRIPT" "$1" >/dev/null 2>&1; then - fail "the script mutation changed nothing — the probe table's shape has moved, so this case would pass vacuously" - return 1 - fi -} - -echo "" -echo "--- exits 1 and says so when every probe row names a directory that does not exist ---" -FIXTURE5D="$(make_fixture)" -FIXTURES+=("$FIXTURE5D") -SCRATCH5D="$(mktemp -d)" -FIXTURES+=("$SCRATCH5D") -sed 's/^skill-audit|/skill-auditX|/; s/^agent-audit|/agent-auditX|/' "$SCRIPT" > "$SCRATCH5D/drifted.sh" -NOPROBE_OUT="" -NOPROBE_RC=0 -if assert_mutated "$SCRATCH5D/drifted.sh"; then - NOPROBE_OUT="$(bash "$SCRATCH5D/drifted.sh" "$FIXTURE5D" 2>&1)" || NOPROBE_RC=$? - if [[ $NOPROBE_RC -eq 0 ]]; then - fail "a probe table naming no existing skill directory exited 0 — the glob-coverage section checked nothing and reported success" - elif ! printf '%s\n' "$NOPROBE_OUT" | grep -q "no probe path was checked"; then - fail "did not report that zero probe paths were checked: $(printf '%s' "$NOPROBE_OUT" | tr '\n' ' ')" - elif ! printf '%s\n' "$NOPROBE_OUT" | grep -q "failed: 1 error(s)"; then - fail "drifted probe rows failed for reasons beyond the empty probe table, so this guard is not provably what fired: $(printf '%s' "$NOPROBE_OUT" | tr '\n' ' ')" - else - pass "a probe table whose rows name no existing skill directory fails with that guard as the sole error" - fi -fi - -echo "" -echo "--- exits 1 and says so when the probe table is empty ---" -FIXTURE5D2="$(make_fixture)" -FIXTURES+=("$FIXTURE5D2") -SCRATCH5D2="$(mktemp -d)" -FIXTURES+=("$SCRATCH5D2") -# The other reachable trigger: the heredoc gutted outright by a bad merge or a -# truncated edit. `done <<'EOF_PROBE'` with no rows between the delimiters is -# valid bash — the loop body simply never runs. -awk ' - /^done <<.EOF_PROBE.$/ { print; inblk = 1; next } - inblk && /^EOF_PROBE$/ { print; inblk = 0; next } - inblk { next } - { print } -' "$SCRIPT" > "$SCRATCH5D2/gutted.sh" -EMPTYTBL_OUT="" -EMPTYTBL_RC=0 -if assert_mutated "$SCRATCH5D2/gutted.sh"; then - EMPTYTBL_OUT="$(bash "$SCRATCH5D2/gutted.sh" "$FIXTURE5D2" 2>&1)" || EMPTYTBL_RC=$? - if [[ $EMPTYTBL_RC -eq 0 ]]; then - fail "an empty probe table exited 0 — the glob-coverage section verified nothing and reported success" - elif ! printf '%s\n' "$EMPTYTBL_OUT" | grep -q "no probe path was checked"; then - fail "did not report that zero probe paths were checked: $(printf '%s' "$EMPTYTBL_OUT" | tr '\n' ' ')" - elif ! printf '%s\n' "$EMPTYTBL_OUT" | grep -q "failed: 1 error(s)"; then - fail "an empty probe table failed for reasons beyond the guard: $(printf '%s' "$EMPTYTBL_OUT" | tr '\n' ' ')" - else - pass "an emptied probe heredoc fails with that guard as the sole error" - fi -fi - -# --- 5e. Positive: the check does real work against THIS repo --- -# Every case above runs against a synthetic fixture, so the whole suite could be -# green while the script inspected nothing at all in the repo it is wired into at -# pre-push. The summary line carries the counts; assert they are non-zero. -# -# BOTH counts, not just the probe count. The `.vale.ini` half of that line was a -# hardcoded `2` in each branch of the summary — true on any clean run, since a -# missing or unreadable file errs out before the summary is reached, but a -# constant states what the author expected rather than what the run inspected, -# and extracting only the probe count left it asserted by nothing. It is computed -# now, so the count is worth reading and worth pinning. -echo "" -echo "--- reports a non-zero number of inspected targets against this repo ---" -REAL_OUT="" -REAL_RC=0 -REAL_OUT="$(bash "$SCRIPT" "$REPO_ROOT" 2>&1)" || REAL_RC=$? -REAL_PROBES="$(printf '%s\n' "$REAL_OUT" | sed -n 's/.*checked, \([0-9][0-9]*\) glob probe(s).*/\1/p')" -REAL_INIS="$(printf '%s\n' "$REAL_OUT" | sed -n 's/.*: \([0-9][0-9]*\) \.vale\.ini file(s) checked.*/\1/p')" -if [[ $REAL_RC -ne 0 ]]; then - fail "exited non-zero against this repo's real Vale copies" - printf '%s\n' "$REAL_OUT" | sed 's/^/ /' -elif [[ -z "$REAL_PROBES" || -z "$REAL_INIS" ]]; then - fail "a clean run against this repo reported no inspected-target counts, so 'it checked something' is unverifiable: $(printf '%s' "$REAL_OUT" | tr '\n' ' ')" -elif [[ "$REAL_PROBES" -lt 1 ]]; then - fail "a clean run against this repo verified $REAL_PROBES glob probes — a pass that inspected nothing" -elif [[ "$REAL_INIS" -lt 2 ]]; then - fail "a clean run against this repo reported $REAL_INIS .vale.ini file(s) checked — both copies' configs must be inspected" -else - pass "inspects $REAL_INIS .vale.ini file(s) and $REAL_PROBES glob probe(s) against this repo, and exits 0" -fi - -# --- 5b. Exits 1 when REPO_ROOT does not exist --- -# A nonexistent path used to fall through to the "neither copy present" no-op -# (test 5 above) and exit 0 — indistinguishable from a real, verified in-sync -# result. That guard is for a repo legitimately missing kyberforge, not a -# typo'd or stale path. -echo "" -echo "--- exits 1 when REPO_ROOT does not exist ---" -if bash "$SCRIPT" "/nonexistent/path/$(date +%s)-$$" > /dev/null 2>&1; then - fail "exited 0 for a nonexistent REPO_ROOT — expected exit 1" -else - pass "exits non-zero for a nonexistent REPO_ROOT" -fi - -# --- 6. Exits 1 when only one of the two copies is present --- -# The no-op guard used `||`, so a single missing copy also exited 0 — a deleted -# or renamed copy passed the sync check silently. -echo "" -echo "--- exits 1 when only one of the two copies is present ---" -FIXTURE6="$(make_fixture)" -FIXTURE7="$(make_fixture)" -FIXTURES+=("$FIXTURE6" "$FIXTURE7") -rm -rf "$FIXTURE6/plugins/kyberforge/.apm/skills/skill-audit" -rm -rf "$FIXTURE7/plugins/kyberforge/.apm/skills/agent-audit" -if bash "$SCRIPT" "$FIXTURE6" > /dev/null 2>&1; then - fail "exited 0 when only agent-audit is present — expected exit 1" -else - pass "exits non-zero when skill-audit's copy is missing but agent-audit's is present" -fi -if bash "$SCRIPT" "$FIXTURE7" > /dev/null 2>&1; then - fail "exited 0 when only skill-audit is present — expected exit 1" -else - pass "exits non-zero when agent-audit's canonical copy is missing but skill-audit's is present" -fi - -# --- 7. Exits 1 when a .vale.ini is missing entirely --- -# Without it vale falls back to an upward config search and lints the file with -# whatever config it happens to find, which is not a failure anyone sees. -echo "" -echo "--- exits 1 when a .vale.ini is missing ---" -FIXTURE8="$(make_fixture)" -FIXTURES+=("$FIXTURE8") -rm -f "$FIXTURE8/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" -if bash "$SCRIPT" "$FIXTURE8" > /dev/null 2>&1; then - fail "exited 0 when skill-audit's .vale.ini is missing — expected exit 1" -else - pass "exits non-zero when a .vale.ini is missing" -fi - -# --- 7b. Exits 1, saying so, when a .vale.ini is present but cannot be read --- -# Every assertion in that loop is a grep, and grep exits 2 on a read error: the -# two `grep -q` checks then misreport a file whose StylesPath and BasedOnStyles -# may be perfectly fine, and the override capture swallows the error into an -# empty result that reads as "no findings". So the exit code alone proves -# nothing here — the check already exits 1 either way, just with the wrong -# reason — and this case asserts the MESSAGE. Deleting the readability guard -# leaves the exit code at 1 and the diagnosis wrong, which is exactly the -# mutation the assertion below kills. -# -# The unreadable path is a DIRECTORY, not a mode-000 file, and that is the whole -# point of the case: `cat` on a directory fails for every uid, while a mode-000 -# file is readable by root, which is what this repo's dev environment and its -# pre-push hooks run as. A permission-based fixture would pass or fail depending -# on the invoking uid; this one does not. -echo "" -echo "--- exits 1 and says so when a .vale.ini exists but cannot be read ---" -FIXTURE8B="$(make_fixture)" -FIXTURES+=("$FIXTURE8B") -UNREADABLE_INI="$FIXTURE8B/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" -rm -f "$UNREADABLE_INI" -mkdir -p "$UNREADABLE_INI" -UNREADABLE_OUT="" -UNREADABLE_RC=0 -UNREADABLE_OUT="$(bash "$SCRIPT" "$FIXTURE8B" 2>&1)" || UNREADABLE_RC=$? -if [[ -e "$UNREADABLE_INI" ]] && cat "$UNREADABLE_INI" >/dev/null 2>&1; then - fail "the fixture's .vale.ini is still readable, so this case proves nothing about the unreadable branch" -elif [[ $UNREADABLE_RC -eq 0 ]]; then - fail "exited 0 when skill-audit's .vale.ini could not be read — expected exit 1" -elif ! printf '%s\n' "$UNREADABLE_OUT" | grep -q "could not be read"; then - fail "failed for the wrong reason on an unreadable .vale.ini — the readability guard did not fire, so the greps misdiagnosed it: $(printf '%s' "$UNREADABLE_OUT" | tr '\n' ' ')" -else - pass "exits non-zero and reports an unreadable .vale.ini as unreadable, not as missing or malformed" -fi - -# --- 8. Exits 1 when the shared StylesPath line is dropped from either copy --- -# StylesPath resolves relative to the .vale.ini, which is the only reason the -# bundled styles are found from a consuming repo's clone prefix. -# Run with vale masked: a dropped StylesPath also stops vale finding the styles, -# so with vale on PATH the glob probe fails too and this case would still exit 1 -# with the StylesPath assertion itself deleted. Masking makes the text assertion -# the only thing that can produce the verdict. -echo "" -echo "--- exits 1 when StylesPath is missing from either .vale.ini ---" -FIXTURE9="$(make_fixture)" -FIXTURE10="$(make_fixture)" -FIXTURES+=("$FIXTURE9" "$FIXTURE10") -break_glob "$FIXTURE9/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \ - 'StylesPath = styles' 'StylesPath = elsewhere' -break_glob "$FIXTURE10/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - 'StylesPath = styles' 'StylesPath = elsewhere' -if run_check_no_vale "$FIXTURE9" > /dev/null 2>&1; then - fail "exited 0 when skill-audit's .vale.ini lost StylesPath — expected exit 1" -else - pass "exits non-zero when skill-audit's .vale.ini lost StylesPath" -fi -if run_check_no_vale "$FIXTURE10" > /dev/null 2>&1; then - fail "exited 0 when agent-audit's .vale.ini lost StylesPath — expected exit 1" -else - pass "exits non-zero when agent-audit's .vale.ini lost StylesPath" -fi - -# --- 9. Exits 1 when no section's BasedOnStyles names Kyberforge --- -# Every rule the prefilter gates on lives in that style, so a section that keeps -# its glob but loses the style lints the file and reports nothing. -echo "" -echo "--- exits 1 when BasedOnStyles no longer names Kyberforge ---" -FIXTURE11="$(make_fixture)" -FIXTURES+=("$FIXTURE11") -break_glob "$FIXTURE11/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - 'BasedOnStyles = Kyberforge' 'BasedOnStyles = KyberforgeCopilot' -if run_check_no_vale "$FIXTURE11" > /dev/null 2>&1; then - fail "exited 0 when agent-audit's .vale.ini stopped naming Kyberforge — expected exit 1" -else - pass "exits non-zero when a .vale.ini no longer names the Kyberforge style" -fi - -# --- 9b. Exits 1 when a per-rule override leaves a rule at anything but error --- -# The third way to switch a rule off without touching a style file or a glob. -# Per ADR-0013, every rule is `level: error` and every alert is a FAIL -- there -# is no ignorable tier. Vale's exit code keys on `error` -# alerts alone, so any such override leaves the glob intact, the styles -# byte-identical, and the run at `0 errors`, exit 0, `Passed`. -# -# Asserted as an ALLOWLIST because that is vale 3.15.2's own semantic, verified -# by enumerating the value space: only the exact tokens `YES` and `error` keep a -# rule blocking. `warning`/`suggestion` downgrade it (alert printed, exit 0 -- -# invisible, since pre-commit swallows a passing hook's output); EVERY other -# value silences it outright, including `false`, `0`, `off`, an empty value, -# `garbage`, and lowercase `yes`. That last one is why a blocklist of -# `NO|warning|suggestion` was not enough: `= yes` reads as "enabled" to a human -# and disables the rule. Case 10's glob probe backstops none of this -- it keys -# on one Kyberforge.VagueWording alert, so DescriptionOpener, PaddingPhrase, -# SentenceOpenerThereIs and ProactivePhrase can each be retired underneath it, -# which is why the cases below deliberately target rules that probe never sees. -# -# Two cases below are about comment forms, and they are NOT symmetric in vale: -# `error # note` (spaced) is stripped by vale and stays live, while `error# note` -# (no space) is not stripped and silences the rule. The gate demands a bare -# token, so it flags both -- deliberately stricter than vale for the spaced form, -# and the only way to catch the no-space form without reimplementing vale's -# comment parsing. `Kyberforge.Vague2` covers rule names carrying a digit: such a -# rule is genuinely silenced by `= NO`, and an alpha-only name class in the gate -# would not even see the line. -echo "" -echo "--- exits 1 when a .vale.ini overrides a Kyberforge rule to anything but YES/error ---" -while IFS= read -r override; do - [[ -n "$override" ]] || continue - # `` stands in for a bare `Rule =` with no value at all, which the - # heredoc cannot carry as a trailing space without a linter eating it. - override="${override//}" - FIXTURE_OV="$(make_fixture)" - FIXTURES+=("$FIXTURE_OV") - echo "$override" >> "$FIXTURE_OV/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" - if run_check_no_vale "$FIXTURE_OV" > /dev/null 2>&1; then - fail "exited 0 with '$override' in skill-audit's .vale.ini -- expected exit 1" - else - pass "exits non-zero on '$override'" - fi -done <<'EOF_OVERRIDES' -Kyberforge.SentenceOpenerThereIs = NO -Kyberforge.VagueWording = warning -Kyberforge.SentenceOpenerThereIs = suggestion -Kyberforge.SentenceOpenerThereIs = false -Kyberforge.DescriptionOpener = 0 -Kyberforge.PaddingPhrase = off -Kyberforge.SentenceOpenerThereIs = yes -Kyberforge.DescriptionOpener = garbage -Kyberforge.PaddingPhrase = -Kyberforge.SentenceOpenerThereIs = NO # keep quiet -Kyberforge.DescriptionOpener = error# silenced, vale strips no comment without a space -Kyberforge.PaddingPhrase = error; silenced too, same no-space rule for ';' -Kyberforge.DescriptionOpener = error # stripped by vale, still rejected: bare token required -Kyberforge.Vague2 = NO -Kyberforge.Vague_2 = NO -Kyberforge.Vague-2 = NO -EOF_OVERRIDES -# Same in agent-audit's copy: the check runs over both .vale.ini files, and a -# rule retired in only the canonical copy is the likelier direction. `= false` -# on ProactivePhrase is the sharpest shape -- one word off the original defect, -# on a KyberforgeCopilot rule no glob probe covers. -FIXTURE_OV_AGENT="$(make_fixture)" -FIXTURES+=("$FIXTURE_OV_AGENT") -echo "KyberforgeCopilot.ProactivePhrase = false" \ - >> "$FIXTURE_OV_AGENT/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" -if run_check_no_vale "$FIXTURE_OV_AGENT" > /dev/null 2>&1; then - fail "exited 0 with 'KyberforgeCopilot.ProactivePhrase = false' in agent-audit's .vale.ini -- expected exit 1" -else - pass "exits non-zero when agent-audit's copy retires a KyberforgeCopilot rule" -fi -# The two allowlisted values must NOT trip the assertion -- otherwise it would -# fire on any legitimate explicit enablement. Kept as a positive case so an -# over-broad tightening of the regex shows up here rather than in the repo. -FIXTURE_OV_OK="$(make_fixture)" -FIXTURES+=("$FIXTURE_OV_OK") -{ - echo "Kyberforge.SentenceOpenerThereIs = YES" - echo "Kyberforge.VagueWording = error" -} >> "$FIXTURE_OV_OK/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" -if bash "$SCRIPT" "$FIXTURE_OV_OK" > /dev/null 2>&1; then - pass "an explicit '= YES' / '= error' override is not flagged" -else - fail "flagged an explicit '= YES' / '= error' override -- those are the two values that keep a rule blocking" - bash "$SCRIPT" "$FIXTURE_OV_OK" 2>&1 | sed 's/^/ /' || true -fi - -# --- 9c. Exits 1 when agent-audit ships KyberforgeCopilot but never loads it --- -# Case 9 asserts only that Kyberforge is named, because skill-audit's copy -# legitimately has no Copilot style. So dropping just `, KyberforgeCopilot` from -# agent-audit's [**/*.agent.md] section unloaded the whole style silently: no -# glob broke, the styles/ diff stayed clean (the directory is still shipped, -# only never loaded), the two .vale.ini files are deliberately unequal so no -# equality check applies, and case 10's probe still passed because it keys on a -# Kyberforge alert. Verified dead by probing a `.agent.md` carrying -# "Use proactively": 0 alerts under the broken config, KyberforgeCopilot. -# ProactivePhrase under the shipped one. ADR-0013 scopes the style to -# `.agent.md` files only, for the Copilot-only 'Use proactively has no effect' -# check, so shipping it unloaded is drift. -echo "" -echo "--- exits 1 when the shipped KyberforgeCopilot style is named by no BasedOnStyles ---" -FIXTURE11C="$(make_fixture)" -FIXTURES+=("$FIXTURE11C") -break_glob "$FIXTURE11C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - 'BasedOnStyles = Kyberforge, KyberforgeCopilot' 'BasedOnStyles = Kyberforge' -if run_check_no_vale "$FIXTURE11C" > /dev/null 2>&1; then - fail "exited 0 when KyberforgeCopilot was dropped from BasedOnStyles -- expected exit 1" -else - pass "exits non-zero when a shipped KyberforgeCopilot style is never loaded" -fi -# The assertion is conditional on the style being shipped: a copy with no -# KyberforgeCopilot directory (skill-audit's, by design) must stay clean -- -# case 13 below covers the shipped-and-loaded pairing. - -# --- 10. Exits 1 when a glob section stops matching the shape its hook lints --- -# One case per glob section, because each covers a file shape the others don't: -# agent-audit's [**/*.agent.md] is the only section covering a Copilot agent file -# outside an agents/ directory, so breaking it alone is invisible to the others. -echo "" -echo "--- exits 1 when a .vale.ini glob no longer matches its hook's file shape ---" -FIXTURE12="$(make_fixture)" -FIXTURE13="$(make_fixture)" -FIXTURE14="$(make_fixture)" -FIXTURES+=("$FIXTURE12" "$FIXTURE13" "$FIXTURE14") -break_glob "$FIXTURE12/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \ - '[**/SKILL.md]' '[**/NOMATCH.md]' -break_glob "$FIXTURE13/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - '[**/agents/*.md]' '[**/NOMATCH-agents/*.md]' -break_glob "$FIXTURE14/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - '[**/*.agent.md]' '[**/*.NOMATCH.md]' -if bash "$SCRIPT" "$FIXTURE12" > /dev/null 2>&1; then - fail "exited 0 when skill-audit's SKILL.md glob matched nothing — expected exit 1" -else - pass "exits non-zero when skill-audit's SKILL.md glob matches nothing" -fi -if bash "$SCRIPT" "$FIXTURE13" > /dev/null 2>&1; then - fail "exited 0 when agent-audit's agents/*.md glob matched nothing — expected exit 1" -else - pass "exits non-zero when agent-audit's agents/*.md glob matches nothing" -fi -if bash "$SCRIPT" "$FIXTURE14" > /dev/null 2>&1; then - fail "exited 0 when agent-audit's *.agent.md glob matched nothing — expected exit 1" -else - pass "exits non-zero when agent-audit's *.agent.md glob matches nothing" -fi - -# --- 10b. Exits 1 when a glob is narrowed to this repo's own plugins/ layout --- -# Every probe path used to start with `plugins/`, so a glob narrowed from a -# filename shape to a location (`[**/SKILL.md]` -> `[**/.apm/skills/*/SKILL.md]`) -# still matched all of them and the check passed -- while a project-scope -# `.claude/skills/foo/SKILL.md` started linting as `0 errors ... in 0 files`, -# exit 0, hook `Passed`: the exact failure the script's own header comment says -# it exists to catch. A `SKILL.md` outside `plugins/` (e.g. project-scope -# `.claude/skills/foo/SKILL.md`) still matches `[**/SKILL.md]` and gets linted -# normally -- the globs constrain filename shape, not location. -# These narrowings are still valid glob syntax and break no `plugins/`-shaped -# file, so only a non-`plugins/` probe path catches them. -echo "" -echo "--- exits 1 when a .vale.ini glob is narrowed from a filename shape to a location ---" -FIXTURE14B="$(make_fixture)" -FIXTURE14C="$(make_fixture)" -FIXTURES+=("$FIXTURE14B" "$FIXTURE14C") -break_glob "$FIXTURE14B/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \ - '[**/SKILL.md]' '[**/.apm/skills/*/SKILL.md]' -break_glob "$FIXTURE14C/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - '[**/agents/*.md]' '[**/.apm/agents/*.md]' -if bash "$SCRIPT" "$FIXTURE14B" > /dev/null 2>&1; then - fail "exited 0 when skill-audit's glob stopped covering a SKILL.md outside plugins/ -- expected exit 1" -else - pass "exits non-zero when skill-audit's glob stops covering a project-scope SKILL.md" -fi -if bash "$SCRIPT" "$FIXTURE14C" > /dev/null 2>&1; then - fail "exited 0 when agent-audit's glob stopped covering an agents/*.md outside plugins/ -- expected exit 1" -else - pass "exits non-zero when agent-audit's glob stops covering a project-scope agents/*.md" -fi - -# --- 11. Exits 1 when a probe path falls out of every hook's `files:` regex --- -# The probe paths are hardcoded, so they can silently stop representing anything -# the hooks lint. Rescoping the shipped agent hook away from the `.agent.md` -# shape has to fail here rather than leave a probe testing a shape no hook -# matches any more. -echo "" -echo "--- exits 1 when a probe path matches no hook's files: regex ---" -FIXTURE16="$(make_fixture)" -FIXTURES+=("$FIXTURE16") -break_glob "$FIXTURE16/.pre-commit-hooks.yaml" \ - "files: '(^|/)agents/[^/]+\\.md\$|\\.agent\\.md\$'" "files: '(^|/)agents/[^/]+\\.md\$'" -if bash "$SCRIPT" "$FIXTURE16" > /dev/null 2>&1; then - fail "exited 0 when the agent hook was rescoped away from .agent.md — expected exit 1" -else - pass "exits non-zero when a probe path is in no hook's scope any more" -fi - -# --- 11b. Exits 1 when the local config's files: regex narrows out of sync -# with the canonical .pre-commit-hooks.yaml regex --- -# hook_file_regexes() used to union the two manifests' `files:` regexes before -# checking probe coverage, so a probe that matched only the old, looser -# .pre-commit-hooks.yaml pattern still passed as "in scope" even after -# .pre-commit-config.yaml's copy of the same hook was narrowed away from it. -# That is exactly the shape of rescoping this repo's own agent hook went -# through (SKILL/agent `.md` -> `.apm/.../*.agent.md`): the local hook quietly -# stopped linting a shape the shipped, external-facing manifest still claims -# to cover, and nothing caught it. Reproduce it directly: narrow only the -# fixture's local config regex (leave .pre-commit-hooks.yaml as shipped) and -# assert the check now flags the disagreement instead of passing silently. -echo "" -echo "--- exits 1 when .pre-commit-config.yaml's files: regex drifts out of sync with .pre-commit-hooks.yaml's ---" -FIXTURE16B="$(make_fixture)" -FIXTURES+=("$FIXTURE16B") -break_glob "$FIXTURE16B/.pre-commit-config.yaml" \ - "files: '^plugins/[^/]+/\\.apm/agents/[^/]+\\.agent\\.md\$'" \ - "files: '^plugins/kyberforge/\\.apm/agents/[^/]+\\.agent\\.md\$'" -if bash "$SCRIPT" "$FIXTURE16B" > /dev/null 2>&1; then - fail "exited 0 when the local config regex narrowed out of sync with .pre-commit-hooks.yaml — expected exit 1" -else - pass "exits non-zero when the local config regex narrows out of sync with the canonical .pre-commit-hooks.yaml regex" -fi - -# --- 12. The text-level assertions hold on a machine without vale --- -# They are the fallback when the glob probe cannot run. With vale on PATH the -# probe fails on these same mutations, so it would mask them: only masking vale -# proves a clean run here means the text assertions themselves ran. -echo "" -echo "--- the StylesPath / BasedOnStyles assertions still gate with vale masked off PATH ---" -VALE_DIR="$(dirname "$(command -v vale 2>/dev/null || echo /nonexistent/vale)")" -PATH_NO_VALE="$(printf '%s' "$PATH" | tr ':' '\n' | grep -vxF "$VALE_DIR" | paste -sd: -)" -if (PATH="$PATH_NO_VALE"; command -v vale >/dev/null 2>&1); then - fail "could not mask vale off PATH — the vale-absent fallback was not exercised" -else - FIXTURE17="$(make_fixture)" - FIXTURE18="$(make_fixture)" - FIXTURE19="$(make_fixture)" - FIXTURES+=("$FIXTURE17" "$FIXTURE18" "$FIXTURE19") - break_glob "$FIXTURE18/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \ - 'StylesPath = styles' 'StylesPath = elsewhere' - break_glob "$FIXTURE19/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" \ - 'BasedOnStyles = Kyberforge' 'BasedOnStyles = KyberforgeCopilot' - # 12a. Missing vale is a HARD FAILURE, not a warning — even on copies that are - # otherwise perfectly in sync. It used to be a warning, and a warning made the - # six glob probes self-disable on the machine that most needed them: applying - # the one-character typo `[**/SKILL.md]` -> `[**/SKILLS.md]` and running with - # vale off PATH exited 0, its sole output a stderr line pre-commit swallows, - # so the pre-push hook reported `Passed`. That is the exact defect the - # glob-coverage section exists to catch, disabled by the absence of the tool - # that catches it. Assert the MESSAGE: exit 1 has a dozen causes here and the - # fixture is in sync, so the code alone would not distinguish this from any - # other finding. - NOVALE_OUT="" - NOVALE_RC=0 - NOVALE_OUT="$(PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE17" 2>&1)" || NOVALE_RC=$? - if [[ $NOVALE_RC -eq 0 ]]; then - fail "exited 0 on in-sync copies with vale unavailable — a run that could not verify glob coverage must not report success" - elif ! printf '%s\n' "$NOVALE_OUT" | grep -q "vale is not installed, so none of the .vale.ini glob-coverage probes ran"; then - fail "failed without vale for the wrong reason — the missing-binary guard did not fire: $(printf '%s' "$NOVALE_OUT" | tr '\n' ' ')" - else - pass "hard-fails, saying so, when vale is unavailable and no opt-out is set" - fi - - # 12b. The opt-out is the only way to get a clean exit without vale, and it has - # to be set deliberately. Absence of the binary must never imply it. - if PATH="$PATH_NO_VALE" CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 \ - bash "$SCRIPT" "$FIXTURE17" > /dev/null 2>&1; then - pass "exits 0 on in-sync copies with vale unavailable and the explicit opt-out set" - else - fail "exited non-zero on in-sync copies with vale unavailable and CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 — the opt-out does not work" - fi - - # 12c/12d. The text assertions still gate under the opt-out. This is what the - # opt-out has to preserve: masking vale makes the assertion under test the only - # thing that can produce the verdict (with vale present, a dropped StylesPath - # also breaks the probe, so these cases would still exit 1 with the assertion - # itself deleted). - if PATH="$PATH_NO_VALE" CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 \ - bash "$SCRIPT" "$FIXTURE18" > /dev/null 2>&1; then - fail "exited 0 on a dropped StylesPath with vale unavailable — expected exit 1" - else - pass "exits non-zero on a dropped StylesPath with vale unavailable" - fi - if PATH="$PATH_NO_VALE" CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 \ - bash "$SCRIPT" "$FIXTURE19" > /dev/null 2>&1; then - fail "exited 0 on a BasedOnStyles that dropped Kyberforge with vale unavailable — expected exit 1" - else - pass "exits non-zero on a BasedOnStyles that dropped Kyberforge with vale unavailable" - fi - - # 12e. An opted-out clean run must still say it verified nothing — otherwise - # the opt-out just reintroduces the silent vacuous pass under a new name. - OPTOUT_OUT="$(PATH="$PATH_NO_VALE" CHECK_VALE_STYLE_SYNC_ALLOW_MISSING_VALE=1 \ - bash "$SCRIPT" "$FIXTURE17" 2>&1)" - if printf '%s\n' "$OPTOUT_OUT" | grep -q "glob coverage was NOT verified" \ - && printf '%s\n' "$OPTOUT_OUT" | grep -q "0 glob probe(s) verified"; then - pass "an opted-out clean run reports that glob coverage was not verified" - else - fail "an opted-out clean run did not say it verified no glob coverage — it looks identical to a verified one: $(printf '%s' "$OPTOUT_OUT" | tr '\n' ' ')" - fi - - # 12f. The typo the whole section exists to catch must fail with vale absent - # and the opt-out set, or not at all — never pass. It cannot be caught without - # vale, so the opt-out must not turn it into a green run by accident: with the - # opt-out this fixture legitimately passes, which is precisely why the opt-out - # is gated on an env var and 12a is the default. - FIXTURE19B="$(make_fixture)" - FIXTURES+=("$FIXTURE19B") - break_glob "$FIXTURE19B/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" \ - '[**/SKILL.md]' '[**/SKILLS.md]' - if PATH="$PATH_NO_VALE" bash "$SCRIPT" "$FIXTURE19B" > /dev/null 2>&1; then - fail "the one-character glob typo exited 0 with vale off PATH — the probe self-disabled on the exact defect it exists to catch" - else - pass "the one-character glob typo does not exit 0 with vale off PATH" - fi -fi - -# --- 13. The intentional agent-audit-only divergence is NOT flagged --- -# The two .vale.ini files are deliberately different: agent-audit ships an extra -# [**/*.agent.md] section and the KyberforgeCopilot style. A check that diffed -# them would fail the repo as it stands, so assert the divergence is really in -# the fixture before asserting the check tolerates it — otherwise this case would -# still pass if the fixture had quietly stopped carrying it. -echo "" -echo "--- exits 0 despite agent-audit's KyberforgeCopilot divergence ---" -FIXTURE15="$(make_fixture)" -FIXTURES+=("$FIXTURE15") -AGENT_INI15="$FIXTURE15/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/.vale.ini" -SKILL_INI15="$FIXTURE15/plugins/kyberforge/.apm/skills/skill-audit/assets/vale/.vale.ini" -if ! grep -q "KyberforgeCopilot" "$AGENT_INI15" \ - || grep -q "KyberforgeCopilot" "$SKILL_INI15" \ - || [[ ! -d "$FIXTURE15/plugins/kyberforge/.apm/skills/agent-audit/assets/vale/styles/KyberforgeCopilot" ]]; then - fail "the fixture no longer carries the agent-audit-only KyberforgeCopilot divergence, so tolerating it proves nothing" -elif bash "$SCRIPT" "$FIXTURE15" > /dev/null 2>&1; then - pass "exits 0 with agent-audit's extra KyberforgeCopilot section and style present" -else - fail "flagged the intentional agent-audit-only KyberforgeCopilot divergence — expected exit 0" - bash "$SCRIPT" "$FIXTURE15" 2>&1 | sed 's/^/ /' || true -fi - -echo "" -echo "Results: $PASS passed, $FAIL failed" -[[ $FAIL -eq 0 ]] diff --git a/tests/test-skill-size-check.sh b/tests/test-skill-size-check.sh index 07088e7..7fabdbd 100755 --- a/tests/test-skill-size-check.sh +++ b/tests/test-skill-size-check.sh @@ -10,15 +10,23 @@ # routing targets. # # The constant-agreement block below is the load-bearing part: all three copies -# (this hook, skill-audit's validate.sh, agent-audit's validate.sh) are +# (this hook, the auditor's skill flow, the auditor's agent flow) are # hand-duplicated because a cache-installed plugin cannot read outside its own # directory, and nothing but these assertions stops them drifting. +# +# ADR-0025 merged skill-audit and agent-audit into factory-audit, which moved two +# of those copies but did not reduce them to one: the constants live in the two +# mode libraries validate.sh sources, and the two libraries still declare them +# separately. So the comparisons below read lib-checks-skill.sh and +# lib-checks-agent.sh directly rather than the entry point, which declares none +# of them — grepping validate.sh would find nothing and report every constant as +# , or worse, silently agree that two empty values match. set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" SCRIPT="$REPO_ROOT/scripts/skill-size-check.sh" -VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh" -AGENT_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh" +VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh" +AGENT_VALIDATE="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh" PASS=0 FAIL=0 @@ -81,26 +89,26 @@ fi MAX_WORDS="$(grep -oE '^MAX_WORDS=[0-9]+' "$SCRIPT" | cut -d= -f2)" MAX_LINES="$(grep -oE '^MAX_LINES=[0-9]+' "$SCRIPT" | cut -d= -f2)" -# The audit (skill-audit/scripts/validate.sh) duplicates both ceilings, because +# The audit (factory-audit's lib-checks-skill.sh) duplicates both ceilings, because # a cache-installed plugin's scripts cannot read files outside the plugin # directory. Nothing but this assertion stops the copies drifting, and drift # means a SKILL.md passes its own audit and is then rejected by the commit hook. echo "" -echo "--- the hook and skill-audit's validate.sh agree on both ceilings ---" +echo "--- the hook and factory-audit's skill checks agree on both ceilings ---" if [[ ! -f "$VALIDATE" ]]; then - fail "skill-audit validate.sh not found at $VALIDATE" + fail "factory-audit lib-checks-skill.sh not found at $VALIDATE" else V_MAX_WORDS="$(grep -oE '^MAX_WORDS = [0-9]+' "$VALIDATE" | grep -oE '[0-9]+')" V_MAX_LINES="$(grep -oE '^MAX_LINES = [0-9]+' "$VALIDATE" | grep -oE '[0-9]+')" if [[ "$V_MAX_WORDS" == "$MAX_WORDS" ]]; then pass "both enforce MAX_WORDS=$MAX_WORDS" else - fail "MAX_WORDS drift: hook says $MAX_WORDS, validate.sh says ${V_MAX_WORDS:-}" + fail "MAX_WORDS drift: hook says $MAX_WORDS, lib-checks-skill.sh says ${V_MAX_WORDS:-}" fi if [[ "$V_MAX_LINES" == "$MAX_LINES" ]]; then pass "both enforce MAX_LINES=$MAX_LINES" else - fail "MAX_LINES drift: hook says $MAX_LINES, validate.sh says ${V_MAX_LINES:-}" + fail "MAX_LINES drift: hook says $MAX_LINES, lib-checks-skill.sh says ${V_MAX_LINES:-}" fi fi @@ -108,11 +116,11 @@ fi # ADR-0020 constants # --------------------------------------------------------------------------- # Three hand-maintained copies, for the same cache-isolation reason as -# MAX_WORDS/MAX_LINES above. skill-audit carries all four; agent-audit carries +# MAX_WORDS/MAX_LINES above. The skill flow carries all four; the agent flow carries # only the two description constants, because ADR-0020 deliberately gives # agents NO body word gate (a skill body competes with the caller's live # conversation; an agent body becomes the system prompt of a fresh context). -# The absence of BODY_* in agent-audit is asserted below so a well-meaning +# The absence of BODY_* in the agent flow is asserted below so a well-meaning # "consistency" edit that adds them fails here rather than contradicting the # ADR silently. DESC_SUGGEST_CHARS="$(grep -oE '^DESC_SUGGEST_CHARS=[0-9]+' "$SCRIPT" | cut -d= -f2)" @@ -132,21 +140,21 @@ for pair in "DESC_SUGGEST_CHARS:$DESC_SUGGEST_CHARS" "DESC_MAX_CHARS:$DESC_MAX_C done echo "" -echo "--- the hook and skill-audit's validate.sh agree on all four ADR-0020 constants ---" +echo "--- the hook and factory-audit's skill checks agree on all four ADR-0020 constants ---" for const in DESC_SUGGEST_CHARS DESC_MAX_CHARS BODY_SUGGEST_WORDS BODY_MAX_WORDS; do hook_value="$(grep -oE "^${const}=[0-9]+" "$SCRIPT" | cut -d= -f2)" audit_value="$(grep -oE "^${const} = [0-9]+" "$VALIDATE" | grep -oE '[0-9]+' || true)" if [[ -n "$hook_value" && "$hook_value" == "$audit_value" ]]; then pass "both enforce $const=$hook_value" else - fail "$const drift: hook says ${hook_value:-}, skill-audit validate.sh says ${audit_value:-}" + fail "$const drift: hook says ${hook_value:-}, lib-checks-skill.sh says ${audit_value:-}" fi done echo "" -echo "--- the hook and agent-audit's validate.sh agree on the description constants ---" +echo "--- the hook and factory-audit's agent checks agree on the description constants ---" if [[ ! -f "$AGENT_VALIDATE" ]]; then - fail "agent-audit validate.sh not found at $AGENT_VALIDATE" + fail "factory-audit lib-checks-agent.sh not found at $AGENT_VALIDATE" else for const in DESC_SUGGEST_CHARS DESC_MAX_CHARS; do hook_value="$(grep -oE "^${const}=[0-9]+" "$SCRIPT" | cut -d= -f2)" @@ -154,15 +162,15 @@ else if [[ -n "$hook_value" && "$hook_value" == "$agent_value" ]]; then pass "both enforce $const=$hook_value" else - fail "$const drift: hook says ${hook_value:-}, agent-audit validate.sh says ${agent_value:-}" + fail "$const drift: hook says ${hook_value:-}, lib-checks-agent.sh says ${agent_value:-}" fi done echo "" - echo "--- agent-audit declares NO body word gate (ADR-0020 is explicit about this) ---" + echo "--- the agent flow declares NO body word gate (ADR-0020 is explicit about this) ---" if grep -qE '^BODY_(SUGGEST|MAX)_WORDS = ' "$AGENT_VALIDATE"; then - fail "agent-audit validate.sh declares a body word gate — ADR-0020 gives agents the description gates and NO body word gate" + fail "lib-checks-agent.sh declares a body word gate — ADR-0020 gives agents the description gates and NO body word gate" else - pass "agent-audit validate.sh declares no BODY_*_WORDS constant" + pass "lib-checks-agent.sh declares no BODY_*_WORDS constant" fi fi @@ -188,7 +196,7 @@ make_line_fixture() { # The line ceiling is inclusive of the limit itself, enforced via `>` — so # exactly $MAX_LINES must pass and $((MAX_LINES + 1)) must fail. This matches -# skill-audit/scripts/validate.sh's `line_count <= 500` pass condition; the two +# factory-audit's lib-checks-skill.sh `line_count <= 500` pass condition; the two # previously disagreed at exactly $MAX_LINES lines, so a SKILL.md could pass its # own audit and still be blocked by the commit hook. echo "" @@ -514,7 +522,7 @@ fi # A skill carrying `disable-model-invocation: true` is removed from the # model-visible listing entirely — it is not preloaded, and the Skill tool # refuses to call it — so its description is never matched against user intent. -# ADR-0020, skill-author Step 2 and skill-audit's own Step 0 all give it ONE +# ADR-0020, skill-author Step 2 and factory-audit's own Gotchas all give it ONE # plain human-facing sentence: no trigger list, no boundary clause. No validator # knew the field existed, so the boundary-clause SUGGESTION fired on exactly the # shape the contract mandates, and its remedy — "so the router knows where NOT diff --git a/tests/test-vale-hooks-consumer.sh b/tests/test-vale-hooks-consumer.sh index e705480..4d8d087 100755 --- a/tests/test-vale-hooks-consumer.sh +++ b/tests/test-vale-hooks-consumer.sh @@ -36,12 +36,16 @@ export PRE_COMMIT_HOME="$WORK/pc-home" mkdir -p "$HOOK_REPO/plugins/kyberforge/.apm/skills" "$HOOK_REPO/scripts" cp "$REPO_ROOT/.pre-commit-hooks.yaml" "$HOOK_REPO/" cp "$REPO_ROOT/scripts/skill-size-check.sh" "$HOOK_REPO/scripts/" -for skill in skill-audit agent-audit; do - mkdir -p "$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill" - cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/scripts" \ - "$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/assets" \ - "$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill/" -done +# One skill since ADR-0025 merged skill-audit and agent-audit into factory-audit, +# and one vale-wrap.sh with it. Both Vale hook IDs still ship and both are still +# registered by the consumer below — they now point at the same entry and differ +# only in their `files:` scope, which is exactly what the per-hook attribution in +# case 1 exists to prove is still true. +skill=factory-audit +mkdir -p "$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill" +cp -R "$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/scripts" \ + "$REPO_ROOT/plugins/kyberforge/.apm/skills/$skill/assets" \ + "$HOOK_REPO/plugins/kyberforge/.apm/skills/$skill/" git -C "$HOOK_REPO" init -q git -C "$HOOK_REPO" add -A git -C "$HOOK_REPO" -c user.email=test@example.invalid -c user.name=test commit -qm "hook repo" diff --git a/tests/test-vale-wrap.sh b/tests/test-vale-wrap.sh index 29688b0..7fd36fc 100755 --- a/tests/test-vale-wrap.sh +++ b/tests/test-vale-wrap.sh @@ -6,23 +6,117 @@ set -euo pipefail REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" -# skill-audit's copy is used here (not agent-audit's) because every fixture below is a -# SKILL.md — only skill-audit's .vale.ini has the [**/SKILL.md] glob section. vale-wrap.sh -# itself is an identical copy in both skills, so which one SCRIPT points at doesn't matter. -SKILL_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/skill-audit" -SCRIPT="$SKILL_AUDIT/scripts/vale-wrap.sh" -VALE_CONFIG="$SKILL_AUDIT/assets/vale/.vale.ini" +# ADR-0025 merged skill-audit and agent-audit, so there is now ONE vale-wrap.sh and +# ONE .vale.ini. Every fixture below is a SKILL.md, matched by the merged config's +# [**/SKILL.md] section — which is the same section the pre-merge skill-audit config +# carried, unchanged, so no fixture's expected verdict moves. +FACTORY_AUDIT="$REPO_ROOT/plugins/kyberforge/.apm/skills/factory-audit" +SCRIPT="$FACTORY_AUDIT/scripts/vale-wrap.sh" +VALE_CONFIG="$FACTORY_AUDIT/assets/vale/.vale.ini" PASS=0 FAIL=0 pass() { echo " PASS: $1"; PASS=$((PASS + 1)); } fail() { echo " FAIL: $1"; FAIL=$((FAIL + 1)); } +# Vale absent skips the Vale-DEPENDENT cases, not the suite. An early `exit 77` +# here used to skip everything, including the checks that are plain greps and +# awk over the config and the two hook manifests (cases 16, 26-28's static +# halves, 31 Parts A/B, 32, 33) -- so a machine without vale reported a skip +# while never looking at a manifest it could have read. Those still run; the +# suite exits 77 at the end only if they all passed, so run-tests.sh keeps +# reporting SKIPPED and `--strict` keeps turning that skip into a failure. A +# static FAIL still exits 1, because a real defect is not a setup error. +VALE_AVAILABLE=true if ! command -v vale &>/dev/null; then - echo "SKIP: vale is not installed — skipping (matches skill-audit/agent-audit's own fallback behavior)" - exit 77 + VALE_AVAILABLE=false + echo "SKIP: vale is not installed — Vale-dependent cases skipped (matches factory-audit's own fallback behavior); the static cases still run" fi +# The fixture handles the skippable regions below create. Seeded empty so the +# EXIT trap's `rm -rf` does not trip `set -u` on a region that never ran. +FIXTURE1="" FIXTURE2="" FIXTURE3="" FIXTURE4="" FIXTURE5="" FIXTURE6="" +FIXTURE7="" FIXTURE8="" FIXTURE10="" FIXTURE11="" FIXTURE12="" STUB13="" +FIXTURE14="" FIXTURE17="" FIXTURE18="" + +# --- 0. The shipped Vale config can load at all ------------------------------ +# A missing `.vale.ini`, a missing `StylesPath`, or a `BasedOnStyles` naming a +# style directory that is not there all stop vale before it lints anything +# (`path ... does not exist`, `style 'Kyberforge' does not exist on +# StylesPath`, rc 2). Every Vale-dependent case below then failed on its own +# generic symptom -- nine "vale printed no summary line" failures across cases +# 28-31 alone, none naming the cause. This names it once and holds the +# Vale-dependent cases back instead. Static on purpose: it needs no vale, so it +# runs on the machines that skip everything else. +# +# `BasedOnStyles =` left EMPTY is deliberately not a defect here: vale loads +# that config and lints the file with no style, which is the silent case 28 +# exists to catch, and catching it here instead would leave 28's +# style-not-loaded branch untested. +vale_config_defects0() { + local cfg="$1" dir sp line names name bad="" + if [[ ! -f "$cfg" ]]; then + printf '%s' "[$cfg does not exist] " + return 0 + fi + # Decided by ACTUALLY READING the file, not by `[[ -r ]]`. `-r` is access(2), + # which answers "would the permission bits allow it" -- and for uid 0 that is + # yes even on a mode-000 file. This repo's dev environment is root, so an + # `[[ ! -r ]]` guard could never fire in the one place it exists to fire: it + # was untestable because it was dead. A read attempt is also the stricter + # question, catching EISDIR and EIO, which access(2) reports on neither. + # `cat`, not a bare `< "$cfg"` redirect: opening a directory for reading + # succeeds, only the read fails. (scripts/check-vale-style-sync.sh carried + # this reasoning before ADR-0025 deleted it; the hazard did not go with it.) + if ! cat "$cfg" > /dev/null 2>&1; then + printf '%s' "[$cfg exists but could not be read] " + return 0 + fi + dir="$(dirname "$cfg")" + sp="$({ grep -E '^[[:space:]]*StylesPath[[:space:]]*=' "$cfg" || true; } | tail -1)" + sp="${sp#*=}" + sp="${sp#"${sp%%[![:space:]]*}"}" + sp="${sp%"${sp##*[![:space:]]}"}" + if [[ -z "$sp" ]]; then + printf '%s' "[${cfg##*/} sets no StylesPath, so vale cannot find any style it names] " + return 0 + fi + [[ "$sp" == /* ]] || sp="$dir/$sp" + if [[ ! -d "$sp" ]]; then + printf '%s' "[StylesPath resolves to $sp, which is not a directory] " + return 0 + fi + while IFS= read -r line; do + names="${line#*=}" + while IFS= read -r name; do + name="${name#"${name%%[![:space:]]*}"}" + name="${name%"${name##*[![:space:]]}"}" + # `Vale` is vale's built-in style and has no directory. + [[ -n "$name" && "$name" != "Vale" ]] || continue + [[ -d "$sp/$name" ]] || bad+="[BasedOnStyles names '$name', but $sp/$name does not exist] " + done <&1) || true } +# ===== BEGIN VALE-DEPENDENT REGION (cases 1-15) ============================== +# Not re-indented, so the case bodies stay diffable against their history. The +# matching `fi` is marked END with the same case range. +if [[ "$VALE_READY" == true ]]; then + # --- 1. A known-bad single-line description is caught (sanity check on Vale itself) --- echo "" echo "--- catches vague wording in a single-line description ---" @@ -219,7 +318,7 @@ echo "--- resolves a cwd-relative --config from a subdirectory (equals and two-a FIXTURE8="$(mktemp -d)" (cd "$FIXTURE8" && git init -q) cp "$VALE_CONFIG" "$FIXTURE8/.vale.ini" -cp -r "$SKILL_AUDIT/assets/vale/styles" "$FIXTURE8/styles" +cp -r "$FACTORY_AUDIT/assets/vale/styles" "$FIXTURE8/styles" mkdir -p "$FIXTURE8/plugins/testplugin/skills/zzzskill" { echo "---" @@ -432,6 +531,9 @@ else fail "a path with a space was dropped from the directory walk" fi +fi +# ===== END VALE-DEPENDENT REGION (cases 1-15) ================================ + # --- 16. No unguarded `"${arr[@]}"` expansion survives in any script that runs # on macOS. bash before 4.4 — including the 3.2 that macOS still ships as # /bin/bash — treats that form on an *empty* array as an unbound variable under @@ -618,8 +720,14 @@ bash32_glob() { # script trips this assertion — correct, because at that point the glob is dead # weight and should be deleted from the table deliberately, not left to pass # vacuously. +# +# `scripts` dropped from 10 to 8 with ADR-0025: the merge retired +# scripts/check-vale-style-sync.sh and scripts/sync-vale-styles.sh, both of which +# existed only to keep two copies of the Vale config in step, leaving 9 files. +# The floor moves with the count on a deliberate deletion — it is a guard against +# a broken or renamed PATH resolving to nothing, never a headcount to maintain. BASH32_GLOB_NAMES=(scripts tests plugins providers) -BASH32_GLOB_FLOORS=(10 14 10 1) +BASH32_GLOB_FLOORS=(8 14 10 1) BASH32_SCRIPTS=() BASH32_IDX=0 while [[ $BASH32_IDX -lt ${#BASH32_GLOB_NAMES[@]} ]]; do @@ -671,6 +779,9 @@ else pass "all ${#BASH32_SCRIPTS[@]} scanned scripts are free of every bash-4-only construct this case checks for" fi +# ===== BEGIN VALE-DEPENDENT REGION (cases 17-18) ============================= +if [[ "$VALE_READY" == true ]]; then + # --- 17. The invocations whose arrays are closest to empty actually run. Under # a bash older than 4.4 this is genuine macOS-shell coverage; on a modern bash it # degrades to a smoke test, so the pass message names the shell that really ran. @@ -723,6 +834,9 @@ else fail "a path argument with a space was split by the array expansion: $OUT18" fi +fi +# ===== END VALE-DEPENDENT REGION (cases 17-18) =============================== + # The cases below share one cleanup list. The per-case trap rebuilding above # does not scale past the fixture count it already carries, and this trap is # installed last, so it is the one that runs. @@ -736,6 +850,11 @@ cleanup_all() { } trap cleanup_all EXIT +# ===== BEGIN VALE-DEPENDENT REGION (cases 19-25) ============================= +# Case 25 is static, but it audits the fixtures case 19 creates, so it cannot +# run without 19 and is held back with it. +if [[ "$VALE_READY" == true ]]; then + # --- 19. Every YAML form whose parsed value is joined back out of 2+ physical # lines breaks the `text.frontmatter.description` scope identically, not just # the `>` folded block the flattener originally handled: a plain scalar wrapped @@ -1067,6 +1186,9 @@ else pass "all ${#FORM_FIXTURES19[@]} form fixtures are registered for cleanup" fi +fi +# ===== END VALE-DEPENDENT REGION (cases 19-25) =============================== + # --- 26. Case 16's shell-special-array exemption is pinned to a fixture. No file # the scan currently reads expands any of those arrays, so the exemption is inert # in practice: it could be deleted, or quietly widened to cover an array that can @@ -1196,6 +1318,1201 @@ else pass "the seeding exemption switches on only for resolvable directives, and every directive in the scanned corpus resolves" fi +# --- 28-30. `.vale.ini` glob coverage --------------------------------------- +# +# Rehomed from scripts/check-vale-style-sync.sh, deleted by ADR-0025. That +# script's text-level assertions diffed skill-audit's Vale copy against +# agent-audit's and went moot when the merge into factory-audit left one copy. +# Its GLOB-COVERAGE probes did not. Merging the copies does not make a glob typo +# impossible -- a typo in any one of the three sections of the surviving +# `.vale.ini` still skips that file shape -- and this file, which already owns +# the wrapper's behaviour against that same config, is where they belong. +# +# The original comment, preserved because it names the failure mode rather than +# describing the check: nothing in the repo read the `.vale.ini` at all, "and +# that is what let a one-character glob typo silently disable the prefilter for +# a whole file type: the hook still MATCHES the file via its `files:` regex, so +# pre-commit reports neither `Skipped` nor an error; vale lints zero files, +# prints `0 errors ... in 1 file` and exits 0, and the hook shows `Passed`." +# Measured again against the merged config while rehoming these: a mutated +# `[**/SKILL.md]` -> `[**/SKILLS.md]` reports `0 errors ... in 0 files` and +# exits 0. Every gate in this repo reads that as a pass. +# +# ADR-0014's reason for two hook IDs was this same problem, and .pre-commit- +# hooks.yaml still carries both IDs after the merge for that reason. + +# One representative path per file shape the prefilter is supposed to cover, +# tagged with the `.vale.ini` section that is supposed to cover it and with +# whether that path is covered by that section ALONE. +# +# `isolating` is load-bearing, not decoration. `.apm/agents/demo.agent.md` +# matches BOTH `[**/agents/*.md]` and `[**/*.agent.md]`, so breaking either one +# leaves it linted by the other and a probe on it alone would prove nothing +# about which section is live. `copilot/demo.agent.md` -- a Copilot agent file +# outside any `agents/` directory -- is what pins `[**/*.agent.md]` on its own, +# and bare `demo.md` under `agents/` is what pins `[**/agents/*.md]`. Case 29 +# asserts that separation instead of trusting this column. +# +# The two `.claude/`-prefixed rows carry the location-independence property +# (original comment, unchanged in substance): a `SKILL.md` outside `plugins/` +# still matches `[**/SKILL.md]` and gets linted normally -- the globs constrain +# filename shape, not location. Every other row starts with `plugins/`, so +# narrowing a glob to a `plugins/`-shaped path (`[**/SKILL.md]` -> +# `[**/.apm/skills/*/SKILL.md]`) left all of them matching while the +# project-scope shape started linting as `0 errors ... in 0 files`. +# +# `demo.md` (bare, no `.agent.md` suffix) exercises `[**/agents/*.md]` in +# isolation, not because any current `.apm/agents/*` file has that shape -- per +# ADR-0016 they are all `*.agent.md`. `.pre-commit-hooks.yaml`'s agent regex +# still covers the bare shape, which is what keeps the row honest. +PROBE_TABLE28="$(cat <<'EOF_PROBE28' +plugins/demo/.apm/skills/demo/SKILL.md|[**/SKILL.md]|isolating +.claude/skills/demo/SKILL.md|[**/SKILL.md]|isolating +plugins/demo/.apm/agents/demo.md|[**/agents/*.md]|isolating +.claude/agents/demo.md|[**/agents/*.md]|isolating +plugins/demo/.apm/agents/demo.agent.md|[**/*.agent.md]|overlapping +copilot/demo.agent.md|[**/*.agent.md]|isolating +EOF_PROBE28 +)" + +VALE_ASSETS28="$FACTORY_AUDIT/assets/vale" + +# The probe file carries a description with a token Kyberforge.VagueWording +# flags, so a config whose glob matches but whose BasedOnStyles lost Kyberforge +# fails too: it would lint the file and report nothing. `Use proactively` is +# there for case 30 and is inert for the other two cases -- no Kyberforge rule +# matches it, only KyberforgeCopilot.ProactivePhrase does. +PROBE_DESC28="Use when the caller wants a probe that helps with things. Use proactively." + +# One tree holding every probe path, reused by all three cases. The config is +# always passed as an absolute path from outside the tree, so one tree serves +# the real config and the mutated copies alike. +build_probe_tree28() { + local dir rel + dir="$(mktemp -d)" + while IFS='|' read -r rel _ _; do + [[ -n "$rel" ]] || continue + mkdir -p "$dir/$(dirname "$rel")" + { + echo "---" + echo "name: probe" + echo "description: $PROBE_DESC28" + echo "---" + echo "" + echo "Body." + } > "$dir/$rel" + done <&1) | sed -E 's/\x1b\[[0-9;]*m//g'; } || true +} + +# The file count out of vale's own summary line, e.g. `... in 0 files.` Empty +# output means no summary line at all, which the callers treat as a failure +# rather than as zero -- vale not running and vale scanning nothing are +# different defects and must not report the same way. +files_scanned28() { + printf '%s\n' "$1" \ + | { grep -oE 'in [0-9]+ files?\.' || true; } \ + | { grep -oE '[0-9]+' || true; } \ + | tail -1 +} + +# Prints `|` for every hook in the given pre-commit manifest +# whose entry is factory-audit's vale-wrap.sh -- either manifest, since the two +# carry the same two hooks in the same shape. Records are delimited by their +# `- id:` line, so this does not depend on `entry:` preceding `files:` within a +# record. Case 28 wants the regexes alone and case 32 needs to know which hook +# each belongs to, so the id is carried here and dropped by the wrapper below. +hook_records28() { + local manifest="$1" id raw + [[ -f "$manifest" ]] || return 0 + awk ' + function flush() { + if (entry ~ /factory-audit\/scripts\/vale-wrap\.sh/ && files != "") print id "|" files + id = ""; entry = ""; files = "" + } + /^[ \t]*-[ \t]*id:/ { flush(); id = $0; sub(/^[ \t]*-[ \t]*id:[ \t]*/, "", id) } + /^[ \t]*entry:/ { entry = $0 } + /^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) } + END { flush() } + ' "$manifest" | while IFS='|' read -r id raw; do + # Strip the surrounding YAML quotes; the regex itself never carries them. + # `read -r id raw` splits on the FIRST `|` only, so a regex containing an + # alternation survives intact in $raw. + raw="${raw%\'}"; raw="${raw#\'}" + raw="${raw%\"}"; raw="${raw#\"}" + printf '%s|%s\n' "$id" "$raw" + done +} + +# Prints the `files:` regex of every hook in the given manifest whose entry is +# factory-audit's vale-wrap.sh. +hook_file_regexes28() { + hook_records28 "$1" | cut -d'|' -f2- +} + +matches_any_regex28() { + local rel="$1" regexes="$2" re + [[ -n "$regexes" ]] || return 1 + while IFS= read -r re; do + [[ -n "$re" ]] || continue + if printf '%s\n' "$rel" | grep -Eq "$re"; then + return 0 + fi + done <|` or `FAIL||` line per probe row, for +# the config at $1. A function rather than an inline loop so Part B can hold a +# mutated copy to this exact logic -- a second, "equivalent" loop for the +# fixture would prove nothing about the live check. With $2 = false only the +# hook-scope half runs: that half reads .pre-commit-hooks.yaml, not vale, so a +# machine without vale still gets it. +probe_coverage28() { + local cfg="$1" with_vale="$2" rel sec report count + while IFS='|' read -r rel sec _; do + [[ -n "$rel" ]] || continue + if ! matches_any_regex28 "$rel" "$HOOK_REGEXES28"; then + # Original wording: the probe path is stale, or the hook was rescoped away + # from a shape it still needs to lint. Either way the row below stops + # describing anything the push gate actually hands to vale. + echo "FAIL|$rel|$rel matches no 'files:' regex of any factory-audit vale hook in .pre-commit-hooks.yaml — the probe path is stale, or the hook was rescoped away from a shape it still needs to lint" + continue + fi + if [[ "$with_vale" != true ]]; then + echo "PASS|$rel|$rel is in scope of a published factory-audit vale hook (glob coverage not checked: Vale-dependent half held back)" + continue + fi + report="$(vale_report28 "$cfg" "$TREE28" "$rel")" + count="$(files_scanned28 "$report")" + if [[ -z "$count" ]]; then + echo "FAIL|$rel|vale printed no summary line for $rel, so it is not known whether anything was scanned: ${report:-}" + elif [[ "$count" -eq 0 ]]; then + echo "FAIL|$rel|$sec scanned 0 files for $rel — the section's glob covers no path of that shape, so vale exits 0 and every gate reads it as a pass" + elif ! printf '%s\n' "$report" | grep -qF "Kyberforge.VagueWording"; then + echo "FAIL|$rel|$sec scanned $rel but raised no Kyberforge alert — the glob matches but the style is not loaded, which lints the file and reports nothing" + else + echo "PASS|$rel|$sec scans $rel ($count file) and raises a Kyberforge alert" + fi + done < "$SDIR28/.vale.ini" + if cmp -s "$VALE_ASSETS28/.vale.ini" "$SDIR28/.vale.ini"; then + STYLE_MUT_FAILS28+="[$MSEC28: the mutation left the config unchanged, so nothing was tested] " + continue + fi + RESULTS28="$(probe_coverage28 "$SDIR28/.vale.ini" true)" + while IFS='|' read -r REL28 ROWSEC28 ISO28; do + [[ -n "$REL28" && "$ISO28" == "isolating" ]] || continue + LINE28="$(printf '%s\n' "$RESULTS28" | { grep -F "|$REL28|" || true; } | head -1)" + if [[ "$ROWSEC28" == "$MSEC28" ]]; then + printf '%s\n' "$LINE28" | grep -qF "but raised no Kyberforge alert" \ + || STYLE_MUT_FAILS28+="[$MSEC28 lost Kyberforge but $REL28 did not fail as style-not-loaded: ${LINE28:-}] " + else + [[ "$LINE28" == PASS\|* ]] \ + || STYLE_MUT_FAILS28+="[$MSEC28 lost Kyberforge and unrelated probe $REL28 stopped passing, so the mutation was not confined to one section: ${LINE28:-}] " + fi + done < + # `[**/SKILLZZ.md]`, the same one-token shape as the `SKILL.md`/`SKILLS.md` + # typo the original probes were written for. Applied to the header line only, + # so BasedOnStyles and StylesPath are untouched and the mutation isolates the + # glob. + MUTATED29="$(printf '%s\n' "$SEC29" | sed -E 's/\.([^.]*)\]$/ZZ.\1]/')" + if [[ "$MUTATED29" == "$SEC29" ]]; then + MUTATION_FAILS29+="[$SEC29: the mutation rule left the header unchanged, so nothing was tested] " + continue + fi + # Exact-line replacement via awk: the header is a bracket expression, so a + # sed pattern built from it would be read as a character class. + awk -v old="$SEC29" -v new="$MUTATED29" '$0 == old { print new; next } { print }' \ + "$VALE_ASSETS28/.vale.ini" > "$DIR29/.vale.ini" + if ! grep -qF -- "$MUTATED29" "$DIR29/.vale.ini"; then + MUTATION_FAILS29+="[$SEC29: the mutated header never reached the copied config] " + continue + fi + while IFS='|' read -r REL29 ROWSEC29 ISO29; do + [[ -n "$REL29" ]] || continue + COUNT29="$(files_scanned28 "$(vale_report28 "$DIR29/.vale.ini" "$TREE28" "$REL29")")" + [[ -n "$COUNT29" ]] || { MUTATION_FAILS29+="[$SEC29 broken: no vale summary for $REL29] "; continue; } + if [[ "$ROWSEC29" == "$SEC29" && "$ISO29" == "isolating" ]]; then + [[ "$COUNT29" -eq 0 ]] || MUTATION_FAILS29+="[$SEC29 broken but $REL29 still scanned $COUNT29 file(s), so this section's probes cannot detect a typo in it] " + else + [[ "$COUNT29" -gt 0 ]] || MUTATION_FAILS29+="[$SEC29 broken and unrelated probe $REL29 stopped scanning, so the mutation was not confined to one section] " + fi + done < "$UNLOAD30/.vale.ini" +LEAK30="$(mktemp -d)" +new_fixture "$LEAK30" +cp -r "$VALE_ASSETS28/." "$LEAK30/" +rewrite_styles30 "$VALE_ASSETS28/.vale.ini" "[**/SKILL.md]" "Kyberforge, KyberforgeCopilot" > "$LEAK30/.vale.ini" +UNLOAD_FAILS30="$(copilot_scope_failures30 "$UNLOAD30/.vale.ini")" +LEAK_FAILS30="$(copilot_scope_failures30 "$LEAK30/.vale.ini")" +if cmp -s "$VALE_ASSETS28/.vale.ini" "$UNLOAD30/.vale.ini" || cmp -s "$VALE_ASSETS28/.vale.ini" "$LEAK30/.vale.ini"; then + fail "a Copilot-scope mutation left the copied config unchanged, so Part B mutated nothing and proves nothing about Part A" +elif ! printf '%s' "$UNLOAD_FAILS30" | grep -qF "[copilot/demo.agent.md is an agent file but raised no ProactivePhrase alert"; then + fail "dropping KyberforgeCopilot from [**/*.agent.md] did not fail Part A, so an unloaded Copilot style would pass silently again: ${UNLOAD_FAILS30:-}" +elif ! printf '%s' "$LEAK_FAILS30" | grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is not an .agent.md file but raised a ProactivePhrase alert"; then + fail "adding KyberforgeCopilot to [**/SKILL.md] did not fail Part A, so the style could leak past ADR-0013's scope unnoticed: ${LEAK_FAILS30:-}" +else + pass "unloading KyberforgeCopilot from .agent.md files and leaking it onto SKILL.md files are each caught by Part A" +fi +fi +fi + +# --- 31. No Kyberforge rule is overridden out of its blocking level ---------- +# +# Also rehomed from the deleted scripts/check-vale-style-sync.sh (ADR-0025). +# This one is not a glob probe and not a copy diff: it is a third class the +# merge took out with the script, and cases 28-30 cannot backstop it. They key +# on `Kyberforge.VagueWording` and `KyberforgeCopilot.ProactivePhrase`, so +# DescriptionOpener, PaddingPhrase, SentenceOpenerThereIs and CompositionNote +# can each be retired underneath a passing probe. +# +# The original's comments, verbatim: +# +# Per-rule overrides are the third way to retire a rule without touching a +# style file or a glob. Per ADR-0013, every rule is `level: error` and every +# alert is a FAIL — there is no ignorable tier. Vale's exit +# code keys on `error` alerts alone, so any override that leaves a rule at +# anything other than `error` still lints the file, still exits 0, and still +# shows `Passed` in pre-commit. The glob probe below cannot backstop this: it +# keys on one `Kyberforge.VagueWording` alert, so DescriptionOpener, +# PaddingPhrase, SentenceOpenerThereIs and ProactivePhrase can each be retired +# underneath a passing probe. +# +# Asserted as an ALLOWLIST, not a blocklist of `NO|warning|suggestion`, because +# that is vale 3.15.2's own semantic: only the exact tokens `YES` and `error` +# keep a rule blocking. `warning`/`suggestion` downgrade it (alert still +# printed, exit 0 — invisible, since pre-commit swallows a passing hook's +# output); every other value — `NO`, `false`, `0`, `off`, `n`, empty, +# `garbage`, and lowercase `yes`, `true`, `1`, `on` — silences the rule +# outright. Lowercase `yes` is the trap a blocklist cannot cover: it reads as +# "enabled" to a human and disables the rule. Verified by enumerating the +# value space against vale 3.15.2. +# +# The allowlist demands a BARE `YES`/`error` with nothing after it, which also +# rejects `error # note` and `error ; note`. Vale itself strips those — a +# whitespace-preceded `#` or `;` comment is removed and the rule stays live — +# so rejecting them is deliberately stricter than vale, not a workaround for +# it. Uniformity is worth more here than the ability to annotate a line that +# should not exist: no shipped `.vale.ini` has any override line at all, and +# the failure mode is a loud false positive rather than a silent pass. The +# genuine hazard is the no-space form — `error# note` and `error; note` are +# NOT stripped and silence the rule outright — and a rule that demands a bare +# token catches those without having to reimplement vale's comment parsing. +# +# `[A-Za-z0-9_-]` on both halves of the name, not `[A-Za-z]`: a rule named +# `Kyberforge.Vague2` is genuinely silenced by `= NO` (verified: 1 error -> +# 0 errors), so an alpha-only class would let a digit-bearing rule name slip +# past the gate. All five current rule names are pure alpha, so this is +# forward cover, not a live hole. + +# The detector, with the original's two regexes unchanged. Part A runs it +# against the real config and Part B runs it against synthesised lines, so the +# thing Part B proves is the same thing Part A relies on — a second, "equivalent" +# copy of the regex for the fixtures would prove nothing about the live check. +# +# Original comment on the missing `-q`, kept because it names the trap: No +# `grep -q` in that pipeline on purpose: `-q` exits on its first match, and +# under `set -o pipefail` the resulting SIGPIPE on the upstream grep would make +# the whole pipeline report 141 and read as "no findings". +bad_overrides31() { + grep -E '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=' "$1" \ + | grep -Ev '^[[:space:]]*Kyberforge[A-Za-z0-9_-]*\.[A-Za-z0-9_-]+[[:space:]]*=[[:space:]]*(YES|error)[[:space:]]*$' \ + || true +} + +echo "" +echo "--- no .vale.ini override downgrades a Kyberforge rule out of its blocking level ---" + +# Part A: the live assertion, against the shipped config. A plain grep, so it +# runs without vale; held back only when case 0 found the config missing or +# unreadable, where grepping it would read as "no bad override" and pass. +BAD31="" +[[ "$VALE_CONFIG_OK" != true ]] || BAD31="$(bad_overrides31 "$VALE_ASSETS28/.vale.ini")" +if [[ "$VALE_CONFIG_OK" != true ]]; then + : +elif [[ -n "$BAD31" ]]; then + fail ".vale.ini overrides a Kyberforge rule to something other than a bare YES or error (first: '${BAD31%%$'\n'*}') — every rule in this prefilter is level: error and every alert is a FAIL, and any other value downgrades or silences the rule while vale still exits 0. A trailing comment is rejected too: vale strips a spaced '# ...' but not 'error# ...', so this asks for the bare token rather than guessing which form you meant" +else + pass "the shipped .vale.ini carries no Kyberforge rule override outside the bare YES/error allowlist" +fi + +# Part B: the detector is held to every rule that actually ships, enumerated +# from the style directories on disk rather than from a list in this file. A +# list would have to be remembered; the styles directory cannot be forgotten, +# because adding a rule IS adding a file to it. This is what makes Part A +# forward-covering: a rule added later under a style whose name the detector's +# `Kyberforge[A-Za-z0-9_-]*` class does not match fails here the day it lands, +# rather than being silently un-gated. +RULES31="$( + for RULE_FILE31 in "$VALE_ASSETS28"/styles/*/*.yml "$VALE_ASSETS28"/styles/*/*.yaml; do + [[ -f "$RULE_FILE31" ]] || continue + STYLE_DIR31="${RULE_FILE31%/*}" + RULE_BASE31="${RULE_FILE31##*/}" + RULE_BASE31="${RULE_BASE31%.yml}" + printf '%s.%s\n' "${STYLE_DIR31##*/}" "${RULE_BASE31%.yaml}" + done +)" +RULE_COUNT31="$(printf '%s\n' "$RULES31" | grep -c . || true)" + +# Both halves are asserted. A detector that flags everything would pass the +# reject rows while making Part A a permanent false alarm; one that flags +# nothing would pass the accept rows while making Part A vacuous. Only holding +# it to both tells the two apart. +VARIANTS31="$(cat <<'EOF_VAR31' +reject|NO +reject|warning +reject|suggestion +reject|yes +reject|true +reject| +reject|error # note +reject|error# note +reject|error; note +accept|YES +accept|error +EOF_VAR31 +)" + +DETECTOR_FAILS31="" +SYNTH31="$(mktemp -d)" +new_fixture "$SYNTH31" +if [[ "$RULE_COUNT31" -eq 0 ]]; then + fail "no rule file was found under $VALE_ASSETS28/styles/*/ — the enumeration is empty, so Part B verifies nothing and Part A's forward cover is unproven" +else + while IFS= read -r RULE31; do + [[ -n "$RULE31" ]] || continue + while IFS='|' read -r VERDICT31 VALUE31; do + [[ -n "$VERDICT31" ]] || continue + # Written under a real section header so the fixture is a config a human + # could plausibly ship, not a bare fragment. + { + echo "StylesPath = styles" + echo "" + echo "[**/SKILL.md]" + echo "BasedOnStyles = Kyberforge" + echo "$RULE31 = $VALUE31" + } > "$SYNTH31/probe.ini" + GOT31="$(bad_overrides31 "$SYNTH31/probe.ini")" + if [[ "$VERDICT31" == "reject" && -z "$GOT31" ]]; then + DETECTOR_FAILS31+="[$RULE31 = '$VALUE31' was NOT flagged, so this value could silence or downgrade the rule with nothing noticing] " + elif [[ "$VERDICT31" == "accept" && -n "$GOT31" ]]; then + DETECTOR_FAILS31+="[$RULE31 = '$VALUE31' WAS flagged, so the allowlist rejects a value that keeps the rule blocking] " + fi + done < 0 errors"); asserting +# it here means the case cannot quietly degrade into guarding a failure mode +# vale no longer has. One rule is named deliberately rather than looping the +# enumeration: the mechanism under test is vale's override handling, which is +# not per-rule, and VagueWording is the rule the shared probe description +# already trips, so no second fixture shape is needed. +# +# `copilot/demo.agent.md` because the override is appended at the end of the +# file, which places it in the last section — `[**/*.agent.md]`. +if [[ "$VALE_READY" == true ]]; then +OVERRIDE_DIR31="$(mktemp -d)" +new_fixture "$OVERRIDE_DIR31" +cp -r "$VALE_ASSETS28/." "$OVERRIDE_DIR31/" +printf 'Kyberforge.VagueWording = NO\n' >> "$OVERRIDE_DIR31/.vale.ini" +SILENCED31="$(vale_report28 "$OVERRIDE_DIR31/.vale.ini" "$TREE28" "copilot/demo.agent.md")" +SILENCED_COUNT31="$(files_scanned28 "$SILENCED31")" +if [[ -z "$SILENCED_COUNT31" || "$SILENCED_COUNT31" -eq 0 ]]; then + fail "the override fixture scanned no file at all, so the disappearance of the VagueWording alert proves nothing about overrides" +elif printf '%s\n' "$SILENCED31" | grep -qF "Kyberforge.VagueWording"; then + # Reported as a FAIL, not a pass. It is not a defect in the config, but it + # means Parts A and B are guarding a failure mode this vale build no longer + # has — and a guard that guards nothing while reporting PASS is the same + # vacuous pass the whole case exists to close. Someone has to look. + fail "an '= NO' override no longer silences the rule in this vale build, so the allowlist in Parts A and B is guarding a failure mode that no longer exists — re-verify against this vale version before trusting or removing it" +else + pass "an '= NO' override silences a rule while vale still scans the file and exits 0, which is the silent downgrade the allowlist above exists to catch" +fi +fi + +# --- 32. Every vale prefilter hook in .pre-commit-config.yaml still selects a +# live, correctly-classed corpus ---------------------------------------------- +# +# The one assertion of the deleted scripts/check-vale-style-sync.sh (ADR-0025) +# that cases 28-31 did not rehome, re-scoped onto what is actually at risk. The +# original compared the `files:` regexes across the two manifests, selecting each +# hook by its `entry:` (`entry ~ skill "/scripts/vale-wrap.sh"`, old line 220) -- +# which worked only because the two skills gave the two hooks two distinct entry +# paths. After the merge both hooks share one `entry:`, so that selector can no +# longer tell them apart and a faithful port would have to key on hook `id:` +# instead. Case 33 is that port; this case covers the separate question of +# whether each local hook selects a live corpus at all. The zero-match half of the hole stands on its own, and nothing +# else in the repo covers it: tests/test-vale-hooks-consumer.sh synthesises its +# own consumer config out of `.pre-commit-hooks.yaml` and never reads the local +# one, and cases 28-30 read `.pre-commit-hooks.yaml` too. This repo's OWN +# prefilter regexes -- `.pre-commit-config.yaml`'s vale-audit-prefilter-skill and +# vale-audit-prefilter-agent -- are therefore asserted by no test at all. Narrow +# either one to match zero files and every gate still passes: pre-commit does not +# error on a hook that matches nothing, it simply never runs it. That is the same +# silent-zero failure mode case 28 guards on the vale side of this pipeline, one +# layer up -- there the glob scans 0 files and exits 0, here the hook is handed 0 +# files and never starts. +# +# Two properties, because matching SOMETHING is not the same as matching the +# right thing: a regex loosened to `^plugins/` would match hundreds of files and +# clear a bare non-emptiness check while handing vale a corpus it has no glob +# for. So each hook must also select only its own artifact class -- ADR-0014's +# reason for two hook IDs, carried across the merge by ADR-0025's comment in the +# config, is precisely that the two scopes stay independently addressable. + +echo "" +echo "--- each .pre-commit-config.yaml vale prefilter hook matches a real, correctly-classed file ---" + +PC_CONFIG32="$REPO_ROOT/.pre-commit-config.yaml" +# The corpus pre-commit itself draws from. A `files:` regex is matched against +# repository paths, so a regex matching no tracked path matches nothing the gate +# will ever hand the hook. +REPO_FILES32="$(cd "$REPO_ROOT" && git ls-files)" + +# Prints one failure token per defect; empty output means every prefilter hook in +# $1 is scoped to a live corpus of its own artifact class. The config path is an +# argument so Part B can run this exact function against a mutated copy -- a +# second, "equivalent" implementation for the fixture would prove nothing about +# the live check. +prefilter_scope_failures32() { + local config="$1" files="$2" + local records id re class matched count offenders offending m + local seen_skill=false seen_agent=false bad="" + records="$(hook_records28 "$config")" + if [[ -z "$records" ]]; then + printf '%s' "[no hook whose entry is factory-audit's vale-wrap.sh was parsed out of ${config##*/} at all, so nothing below was checked] " + return 0 + fi + while IFS='|' read -r id re; do + [[ -n "$id" ]] || continue + case "$id" in + *-skill) class="skill" ;; + *-agent) class="agent" ;; + *) + bad+="[$id is a vale prefilter hook whose id names neither artifact class, so this case cannot tell which corpus it is meant to select] " + continue + ;; + esac + # Recorded as soon as the class is known, not at the end of the iteration: + # a hook that fails a check below is still a hook that EXISTS, and reporting + # it as missing as well would bury the real defect under a second message. + [[ "$class" != skill ]] || seen_skill=true + [[ "$class" != agent ]] || seen_agent=true + matched="$(printf '%s\n' "$files" | { grep -E "$re" || true; })" + count="$(printf '%s\n' "$matched" | grep -c . || true)" + if [[ "$count" -eq 0 ]]; then + bad+="[$id: 'files: $re' matches no tracked file in this repo, so pre-commit never runs it and the prefilter is off for the whole $class corpus while every gate still reports a pass] " + continue + fi + offenders="" + offending=0 + while IFS= read -r m; do + [[ -n "$m" ]] || continue + # Per ADR-0016 every agent file is `.agent.md` under an `agents/` + # directory; the skill corpus is SKILL.md files. Asserting the shape of + # every matched path is what rejects a regex loosened to a wider scope, + # and it makes the two corpora disjoint by construction. + if [[ "$class" == skill ]]; then + [[ "$m" != */SKILL.md ]] || continue + else + [[ "$m" != */agents/*.agent.md ]] || continue + fi + offending=$((offending + 1)) + # A loosened regex can select hundreds of paths; three name the shape of + # the leak, and the count carries the scale. + [[ "$offending" -gt 3 ]] || offenders+="$m " + done < "$MUT32/.pre-commit-config.yaml" +MUT_RECORDS32="$(hook_records28 "$MUT32/.pre-commit-config.yaml")" +MUT_FAILS32="$(prefilter_scope_failures32 "$MUT32/.pre-commit-config.yaml" "$REPO_FILES32")" +if ! printf '%s\n' "$MUT_RECORDS32" | grep -q 'zzz-no-such-path'; then + fail "the narrowed regexes never reached the copied config, so Part B narrowed nothing and proves nothing about Part A" +elif [[ "$(printf '%s\n' "$MUT_RECORDS32" | grep -c .)" -ne 2 ]]; then + fail "the mutated config did not parse back as two prefilter hooks, so any failure below would come from the parser, not from the narrowing" +elif ! printf '%s' "$MUT_FAILS32" | grep -qF "vale-audit-prefilter-skill: 'files: ^zzz-no-such-path/"; then + fail "narrowing the skill hook's regex to match zero files did not fail this check, so Part A cannot detect a prefilter that has been silently switched off for SKILL.md files" +elif ! printf '%s' "$MUT_FAILS32" | grep -qF "vale-audit-prefilter-agent: 'files: ^zzz-no-such-path/"; then + fail "narrowing the agent hook's regex to match zero files did not fail this check, so Part A cannot detect a prefilter that has been silently switched off for agent files" +else + pass "narrowing either hook's 'files:' regex to match zero files is caught by Part A, which is what makes its pass mean something" +fi + +# --- 33. The local and published vale hooks agree on every shared file shape -- +# +# The cross-manifest `files:` agreement check of the deleted +# scripts/check-vale-style-sync.sh (ADR-0025), ported. Case 32 does not cover +# it: a local regex narrowed from `^plugins/[^/]+/...` to +# `^plugins/kyberforge/...` still matches tracked files, all of them SKILL.md, +# so it clears both of 32's properties while silently dropping every other +# plugin's skills out of this repo's prefilter. Measured before this case +# existed: that exact narrowing left the whole suite green. +# +# The original's comment on why the two manifests are compared per hook rather +# than unioned, verbatim in substance: "A union here previously let a probe that +# matched only the older, looser .pre-commit-hooks.yaml pattern read as 'in +# scope' even after .pre-commit-config.yaml's copy of the same hook had been +# narrowed away from it -- silently masking exactly the kind of hook-rescoping +# drift this script exists to catch." +# +# What changed in the port is the selector and nothing else. The original found +# each manifest's hook by `entry ~ skill "/scripts/vale-wrap.sh"`, which told the +# two hooks apart only because two skills gave them two entry paths; after the +# merge both hooks share one entry. They are paired by `id:` instead, from an +# explicit table. The pairing is explicit rather than inferred from an id suffix +# so that renaming either id fails here by name instead of quietly dropping a +# class out of the comparison. The probe table and its shared/hooks-only scopes +# are the original's rows, unchanged: +# +# shared -- a shape this repo's own layout has, so both manifests must +# agree on it. This is what catches the narrowing above. +# hooks-only -- a shape only the layout-agnostic published manifest has to +# cover. `.pre-commit-config.yaml` pinning this repo's own +# `plugins/*/.apm/` layout is by design, not drift; bare +# `agents/demo.md` is hooks-only because per ADR-0016 every +# `.apm/agents/` file is `*.agent.md`. +# +# A probe in scope of neither manifest fails too, exactly as in the original: +# the probe path is stale, or both hooks were rescoped away from it. + +HOOK_PAIRS33="$(cat <<'EOF_PAIRS33' +skill|kyberforge-vale-audit-skill|vale-audit-prefilter-skill +agent|kyberforge-vale-audit-agent|vale-audit-prefilter-agent +EOF_PAIRS33 +)" + +PROBES33="$(cat <<'EOF_PROBES33' +skill|plugins/demo/.apm/skills/demo/SKILL.md|shared +skill|.claude/skills/demo/SKILL.md|hooks-only +agent|plugins/demo/.apm/agents/demo.md|hooks-only +agent|plugins/demo/.apm/agents/demo.agent.md|shared +agent|.claude/agents/demo.md|hooks-only +agent|copilot/demo.agent.md|hooks-only +EOF_PROBES33 +)" + +# Prints the `files:` regex of the hook whose id is exactly $2 in manifest $1, +# or nothing. Records are delimited by their `- id:` line, as in +# hook_records28, but selected by id alone: after the merge `entry:` no longer +# distinguishes them. +hook_regex_by_id33() { + local manifest="$1" want="$2" raw + [[ -f "$manifest" ]] || return 0 + raw="$(WANT="$want" awk ' + function flush() { + if (id == ENVIRON["WANT"] && files != "") print files + id = ""; files = "" + } + /^[ \t]*-[ \t]*id:/ { flush(); id = $0; sub(/^[ \t]*-[ \t]*id:[ \t]*/, "", id); sub(/[ \t]+$/, "", id) } + /^[ \t]*files:/ { files = $0; sub(/^[ \t]*files:[ \t]*/, "", files) } + END { flush() } + ' "$manifest" | head -1)" + raw="${raw%\'}"; raw="${raw#\'}" + raw="${raw%\"}"; raw="${raw#\"}" + printf '%s' "$raw" +} + +# Prints one failure token per defect for published manifest $1 against local +# manifest $2; empty output means every probe is in scope of at least one of its +# class's hooks and every shared probe is in scope of both. Manifest paths are +# arguments so Part B runs this exact function against mutated copies. +cross_manifest_failures33() { + local published="$1" local_cfg="$2" + local class pub_id loc_id rel scope pub_re loc_re in_hooks in_config + local bad="" checked=0 shared_classes="" missing_classes="" + while IFS='|' read -r class pub_id loc_id; do + [[ -n "$class" ]] || continue + if [[ -z "$(hook_regex_by_id33 "$published" "$pub_id")" ]]; then + bad+="[no hook with id '$pub_id' and a files: regex in ${published##*/}, so the $class class is compared against nothing] " + missing_classes+="$class " + fi + if [[ -z "$(hook_regex_by_id33 "$local_cfg" "$loc_id")" ]]; then + bad+="[no hook with id '$loc_id' and a files: regex in ${local_cfg##*/}, so the $class class is compared against nothing] " + missing_classes+="$class " + fi + done < "$MUT33/skill.yaml" +LINE33='^[ \t]*files:' narrow_config33 "$PC_CONFIG32" \ + '^plugins/[^/]+/\.apm/agents/' '^plugins/kyberforge/\.apm/agents/' > "$MUT33/agent.yaml" +LINE33='^[ \t]*-[ \t]*id:' narrow_config33 "$PC_CONFIG32" \ + 'vale-audit-prefilter-skill' 'vale-audit-prefilter-skill-renamed' > "$MUT33/id.yaml" +SKILL_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/skill.yaml")" +AGENT_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/agent.yaml")" +ID_FAILS33="$(cross_manifest_failures33 "$PUBLISHED33" "$MUT33/id.yaml")" +if ! grep -qF '^plugins/kyberforge/\.apm/skills/' "$MUT33/skill.yaml" \ + || ! grep -qF '^plugins/kyberforge/\.apm/agents/' "$MUT33/agent.yaml" \ + || ! grep -qF 'vale-audit-prefilter-skill-renamed' "$MUT33/id.yaml"; then + fail "a mutation never reached its copied config, so Part B mutated nothing and proves nothing about Part A" +elif ! printf '%s' "$SKILL_FAILS33" | grep -qF "[plugins/demo/.apm/skills/demo/SKILL.md is in scope of kyberforge-vale-audit-skill"; then + fail "narrowing the local skill hook to ^plugins/kyberforge/ did not fail Part A, so the prefilter can drop every other plugin's skills with every gate green: ${SKILL_FAILS33:-}" +elif [[ -z "$AGREE_FAILS33" ]] && printf '%s' "$SKILL_FAILS33" | grep -qF "kyberforge-vale-audit-agent"; then + # Only meaningful against a clean base: when Part A already failed, the copy + # inherits that defect, and reporting it again here would be one defect twice. + fail "narrowing only the skill hook also reported an agent-class defect, so the comparison is not confined to its class: $SKILL_FAILS33" +elif ! printf '%s' "$AGENT_FAILS33" | grep -qF "[plugins/demo/.apm/agents/demo.agent.md is in scope of kyberforge-vale-audit-agent"; then + fail "narrowing the local agent hook to ^plugins/kyberforge/ did not fail Part A: ${AGENT_FAILS33:-}" +elif ! printf '%s' "$ID_FAILS33" | grep -qF "[no hook with id 'vale-audit-prefilter-skill' and a files: regex in id.yaml"; then + fail "renaming the local skill hook's id did not fail Part A by name, so the skill class could fall out of the comparison silently: ${ID_FAILS33:-}" +else + pass "narrowing either local hook to one plugin, or renaming one, is caught by Part A" +fi + +# --- 34. Every glob section loads a real style, asserted without vale -------- +# +# Case 28 asks this behaviourally -- it lints a probe through the real config +# and watches for a Kyberforge alert -- but every probe of it that can answer +# the question is behind `VALE_READY`. On a machine with no vale those probes +# do not run, the config is then read by case 0 alone, and case 0 deliberately +# exempts an EMPTY `BasedOnStyles` so that 28's style-not-loaded branch keeps +# something left to detect. Between the two, an emptied `BasedOnStyles` is +# caught by nothing in a non-strict run: vale loads that config, lints every +# file the section matches with NO rule, prints `0 errors ... in 1 file` and +# exits 0, and every gate in this repo reads that as a pass. That is the +# silent-pass class ADR-0013 exists to prevent, and `--strict` -- which a +# consumer's clone does not run -- is the only thing standing in front of it. +# +# So this case asks the same question of the config TEXT, with no dependency on +# vale being installed. It is a separate case rather than an addition to either +# neighbour: case 0's exemption and case 28's behavioural branch both stay +# exactly as documented, and no property is owned twice. +# +# The StylesPath half is the deleted scripts/check-vale-style-sync.sh's +# requirement (ADR-0025), restored. `vale_config_defects0` absolutizes with +# `[[ "$sp" == /* ]] || sp="$dir/$sp"`, which ACCEPTS an absolute StylesPath -- +# a path that resolves on the machine that wrote it and on no other. Relative +# resolution against the config's own directory is the only reason the bundled +# styles are found under a CONSUMING repo's clone prefix, so an absolute one +# passes every check here and hard-fails every external consumer of +# .pre-commit-hooks.yaml. It is asserted here, not in case 0, to keep case 0's +# scope the one its comment describes. + +VALE_ASSETS34="$FACTORY_AUDIT/assets/vale" + +# `
|` for every `[glob]` section of $1, in +# file order. A section carrying no BasedOnStyles at all prints an empty value +# rather than no row: an absent key and an emptied one are the same defect and +# must report as one. Keys above the first header are vale's global scope, not +# a section, and are skipped -- a global BasedOnStyles does not make a section +# that overrides it with an empty one load anything. +section_styles34() { + awk ' + function trim(s) { gsub(/^[ \t]+|[ \t]+$/, "", s); return s } + /^[ \t]*\[.*\][ \t]*$/ { + if (sec != "") print sec "|" val + sec = trim($0); val = ""; next + } + sec != "" && /^[ \t]*BasedOnStyles[ \t]*=/ { + val = $0; sub(/^[^=]*=/, "", val); val = trim(val) + } + END { if (sec != "") print sec "|" val } + ' "$1" +} + +# One failure token per defect in the config at $1; empty output means every +# section would load at least one real style for a consumer. The section floor +# is here for the same reason case 28 Part A and case 33 carry theirs: a config +# whose sections were all deleted lints nothing at all, and without a floor this +# function would report it clean. +style_load_defects34() { + local cfg="$1" dir sp sec names name bad="" sections=0 + if [[ ! -f "$cfg" ]]; then + printf '%s' "[$cfg does not exist] " + return 0 + fi + dir="$(dirname "$cfg")" + sp="$({ grep -E '^[[:space:]]*StylesPath[[:space:]]*=' "$cfg" || true; } | tail -1)" + sp="${sp#*=}" + sp="${sp#"${sp%%[![:space:]]*}"}" + sp="${sp%"${sp##*[![:space:]]}"}" + if [[ -z "$sp" ]]; then + printf '%s' "[${cfg##*/} sets no StylesPath, so no section can name a style that loads] " + return 0 + fi + if [[ "$sp" == /* ]]; then + printf '%s' "[${cfg##*/} sets the ABSOLUTE StylesPath '$sp'; it resolves only on the machine that wrote it, and a consuming repo's clone -- which is the only reason .pre-commit-hooks.yaml ships these styles -- gets 'path does not exist'] " + return 0 + fi + if [[ ! -d "$dir/$sp" ]]; then + printf '%s' "[StylesPath resolves to $dir/$sp, which is not a directory] " + return 0 + fi + while IFS='|' read -r sec names; do + [[ -n "$sec" ]] || continue + sections=$((sections + 1)) + if [[ -z "$names" ]]; then + bad+="[$sec declares no non-empty BasedOnStyles, so vale lints every file it matches with no rule loaded, prints '0 errors' and exits 0] " + continue + fi + while IFS= read -r name; do + name="${name#"${name%%[![:space:]]*}"}" + name="${name%"${name##*[![:space:]]}"}" + # `Vale` is vale's built-in style and has no directory, exactly as in + # vale_config_defects0. + [[ -n "$name" && "$name" != "Vale" ]] || continue + [[ -d "$dir/$sp/$name" ]] || bad+="[$sec names style '$name', but $dir/$sp/$name does not exist] " + done < "$MDIR34/.vale.ini" + if cmp -s "$VALE_ASSETS34/.vale.ini" "$MDIR34/.vale.ini"; then + MUT_FAILS34+="[$MSEC34: the mutation left the config unchanged, so nothing was tested] " + continue + fi + MDEFECTS34="$(style_load_defects34 "$MDIR34/.vale.ini")" + grep -qF "[$MSEC34 declares no non-empty BasedOnStyles" <<< "$MDEFECTS34" \ + || MUT_FAILS34+="[$MSEC34 lost its BasedOnStyles and Part A did not fail on it: ${MDEFECTS34:-}] " + done < "$ABS34/.vale.ini" + ABS_DEFECTS34="$(style_load_defects34 "$ABS34/.vale.ini")" + if ! grep -qF "StylesPath = $ABS34/styles" "$ABS34/.vale.ini"; then + fail "the absolute-StylesPath mutation never reached its copied config, so Part B proves nothing about that branch" + elif [[ ! -d "$ABS34/styles" ]]; then + fail "the absolute StylesPath mutation points at a directory that does not exist, so a failure below would not be about absoluteness at all" + elif ! grep -qF "sets the ABSOLUTE StylesPath" <<< "$ABS_DEFECTS34"; then + fail "an absolute StylesPath naming a REAL styles directory passed Part A, so the config can ship a path that resolves on this machine alone: ${ABS_DEFECTS34:-}" + elif [[ -n "$MUT_FAILS34" ]]; then + fail "Part A does not catch a section whose BasedOnStyles was emptied, which is the silent case this whole case exists for: $MUT_FAILS34" + else + pass "each section, with its BasedOnStyles emptied in a copy of the config, fails Part A by name, and so does an absolute StylesPath pointing at a real styles directory" + fi +fi + echo "" echo "Results: $PASS passed, $FAIL failed" -[[ $FAIL -eq 0 ]] +[[ $FAIL -eq 0 ]] || exit 1 +if [[ "$VALE_AVAILABLE" != true ]]; then + # run-tests.sh records the first `SKIP` line printed above as the reason. + echo "Every static case passed, but vale is not installed and the Vale-dependent cases did not run — exiting 77 so run-tests.sh reports this suite as skipped, and --strict fails it." + exit 77 +fi