refactor(kyberforge)!: merge skill-audit and agent-audit into factory-audit
Why The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser (93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line test suite. Sync-by-gate had already failed once: at484357athe two parser copies drifted into different spellings of the bullet loop while a docstring asserted they were identical. That drift was behaviour-neutral and was re-unified by hand at598a7c3, so the copies were identical at merge time — but nothing had caught it, and the next drift need not be neutral. Implementation Notes Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching across skill directories, which is why two separate skills needed embedded copies; two files inside ONE skill may source a third. That is the whole reason the merge removes duplication rather than relocating it. The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path before any validation, and its table mirrors validate.sh's detection exactly: a directory holding SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory (agent); anything else stops without running a validator. Steps 1-3 live in references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the other artifact type, the body restarts at Step 0. Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that matches a live sibling reads as ownership rather than membership. The description carries one arrow per boundary target, because ADR-0020 resolves only the first target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated "audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16% of the body: no size SUGGESTIONs. The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both. tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources the resolver and that no third copy exists. The entry scripts classify the target from its resolved parent directory, so a bare agent filename typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is missing, rather than dying with exit 1, the tier the flows relay as real findings. The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's status (`f || RC=$?`) disables errexit for its entire body, and no subshell or `set -e` inside can re-arm it once the call sits in a condition context (measured, both spellings). Their error paths use `exit`, which is unaffected either way; this keeps errexit armed for anything added later. Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's dev environment is root -- so the guard could never fire where it exists to fire. A read attempt is also the stricter question, catching EIO. This is the reasoning scripts/check-vale-style-sync.sh carried before this commit deleted it; the hazard did not go with it. All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared, the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two remaining bare cd calls take absolute paths, which CDPATH is never consulted for. Impact BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to 2.0.0 (catalog 0.4.7). Check logic is unchanged: differential runs of the old and new validators across every skill and agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename. One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change, deliberately (full table in ADR-0025): - a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is by path shape, so a shape-matching path that is simply absent used to reach the validator and come back as a FAIL against a file that never existed; - an unshaped target exits 2 under the generic "matches neither" message, and a directory with no SKILL.md under a third, distinct one -- three exit-2 messages, not one; - a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding about the artifact rather than a usage error; - a SKILL.md file path is audited as its skill directory instead of refused; - a .md agent outside an agents/ directory is refused rather than audited; - a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2. validate-provenance.sh already exited 2 for the last two; validate.sh now matches it. .pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both files: regexes are unchanged; only entry: and description: moved. scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21), tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite. Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8. ADR: 0025 BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by factory-audit, which auto-detects whether it was handed a skill directory or an agent file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
@@ -1,90 +0,0 @@
|
||||
---
|
||||
name: agent-audit
|
||||
description: >
|
||||
Use when the user wants an agent definition audited — "audit this agent",
|
||||
"review my agent file", "is this ready to ship" — or after hand-editing an
|
||||
agent outside agent-author. Not applying fixes -> agent-author. Not a skill
|
||||
directory -> skill-audit.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
version: "1.0.2"
|
||||
category: factory
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- Agents take the same 250/400-character description gates as skills and **no body word gate at all** — an agent body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer. Judge an over-long agent body through the delegation check, never by word count.
|
||||
- At plugin/APM scope the agent is a single vendor-neutral file by design, so provider safety stops meaning Claude-Code-versus-Copilot field leakage there.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <agent-file>
|
||||
bash scripts/validate-provenance.sh <agent-file>
|
||||
bash scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
|
||||
```
|
||||
|
||||
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
|
||||
|
||||
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, `references/field-inventory.md` missing — read `references/validation-scripts.md`; what these scripts measure is not reproducible by reading.
|
||||
|
||||
`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — a bad argument or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same 2 tier.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
|
||||
|
||||
## Step 2 — Read the agent and load its scope contract
|
||||
|
||||
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
|
||||
|
||||
| Scope | Read |
|
||||
|---|---|
|
||||
| plugin/APM | `references/scope-plugin-apm.md` |
|
||||
| project, user | `references/scope-project-user.md` |
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body, delegation, comment-discipline | `references/body-and-delegation.md` |
|
||||
|
||||
Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with a coverage line naming every dimension checked. At project/user scope:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
|
||||
```
|
||||
|
||||
At plugin/APM scope, drop `pair-consistency` — there is no pair to check.
|
||||
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run agent-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
@@ -1,90 +0,0 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Sources
|
||||
|
||||
## context7-websites-code-claude
|
||||
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/finding-criteria.md, references/field-inventory.md, references/description-quality.md, references/body-and-delegation.md, references/scope-plugin-apm.md, references/scope-project-user.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-creating
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-finding-installing
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-marketplace
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-sdk-custom-agents
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
@@ -1,47 +0,0 @@
|
||||
# scripts/
|
||||
|
||||
Executable code bundled with this skill. Agents run scripts in this directory
|
||||
to perform repeatable operations rather than reinventing the logic each run.
|
||||
|
||||
## When to add a script
|
||||
|
||||
Add a script when agents independently reinvent the same logic across runs —
|
||||
building the same parser, chart, or validation routine from scratch each time.
|
||||
Bundle it here once, tested and reliable.
|
||||
|
||||
## Script requirements (agentskills.io)
|
||||
|
||||
Scripts must be designed for non-interactive, agentic execution:
|
||||
|
||||
- **No interactive prompts** — agents run in non-interactive shells.
|
||||
Accept all input via flags, env vars, or stdin. A script that blocks on
|
||||
TTY input hangs indefinitely.
|
||||
- **Expose `--help`** — this is how agents learn your script's interface.
|
||||
Keep the output concise; it enters the agent's context window.
|
||||
- **Structured output** — write data (JSON, CSV, TSV) to stdout.
|
||||
Write progress, warnings, and diagnostics to stderr.
|
||||
- **Idempotent** — prefer "create if not exists" over "create and fail on
|
||||
duplicate". Agents may retry on failure.
|
||||
- **Meaningful exit codes** — `0` for success, non-zero for failure.
|
||||
Use distinct codes for different failure types; document them in `--help`.
|
||||
- **Dry-run support** — add `--dry-run` for destructive operations.
|
||||
|
||||
## Self-contained scripts
|
||||
|
||||
Bundle dependencies inline so the agent can run the script with a single command.
|
||||
|
||||
Python (PEP 723 + uv):
|
||||
```python
|
||||
# /// script
|
||||
# dependencies = ["requests>=2.31,<3"]
|
||||
# requires-python = ">=3.11"
|
||||
# ///
|
||||
import requests
|
||||
```
|
||||
```bash
|
||||
uv run scripts/my-script.py
|
||||
```
|
||||
|
||||
## If no scripts are needed
|
||||
|
||||
Delete this README and the `scripts/` directory entirely.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,33 +0,0 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## When to add tests
|
||||
|
||||
Add tests here when the skill has scripts in `scripts/` that are complex enough
|
||||
to break silently — validators, parsers, generators, anything with branching
|
||||
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
|
||||
belongs here, not in `scripts/`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Tests require [bats-support](https://github.com/bats-core/bats-support) and
|
||||
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
|
||||
helpers from the repo root's `tests/test_helper/`.
|
||||
|
||||
From the repo root:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests for this skill (from the repo root):
|
||||
|
||||
```bash
|
||||
bats <destination-dir>/agent-audit/tests/
|
||||
```
|
||||
|
||||
## If no tests are needed
|
||||
|
||||
Delete this README and the `tests/` directory entirely.
|
||||
@@ -3,7 +3,7 @@ name: agent-author
|
||||
description: >
|
||||
Use when the user wants to create a new agent definition file from scratch, or
|
||||
apply grill findings, audit findings, or inline feedback to an existing one.
|
||||
Not read-only review -> `agent-audit`. Not skills -> `skill-author`.
|
||||
Not read-only review -> `factory-audit`. Not skills -> `skill-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
version: "1.0.2"
|
||||
@@ -18,7 +18,7 @@ metadata:
|
||||
|
||||
- At plugin/APM scope `tools` and every Claude-only field are omitted entirely, not merely ignored: `apm compile` copies frontmatter verbatim to both harnesses, so fencing a read-only agent with `tools:` is wrong on one of them. `disallowedTools` is the one restriction that survives (ADR-0016).
|
||||
- That fence is partial. It denies only the tools it names, never `Bash`, which a plugin-scope agent inherits — a shell redirect still writes. State the read-only boundary in the body too.
|
||||
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is an `agent-audit` FAIL.
|
||||
- An agent body carries no word gate; delegation replaces it. A plugin/APM agent is one file with no sibling `references/` directory, so it cannot disclose to itself, only invoke skills — and a body restating a procedure an invocable skill owns is a `factory-audit` FAIL.
|
||||
- Duplicate `name` values in one scope: Claude Code discards one silently. Verify uniqueness before shipping.
|
||||
|
||||
## Step 1 — Dispatch
|
||||
@@ -29,7 +29,7 @@ metadata:
|
||||
| A file exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| A file exists, no signals | Stop and ask | — |
|
||||
|
||||
Signals: grill output, `agent-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
Signals: grill output, `factory-audit` findings, inline feedback, session context describing what went wrong. With none, ask: "No improvement signals found. Did you mean to create a new agent, or do you have feedback to apply?"
|
||||
|
||||
Read only the reference for the resolved flow. Capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
@@ -48,7 +48,7 @@ Read only the file for the resolved scope; the other describes fields this run c
|
||||
|
||||
Before writing or editing a `description`, or restructuring a body, read `references/contract.md` — the three-part shape, banned content, the delegation rule and the body pattern.
|
||||
|
||||
Gates `agent-audit` enforces at every scope:
|
||||
Gates `factory-audit` enforces at every scope:
|
||||
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <name>` that resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only: an agent's `name` and `description` is preloaded into every session exactly as a skill's is.
|
||||
- **Body** — no word gate, and a delegation check in its place: name the skill to invoke rather than restating what it does.
|
||||
@@ -58,7 +58,7 @@ At every scope, five tools reach no subagent whatever `tools` says — `AskUserQ
|
||||
|
||||
## Step 4 — Validate and close
|
||||
|
||||
Invoke `agent-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
|
||||
Invoke `factory-audit` on each file written and resolve every FAIL before reporting done. It checks the field allowlist, name-to-stem match, leftover placeholders and template comments, the description budget and the Copilot body limit — do not hand-check those.
|
||||
|
||||
At plugin/APM scope bump the resolved package's `apm.yml` `version` — **minor** on create, **patch** on improve — because consumers compare it to detect updates. Project and user scope have no manifest.
|
||||
|
||||
|
||||
@@ -7,4 +7,4 @@ All three scaffold the `description` in the three-part ADR-0020 shape — a `Use
|
||||
|
||||
- **`claude-code.md`** — Claude Code agent definition template (project/user scope). Includes all supported frontmatter fields (required and optional) with inline guidance comments and `FILL IN:` placeholders.
|
||||
- **`copilot.agent.md.template`** — Copilot CLI agent definition template (CLI format, project/user scope). Excludes cloud/IDE-only fields (`target`, `user-invocable`, `disable-model-invocation`, `mcp-servers`) and Claude Code-only fields. Uses Copilot tool aliases (`execute`, `read`, `edit`, `search`, `agent`, `web`).
|
||||
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md` — the authoritative list, read from there as data by `agent-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment).
|
||||
- **`apm-agent.md`** — Vendor-neutral APM agent definition template (plugin/APM scope). Frontmatter is limited to the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md` — the authoritative list, read from there as data by `factory-audit`'s `validate.sh`; this file deliberately does not restate it. No `tools` and no Claude-only knobs, since `apm compile` copies frontmatter verbatim to both the Claude Code and Copilot CLI targets with no per-target integrator; `disallowedTools` is scaffolded as an opt-in comment because a denylist, unlike the `tools` allowlist, survives that copy (ADR-0016 and its 2026-08-14 amendment).
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
anything, so a harness-specific value is wrong on at least one of them.
|
||||
|
||||
This template does not restate the permitted-field list. The authoritative
|
||||
list is the `apm-agent-allowlist` section of agent-audit's
|
||||
references/field-inventory.md, which agent-audit's validate.sh reads from
|
||||
list is the `apm-agent-allowlist` section of factory-audit's
|
||||
references/agent-field-inventory.md, which factory-audit's validate.sh reads from
|
||||
there as data — a list copied into a template goes stale one step further
|
||||
out than the list itself. Every field scaffolded below is on it; before
|
||||
adding any other field, check that section.
|
||||
@@ -37,7 +37,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
|
||||
deleted.
|
||||
Never write "Use proactively" here. It steers the Claude Code runtime and does
|
||||
nothing anywhere else, and this file compiles to a Copilot `.agent.md` too, where
|
||||
agent-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL.
|
||||
factory-audit's KyberforgeCopilot.ProactivePhrase rule grades it a hard FAIL.
|
||||
The phrase is CC-only; at this scope, a precise trigger clause does that job.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not prose or style linting -> `lint-runner`." -->
|
||||
@@ -69,7 +69,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
@@ -18,7 +18,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
|
||||
"Use proactively" is valid HERE and only here: it steers the Claude Code runtime
|
||||
to offer this agent unprompted. Add it only if that is what you want. If you add
|
||||
it, leave it OUT of the Copilot half of the pair — the phrase does nothing there
|
||||
and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The
|
||||
and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL. The
|
||||
pair must describe the same job; it does not have to be byte-identical.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
merges. Not prose or style linting -> `lint-runner`." -->
|
||||
@@ -31,7 +31,7 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
|
||||
Omit Agent entirely to prevent this agent from spawning subagents.
|
||||
Never available to subagents regardless of tools field:
|
||||
AskUserQuestion, EnterPlanMode, ExitPlanMode, ScheduleWakeup, WaitForMcpServers
|
||||
Listing any of them is a finding: agent-audit enforces the flat rule. -->
|
||||
Listing any of them is a finding: factory-audit enforces the flat rule. -->
|
||||
|
||||
<!-- model: sonnet
|
||||
Optional. Aliases: sonnet, opus, haiku, fable. Or full model ID.
|
||||
@@ -90,7 +90,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
@@ -20,8 +20,8 @@ description: FILL IN: Use when <trigger>. <One capability clause.> Not <thing> -
|
||||
250 characters is the target, 400 the hard ceiling (ADR-0020).
|
||||
Do not open with an action verb ("Reviews...", "Analyzes...") — that rule was deleted.
|
||||
Never write "Use proactively" here. It steers the Claude Code runtime and does nothing
|
||||
in Copilot, and agent-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL.
|
||||
Otherwise keep the wording matched to the Claude Code half of the pair: agent-audit
|
||||
in Copilot, and factory-audit's KyberforgeCopilot.ProactivePhrase grades it a hard FAIL.
|
||||
Otherwise keep the wording matched to the Claude Code half of the pair: factory-audit
|
||||
checks that both halves describe the same job, not that they are byte-identical, so
|
||||
dropping the CC-only phrase here is not a pair-consistency finding.
|
||||
Example: "Use when a diff needs checking for injected credentials before it
|
||||
@@ -58,7 +58,7 @@ You are a FILL IN: role description. When invoked, FILL IN: primary action.
|
||||
|
||||
<!-- Delegate, don't restate. If an installed skill already owns a procedure this agent
|
||||
needs, name it ("invoke `git-commits`") instead of transcribing it — a body that
|
||||
restates a procedure an invocable skill owns is an agent-audit FAIL. One job per
|
||||
restates a procedure an invocable skill owns is a factory-audit FAIL. One job per
|
||||
agent. Delete this comment before shipping. -->
|
||||
|
||||
## Inputs
|
||||
|
||||
@@ -7,7 +7,7 @@ source_keys:
|
||||
# The agent description and body contract
|
||||
|
||||
House contract. The counts and the boundary targets are enforced by
|
||||
`agent-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
|
||||
`factory-audit`'s `scripts/validate.sh`; the prose patterns by the Vale styles it bundles; the
|
||||
judgment calls by its reference files.
|
||||
|
||||
## Why the budget exists
|
||||
@@ -54,7 +54,7 @@ appear depends on the file:
|
||||
| Vendor-neutral `.apm/agents/<name>.agent.md` (plugin/APM scope) | **Never.** Same Vale rule, same hard FAIL — the file matches the `**/*.agent.md` glob, and it compiles to a real Copilot agent downstream. |
|
||||
|
||||
A pair whose Claude Code half carries the phrase and whose Copilot half omits it is correct, not
|
||||
inconsistent: `agent-audit` checks that both halves describe the same job, not that they match
|
||||
inconsistent: `factory-audit` checks that both halves describe the same job, not that they match
|
||||
word for word.
|
||||
|
||||
Indirect triggers ("even if the user doesn't say X") take a similar conditional at every scope:
|
||||
@@ -126,7 +126,7 @@ One job per agent. An agent covering two jobs gets delegated to for the wrong on
|
||||
**Delegation discipline replaces the word gate.** A plugin/APM agent is a single file with no
|
||||
sibling `references/` directory: it cannot disclose progressively to itself, so its only way to
|
||||
stay short is to *invoke* rather than *restate*. A body that transcribes a procedure a skill it
|
||||
can invoke already owns is an `agent-audit` FAIL, and the fix is one line — "invoke `<skill>`".
|
||||
can invoke already owns is a `factory-audit` FAIL, and the fix is one line — "invoke `<skill>`".
|
||||
|
||||
- Restating: "To commit, check the message against Conventional Commits: type, scope,
|
||||
description; header under 100 chars; …"
|
||||
|
||||
@@ -22,7 +22,7 @@ Before touching the filesystem, confirm you have:
|
||||
|
||||
If any are missing, stop and ask before proceeding.
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
|
||||
`factory-audit` runs the validation in `SKILL.md` Step 4. It ships with the kyberforge plugin and
|
||||
is co-installed with this skill; if it is unavailable, stop and ask the user to install
|
||||
kyberforge before continuing.
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ When the same agent `name` appears at multiple scopes, **user scope wins over pr
|
||||
Field rules are per scope and live with the scope: `references/plugin-scope.md` for the single
|
||||
vendor-neutral file, `references/project-user-scope.md` for the Claude Code / Copilot pair. Read
|
||||
one, not both. The short version is that plugin/APM frontmatter is an allowlist read from
|
||||
`agent-audit`'s `references/field-inventory.md`, narrow because `apm compile` copies frontmatter
|
||||
`factory-audit`'s `references/agent-field-inventory.md`, narrow because `apm compile` copies frontmatter
|
||||
verbatim to every target (ADR-0016), while project and user scope carry the full per-provider
|
||||
field sets.
|
||||
|
||||
|
||||
@@ -14,9 +14,9 @@ Confirm the agent file (or, at project and user scope, the pair) exists and that
|
||||
improvement signal is present in the conversation or in a referenced file.
|
||||
|
||||
If no signals are present, stop: "This skill applies existing signals to an agent. For a blind
|
||||
review, run `agent-audit` instead."
|
||||
review, run `factory-audit` instead."
|
||||
|
||||
`agent-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
|
||||
`factory-audit` runs the validation in `SKILL.md` Step 4 and is co-installed with this skill; if
|
||||
it is unavailable, stop and ask the user to install the kyberforge plugin before continuing.
|
||||
|
||||
**Partial pair — project and user scope only.** If one provider file exists and the other does
|
||||
@@ -53,7 +53,7 @@ Edit whichever file the signals point to.
|
||||
scoped to the cases you have seen overfits and performs worse on new input.
|
||||
|
||||
**Delegate rather than grow.** An agent body has no word ceiling, but a body that restates a
|
||||
procedure a skill it can invoke already owns is an `agent-audit` FAIL. When a signal reports a
|
||||
procedure a skill it can invoke already owns is a `factory-audit` FAIL. When a signal reports a
|
||||
missing procedure, check first whether an installed skill owns it and name that skill instead of
|
||||
transcribing it. See `references/contract.md`.
|
||||
|
||||
@@ -81,7 +81,7 @@ the matching `sources.md` entry — the create flow's Step 3 has the rules.
|
||||
|
||||
**Check for regressions before handing back.** `SKILL.md` Step 4 tells you to resolve every FAIL,
|
||||
which says nothing about a check that passed *before* these edits and no longer does. Compare the
|
||||
closing `agent-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or
|
||||
closing `factory-audit` against the agent's pre-edit state — a PASS that has become a SUGGESTION, or
|
||||
a SUGGESTION that has become a FAIL, is damage this flow caused and is in scope for it. Only the
|
||||
improve flow can make that comparison; the create flow has no prior state to compare against.
|
||||
|
||||
|
||||
@@ -12,9 +12,9 @@ a Copilot marker — the file is vendor-neutral.
|
||||
|
||||
## Frontmatter
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `agent-audit`'s
|
||||
`references/field-inventory.md`. Read them from there as data — that section is the single source
|
||||
of truth, `agent-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
|
||||
The permitted keys are the `apm-agent-allowlist` section of `factory-audit`'s
|
||||
`references/agent-field-inventory.md`. Read them from there as data — that section is the single source
|
||||
of truth, `factory-audit`'s `validate.sh` parses it at load time, and it changes. Any restatement of
|
||||
the roster, here or in a template or in script output, goes stale one step further out than the
|
||||
list itself.
|
||||
|
||||
@@ -60,7 +60,7 @@ when research sources informed the agent, with slugs matching H2 headings in the
|
||||
Follow the Body section of `references/contract.md`: role instruction, one job, and delegation
|
||||
to installed skills instead of transcribed procedure.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
## Before invoking `factory-audit`
|
||||
|
||||
- [ ] `name` kebab-case, matching the filename stem, unique in scope
|
||||
- [ ] `description` written to `references/contract.md`
|
||||
|
||||
@@ -92,7 +92,7 @@ Both formats truncate a body past **30,000 characters** silently.
|
||||
Copilot has no `permissionMode`, `maxTurns`, `isolation`, `memory`, `effort`, `hooks` or
|
||||
`mcpServers`. Never let those cross over from the Claude Code file.
|
||||
|
||||
## Before invoking `agent-audit`
|
||||
## Before invoking `factory-audit`
|
||||
|
||||
Both files:
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ All scripts in this skill must follow these rules:
|
||||
- **Idempotent** — "create if not exists" per file. The scaffold script skips any file that already exists; agents may safely re-run it.
|
||||
- **Meaningful exit codes** — `0` success, `1` invalid arguments or precondition failure. Document in `--help`.
|
||||
- **Self-contained** — no external package installs at runtime. The script uses only bash builtins and POSIX tools (`sed`, `mkdir`, `cat`).
|
||||
- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `agent-audit`'s `references/field-inventory.md`, which `agent-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure.
|
||||
- **No restated field rosters** — no script output, in `--help` or in next-steps guidance, enumerates permitted, forbidden, or required frontmatter fields. Point at the `apm-agent-allowlist` section of `factory-audit`'s `references/agent-field-inventory.md`, which `factory-audit`'s `validate.sh` reads from there as data. A roster copied into script output goes stale one step further out than the list itself: the next-steps hint `(name, description, model, body only)` kept printing after ADR-0016's 2026-08-14 amendment added `disallowedTools` to the permitted set. `tests/new-agent.bats` enforces this for the plugin/APM branch — naming some allowlisted fields but not all is a failure.
|
||||
|
||||
## Template variables
|
||||
|
||||
|
||||
@@ -25,8 +25,8 @@ Arguments:
|
||||
its frontmatter verbatim to every target with no
|
||||
per-target field integrator, so the permitted
|
||||
field set is narrow — see the apm-agent-allowlist
|
||||
section of agent-audit's
|
||||
references/field-inventory.md and ADR-0016)
|
||||
section of factory-audit's
|
||||
references/agent-field-inventory.md and ADR-0016)
|
||||
→ creates <package-root>/sources.md (if absent)
|
||||
project scope : no type:-bearing apm.yml found; root is a
|
||||
project directory
|
||||
@@ -109,7 +109,7 @@ is_apm_package_manifest() {
|
||||
|
||||
# --- Walk-up package-root detection ---
|
||||
#
|
||||
# Mirrors agent-audit's validate.sh scope walk-up, with apm.yml + type: swapped
|
||||
# Mirrors factory-audit's validate.sh scope walk-up, with apm.yml + type: swapped
|
||||
# in for the old plugin.json marker. Starting at ROOT, walk upward:
|
||||
# - an apm.yml with a top-level `type:` field marks an APM package root
|
||||
# (plugin/APM scope) — stop and return it.
|
||||
@@ -266,13 +266,13 @@ fi
|
||||
# A roster restated in terminal output goes stale one step further out than the list
|
||||
# itself: the old "(name, description, model, body only)" hint outlived ADR-0016's
|
||||
# 2026-08-14 amendment, which added disallowedTools to the permitted set. Point at the
|
||||
# scaffolded file's own comments for what to fill, and at agent-audit's validate.sh —
|
||||
# which reads the allowlist from field-inventory.md as data — for what is permitted.
|
||||
AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../agent-audit/scripts" 2>/dev/null && pwd || true)"
|
||||
# scaffolded file's own comments for what to fill, and at factory-audit's validate.sh —
|
||||
# which reads the allowlist from agent-field-inventory.md as data — for what is permitted.
|
||||
AUDIT_SCRIPTS="$(cd "$SKILL_ROOT/../factory-audit/scripts" 2>/dev/null && pwd || true)"
|
||||
if [[ -n "$AUDIT_SCRIPTS" && -f "$AUDIT_SCRIPTS/validate.sh" ]]; then
|
||||
VALIDATE_HINT="$AUDIT_SCRIPTS/validate.sh"
|
||||
else
|
||||
VALIDATE_HINT="agent-audit's scripts/validate.sh"
|
||||
VALIDATE_HINT="factory-audit's scripts/validate.sh"
|
||||
fi
|
||||
|
||||
if [[ "$created_any" == false ]]; then
|
||||
@@ -290,7 +290,7 @@ else
|
||||
echo " 2. Populate $SOURCES_DIR/sources.md with research sources, or delete it" >&2
|
||||
echo " 3. Validate: $VALIDATE_HINT $APM_FILE" >&2
|
||||
echo " It checks the frontmatter against the apm-agent-allowlist section of" >&2
|
||||
echo " agent-audit's references/field-inventory.md, the authoritative field list." >&2
|
||||
echo " factory-audit's references/agent-field-inventory.md, the authoritative field list." >&2
|
||||
else
|
||||
echo " 1. Fill in $CC_FILE — replace every FILL IN: placeholder. Optional fields are" >&2
|
||||
echo " scaffolded there as commented blocks; uncomment the ones that apply." >&2
|
||||
|
||||
@@ -77,14 +77,14 @@ teardown() {
|
||||
assert_failure
|
||||
}
|
||||
|
||||
# The permitted set is read from the same data agent-audit's validate.sh reads --
|
||||
# the apm-agent-allowlist section of agent-audit's field-inventory.md -- rather than
|
||||
# The permitted set is read from the same data factory-audit's validate.sh reads --
|
||||
# the apm-agent-allowlist section of factory-audit's agent-field-inventory.md -- rather than
|
||||
# restated here. A hardcoded copy drifts: this assertion listed four fields and went
|
||||
# on passing after ADR-0016's amendment added disallowedTools, and would have
|
||||
# rejected a scaffolded agent that legitimately carried it.
|
||||
@test "plugin/APM scope: frontmatter carries only allowlisted fields" {
|
||||
inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md"
|
||||
[ -f "$inventory" ] || fail "field-inventory.md not found at $inventory"
|
||||
inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md"
|
||||
[ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory"
|
||||
allowlist="$(awk '
|
||||
/^## apm-agent-allowlist$/ { insection = 1; next }
|
||||
insection && /^##/ { exit }
|
||||
@@ -99,7 +99,7 @@ teardown() {
|
||||
keys="$(grep -oE '^[a-zA-Z][a-zA-Z0-9_-]*:' <<< "$fm" | sed 's/:$//' | sort -u)"
|
||||
for key in $keys; do
|
||||
if ! grep -qw "$key" <<< "$allowlist"; then
|
||||
fail "frontmatter key '$key' is not in field-inventory.md's apm-agent-allowlist ($allowlist)"
|
||||
fail "frontmatter key '$key' is not in agent-field-inventory.md's apm-agent-allowlist ($allowlist)"
|
||||
fi
|
||||
done
|
||||
}
|
||||
@@ -111,8 +111,8 @@ teardown() {
|
||||
# must name every one of them, so a partial restatement -- the only shape that can go
|
||||
# stale silently -- fails. Naming none, the current design, passes.
|
||||
@test "plugin/APM scope: next-steps guidance does not partially restate the allowlist" {
|
||||
inventory="$BATS_TEST_DIRNAME/../../agent-audit/references/field-inventory.md"
|
||||
[ -f "$inventory" ] || fail "field-inventory.md not found at $inventory"
|
||||
inventory="$BATS_TEST_DIRNAME/../../factory-audit/references/agent-field-inventory.md"
|
||||
[ -f "$inventory" ] || fail "agent-field-inventory.md not found at $inventory"
|
||||
allowlist="$(awk '
|
||||
/^## apm-agent-allowlist$/ { insection = 1; next }
|
||||
insection && /^##/ { exit }
|
||||
@@ -134,7 +134,7 @@ teardown() {
|
||||
fi
|
||||
done
|
||||
if [ -n "$named" ] && [ -n "$missing" ]; then
|
||||
fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at field-inventory.md instead of restating it."
|
||||
fail "next-steps names allowlisted field(s)$named but omits$missing -- a partial roster. Point at agent-field-inventory.md instead of restating it."
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
77
plugins/kyberforge/.apm/skills/factory-audit/SKILL.md
Normal file
77
plugins/kyberforge/.apm/skills/factory-audit/SKILL.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: factory-audit
|
||||
description: >
|
||||
Use when the user wants a skill directory or agent definition audited,
|
||||
including "is this ready to ship", or after hand-editing one outside its
|
||||
author skill. Not applying skill fixes -> skill-author. Not applying agent
|
||||
fixes -> agent-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
version: "1.0.1"
|
||||
category: factory
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A file carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 0 — Dispatch
|
||||
|
||||
Resolve the flow from the target path **before running anything**. The two flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches.
|
||||
|
||||
| Target | Flow | Read |
|
||||
|---|---|---|
|
||||
| A directory containing `SKILL.md` | skill | `references/skill-flow.md` |
|
||||
| A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` |
|
||||
| A file named `*.agent.md` | agent | `references/agent-flow.md` |
|
||||
| A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` |
|
||||
| Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — |
|
||||
|
||||
Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4.
|
||||
|
||||
On the last row, stop: run no validator and tell the user the two accepted shapes — a skill directory (or its `SKILL.md`), or an agent file (`*.agent.md`, or a `.md` directly under an `agents/` directory). Guessing a flow audits the path against the wrong spec.
|
||||
|
||||
The scripts re-detect the flow from the path. If `validate.sh` reports on the other artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with the coverage line for the flow you took, naming every dimension checked.
|
||||
|
||||
Skill flow:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Agent flow:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
|
||||
```
|
||||
|
||||
On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check.
|
||||
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow. Do not apply fixes — report and propose only.
|
||||
@@ -1,5 +1,8 @@
|
||||
StylesPath = styles
|
||||
|
||||
[**/SKILL.md]
|
||||
BasedOnStyles = Kyberforge
|
||||
|
||||
[**/agents/*.md]
|
||||
BasedOnStyles = Kyberforge
|
||||
|
||||
@@ -30,7 +30,7 @@ prompt of a fresh context that has nothing else in it. The rationale for the 900
|
||||
not transfer, so:
|
||||
|
||||
- **Never report an agent body as too long on a word count.** There is no number to cite.
|
||||
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate.bats` pins its absence with a
|
||||
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate-agent.bats` pins its absence with a
|
||||
body far past 900 words that must still pass, and adding one would contradict the ADR.
|
||||
- The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which
|
||||
`validate.sh` already reports as a SUGGESTION because content past it is silently truncated.
|
||||
@@ -100,6 +100,6 @@ frontmatter block that still contains one.
|
||||
|
||||
## Where the criteria live
|
||||
|
||||
Every FAIL and SUGGESTION criterion for these dimensions is in `references/finding-criteria.md`,
|
||||
Every FAIL and SUGGESTION criterion for these dimensions is in `references/agent-finding-criteria.md`,
|
||||
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
|
||||
puts the body, delegation or comment-discipline dimension in play.
|
||||
@@ -91,6 +91,6 @@ description: >
|
||||
|
||||
## Where the criteria live
|
||||
|
||||
Every FAIL and SUGGESTION criterion for this dimension is in `references/finding-criteria.md`,
|
||||
Every FAIL and SUGGESTION criterion for this dimension is in `references/agent-finding-criteria.md`,
|
||||
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
|
||||
puts the description dimension in play.
|
||||
@@ -20,7 +20,7 @@ Two rules on using it:
|
||||
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
||||
to read; it does not settle a close call on its own.
|
||||
|
||||
## description — `references/description-quality.md`
|
||||
## description — `references/agent-description-quality.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -49,7 +49,7 @@ Flag as FAIL if:
|
||||
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
|
||||
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0 of
|
||||
`references/description-quality.md`.
|
||||
`references/agent-description-quality.md`.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
@@ -68,7 +68,7 @@ What is left to judgment is semantic and the script cannot reach it: whether a t
|
||||
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
||||
files manually") should have named one.
|
||||
|
||||
## body, delegation and comment-discipline — `references/body-and-delegation.md`
|
||||
## body, delegation and comment-discipline — `references/agent-body-and-delegation.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Agent Flow
|
||||
|
||||
Steps 1 to 3 for an agent definition — the target Step 0 matched as a `*.agent.md` file, or as a
|
||||
`.md` file whose immediate parent directory is `agents/`. Work them in order, then return to
|
||||
`SKILL.md` Step 4 to report.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- An agent takes the skill description gates (250 characters SUGGESTION, 400 FAIL) and **no body word gate at all**. Its body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer and no number exists to cite. Judge an over-long agent body through the delegation check.
|
||||
- **Plugin/APM scope only:** provider safety means survival of a verbatim copy to every target, not Claude-Code-versus-Copilot field leakage — `references/agent-scope-plugin-apm.md` carries the contract.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <agent-file>
|
||||
bash scripts/validate-provenance.sh <agent-file>
|
||||
bash scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
|
||||
```
|
||||
|
||||
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
|
||||
|
||||
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, a `scripts/lib-*.sh` library or `references/agent-field-inventory.md` missing — read `references/agent-validation-scripts.md`; what these scripts measure is not reproducible by reading.
|
||||
|
||||
`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same tiers: **1** is real findings, **2** is never ran — report that as `### Structure` unverified, never as a failure.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
|
||||
|
||||
## Step 2 — Read the agent and load its scope contract
|
||||
|
||||
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
|
||||
|
||||
| Scope | Read |
|
||||
|---|---|
|
||||
| plugin/APM | `references/agent-scope-plugin-apm.md` |
|
||||
| project, user | `references/agent-scope-project-user.md` |
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/agent-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/agent-description-quality.md` |
|
||||
| body, delegation, comment-discipline | `references/agent-body-and-delegation.md` |
|
||||
|
||||
Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -18,13 +18,13 @@ cannot carry either (ADR-0016). That single fact drives everything below.
|
||||
|
||||
## Frontmatter allowlist
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `references/field-inventory.md`. Read
|
||||
The permitted keys are the `apm-agent-allowlist` section of `references/agent-field-inventory.md`. Read
|
||||
them from there. Do not recite the list in a finding, do not work from memory, and do not trust any
|
||||
restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it
|
||||
has changed before, and `validate.sh` parses that same section at load time, so a recitation is a
|
||||
copy that can disagree with the check the agent just ran.
|
||||
|
||||
`field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
|
||||
`agent-field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
|
||||
is not. Read that note before arguing with a finding about it.
|
||||
|
||||
## Dimension routing
|
||||
@@ -55,4 +55,4 @@ known upstream schema limitation (ADR-0016), not an authoring mistake, and the f
|
||||
give the author visibility into the gap rather than to imply the schema can be made to close it.
|
||||
|
||||
Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it.
|
||||
A denylist-shaped restriction is the available half of that — see `field-inventory.md`.
|
||||
A denylist-shaped restriction is the available half of that — see `agent-field-inventory.md`.
|
||||
@@ -30,7 +30,7 @@ the finding, not presented as a platform spec failure.
|
||||
| everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure |
|
||||
|
||||
The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of
|
||||
`references/field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
|
||||
`references/agent-field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
|
||||
those same sections, so any restatement is a copy that can disagree with the check (ADR-0009).
|
||||
|
||||
## Field and naming rules that differ by provider
|
||||
@@ -42,7 +42,7 @@ those same sections, so any restatement is a copy that can disagree with the che
|
||||
- `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent
|
||||
unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags
|
||||
it. The Copilot equivalent is `disable-model-invocation`, which changes the description contract
|
||||
entirely — see `references/description-quality.md`, Step 0.
|
||||
entirely — see `references/agent-description-quality.md`, Step 0.
|
||||
|
||||
## Pair consistency
|
||||
|
||||
@@ -54,6 +54,6 @@ Check that:
|
||||
- The two files describe the **same job**. Divergent capability claims across the pair mean one
|
||||
half was edited and the other was not, which is the defect this dimension exists to catch.
|
||||
- Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is
|
||||
the Step 0 case in `references/description-quality.md`, not a pair-consistency finding.
|
||||
the Step 0 case in `references/agent-description-quality.md`, not a pair-consistency finding.
|
||||
|
||||
Keep `pair-consistency` in the Step 4 coverage line at these scopes.
|
||||
@@ -50,18 +50,18 @@ kebab-case; no `FILL IN:` placeholders in the description or body; the descripti
|
||||
characters measured on the folded YAML value.
|
||||
|
||||
**Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter;
|
||||
no frontmatter key outside the `apm-agent-allowlist` section of `references/field-inventory.md` —
|
||||
no frontmatter key outside the `apm-agent-allowlist` section of `references/agent-field-inventory.md` —
|
||||
open that file, do not work from memory.
|
||||
|
||||
**Project/user scope:** the counterpart file exists; `name` matches the filename stem in the
|
||||
Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in
|
||||
the Copilot file and none from `copilot-only-fields` in the CC file, both read from
|
||||
`references/field-inventory.md`.
|
||||
`references/agent-field-inventory.md`.
|
||||
|
||||
## Script-specific failures
|
||||
|
||||
- **`Error: field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
|
||||
`references/field-inventory.md` at load time and refuses to run without it, rather than falling
|
||||
- **`Error: agent-field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
|
||||
`references/agent-field-inventory.md` at load time and refuses to run without it, rather than falling
|
||||
back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not
|
||||
work around it.
|
||||
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full
|
||||
@@ -205,5 +205,5 @@ Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
|
||||
```
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -83,5 +83,5 @@ description: >
|
||||
|
||||
(`data-model` is illustrative. In a real description the target has to resolve.)
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -15,8 +15,11 @@ Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification per
|
||||
directories; this house does not, because an unlisted directory is content no auditor and no host
|
||||
knows to look at. Flag any other directory as a FAIL.
|
||||
|
||||
- `scripts/` holds only executable code an agent can run. Test files (`.bats`, `*_test.*`,
|
||||
`test_*.sh`) there are a FAIL — they belong in `tests/`.
|
||||
- `scripts/` holds only executable code an agent can run, and the sourced libraries those entry
|
||||
points load. A `lib-*.sh` that is never invoked on its own belongs here beside the entry point
|
||||
that sources it — it is executable code, not documentation, so do not flag it for failing to run
|
||||
standalone. Test files (`.bats`, `*_test.*`, `test_*.sh`) there are a FAIL — they belong in
|
||||
`tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
|
||||
@@ -38,9 +41,9 @@ Resolve before flagging, twice over:
|
||||
surrounding prose presents it as the form to copy.
|
||||
|
||||
**Referring to another skill's file.** There is one sanctioned spelling, and it is possessive:
|
||||
`skill-audit's references/validation-scripts.md`. Write the skill by name and let the reader
|
||||
`skill-author's references/contract.md`. Write the skill by name and let the reader
|
||||
resolve it — do not spell the repo path. The full path is the thing this section forbids, and
|
||||
`references/validation-scripts.md` on its own is a hard ERROR from the gate, which
|
||||
`references/contract.md` on its own is a hard ERROR from the gate, which
|
||||
requires an unqualified `references/` pointer to exist in the skill's OWN directory. The
|
||||
possessive form is the only spelling both rules accept; the gate recognises it and skips the
|
||||
on-disk check. Flag any other spelling of a cross-skill reference.
|
||||
@@ -65,5 +68,5 @@ The skill has to agree with itself. Two checks:
|
||||
- Placeholder READMEs inside `scripts/`, `tests/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -19,7 +19,7 @@ Two rules on using it:
|
||||
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
||||
to read; it does not settle a close call on its own.
|
||||
|
||||
## description — `references/description-quality.md`
|
||||
## description — `references/skill-description-quality.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -43,7 +43,7 @@ Flag as FAIL if:
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of
|
||||
`references/description-quality.md`.
|
||||
`references/skill-description-quality.md`.
|
||||
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
||||
of the 400-character house ceiling above.
|
||||
|
||||
@@ -62,7 +62,7 @@ left to judgment here is semantic and the script cannot reach it: whether a targ
|
||||
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
||||
files manually") should have named one.
|
||||
|
||||
## body-discipline — `references/body-discipline.md`
|
||||
## body-discipline — `references/skill-body-discipline.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -84,7 +84,7 @@ Flag as SUGGESTION if:
|
||||
- Gotchas are correct but placed late in the body rather than near the top
|
||||
- Content that only one branch reaches is inlined where a `references/` file would serve
|
||||
|
||||
## patterns — `references/patterns.md`
|
||||
## patterns — `references/skill-patterns.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -101,7 +101,7 @@ Flag as SUGGESTION if:
|
||||
- An output template is present but permissive where the consumer needs it exact
|
||||
- A conditional reference names a trigger that is real but broader than the branch it guards
|
||||
|
||||
## file-structure and internal-consistency — `references/file-structure.md`
|
||||
## file-structure and internal-consistency — `references/skill-file-structure.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -117,7 +117,7 @@ Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
|
||||
## formatting and scripts — `references/formatting-and-scripts.md`
|
||||
## formatting and scripts — `references/skill-formatting-and-scripts.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Skill Flow
|
||||
|
||||
Steps 1 to 3 for a skill directory — the target Step 0 matched as a directory containing
|
||||
`SKILL.md`, or as a `SKILL.md` file, in which case `<skill-dir>` below is its parent directory.
|
||||
Work them in order, then return to `SKILL.md` Step 4 to report.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- A skill takes two independent length families, and it can sit inside one while failing the other — so report them separately. The 500-line / 2,770-word pair counts the **whole file** for spec conformance. The 250/400-character and 600/900-word pair is the house context budget, and its word half counts the **body only**.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
bash scripts/validate-provenance.sh <skill-dir>
|
||||
bash scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation. It exits **0** when no check failed, **1** on real findings, and **2** when it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr. Report an exit 2 as `### Structure` unverified, quoting that reason, never as a failure or a pass.
|
||||
|
||||
Read `references/skill-validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback.
|
||||
|
||||
`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/skill-validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/skill-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/skill-description-quality.md` |
|
||||
| body-discipline | `references/skill-body-discipline.md` |
|
||||
| patterns | `references/skill-patterns.md` |
|
||||
| file-structure, internal-consistency | `references/skill-file-structure.md` |
|
||||
| formatting, scripts | `references/skill-formatting-and-scripts.md` |
|
||||
|
||||
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -44,5 +44,5 @@ follow from that:
|
||||
undocumented one is a coin flip.
|
||||
- **`--dry-run` present for destructive operations.**
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -10,7 +10,7 @@ Upstream source: agentskills.io — best-practices (instruction patterns), speci
|
||||
|
||||
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
|
||||
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
|
||||
*content* is judged in `references/body-discipline.md`.
|
||||
*content* is judged in `references/skill-body-discipline.md`.
|
||||
|
||||
## The constructs and when each is right
|
||||
|
||||
@@ -50,5 +50,5 @@ forms are judgment.
|
||||
`references/` when only one dispatch branch produces that output. A template inlined for a branch
|
||||
most invocations never take is body-discipline padding.
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/finding-criteria.md`,
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,153 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Sources
|
||||
|
||||
<!-- agentskills.io/llms.txt was used for initial source discovery and is not listed below; it contributed no skill file content directly. -->
|
||||
|
||||
## agentskills-home
|
||||
|
||||
- **URL:** https://agentskills.io/home.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-spec
|
||||
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-description-quality.md, references/skill-patterns.md, references/skill-file-structure.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-patterns.md, references/skill-finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-description-quality.md, references/skill-finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-evaluating-skills
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
|
||||
- **Contributing files:** (none — eval workflow not directly informing audit dimensions)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-using-scripts
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/quickstart.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
|
||||
- **Contributing files:** (none — creation guide not directly informing audit criteria)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-websites-code-claude
|
||||
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-creating
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-finding-installing
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-marketplace
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-sdk-custom-agents
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
@@ -1,127 +1,54 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate.sh <skill-dir>
|
||||
|
||||
Validate a skill directory against the agentskills.io specification.
|
||||
|
||||
Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: skill-dir is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||||
# boundary — same description, two verdicts, depending on which reader ran.
|
||||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||||
# dependencies are declared the same way.
|
||||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||||
# dependency sends the reader to install the wrong thing.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 -u - "$1" <<'PYTHON'
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
import yaml
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and this script's own message text carries em dashes (the
|
||||
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
||||
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
||||
# every check has already run, which loses the whole report and (here) flips a
|
||||
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
||||
# resolver block below pins the reads; this pins the writes.
|
||||
# lib-boundary-resolver.sh — SOURCED, never executed.
|
||||
#
|
||||
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
||||
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
||||
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
||||
# byte-identity across all three.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
|
||||
if not os.path.isfile(skill_md):
|
||||
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
|
||||
def fail(msg):
|
||||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||||
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
|
||||
# Both repo callers capture `2>&1`, so nothing a human reads moves.
|
||||
global failed
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# skill-audit's Step 4 report counts them into its `PASS (N suggestions)`
|
||||
# result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
def info(msg):
|
||||
# A check that DECLINED to run says so out loud, rather than passing
|
||||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||||
print(f"INFO {msg}")
|
||||
|
||||
# The ADR-0020 shared boundary resolver, as ONE copy for this skill. Both of
|
||||
# validate.sh's modes compose it into the Python program they run, so the
|
||||
# skill-mode and agent-mode check suites resolve boundary targets through the
|
||||
# same code rather than through two copies that can drift apart.
|
||||
#
|
||||
# The resolver is Python, and bash cannot source Python, so the block is held
|
||||
# in a shell variable filled from a QUOTED here-doc: nothing inside it is
|
||||
# expanded, substituted or rewritten, and the text between the two markers
|
||||
# below is therefore byte-identical to the copy in scripts/skill-size-check.sh
|
||||
# that tests/test-adr0020-contract.sh hashes. The markers stay on lines of
|
||||
# their own, at column 0, exactly once each, so `sed -n '/^BEGIN$/,/^END$/p'`
|
||||
# extracts the same span here as it does from the scripts the test already
|
||||
# reads. Edit one copy, then paste it over the others.
|
||||
#
|
||||
# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`.
|
||||
# This file is sourced by validate.sh before the mode-specific python3/PyYAML
|
||||
# preflight runs, so a `cat` here made coreutils a hard dependency ahead of
|
||||
# python3: on a PATH with neither, the script exited 127 naming `cat` instead of
|
||||
# reaching the preflight that names python3. `read -r -d ''` reads to a NUL that
|
||||
# never arrives and so returns non-zero at EOF — hence the `|| true` — and it
|
||||
# keeps the last line's newline, which the joining newline in the caller would
|
||||
# otherwise double — hence the single strip after the delimiter. It removes
|
||||
# exactly ONE newline, never a run: blank lines at the end of a chunk are part
|
||||
# of the program text the entry script reassembles, and stripping every
|
||||
# trailing newline deleted them. The here-doc itself is unchanged: still
|
||||
# QUOTED, still byte-identical between its markers.
|
||||
#
|
||||
# Self-containment (agentskills.io, skill-author/references/deployment-modes.md)
|
||||
# binds BETWEEN skills, not within one: a cache-installed plugin copies each
|
||||
# skill's own directory whole, so a sibling file in this same scripts/ directory
|
||||
# travels with the skill and is always readable. That is why this is sourced
|
||||
# here and duplicated across skill boundaries elsewhere.
|
||||
#
|
||||
# Consumed by: validate.sh (both modes), via $KYBERFORGE_RESOLVER_PY.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_RESOLVER_PY <<'KYBERFORGE_ADR0020_RESOLVER_PY' || true
|
||||
# ===== BEGIN ADR-0020 SHARED BOUNDARY RESOLVER =====
|
||||
# ONE resolver, embedded VERBATIM in three scripts:
|
||||
# ONE resolver, embedded VERBATIM in two scripts (ADR-0025 retired the third):
|
||||
# scripts/skill-size-check.sh
|
||||
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate.sh
|
||||
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate.sh
|
||||
# The block between these markers must stay byte-identical in all three. It is
|
||||
# copied rather than imported because a cache-installed plugin's scripts cannot
|
||||
# read files outside their own plugin directory, so there is no single file all
|
||||
# three can share (same constraint that forces the ADR-0020 constants to be
|
||||
# duplicated). Edit one copy, then paste it over the other two.
|
||||
# plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh
|
||||
# The block between these markers must stay byte-identical in both. It is copied
|
||||
# rather than imported because a cache-installed plugin's scripts cannot read
|
||||
# files outside their own plugin directory, and this repo-root hook resolves via
|
||||
# .pre-commit-hooks.yaml, where entry[0] is the only token pre-commit rewrites --
|
||||
# so no single file is reachable by both (the same constraint that duplicates the
|
||||
# ADR-0020 constants). Edit one copy, then paste it over the other.
|
||||
#
|
||||
# Requires: glob, os, re, yaml (imported by the host script; PyYAML is a hard
|
||||
# dependency, preflighted in bash before the interpreter starts).
|
||||
@@ -155,9 +82,9 @@ def read_text(path):
|
||||
# The set of names a boundary clause may resolve against is derived from an
|
||||
# AUTHORING ROOT found by walking up FROM THE TARGET FILE. It is NEVER derived
|
||||
# from this script's own location: deriving it from ${BASH_SOURCE} leaked
|
||||
# holocron's 39-skill universe into every consumer repo that ran this hook
|
||||
# through pre-commit, so a consumer skill routing to `skill-audit` resolved
|
||||
# against a plugin it had never installed.
|
||||
# holocron's whole skill universe into every consumer repo that ran this hook
|
||||
# through pre-commit, so a consumer skill routing to a holocron skill such as
|
||||
# `factory-audit` resolved against a plugin it had never installed.
|
||||
#
|
||||
# An authoring root is the nearest ancestor holding plugins/*/.apm/skills/ or
|
||||
# plugins/*/.apm/agents/ (a plugin monorepo), falling back to the nearest
|
||||
@@ -417,8 +344,8 @@ def known_targets(start_dir):
|
||||
# written yet, and any new process chain re-arms it. Unexercised is not the
|
||||
# same as unnecessary, and the branch it guards is still load-bearing: the
|
||||
# bare-arrow rule is the sole extractor for three real targets in
|
||||
# kyberforge's audit skills (agent-audit -> agent-author, agent-audit ->
|
||||
# skill-audit, skill-audit -> skill-author), all written unbackticked.
|
||||
# kyberforge (factory-audit -> skill-author, factory-audit ->
|
||||
# agent-author, apm-orchestrate -> apm-install), all written unbackticked.
|
||||
# * A backticked hyphenated token counts only inside a boundary sentence.
|
||||
# Unconditionally, `pre-push` or `commit-msg` in a TRIGGER clause is a hard
|
||||
# FAIL with no escape hatch. Gating it costs nothing (measured over this
|
||||
@@ -1051,7 +978,7 @@ def hand_invoked(fm_text):
|
||||
#
|
||||
# Both read a FENCE-MASKED copy of the body. Scanning the raw body made a
|
||||
# ```-fenced example a hard ERROR — and the skills most likely to carry one are
|
||||
# skill-author and skill-audit, which DOCUMENT the references/ convention — and
|
||||
# skill-author and factory-audit, which DOCUMENT the references/ convention — and
|
||||
# let a `## Gotchas` heading inside a fenced block stand in for the real
|
||||
# section. Masking preserves every byte offset (content becomes spaces,
|
||||
# newlines stay), so a span found in the mask slices the original.
|
||||
@@ -1076,12 +1003,12 @@ REFERENCE_POINTER = re.compile(
|
||||
REFERENCE_PAST = re.compile(
|
||||
r'\b(?:removed|deleted|renamed|superseded|replaced|obsolete|deprecated'
|
||||
r'|former|formerly|gone|no longer|used to)\b', re.I)
|
||||
# A pointer QUALIFIED by another skill's name — "skill-audit's
|
||||
# references/validation-scripts.md" — names a file that is deliberately NOT in
|
||||
# A pointer QUALIFIED by another skill's name — "factory-audit's
|
||||
# references/skill-validation-scripts.md" — names a file that is deliberately NOT in
|
||||
# this skill's directory. Requiring it on the local disk left NO legal spelling
|
||||
# for a cross-skill reference at all: the only alternative, a full repo path
|
||||
# (`plugins/kyberforge/.apm/skills/skill-audit/references/...`), is itself a
|
||||
# FAIL under skill-audit's own file-structure rubric, because a path that climbs
|
||||
# (`plugins/kyberforge/.apm/skills/factory-audit/references/...`), is itself a
|
||||
# FAIL under factory-audit's own file-structure rubric, because a path climbing
|
||||
# out of the skill directory stops resolving once the plugin is cache-installed.
|
||||
# The possessive form is the sanctioned spelling, and it is skipped here. It is
|
||||
# not checked further — this function has no way to locate another skill's
|
||||
@@ -1173,505 +1100,5 @@ def missing_reference_pointers(body, skill_dir):
|
||||
missing.add('references/' + match.group(1))
|
||||
return sorted(missing)
|
||||
# ===== END ADR-0020 SHARED BOUNDARY RESOLVER =====
|
||||
|
||||
|
||||
# A leading BOM is stripped before anything is parsed or counted. It changes
|
||||
# neither count below — it is not a line separator and str.split() does not
|
||||
# treat it as whitespace — but it did defeat the frontmatter match.
|
||||
try:
|
||||
content = strip_bom(read_text(skill_md))
|
||||
except EncodingError as exc:
|
||||
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
|
||||
f"hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = FRONTMATTER_RE.match(content)
|
||||
if not fm_match:
|
||||
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
|
||||
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
|
||||
"spaces after either marker and CRLF endings are all tolerated). Nothing "
|
||||
"downstream can be measured, so this is a hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
fm = fm_match.group(1)
|
||||
body_start = fm_match.end()
|
||||
|
||||
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
|
||||
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
|
||||
# by `description: ...` captured the NEXT KEY as the name and reported a
|
||||
# mismatch instead of an absence. Same class of bug as the `description:` one
|
||||
# the shared resolver's description_value() docstring records.
|
||||
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first.
|
||||
try:
|
||||
desc = description_value(fm)
|
||||
except FrontmatterError as exc:
|
||||
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or a
|
||||
# description of the wrong type. Do not prefix a diagnosis here; the last
|
||||
# one named a syntax error for two failures that have none.
|
||||
fail(f"{exc}. Nothing downstream can be measured, so this is a hard "
|
||||
f"failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation:
|
||||
# true` takes the skill out of the model-visible listing entirely, so the
|
||||
# trigger/capability/boundary rules and the 250-character routing target do not
|
||||
# apply to it — the audit's own references/description-quality.md Step 0 says
|
||||
# so, and until this line existed no check here knew the field existed. What the
|
||||
# flag does NOT lift: the body word budget and the 400-character description
|
||||
# ceiling. See the shared resolver's hand_invoked().
|
||||
by_hand = hand_invoked(fm)
|
||||
|
||||
# --- Checks ---
|
||||
|
||||
# name present
|
||||
if name:
|
||||
ok(f"name present: '{name}'")
|
||||
else:
|
||||
fail("name field is missing or empty")
|
||||
|
||||
# name matches directory
|
||||
if name and dir_name:
|
||||
if name == dir_name:
|
||||
ok(f"name '{name}' matches directory '{dir_name}'")
|
||||
else:
|
||||
fail(f"name '{name}' does not match directory '{dir_name}'")
|
||||
|
||||
# name length
|
||||
if name:
|
||||
if len(name) <= 64:
|
||||
ok(f"name length {len(name)} chars (limit: 64)")
|
||||
else:
|
||||
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
|
||||
|
||||
# name format
|
||||
if name:
|
||||
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
|
||||
ok("name format valid (kebab-case)")
|
||||
else:
|
||||
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
|
||||
|
||||
# description present
|
||||
if desc:
|
||||
ok("description present")
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
|
||||
# but not backtick-quoted references like `FILL IN:` used in instructions.
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
# description contains unfilled placeholder
|
||||
if desc and PLACEHOLDER_RE.search(desc):
|
||||
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
|
||||
else:
|
||||
if desc:
|
||||
ok("description has no unfilled placeholders")
|
||||
|
||||
# --- ADR-0022: metadata.version is mandatory -------------------------------
|
||||
# FAIL, not SUGGESTION, and the tier is set by the gate rather than by taste.
|
||||
# `.pre-commit-config.yaml`'s `skill-size-check` hook REJECTS a SKILL.md with
|
||||
# no `metadata.version`, and rejects a value that is not three-part semver.
|
||||
# skill-author's Step 4 says to run this audit and "resolve every FAIL", so any
|
||||
# tier below FAIL lets that step report done on a skill the commit gate then
|
||||
# refuses — the same audit-disagrees-with-the-gate failure the MAX_LINES note
|
||||
# below warns about, arrived at from the other direction. Verified before this
|
||||
# check existed: a SKILL.md with no `metadata:` block at all reported "All
|
||||
# checks passed".
|
||||
#
|
||||
# The rule is DUPLICATED from that hook for the same cache-isolation reason as
|
||||
# every other constant here — an installed plugin's scripts cannot read the
|
||||
# repo-root config. Keep the two in step: this check must accept exactly what
|
||||
# the hook accepts.
|
||||
SEMVER_RE = re.compile(r'^\d+\.\d+\.\d+$')
|
||||
|
||||
try:
|
||||
fm_data = yaml.safe_load(fm)
|
||||
except Exception:
|
||||
# Unreachable in practice: description_value() above parses the same text
|
||||
# and hard-exits on a YAML error, so anything arriving here already parsed.
|
||||
fm_data = None
|
||||
metadata_block = fm_data.get('metadata') if isinstance(fm_data, dict) else None
|
||||
|
||||
if not isinstance(metadata_block, dict) or metadata_block.get('version') is None:
|
||||
fail("frontmatter has no metadata.version — ADR-0022 makes it mandatory for "
|
||||
"every skill, and the skill-size-check pre-commit hook rejects the file "
|
||||
"without it. Add `metadata:` / ` version: \"1.0.0\"` (new skills start "
|
||||
"at \"0.1.0\")")
|
||||
else:
|
||||
version_value = metadata_block['version']
|
||||
# NOT str()-coerced blind: `version: 1.0` is a YAML float, and its "1.0"
|
||||
# spelling is exactly the two-part value the hook rejects — coercing and
|
||||
# then matching keeps this check and the hook agreeing on that case.
|
||||
version_text = version_value if isinstance(version_value, str) else str(version_value)
|
||||
version_text = version_text.strip()
|
||||
if SEMVER_RE.match(version_text):
|
||||
ok(f"metadata.version present: '{version_text}' (ADR-0022)")
|
||||
else:
|
||||
fail(f"metadata.version '{version_text}' is not three-part semver — the "
|
||||
f"skill-size-check pre-commit hook rejects it. Use MAJOR.MINOR.PATCH, "
|
||||
f"e.g. \"1.0.0\"")
|
||||
|
||||
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
|
||||
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
|
||||
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
|
||||
# outside the plugin directory once the plugin is cache-installed, so there is
|
||||
# no single source to share. Keep the two in sync by hand: if they drift, this
|
||||
# audit will report a skill ready to ship that the commit hook then rejects.
|
||||
MAX_LINES = 500
|
||||
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
|
||||
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
|
||||
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. agent-audit/scripts/validate.sh holds a third copy of the two
|
||||
# description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
else:
|
||||
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
|
||||
|
||||
# str.split() with no argument splits on runs of whitespace, matching the
|
||||
# `wc -w` the hook uses, and counts the whole file including frontmatter.
|
||||
word_count = len(content.split())
|
||||
if word_count <= MAX_WORDS:
|
||||
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
elif by_hand:
|
||||
ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character "
|
||||
f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- Reference pointers must exist -----------------------------------------
|
||||
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
|
||||
# on disk is a hard break, and until this check existed nothing in the
|
||||
# gate/audit/vale stack noticed it — all three exited 0.
|
||||
missing_refs = missing_reference_pointers(body, skill_dir)
|
||||
for ref in missing_refs:
|
||||
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
|
||||
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
|
||||
if not missing_refs:
|
||||
ok("all referenced references/ files exist")
|
||||
|
||||
# --- Gotchas discipline -----------------------------------------------------
|
||||
# SUGGESTION on both counts: the measurement is deterministic, but whether a
|
||||
# given gotcha earns its place in the body is the auditor's judgment.
|
||||
gotchas = gotcha_stats(body)
|
||||
if gotchas is not None:
|
||||
gotcha_entries, gotcha_words = gotchas
|
||||
if gotcha_entries > GOTCHA_MAX_ENTRIES:
|
||||
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
|
||||
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
|
||||
f"missing references/ file or a design problem written up as a warning")
|
||||
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
|
||||
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
|
||||
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
|
||||
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
|
||||
f"parts to references/ and keep the section for live traps")
|
||||
|
||||
# --- ADR-0020: boundary clause present -------------------------------------
|
||||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||||
# this particular skill warrants a boundary clause is judgment. Both accepted
|
||||
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
|
||||
#
|
||||
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not
|
||||
# parse" are different findings, and reporting the first for the second sends
|
||||
# the author hunting for a problem that is not there (issue #110).
|
||||
#
|
||||
# Skipped entirely for a hand-invoked skill — the contract gives it one plain
|
||||
# sentence with no boundary clause, so the finding would be wrong and its remedy
|
||||
# names a router that cannot see the skill (issue #108).
|
||||
if desc and by_hand:
|
||||
ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger "
|
||||
"rules do not apply; audited as one plain human-facing sentence")
|
||||
elif desc:
|
||||
status = boundary_clause_status(desc)
|
||||
if status == 'present':
|
||||
ok("description has a boundary clause")
|
||||
elif status == 'absent':
|
||||
suggest("description has no boundary clause — add the prose form (\"Do not use "
|
||||
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||||
"so the router knows where NOT to send this skill")
|
||||
else:
|
||||
suggest("description has an arrow boundary clause (\"Not X -> y\") from which no "
|
||||
"target could be read, so the dangling-target check did not run on it — "
|
||||
"the clause is PRESENT and unparsed, not missing. Most often the target is "
|
||||
"a single word, which is deliberately not matchable bare because "
|
||||
"`research`, `triage` and `forge` are all ordinary English: write it as "
|
||||
"`name` or /name")
|
||||
# One arrow, one target. A second name after the same arrow is resolved by
|
||||
# nothing and reported by nothing, so the clause claims coverage it does not
|
||||
# have and this script printed "1 of 1 boundary target(s) resolve" on a
|
||||
# clause naming two (issue #107).
|
||||
for first, second in multi_target_arrow_clauses(desc):
|
||||
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
||||
f"'{second}') and only the first is resolved — the second is checked by "
|
||||
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
||||
f"Not Y -> {second}.\"")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# The resolution universe comes from the SKILL's own location: the authoring
|
||||
# root above it (every sibling plugin in the monorepo), its own apm package, and
|
||||
# the packages that package declares in apm.yml dependencies.apm. It is never
|
||||
# derived from this script's own path, and — when an authoring root exists — it
|
||||
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
|
||||
# run `apm install` return the same verdict. See the shared resolver's header.
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
if routing_targets and not known:
|
||||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||||
f"determined for this path (no authoring root above it, no apm package "
|
||||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||||
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
|
||||
elif routing_targets:
|
||||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||||
# route notation or its own sentence corroborates it by naming another
|
||||
# target that resolves. See the shared resolver's CORROBORATION note.
|
||||
unresolved, soft = unresolved_targets(desc, known)
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||||
f"target sends the router nowhere")
|
||||
for target in soft:
|
||||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||||
f"file format or an English compound. If it IS a route, write it as "
|
||||
f"`/{target}` or `-> {target}` and it will be checked properly")
|
||||
if not unresolved:
|
||||
# Counts the targets that ACTUALLY resolve, not every target found:
|
||||
# a confirm-only target (one used attributively — see the resolver's
|
||||
# ATTRIBUTIVE USE note) is exempt from the failure above, so
|
||||
# reporting it as resolved would be a false claim.
|
||||
resolved = [t for t in routing_targets if normalize_target(t) in known]
|
||||
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(resolved) if resolved else '(none)'}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
# A here-doc body is DATA, not command position. Every script in this corpus
|
||||
# carries a `usage() { cat <<EOF ... EOF; }`, and prose wrapped inside one puts
|
||||
# ordinary English at the start of a line — "read is reported as an INFO ..."
|
||||
# in this skill's own validate-provenance.sh, which made skill-audit hard-FAIL
|
||||
# on its own script. Reflowing that one sentence would have cleared the finding
|
||||
# and left the cause: every future usage text is one wrap away from the same
|
||||
# false positive, and the remedy an author reaches for is contorting working
|
||||
# source, which the note above records has already happened twice.
|
||||
#
|
||||
# Detection is deliberately conservative in the direction that matters. A
|
||||
# here-doc body is skipped only when its terminator is actually found further
|
||||
# down the file; an opener with no terminator — the shape a stray `<<` inside a
|
||||
# string would produce — is ignored rather than allowed to swallow the tail,
|
||||
# because swallowing the tail is a false NEGATIVE and this check exists to fail
|
||||
# closed. `<<<` here-strings open nothing and are excluded by the lookbehind.
|
||||
HEREDOC_START_RE = re.compile(r'(?<!<)<<-?\s*(["\']?)([A-Za-z_][A-Za-z0-9_]*)\1')
|
||||
|
||||
|
||||
def heredoc_delimiter(line):
|
||||
"""The here-doc terminator this line opens, or None."""
|
||||
m = HEREDOC_START_RE.search(line)
|
||||
return m.group(2) if m else None
|
||||
|
||||
|
||||
def heredoc_body_indices(lines):
|
||||
"""Line indices that are here-doc BODY (plus its terminator), not code."""
|
||||
skip = set()
|
||||
i, n = 0, len(lines)
|
||||
while i < n:
|
||||
stripped = lines[i].strip()
|
||||
delim = None if stripped.startswith('#') else heredoc_delimiter(lines[i])
|
||||
if delim:
|
||||
# `<<-` allows an indented terminator, so compare stripped.
|
||||
for j in range(i + 1, n):
|
||||
if lines[j].strip() == delim:
|
||||
skip.update(range(i + 1, j + 1))
|
||||
i = j
|
||||
break
|
||||
i += 1
|
||||
return skip
|
||||
|
||||
|
||||
# The here-doc exemption applies to the `read` heuristic ONLY, and the
|
||||
# asymmetry is the point. `read` is an ordinary English verb, so any prose a
|
||||
# script prints is one line-wrap away from opening with it. `input(` is not a
|
||||
# word — a line beginning `input(` inside a here-doc is an embedded Python
|
||||
# program pausing for a keypress, which is exactly what this check is for, and
|
||||
# these scripts embed Python in a here-doc as a matter of course. Exempting the
|
||||
# whole body would have disarmed the check across every script in the corpus.
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
lines = source.splitlines()
|
||||
in_heredoc = heredoc_body_indices(lines)
|
||||
for idx, line in enumerate(lines):
|
||||
stripped = line.strip()
|
||||
if idx in in_heredoc:
|
||||
if re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
continue
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
scripts = [f for f in os.listdir(scripts_dir)
|
||||
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
|
||||
for fname in scripts:
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
try:
|
||||
sc = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
# The executable-bit check below still runs — one unreadable byte
|
||||
# must not silently drop a second, independent check.
|
||||
sc = None
|
||||
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
|
||||
f"interactive prompts")
|
||||
interactive = interactive_reads(sc) if sc is not None else []
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
elif sc is not None:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
if os.access(fpath, os.X_OK):
|
||||
ok(f"scripts/{fname}: is executable")
|
||||
else:
|
||||
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
if suggestions:
|
||||
# Feeds skill-audit's Step 4 `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
PYTHON
|
||||
KYBERFORGE_ADR0020_RESOLVER_PY
|
||||
KYBERFORGE_RESOLVER_PY="${KYBERFORGE_RESOLVER_PY%$'\n'}"
|
||||
683
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh
Executable file
683
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh
Executable file
@@ -0,0 +1,683 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-checks-agent.sh — SOURCED, never executed.
|
||||
#
|
||||
# agent-audit's structural check suite: everything in its validate.sh that is
|
||||
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
|
||||
# resolver's markers. validate.sh reassembles
|
||||
#
|
||||
# $KYBERFORGE_AGENT_PREAMBLE_PY
|
||||
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
|
||||
# $KYBERFORGE_AGENT_BODY_PY
|
||||
#
|
||||
# in that order — the order the resolver block sat in the original file — and
|
||||
# feeds the result to python3 with the agent file as argv[1] and this scripts/
|
||||
# directory as argv[2]. argv[2] is what the preamble resolves the frontmatter
|
||||
# allowlist against, and the body's detect_scope() walk-up and provider
|
||||
# detection are unchanged from agent-audit's copy.
|
||||
#
|
||||
# The dimension vocabulary and the tiers here are agent-audit's and are
|
||||
# deliberately NOT reconciled with lib-checks-skill.sh's. Agents take the
|
||||
# ADR-0020 description gates and no body word gate at all; adding one would
|
||||
# contradict the ADR.
|
||||
#
|
||||
# Consumed by: validate.sh, agent mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
kyberforge_agent_preflight() {
|
||||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||||
# boundary — same description, two verdicts, depending on which reader ran.
|
||||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||||
# dependencies are declared the same way.
|
||||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||||
# dependency sends the reader to install the wrong thing.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
|
||||
# lib-provenance-*.sh has always exited 2 here; this matches it.
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||||
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||||
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_AGENT_PREAMBLE_PY <<'KYBERFORGE_AGENT_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
import yaml
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and this script's own message text carries em dashes (the
|
||||
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
||||
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
||||
# every check has already run, which loses the whole report and (here) flips a
|
||||
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
||||
# resolver block below pins the reads; this pins the writes.
|
||||
#
|
||||
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
||||
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
||||
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
||||
# byte-identity across all three.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
agent_file = os.path.abspath(sys.argv[1])
|
||||
script_dir = sys.argv[2]
|
||||
|
||||
fname = os.path.basename(agent_file)
|
||||
|
||||
# --- Detect provider (check .agent.md before .md) ---
|
||||
if fname.endswith('.agent.md'):
|
||||
provider = 'copilot'
|
||||
name_stem = fname[:-len('.agent.md')]
|
||||
elif fname.endswith('.md'):
|
||||
provider = 'claude-code'
|
||||
name_stem = fname[:-len('.md')]
|
||||
else:
|
||||
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# --- Load agent-field-inventory.md ---
|
||||
# The merged skill holds one references/ directory for both modes, so every
|
||||
# flow-specific file is name-prefixed and the agent half of the inventory is
|
||||
# agent-field-inventory.md (ADR-0025). There is no fallback to the pre-merge
|
||||
# `field-inventory.md` spelling: agent-audit no longer exists, so a file at that
|
||||
# name would be a stray, and silently reading it would mean auditing against an
|
||||
# inventory this skill does not ship.
|
||||
inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'agent-field-inventory.md'))
|
||||
if not os.path.isfile(inv_path):
|
||||
print(f"Error: agent-field-inventory.md not found at {inv_path}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# Encoding is pinned to UTF-8 rather than inherited from the locale: under
|
||||
# LC_ALL=C the inherited default is ASCII, and this file legitimately carries
|
||||
# non-ASCII prose. read_text() in the shared resolver block below does the same
|
||||
# thing for every other file; this one is read before that block is defined.
|
||||
try:
|
||||
with open(inv_path, encoding='utf-8') as f:
|
||||
inv_content = f.read()
|
||||
except UnicodeDecodeError as exc:
|
||||
print(f"Error: agent-field-inventory.md at {inv_path} is not valid UTF-8 "
|
||||
f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.",
|
||||
file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
def parse_section_tokens(content, section_name):
|
||||
lines = content.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
if line.strip() == f'## {section_name}':
|
||||
for j in range(i + 1, len(lines)):
|
||||
stripped = lines[j].strip()
|
||||
if stripped and not stripped.startswith('#') and not stripped.startswith('---'):
|
||||
return set(stripped.split())
|
||||
return set()
|
||||
|
||||
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
|
||||
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
|
||||
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
|
||||
|
||||
# Tools the runtime withholds from subagents regardless of the tools field
|
||||
SUBAGENT_UNAVAILABLE_TOOLS = {
|
||||
'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers',
|
||||
}
|
||||
|
||||
# Copilot body length limit (chars) — content beyond this is silently truncated
|
||||
COPILOT_BODY_LIMIT = 30000
|
||||
|
||||
# ADR-0020 description budget. An agent's name + description is preloaded into
|
||||
# every session exactly like a skill's, so agents take the SAME description
|
||||
# gates. These two constants are DUPLICATED in three places:
|
||||
# scripts/skill-size-check.sh, lib-checks-skill.sh beside this file, and here.
|
||||
# The repo-root hook's copy cannot be shared with this skill — a cache-installed
|
||||
# plugin's scripts cannot read files outside their own plugin directory, and the
|
||||
# hook cannot reach inside the plugin. The two copies INSIDE this skill could be
|
||||
# shared (ADR-0025: two files in one skill may source a third), and are not only
|
||||
# because each mode library is a verbatim lift of the pre-merge suite whose
|
||||
# constants sit in its Python preamble; hoisting them is a separate change.
|
||||
# tests/test-skill-size-check.sh asserts all three agree, so drift fails CI
|
||||
# rather than silently diverging.
|
||||
#
|
||||
# Agents deliberately take NO body word gate, and adding one here would
|
||||
# contradict ADR-0020: a skill body is loaded into the caller's context and
|
||||
# competes with the live conversation, while an agent body becomes the system
|
||||
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
|
||||
# not transfer. Agent body length falls out of the delegation rule instead.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
|
||||
# --- Helpers (shared by every scope) ---
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def fail(msg):
|
||||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||||
# stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo
|
||||
# callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture
|
||||
# `2>&1`, so nothing a human reads moves.
|
||||
global failed
|
||||
failed = True
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
|
||||
def suggest(msg):
|
||||
suggestions.append(msg)
|
||||
|
||||
def info(msg):
|
||||
# A check that DECLINED to run says so out loud, rather than passing
|
||||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||||
print(f"INFO {msg}")
|
||||
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
|
||||
KYBERFORGE_AGENT_PREAMBLE
|
||||
KYBERFORGE_AGENT_PREAMBLE_PY="${KYBERFORGE_AGENT_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_AGENT_BODY_PY <<'KYBERFORGE_AGENT_BODY' || true
|
||||
|
||||
|
||||
def parse_frontmatter(content):
|
||||
m = FRONTMATTER_RE.match(strip_bom(content))
|
||||
if not m:
|
||||
return None, content
|
||||
return m.group(1), strip_bom(content)[m.end():]
|
||||
|
||||
def extract_field(fm, field):
|
||||
"""The raw text after `field:` ON ITS OWN LINE, or None.
|
||||
|
||||
The character class is `[^\\S\\r\\n]`, never `\\s`: under re.MULTILINE a
|
||||
`\\s*` after the colon crosses the newline, so `description:` with no value
|
||||
followed by `model: sonnet` captured `model: sonnet` as the description.
|
||||
That made the value look present, skipped the "missing or empty" failure,
|
||||
and then every ADR-0020 gate early-returned on the genuinely empty folded
|
||||
value — a valueless description exited 0 with zero output on a BLOCKING
|
||||
pre-push gate. This function is now used only for fields with no folding
|
||||
semantics (name, tools); description goes through description_value(), the
|
||||
shared resolver's YAML reader, which is the only thing that can see through
|
||||
`>`, `null`, `''` and a quoted `"description"` key alike.
|
||||
"""
|
||||
m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
def get_frontmatter_keys(fm):
|
||||
keys = set()
|
||||
for line in fm.splitlines():
|
||||
m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line)
|
||||
if m:
|
||||
keys.add(m.group(1))
|
||||
return keys
|
||||
|
||||
def agent_description(fm, local_fname):
|
||||
"""The folded description VALUE, or None if it could not be read."""
|
||||
try:
|
||||
return description_value(fm)
|
||||
except FrontmatterError as exc:
|
||||
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or
|
||||
# a description of the wrong type. Do not prefix a diagnosis here; the
|
||||
# last one named a syntax error for two failures that have none.
|
||||
fail(f"{exc} — the ADR-0020 description and boundary-target gates could "
|
||||
f"not run — {local_fname}")
|
||||
return None
|
||||
|
||||
def check_description_budget(value, local_fname, by_hand=False):
|
||||
"""ADR-0020 description gates — identical for every scope.
|
||||
|
||||
`by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent
|
||||
carrying `disable-model-invocation: true` is absent from the model-visible
|
||||
listing, so the 250-character SUGGESTION — a routing-quality budget — has
|
||||
no listing to apply to. The 400-character ceiling is unaffected.
|
||||
"""
|
||||
if not value:
|
||||
return
|
||||
dlen = len(value)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body — {local_fname}")
|
||||
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers "
|
||||
f"— {local_fname}")
|
||||
|
||||
def check_boundary(value, fpath, local_fname, by_hand=False):
|
||||
"""ADR-0020 boundary clause + resolvable boundary targets.
|
||||
|
||||
agent-author's SKILL.md states that an agent's boundary targets must
|
||||
resolve, but until this ran no script checked it — the contract was
|
||||
documented and unenforced. The resolution universe is derived from the
|
||||
AGENT FILE's own location (the authoring root above it, its own apm
|
||||
package, and that package's declared apm dependencies), never from this
|
||||
script's path, and — when an authoring root exists — never from a deployed
|
||||
.claude/ tree, so a fresh clone and a machine that has run `apm install`
|
||||
return the same verdict.
|
||||
"""
|
||||
if not value:
|
||||
return
|
||||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||||
# this particular agent warrants a boundary clause is judgment. All four
|
||||
# agents in this corpus currently lack one.
|
||||
#
|
||||
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could
|
||||
# not parse" are different findings (issue #110). And a hand-invoked agent is
|
||||
# exempt from the clause altogether (issue #108) — the boundary-target
|
||||
# resolution below still runs, because a target it DOES name should still
|
||||
# resolve.
|
||||
status = boundary_clause_status(value) if not by_hand else 'present'
|
||||
if status == 'absent':
|
||||
suggest(f"description has no boundary clause — add the prose form (\"Do not use "
|
||||
f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||||
f"so the router knows where NOT to send this agent — {local_fname}")
|
||||
elif status == 'unparsed':
|
||||
suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no "
|
||||
f"target could be read, so the dangling-target check did not run on it — "
|
||||
f"the clause is PRESENT and unparsed, not missing. Most often the target "
|
||||
f"is a single word, which is deliberately not matchable bare: write it as "
|
||||
f"`name` or /name — {local_fname}")
|
||||
if not by_hand:
|
||||
# One arrow, one target: a second name after the same arrow is resolved
|
||||
# by nothing and reported by nothing (issue #107).
|
||||
for first, second in multi_target_arrow_clauses(value):
|
||||
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
||||
f"'{second}') and only the first is resolved — the second is checked by "
|
||||
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
||||
f"Not Y -> {second}.\" — {local_fname}")
|
||||
targets = boundary_targets(value)
|
||||
if not targets:
|
||||
return
|
||||
known = known_targets(os.path.dirname(os.path.abspath(fpath)))
|
||||
if not known:
|
||||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||||
f"determined for this path (no authoring root above it, no apm package "
|
||||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||||
f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}")
|
||||
return
|
||||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||||
# route notation or its own sentence corroborates it by naming another target
|
||||
# that resolves. See the shared resolver's CORROBORATION note.
|
||||
blocking, reported = unresolved_targets(value, known)
|
||||
for target in blocking:
|
||||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||||
f"target sends the router nowhere — {local_fname}")
|
||||
for target in reported:
|
||||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||||
f"file format or an English compound. If it IS a route, write it as "
|
||||
f"`/{target}` or `-> {target}` and it will be checked properly — "
|
||||
f"{local_fname}")
|
||||
|
||||
def extract_tools_list(fm):
|
||||
"""Tool names from the `tools` field — inline scalar OR YAML block sequence.
|
||||
|
||||
Read off the PARSED mapping, never off extract_field(). That function's
|
||||
capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:`
|
||||
written as a block sequence — the shape Copilot agent files use — captured
|
||||
nothing at all and the subagent-unavailable-tool check silently stopped
|
||||
firing on exactly the files it was written for. Both spellings are legal
|
||||
YAML, so both are read here.
|
||||
"""
|
||||
try:
|
||||
data = yaml.safe_load(fm)
|
||||
except Exception:
|
||||
# Not this function's failure to report: the frontmatter's validity is
|
||||
# decided (and failed) by agent_description() on the same text.
|
||||
return set()
|
||||
if not isinstance(data, dict):
|
||||
return set()
|
||||
val = data.get('tools')
|
||||
if isinstance(val, list):
|
||||
items = [str(item).strip() for item in val]
|
||||
elif isinstance(val, str):
|
||||
items = re.split(r'[\s,]+', val.strip())
|
||||
else:
|
||||
return set()
|
||||
return {item for item in items if item}
|
||||
|
||||
def is_copilot_cloud_ide(fpath):
|
||||
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
|
||||
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
|
||||
|
||||
# --- Detect scope ---
|
||||
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
|
||||
# exactly one of the four package content types. Group 1 captures an optional
|
||||
# opening quote; \1 requires the same character (or nothing) to close it, so
|
||||
# "skill" and '"skill"' both match but a mismatched quote doesn't. The value
|
||||
# must then be followed by whitespace or end-of-line — not just a non-word
|
||||
# character — so a malformed value like `prompts-only` is correctly rejected
|
||||
# instead of false-matching on the `prompts` prefix.
|
||||
APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
|
||||
|
||||
def find_apm_package_root(apm_yml_path):
|
||||
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
|
||||
manifest, not a type:-less marketplace-only apm.yml)."""
|
||||
# errors='replace', not a hard failure: this only asks whether a `type:`
|
||||
# line exists, and a stray undecodable byte elsewhere in someone else's
|
||||
# apm.yml must not abort scope detection.
|
||||
with open(apm_yml_path, encoding='utf-8', errors='replace') as f:
|
||||
for line in f:
|
||||
if APM_TYPE_RE.match(line):
|
||||
return True
|
||||
return False
|
||||
|
||||
def detect_scope(start_dir):
|
||||
home = os.path.expanduser('~')
|
||||
original_start = os.path.abspath(start_dir)
|
||||
# Agent files conventionally live exactly two path segments below their
|
||||
# scope root — <root>/.claude/agents, <root>/.github/agents,
|
||||
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
|
||||
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
|
||||
# recovers the same root new-agent.sh would have been invoked with to
|
||||
# produce this exact file, independent of how far the walk below has to
|
||||
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
|
||||
# `root` vs `current` distinction even though validate.sh is handed a
|
||||
# file's directory, not the scope root itself.
|
||||
#
|
||||
# That arithmetic is only trustworthy when the path actually has this
|
||||
# shape: parent directory literally named "agents", grandparent one of
|
||||
# the four known scope-dir names. A hand-placed or otherwise
|
||||
# non-conventional agent file (never produced by new-agent.sh) has no
|
||||
# such guarantee — blindly trusting two-segments-up there could point at
|
||||
# an unrelated ancestor. conventional_shape gates every use of
|
||||
# conventional_root below; when it's false, the walked-to `current`
|
||||
# directory is used instead, the same fallback this function used before
|
||||
# conventional_root existed.
|
||||
scope_dir_name = os.path.basename(os.path.dirname(original_start))
|
||||
conventional_shape = (
|
||||
os.path.basename(original_start) == 'agents'
|
||||
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
|
||||
)
|
||||
conventional_root = os.path.dirname(os.path.dirname(original_start))
|
||||
current = original_start
|
||||
while True:
|
||||
# The filesystem root is never a candidate, the same guard the shared
|
||||
# resolver's walk-up loops carry. Without it a file under a marker-less
|
||||
# temp directory walked all the way to `/` and returned it as the scope
|
||||
# root, which then reported `counterpart file not found:
|
||||
# /.claude/agents/<name>.md` — a path that names someone else's machine,
|
||||
# not the user's project. When the walk runs out, the agent file's own
|
||||
# directory (or its conventional root) is the honest answer.
|
||||
if _is_fs_root(current):
|
||||
return 'project', conventional_root if conventional_shape else original_start
|
||||
apm_yml = os.path.join(current, 'apm.yml')
|
||||
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
|
||||
return 'plugin', current
|
||||
# $HOME is the user-scope boundary — checked before the .git test
|
||||
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
|
||||
# can't shadow user scope by being its own .git repo. 'user' scope
|
||||
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
|
||||
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
|
||||
# conventional two-segments-below-root depth (i.e. $HOME IS that
|
||||
# root, matching the real ~/.claude/agents or ~/.copilot/agents
|
||||
# shape). Any other walk-up into $HOME — a marker-less directory
|
||||
# nested deeper than that convention — resolves to project scope
|
||||
# instead: a stray directory under $HOME can't be silently
|
||||
# redirected into the shared global ~/.claude or ~/.copilot agent
|
||||
# directories.
|
||||
if current == home:
|
||||
if original_start == home or (conventional_shape and conventional_root == home):
|
||||
return 'user', home
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
|
||||
# in a git worktree — exists() covers both. Returns conventional_root,
|
||||
# not current: new-agent.sh's project-scope file placement always
|
||||
# uses its `$ROOT` argument directly, never the walked-up `.git`
|
||||
# location, so a <root> one or more levels below the repo's .git
|
||||
# (a subdirectory of a larger git-tracked tree — explicitly a
|
||||
# supported case per new-agent.sh's usage text) must resolve to the
|
||||
# same root new-agent.sh actually wrote to, not to the .git dir —
|
||||
# unless the path lacks the conventional shape, in which case that
|
||||
# arithmetic isn't trustworthy and current is used instead.
|
||||
if os.path.exists(os.path.join(current, '.git')):
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
parent = os.path.dirname(current)
|
||||
if parent == current:
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
current = parent
|
||||
|
||||
agent_dir = os.path.dirname(agent_file)
|
||||
scope, scope_root = detect_scope(agent_dir)
|
||||
|
||||
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
|
||||
def check_apm_agent_file(fpath, allowlist, stem):
|
||||
local_fname = os.path.basename(fpath)
|
||||
try:
|
||||
content = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
||||
f"failure, not a skip — {local_fname}")
|
||||
return
|
||||
except OSError as exc:
|
||||
# A path that cannot be opened gets a FAIL line naming it, not a bare
|
||||
# FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes
|
||||
# this path for an agent file deleted from the worktree but still
|
||||
# tracked in the index — a real, expected state, and the caller needs to
|
||||
# be told which file, not handed an interpreter stack.
|
||||
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
||||
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
fm, body = parse_frontmatter(content)
|
||||
if fm is None:
|
||||
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
||||
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
||||
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
||||
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
# The apm-agent.md template embeds its authoring guidance as HTML
|
||||
# comments inside the frontmatter block (so they render invisible in a
|
||||
# Markdown preview but stay visible in the raw file). get_frontmatter_keys
|
||||
# silently ignores any line that isn't a `key:` match, so a comment left
|
||||
# behind at ship time would otherwise pass unnoticed — yet apm compile
|
||||
# copies this frontmatter verbatim to both harnesses, and `<!-- -->` is
|
||||
# not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016).
|
||||
if re.search(r'<!--|-->', fm):
|
||||
fail(f"frontmatter still contains template HTML comments (<!-- ... -->) "
|
||||
f"— delete them before shipping — {local_fname}")
|
||||
|
||||
# Allowlist: the permitted keys are data, read at load time from
|
||||
# references/agent-field-inventory.md's `## apm-agent-allowlist` section — do not
|
||||
# restate them here, or this comment goes stale the next time that line
|
||||
# changes. apm compile verbatim-copies frontmatter to every target, so a key
|
||||
# outside the list is unsafe on at least one harness (ADR-0016). Note the
|
||||
# list admits denylist-shaped restrictions (disallowedTools) but never
|
||||
# allowlist-shaped ones (tools), whose value shape differs per harness.
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key not in allowlist:
|
||||
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
|
||||
f"({', '.join(sorted(allowlist))}) — {local_fname}")
|
||||
|
||||
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
|
||||
name_val = extract_field(fm, 'name')
|
||||
if not name_val:
|
||||
fail(f"name field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
if name_val != stem:
|
||||
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
||||
|
||||
# description — required, non-empty, no placeholder
|
||||
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
||||
# it on extract_field's raw capture is what let `description:` with no value
|
||||
# pass this gate in total silence: the capture picked up the next key, so
|
||||
# "missing or empty" never fired, and every ADR-0020 check below then
|
||||
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
||||
folded = agent_description(fm, local_fname)
|
||||
if folded is None:
|
||||
pass # frontmatter is not valid YAML — agent_description already failed
|
||||
elif not folded:
|
||||
fail(f"description field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(folded):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
by_hand = hand_invoked(fm)
|
||||
check_description_budget(folded, local_fname, by_hand)
|
||||
check_boundary(folded, fpath, local_fname, by_hand)
|
||||
|
||||
# body — required, non-empty, no placeholder; same Copilot truncation risk
|
||||
# applies since this file compiles verbatim into a real Copilot file downstream.
|
||||
if not body.strip():
|
||||
fail(f"system prompt body is empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(body):
|
||||
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
if len(body) > COPILOT_BODY_LIMIT:
|
||||
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
|
||||
f"content beyond the limit is silently truncated by the Copilot runtime "
|
||||
f"once apm compile emits it downstream — {local_fname}")
|
||||
|
||||
if scope == 'plugin':
|
||||
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
sys.exit(1 if failed else 0)
|
||||
|
||||
# --- Project/user scope: unchanged CC/Copilot pair validation ---
|
||||
|
||||
# --- Derive counterpart path ---
|
||||
if scope == 'project':
|
||||
if provider == 'claude-code':
|
||||
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
|
||||
counterpart_provider = 'copilot'
|
||||
else:
|
||||
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
|
||||
counterpart_provider = 'claude-code'
|
||||
else: # user
|
||||
home = os.path.expanduser('~')
|
||||
if provider == 'claude-code':
|
||||
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
|
||||
counterpart_provider = 'copilot'
|
||||
else:
|
||||
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
|
||||
counterpart_provider = 'claude-code'
|
||||
|
||||
def check_file(fpath, file_provider):
|
||||
local_fname = os.path.basename(fpath)
|
||||
try:
|
||||
content = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
||||
f"failure, not a skip — {local_fname}")
|
||||
return
|
||||
except OSError as exc:
|
||||
# Same reason as check_apm_agent_file's: a diagnostic naming the path
|
||||
# beats a FileNotFoundError traceback. The counterpart is pre-checked at
|
||||
# the bottom of this script, but agent_file itself never was.
|
||||
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
||||
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
fm, body = parse_frontmatter(content)
|
||||
if fm is None:
|
||||
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
||||
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
||||
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
||||
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
# name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents
|
||||
cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath))
|
||||
name_val = extract_field(fm, 'name')
|
||||
if not cloud_ide:
|
||||
if not name_val:
|
||||
fail(f"name field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
# Stem check applies to Copilot CLI only; CC docs say filename need not match name
|
||||
if file_provider == 'copilot':
|
||||
stem = local_fname[:-len('.agent.md')]
|
||||
if name_val != stem:
|
||||
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
||||
elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
# cloud/IDE: name is optional, but if present it must be valid
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
|
||||
# description
|
||||
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
||||
# it on extract_field's raw capture is what let `description:` with no value
|
||||
# pass this gate in total silence: the capture picked up the next key, so
|
||||
# "missing or empty" never fired, and every ADR-0020 check below then
|
||||
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
||||
folded = agent_description(fm, local_fname)
|
||||
if folded is None:
|
||||
pass # frontmatter is not valid YAML — agent_description already failed
|
||||
elif not folded:
|
||||
fail(f"description field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(folded):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
by_hand = hand_invoked(fm)
|
||||
check_description_budget(folded, local_fname, by_hand)
|
||||
check_boundary(folded, fpath, local_fname, by_hand)
|
||||
|
||||
# body
|
||||
if not body.strip():
|
||||
fail(f"system prompt body is empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(body):
|
||||
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
# Copilot body length limit
|
||||
if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT:
|
||||
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}")
|
||||
|
||||
# CC-only fields in Copilot file
|
||||
if file_provider == 'copilot':
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key in cc_only_fields:
|
||||
fail(f"CC-only field '{key}' present in Copilot file — {local_fname}")
|
||||
|
||||
# Copilot-only fields in CC file
|
||||
if file_provider == 'claude-code':
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key in copilot_only_fields:
|
||||
fail(f"Copilot-only field '{key}' present in CC file — {local_fname}")
|
||||
|
||||
# Subagent-unavailable tools listed in tools field
|
||||
tools = extract_tools_list(fm)
|
||||
unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS
|
||||
for tool in sorted(unavailable):
|
||||
suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}")
|
||||
|
||||
# --- Check counterpart exists ---
|
||||
if not os.path.isfile(counterpart):
|
||||
fail(f"counterpart file not found: {counterpart}")
|
||||
sys.exit(1)
|
||||
|
||||
# --- Check both files ---
|
||||
check_file(agent_file, provider)
|
||||
check_file(counterpart, counterpart_provider)
|
||||
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
|
||||
sys.exit(1 if failed else 0)
|
||||
KYBERFORGE_AGENT_BODY
|
||||
KYBERFORGE_AGENT_BODY_PY="${KYBERFORGE_AGENT_BODY_PY%$'\n'}"
|
||||
621
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh
Executable file
621
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh
Executable file
@@ -0,0 +1,621 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-checks-skill.sh — SOURCED, never executed.
|
||||
#
|
||||
# skill-audit's structural check suite: everything in its validate.sh that is
|
||||
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
|
||||
# resolver's markers. validate.sh reassembles
|
||||
#
|
||||
# $KYBERFORGE_SKILL_PREAMBLE_PY
|
||||
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
|
||||
# $KYBERFORGE_SKILL_BODY_PY
|
||||
#
|
||||
# in that order — the order the resolver block sat in the original file — and
|
||||
# feeds the result to python3, so every check runs against the same names it
|
||||
# always did.
|
||||
#
|
||||
# The dimension vocabulary, the message wording and the PASS/FAIL/SUGGESTION/
|
||||
# INFO tiers here are skill-audit's and are deliberately NOT reconciled with
|
||||
# lib-checks-agent.sh's. The two suites disagree on purpose: a skill body is
|
||||
# loaded into the caller's context, an agent body becomes the system prompt of
|
||||
# a fresh one, so ADR-0020 gives skills a body word budget and agents none.
|
||||
#
|
||||
# Consumed by: validate.sh, skill mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
kyberforge_skill_preflight() {
|
||||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||||
# boundary — same description, two verdicts, depending on which reader ran.
|
||||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||||
# dependencies are declared the same way.
|
||||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||||
# dependency sends the reader to install the wrong thing.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
|
||||
# lib-provenance-*.sh has always exited 2 here; this matches it.
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||||
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_SKILL_PREAMBLE_PY <<'KYBERFORGE_SKILL_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
import yaml
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and this script's own message text carries em dashes (the
|
||||
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
||||
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
||||
# every check has already run, which loses the whole report and (here) flips a
|
||||
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
||||
# resolver block below pins the reads; this pins the writes.
|
||||
#
|
||||
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
||||
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
||||
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
||||
# byte-identity across all three.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
|
||||
if not os.path.isfile(skill_md):
|
||||
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
|
||||
def fail(msg):
|
||||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||||
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
|
||||
# Both repo callers capture `2>&1`, so nothing a human reads moves.
|
||||
global failed
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# factory-audit's SKILL.md Step 4 report counts them into its
|
||||
# `PASS (N suggestions)` result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
def info(msg):
|
||||
# A check that DECLINED to run says so out loud, rather than passing
|
||||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||||
print(f"INFO {msg}")
|
||||
|
||||
|
||||
KYBERFORGE_SKILL_PREAMBLE
|
||||
KYBERFORGE_SKILL_PREAMBLE_PY="${KYBERFORGE_SKILL_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_SKILL_BODY_PY <<'KYBERFORGE_SKILL_BODY' || true
|
||||
|
||||
|
||||
# A leading BOM is stripped before anything is parsed or counted. It changes
|
||||
# neither count below — it is not a line separator and str.split() does not
|
||||
# treat it as whitespace — but it did defeat the frontmatter match.
|
||||
try:
|
||||
content = strip_bom(read_text(skill_md))
|
||||
except EncodingError as exc:
|
||||
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
|
||||
f"hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = FRONTMATTER_RE.match(content)
|
||||
if not fm_match:
|
||||
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
|
||||
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
|
||||
"spaces after either marker and CRLF endings are all tolerated). Nothing "
|
||||
"downstream can be measured, so this is a hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
fm = fm_match.group(1)
|
||||
body_start = fm_match.end()
|
||||
|
||||
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
|
||||
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
|
||||
# by `description: ...` captured the NEXT KEY as the name and reported a
|
||||
# mismatch instead of an absence. Same class of bug as the `description:` one
|
||||
# the shared resolver's description_value() docstring records.
|
||||
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first.
|
||||
try:
|
||||
desc = description_value(fm)
|
||||
except FrontmatterError as exc:
|
||||
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or a
|
||||
# description of the wrong type. Do not prefix a diagnosis here; the last
|
||||
# one named a syntax error for two failures that have none.
|
||||
fail(f"{exc}. Nothing downstream can be measured, so this is a hard "
|
||||
f"failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation:
|
||||
# true` takes the skill out of the model-visible listing entirely, so the
|
||||
# trigger/capability/boundary rules and the 250-character routing target do not
|
||||
# apply to it — the audit's own references/skill-description-quality.md Step 0 says
|
||||
# so, and until this line existed no check here knew the field existed. What the
|
||||
# flag does NOT lift: the body word budget and the 400-character description
|
||||
# ceiling. See the shared resolver's hand_invoked().
|
||||
by_hand = hand_invoked(fm)
|
||||
|
||||
# --- Checks ---
|
||||
|
||||
# name present
|
||||
if name:
|
||||
ok(f"name present: '{name}'")
|
||||
else:
|
||||
fail("name field is missing or empty")
|
||||
|
||||
# name matches directory
|
||||
if name and dir_name:
|
||||
if name == dir_name:
|
||||
ok(f"name '{name}' matches directory '{dir_name}'")
|
||||
else:
|
||||
fail(f"name '{name}' does not match directory '{dir_name}'")
|
||||
|
||||
# name length
|
||||
if name:
|
||||
if len(name) <= 64:
|
||||
ok(f"name length {len(name)} chars (limit: 64)")
|
||||
else:
|
||||
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
|
||||
|
||||
# name format
|
||||
if name:
|
||||
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
|
||||
ok("name format valid (kebab-case)")
|
||||
else:
|
||||
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
|
||||
|
||||
# description present
|
||||
if desc:
|
||||
ok("description present")
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
|
||||
# but not backtick-quoted references like `FILL IN:` used in instructions.
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
# description contains unfilled placeholder
|
||||
if desc and PLACEHOLDER_RE.search(desc):
|
||||
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
|
||||
else:
|
||||
if desc:
|
||||
ok("description has no unfilled placeholders")
|
||||
|
||||
# --- ADR-0022: metadata.version is mandatory -------------------------------
|
||||
# FAIL, not SUGGESTION, and the tier is set by the gate rather than by taste.
|
||||
# `.pre-commit-config.yaml`'s `skill-size-check` hook REJECTS a SKILL.md with
|
||||
# no `metadata.version`, and rejects a value that is not three-part semver.
|
||||
# skill-author's Step 4 says to run this audit and "resolve every FAIL", so any
|
||||
# tier below FAIL lets that step report done on a skill the commit gate then
|
||||
# refuses — the same audit-disagrees-with-the-gate failure the MAX_LINES note
|
||||
# below warns about, arrived at from the other direction. Verified before this
|
||||
# check existed: a SKILL.md with no `metadata:` block at all reported "All
|
||||
# checks passed".
|
||||
#
|
||||
# The rule is DUPLICATED from that hook for the same cache-isolation reason as
|
||||
# every other constant here — an installed plugin's scripts cannot read the
|
||||
# repo-root config. Keep the two in step: this check must accept exactly what
|
||||
# the hook accepts.
|
||||
SEMVER_RE = re.compile(r'^\d+\.\d+\.\d+$')
|
||||
|
||||
try:
|
||||
fm_data = yaml.safe_load(fm)
|
||||
except Exception:
|
||||
# Unreachable in practice: description_value() above parses the same text
|
||||
# and hard-exits on a YAML error, so anything arriving here already parsed.
|
||||
fm_data = None
|
||||
metadata_block = fm_data.get('metadata') if isinstance(fm_data, dict) else None
|
||||
|
||||
if not isinstance(metadata_block, dict) or metadata_block.get('version') is None:
|
||||
fail("frontmatter has no metadata.version — ADR-0022 makes it mandatory for "
|
||||
"every skill, and the skill-size-check pre-commit hook rejects the file "
|
||||
"without it. Add `metadata:` / ` version: \"1.0.0\"` (new skills start "
|
||||
"at \"0.1.0\")")
|
||||
else:
|
||||
version_value = metadata_block['version']
|
||||
# NOT str()-coerced blind: `version: 1.0` is a YAML float, and its "1.0"
|
||||
# spelling is exactly the two-part value the hook rejects — coercing and
|
||||
# then matching keeps this check and the hook agreeing on that case.
|
||||
version_text = version_value if isinstance(version_value, str) else str(version_value)
|
||||
version_text = version_text.strip()
|
||||
if SEMVER_RE.match(version_text):
|
||||
ok(f"metadata.version present: '{version_text}' (ADR-0022)")
|
||||
else:
|
||||
fail(f"metadata.version '{version_text}' is not three-part semver — the "
|
||||
f"skill-size-check pre-commit hook rejects it. Use MAJOR.MINOR.PATCH, "
|
||||
f"e.g. \"1.0.0\"")
|
||||
|
||||
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
|
||||
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
|
||||
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
|
||||
# outside the plugin directory once the plugin is cache-installed, so there is
|
||||
# no single source to share. Keep the two in sync by hand: if they drift, this
|
||||
# audit will report a skill ready to ship that the commit hook then rejects.
|
||||
MAX_LINES = 500
|
||||
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
|
||||
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
|
||||
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. lib-checks-agent.sh, beside this file, holds a third copy of the
|
||||
# two description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
else:
|
||||
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
|
||||
|
||||
# str.split() with no argument splits on runs of whitespace, matching the
|
||||
# `wc -w` the hook uses, and counts the whole file including frontmatter.
|
||||
word_count = len(content.split())
|
||||
if word_count <= MAX_WORDS:
|
||||
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
elif by_hand:
|
||||
ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character "
|
||||
f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- Reference pointers must exist -----------------------------------------
|
||||
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
|
||||
# on disk is a hard break, and until this check existed nothing in the
|
||||
# gate/audit/vale stack noticed it — all three exited 0.
|
||||
missing_refs = missing_reference_pointers(body, skill_dir)
|
||||
for ref in missing_refs:
|
||||
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
|
||||
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
|
||||
if not missing_refs:
|
||||
ok("all referenced references/ files exist")
|
||||
|
||||
# --- Gotchas discipline -----------------------------------------------------
|
||||
# SUGGESTION on both counts: the measurement is deterministic, but whether a
|
||||
# given gotcha earns its place in the body is the auditor's judgment.
|
||||
gotchas = gotcha_stats(body)
|
||||
if gotchas is not None:
|
||||
gotcha_entries, gotcha_words = gotchas
|
||||
if gotcha_entries > GOTCHA_MAX_ENTRIES:
|
||||
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
|
||||
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
|
||||
f"missing references/ file or a design problem written up as a warning")
|
||||
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
|
||||
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
|
||||
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
|
||||
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
|
||||
f"parts to references/ and keep the section for live traps")
|
||||
|
||||
# --- ADR-0020: boundary clause present -------------------------------------
|
||||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||||
# this particular skill warrants a boundary clause is judgment. Both accepted
|
||||
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
|
||||
#
|
||||
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not
|
||||
# parse" are different findings, and reporting the first for the second sends
|
||||
# the author hunting for a problem that is not there (issue #110).
|
||||
#
|
||||
# Skipped entirely for a hand-invoked skill — the contract gives it one plain
|
||||
# sentence with no boundary clause, so the finding would be wrong and its remedy
|
||||
# names a router that cannot see the skill (issue #108).
|
||||
if desc and by_hand:
|
||||
ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger "
|
||||
"rules do not apply; audited as one plain human-facing sentence")
|
||||
elif desc:
|
||||
status = boundary_clause_status(desc)
|
||||
if status == 'present':
|
||||
ok("description has a boundary clause")
|
||||
elif status == 'absent':
|
||||
suggest("description has no boundary clause — add the prose form (\"Do not use "
|
||||
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||||
"so the router knows where NOT to send this skill")
|
||||
else:
|
||||
suggest("description has an arrow boundary clause (\"Not X -> y\") from which no "
|
||||
"target could be read, so the dangling-target check did not run on it — "
|
||||
"the clause is PRESENT and unparsed, not missing. Most often the target is "
|
||||
"a single word, which is deliberately not matchable bare because "
|
||||
"`research`, `triage` and `forge` are all ordinary English: write it as "
|
||||
"`name` or /name")
|
||||
# One arrow, one target. A second name after the same arrow is resolved by
|
||||
# nothing and reported by nothing, so the clause claims coverage it does not
|
||||
# have and this script printed "1 of 1 boundary target(s) resolve" on a
|
||||
# clause naming two (issue #107).
|
||||
for first, second in multi_target_arrow_clauses(desc):
|
||||
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
||||
f"'{second}') and only the first is resolved — the second is checked by "
|
||||
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
||||
f"Not Y -> {second}.\"")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# The resolution universe comes from the SKILL's own location: the authoring
|
||||
# root above it (every sibling plugin in the monorepo), its own apm package, and
|
||||
# the packages that package declares in apm.yml dependencies.apm. It is never
|
||||
# derived from this script's own path, and — when an authoring root exists — it
|
||||
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
|
||||
# run `apm install` return the same verdict. See the shared resolver's header.
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
if routing_targets and not known:
|
||||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||||
f"determined for this path (no authoring root above it, no apm package "
|
||||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||||
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
|
||||
elif routing_targets:
|
||||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||||
# route notation or its own sentence corroborates it by naming another
|
||||
# target that resolves. See the shared resolver's CORROBORATION note.
|
||||
unresolved, soft = unresolved_targets(desc, known)
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||||
f"target sends the router nowhere")
|
||||
for target in soft:
|
||||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||||
f"file format or an English compound. If it IS a route, write it as "
|
||||
f"`/{target}` or `-> {target}` and it will be checked properly")
|
||||
if not unresolved:
|
||||
# Counts the targets that ACTUALLY resolve, not every target found:
|
||||
# a confirm-only target (one used attributively — see the resolver's
|
||||
# ATTRIBUTIVE USE note) is exempt from the failure above, so
|
||||
# reporting it as resolved would be a false claim.
|
||||
resolved = [t for t in routing_targets if normalize_target(t) in known]
|
||||
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(resolved) if resolved else '(none)'}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
# A here-doc body is DATA, not command position. Every script in this corpus
|
||||
# carries a `usage() { cat <<EOF ... EOF; }`, and prose wrapped inside one puts
|
||||
# ordinary English at the start of a line — "read is reported as an INFO ..."
|
||||
# in this skill's own validate-provenance.sh, which made the skill audit
|
||||
# hard-FAIL on its own script. Reflowing that one sentence would have cleared the finding
|
||||
# and left the cause: every future usage text is one wrap away from the same
|
||||
# false positive, and the remedy an author reaches for is contorting working
|
||||
# source, which the note above records has already happened twice.
|
||||
#
|
||||
# Detection is deliberately conservative in the direction that matters. A
|
||||
# here-doc body is skipped only when its terminator is actually found further
|
||||
# down the file; an opener with no terminator — the shape a stray `<<` inside a
|
||||
# string would produce — is ignored rather than allowed to swallow the tail,
|
||||
# because swallowing the tail is a false NEGATIVE and this check exists to fail
|
||||
# closed. `<<<` here-strings open nothing and are excluded by the lookbehind.
|
||||
HEREDOC_START_RE = re.compile(r'(?<!<)<<-?\s*(["\']?)([A-Za-z_][A-Za-z0-9_]*)\1')
|
||||
|
||||
|
||||
def heredoc_delimiter(line):
|
||||
"""The here-doc terminator this line opens, or None."""
|
||||
m = HEREDOC_START_RE.search(line)
|
||||
return m.group(2) if m else None
|
||||
|
||||
|
||||
def heredoc_body_indices(lines):
|
||||
"""Line indices that are here-doc BODY (plus its terminator), not code."""
|
||||
skip = set()
|
||||
i, n = 0, len(lines)
|
||||
while i < n:
|
||||
stripped = lines[i].strip()
|
||||
delim = None if stripped.startswith('#') else heredoc_delimiter(lines[i])
|
||||
if delim:
|
||||
# `<<-` allows an indented terminator, so compare stripped.
|
||||
for j in range(i + 1, n):
|
||||
if lines[j].strip() == delim:
|
||||
skip.update(range(i + 1, j + 1))
|
||||
i = j
|
||||
break
|
||||
i += 1
|
||||
return skip
|
||||
|
||||
|
||||
# The here-doc exemption applies to the `read` heuristic ONLY, and the
|
||||
# asymmetry is the point. `read` is an ordinary English verb, so any prose a
|
||||
# script prints is one line-wrap away from opening with it. `input(` is not a
|
||||
# word — a line beginning `input(` inside a here-doc is an embedded Python
|
||||
# program pausing for a keypress, which is exactly what this check is for, and
|
||||
# these scripts embed Python in a here-doc as a matter of course. Exempting the
|
||||
# whole body would have disarmed the check across every script in the corpus.
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
lines = source.splitlines()
|
||||
in_heredoc = heredoc_body_indices(lines)
|
||||
for idx, line in enumerate(lines):
|
||||
stripped = line.strip()
|
||||
if idx in in_heredoc:
|
||||
if re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
continue
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
scripts = [f for f in os.listdir(scripts_dir)
|
||||
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
|
||||
for fname in scripts:
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
try:
|
||||
sc = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
# The executable-bit check below still runs — one unreadable byte
|
||||
# must not silently drop a second, independent check.
|
||||
sc = None
|
||||
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
|
||||
f"interactive prompts")
|
||||
interactive = interactive_reads(sc) if sc is not None else []
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
elif sc is not None:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
if os.access(fpath, os.X_OK):
|
||||
ok(f"scripts/{fname}: is executable")
|
||||
else:
|
||||
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
if suggestions:
|
||||
# Feeds SKILL.md Step 4's `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
KYBERFORGE_SKILL_BODY
|
||||
KYBERFORGE_SKILL_BODY_PY="${KYBERFORGE_SKILL_BODY_PY%$'\n'}"
|
||||
134
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-contributing-files.sh
Executable file
134
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-contributing-files.sh
Executable file
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-contributing-files.sh — SOURCED, never executed.
|
||||
#
|
||||
# The shared Contributing-files parser, as ONE copy for this skill. Both of
|
||||
# validate-provenance.sh's modes compose it into the Python program they run.
|
||||
# Before the merge it was embedded twice. The two copies drifted once
|
||||
# (484357a) into different spellings of the bullet loop — behaviourally
|
||||
# identical, but unchecked while a docstring asserted they matched — and were
|
||||
# re-unified at 598a7c3, so they were byte-identical when ADR-0025 merged them.
|
||||
# tests/test-adr0020-contract.sh pins this file as the sole authority so that
|
||||
# a second copy cannot reappear.
|
||||
#
|
||||
# Held in a shell variable filled from a QUOTED here-doc for the same reason as
|
||||
# lib-boundary-resolver.sh's block: nothing inside is expanded, so the text
|
||||
# between the two markers below stays byte-identical to the copies the contract
|
||||
# test reads, and the markers stay on lines of their own, at column 0, exactly
|
||||
# once each, so the same sed range extracts the same span.
|
||||
#
|
||||
# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`.
|
||||
# validate-provenance.sh sources this file before either mode's python3
|
||||
# preflight, so a `cat` here made coreutils a hard dependency ahead of python3:
|
||||
# on a PATH with neither, the script exited 127 naming `cat` instead of reaching
|
||||
# the preflight that names python3. `read -r -d ''` reads to a NUL that never
|
||||
# arrives and so returns non-zero at EOF — hence the `|| true` — and it keeps the
|
||||
# last line's newline, which the joining newline in the caller would otherwise
|
||||
# double — hence the single strip after the delimiter. It removes exactly ONE
|
||||
# newline, never a run: blank lines at the end of a chunk are program text, and
|
||||
# stripping every trailing newline deleted them. The here-doc itself is
|
||||
# unchanged.
|
||||
#
|
||||
# Consumed by: validate-provenance.sh (both modes), via
|
||||
# $KYBERFORGE_CONTRIBUTING_FILES_PY.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_CONTRIBUTING_FILES_PY <<'KYBERFORGE_CONTRIBUTING_FILES' || true
|
||||
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
|
||||
# ONE parser, and since ADR-0025 exactly one copy of it: this file, sourced by
|
||||
# validate-provenance.sh for both the skill and the agent flow. It used to be
|
||||
# embedded verbatim in skill-audit's and agent-audit's separate
|
||||
# validate-provenance.sh copies, because a cache-installed plugin's scripts
|
||||
# cannot read files outside their own plugin directory and no single file was
|
||||
# reachable by both skills. Merging those skills removed that constraint: two
|
||||
# files in ONE skill directory can source a third. The two copies had drifted
|
||||
# once before (cosmetically, and re-unified before the merge) while a docstring
|
||||
# claimed they had not, which is why tests/test-adr0020-contract.sh now pins
|
||||
# this file as the SOLE authority — that it exists, that validate-provenance.sh
|
||||
# sources it, and that nothing anywhere has re-inlined the parser. Do not paste
|
||||
# this block into a caller.
|
||||
#
|
||||
# Before the contract test pinned it, the agent-side copy's docstring merely
|
||||
# ASSERTED the two copies were "behaviourally identical" and nothing checked
|
||||
# it — which is how the two diverged spellings of the bullet loop went
|
||||
# unnoticed at 484357a.
|
||||
#
|
||||
# Requires: re (imported by the host script).
|
||||
|
||||
|
||||
def parse_contributing_files(content, slug):
|
||||
"""Find the Contributing files for a given slug H2 in content.
|
||||
|
||||
Both authored forms are accepted, because both are in use across the
|
||||
corpus and only recognising the first silently skipped the contributing-
|
||||
file checks on every sources.md written the other way:
|
||||
|
||||
- **Contributing files:** SKILL.md, references/a.md
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (what this source contributed)
|
||||
- references/a.md (what this source contributed)
|
||||
|
||||
Returns a list of paths with any trailing parenthetical note stripped.
|
||||
Note the bullet form's notes may themselves contain commas, so the list
|
||||
is built per bullet rather than by splitting the joined value.
|
||||
|
||||
The three return values are NOT interchangeable, and callers depend on
|
||||
the distinction:
|
||||
|
||||
[path, ...] the entry names contributing files
|
||||
[] the entry EXPLICITLY records "(none)"
|
||||
None the entry says nothing this parser can read
|
||||
|
||||
Only an explicit "(none)" yields []. A "Contributing files:" heading
|
||||
followed by a numbered list, by `*` bullets, or by prose parses nothing
|
||||
and returns None, never [] — a caller reads [] as a deliberate "no
|
||||
contributing files" record and SKIPS its check on that basis, so a parse
|
||||
failure returning [] would silently disable the check instead of leaving
|
||||
the unreadable entry exposed to it.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
|
||||
re.MULTILINE | re.DOTALL
|
||||
)
|
||||
m = pattern.search(content)
|
||||
if not m:
|
||||
return None
|
||||
block = m.group(1)
|
||||
|
||||
def strip_note(entry):
|
||||
# "references/a.md (why)" -> "references/a.md"
|
||||
return re.sub(r'\s*\(.*$', '', entry).strip()
|
||||
|
||||
# Inline form: value on the same line, comma-separated, no notes.
|
||||
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
|
||||
if cf_m:
|
||||
value = cf_m.group(1).strip()
|
||||
if value.startswith("(none"):
|
||||
return []
|
||||
return [p for p in (strip_note(x) for x in value.split(","))
|
||||
if p] or None
|
||||
|
||||
# Bullet form: heading on its own line, one file per following bullet.
|
||||
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
|
||||
if not cf_m:
|
||||
return None
|
||||
files = []
|
||||
for line in block[cf_m.end():].splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
if files:
|
||||
break
|
||||
continue
|
||||
if not line.startswith("- "):
|
||||
break
|
||||
entry = line[2:].strip()
|
||||
if entry.startswith("(none"):
|
||||
return []
|
||||
entry = strip_note(entry)
|
||||
if entry:
|
||||
files.append(entry)
|
||||
return files or None
|
||||
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
|
||||
KYBERFORGE_CONTRIBUTING_FILES
|
||||
KYBERFORGE_CONTRIBUTING_FILES_PY="${KYBERFORGE_CONTRIBUTING_FILES_PY%$'\n'}"
|
||||
@@ -1,7 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
# lib-provenance-agent.sh — SOURCED, never executed.
|
||||
#
|
||||
# agent-audit's provenance suite: its validate-provenance.sh, minus the shared
|
||||
# Contributing-files parser (lib-contributing-files.sh holds the one copy) and
|
||||
# minus the --help dispatch that validate-provenance.sh now owns. The bash
|
||||
# argument handling, the preconditions and every exit code are lifted verbatim,
|
||||
# except the extension check, which the dispatcher made unreachable (see
|
||||
# kyberforge_prov_agent_run).
|
||||
#
|
||||
# The two provenance modes have DIFFERENT exit contracts and they are NOT
|
||||
# unified. Agent mode prints NOTHING on a clean run, and exits 0 silently when
|
||||
# the scope walk-up finds no type:-bearing apm.yml above the agent file — that
|
||||
# is a verdict about a real file ("this agent is user or project scope, so
|
||||
# plugin-scope provenance does not apply"), not a rejected input, and
|
||||
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it. Skill mode
|
||||
# (lib-provenance-skill.sh) has no such verdict and instead treats exit 0 with
|
||||
# output as INFO-only findings. Neither contract may be spelled with the
|
||||
# other's codes.
|
||||
#
|
||||
# Agent mode also has no check 9, so it takes no --base-ref flag: a --base-ref
|
||||
# passed alongside an agent target is an extra argument and is rejected with
|
||||
# exit 2, exactly as before the merge.
|
||||
#
|
||||
# Consumed by: validate-provenance.sh, agent mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
usage() {
|
||||
kyberforge_prov_agent_usage() {
|
||||
cat <<EOF
|
||||
Usage: validate-provenance.sh <agent-file>
|
||||
|
||||
@@ -52,91 +77,98 @@ Checks performed:
|
||||
4 Contributing files back-reference the parent slug in their source_keys
|
||||
5 Research doc field present and not placeholder
|
||||
|
||||
This script has no counterpart to skill-audit's checks 6, 7 and 8 (Research
|
||||
Agent mode has no counterpart to skill mode's checks 6, 7 and 8 (Research
|
||||
doc field / upstream forward / upstream reverse are numbered 6, 7, 8 there and
|
||||
5 here): an agent at plugin scope is a single file with a plugin-root
|
||||
sources.md, so there is no references/ tree to walk and no upstream research
|
||||
source index to cross-check. parse_status() and the sources.md-basename gate
|
||||
that those checks need exist only in the skill-audit copy.
|
||||
that those checks need exist only in lib-provenance-skill.sh.
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Usage and environment problems exit 2, findings exit 1. See the usage text
|
||||
# above for why the two must not share a code, and for why "not plugin scope"
|
||||
# is neither of them. This is a deliberate divergence from validate.sh, which
|
||||
# has no 2 tier for content: validate.sh always prints PASS lines, so a usage
|
||||
# error there is visibly not a findings report. This script prints NOTHING on a
|
||||
# clean run, so exit 1 plus empty stdout was the only signal a caller got
|
||||
# either way.
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: agent-file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Extra positional arguments were silently dropped, so a typo'd flag or a second
|
||||
# path looked like it had been honoured.
|
||||
if [[ $# -gt 1 ]]; then
|
||||
echo "Error: expected exactly one argument, got $#: $*" >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# python3 is a HARD dependency. Without this preflight a missing interpreter
|
||||
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
|
||||
# caller maps to anything, from a message that names this script's line number
|
||||
# rather than the missing dependency.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A path that does not exist, or exists but is not a regular file, used to reach
|
||||
# the Python body, get os.path.dirname()'d into some ancestor directory and then
|
||||
# either report a silent exit 0 (no package above it) or — worse — audit a
|
||||
# DIFFERENT agent's package while naming the typo'd path. A typo'd target was
|
||||
# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a
|
||||
# nonexistent path for exactly this reason.
|
||||
#
|
||||
# This is decided from the argument alone, before any walk-up runs, so it cannot
|
||||
# collide with the not-plugin-scope exit 0: that verdict is only ever reached by
|
||||
# a file that got past here.
|
||||
if [[ ! -e "$1" ]]; then
|
||||
echo "Error: no such file: $1" >&2
|
||||
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
|
||||
echo " Fix: pass the path of the agent file to validate." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ ! -f "$1" ]]; then
|
||||
echo "Error: not a regular file: $1" >&2
|
||||
echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2
|
||||
echo " Fix: pass the agent file itself — .apm/agents/<name>.agent.md — not its parent directory." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# The extension check used to live inside the Python body. It stays exit 2 and
|
||||
# keeps its wording; it moves up here so that every "this argument is not
|
||||
# auditable" verdict is reached in one place, before the interpreter starts and
|
||||
# before the scope walk-up can turn a bad argument into a silent exit 0.
|
||||
case "$1" in
|
||||
*.agent.md | *.md) ;;
|
||||
*)
|
||||
echo "Error: unrecognized extension '$(basename "$1")' — expected .md or .agent.md" >&2
|
||||
kyberforge_prov_agent_run() {
|
||||
# Usage and environment problems exit 2, findings exit 1. See the usage text
|
||||
# above for why the two must not share a code, and for why "not plugin scope"
|
||||
# is neither of them. This is a deliberate divergence from validate.sh, which
|
||||
# has no 2 tier for content: validate.sh always prints PASS lines, so a usage
|
||||
# error there is visibly not a findings report. This script prints NOTHING on a
|
||||
# clean run, so exit 1 plus empty stdout was the only signal a caller got
|
||||
# either way.
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: agent-file is required." >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_agent_usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
|
||||
python3 -u - "$1" <<'PYTHON'
|
||||
# Extra positional arguments were silently dropped, so a typo'd flag or a second
|
||||
# path looked like it had been honoured.
|
||||
if [[ $# -gt 1 ]]; then
|
||||
echo "Error: expected exactly one argument, got $#: $*" >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_agent_usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# python3 is a HARD dependency. Without this preflight a missing interpreter
|
||||
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
|
||||
# caller maps to anything, from a message that names this script's line number
|
||||
# rather than the missing dependency.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A path that does not exist, or exists but is not a regular file, used to reach
|
||||
# the Python body, get os.path.dirname()'d into some ancestor directory and then
|
||||
# either report a silent exit 0 (no package above it) or — worse — audit a
|
||||
# DIFFERENT agent's package while naming the typo'd path. A typo'd target was
|
||||
# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a
|
||||
# nonexistent path for exactly this reason.
|
||||
#
|
||||
# This is decided from the argument alone, before any walk-up runs, so it cannot
|
||||
# collide with the not-plugin-scope exit 0: that verdict is only ever reached by
|
||||
# a file that got past here.
|
||||
if [[ ! -e "$1" ]]; then
|
||||
echo "Error: no such file: $1" >&2
|
||||
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
|
||||
echo " Fix: pass the path of the agent file to validate." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ ! -f "$1" ]]; then
|
||||
echo "Error: not a regular file: $1" >&2
|
||||
echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2
|
||||
echo " Fix: pass the agent file itself — .apm/agents/<name>.agent.md — not its parent directory." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# No extension check here. The pre-merge script carried one ("unrecognized
|
||||
# extension — expected .md or .agent.md"), but validate-provenance.sh only
|
||||
# dispatches a *.agent.md, or a *.md directly under an agents/ directory, to
|
||||
# this function, and the argument-count check above guarantees $1 IS that
|
||||
# target — so the check could never fire. The "no such file" and "not a
|
||||
# regular file" checks stay: a nonexistent x.agent.md and a FIFO named
|
||||
# x.agent.md both pass the dispatcher and both still reach them.
|
||||
|
||||
# The Python program, reassembled in the order the parser block sat in before
|
||||
# the merge: preamble, shared parser, body.
|
||||
local prog="$KYBERFORGE_PROV_AGENT_PREAMBLE_PY
|
||||
$KYBERFORGE_CONTRIBUTING_FILES_PY
|
||||
$KYBERFORGE_PROV_AGENT_BODY_PY"
|
||||
|
||||
local rc=0
|
||||
python3 -u - "$1" <<< "$prog" || rc=$?
|
||||
# The findings code travels in KYBERFORGE_PROV_RC and this function returns 0,
|
||||
# so the caller can invoke it UNTESTED. See lib-provenance-skill.sh for why:
|
||||
# testing a function's status disables errexit for its whole body.
|
||||
KYBERFORGE_PROV_RC="$rc"
|
||||
return 0
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_PREAMBLE_PY <<'KYBERFORGE_PROV_AGENT_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
@@ -291,99 +323,10 @@ def parse_source_keys(fm):
|
||||
def parse_h2_slugs(content):
|
||||
return re.findall(r'^## (.+)$', content, re.MULTILINE)
|
||||
|
||||
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
|
||||
# ONE parser, embedded VERBATIM in two scripts:
|
||||
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
|
||||
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
|
||||
# The block between these markers must stay byte-identical in both. It is
|
||||
# copied rather than imported because a cache-installed plugin's scripts cannot
|
||||
# read files outside their own plugin directory, so there is no single file both
|
||||
# can share — the same constraint that forces the ADR-0020 boundary resolver to
|
||||
# be duplicated across three scripts. Edit one copy, then paste it over the
|
||||
# other.
|
||||
#
|
||||
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
|
||||
# it did, the agent-audit copy's docstring merely ASSERTED the two were
|
||||
# "behaviourally identical" and nothing checked it — which is how the two
|
||||
# already-diverged spellings of the bullet loop went unnoticed.
|
||||
#
|
||||
# Requires: re (imported by the host script).
|
||||
KYBERFORGE_PROV_AGENT_PREAMBLE
|
||||
KYBERFORGE_PROV_AGENT_PREAMBLE_PY="${KYBERFORGE_PROV_AGENT_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
|
||||
def parse_contributing_files(content, slug):
|
||||
"""Find the Contributing files for a given slug H2 in content.
|
||||
|
||||
Both authored forms are accepted, because both are in use across the
|
||||
corpus and only recognising the first silently skipped the contributing-
|
||||
file checks on every sources.md written the other way:
|
||||
|
||||
- **Contributing files:** SKILL.md, references/a.md
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (what this source contributed)
|
||||
- references/a.md (what this source contributed)
|
||||
|
||||
Returns a list of paths with any trailing parenthetical note stripped.
|
||||
Note the bullet form's notes may themselves contain commas, so the list
|
||||
is built per bullet rather than by splitting the joined value.
|
||||
|
||||
The three return values are NOT interchangeable, and callers depend on
|
||||
the distinction:
|
||||
|
||||
[path, ...] the entry names contributing files
|
||||
[] the entry EXPLICITLY records "(none)"
|
||||
None the entry says nothing this parser can read
|
||||
|
||||
Only an explicit "(none)" yields []. A "Contributing files:" heading
|
||||
followed by a numbered list, by `*` bullets, or by prose parses nothing
|
||||
and returns None, never [] — a caller reads [] as a deliberate "no
|
||||
contributing files" record and SKIPS its check on that basis, so a parse
|
||||
failure returning [] would silently disable the check instead of leaving
|
||||
the unreadable entry exposed to it.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
|
||||
re.MULTILINE | re.DOTALL
|
||||
)
|
||||
m = pattern.search(content)
|
||||
if not m:
|
||||
return None
|
||||
block = m.group(1)
|
||||
|
||||
def strip_note(entry):
|
||||
# "references/a.md (why)" -> "references/a.md"
|
||||
return re.sub(r'\s*\(.*$', '', entry).strip()
|
||||
|
||||
# Inline form: value on the same line, comma-separated, no notes.
|
||||
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
|
||||
if cf_m:
|
||||
value = cf_m.group(1).strip()
|
||||
if value.startswith("(none"):
|
||||
return []
|
||||
return [p for p in (strip_note(x) for x in value.split(","))
|
||||
if p] or None
|
||||
|
||||
# Bullet form: heading on its own line, one file per following bullet.
|
||||
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
|
||||
if not cf_m:
|
||||
return None
|
||||
files = []
|
||||
for line in block[cf_m.end():].splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
if files:
|
||||
break
|
||||
continue
|
||||
if not line.startswith("- "):
|
||||
break
|
||||
entry = line[2:].strip()
|
||||
if entry.startswith("(none"):
|
||||
return []
|
||||
entry = strip_note(entry)
|
||||
if entry:
|
||||
files.append(entry)
|
||||
return files or None
|
||||
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_BODY_PY <<'KYBERFORGE_PROV_AGENT_BODY' || true
|
||||
|
||||
def parse_research_docs(content, slug):
|
||||
"""Every Research doc value under a given slug H2, in document order.
|
||||
@@ -629,4 +572,5 @@ for slug in unique_slugs:
|
||||
|
||||
print_findings()
|
||||
sys.exit(1 if has_fail else 0)
|
||||
PYTHON
|
||||
KYBERFORGE_PROV_AGENT_BODY
|
||||
KYBERFORGE_PROV_AGENT_BODY_PY="${KYBERFORGE_PROV_AGENT_BODY_PY%$'\n'}"
|
||||
@@ -1,7 +1,30 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
# lib-provenance-skill.sh — SOURCED, never executed.
|
||||
#
|
||||
# skill-audit's provenance suite: its validate-provenance.sh, minus the shared
|
||||
# Contributing-files parser (lib-contributing-files.sh holds the one copy) and
|
||||
# minus the --help dispatch that validate-provenance.sh now owns. The bash
|
||||
# argument handling, the preconditions and every exit code are lifted verbatim.
|
||||
#
|
||||
# The two provenance modes have DIFFERENT exit contracts and they are NOT
|
||||
# unified. Skill mode exits 0 with output whenever the only findings are INFO
|
||||
# — a check that could not run, announced rather than skipped silently — so a
|
||||
# caller must read exit 0 plus output as INFO-only findings, not as noise.
|
||||
# Agent mode (lib-provenance-agent.sh) prints nothing at all on a clean run and
|
||||
# additionally exits 0 SILENTLY when the scope walk-up finds no plugin package.
|
||||
# Skill mode has no such verdict: it has already hard-failed (exit 2) on a
|
||||
# directory that is not a skill before the interpreter starts.
|
||||
#
|
||||
# Skill mode also owns the --base-ref=<ref> flag (check 9) and the
|
||||
# VALIDATE_PROVENANCE_BASE_REF environment variable. Agent mode has no check 9
|
||||
# and takes no flags, so a --base-ref passed to an agent target is still an
|
||||
# extra argument and is still rejected, exactly as before the merge.
|
||||
#
|
||||
# Consumed by: validate-provenance.sh, skill mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
usage() {
|
||||
kyberforge_prov_skill_usage() {
|
||||
cat <<EOF
|
||||
Usage: validate-provenance.sh <skill-dir> [--base-ref=<ref>]
|
||||
|
||||
@@ -64,102 +87,121 @@ Checks performed:
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
kyberforge_prov_skill_run() {
|
||||
local arg
|
||||
# --base-ref=<ref> is the only recognised flag, for check 9's base-ref
|
||||
# override. It is pulled out before the positional-count checks below so it
|
||||
# never counts against them — a caller passing it alongside skill-dir sees
|
||||
# the same argument-count behaviour as one who does not pass it at all, and a
|
||||
# genuinely extra positional argument is still rejected.
|
||||
#
|
||||
# BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to
|
||||
# the empty string, and deliberately NOT declared `local`: `local X` with no
|
||||
# value still leaves X unset, but declaring it at all would scope it away from
|
||||
# a future caller that wants to inspect it. `--base-ref=` (given, but empty)
|
||||
# and "no flag at all" are different instructions — the first says "use the
|
||||
# default resolution, ignoring the environment", the second says "fall back to
|
||||
# the environment" — and an empty-string initialiser collapsed them:
|
||||
# `${BASE_REF_OVERRIDE:-$ENV}` treats an empty flag value as absent, so the
|
||||
# environment variable won and the usage text's "the flag wins if both are
|
||||
# given" was false for exactly that spelling.
|
||||
declare -a POSITIONAL_ARGS=()
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--base-ref=*)
|
||||
BASE_REF_OVERRIDE="${arg#--base-ref=}"
|
||||
;;
|
||||
*)
|
||||
POSITIONAL_ARGS+=("$arg")
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# --base-ref=<ref> is the only recognised flag, for check 9's base-ref
|
||||
# override. It is pulled out before the positional-count checks below so it
|
||||
# never counts against them — a caller passing it alongside skill-dir sees
|
||||
# the same argument-count behaviour as one who does not pass it at all, and a
|
||||
# genuinely extra positional argument is still rejected.
|
||||
#
|
||||
# BASE_REF_OVERRIDE is deliberately left UNSET here rather than initialised to
|
||||
# the empty string. `--base-ref=` (given, but empty) and "no flag at all" are
|
||||
# different instructions — the first says "use the default resolution, ignoring
|
||||
# the environment", the second says "fall back to the environment" — and an
|
||||
# empty-string initialiser collapsed them: `${BASE_REF_OVERRIDE:-$ENV}` treats
|
||||
# an empty flag value as absent, so the environment variable won and the usage
|
||||
# text's "the flag wins if both are given" was false for exactly that spelling.
|
||||
declare -a POSITIONAL_ARGS=()
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--base-ref=*)
|
||||
BASE_REF_OVERRIDE="${arg#--base-ref=}"
|
||||
;;
|
||||
*)
|
||||
POSITIONAL_ARGS+=("$arg")
|
||||
;;
|
||||
esac
|
||||
done
|
||||
# Usage and environment problems exit 2, findings exit 1. See the usage text
|
||||
# above for why the two must not share a code. This is a deliberate divergence
|
||||
# from validate.sh, which has no 2 tier: validate.sh always prints PASS lines,
|
||||
# so a usage error there is visibly not a findings report. This script prints
|
||||
# NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a
|
||||
# caller got either way.
|
||||
if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then
|
||||
echo "Error: skill-dir is required." >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_skill_usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Usage and environment problems exit 2, findings exit 1. See the usage text
|
||||
# above for why the two must not share a code. This is a deliberate divergence
|
||||
# from validate.sh, which has no 2 tier: validate.sh always prints PASS lines,
|
||||
# so a usage error there is visibly not a findings report. This script prints
|
||||
# NOTHING on a clean run, so exit 1 plus empty stdout was the only signal a
|
||||
# caller got either way.
|
||||
if [[ ${#POSITIONAL_ARGS[@]} -lt 1 ]]; then
|
||||
echo "Error: skill-dir is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
# Extra positional arguments were silently dropped, so a typo'd flag or a second
|
||||
# path looked like it had been honoured.
|
||||
if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then
|
||||
echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_skill_usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Extra positional arguments were silently dropped, so a typo'd flag or a second
|
||||
# path looked like it had been honoured.
|
||||
if [[ ${#POSITIONAL_ARGS[@]} -gt 1 ]]; then
|
||||
echo "Error: expected exactly one argument, got ${#POSITIONAL_ARGS[@]}: ${POSITIONAL_ARGS[*]}" >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
local SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}"
|
||||
|
||||
SKILL_DIR_ARG="${POSITIONAL_ARGS[0]}"
|
||||
# The flag wins over the environment variable whenever the flag was GIVEN —
|
||||
# `+x` tests for presence, not for a non-empty value, which is the distinction
|
||||
# `:-` could not make. An empty result either way tells the Python body to fall
|
||||
# back to `git merge-base HEAD origin/main`.
|
||||
local BASE_REF
|
||||
if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then
|
||||
BASE_REF="$BASE_REF_OVERRIDE"
|
||||
else
|
||||
BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}"
|
||||
fi
|
||||
|
||||
# The flag wins over the environment variable whenever the flag was GIVEN —
|
||||
# `+x` tests for presence, not for a non-empty value, which is the distinction
|
||||
# `:-` could not make. An empty result either way tells the Python body to fall
|
||||
# back to `git merge-base HEAD origin/main`.
|
||||
if [[ -n "${BASE_REF_OVERRIDE+x}" ]]; then
|
||||
BASE_REF="$BASE_REF_OVERRIDE"
|
||||
else
|
||||
BASE_REF="${VALIDATE_PROVENANCE_BASE_REF:-}"
|
||||
fi
|
||||
# python3 is a HARD dependency. Without this preflight a missing interpreter
|
||||
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
|
||||
# caller maps to anything, from a message that names this script's line number
|
||||
# rather than the missing dependency.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# python3 is a HARD dependency. Without this preflight a missing interpreter
|
||||
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
|
||||
# caller maps to anything, from a message that names this script's line number
|
||||
# rather than the missing dependency.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 2
|
||||
fi
|
||||
# A path that is not a directory, or a directory that is not a skill, used to
|
||||
# reach the Python body, find no sources.md and no source_keys, take the
|
||||
# "nothing to validate" early exit and report exit 0 with no output — which
|
||||
# references/skill-validation-scripts.md explicitly told the auditor to read as a
|
||||
# pass. A typo'd target was therefore indistinguishable from a clean skill.
|
||||
# vale-wrap.sh hard-errors on a nonexistent path for exactly this reason.
|
||||
if [[ ! -d "$SKILL_DIR_ARG" ]]; then
|
||||
echo "Error: not a directory: $SKILL_DIR_ARG" >&2
|
||||
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
|
||||
echo " Fix: pass the path of the skill directory to validate." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A path that is not a directory, or a directory that is not a skill, used to
|
||||
# reach the Python body, find no sources.md and no source_keys, take the
|
||||
# "nothing to validate" early exit and report exit 0 with no output — which
|
||||
# references/validation-scripts.md explicitly told the auditor to read as a
|
||||
# pass. A typo'd target was therefore indistinguishable from a clean skill.
|
||||
# vale-wrap.sh hard-errors on a nonexistent path for exactly this reason.
|
||||
if [[ ! -d "$SKILL_DIR_ARG" ]]; then
|
||||
echo "Error: not a directory: $SKILL_DIR_ARG" >&2
|
||||
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
|
||||
echo " Fix: pass the path of the skill directory to validate." >&2
|
||||
exit 2
|
||||
fi
|
||||
if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then
|
||||
echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2
|
||||
echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2
|
||||
echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ ! -f "$SKILL_DIR_ARG/SKILL.md" ]]; then
|
||||
echo "Error: not a skill directory (no SKILL.md): $SKILL_DIR_ARG" >&2
|
||||
echo " Why: a directory with no SKILL.md has no provenance chain to validate, and reporting that as a pass hides the wrong-target mistake." >&2
|
||||
echo " Fix: pass the skill directory itself, not its parent or its references/ subdirectory." >&2
|
||||
exit 2
|
||||
fi
|
||||
# The Python program, reassembled in the order the parser block sat in before
|
||||
# the merge: preamble, shared parser, body.
|
||||
local prog="$KYBERFORGE_PROV_SKILL_PREAMBLE_PY
|
||||
$KYBERFORGE_CONTRIBUTING_FILES_PY
|
||||
$KYBERFORGE_PROV_SKILL_BODY_PY"
|
||||
|
||||
python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<'PYTHON'
|
||||
local rc=0
|
||||
python3 -u - "$SKILL_DIR_ARG" "$BASE_REF" <<< "$prog" || rc=$?
|
||||
# The findings code travels in KYBERFORGE_PROV_RC and this function returns 0,
|
||||
# so the caller can invoke it UNTESTED. Testing a function's status (`f || RC=$?`)
|
||||
# disables errexit for its entire body, which would leave every command above
|
||||
# unguarded -- and no subshell or `set -e` inside can re-arm it once the call
|
||||
# sits in a condition context. Error paths above use `exit`, which is unaffected
|
||||
# either way; this keeps errexit armed for anything added later.
|
||||
KYBERFORGE_PROV_RC="$rc"
|
||||
return 0
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_PREAMBLE_PY <<'KYBERFORGE_PROV_SKILL_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
@@ -297,99 +339,10 @@ def parse_h2_slugs(content):
|
||||
"""Return list of H2 heading values from a markdown file."""
|
||||
return re.findall(r'^## (.+)$', content, re.MULTILINE)
|
||||
|
||||
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
|
||||
# ONE parser, embedded VERBATIM in two scripts:
|
||||
# plugins/kyberforge/.apm/skills/skill-audit/scripts/validate-provenance.sh
|
||||
# plugins/kyberforge/.apm/skills/agent-audit/scripts/validate-provenance.sh
|
||||
# The block between these markers must stay byte-identical in both. It is
|
||||
# copied rather than imported because a cache-installed plugin's scripts cannot
|
||||
# read files outside their own plugin directory, so there is no single file both
|
||||
# can share — the same constraint that forces the ADR-0020 boundary resolver to
|
||||
# be duplicated across three scripts. Edit one copy, then paste it over the
|
||||
# other.
|
||||
#
|
||||
# tests/test-adr0020-contract.sh hashes both copies and fails on drift. Before
|
||||
# it did, the agent-audit copy's docstring merely ASSERTED the two were
|
||||
# "behaviourally identical" and nothing checked it — which is how the two
|
||||
# already-diverged spellings of the bullet loop went unnoticed.
|
||||
#
|
||||
# Requires: re (imported by the host script).
|
||||
KYBERFORGE_PROV_SKILL_PREAMBLE
|
||||
KYBERFORGE_PROV_SKILL_PREAMBLE_PY="${KYBERFORGE_PROV_SKILL_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
|
||||
def parse_contributing_files(content, slug):
|
||||
"""Find the Contributing files for a given slug H2 in content.
|
||||
|
||||
Both authored forms are accepted, because both are in use across the
|
||||
corpus and only recognising the first silently skipped the contributing-
|
||||
file checks on every sources.md written the other way:
|
||||
|
||||
- **Contributing files:** SKILL.md, references/a.md
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (what this source contributed)
|
||||
- references/a.md (what this source contributed)
|
||||
|
||||
Returns a list of paths with any trailing parenthetical note stripped.
|
||||
Note the bullet form's notes may themselves contain commas, so the list
|
||||
is built per bullet rather than by splitting the joined value.
|
||||
|
||||
The three return values are NOT interchangeable, and callers depend on
|
||||
the distinction:
|
||||
|
||||
[path, ...] the entry names contributing files
|
||||
[] the entry EXPLICITLY records "(none)"
|
||||
None the entry says nothing this parser can read
|
||||
|
||||
Only an explicit "(none)" yields []. A "Contributing files:" heading
|
||||
followed by a numbered list, by `*` bullets, or by prose parses nothing
|
||||
and returns None, never [] — a caller reads [] as a deliberate "no
|
||||
contributing files" record and SKIPS its check on that basis, so a parse
|
||||
failure returning [] would silently disable the check instead of leaving
|
||||
the unreadable entry exposed to it.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
|
||||
re.MULTILINE | re.DOTALL
|
||||
)
|
||||
m = pattern.search(content)
|
||||
if not m:
|
||||
return None
|
||||
block = m.group(1)
|
||||
|
||||
def strip_note(entry):
|
||||
# "references/a.md (why)" -> "references/a.md"
|
||||
return re.sub(r'\s*\(.*$', '', entry).strip()
|
||||
|
||||
# Inline form: value on the same line, comma-separated, no notes.
|
||||
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
|
||||
if cf_m:
|
||||
value = cf_m.group(1).strip()
|
||||
if value.startswith("(none"):
|
||||
return []
|
||||
return [p for p in (strip_note(x) for x in value.split(","))
|
||||
if p] or None
|
||||
|
||||
# Bullet form: heading on its own line, one file per following bullet.
|
||||
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
|
||||
if not cf_m:
|
||||
return None
|
||||
files = []
|
||||
for line in block[cf_m.end():].splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
if files:
|
||||
break
|
||||
continue
|
||||
if not line.startswith("- "):
|
||||
break
|
||||
entry = line[2:].strip()
|
||||
if entry.startswith("(none"):
|
||||
return []
|
||||
entry = strip_note(entry)
|
||||
if entry:
|
||||
files.append(entry)
|
||||
return files or None
|
||||
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_SKILL_BODY_PY <<'KYBERFORGE_PROV_SKILL_BODY' || true
|
||||
|
||||
def parse_research_docs(content, slug):
|
||||
"""Every Research doc value under a given slug H2, in document order.
|
||||
@@ -1195,4 +1148,5 @@ else:
|
||||
|
||||
print_findings()
|
||||
sys.exit(1 if has_fail else 0)
|
||||
PYTHON
|
||||
KYBERFORGE_PROV_SKILL_BODY
|
||||
KYBERFORGE_PROV_SKILL_BODY_PY="${KYBERFORGE_PROV_SKILL_BODY_PY%$'\n'}"
|
||||
@@ -189,7 +189,16 @@ for arg in "$@"; do
|
||||
done
|
||||
|
||||
if [[ "$config_given" == false ]]; then
|
||||
vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini")
|
||||
# `scripts/../assets/vale` begins with neither `/` nor `.`, so `cd` consults
|
||||
# CDPATH for it — and when a CDPATH entry supplies the directory, `cd` PRINTS
|
||||
# the directory it chose. A bare `$(cd ... && pwd)` therefore captured TWO
|
||||
# lines, and the chosen directory could be an unrelated tree entirely: with
|
||||
# CDPATH=/tmp/decoy and /tmp/decoy/scripts present, this resolved to
|
||||
# /tmp/decoy/assets/vale and vale died on a two-line --config path. CDPATH is
|
||||
# cleared for the one command, `--` ends option parsing for a directory named
|
||||
# like a flag, and stdout is discarded so only `pwd` is captured. Same fix as
|
||||
# validate.sh and validate-provenance.sh apply to their SCRIPT_DIR.
|
||||
vale_args+=(--config "$(CDPATH='' cd -- "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" > /dev/null && pwd)/.vale.ini")
|
||||
fi
|
||||
|
||||
if [[ ${#path_args[@]} -eq 0 ]]; then
|
||||
@@ -506,7 +515,7 @@ for arg in ${path_args[@]+"${path_args[@]}"}; do
|
||||
while IFS= read -r -d '' rel; do
|
||||
mkdir -p "$dest/$(dirname "$rel")"
|
||||
cp "$arg/$rel" "$dest/$rel"
|
||||
done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0)
|
||||
done < <(CDPATH='' cd -- "$arg" && find -L . -name .git -prune -o -type f -print0)
|
||||
while IFS= read -r -d '' md; do
|
||||
flatten "$md" "$md"
|
||||
done < <(find "$dest" -type f -name '*.md' -print0)
|
||||
324
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh
Executable file
324
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh
Executable file
@@ -0,0 +1,324 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# The ONE entry point for provenance validation. It auto-detects whether the
|
||||
# target is a skill directory or an agent definition file — the same rule
|
||||
# validate.sh uses — and runs the matching suite from lib-provenance-skill.sh or
|
||||
# lib-provenance-agent.sh. The Contributing-files parser both suites need is
|
||||
# sourced once, from lib-contributing-files.sh, instead of being embedded twice.
|
||||
#
|
||||
# The two suites have DIFFERENT exit contracts, and merging the entry point does
|
||||
# not merge those:
|
||||
#
|
||||
# skill mode exits 0 with output when the only findings are INFO — a check
|
||||
# that could not run, announced rather than skipped silently. A
|
||||
# caller must read exit 0 plus output as INFO-only findings.
|
||||
# agent mode prints nothing at all on a clean run, and exits 0 SILENTLY when
|
||||
# the scope walk-up finds no plugin package above the agent file.
|
||||
# That is a verdict about a real file, not a rejected input;
|
||||
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it.
|
||||
#
|
||||
# Exit 2 means the argument is not auditable at all — missing, doubled, the
|
||||
# wrong shape, or an environment problem. It is never a finding.
|
||||
|
||||
# --- Path splitting, with bash builtins only -------------------------------
|
||||
# dirname and basename are EXTERNAL commands, and every call below happens
|
||||
# before the mode's python3 preflight. Using them put coreutils ahead of python3
|
||||
# in the dependency order: on a PATH carrying neither, this script died at exit
|
||||
# 127 naming `dirname` (and, through the sourced libraries, `cat`) instead of
|
||||
# reaching the preflight that names python3 — the exact failure
|
||||
# tests/test-adr0020-contract.sh assertion 2 exists to prevent. The pre-merge
|
||||
# validate-provenance.sh was one self-contained file that reached its preflight
|
||||
# on builtins alone; these two functions, plus the `read`-based loaders in the
|
||||
# sourced libraries, restore that property. `cd` and `pwd` are builtins and may
|
||||
# stay.
|
||||
#
|
||||
# They reproduce dirname/basename semantics for the shapes this script sees:
|
||||
# trailing slashes are stripped, a path with no slash yields "." / itself, and
|
||||
# "/" yields "/".
|
||||
_kf_dirname() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
if [[ "$_p" != */* ]]; then
|
||||
printf '%s' "."
|
||||
return 0
|
||||
fi
|
||||
_p="${_p%/*}"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ -z "$_p" ]]; then
|
||||
_p="/"
|
||||
fi
|
||||
printf '%s' "$_p"
|
||||
}
|
||||
|
||||
_kf_basename() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${_p##*/}"
|
||||
}
|
||||
|
||||
# --- The target's parent directory NAME, resolved -------------------------
|
||||
# The agent rule tests the NAME of the target's parent directory. Reading that
|
||||
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
|
||||
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
|
||||
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
|
||||
# directory was refused as matching neither shape, by an error message naming
|
||||
# that exact shape as valid. The pre-merge agent validator had no path-shape
|
||||
# gate and worked from any working directory.
|
||||
#
|
||||
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
|
||||
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
|
||||
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
|
||||
# still addressed as agents/, which is what the literal test always honoured.
|
||||
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
|
||||
# that cannot be entered — a typo'd path — falls back to the literal name, so
|
||||
# the neither-shape error still fires for it.
|
||||
_kf_parent_name() {
|
||||
local _dir _resolved
|
||||
_dir="$(_kf_dirname "$1")"
|
||||
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
|
||||
_kf_basename "$_resolved"
|
||||
else
|
||||
_kf_basename "$_dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- This script's own directory, and the libraries beside it -------------
|
||||
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
|
||||
# CDPATH exported and the relative invocation the flow references prescribe
|
||||
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
|
||||
# and could resolve through CDPATH to an unrelated directory and source a
|
||||
# same-named file from there. CDPATH is cleared for the one command, `--` ends
|
||||
# option parsing for a directory named like a flag, and stdout is discarded so
|
||||
# only `pwd` is captured.
|
||||
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
|
||||
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
|
||||
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
|
||||
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A sourced library that is missing or unreadable used to kill the script under
|
||||
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
|
||||
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
|
||||
# partial install, or a copy or symlink of this one file taken out of scripts/,
|
||||
# was therefore reported as a failing audit. Checked explicitly instead, and
|
||||
# exit 2, which the same references read as "it never ran".
|
||||
_kf_require_lib() {
|
||||
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
|
||||
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
|
||||
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
|
||||
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
# Each mode's own usage text lives in that mode's library, verbatim, so usage()
|
||||
# needs the libraries — but `--help` must not. Sourcing them unconditionally at
|
||||
# the top made a missing lib-*.sh turn `--help` into exit 2, so the one command
|
||||
# that explains how to use the script was the one command a partial install
|
||||
# could not answer. validate.sh's usage() is self-contained and always works;
|
||||
# this restores the same property without copying the per-mode text down here
|
||||
# and letting it drift from the libraries that own it. When a library is gone,
|
||||
# the shared half of the usage still prints and the mode's half says why it
|
||||
# cannot.
|
||||
#
|
||||
# The two call sites below are spelled out rather than folded into one helper
|
||||
# taking the library as a parameter: a parameterized `.` is a non-constant
|
||||
# source, which is SC1090 at warning severity — the level .pre-commit-config.yaml
|
||||
# runs shellcheck at — and the only way to silence it, a `source=/dev/null`
|
||||
# directive, is a directive that resolves to nothing, which
|
||||
# tests/test-vale-wrap.sh part C rejects outright because a non-resolving
|
||||
# directive silently disarms that file's array-seeding exemption. Two literal
|
||||
# sources with two real directives cost a few lines and keep both gates honest.
|
||||
_kf_lib_readable() {
|
||||
[[ -f "$SCRIPT_DIR/$1" && -r "$SCRIPT_DIR/$1" ]]
|
||||
}
|
||||
|
||||
_kf_usage_lib_missing() {
|
||||
echo "(This mode's usage lives in $1, which is missing or unreadable in"
|
||||
echo "$SCRIPT_DIR. Reinstall the factory-audit skill to restore it. Note that"
|
||||
echo "an audit cannot run in this state either — it would exit 2.)"
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate-provenance.sh <skill-dir> [--base-ref=<ref>]
|
||||
validate-provenance.sh <agent-file>
|
||||
|
||||
Validate that a skill's or an agent's sources provenance chain is complete and
|
||||
internally consistent. The mode is detected from the target:
|
||||
|
||||
skill mode the target is a directory (a skill directory contains SKILL.md),
|
||||
or the target IS a SKILL.md file.
|
||||
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
||||
directory is named 'agents' (.apm/agents, .claude/agents,
|
||||
.github/agents, .copilot/agents).
|
||||
|
||||
The two modes have different checks, different exit contracts and different
|
||||
flags — --base-ref belongs to skill mode's check 9 and agent mode has no
|
||||
check 9 — so each mode's own usage follows below, verbatim.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (or nothing to validate; in agent mode, also "not plugin
|
||||
scope")
|
||||
1 One or more checks failed
|
||||
2 Usage error, the target matches neither a skill directory nor an agent
|
||||
file this script can read, or a lib-*.sh beside this script is missing or
|
||||
unreadable
|
||||
|
||||
An exit code of 2 is NOT a finding. SKILL.md tells the auditor to surface a
|
||||
non-zero exit as findings, so a usage error leaving exit 1 with nothing on
|
||||
stdout was indistinguishable from a clean-but-failing run. Environment and
|
||||
argument problems exit 2; only real findings exit 1.
|
||||
|
||||
=== skill mode ===
|
||||
EOF
|
||||
if _kf_lib_readable lib-provenance-skill.sh; then
|
||||
# shellcheck source=lib-provenance-skill.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-skill.sh"
|
||||
kyberforge_prov_skill_usage
|
||||
else
|
||||
_kf_usage_lib_missing lib-provenance-skill.sh
|
||||
fi
|
||||
cat <<EOF
|
||||
|
||||
=== agent mode ===
|
||||
EOF
|
||||
if _kf_lib_readable lib-provenance-agent.sh; then
|
||||
# shellcheck source=lib-provenance-agent.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-agent.sh"
|
||||
kyberforge_prov_agent_usage
|
||||
else
|
||||
_kf_usage_lib_missing lib-provenance-agent.sh
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Sourced only once an audit is actually going to be attempted. It sat at the
|
||||
# top of the file until `--help` on a partial install exited 2 instead of
|
||||
# printing usage; see the comment above _kf_lib_readable for the whole story.
|
||||
# usage() loads the two provenance libraries on its own when it needs them, so
|
||||
# nothing here is reached by the --help path.
|
||||
_kf_require_lib lib-contributing-files.sh
|
||||
# shellcheck source=lib-contributing-files.sh
|
||||
. "$SCRIPT_DIR/lib-contributing-files.sh"
|
||||
_kf_require_lib lib-provenance-skill.sh
|
||||
# shellcheck source=lib-provenance-skill.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-skill.sh"
|
||||
_kf_require_lib lib-provenance-agent.sh
|
||||
# shellcheck source=lib-provenance-agent.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-agent.sh"
|
||||
|
||||
# --- Detect the mode -------------------------------------------------------
|
||||
# The first non-flag argument decides the mode. Only the mode is decided here:
|
||||
# the argument COUNT, the flag rules and every precondition belong to the mode's
|
||||
# own suite and are applied there, unchanged, over the original "$@". So a
|
||||
# --base-ref handed to an agent target is still an extra argument and is still
|
||||
# rejected, and a second positional is still rejected by whichever mode it
|
||||
# reaches.
|
||||
# _saw_positional is tracked separately because an EMPTY positional and NO
|
||||
# positional are different mistakes with different fixes, and `-z "$TARGET"`
|
||||
# alone cannot tell them apart: `validate-provenance.sh ""` — an unquoted shell
|
||||
# variable that expanded to nothing, the usual way this happens — was reported
|
||||
# as "only flags were given", which is false and sends the reader looking for a
|
||||
# flag they did not type instead of at the variable that came up empty.
|
||||
TARGET=""
|
||||
_saw_positional=false
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
--base-ref=*) ;;
|
||||
*) TARGET="$_arg"; _saw_positional=true; break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ "$_saw_positional" == false ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo " Why: only flags were given, so there is no target to detect a mode from." >&2
|
||||
echo " Fix: pass the skill directory, or the agent file, as a positional argument." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ -z "$TARGET" ]]; then
|
||||
echo "Error: the target argument is an empty string." >&2
|
||||
echo " Why: a positional argument was passed, but it is empty, so there is no path to detect a mode from — usually an unquoted or unset shell variable expanding to nothing at the call site, not a missing argument." >&2
|
||||
echo " Fix: check the variable that supplies the target, and pass the skill directory, or the agent file, as a non-empty positional argument." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
TARGET_BASE="$(_kf_basename "$TARGET")"
|
||||
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
|
||||
|
||||
if [[ -d "$TARGET" ]]; then
|
||||
if [[ -f "$TARGET/SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
else
|
||||
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would run the wrong provenance checks." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
||||
exit 2
|
||||
fi
|
||||
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
||||
MODE=agent
|
||||
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
||||
MODE=agent
|
||||
else
|
||||
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would report a silent pass on a typo'd target, which is the failure both suites' preconditions exist to prevent." >&2
|
||||
echo " Fix: pass one of those two shapes." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# In skill mode a SKILL.md target names its directory. The token is replaced in
|
||||
# place rather than assumed to be $1, because --base-ref may precede it; the
|
||||
# suite's own preconditions then apply to that directory, exactly as before the
|
||||
# merge.
|
||||
if [[ "$MODE" == skill && "$TARGET_BASE" == "SKILL.md" && ! -d "$TARGET" ]]; then
|
||||
declare -a _rewritten=()
|
||||
_replaced=false
|
||||
for _arg in "$@"; do
|
||||
if [[ "$_replaced" == false && "$_arg" == "$TARGET" ]]; then
|
||||
_rewritten+=("$(_kf_dirname "$TARGET")")
|
||||
_replaced=true
|
||||
else
|
||||
_rewritten+=("$_arg")
|
||||
fi
|
||||
done
|
||||
# Guarded expansion: bash 3.2 under `set -u` aborts on "${arr[@]}" when the
|
||||
# array is empty, and the loop above cannot prove non-emptiness to a static
|
||||
# scan. tests/test-vale-wrap.sh enforces bash-3.2 portability across this tree.
|
||||
set -- ${_rewritten[@]+"${_rewritten[@]}"}
|
||||
fi
|
||||
|
||||
# Called UNTESTED, on purpose: `f || RC=$?` would disable errexit for the whole
|
||||
# function body. Each run function stashes its findings code in
|
||||
# KYBERFORGE_PROV_RC and returns 0; its error paths exit directly.
|
||||
KYBERFORGE_PROV_RC=0
|
||||
case "$MODE" in
|
||||
skill) kyberforge_prov_skill_run "$@" ;;
|
||||
agent) kyberforge_prov_agent_run "$@" ;;
|
||||
esac
|
||||
RC="$KYBERFORGE_PROV_RC"
|
||||
|
||||
exit "$RC"
|
||||
255
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh
Executable file
255
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh
Executable file
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# The ONE entry point for structural validation. It auto-detects whether the
|
||||
# target is a skill directory or an agent definition file and runs the matching
|
||||
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh
|
||||
# and are unchanged from the skill-audit / agent-audit scripts they came from.
|
||||
# The ADR-0020 boundary resolver both of them need is sourced once, from
|
||||
# lib-boundary-resolver.sh, instead of being embedded twice.
|
||||
#
|
||||
# Detection never guesses. A target that matches neither shape is a hard exit 2
|
||||
# naming the mismatch, because the alternative — picking a mode and letting the
|
||||
# suite fail on its own terms — reports a skill-shaped finding about an agent
|
||||
# file, or the reverse, and sends the reader after the wrong problem.
|
||||
|
||||
# --- Path splitting, with bash builtins only -------------------------------
|
||||
# dirname and basename are EXTERNAL commands, and every call below happens
|
||||
# before the mode-specific python3/PyYAML preflight. Using them put coreutils
|
||||
# ahead of python3 in the dependency order: on a PATH carrying neither, this
|
||||
# script died at exit 127 naming `dirname` instead of reaching the preflight
|
||||
# that names python3 — the exact failure tests/test-adr0020-contract.sh
|
||||
# assertion 2 exists to prevent ("the two are checked separately so the message
|
||||
# names the thing to install rather than the wrong one"). The pre-merge
|
||||
# validate.sh was one self-contained file that reached its preflight on builtins
|
||||
# alone; these two functions restore that property. `cd` and `pwd` are builtins
|
||||
# and may stay.
|
||||
#
|
||||
# They reproduce dirname/basename semantics for the shapes this script sees:
|
||||
# trailing slashes are stripped, a path with no slash yields "." / itself, and
|
||||
# "/" yields "/".
|
||||
_kf_dirname() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
if [[ "$_p" != */* ]]; then
|
||||
printf '%s' "."
|
||||
return 0
|
||||
fi
|
||||
_p="${_p%/*}"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ -z "$_p" ]]; then
|
||||
_p="/"
|
||||
fi
|
||||
printf '%s' "$_p"
|
||||
}
|
||||
|
||||
_kf_basename() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${_p##*/}"
|
||||
}
|
||||
|
||||
# --- The target's parent directory NAME, resolved -------------------------
|
||||
# The agent rule tests the NAME of the target's parent directory. Reading that
|
||||
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
|
||||
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
|
||||
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
|
||||
# directory was refused as matching neither shape, by an error message naming
|
||||
# that exact shape as valid. The pre-merge agent validator had no path-shape
|
||||
# gate and worked from any working directory.
|
||||
#
|
||||
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
|
||||
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
|
||||
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
|
||||
# still addressed as agents/, which is what the literal test always honoured.
|
||||
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
|
||||
# that cannot be entered — a typo'd path — falls back to the literal name, so
|
||||
# the neither-shape error still fires for it.
|
||||
_kf_parent_name() {
|
||||
local _dir _resolved
|
||||
_dir="$(_kf_dirname "$1")"
|
||||
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
|
||||
_kf_basename "$_resolved"
|
||||
else
|
||||
_kf_basename "$_dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- This script's own directory, and the libraries beside it -------------
|
||||
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
|
||||
# CDPATH exported and the relative invocation the flow references prescribe
|
||||
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
|
||||
# and could resolve through CDPATH to an unrelated directory and source a
|
||||
# same-named file from there. CDPATH is cleared for the one command, `--` ends
|
||||
# option parsing for a directory named like a flag, and stdout is discarded so
|
||||
# only `pwd` is captured.
|
||||
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
|
||||
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
|
||||
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
|
||||
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A sourced library that is missing or unreadable used to kill the script under
|
||||
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
|
||||
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
|
||||
# partial install, or a copy or symlink of this one file taken out of scripts/,
|
||||
# was therefore reported as a failing audit. Checked explicitly instead, and
|
||||
# exit 2, which the same references read as "it never ran".
|
||||
_kf_require_lib() {
|
||||
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
|
||||
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
|
||||
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
|
||||
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate.sh <skill-dir | agent-file>
|
||||
|
||||
Validate a skill directory against the agentskills.io specification, or an agent
|
||||
definition file against the agent definition spec. The mode is detected from the
|
||||
target:
|
||||
|
||||
skill mode the target is a directory (a skill directory contains SKILL.md),
|
||||
or the target IS a SKILL.md file.
|
||||
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
||||
directory is named 'agents' (.apm/agents, .claude/agents,
|
||||
.github/agents, .copilot/agents).
|
||||
|
||||
Skill mode audits the directory named by <skill-dir>.
|
||||
|
||||
Agent mode: at plugin/APM scope, <agent-file> is a single vendor-neutral
|
||||
.apm/agents/<name>.agent.md file with no counterpart. Its frontmatter allowlist
|
||||
is not restated here: it is read at load time from the apm-agent-allowlist
|
||||
section of references/agent-field-inventory.md, which is the authoritative list.
|
||||
At project or user scope, <agent-file> is either half of a Claude Code .md /
|
||||
Copilot .agent.md pair.
|
||||
|
||||
Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
agent-file Path to the agent file (or either half of a project/user-scope pair).
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
2 Nothing was audited (no argument, the target matches neither shape, the
|
||||
target does not exist, an unrecognized file extension, a missing
|
||||
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
|
||||
beside this script)
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
TARGET="$1"
|
||||
|
||||
# --- The target has to be there --------------------------------------------
|
||||
# Only the directory branch below stats the target; the *.agent.md, the
|
||||
# agents/-parent and the SKILL.md branches classify on NAME alone, so a typo'd
|
||||
# path matching one of those shapes was handed to python3 and came back as a
|
||||
# FAIL at exit 1 — the findings tier, for a target that was never there to have
|
||||
# findings about. The tiers are: 2 nothing is at this path so no check ran, 1
|
||||
# something is there and it is broken.
|
||||
#
|
||||
# This runs on the TYPED path, before the SKILL.md -> parent-directory rewrite
|
||||
# further down: rewritten first, a missing `docs/SKILL.md` would be tested as
|
||||
# `docs`, which exists, and the guard would miss it.
|
||||
#
|
||||
# `-L` deliberately rescues what `-e` rejects. A dangling symlink and a symlink
|
||||
# loop are both FALSE to -e but TRUE to -L, and neither belongs here: something
|
||||
# IS at that path, it just cannot be opened, and "exists but unreadable" is a
|
||||
# real finding the agent suite's check_file reports as a FAIL naming the file.
|
||||
# Catching them here would replace that FAIL with a false "does not exist".
|
||||
if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then
|
||||
echo "Error: '$TARGET' does not exist." >&2
|
||||
echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2
|
||||
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- Detect the mode -------------------------------------------------------
|
||||
# Pure path and stat inspection, no interpreter and no external command needed,
|
||||
# so it runs before the python3/PyYAML preflight — which is mode-specific,
|
||||
# because each suite names the gates it would otherwise skip.
|
||||
TARGET_BASE="$(_kf_basename "$TARGET")"
|
||||
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
|
||||
|
||||
if [[ -d "$TARGET" ]]; then
|
||||
if [[ -f "$TARGET/SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
else
|
||||
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
||||
exit 2
|
||||
fi
|
||||
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
TARGET="$(_kf_dirname "$TARGET")"
|
||||
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
||||
MODE=agent
|
||||
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
||||
MODE=agent
|
||||
else
|
||||
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2
|
||||
echo " Fix: pass one of those two shapes." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- Run the matching suite ------------------------------------------------
|
||||
# Each suite is reassembled in the order the resolver block sat in before the
|
||||
# merge — preamble, resolver, body — so every check runs against exactly the
|
||||
# names and the order it always did.
|
||||
RC=0
|
||||
case "$MODE" in
|
||||
skill)
|
||||
_kf_require_lib lib-boundary-resolver.sh
|
||||
# shellcheck source=lib-boundary-resolver.sh
|
||||
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
||||
_kf_require_lib lib-checks-skill.sh
|
||||
# shellcheck source=lib-checks-skill.sh
|
||||
. "$SCRIPT_DIR/lib-checks-skill.sh"
|
||||
kyberforge_skill_preflight
|
||||
PROG="$KYBERFORGE_SKILL_PREAMBLE_PY
|
||||
$KYBERFORGE_RESOLVER_PY
|
||||
$KYBERFORGE_SKILL_BODY_PY"
|
||||
python3 -u - "$TARGET" <<< "$PROG" || RC=$?
|
||||
;;
|
||||
agent)
|
||||
_kf_require_lib lib-boundary-resolver.sh
|
||||
# shellcheck source=lib-boundary-resolver.sh
|
||||
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
||||
_kf_require_lib lib-checks-agent.sh
|
||||
# shellcheck source=lib-checks-agent.sh
|
||||
. "$SCRIPT_DIR/lib-checks-agent.sh"
|
||||
kyberforge_agent_preflight
|
||||
PROG="$KYBERFORGE_AGENT_PREAMBLE_PY
|
||||
$KYBERFORGE_RESOLVER_PY
|
||||
$KYBERFORGE_AGENT_BODY_PY"
|
||||
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
|
||||
;;
|
||||
esac
|
||||
|
||||
exit "$RC"
|
||||
94
plugins/kyberforge/.apm/skills/factory-audit/tests/README.md
Normal file
94
plugins/kyberforge/.apm/skills/factory-audit/tests/README.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## When to add tests
|
||||
|
||||
Add tests here when the skill has scripts in `scripts/` that are complex enough
|
||||
to break silently — validators, parsers, generators, anything with branching
|
||||
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
|
||||
belongs here, not in `scripts/`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Tests require [bats-support](https://github.com/bats-core/bats-support) and
|
||||
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
|
||||
helpers from the repo root's `tests/test_helper/`.
|
||||
|
||||
From the repo root:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests for this skill (from the repo root):
|
||||
|
||||
```bash
|
||||
bats plugins/kyberforge/.apm/skills/factory-audit/tests/
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `validate-skill.bats` | `scripts/validate.sh` against skill directories |
|
||||
| `validate-agent.bats` | `scripts/validate.sh` against agent files |
|
||||
| `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories |
|
||||
| `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files |
|
||||
|
||||
## Two scripts, four suites
|
||||
|
||||
`factory-audit` merges what were two skills — `skill-audit` and `agent-audit` —
|
||||
each of which shipped its own `validate.sh` and `validate-provenance.sh`. The
|
||||
merged skill has **one** of each. Every suite here invokes one of those two
|
||||
scripts; the four files are two scripts × two artifact types, not four scripts.
|
||||
|
||||
`validate-skill.bats` and `validate-agent.bats` run the same
|
||||
`scripts/validate.sh` and differ only in the fixtures they point it at. The two
|
||||
provenance suites stand in the same relation to `scripts/validate-provenance.sh`.
|
||||
Do not add a third script path here on the assumption that a differently named
|
||||
suite must mean a differently named script.
|
||||
|
||||
### Auto-detection is pinned across the pair
|
||||
|
||||
Each entry point decides for itself what it was handed. ADR-0025 states the
|
||||
rule: a directory containing `SKILL.md` takes the skill flow; an `.agent.md`
|
||||
file, or a file under a directory named `agents/`, takes the agent flow.
|
||||
Anything else is rejected rather than guessed at. That behaviour is new with the
|
||||
merge — before it, each script was hard-wired to one artifact type and nothing
|
||||
about classification could be wrong — so it is asserted from both sides rather
|
||||
than in one place:
|
||||
|
||||
- the skill-side suites pin the skill-directory classification and the
|
||||
neither-shape rejection,
|
||||
- the agent-side suites pin the two agent rules *separately* — `.agent.md` in a
|
||||
directory that is not `agents/`, and a plain `.md` under `.apm/agents/` — so
|
||||
that a detector implementing only one of them cannot pass both. Plus a control
|
||||
asserting an agent file never picks up a skill-only gate.
|
||||
|
||||
Both skill-side suites additionally pin the `SKILL.md` **file** path, not just
|
||||
the directory: a pre-commit `files:` hook matches files, so every hook-driven
|
||||
invocation hands over a `SKILL.md` path. Each entry point resolves it to the
|
||||
directory, and the suites assert the two spellings produce the identical
|
||||
verdict rather than merely that the file spelling survives.
|
||||
|
||||
A misclassification is silent and total — the wrong rubric runs end to end and
|
||||
reports the artifact clean against gates that never applied to it — and no other
|
||||
fixture in these suites would notice, because every other fixture is already the
|
||||
shape its own suite expects.
|
||||
|
||||
### The two provenance exit contracts are different on purpose
|
||||
|
||||
`scripts/validate-provenance.sh` does **not** behave identically in its two
|
||||
modes, and the difference is deliberate:
|
||||
|
||||
| Mode | Successful run |
|
||||
|------|----------------|
|
||||
| skill | may exit 0 **with** output — INFO findings print, status stays 0 |
|
||||
| agent | exits 0 and prints **nothing** |
|
||||
|
||||
Each half is asserted from its own side, by the `exit contract:` test in each
|
||||
provenance suite. Both are asserted on purpose: a merge that collapsed one
|
||||
contract into the other would still satisfy whichever side was left unasserted,
|
||||
so a single-sided pin would go green on exactly the defect it exists to catch.
|
||||
@@ -131,7 +131,22 @@ EOF
|
||||
}
|
||||
|
||||
@test "user scope: agent file directly in \$HOME (start dir IS exactly \$HOME, no walk-up) resolves to user scope" {
|
||||
local fake_home="$TMPDIR/fakehome-direct"
|
||||
# The fake $HOME is named `agents` on purpose, and it is a fixture detail
|
||||
# forced by the merged entry point rather than anything the scope resolver
|
||||
# cares about. The case under test needs the agent file's own directory to BE
|
||||
# $HOME (detect_scope's `original_start == home` branch), AND the input has
|
||||
# to be the Claude Code half — a plain .md — because that is the only half
|
||||
# whose counterpart path differs between user scope ($HOME/.copilot/agents)
|
||||
# and project scope ($HOME/.github/agents). With a *.agent.md input the two
|
||||
# scopes derive the identical counterpart and the test would pass without
|
||||
# discriminating. Post-merge, a plain .md only classifies as an agent when
|
||||
# its parent directory is named `agents`, so the two requirements can only be
|
||||
# satisfied together by a $HOME whose basename is `agents`.
|
||||
#
|
||||
# detect_scope's conventional_shape stays FALSE here (the grandparent is not
|
||||
# one of .claude/.github/.copilot/.apm), so this still exercises the
|
||||
# no-walk-up branch and not the two-segments-up arithmetic.
|
||||
local fake_home="$TMPDIR/fakehome-direct/agents"
|
||||
mkdir -p "$fake_home" "$fake_home/.copilot/agents"
|
||||
cat > "$fake_home/my-agent.md" <<EOF
|
||||
---
|
||||
@@ -267,13 +282,22 @@ EOF
|
||||
@test "project scope: a non-conventional path (agent file not directly under a literal 'agents' dir) falls back to the nearest .git boundary instead of two-segments-up arithmetic" {
|
||||
local outer="$TMPDIR/outer-repo"
|
||||
local pkg="$outer/pkgA"
|
||||
mkdir -p "$pkg/.git" "$pkg/.github/agents" "$pkg/extra"
|
||||
mkdir -p "$pkg/.git" "$pkg/.claude/agents" "$pkg/extra"
|
||||
# Misplaced file: sits two path segments below $outer (pkgA/extra), which
|
||||
# matches the conventional_root arithmetic by coincidence, but its
|
||||
# immediate parent dir is "extra", not "agents" — conventional_shape is
|
||||
# false, so the fix must fall back to the nearest .git boundary (pkgA),
|
||||
# not trust $outer.
|
||||
cat > "$pkg/extra/my-agent.md" <<EOF
|
||||
#
|
||||
# The *.agent.md suffix is a post-merge fixture requirement, not part of what
|
||||
# is being measured: the merged entry point classifies a plain .md as an
|
||||
# agent only when its parent directory is named `agents`, and the whole point
|
||||
# of this fixture is that the parent is `extra`. The suffix is what keeps the
|
||||
# file classifiable while leaving conventional_shape false. It flips which
|
||||
# half of the pair is the input, so the counterpart below is the Claude Code
|
||||
# half rather than the Copilot one; the scope arithmetic under test is
|
||||
# untouched by that.
|
||||
cat > "$pkg/extra/my-agent.agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
@@ -283,9 +307,9 @@ You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
# Counterpart at the nearest-.git root (pkgA), not at $outer — if the
|
||||
# arithmetic were trusted here, validate.sh would look for a counterpart
|
||||
# at $outer/.github/agents/my-agent.agent.md, which doesn't exist, and
|
||||
# at $outer/.claude/agents/my-agent.md, which doesn't exist, and
|
||||
# false-FAIL.
|
||||
cat > "$pkg/.github/agents/my-agent.agent.md" <<EOF
|
||||
cat > "$pkg/.claude/agents/my-agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
@@ -293,7 +317,7 @@ description: A valid agent description.
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$pkg/extra/my-agent.md"
|
||||
run bash "$SCRIPT" "$pkg/extra/my-agent.agent.md"
|
||||
assert_success
|
||||
refute_output --partial "counterpart file not found"
|
||||
}
|
||||
@@ -873,21 +897,42 @@ make_tools_pair() {
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# A file that cannot be read
|
||||
# A file that is not there, and a file that cannot be read
|
||||
# ---------------------------------------------------------------------------
|
||||
# scripts/check-apm-agents-valid.sh derives its expected agent-file set from
|
||||
# `git ls-files`, so it hands this script paths that are tracked but absent from
|
||||
# the worktree — a real and expected state, not a corner case. That used to exit
|
||||
# 1 with a bare FileNotFoundError traceback and no FAIL line at all: non-zero, so
|
||||
# the gate blocked, but with an interpreter stack instead of a diagnostic naming
|
||||
# the file. Both scope paths are covered because they are separate call sites
|
||||
# (check_apm_agent_file and check_file) and each needed its own handler.
|
||||
# These are two different tiers and the split is the point.
|
||||
#
|
||||
# `is-a-dir.agent.md` is a DIRECTORY rather than a chmod 000 file on purpose:
|
||||
# these tests run as root in CI, where mode bits do not deny anything and a
|
||||
# permissions fixture would be silently readable and prove nothing.
|
||||
# NOT THERE is exit 2: nothing is at the path, so no check ran. The entry
|
||||
# point's existence guard catches it before python3 is invoked, because the
|
||||
# *.agent.md and agents/-parent branches classify on NAME alone and a typo'd
|
||||
# path used to sail through them into python and come back as a FAIL at exit 1
|
||||
# — the FINDINGS tier, for a target that was never there to have findings
|
||||
# about. (Before that it was worse still: a bare FileNotFoundError traceback
|
||||
# and no FAIL line at all.) Both scope paths are covered because they were
|
||||
# separate call sites, check_apm_agent_file and check_file, and each needed its
|
||||
# own handler; the guard now short-circuits both, and the traceback assertions
|
||||
# below are kept to pin that the diagnostic is still a sentence and not a stack.
|
||||
#
|
||||
# CANNOT BE READ is exit 1: something IS at the path and opening it fails, which
|
||||
# is a real finding, and it must still reach check_file's `except OSError` arm.
|
||||
# That is why the guard tests `! -e && ! -L` rather than `! -e` alone — a
|
||||
# dangling symlink and a symlink loop are both false to -e and true to -L. The
|
||||
# unreadable/nonexistent pairs below sit together so that distinction cannot be
|
||||
# collapsed by accident.
|
||||
#
|
||||
# The unreadable fixture is a SYMLINK LOOP rather than a chmod 000 file: these
|
||||
# tests run as root in CI, where mode bits do not deny anything and a permissions
|
||||
# fixture would be silently readable and prove nothing.
|
||||
#
|
||||
# It used to be a DIRECTORY named `is-a-dir.agent.md`, which served the same
|
||||
# purpose pre-merge. That no longer reaches the handler: the merged entry point
|
||||
# classifies on shape first, and a directory is a skill-shaped target whatever it
|
||||
# is named, so it is rejected at exit 2 before python3 is invoked. A symlink loop
|
||||
# is a FILE by the detector's reckoning, classifies as an agent on its name, and
|
||||
# still raises OSError on open — so it exercises the same `except OSError` arm.
|
||||
# The directory-named-like-an-agent case did not go away with the fixture; it is
|
||||
# pinned in its own test immediately below.
|
||||
|
||||
@test "a nonexistent plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
|
||||
@test "a nonexistent plugin/APM-scope agent file exits 2 naming the path, never the findings tier" {
|
||||
local root="$TMPDIR/pkg"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
@@ -896,15 +941,45 @@ version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.apm/agents/absent.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "FAIL"
|
||||
assert_output --partial "could not be read"
|
||||
assert_output --partial "absent.agent.md"
|
||||
# The literal 2, not assert_failure: 1 would pass a bare assert_failure, and
|
||||
# 1 is exactly the wrong answer here — it is the tier that says the file was
|
||||
# audited and has findings.
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "Error: '$root/.apm/agents/absent.agent.md' does not exist."
|
||||
# Kept from when this path reached python3 and returned a stack. Trivially
|
||||
# true now that the guard short-circuits before the interpreter, and kept
|
||||
# anyway: the assertion is "a reader gets a sentence, not a traceback", and
|
||||
# that intent outlives the mechanism that currently satisfies it.
|
||||
refute_output --partial "Traceback"
|
||||
refute_output --partial "FileNotFoundError"
|
||||
}
|
||||
|
||||
@test "an unreadable plugin/APM-scope agent file gets a FAIL naming the path, not a traceback" {
|
||||
local root="$TMPDIR/pkg-loop"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
ln -s "unreadable.agent.md" "$root/.apm/agents/unreadable.agent.md"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.apm/agents/unreadable.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "FAIL"
|
||||
assert_output --partial "could not be read"
|
||||
assert_output --partial "unreadable.agent.md"
|
||||
refute_output --partial "Traceback"
|
||||
refute_output --partial "OSError"
|
||||
}
|
||||
|
||||
@test "a DIRECTORY named like an agent file is rejected by shape, not audited as one" {
|
||||
# The other half of the case the fixture above used to cover, kept as its own
|
||||
# test so nothing is lost in the swap. A directory cannot be an agent target
|
||||
# under ADR-0025's rule at all, so it never reaches check_apm_agent_file's
|
||||
# OSError arm — it is refused at classification time. Exit 2 (not auditable)
|
||||
# rather than 1 (audited and failing), and still a message naming the path:
|
||||
# the failure mode this guards against is a silent exit 0 or a bare non-zero
|
||||
# with an interpreter stack for a diagnostic.
|
||||
local root="$TMPDIR/pkg-dir"
|
||||
mkdir -p "$root/.apm/agents/is-a-dir.agent.md"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
@@ -913,18 +988,19 @@ version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.apm/agents/is-a-dir.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "FAIL"
|
||||
assert_output --partial "could not be read"
|
||||
assert_output --partial "is-a-dir.agent.md"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "Error: '$root/.apm/agents/is-a-dir.agent.md' is a directory with no SKILL.md in it."
|
||||
refute_output --partial "Traceback"
|
||||
refute_output --partial "IsADirectoryError"
|
||||
}
|
||||
|
||||
@test "a nonexistent project-scope agent file gets a FAIL naming the path, not a traceback" {
|
||||
# The counterpart is pre-checked before either file is opened, so this
|
||||
# exercises the OTHER call site: the counterpart exists, the named file does
|
||||
# not, and check_file is what has to report it.
|
||||
@test "a nonexistent project-scope agent file exits 2 naming the path, never the findings tier" {
|
||||
# The counterpart exists and the named file does not. Before the guard this
|
||||
# exercised the OTHER call site — check_file, reached after the counterpart
|
||||
# pre-check — and the fact that the counterpart is present is what made it a
|
||||
# distinct case. The guard is upstream of scope detection entirely, so both
|
||||
# scopes now stop at the same place; the fixture is kept so a guard that ever
|
||||
# regresses to a scope-specific check is caught on both paths.
|
||||
local root="$TMPDIR/proj-missing"
|
||||
mkdir -p "$root/.git" "$root/.claude/agents" "$root/.github/agents"
|
||||
cat > "$root/.github/agents/my-agent.agent.md" <<EOF
|
||||
@@ -936,10 +1012,11 @@ description: A valid agent description. Do not use for anything else.
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.claude/agents/my-agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "FAIL"
|
||||
assert_output --partial "could not be read"
|
||||
assert_output --partial "my-agent.md"
|
||||
# The literal 2, not assert_failure — see the plugin-scope twin above.
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "Error: '$root/.claude/agents/my-agent.md' does not exist."
|
||||
# Trivially true now that the guard short-circuits before python3; kept
|
||||
# because the intent — a sentence, not an interpreter stack — is unchanged.
|
||||
refute_output --partial "Traceback"
|
||||
refute_output --partial "FileNotFoundError"
|
||||
}
|
||||
@@ -963,3 +1040,180 @@ EOF
|
||||
refute_output --partial "UnicodeEncodeError"
|
||||
refute_output --partial "Traceback"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-detection — the merged entry point classifies its own target
|
||||
#
|
||||
# NEW with the factory-audit merge, and new behaviour rather than a ported
|
||||
# case: scripts/validate.sh is now ONE entry point for both artifact types and
|
||||
# works out from the target which rubric to run. This file is the AGENT half of
|
||||
# that contract; validate-skill.bats holds the skill half and the
|
||||
# neither-shape rejection. Same script in all four, different fixtures.
|
||||
#
|
||||
# ADR-0025 states the rule: "A directory containing SKILL.md takes the skill
|
||||
# flow; an .agent.md file or a file under agents/ takes the agent flow." That is
|
||||
# TWO independent rules on the agent side, and each is pinned on its own below,
|
||||
# because either one alone would make the other look like it worked:
|
||||
#
|
||||
# 1. the filename ends in .agent.md, wherever it sits
|
||||
# 2. the file sits under a directory named agents/, whatever it is called
|
||||
# (.apm/agents/, .claude/agents/, .github/agents/ — the same shape the
|
||||
# exported Vale hook matches with `(^|/)agents/[^/]+\.md$`)
|
||||
#
|
||||
# A misclassification here is silent and total — the skill rubric would run
|
||||
# against an agent file, so ADR-0022's metadata.version and ADR-0020's 900-word
|
||||
# body ceiling would both be applied to an artifact that ADR-0020 explicitly
|
||||
# exempts from the latter (see the 1,500-word body test above, which AGENTS.md
|
||||
# cites as the pin for that asymmetry).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "auto-detect: a *.agent.md file is audited in AGENT mode wherever it sits" {
|
||||
# Rule 1 in isolation. The directory is deliberately named `defs/`, not
|
||||
# `agents/`, so rule 2 cannot reach this fixture and the extension is the
|
||||
# only thing that can classify it. Put it under agents/ and both rules match,
|
||||
# and a detector implementing only rule 2 would still pass this test.
|
||||
local root="$TMPDIR/pkg-by-extension"
|
||||
mkdir -p "$root/defs"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
cat > "$root/defs/my-agent.agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description. Do not use for anything else.
|
||||
target: cli
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/defs/my-agent.agent.md"
|
||||
# 'target' is a Copilot-only field, and the vendor-field allowlist that
|
||||
# rejects it exists only in the agent rubric. The finding is therefore proof
|
||||
# of WHICH rubric ran, not just that something went wrong. Exit 1 exactly:
|
||||
# a bare assert_failure also accepts exit 2, which is a refusal to audit, and
|
||||
# "target" is too common a word to tell a findings report from a refusal.
|
||||
assert_failure 1
|
||||
assert_output --partial "target"
|
||||
}
|
||||
|
||||
@test "auto-detect: a file under .apm/agents/ is audited in AGENT mode even without the .agent.md suffix" {
|
||||
# Rule 2 in isolation. `.apm/agents/` is the vendor-neutral authoring
|
||||
# directory, so anything sitting in it is an agent definition whatever it is
|
||||
# called — and the plain `.md` suffix here is deliberate, so that rule 1
|
||||
# cannot rescue the fixture and rule 2 is the only thing being measured.
|
||||
local root="$TMPDIR/pkg-by-location"
|
||||
mkdir -p "$root/.apm/agents"
|
||||
cat > "$root/apm.yml" <<EOF
|
||||
name: test-package
|
||||
version: 0.1.0
|
||||
type: skill
|
||||
EOF
|
||||
cat > "$root/.apm/agents/my-agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description. Do not use for anything else.
|
||||
tools: Read Edit
|
||||
---
|
||||
|
||||
You are a test agent. When invoked, do the thing.
|
||||
EOF
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
|
||||
# 'tools' sits outside the plugin/APM allowlist — again agent-rubric-only
|
||||
# vocabulary, so the finding names the rubric that produced it. Exit 1
|
||||
# exactly, for the same reason as the case above: exit 2 is a refusal, not a
|
||||
# findings report, and "tools" alone cannot tell the two apart.
|
||||
assert_failure 1
|
||||
assert_output --partial "tools"
|
||||
}
|
||||
|
||||
@test "auto-detect: an agent file never picks up a skill-only gate" {
|
||||
# The control for both cases above. Without it, either is satisfied by a
|
||||
# detector that runs BOTH rubrics and unions the findings — which would
|
||||
# report every agent file in the corpus as missing a metadata.version it is
|
||||
# not required to have.
|
||||
local root="$TMPDIR/pkg-clean-detect"
|
||||
make_apm_agent "$root" "my-agent"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
assert_success
|
||||
refute_output --partial "metadata.version"
|
||||
refute_output --partial "900-word"
|
||||
refute_output --partial "SKILL.md"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry-point robustness: how the target and the libraries are located
|
||||
# ---------------------------------------------------------------------------
|
||||
# Three defects in the merged entry point's own plumbing, none of which a valid
|
||||
# absolute-path target exercises:
|
||||
#
|
||||
# - The agents/-parent rule read the parent's NAME off the argument text, so a
|
||||
# bare `my-agent.md` typed from inside the agents/ directory had a parent of
|
||||
# "." and was refused as matching neither shape — by a message naming that
|
||||
# exact shape as valid.
|
||||
# - A missing lib-*.sh killed the script under `set -e` with exit 1, the tier
|
||||
# the flow references relay verbatim as real findings.
|
||||
# - SCRIPT_DIR was resolved with a bare `cd`, which under an exported CDPATH
|
||||
# prints the directory it chose and can choose a DIFFERENT one — so a
|
||||
# relative invocation could source same-named libraries from elsewhere.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "entry point: a bare filename typed from inside an agents/ directory is audited, not refused" {
|
||||
# The plain .md suffix is deliberate: *.agent.md is accepted on its name
|
||||
# alone, so only the agents/-parent rule is measured here.
|
||||
local root="$TMPDIR/pkg-bare"
|
||||
make_apm_agent "$root" "my-agent"
|
||||
mv "$root/.apm/agents/my-agent.agent.md" "$root/.apm/agents/my-agent.md"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
|
||||
local abs_status="$status"
|
||||
# The literal 0, not `-ne 2`. The three invocations below are only compared
|
||||
# against this one, so an anchor of "anything but 2" left the whole test
|
||||
# green under a regression that made all three exit 1 — every verdict here
|
||||
# was self-referential. make_apm_agent builds a CLEAN agent, so the
|
||||
# absolute-path baseline is 0 and nothing weaker belongs here. Its provenance
|
||||
# twin in validate-provenance-agent.bats pins the same literal.
|
||||
[ "$abs_status" -eq 0 ]
|
||||
|
||||
cd "$root/.apm/agents"
|
||||
run bash "$SCRIPT" my-agent.md
|
||||
refute_output --partial "matches neither"
|
||||
[ "$status" -eq "$abs_status" ]
|
||||
|
||||
run bash "$SCRIPT" ./my-agent.md
|
||||
refute_output --partial "matches neither"
|
||||
[ "$status" -eq "$abs_status" ]
|
||||
}
|
||||
|
||||
@test "entry point: a missing mode library exits 2 naming it, never exit 1" {
|
||||
# Exit 1 is the findings tier. A validate.sh separated from its libraries is
|
||||
# an install problem, and has to land in the "never ran" tier instead.
|
||||
local root="$TMPDIR/pkg-lone"
|
||||
make_apm_agent "$root" "my-agent"
|
||||
local lone="$TMPDIR/lone-scripts"
|
||||
cp -R "$(dirname "$SCRIPT")" "$lone"
|
||||
rm "$lone/lib-checks-agent.sh"
|
||||
run bash "$lone/validate.sh" "$root/.apm/agents/my-agent.agent.md"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "required library '$lone/lib-checks-agent.sh' is missing or unreadable"
|
||||
refute_output --partial "No such file or directory"
|
||||
}
|
||||
|
||||
@test "entry point: an exported CDPATH cannot redirect which libraries are sourced" {
|
||||
# A decoy scripts/ directory reachable through CDPATH, holding libraries that
|
||||
# announce themselves and abort. The relative invocation `bash
|
||||
# scripts/validate.sh` is the one the flow references prescribe, and it is the
|
||||
# one CDPATH applies to.
|
||||
local root="$TMPDIR/pkg-cdpath"
|
||||
make_apm_agent "$root" "my-agent"
|
||||
local decoy="$TMPDIR/decoy"
|
||||
mkdir -p "$decoy/scripts"
|
||||
local lib
|
||||
for lib in lib-boundary-resolver.sh lib-checks-skill.sh lib-checks-agent.sh; do
|
||||
printf 'echo DECOY-SOURCED; exit 99\n' > "$decoy/scripts/$lib"
|
||||
done
|
||||
cd "$(dirname "$SCRIPT")/.."
|
||||
run env CDPATH="$decoy" bash scripts/validate.sh "$root/.apm/agents/my-agent.agent.md"
|
||||
refute_output --partial "DECOY-SOURCED"
|
||||
assert_success
|
||||
}
|
||||
@@ -586,12 +586,41 @@ EOF
|
||||
# pins it as exit 0 with empty output. Every exit-2 gate is therefore decided
|
||||
# from the ARGUMENT ALONE, before the walk-up runs, so the two can never
|
||||
# collide. The two tests at the end of this block assert both halves.
|
||||
#
|
||||
# PORT NOTE RESOLVED (factory-audit merge): two cases in this block asserted
|
||||
# wording that the skill suite asserted differently for the same class of input
|
||||
# — "agent-file is required" against its "skill-dir is required", and "no such
|
||||
# file" against its "not a directory". Each pre-merge script knew what shape it
|
||||
# was owed; the merged one classifies before it complains. How each pair landed:
|
||||
#
|
||||
# * no argument — NEITHER old wording survives. One entry point takes both
|
||||
# shapes, so it names both: "a skill directory or an agent file is
|
||||
# required." Both suites now assert that one sentence.
|
||||
# * nonexistent path — "no such file" SURVIVES here and did not move. A
|
||||
# nonexistent *.agent.md still classifies as an agent on its name alone, so
|
||||
# agent mode's own precondition is what rejects it. The skill suite's "not a
|
||||
# directory" is the wording that moved, because a nonexistent path with no
|
||||
# agent-shaped name classifies as neither.
|
||||
#
|
||||
# Two further cases in this block did move, and neither moved to the other
|
||||
# suite's wording: a directory handed to this script is now "is a directory with
|
||||
# no SKILL.md in it" rather than "not a regular file", and an unrecognized
|
||||
# extension is "matches neither a skill directory nor an agent file" rather than
|
||||
# "unrecognized extension". Both are classification verdicts now, reached before
|
||||
# either mode's preconditions run.
|
||||
#
|
||||
# The invariant every case in this block pins is unchanged and still pinned at
|
||||
# every site: exit 2 with an explanatory message on stderr, never a silent exit
|
||||
# 0 or a bare 1 with no output.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "exit 2: no arguments is a usage error, not a finding" {
|
||||
run bash "$SCRIPT"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "agent-file is required"
|
||||
assert_output --partial "Error: a skill directory or an agent file is required."
|
||||
# The usage block has to follow the error, or "required" names no shape the
|
||||
# caller can act on.
|
||||
assert_output --partial "Usage: validate-provenance.sh"
|
||||
}
|
||||
|
||||
@test "exit 2: a second positional argument is rejected instead of silently dropped" {
|
||||
@@ -610,20 +639,29 @@ EOF
|
||||
}
|
||||
|
||||
@test "exit 2: a directory is not an agent file" {
|
||||
# Post-merge this is a CLASSIFICATION rejection, not agent mode's old
|
||||
# "not a regular file" precondition: a directory is half of a skill target,
|
||||
# so the detector reaches for SKILL.md, does not find one, and says so. The
|
||||
# thing being pinned is the same — a directory handed to a script that
|
||||
# audits agent FILES exits 2 with a message, never a silent 0.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
run bash "$SCRIPT" "$root/.apm/agents"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "not a regular file"
|
||||
assert_output --partial "Error: '$root/.apm/agents' is a directory with no SKILL.md in it."
|
||||
}
|
||||
|
||||
@test "exit 2: an unrecognized extension is rejected before the walk-up runs" {
|
||||
# Also a classification rejection now. .txt under .apm/agents/ is neither
|
||||
# shape: the agents/-parent rule only admits .md, and this is not *.agent.md.
|
||||
# Rejected on the name alone, so the walk-up still never runs — which is what
|
||||
# keeps this case from colliding with the silent not-plugin-scope exit 0.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
echo "not an agent" > "$root/.apm/agents/my-agent.txt"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.txt"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "unrecognized extension"
|
||||
assert_output --partial "Error: '$root/.apm/agents/my-agent.txt' matches neither a skill directory nor an agent file."
|
||||
}
|
||||
|
||||
@test "exit 2: a PATH with no python3 names the missing dependency instead of exiting 127" {
|
||||
@@ -632,6 +670,22 @@ EOF
|
||||
make_clean_agent "$root"
|
||||
local emptybin="$TMPDIR/emptybin"
|
||||
mkdir -p "$emptybin"
|
||||
# dirname and basename are deliberately ABSENT, and that absence is load-
|
||||
# bearing. The invariant: nothing external is needed to reach the python3
|
||||
# preflight. The entry point resolves SCRIPT_DIR and classifies the target
|
||||
# with bash builtins (_kf_dirname, _kf_basename, _kf_parent_name) and the
|
||||
# mode libraries build their Python bodies with `read` heredocs, so a
|
||||
# genuinely empty PATH reaches the preflight too. Widening this list to keep
|
||||
# a test green would silently retire that guarantee: the script would die at
|
||||
# 127 naming `dirname` instead of the dependency it actually needs, the
|
||||
# failure tests/test-adr0020-contract.sh assertion 2 exists to prevent. cat
|
||||
# and sed stay so the stub matches the skill suite's and is a DENY OF python3
|
||||
# ALONE rather than a test of "no PATH at all"; bash is not among them
|
||||
# because `env -i` below invokes it by absolute path.
|
||||
local cmd
|
||||
for cmd in cat sed; do
|
||||
ln -s "$(command -v "$cmd")" "$emptybin/$cmd"
|
||||
done
|
||||
local bash_bin
|
||||
bash_bin="$(command -v bash)"
|
||||
run env -i PATH="$emptybin" HOME="$HOME" "$bash_bin" "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
@@ -907,3 +961,187 @@ EOF
|
||||
count="$(printf '%s\n' "$output" | grep -c "^FAIL File is not valid UTF-8" || true)"
|
||||
[ "$count" -eq 1 ]
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-detection — the merged provenance entry point classifies its own target
|
||||
#
|
||||
# NEW with the factory-audit merge, and new behaviour rather than a ported
|
||||
# case: scripts/validate-provenance.sh is now ONE entry point for both artifact
|
||||
# types and works out from the target which rubric to run. This file is the
|
||||
# AGENT half of that contract; validate-provenance-skill.bats holds the skill
|
||||
# half and the neither-shape rejection. Same script in all four suites.
|
||||
#
|
||||
# ADR-0025 states the rule: "A directory containing SKILL.md takes the skill
|
||||
# flow; an .agent.md file or a file under agents/ takes the agent flow." That is
|
||||
# TWO independent rules on the agent side, and each is pinned on its own below,
|
||||
# because either one alone would make the other look like it worked:
|
||||
#
|
||||
# 1. the filename ends in .agent.md, wherever it sits
|
||||
# 2. the file sits under a directory named agents/, whatever it is called
|
||||
# (.apm/agents/, .claude/agents/, .github/agents/ — the same shape the
|
||||
# exported Vale hook matches with `(^|/)agents/[^/]+\.md$`)
|
||||
#
|
||||
# The skill rubric rejects a FILE outright ("not a directory"), so a
|
||||
# misclassified agent file does not produce a wrong finding — it produces a
|
||||
# usage error about a file that was perfectly well-formed, which is why these
|
||||
# assert on a real finding rather than merely on a non-crash.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "auto-detect: a *.agent.md file is checked in AGENT provenance mode wherever it sits" {
|
||||
# Rule 1 in isolation. The directory is deliberately named `defs/`, not
|
||||
# `agents/`, so rule 2 cannot reach this fixture and the extension is the
|
||||
# only thing that can classify it.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
mkdir -p "$root/defs"
|
||||
cat > "$root/defs/my-agent.agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
source_keys:
|
||||
- ghost-source
|
||||
---
|
||||
|
||||
You are a test agent.
|
||||
EOF
|
||||
make_sources_md "$root" "my-source" "(none)"
|
||||
run bash "$SCRIPT" "$root/defs/my-agent.agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "source_keys slug 'ghost-source' not found in sources.md"
|
||||
# Check 9 is skill-only. It must not appear here at any tier — its presence
|
||||
# would mean the skill rubric had been run against an agent file.
|
||||
refute_output --partial "Check 9"
|
||||
}
|
||||
|
||||
@test "auto-detect: a file under .apm/agents/ is checked in AGENT provenance mode even without the .agent.md suffix" {
|
||||
# Rule 2 in isolation. `.apm/agents/` is the vendor-neutral authoring
|
||||
# directory, so anything sitting in it is an agent definition whatever it is
|
||||
# called — and the plain `.md` suffix here is deliberate, so that rule 1
|
||||
# cannot rescue the fixture and rule 2 is the only thing being measured.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
cat > "$root/.apm/agents/my-agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
source_keys:
|
||||
- ghost-source
|
||||
---
|
||||
|
||||
You are a test agent.
|
||||
EOF
|
||||
make_sources_md "$root" "my-source" "(none)"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.md"
|
||||
assert_failure
|
||||
assert_output --partial "source_keys slug 'ghost-source' not found in sources.md"
|
||||
refute_output --partial "Check 9"
|
||||
}
|
||||
|
||||
@test "--base-ref is SKILL-mode only: handing it an agent target is an error, not a silent no-op" {
|
||||
# --base-ref belongs to skill mode's check 9, and agent mode has no check 9
|
||||
# at any tier. The merged entry point does NOT own the flag rules — it skips
|
||||
# --base-ref only far enough to find the positional that decides the mode,
|
||||
# then hands the original "$@" to the mode's own suite. So in agent mode the
|
||||
# flag is still just a second argument and is still rejected as one. Silently
|
||||
# accepting and ignoring it is the failure worth pinning: a caller who passed
|
||||
# --base-ref believes a diff ran, and nothing would tell them otherwise.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
make_clean_agent "$root"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md" --base-ref=HEAD
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "expected exactly one argument"
|
||||
}
|
||||
|
||||
@test "--base-ref before an agent target is rejected too, not swallowed by the mode detector" {
|
||||
# The detector skips leading --base-ref= tokens to find the positional. That
|
||||
# skip must not launder the flag out of the argument list: the suite still
|
||||
# sees two arguments and still rejects them. Asserted separately from the
|
||||
# trailing-flag case because the skip loop only runs on this ordering, so a
|
||||
# detector that consumed the flag instead of stepping over it would pass the
|
||||
# test above and fail only here.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
make_clean_agent "$root"
|
||||
run bash "$SCRIPT" --base-ref=HEAD "$root/.apm/agents/my-agent.agent.md"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "expected exactly one argument"
|
||||
}
|
||||
|
||||
@test "exit contract: AGENT mode prints NOTHING on a successful run" {
|
||||
# The other half of the contract pinned in validate-provenance-skill.bats:
|
||||
# skill mode is allowed to exit 0 with INFO findings on stdout; agent mode
|
||||
# says nothing at all when it passes.
|
||||
#
|
||||
# Both halves are asserted explicitly and on purpose. A merge that collapsed
|
||||
# one contract into the other would still satisfy whichever side was left
|
||||
# unasserted, so a single-sided pin would go green on exactly the defect it
|
||||
# was written to catch. assert_output "" is an equality assertion, not a
|
||||
# --partial one, so any added banner, mode line or progress note trips it.
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
make_agent_with_source_keys "$root"
|
||||
make_sources_md "$root"
|
||||
run bash "$SCRIPT" "$root/.apm/agents/my-agent.agent.md"
|
||||
[ "$status" -eq 0 ]
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Entry-point robustness: how the target and the libraries are located
|
||||
# ---------------------------------------------------------------------------
|
||||
# The same three plumbing defects validate-agent.bats pins for validate.sh — a
|
||||
# bare filename inside agents/ refused as neither shape, a missing lib-*.sh
|
||||
# reported as exit 1 findings, and a CDPATH-redirected SCRIPT_DIR — pinned here
|
||||
# for the provenance entry point, which resolves both the same way.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "entry point: a bare filename typed from inside an agents/ directory is audited, not refused" {
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
cat > "$root/.apm/agents/my-agent.md" <<EOF
|
||||
---
|
||||
name: my-agent
|
||||
description: A valid agent description.
|
||||
---
|
||||
|
||||
You are a test agent.
|
||||
EOF
|
||||
cd "$root/.apm/agents"
|
||||
run bash "$SCRIPT" my-agent.md
|
||||
[ "$status" -eq 0 ]
|
||||
assert_output ""
|
||||
|
||||
run bash "$SCRIPT" ./my-agent.md
|
||||
[ "$status" -eq 0 ]
|
||||
assert_output ""
|
||||
}
|
||||
|
||||
@test "entry point: a missing library exits 2 naming it, never exit 1" {
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
make_clean_agent "$root"
|
||||
local lone="$TMPDIR/lone-scripts"
|
||||
cp -R "$(dirname "$SCRIPT")" "$lone"
|
||||
rm "$lone/lib-contributing-files.sh"
|
||||
run bash "$lone/validate-provenance.sh" "$root/.apm/agents/my-agent.agent.md"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "required library '$lone/lib-contributing-files.sh' is missing or unreadable"
|
||||
refute_output --partial "No such file or directory"
|
||||
}
|
||||
|
||||
@test "entry point: an exported CDPATH cannot redirect which libraries are sourced" {
|
||||
local root="$TMPDIR/package"
|
||||
make_package "$root"
|
||||
make_clean_agent "$root"
|
||||
local decoy="$TMPDIR/decoy"
|
||||
mkdir -p "$decoy/scripts"
|
||||
local lib
|
||||
for lib in lib-contributing-files.sh lib-provenance-skill.sh lib-provenance-agent.sh; do
|
||||
printf 'echo DECOY-SOURCED; exit 99\n' > "$decoy/scripts/$lib"
|
||||
done
|
||||
cd "$(dirname "$SCRIPT")/.."
|
||||
run env CDPATH="$decoy" bash scripts/validate-provenance.sh "$root/.apm/agents/my-agent.agent.md"
|
||||
[ "$status" -eq 0 ]
|
||||
assert_output ""
|
||||
}
|
||||
@@ -1291,19 +1291,44 @@ EOF
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cycle 21 — G1: a bad target is a hard error, not a silent pass
|
||||
#
|
||||
# PORT NOTE RESOLVED (factory-audit merge): this block used to assert
|
||||
# "not a directory" for the nonexistent target while the agent suite asserted
|
||||
# "no such file" for its own, because each pre-merge script knew what shape it
|
||||
# was owed. The merged entry point classifies before it complains, so neither
|
||||
# old wording survives at THIS site and the two cases are no longer the same
|
||||
# case:
|
||||
#
|
||||
# * a path that exists as nothing this script recognises (and a path that does
|
||||
# not exist at all, when its name is not *.agent.md and its parent is not
|
||||
# agents/) is "matches neither a skill directory nor an agent file";
|
||||
# * a path that IS a directory but holds no SKILL.md gets its own, more
|
||||
# specific line naming the missing SKILL.md.
|
||||
#
|
||||
# The agent suite's "no such file" wording did NOT move — a nonexistent
|
||||
# *.agent.md path still classifies as an agent and is rejected by agent mode's
|
||||
# own precondition — so that side stays verbatim over there. Both wordings are
|
||||
# now asserted exactly rather than partially-matched, because the invariant both
|
||||
# sides pin is exit 2 with a message that names the path, never a silent exit 0
|
||||
# and never a bare code with no output.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "G1: a nonexistent directory is a hard error (exit 2), not a silent exit 0" {
|
||||
run bash "$SCRIPT" "$TMPDIR/does-not-exist"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "not a directory"
|
||||
assert_output --partial "Error: '$TMPDIR/does-not-exist' matches neither a skill directory nor an agent file."
|
||||
}
|
||||
|
||||
@test "G1: a directory with no SKILL.md is a hard error (exit 2), not a silent exit 0" {
|
||||
# Distinct from the case above on purpose: a directory IS half of a skill
|
||||
# target, so the detector can say something sharper than "neither shape" and
|
||||
# does. The exit code is unchanged from the pre-merge test, which already
|
||||
# pinned 2; only the wording moved, from the suite's own "not a skill
|
||||
# directory" precondition to the dispatcher's classification message.
|
||||
mkdir -p "$TMPDIR/not-a-skill/references"
|
||||
run bash "$SCRIPT" "$TMPDIR/not-a-skill"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "not a skill directory"
|
||||
assert_output --partial "Error: '$TMPDIR/not-a-skill' is a directory with no SKILL.md in it."
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -1496,12 +1521,27 @@ EOF
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cycle 27 — G8: usage and environment errors exit 2, never 1
|
||||
#
|
||||
# PORT NOTE RESOLVED (factory-audit merge): the two suites were written against
|
||||
# two separate scripts and disagreed about the no-argument wording — this file
|
||||
# asserted "skill-dir is required", the agent suite "agent-file is required".
|
||||
# Neither survives. One entry point takes both target shapes, so its no-argument
|
||||
# message names both, and both suites now assert that one sentence. Exit 2 and a
|
||||
# message (not a silent exit 1, and not a bare code with no output) is the
|
||||
# invariant both sides were really pinning, and it is still pinned from both.
|
||||
#
|
||||
# Note the asymmetry with scripts/validate.sh, which exits 1 on no argument:
|
||||
# only validate-provenance.sh carries the exit-2 usage tier, and this test is
|
||||
# what holds it there.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "G8: a missing argument exits 2, not 1" {
|
||||
run bash "$SCRIPT"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "skill-dir is required"
|
||||
assert_output --partial "Error: a skill directory or an agent file is required."
|
||||
# The usage block has to follow the error, or "required" names no shape the
|
||||
# caller can act on.
|
||||
assert_output --partial "Usage: validate-provenance.sh"
|
||||
}
|
||||
|
||||
@test "G8: an extra positional argument is rejected, not silently ignored" {
|
||||
@@ -1517,6 +1557,18 @@ EOF
|
||||
make_clean_skill "$skill"
|
||||
local stub="$TMPDIR/emptybin"
|
||||
mkdir -p "$stub"
|
||||
# dirname and basename are deliberately ABSENT, and that absence is load-
|
||||
# bearing. The invariant: nothing external is needed to reach the python3
|
||||
# preflight. The entry point resolves SCRIPT_DIR and classifies the target
|
||||
# with _kf_dirname/_kf_basename/_kf_parent_name — pure-bash replacements
|
||||
# that exist for precisely this reason — and the mode libraries build their
|
||||
# Python bodies with `read` heredocs. Widening this list to keep a test
|
||||
# green would silently retire that guarantee: on a PATH with neither
|
||||
# coreutils nor python3 the script would die at 127 naming `dirname`
|
||||
# instead of naming the dependency it actually needs, which is the failure
|
||||
# tests/test-adr0020-contract.sh assertion 2 exists to prevent. bash, cat
|
||||
# and sed stay so the stub is a DENY OF python3 ALONE rather than a test of
|
||||
# "no PATH at all" — the diagnostic under test here is the python3 one.
|
||||
for cmd in bash cat sed; do
|
||||
ln -s "$(command -v "$cmd")" "$stub/$cmd"
|
||||
done
|
||||
@@ -1900,3 +1952,111 @@ EOF
|
||||
assert_output --partial "Check 9 skipped — no base ref could be resolved"
|
||||
assert_output --partial "not-a-real-ref"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-detection — the merged provenance entry point classifies its own target
|
||||
#
|
||||
# NEW with the factory-audit merge, and new behaviour rather than a ported
|
||||
# case: scripts/validate-provenance.sh is now ONE entry point for both artifact
|
||||
# types and works out from the target which rubric to run. A DIRECTORY holding
|
||||
# SKILL.md is a skill; a FILE named *.agent.md, or sitting under .apm/agents/,
|
||||
# is an agent.
|
||||
#
|
||||
# The two rubrics are not near-copies of one another — the skill side has a
|
||||
# check 9 the agent side has none of, and reads sources.md from the skill's own
|
||||
# references/ rather than from the package root — so a misclassification is not
|
||||
# a near miss. It runs a set of checks that cannot apply and skips the set that
|
||||
# can, at exit 0.
|
||||
#
|
||||
# This file is the SKILL half plus the neither-shape rejection;
|
||||
# validate-provenance-agent.bats holds the agent half. Same script in both.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "auto-detect: a directory holding SKILL.md is checked in SKILL provenance mode" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_skill_with_source_keys "$skill"
|
||||
make_sources_md "$skill"
|
||||
# No commit_as_base, so check 9 has no repo root and announces the skip.
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
# Check 9 exists only on the skill side — the agent rubric has no check 9 at
|
||||
# any tier — so naming it is positive proof the SKILL rubric ran, rather
|
||||
# than merely that nothing crashed.
|
||||
assert_output --partial "Check 9 skipped — no repo root above the skill directory"
|
||||
}
|
||||
|
||||
@test "auto-detect: a SKILL.md FILE path is checked in SKILL provenance mode, not rejected" {
|
||||
# NEW with the merge and additive: pre-merge, handing the SKILL.md itself to
|
||||
# skill-audit's validate-provenance.sh hit the "not a directory" precondition
|
||||
# and died. It matters because pre-commit `files:` hooks match FILES — the
|
||||
# exported kyberforge-vale-audit-skill hook's regex is (^|/)SKILL\.md$ — so
|
||||
# every hook-driven invocation hands over a SKILL.md path, never its
|
||||
# directory. The entry point rewrites the token to the directory in place.
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_skill_with_source_keys "$skill"
|
||||
make_sources_md "$skill"
|
||||
run bash "$SCRIPT" "$skill/SKILL.md"
|
||||
assert_success
|
||||
# Same positive proof as the directory case: check 9 is skill-only.
|
||||
assert_output --partial "Check 9 skipped — no repo root above the skill directory"
|
||||
}
|
||||
|
||||
@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" {
|
||||
# The rewrite must be transparent, not merely non-fatal. If the two spellings
|
||||
# of the same target could disagree, a pre-commit run and a hand run would
|
||||
# report differently on one skill and neither would be obviously wrong.
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_skill_with_source_keys "$skill"
|
||||
make_sources_md "$skill"
|
||||
|
||||
run bash "$SCRIPT" "$skill"
|
||||
local dir_status="$status"
|
||||
local dir_output="$output"
|
||||
|
||||
run bash "$SCRIPT" "$skill/SKILL.md"
|
||||
[ "$status" -eq "$dir_status" ]
|
||||
[ "$output" = "$dir_output" ]
|
||||
# Guard against the comparison being satisfied by two empty runs: this
|
||||
# fixture has a check-9 INFO to print, so silence here means neither
|
||||
# spelling ran the rubric.
|
||||
refute_output ""
|
||||
}
|
||||
|
||||
@test "exit contract: SKILL mode exits 0 WITH output when the only findings are INFO" {
|
||||
# The two modes' exit contracts are DIFFERENT and the merge must not quietly
|
||||
# unify them. Skill mode is allowed to be chatty on a passing run: INFO
|
||||
# findings print and the status stays 0. Agent mode's opposite half — a
|
||||
# passing run prints nothing at all — is pinned from its own side in
|
||||
# validate-provenance-agent.bats.
|
||||
#
|
||||
# Both halves are asserted explicitly and on purpose. A merge that collapsed
|
||||
# one contract into the other would still satisfy whichever side was left
|
||||
# unasserted, so a single-sided pin would go green on exactly the defect it
|
||||
# was written to catch.
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_skill_with_source_keys "$skill"
|
||||
make_sources_md "$skill"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
[ "$status" -eq 0 ]
|
||||
refute_output ""
|
||||
assert_output --partial "INFO"
|
||||
refute_output --partial "FAIL"
|
||||
}
|
||||
|
||||
@test "auto-detect: a provenance target that is neither a skill directory nor an agent file FAILs, naming what it was handed" {
|
||||
# The detector must not guess. Guessing here is worse than in validate.sh:
|
||||
# this script's whole not-in-scope path is a SILENT exit 0, so a wrong guess
|
||||
# followed by "provenance does not apply at this scope" is indistinguishable
|
||||
# from a clean pass — which is the exact confusion the exit-2 tier above was
|
||||
# created to end.
|
||||
#
|
||||
# A plain .txt file is neither shape under any reading of the contract: not
|
||||
# a directory holding SKILL.md, not *.agent.md, not under .apm/agents/.
|
||||
local dir="$TMPDIR/neither"
|
||||
mkdir -p "$dir"
|
||||
echo "not an artifact of either kind" > "$dir/notes.txt"
|
||||
run bash "$SCRIPT" "$dir/notes.txt"
|
||||
assert_failure
|
||||
refute_output ""
|
||||
assert_output --partial "$dir/notes.txt"
|
||||
}
|
||||
109
plugins/kyberforge/.apm/skills/skill-audit/tests/validate.bats → plugins/kyberforge/.apm/skills/factory-audit/tests/validate-skill.bats
Executable file → Normal file
109
plugins/kyberforge/.apm/skills/skill-audit/tests/validate.bats → plugins/kyberforge/.apm/skills/factory-audit/tests/validate-skill.bats
Executable file → Normal file
@@ -728,7 +728,7 @@ PY
|
||||
# A skill carrying `disable-model-invocation: true` is absent from the
|
||||
# model-visible listing entirely: not preloaded, and the Skill tool refuses to
|
||||
# call it. Its description is never matched against user intent, so
|
||||
# references/description-quality.md Step 0 gives it ONE plain human-facing
|
||||
# references/skill-description-quality.md Step 0 gives it ONE plain human-facing
|
||||
# sentence — no trigger list, no boundary clause — and calls a
|
||||
# missing-boundary-clause finding on such a skill "a wrong finding, not a strict
|
||||
# one". Until this ran, nothing here knew the field existed, so the audit
|
||||
@@ -920,3 +920,110 @@ EOF
|
||||
refute_output --partial "UnicodeEncodeError"
|
||||
refute_output --partial "Traceback"
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auto-detection — the merged entry point classifies its own target
|
||||
#
|
||||
# NEW with the factory-audit merge, and new behaviour rather than a ported
|
||||
# case: scripts/validate.sh is now ONE entry point for both artifact types and
|
||||
# works out from the target which rubric to run. A DIRECTORY holding SKILL.md is
|
||||
# a skill; a FILE named *.agent.md, or sitting under .apm/agents/, is an agent.
|
||||
#
|
||||
# Before the merge each script was hard-wired to one type, so there was nothing
|
||||
# here that could be wrong. Now a misclassification is silent and total: the
|
||||
# wrong rubric runs end to end and reports the artifact clean against gates that
|
||||
# never applied to it, while every gate that did apply goes unrun. Nothing else
|
||||
# in this suite would notice, because every other fixture is a skill directory
|
||||
# and would be classified correctly even by a detector that always guessed
|
||||
# "skill".
|
||||
#
|
||||
# The agent half of the same contract is pinned from the other side, in
|
||||
# validate-agent.bats — same script, same detector, agent-shaped fixtures.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@test "auto-detect: a directory holding SKILL.md is audited in SKILL mode" {
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
run bash "$SCRIPT" "$skill"
|
||||
assert_success
|
||||
# ADR-0022's metadata.version is mandatory for skills and has no agent
|
||||
# analogue whatsoever, so this line is positive proof the SKILL rubric ran —
|
||||
# not merely that the run survived. A bare assert_success would be satisfied
|
||||
# by a detector that classified this as an agent and found nothing to say.
|
||||
assert_output --partial "metadata.version present"
|
||||
# 'counterpart' is agent-mode vocabulary (the CC/Copilot pair check). A skill
|
||||
# directory must never reach a check that has a concept of a counterpart.
|
||||
refute_output --partial "counterpart"
|
||||
}
|
||||
|
||||
@test "auto-detect: a SKILL.md FILE path is audited in SKILL mode, not rejected" {
|
||||
# NEW with the merge and additive rather than ported: pre-merge, handing the
|
||||
# SKILL.md itself to skill-audit's validate.sh hit the directory precondition
|
||||
# and gave a useless exit 1. It matters because pre-commit `files:` hooks
|
||||
# match FILES — the exported kyberforge-vale-audit-skill hook's regex is
|
||||
# (^|/)SKILL\.md$ — so every hook-driven invocation hands over a SKILL.md
|
||||
# path, never the directory above it. The entry point resolves the file to
|
||||
# its directory before dispatching.
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
run bash "$SCRIPT" "$skill/SKILL.md"
|
||||
assert_success
|
||||
# Same positive proof as the directory case: metadata.version is skill-only
|
||||
# and has no agent analogue, so this line says the SKILL rubric ran.
|
||||
assert_output --partial "metadata.version present"
|
||||
refute_output --partial "counterpart"
|
||||
}
|
||||
|
||||
@test "auto-detect: a SKILL.md FILE path and its directory produce the same verdict" {
|
||||
# The resolution must be transparent, not merely non-fatal. If the two
|
||||
# spellings of one target could disagree, a pre-commit run and a hand run
|
||||
# would report differently on the same skill and neither would look wrong.
|
||||
# The name check is the sharp end: it compares `name` against the DIRECTORY
|
||||
# basename, so a target left as the file would compare against "SKILL.md".
|
||||
local skill="$TMPDIR/my-skill"
|
||||
make_valid_skill "$skill"
|
||||
|
||||
run bash "$SCRIPT" "$skill"
|
||||
local dir_status="$status"
|
||||
local dir_output="$output"
|
||||
|
||||
run bash "$SCRIPT" "$skill/SKILL.md"
|
||||
[ "$status" -eq "$dir_status" ]
|
||||
[ "$output" = "$dir_output" ]
|
||||
assert_output --partial "name 'my-skill' matches directory 'my-skill'"
|
||||
}
|
||||
|
||||
@test "auto-detect: a target that is neither a skill directory nor an agent file FAILs, naming what it was handed" {
|
||||
# The detector must not guess. Falling back to either rubric on an
|
||||
# unclassifiable target yields a verdict about rules that were never meant
|
||||
# to apply, and exiting 0 publishes that verdict as a pass — the worst of
|
||||
# the three possible outcomes, because it is the silent one.
|
||||
#
|
||||
# A plain .txt file is neither shape under any reading of the contract: not
|
||||
# a directory holding SKILL.md, not *.agent.md, not under .apm/agents/. The
|
||||
# directory flavour of the same mismatch — a directory with no SKILL.md — is
|
||||
# pinned separately by "fails when SKILL.md is missing" above.
|
||||
local dir="$TMPDIR/neither"
|
||||
mkdir -p "$dir"
|
||||
echo "not an artifact of either kind" > "$dir/notes.txt"
|
||||
run bash "$SCRIPT" "$dir/notes.txt"
|
||||
assert_failure
|
||||
# Non-zero is necessary but not sufficient: a non-zero exit with nothing on
|
||||
# stdout is indistinguishable from a clean-but-failing run, which is the
|
||||
# confusion the exit-2 tier in the provenance suites was created to end.
|
||||
refute_output ""
|
||||
assert_output --partial "$dir/notes.txt"
|
||||
}
|
||||
|
||||
@test "entry point: a missing resolver library in skill mode exits 2 naming it, never exit 1" {
|
||||
# validate-agent.bats pins this for agent mode; each mode sources its own
|
||||
# libraries, so each mode's guard is pinned separately.
|
||||
make_valid_skill "$TMPDIR/my-skill"
|
||||
local lone="$TMPDIR/lone-scripts"
|
||||
cp -R "$(dirname "$SCRIPT")" "$lone"
|
||||
rm "$lone/lib-boundary-resolver.sh"
|
||||
run bash "$lone/validate.sh" "$TMPDIR/my-skill"
|
||||
[ "$status" -eq 2 ]
|
||||
assert_output --partial "required library '$lone/lib-boundary-resolver.sh' is missing or unreadable"
|
||||
refute_output --partial "No such file or directory"
|
||||
}
|
||||
@@ -18,7 +18,7 @@ metadata:
|
||||
|
||||
## Gotchas
|
||||
|
||||
- forge is an optional guided entry point, not a gate — `skill-author`, `skill-audit`, `agent-author`, `agent-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one.
|
||||
- forge is an optional guided entry point, not a gate — `skill-author`, `agent-author`, `factory-audit` and `apm-workflow` all stay directly invokable, and forge never intercepts a direct call to one.
|
||||
- Claude Code's skill-level `context: fork` frontmatter field and the `/fork` subagent command are opposites despite the shared word: `context: fork` isolates (fresh context, no parent access), while `/fork` inherits the full conversation. The route reference each classification loads spends that distinction: `references/author-routes.md` chooses between the two, `references/apm-routes.md` rules the fork out.
|
||||
|
||||
## Step 1 — Grill the intent
|
||||
|
||||
@@ -7,8 +7,8 @@ source_keys:
|
||||
|
||||
Reached from `SKILL.md` Step 2 when the classified artifact is a skill or an agent/subagent
|
||||
definition. Route a skill to `skill-author` and an agent to `agent-author`. The two branches
|
||||
differ on one axis only — which audit skill verifies the result — and everything below applies to
|
||||
both.
|
||||
differ on the author skill only — both verify the result with `factory-audit`, which detects the
|
||||
artifact type itself — and everything below applies to both.
|
||||
|
||||
## Choose fork or inline
|
||||
|
||||
@@ -26,8 +26,8 @@ Fall back to an **inline invocation** — same conversation, no subagent — whe
|
||||
## Two-tier verification
|
||||
|
||||
Both author skills already close out with their own inline audit, in the same context as the
|
||||
authoring work: `skill-author` runs `/skill-audit`, `agent-author` invokes
|
||||
`agent-audit`. That is tier one, and forge does not change it.
|
||||
authoring work: `skill-author` and `agent-author` each invoke `factory-audit` on what they wrote.
|
||||
That is tier one, and forge does not change it.
|
||||
|
||||
Tier two belongs to forge. Once the author skill's run has finished, spin up a separate
|
||||
**clean-context subagent** — fresh, not forked, no inherited context — to independently re-run the
|
||||
|
||||
@@ -1,87 +0,0 @@
|
||||
---
|
||||
name: skill-audit
|
||||
description: >
|
||||
Use when the user wants a skill directory audited against the agentskills.io
|
||||
spec — "audit this skill", "review my SKILL.md", "is this ready to ship" — or
|
||||
after hand-editing a skill outside skill-author. Not applying fixes ->
|
||||
skill-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
version: "1.0.2"
|
||||
category: factory
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A skill carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- `validate.sh` reports two independent length families: the 500-line / 2,770-word pair counts the whole file for spec conformance, while the 250/400-character and 600/900-word pair is the house context budget and its word half counts the **body only**. A skill can sit inside one and fail the other — report them separately.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
bash scripts/validate-provenance.sh <skill-dir>
|
||||
bash scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation.
|
||||
|
||||
Read `references/validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback.
|
||||
|
||||
`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — a usage or environment error, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/description-quality.md` |
|
||||
| body-discipline | `references/body-discipline.md` |
|
||||
| patterns | `references/patterns.md` |
|
||||
| file-structure, internal-consistency | `references/file-structure.md` |
|
||||
| formatting, scripts | `references/formatting-and-scripts.md` |
|
||||
|
||||
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with a coverage line naming every dimension checked:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line, `Run skill-author to address findings.`, whenever there is at least one finding. Do not apply fixes — report and propose only.
|
||||
@@ -1,4 +0,0 @@
|
||||
StylesPath = styles
|
||||
|
||||
[**/SKILL.md]
|
||||
BasedOnStyles = Kyberforge
|
||||
@@ -1,13 +0,0 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -1,7 +0,0 @@
|
||||
extends: existence
|
||||
message: "Description opens with '%s' — use an imperative 'Use when...' opener instead"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This\b'
|
||||
@@ -1,7 +0,0 @@
|
||||
extends: existence
|
||||
message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead"
|
||||
level: error
|
||||
scope: text
|
||||
ignorecase: true
|
||||
raw:
|
||||
- 'see references?/? for (more )?(info|information|details)\b'
|
||||
@@ -1,7 +0,0 @@
|
||||
extends: existence
|
||||
message: "Don't start a sentence with '%s' — name the subject directly"
|
||||
level: error
|
||||
scope: sentence
|
||||
ignorecase: false
|
||||
raw:
|
||||
- '^There\s(is|are)\b'
|
||||
@@ -1,10 +0,0 @@
|
||||
extends: existence
|
||||
message: "Vague capability wording: '%s' — state the capability precisely instead"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- helps with
|
||||
- utilize
|
||||
- assists with
|
||||
- used for
|
||||
@@ -1,59 +0,0 @@
|
||||
# Sources
|
||||
|
||||
<!-- agentskills.io/llms.txt was used for initial source discovery and is not listed below; it contributed no skill file content directly. -->
|
||||
|
||||
## agentskills-home
|
||||
|
||||
- **URL:** https://agentskills.io/home.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
|
||||
- **Contributing files:** SKILL.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-spec
|
||||
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/description-quality.md, references/patterns.md, references/file-structure.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/body-discipline.md, references/patterns.md, references/finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
|
||||
- **Contributing files:** SKILL.md, references/description-quality.md, references/finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-evaluating-skills
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
|
||||
- **Contributing files:** (none — eval workflow not directly informing audit dimensions)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-using-scripts
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md, references/formatting-and-scripts.md, references/finding-criteria.md, references/validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/quickstart.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
|
||||
- **Contributing files:** (none — creation guide not directly informing audit criteria)
|
||||
- **Status:** `extracted`
|
||||
@@ -1,526 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Works around a Vale limitation: the `text.frontmatter.description` NLP scope
|
||||
# silently stops matching once the `description:` value spans 2+ physical lines
|
||||
# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block
|
||||
# scalar (the style used by most skills/agents in this repo), a plain scalar
|
||||
# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped
|
||||
# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed
|
||||
# value keeps exactly the line breaks the source has, and vale matches it fine
|
||||
# (verified against vale 3.15.2), so literal blocks are deliberately left alone.
|
||||
# This script flattens an affected description to a one-line scalar in a scratch
|
||||
# copy — or, for the rare value no inline scalar can spell out verbatim, to a
|
||||
# `|-` literal block with a single content line, which vale matches just as well
|
||||
# (padding with blank lines so every other line number is unchanged), then
|
||||
# runs the real `vale` binary against the copies. Drop-in replacement for calling
|
||||
# `vale` directly: same args, same exit code, bar the two documented divergences
|
||||
# below.
|
||||
#
|
||||
# "Same args" means relative paths — path arguments and the values of the
|
||||
# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against
|
||||
# the caller's current directory, exactly as bare `vale` resolves them. The flag
|
||||
# values are rewritten to absolute form because the run ends up `cd`'d into the
|
||||
# scratch mirror, where a relative one would no longer resolve. (An earlier
|
||||
# version resolved path arguments against the repo root, an invented convention
|
||||
# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse,
|
||||
# silently dropped file arguments that didn't happen to resolve from the repo
|
||||
# root — skipping the flattening this script exists for.)
|
||||
#
|
||||
# Divergence 1: with no `--config` at all, this script's own sibling
|
||||
# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit
|
||||
# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in
|
||||
# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and
|
||||
# hard-fail (E100) for every external consumer. The manifest therefore passes the
|
||||
# script alone, and an explicit `--config` from any other caller still wins.
|
||||
#
|
||||
# Divergence 2: a path-shaped argument that does not exist is a hard error
|
||||
# (exit 2). Bare vale drops it, falls back to reading stdin, and prints
|
||||
# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable
|
||||
# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather
|
||||
# than clean, and `in stdin` does not match that guard, so the silent form would
|
||||
# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way
|
||||
# to keep that guard honest. Linting prose piped on stdin is therefore
|
||||
# unsupported here — it already was, since the no-path handoff closes stdin so
|
||||
# vale can't block on a pipe that will never carry content.
|
||||
#
|
||||
# Vale prints each path exactly as it was handed to it, so the scratch tree
|
||||
# mirrors the caller's absolute cwd: a relative path argument is passed through
|
||||
# verbatim and resolves to its flattened copy, keeping the report byte-identical
|
||||
# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that
|
||||
# property. Only an absolute path outside the cwd is rewritten to its scratch
|
||||
# copy and so reports a scratch path — unavoidable, since a file can only be
|
||||
# read from where it actually is.
|
||||
|
||||
cwd="$(pwd -P)"
|
||||
|
||||
# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 —
|
||||
# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"`
|
||||
# on an empty array as an unbound variable under `set -u`. No expansion site is
|
||||
# reachable while empty on today's control flow, so this is insurance against a
|
||||
# later edit breaking that invariant, not a live fix.
|
||||
vale_args=()
|
||||
path_args=()
|
||||
pending_flag=""
|
||||
config_given=false
|
||||
|
||||
# `--output` takes either one of vale's built-in style names or a template file
|
||||
# path. Only the file form needs absolutizing, and the built-in names have to be
|
||||
# excluded by name *before* the existence test below: a file or directory
|
||||
# literally called `line` in the caller's cwd would otherwise rewrite the
|
||||
# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template]
|
||||
# Runtime error`) where bare vale just uses the built-in. `--path` has no such
|
||||
# names — it is always a path — so the check is keyed on the flag too.
|
||||
is_builtin_output() {
|
||||
case "$2" in
|
||||
line|JSON|CLI) [[ "$1" == "--output" ]] ;;
|
||||
*) false ;;
|
||||
esac
|
||||
}
|
||||
# Absolutizes a `--config` value against the caller's cwd. Shared by both
|
||||
# argument forms below — separated (`--config X`) and joined (`--config=X`)
|
||||
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
|
||||
# one place instead of being duplicated per form.
|
||||
abs_config_value() {
|
||||
if [[ "$1" == /* ]]; then
|
||||
printf '%s' "$1"
|
||||
else
|
||||
printf '%s' "$cwd/$1"
|
||||
fi
|
||||
}
|
||||
for arg in "$@"; do
|
||||
if [[ -n "$pending_flag" ]]; then
|
||||
# Value of a separated two-argv flag. It is never a lint target, however
|
||||
# file-like it looks. The run ends up `cd`'d into the scratch mirror, so a
|
||||
# value naming a file has to be absolutized here or it stops resolving.
|
||||
case "$pending_flag" in
|
||||
--config)
|
||||
# Always a path, and required to exist.
|
||||
vale_args+=("$(abs_config_value "$arg")")
|
||||
;;
|
||||
--output|--path)
|
||||
# See `is_builtin_output` above for why the built-in `--output` names
|
||||
# are excluded first. Anything that names nothing is passed through and
|
||||
# left for vale to interpret.
|
||||
if is_builtin_output "$pending_flag" "$arg"; then
|
||||
vale_args+=("$arg")
|
||||
elif [[ "$arg" != /* && -e "$arg" ]]; then
|
||||
vale_args+=("$cwd/$arg")
|
||||
else
|
||||
vale_args+=("$arg")
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
vale_args+=("$arg")
|
||||
;;
|
||||
esac
|
||||
pending_flag=""
|
||||
continue
|
||||
fi
|
||||
case "$arg" in
|
||||
--config)
|
||||
vale_args+=("$arg")
|
||||
pending_flag="$arg"
|
||||
config_given=true
|
||||
continue
|
||||
;;
|
||||
--config=*)
|
||||
vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
|
||||
config_given=true
|
||||
continue
|
||||
;;
|
||||
# Same cwd-relative resolution for the `--flag=value` spelling of the two
|
||||
# other path-valued flags.
|
||||
--output=*|--path=*)
|
||||
flag_val="${arg#*=}"
|
||||
if is_builtin_output "${arg%%=*}" "$flag_val"; then
|
||||
vale_args+=("$arg")
|
||||
elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then
|
||||
vale_args+=("${arg%%=*}=$cwd/$flag_val")
|
||||
else
|
||||
vale_args+=("$arg")
|
||||
fi
|
||||
continue
|
||||
;;
|
||||
# Vale's remaining value-taking flags, per `vale --help` (3.x). In the
|
||||
# separated two-argv form the value must not be classified as a lint target
|
||||
# — `--output tmpl.tmpl` names a real template file, and treating it as
|
||||
# input both lints the template and reorders argv so vale sees
|
||||
# `--output --no-wrap`. The `--flag=value` form needs no entry here: it
|
||||
# starts with `-` and falls through to vale untouched. A value flag added by
|
||||
# some future vale release is simply absent from this list and lands back on
|
||||
# today's behaviour, so this list going stale is never worse than not having
|
||||
# it.
|
||||
--ext|--filter|--glob|--minAlertLevel|--output|--path)
|
||||
vale_args+=("$arg")
|
||||
pending_flag="$arg"
|
||||
continue
|
||||
;;
|
||||
# Vale's subcommands are bare words that name no file, so they would trip
|
||||
# the not-found error below. A lint target literally named `sync` (no
|
||||
# extension, no slash) is misread as the subcommand — accepted, because the
|
||||
# alternative is failing every `vale-wrap.sh ls-config`.
|
||||
ls-config|ls-dirs|ls-metrics|ls-vars|sync)
|
||||
vale_args+=("$arg")
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
if [[ "$arg" == -* ]]; then
|
||||
vale_args+=("$arg")
|
||||
continue
|
||||
fi
|
||||
# Everything left is a lint target: `vale [options] [input...]` has no third
|
||||
# kind of argument. See divergence 2 above for why a missing one is fatal here.
|
||||
if [[ ! -e "$arg" ]]; then
|
||||
echo "vale-wrap.sh: no such file or directory: $arg" >&2
|
||||
exit 2
|
||||
fi
|
||||
# An absolute path inside the caller's cwd is relativized so the report cites
|
||||
# a path that resolves against the real tree. Left absolute, it would be
|
||||
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
|
||||
# path to a file that is deleted on exit, which reads as a bug in any report
|
||||
# quoting it. Absolute paths outside the cwd have no relative form and keep
|
||||
# the scratch-path behaviour documented above.
|
||||
if [[ "$arg" == "$cwd"/* ]]; then
|
||||
path_args+=("${arg#"$cwd"/}")
|
||||
else
|
||||
path_args+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "$config_given" == false ]]; then
|
||||
vale_args+=(--config "$(cd "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" && pwd)/.vale.ini")
|
||||
fi
|
||||
|
||||
if [[ ${#path_args[@]} -eq 0 ]]; then
|
||||
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
|
||||
# block waiting on a pipe that will never carry content.
|
||||
exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null
|
||||
fi
|
||||
|
||||
# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing)
|
||||
# is a GNU extension the BSD realpath on macOS doesn't have — and every dest
|
||||
# below is a path that doesn't exist yet. python3 is already a hard dependency.
|
||||
abspath() {
|
||||
python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1"
|
||||
}
|
||||
|
||||
flatten() {
|
||||
# Two call shapes: `flatten src dest` (dest already resolved and inside the
|
||||
# scratch tree — the per-markdown-file calls in the directory branch below)
|
||||
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
|
||||
# branch further down) additionally resolves `raw_dest` the way a separate
|
||||
# `abspath` call used to, applies the same sandbox-escape guard, and prints
|
||||
# the resolved path — folding two python3 spawns per file into one.
|
||||
python3 - "$@" <<'PYTHON'
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
src, dest_input = sys.argv[1], sys.argv[2]
|
||||
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
|
||||
|
||||
if tmpdir is None:
|
||||
dest = dest_input
|
||||
else:
|
||||
dest = os.path.abspath(dest_input)
|
||||
if not dest.startswith(tmpdir + os.sep):
|
||||
print(
|
||||
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
|
||||
f"land outside {tmpdir}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||
|
||||
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
|
||||
# a byte-for-byte round trip instead of aborting the whole run on a decode error.
|
||||
with open(src, encoding='utf-8', errors='surrogateescape') as fh:
|
||||
content = fh.read()
|
||||
|
||||
# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\<newline>` is handled
|
||||
# separately in unescape_double because it also swallows the next indentation.
|
||||
DQ_ESCAPES = {
|
||||
'0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n',
|
||||
'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"',
|
||||
'/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028',
|
||||
'P': '\u2029',
|
||||
}
|
||||
|
||||
# First characters that make a plain (unquoted) scalar mean something other than
|
||||
# text: YAML's c-indicator set.
|
||||
PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`'
|
||||
|
||||
|
||||
def unescape_double(text):
|
||||
"""Decode a double-quoted YAML scalar's body to the string YAML parses."""
|
||||
out = []
|
||||
i = 0
|
||||
while i < len(text):
|
||||
char = text[i]
|
||||
if char != '\\':
|
||||
out.append(char)
|
||||
i += 1
|
||||
continue
|
||||
i += 1
|
||||
if i >= len(text):
|
||||
break
|
||||
esc = text[i]
|
||||
if esc == '\n':
|
||||
i += 1
|
||||
while i < len(text) and text[i] in ' \t':
|
||||
i += 1
|
||||
continue
|
||||
if esc in 'xuU':
|
||||
width = {'x': 2, 'u': 4, 'U': 8}[esc]
|
||||
digits = text[i + 1:i + 1 + width]
|
||||
if len(digits) == width:
|
||||
try:
|
||||
out.append(chr(int(digits, 16)))
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
i += 1 + width
|
||||
continue
|
||||
out.append(DQ_ESCAPES.get(esc, esc))
|
||||
i += 1
|
||||
return ''.join(out)
|
||||
|
||||
|
||||
def close_quote(text, quote):
|
||||
"""Index of the closing `quote` in `text`, which starts just past the
|
||||
opening one. None while the scalar is still unterminated."""
|
||||
i = 0
|
||||
while i < len(text):
|
||||
char = text[i]
|
||||
if quote == '"' and char == '\\':
|
||||
i += 2
|
||||
continue
|
||||
if char == quote:
|
||||
if quote == "'" and text[i + 1:i + 2] == "'":
|
||||
i += 2
|
||||
continue
|
||||
return i
|
||||
i += 1
|
||||
return None
|
||||
|
||||
|
||||
def continuation_lines(rest):
|
||||
"""Yield the physical lines of `rest` that continue the value started on the
|
||||
`description:` line. Indentation-based and blank-line-tolerant, per YAML:
|
||||
a blank line (any amount of whitespace) always stays inside; the indent is
|
||||
set by the first content line; the value ends at the first line indented
|
||||
less than that, at any line flush with the key (that is the next mapping
|
||||
key, not a continuation), or at EOF."""
|
||||
indent = None
|
||||
for line in rest.splitlines(keepends=True):
|
||||
text = line.rstrip('\n')
|
||||
if text.strip() == '':
|
||||
yield line
|
||||
continue
|
||||
line_indent = len(text) - len(text.lstrip(' \t'))
|
||||
if line_indent == 0:
|
||||
return
|
||||
if indent is None:
|
||||
indent = line_indent
|
||||
elif line_indent < indent:
|
||||
return
|
||||
yield line
|
||||
|
||||
|
||||
def emit(value):
|
||||
"""Render `value` as a YAML scalar whose source text spells the value out
|
||||
verbatim. Vale locates the description by matching the parsed value back
|
||||
against the source, so a scalar carrying any escape — `''` in a
|
||||
single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the
|
||||
whole `text.frontmatter.description` scope vanish, the same failure this
|
||||
script exists to work around. Verbatim forms only, therefore, tried in
|
||||
descending order of fidelity. The first three occupy one physical line; the
|
||||
`|-` fallback occupies two, which the caller accounts for when padding."""
|
||||
if (value
|
||||
and value[0] not in PLAIN_UNSAFE_FIRST
|
||||
and ': ' not in value
|
||||
and not value.endswith(':')
|
||||
and ' #' not in value):
|
||||
return value # plain: nothing needs escaping at all
|
||||
if "'" not in value:
|
||||
return "'" + value + "'" # single-quoted: only `'` would escape
|
||||
if '"' not in value and '\\' not in value:
|
||||
return '"' + value + '"' # double-quoted: only `"`/`\` would
|
||||
# Last resort: the value needs quoting AND holds an apostrophe AND a double
|
||||
# quote or backslash, so no *inline* scalar can carry it verbatim. A `|-`
|
||||
# literal block can — a block scalar's body has no escape syntax at all, so
|
||||
# `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches
|
||||
# the description scope against it (the header above says the same of the
|
||||
# `|` blocks this script deliberately leaves alone; verified against vale
|
||||
# 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed
|
||||
# value is exactly `value` with no trailing newline.
|
||||
return '|-\n ' + value
|
||||
|
||||
|
||||
fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL)
|
||||
if fm_match:
|
||||
fm = fm_match.group(2)
|
||||
header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE)
|
||||
else:
|
||||
header_m = None
|
||||
|
||||
if header_m:
|
||||
head_start = header_m.start()
|
||||
value_start = header_m.end()
|
||||
header_end = fm.find('\n', value_start)
|
||||
header_end = len(fm) if header_end == -1 else header_end
|
||||
first = fm[value_start:header_end]
|
||||
body_start = header_end + 1
|
||||
indicator = first.rstrip()
|
||||
|
||||
block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator)
|
||||
if block_m and block_m.group(1) == '|':
|
||||
kind = None # literal blocks keep their line breaks; vale is fine
|
||||
elif block_m:
|
||||
kind = 'block' # folded (`>`): the value starts on the next line
|
||||
elif indicator == '':
|
||||
kind = 'block' # bare `description:`: a plain scalar on later lines
|
||||
elif first[:1] == '"':
|
||||
kind = 'double'
|
||||
elif first[:1] == "'":
|
||||
kind = 'single'
|
||||
elif first[:1] in '#&*!':
|
||||
kind = None # comment, anchor, alias or tag — not a plain scalar
|
||||
else:
|
||||
kind = 'plain'
|
||||
|
||||
text = ''
|
||||
value_end = value_start
|
||||
value_lines = 0
|
||||
if kind in ('block', 'plain'):
|
||||
body = ''.join(continuation_lines(fm[body_start:]))
|
||||
value_end = body_start + len(body)
|
||||
if kind == 'block':
|
||||
text = body
|
||||
value_lines = body.count('\n')
|
||||
else:
|
||||
text = fm[value_start:value_end]
|
||||
value_lines = 1 + body.count('\n')
|
||||
if ' #' in text or text.lstrip().startswith('#'):
|
||||
# A `#` opens a comment inside a plain scalar. Folding it in
|
||||
# would lint text YAML never treats as part of the value, so
|
||||
# leave the file alone rather than lint the wrong string.
|
||||
kind = None
|
||||
elif kind in ('double', 'single'):
|
||||
quote = '"' if kind == 'double' else "'"
|
||||
inner_start = value_start + 1
|
||||
acc = fm[inner_start:body_start]
|
||||
idx = close_quote(acc, quote)
|
||||
lines = continuation_lines(fm[body_start:])
|
||||
while idx is None:
|
||||
try:
|
||||
acc += next(lines)
|
||||
except StopIteration:
|
||||
break
|
||||
idx = close_quote(acc, quote)
|
||||
if idx is None:
|
||||
kind = None # unterminated quote: invalid YAML, leave it to vale
|
||||
else:
|
||||
inner = acc[:idx]
|
||||
value_end = inner_start + idx + 1
|
||||
text = unescape_double(inner) if quote == '"' else inner.replace("''", "'")
|
||||
value_lines = 1 + inner.count('\n')
|
||||
|
||||
flat = re.sub(r'\s+', ' ', text).strip()
|
||||
if kind and flat and value_lines >= 2:
|
||||
# `value_end` can land mid-line, just past a closing quote, so extend to
|
||||
# the end of that physical line and carry whatever follows (a trailing
|
||||
# comment) across unchanged.
|
||||
if value_end > 0 and fm[value_end - 1] == '\n':
|
||||
span_end = value_end
|
||||
trailer = ''
|
||||
else:
|
||||
newline = fm.find('\n', value_end)
|
||||
span_end = len(fm) if newline == -1 else newline + 1
|
||||
trailer = fm[value_end:span_end].rstrip('\n')
|
||||
scalar = emit(flat)
|
||||
# A trailing comment carried across from the original line stays on the
|
||||
# `description:` line itself: after a block scalar's `|-` header it is
|
||||
# still a comment, but inside the block body it would become part of the
|
||||
# value.
|
||||
head, newline_sep, block_body = scalar.partition('\n')
|
||||
# The replacement displaces the whole span, so the blank-line pad makes
|
||||
# up the difference between the lines it displaced and the lines it
|
||||
# occupies — every later line number is unchanged. That is one line for
|
||||
# the three inline forms and two for the `|-` block; the span itself is
|
||||
# at least two lines here (`value_lines >= 2` is a precondition), so the
|
||||
# pad count never goes negative.
|
||||
pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n'))
|
||||
new_fm = (fm[:head_start] + 'description: ' + head + trailer
|
||||
+ newline_sep + block_body + '\n' + pad + fm[span_end:])
|
||||
content = (fm_match.group(1) + new_fm + fm_match.group(3)
|
||||
+ content[fm_match.end():])
|
||||
|
||||
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
|
||||
fh.write(content)
|
||||
|
||||
if tmpdir is not None:
|
||||
print(dest)
|
||||
PYTHON
|
||||
}
|
||||
|
||||
tmpdir="$(cd "$(mktemp -d)" && pwd -P)"
|
||||
trap 'rm -rf "$tmpdir"' EXIT
|
||||
|
||||
# Mirror of the caller's cwd inside the scratch tree; relative path arguments
|
||||
# are resolved from here.
|
||||
mirror="$tmpdir$cwd"
|
||||
mkdir -p "$mirror"
|
||||
|
||||
argv_paths=()
|
||||
for arg in ${path_args[@]+"${path_args[@]}"}; do
|
||||
if [[ "$arg" == /* ]]; then
|
||||
raw_dest="$tmpdir$arg"
|
||||
else
|
||||
raw_dest="$mirror/$arg"
|
||||
fi
|
||||
if [[ -d "$arg" ]]; then
|
||||
dest="$(abspath "$raw_dest")"
|
||||
# A path argument with enough leading `..` to climb past the mirror root would
|
||||
# write outside the scratch dir. The real filesystem clamps such a path at
|
||||
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
|
||||
case "$dest" in
|
||||
"$tmpdir"/*) ;;
|
||||
*)
|
||||
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
mkdir -p "$(dirname "$dest")"
|
||||
# A directory is mirrored whole — vale applies its own format filtering to
|
||||
# the tree, so any file dropped here would be silently unlinted — and then
|
||||
# every markdown file in the copy is flattened in place. `.git` is pruned:
|
||||
# vale never lints it and copying it can dwarf the rest of the tree.
|
||||
# `find -L` follows symlinks because vale does: it lints both a symlinked
|
||||
# file and a file under a symlinked directory, and a bare `-type f` walk
|
||||
# would report "0 files" where bare vale reports one. (A symlink loop makes
|
||||
# `find` warn on stderr and carry on, which is also what vale does.) The
|
||||
# second walk needs no `-L`: the mirror is all real files by construction.
|
||||
mkdir -p "$dest"
|
||||
while IFS= read -r -d '' rel; do
|
||||
mkdir -p "$dest/$(dirname "$rel")"
|
||||
cp "$arg/$rel" "$dest/$rel"
|
||||
done < <(cd "$arg" && find -L . -name .git -prune -o -type f -print0)
|
||||
while IFS= read -r -d '' md; do
|
||||
flatten "$md" "$md"
|
||||
done < <(find "$dest" -type f -name '*.md' -print0)
|
||||
else
|
||||
# `abspath` + `flatten` folded into one python3 process — see the comment
|
||||
# atop `flatten` above.
|
||||
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
|
||||
fi
|
||||
if [[ "$arg" == /* ]]; then
|
||||
argv_paths+=("$dest")
|
||||
else
|
||||
argv_paths+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
cd "$mirror"
|
||||
vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"}
|
||||
@@ -1,29 +0,0 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Tests require [bats-support](https://github.com/bats-core/bats-support) and
|
||||
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
|
||||
helpers from the repo root's `tests/test_helper/`.
|
||||
|
||||
From the repo root:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests for this skill (from the repo root):
|
||||
|
||||
```bash
|
||||
bats plugins/kyberforge/.apm/skills/skill-audit/tests/
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `validate.bats` | Bats test suite for `scripts/validate.sh` |
|
||||
| `validate-provenance.bats` | Bats test suite for `scripts/validate-provenance.sh` |
|
||||
@@ -3,7 +3,7 @@ name: skill-author
|
||||
description: >
|
||||
Use when the user wants to create a new skill from scratch, or apply audit
|
||||
findings, grill output, eval results, or inline feedback to an existing one.
|
||||
Not read-only review -> `skill-audit`. Not agent files -> `agent-author`.
|
||||
Not read-only review -> `factory-audit`. Not agent files -> `agent-author`.
|
||||
allowed-tools: Bash Read Write Edit
|
||||
metadata:
|
||||
version: "1.0.2"
|
||||
@@ -21,7 +21,7 @@ metadata:
|
||||
## Gotchas
|
||||
|
||||
- The word gates are two measurements, not two tiers of one rule: the 2,770-word / 500-line spec backstop counts the whole file, Step 3's gate the body alone. Never unify them.
|
||||
- Never spawn a subagent to audit or recheck your own work — run `/skill-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft.
|
||||
- Never spawn a subagent to audit or recheck your own work — run `/factory-audit` inline, in the same context as the edits. Clean-context recheck belongs to `/forge`'s outer loop, and a self-spawned subagent's worktree can be torn down by concurrent cleanup, destroying an uncommitted draft.
|
||||
- Do not create new scripts unless a signal explicitly calls for it. Writing one from scratch requires out-of-scope transcript analysis — flag the opportunity as a suggestion instead.
|
||||
|
||||
## Step 1 — Dispatch
|
||||
@@ -32,7 +32,7 @@ metadata:
|
||||
| Directory exists, at least one improvement signal present | Improve | `references/improve.md` |
|
||||
| Directory exists, no signals | Stop and ask | — |
|
||||
|
||||
Signals: grill output, `/skill-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply.
|
||||
Signals: grill output, `/factory-audit` findings, inline feedback, eval results, session context describing what went wrong. With none, ask whether the user meant to create a new skill or has feedback to apply.
|
||||
|
||||
Read only the reference matching the resolved flow — each is self-contained. If the target sits inside a git worktree, capture `rtk git log --oneline -1` before touching the filesystem; Step 4 needs it.
|
||||
|
||||
@@ -47,7 +47,7 @@ Decide before writing any description: model-invoked or hand-invoked?
|
||||
|
||||
Before writing or editing a description, or restructuring a body, read `references/contract.md` — the banned-content list, boundary form, include/exclude rubric and body patterns.
|
||||
|
||||
Gates `/skill-audit` enforces in both flows:
|
||||
Gates `/factory-audit` enforces in both flows:
|
||||
|
||||
- **Description** — a trigger clause, at most one capability clause, and a boundary clause shaped `Not <thing> -> <skill-name>` whose target resolves to a real skill or agent. 250 characters SUGGESTION, 400 FAIL, value only.
|
||||
- **Body** — decision procedure only: ordered steps, branches, gates, and which reference to load when. 600 words SUGGESTION, 900 FAIL, body only. At two or more mutually exclusive flows a dispatch table is mandatory and each flow gets its own self-contained `references/` file.
|
||||
@@ -55,7 +55,7 @@ Gates `/skill-audit` enforces in both flows:
|
||||
|
||||
## Step 4 — Validate and close
|
||||
|
||||
Run `/skill-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists.
|
||||
Run `/factory-audit` on the resolved skill directory; resolve every FAIL before reporting done. It checks name-to-directory match, placeholders, both size budgets, boundary-target resolution and script hygiene — do not hand-check those. Hand-check the one thing it misses: an empty body reports `PASS SKILL.md body word count 0 (ADR-0020 target: 600)`, so confirm at least one non-empty section exists.
|
||||
|
||||
Bump `metadata.version`: the **minor** version on create (new skills start at `0.1.0`) and the **patch** version on improve.
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ source_keys:
|
||||
|
||||
# The description and body contract
|
||||
|
||||
House contract. Every rule here is enforced by `/skill-audit` —
|
||||
House contract. Every rule here is enforced by `/factory-audit` —
|
||||
`scripts/validate.sh` for the counts and the boundary targets, the bundled Vale styles for the
|
||||
prose patterns, and its reference files for the judgment calls.
|
||||
|
||||
@@ -146,7 +146,7 @@ row's target exists on disk; each row pairs exactly one target with a condition
|
||||
evaluate from the request, never a literal slash invocation; one line after the table names the
|
||||
matched file as the only one to read; and the gates every branch needs sit in the body, not inside
|
||||
one flow's file. That last one is the property the `git-commits` v0.1.2 failure turned on, and it is
|
||||
the one a dispatch split is most likely to break. `skill-audit`'s `references/body-discipline.md`
|
||||
the one a dispatch split is most likely to break. `factory-audit`'s `references/skill-body-discipline.md`
|
||||
carries the audit-side form of the same exemption; the two lists are the same four properties, and
|
||||
an edit to either belongs in both.
|
||||
|
||||
|
||||
@@ -28,8 +28,8 @@ Before touching the filesystem, verify you have:
|
||||
|
||||
If any are missing, stop and ask the user before proceeding.
|
||||
|
||||
**Requires `/skill-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
|
||||
the kyberforge plugin and are co-installed. If `/skill-audit` is unavailable, stop and ask the
|
||||
**Requires `/factory-audit`** — used in `SKILL.md` Step 4 for final validation. Both skills ship in
|
||||
the kyberforge plugin and are co-installed. If `/factory-audit` is unavailable, stop and ask the
|
||||
user to install the kyberforge plugin before continuing.
|
||||
|
||||
## Package-intent gate
|
||||
@@ -175,7 +175,7 @@ If a research `sources.md` is present in the conversation context:
|
||||
`- **Research doc:** <path>` where `<path>` is the relative path from the repo root to the
|
||||
plugin-level research sources file this entry was drawn from (e.g.
|
||||
`plugins/myplugin/docs/research/docs/<topic>/sources.md`). This field is required on every
|
||||
entry — it makes the provenance chain explicit and is validated by `/skill-audit`.
|
||||
entry — it makes the provenance chain explicit and is validated by `/factory-audit`.
|
||||
4. Add `source_keys` to the frontmatter of `SKILL.md` (under `metadata`) listing the slugs of
|
||||
sources that informed it.
|
||||
5. For each file in `references/` that was informed by research sources, add `source_keys`
|
||||
|
||||
@@ -44,7 +44,7 @@ Use `${CLAUDE_PLUGIN_ROOT}` only in hook commands and `.mcp.json` configs — no
|
||||
|
||||
## Standalone mode
|
||||
|
||||
Deployed directly to `~/.agents/skills/<name>/`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/skill-audit`) work if the called skill is also installed.
|
||||
Deployed directly to `~/.agents/skills/<name>/`. No plugin context, no env vars injected. All file references must resolve within the skill directory. Skill invocations (e.g. `/factory-audit`) work if the called skill is also installed.
|
||||
|
||||
## Cross-tool portability
|
||||
|
||||
|
||||
@@ -16,12 +16,12 @@ Confirm the skill directory path exists and that at least one improvement signal
|
||||
conversation or a referenced file.
|
||||
|
||||
If the skill directory is missing, ask for it. If no signals are present, stop: "This skill applies
|
||||
existing signals to a skill. For a blind review without signals, use `/skill-audit` instead."
|
||||
existing signals to a skill. For a blind review without signals, use `/factory-audit` instead."
|
||||
|
||||
Signals can come from anywhere in the conversation or referenced files:
|
||||
|
||||
- Grill session output (most common predecessor in the factory sequence)
|
||||
- `/skill-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- `/factory-audit` findings (PASS/FAIL/SUGGESTION punch list)
|
||||
- Human feedback (feedback.json, inline in conversation, PR or issue comments)
|
||||
- Session context describing what went wrong
|
||||
|
||||
|
||||
@@ -186,4 +186,4 @@ echo " 3. Add docs to references/ if needed (or delete the directory)" >&2
|
||||
echo " 4. Add resources to assets/ if needed (or delete the directory)" >&2
|
||||
echo " 5. Add tests to tests/ if the skill has scripts (or delete the directory)" >&2
|
||||
echo " 6. Populate references/sources.md with research sources, or delete it" >&2
|
||||
echo " 7. Validate: run /skill-audit on $TARGET" >&2
|
||||
echo " 7. Validate: run /factory-audit on $TARGET" >&2
|
||||
|
||||
@@ -60,9 +60,9 @@ teardown() {
|
||||
assert [ -d "$DEST/my-tool-2" ]
|
||||
}
|
||||
|
||||
@test "next-steps output references /skill-audit not validate.sh" {
|
||||
@test "next-steps output references /factory-audit not validate.sh" {
|
||||
run bash "$SCRIPT" my-tool "$DEST"
|
||||
assert_output --partial "/skill-audit"
|
||||
assert_output --partial "/factory-audit"
|
||||
refute_output --partial "validate.sh"
|
||||
}
|
||||
|
||||
|
||||
@@ -41,9 +41,8 @@ Authoring source lives in `.apm/`; it is the only content source and the only th
|
||||
|---|---|
|
||||
| `forge` | Grill an unclassified "I want to add something" request, decide whether it's a skill, agent, plugin, or marketplace entry, then route to the matching author skill |
|
||||
| `skill-author` | Create or improve a skill from scratch, audit findings, or inline feedback |
|
||||
| `skill-audit` | Audit a skill directory against the agentskills.io spec and produce a findings report |
|
||||
| `agent-author` | Author an agent definition file |
|
||||
| `agent-audit` | Audit an agent definition across structure, provider safety, description and body quality, and provenance; produces a findings report |
|
||||
| `factory-audit` | Audit a skill directory or an agent definition — structure, provider safety, description and body quality, and provenance; produces a findings report. Auto-detects which of the two it was handed (ADR-0025) |
|
||||
| `apm-install` | Install or upgrade the apm CLI and set up the agent runtimes it drives (Copilot CLI, Codex, Gemini, generic llm) |
|
||||
| `apm-workflow` | Author apm.yml, scaffold an apm package/marketplace, install dependencies, and compile/pack/publish/audit apm content |
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
name: kyberforge
|
||||
version: 1.6.2
|
||||
version: 2.0.0
|
||||
description: Skills and agents for creating, maintaining, and managing a Claude Code / Copilot CLI plugin marketplace.
|
||||
author:
|
||||
name: Defame1297
|
||||
|
||||
@@ -18,6 +18,6 @@ Upstream reference material gathered during skill authoring. Not shipped with th
|
||||
| `research/docs/claude-code-plugins/` | Claude Code plugin and marketplace manifests, plugin directory layout, agent definition format, `claude plugin validate` behaviour |
|
||||
| `research/docs/github-copilot-plugins/` | Copilot CLI plugin manifest and marketplace format, agent definition format, Copilot extensions and SDK |
|
||||
| `research/docs/microsoft-apm/` | apm CLI reference, `apm.yml` schema, primitive schemas (agent, prompt, instructions, hooks), monorepo repo shapes, marketplace/registries, packing and releasing |
|
||||
| `research/examples/skill-write/` | Upstream skill examples reviewed when authoring skill-author and skill-audit |
|
||||
| `research/examples/skill-write/` | Upstream skill examples reviewed when authoring `skill-author` and the audit skill, now `factory-audit` (ADR-0025) |
|
||||
|
||||
AGENTS.md research moved to `plugins/core/docs/research/docs/agentsmd/` when the AGENTS.md skills landed in the `core` plugin (ADR-0012) — it is no longer part of kyberforge's provenance chain.
|
||||
|
||||
Reference in New Issue
Block a user