refactor(kyberforge)!: merge skill-audit and agent-audit into factory-audit
Why The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser (93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line test suite. Sync-by-gate had already failed once: at484357athe two parser copies drifted into different spellings of the bullet loop while a docstring asserted they were identical. That drift was behaviour-neutral and was re-unified by hand at598a7c3, so the copies were identical at merge time — but nothing had caught it, and the next drift need not be neutral. Implementation Notes Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching across skill directories, which is why two separate skills needed embedded copies; two files inside ONE skill may source a third. That is the whole reason the merge removes duplication rather than relocating it. The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path before any validation, and its table mirrors validate.sh's detection exactly: a directory holding SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory (agent); anything else stops without running a validator. Steps 1-3 live in references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the other artifact type, the body restarts at Step 0. Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that matches a live sibling reads as ownership rather than membership. The description carries one arrow per boundary target, because ADR-0020 resolves only the first target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated "audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16% of the body: no size SUGGESTIONs. The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both. tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources the resolver and that no third copy exists. The entry scripts classify the target from its resolved parent directory, so a bare agent filename typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is missing, rather than dying with exit 1, the tier the flows relay as real findings. The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's status (`f || RC=$?`) disables errexit for its entire body, and no subshell or `set -e` inside can re-arm it once the call sits in a condition context (measured, both spellings). Their error paths use `exit`, which is unaffected either way; this keeps errexit armed for anything added later. Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's dev environment is root -- so the guard could never fire where it exists to fire. A read attempt is also the stricter question, catching EIO. This is the reasoning scripts/check-vale-style-sync.sh carried before this commit deleted it; the hazard did not go with it. All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared, the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two remaining bare cd calls take absolute paths, which CDPATH is never consulted for. Impact BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to 2.0.0 (catalog 0.4.7). Check logic is unchanged: differential runs of the old and new validators across every skill and agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename. One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change, deliberately (full table in ADR-0025): - a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is by path shape, so a shape-matching path that is simply absent used to reach the validator and come back as a FAIL against a file that never existed; - an unshaped target exits 2 under the generic "matches neither" message, and a directory with no SKILL.md under a third, distinct one -- three exit-2 messages, not one; - a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding about the artifact rather than a usage error; - a SKILL.md file path is audited as its skill directory instead of refused; - a .md agent outside an agents/ directory is refused rather than audited; - a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2. validate-provenance.sh already exited 2 for the last two; validate.sh now matches it. .pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both files: regexes are unchanged; only entry: and description: moved. scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21), tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite. Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8. ADR: 0025 BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by factory-audit, which auto-detects whether it was handed a skill directory or an agent file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
This commit is contained in:
77
plugins/kyberforge/.apm/skills/factory-audit/SKILL.md
Normal file
77
plugins/kyberforge/.apm/skills/factory-audit/SKILL.md
Normal file
@@ -0,0 +1,77 @@
|
||||
---
|
||||
name: factory-audit
|
||||
description: >
|
||||
Use when the user wants a skill directory or agent definition audited,
|
||||
including "is this ready to ship", or after hand-editing one outside its
|
||||
author skill. Not applying skill fixes -> skill-author. Not applying agent
|
||||
fixes -> agent-author.
|
||||
allowed-tools: Bash Read
|
||||
metadata:
|
||||
version: "1.0.1"
|
||||
category: factory
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
## Gotchas
|
||||
|
||||
- Do not narrate PASS/FAIL per check while auditing. Gather findings internally and surface them only in the Step 4 report. Narrating each check as you go is the default failure mode here.
|
||||
- A file carrying `disable-model-invocation: true` is hand-invoked — its description is never routed against, so the trigger, capability and boundary rules do not apply. Audit it as one plain human-facing sentence instead.
|
||||
- Vale reporting `0 files` scanned means NOT RUN, not clean. Fall back to full Step 3 judgment for every dimension it would have covered.
|
||||
|
||||
## Step 0 — Dispatch
|
||||
|
||||
Resolve the flow from the target path **before running anything**. The two flows run different validators over different dimension vocabularies, so dispatching after Step 1 means the wrong validator has already produced the wrong findings. The rows mirror the shapes `scripts/validate.sh` accepts; take the first that matches.
|
||||
|
||||
| Target | Flow | Read |
|
||||
|---|---|---|
|
||||
| A directory containing `SKILL.md` | skill | `references/skill-flow.md` |
|
||||
| A file named `SKILL.md` — audit its parent directory | skill | `references/skill-flow.md` |
|
||||
| A file named `*.agent.md` | agent | `references/agent-flow.md` |
|
||||
| A `.md` file whose immediate parent directory is `agents/` (`.apm/agents`, `.claude/agents`, `.github/agents`, `.copilot/agents`) | agent | `references/agent-flow.md` |
|
||||
| Anything else — a missing path, a directory without `SKILL.md`, any other file | none | — |
|
||||
|
||||
Read only the file its row matched. Each carries Steps 1 to 3 — the deterministic checks, the read, and the qualitative audit — and is self-contained. Return here for Step 4.
|
||||
|
||||
On the last row, stop: run no validator and tell the user the two accepted shapes — a skill directory (or its `SKILL.md`), or an agent file (`*.agent.md`, or a `.md` directly under an `agents/` directory). Guessing a flow audits the path against the wrong spec.
|
||||
|
||||
The scripts re-detect the flow from the path. If `validate.sh` reports on the other artifact type than your row, discard what you have and restart here — the flow file, not the script, picked your rubrics, coverage line and remediation line.
|
||||
|
||||
## Step 4 — Report
|
||||
|
||||
Open with the coverage line for the flow you took, naming every dimension checked.
|
||||
|
||||
Skill flow:
|
||||
|
||||
```text
|
||||
Checked: structure · description · body-discipline · patterns · file-structure · formatting · scripts · internal-consistency · provenance
|
||||
```
|
||||
|
||||
Agent flow:
|
||||
|
||||
```text
|
||||
Checked: structure · provider-safety · description · body · delegation · comment-discipline · pair-consistency · provenance
|
||||
```
|
||||
|
||||
On the agent flow at plugin/APM scope, drop `pair-consistency` — there is no pair to check.
|
||||
|
||||
Then output only the dimensions that have findings, grouped under H3 headings, FAILs before SUGGESTIONs within each. Omit clean dimensions — their absence is what confirms they passed.
|
||||
|
||||
Each finding:
|
||||
|
||||
```text
|
||||
FAIL/SUGGESTION <finding> — file:line
|
||||
Why: <why this is a problem>
|
||||
Fix: <exact change — quote before/after where applicable>
|
||||
```
|
||||
|
||||
Close with a `## Result` block holding one line: `PASS`, `PASS (N suggestions)`, or `FAIL (N fails · M suggestions)`, each optionally followed by ` · P info`. INFO findings are observational and never change PASS/FAIL; omit `· P info` when there are none. Add a second line whenever there is at least one finding — `Run skill-author to address findings.` on the skill flow, `Run agent-author to address findings.` on the agent flow. Do not apply fixes — report and propose only.
|
||||
@@ -0,0 +1,10 @@
|
||||
StylesPath = styles
|
||||
|
||||
[**/SKILL.md]
|
||||
BasedOnStyles = Kyberforge
|
||||
|
||||
[**/agents/*.md]
|
||||
BasedOnStyles = Kyberforge
|
||||
|
||||
[**/*.agent.md]
|
||||
BasedOnStyles = Kyberforge, KyberforgeCopilot
|
||||
@@ -0,0 +1,13 @@
|
||||
extends: existence
|
||||
message: "Composition or architecture note in a description: '%s' — a description carries a trigger, one capability clause and a boundary clause only; move this to README.md"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- cross-cutting
|
||||
- shared (skill|agent)
|
||||
- human-facing
|
||||
- entry[- ]point
|
||||
- composes
|
||||
- rather than duplicating
|
||||
- replaces the (old|former|previous)
|
||||
@@ -0,0 +1,7 @@
|
||||
extends: existence
|
||||
message: "Description opens with '%s' — use an imperative 'Use when...' opener instead"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
raw:
|
||||
- '^This\b'
|
||||
@@ -0,0 +1,7 @@
|
||||
extends: existence
|
||||
message: "Generic reference pointer: '%s' — use the specific 'If X, read `references/file.md`' form instead"
|
||||
level: error
|
||||
scope: text
|
||||
ignorecase: true
|
||||
raw:
|
||||
- 'see references?/? for (more )?(info|information|details)\b'
|
||||
@@ -0,0 +1,7 @@
|
||||
extends: existence
|
||||
message: "Don't start a sentence with '%s' — name the subject directly"
|
||||
level: error
|
||||
scope: sentence
|
||||
ignorecase: false
|
||||
raw:
|
||||
- '^There\s(is|are)\b'
|
||||
@@ -0,0 +1,10 @@
|
||||
extends: existence
|
||||
message: "Vague capability wording: '%s' — state the capability precisely instead"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- helps with
|
||||
- utilize
|
||||
- assists with
|
||||
- used for
|
||||
@@ -0,0 +1,7 @@
|
||||
extends: existence
|
||||
message: "'%s' is CC-specific phrasing with no effect in Copilot descriptions — remove it"
|
||||
level: error
|
||||
scope: text.frontmatter.description
|
||||
ignorecase: true
|
||||
tokens:
|
||||
- Use proactively
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Body, Delegation and Comment Discipline Reference
|
||||
|
||||
Upstream source: Claude Code subagent and plugin references, GitHub Copilot custom-agents
|
||||
configuration. House contract: the context budget.
|
||||
|
||||
Read this when judging the **body**, **delegation** and **comment-discipline** dimensions.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## Agents take no body word gate
|
||||
|
||||
A skill body is gated at 600 words SUGGESTION / 900 FAIL; an agent body is deliberately gated at
|
||||
nothing. The two are not the same construct: a skill body is loaded into the caller's live
|
||||
context and competes with the conversation already there, while an agent body *becomes* the system
|
||||
prompt of a fresh context that has nothing else in it. The rationale for the 900-word ceiling does
|
||||
not transfer, so:
|
||||
|
||||
- **Never report an agent body as too long on a word count.** There is no number to cite.
|
||||
- **Never add such a gate to `scripts/validate.sh`.** `tests/validate-agent.bats` pins its absence with a
|
||||
body far past 900 words that must still pass, and adding one would contradict the ADR.
|
||||
- The one length signal that does apply is the Copilot runtime's 30,000-character body limit, which
|
||||
`validate.sh` already reports as a SUGGESTION because content past it is silently truncated.
|
||||
|
||||
Length is judged through the delegation check below instead, which is the defect a word count was
|
||||
standing in for anyway.
|
||||
|
||||
## The delegation check
|
||||
|
||||
A plugin-scope agent is a single `.apm/agents/<name>.agent.md` file with no sibling `references/`
|
||||
directory. It cannot progressively disclose to itself — it can only delegate to skills. So a
|
||||
procedure spelled out in an agent body that a skill the agent invokes already owns is not a
|
||||
shortcut: it is a second copy of that procedure, and the second copy drifts. This is the
|
||||
characteristic agent defect, the way a stale README row is the characteristic skill defect.
|
||||
|
||||
**An agent body that restates a procedure owned by a skill it can invoke is a FAIL.** The Fix is
|
||||
always the same shape: invoke `<skill>` instead.
|
||||
|
||||
How to apply it: for each procedural block in the body — a rule list, a numbered sequence, a
|
||||
constraint table — ask which skill owns that procedure. If the agent names that skill anywhere (its
|
||||
dispatch table, its routing prose, its frontmatter), the block is a restatement and the skill is
|
||||
already there to be invoked.
|
||||
|
||||
Worked example. The three `*-orchestrate` agents exist to compose domain skills — `git-orchestrate`
|
||||
(933 body words), `gitea-orchestrate` (1,199) and `apm-orchestrate` (1,080) — so any step they
|
||||
spell out that the composed skill already owns is the defect. `git-orchestrate:24-31` carries a
|
||||
"Hard rules" list (Conventional Commits types, atomic commits, never commit secrets, git trailers)
|
||||
that `git-commits` owns and that `git-orchestrate:44` routes to by name; `:39` concedes the point
|
||||
outright, noting the sub-skills "carry their own local copies of these rules". Two copies, one
|
||||
authority, and nothing keeping them in step.
|
||||
|
||||
What is **not** a finding under this rule, because no skill owns it:
|
||||
|
||||
- The dispatch table itself — which operation routes to which skill.
|
||||
- Safety gates the agent enforces before dispatching, and refusals it makes on its own authority.
|
||||
- The input contract and the structured output the agent's caller consumes.
|
||||
- Session state the agent carries across skill invocations.
|
||||
|
||||
## What the body is for
|
||||
|
||||
Include what the fresh context lacks:
|
||||
|
||||
- A direct role instruction opening the prompt: `You are a [role]. When invoked, [action].`
|
||||
- One bounded job, stated so the agent knows what it must refuse.
|
||||
- The dispatch, gates, inputs and outputs listed above.
|
||||
- **Error handling** — what the agent does on malformed, missing or contradictory input: stop and
|
||||
report, or degrade to a named fallback. Absent it, the agent invents a recovery, and a
|
||||
subagent's invented recovery is invisible to its caller until the output is wrong.
|
||||
- Non-obvious environment facts and project-specific conventions it cannot infer.
|
||||
- One default per decision point with one escape hatch.
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Comment discipline
|
||||
|
||||
Inspect every comment block in the YAML frontmatter and apply the core test to each: *would the
|
||||
agent get this wrong without this comment?* Template scaffolding — `# Optional. <long
|
||||
explanation>`, more than a line or two of inline guidance per field — belongs to development, not
|
||||
to a shipped file. At plugin/APM scope the stakes are higher than tidiness: `apm compile` copies
|
||||
frontmatter verbatim to every target, `<!-- ... -->` is not valid YAML, and `validate.sh` FAILs a
|
||||
frontmatter block that still contains one.
|
||||
|
||||
## Where the criteria live
|
||||
|
||||
Every FAIL and SUGGESTION criterion for these dimensions is in `references/agent-finding-criteria.md`,
|
||||
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
|
||||
puts the body, delegation or comment-discipline dimension in play.
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Agent Description Quality Reference
|
||||
|
||||
Upstream source: Claude Code subagent reference, GitHub Copilot custom-agents configuration.
|
||||
House contract: the context budget. The house contract is narrower than either
|
||||
platform's schema rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## Why the description is the expensive part
|
||||
|
||||
At startup an agent loads only the `name` and `description` of every installed skill and agent.
|
||||
The body is never seen until the agent is invoked. The description therefore carries the entire
|
||||
triggering burden **and** is paid for in every session, whether the agent fires or not.
|
||||
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the caller takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
- **`disable-model-invocation: true`** — the agent is hand-invoked. Its description is never
|
||||
matched against user intent, so it is not a routing string. It carries **one plain human-facing
|
||||
sentence** stating what the agent does. Audit it for that and nothing else. Reporting a missing
|
||||
trigger clause, a missing boundary clause or absent indirect triggers on a hand-invoked agent is a
|
||||
wrong finding, not a strict one. The field is Copilot-only and not on the vendor-neutral APM
|
||||
allowlist, so this case arises in a Copilot `.agent.md` at project/user scope and nowhere else.
|
||||
Its Claude Code counterpart has no equivalent field and stays model-invoked, so the two halves of
|
||||
the pair carrying differently shaped descriptions is expected there rather than a
|
||||
pair-consistency finding.
|
||||
`user-invocable: false` does not belong in this bullet. The two are separate fields with opposite
|
||||
defaults — `disable-model-invocation` (default `false`) governs runtime auto-selection,
|
||||
`user-invocable` (default `true`) governs manual invocation, and the retired `infer` field was
|
||||
replaced by the pair rather than by either one. So `user-invocable: false` says nothing about
|
||||
whether the agent is model-routed: judge that from `disable-model-invocation` alone, and where
|
||||
that is absent the three-part shape below still applies. `user-invocable` carries no
|
||||
description-quality contract of its own and is out of this file's scope entirely.
|
||||
- **No such flag** — the agent is model-invoked and the rest of this file applies.
|
||||
|
||||
## The three-part shape
|
||||
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This agent ...` —
|
||||
the caller is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source.
|
||||
|
||||
Everything else belongs in the body or in the plugin's `README.md`.
|
||||
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on an agent whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Near-miss exclusions
|
||||
|
||||
Add a boundary clause only where a sibling skill or agent could plausibly steal the activation. Use
|
||||
strong near-misses — queries that share keywords but need something different — not weak ones. One
|
||||
boundary clause per genuine near-miss; a list of four is enumeration wearing a boundary's clothes.
|
||||
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# FAIL — a noun-phrase opener rather than a trigger, capability enumeration in
|
||||
# place of one capability clause, and no boundary clause at all, preloaded into
|
||||
# every session forever. (The live git-orchestrate description, 254 chars.)
|
||||
description: Orchestrates git workflow operations for other agents. Invoke when a
|
||||
caller needs a multi-step or destructive git operation (rebase, force-push, branch
|
||||
deletion) coordinated across domain skills with safety gates, session context, and
|
||||
structured results.
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The operation list and the
|
||||
# safety-gate mechanics are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when an agent caller needs a multi-step or destructive git operation
|
||||
dispatched and safety-gated. Not conversational git help -> git-workflow.
|
||||
```
|
||||
|
||||
## Where the criteria live
|
||||
|
||||
Every FAIL and SUGGESTION criterion for this dimension is in `references/agent-finding-criteria.md`,
|
||||
which Step 3 reads on every run. This file is the reasoning behind them, loaded only when that file
|
||||
puts the description dimension in play.
|
||||
@@ -0,0 +1,49 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
## claude-code-fields
|
||||
|
||||
name description tools disallowedTools model effort maxTurns permissionMode skills mcpServers hooks memory background isolation color initialPrompt
|
||||
|
||||
## claude-code-only-fields
|
||||
|
||||
maxTurns isolation memory permissionMode effort hooks mcpServers disallowedTools skills initialPrompt color background
|
||||
|
||||
## copilot-fields
|
||||
|
||||
name description tools target model disable-model-invocation user-invocable mcp-servers metadata
|
||||
|
||||
## copilot-only-fields
|
||||
|
||||
target disable-model-invocation user-invocable mcp-servers metadata
|
||||
|
||||
## apm-agent-allowlist
|
||||
|
||||
name description model source_keys disallowedTools
|
||||
|
||||
Parsing note: `validate.sh` reads the **first** non-empty, non-`#`, non-`---` line under each
|
||||
heading as a whitespace-separated token list, and stops there. Keep the token line immediately
|
||||
below its heading; explanatory prose goes after it, as here.
|
||||
|
||||
Why `disallowedTools` is on a list that is otherwise vendor-neutral, when `tools` is not
|
||||
(ADR-0016 and its 2026-08-14 amendment): the two are not symmetric. `tools` is an **allowlist**
|
||||
whose vocabulary differs per harness — Claude Code names its own tools, Copilot CLI uses aliases
|
||||
(`execute`/`read`/`edit`/`search`/`agent`/`web`) — so a value correct for one is wrong for the
|
||||
other, and `apm compile` copies frontmatter verbatim with no per-target integrator to reconcile
|
||||
them. `disallowedTools` is a **denylist**, and denying by name is safe under verbatim copy: a name
|
||||
the other harness does not recognise denies nothing, so the worst case is that the fence is absent
|
||||
there, never that the wrong capability is granted. Claude Code honours it for plugin subagents: its
|
||||
plugin agent-definition reference names the fields plugin agents silently ignore (`hooks`,
|
||||
`mcpServers`, `permissionMode`), and `disallowedTools` is not among them.
|
||||
|
||||
`disallowedTools` also appears in `claude-code-only-fields` above, and that stays correct: at
|
||||
project/user scope it is still a Claude-only field and must not appear in a Copilot `.agent.md`.
|
||||
The two lists answer different questions — "may this field cross the CC/Copilot file boundary" for
|
||||
a real pair, versus "is this field safe under verbatim copy to every target" for a single
|
||||
vendor-neutral APM file.
|
||||
@@ -0,0 +1,98 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Finding Criteria
|
||||
|
||||
Every FAIL and SUGGESTION criterion, for every qualitative dimension, and nothing else. The
|
||||
reasoning each criterion stands on, its worked examples and its house rules stay in that
|
||||
dimension's rubric, which Step 3 loads only for a dimension this file puts in play.
|
||||
|
||||
Two rules on using it:
|
||||
|
||||
- A criterion that plainly applies is a finding. Write it up citing file and line.
|
||||
- A criterion that might apply, or whose call the wording here does not settle, is a reason to load
|
||||
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
||||
to read; it does not settle a close call on its own.
|
||||
|
||||
## description — `references/agent-description-quality.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut. Agent
|
||||
descriptions have no platform-documented ceiling of their own, so 400 is the only hard limit
|
||||
there is — do not go looking for a backstop behind it.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared agent", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This agent ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
||||
- **Vague capabilities** ("helps with agents" where "audits an agent definition pair" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **`Use proactively` in a Copilot or vendor-neutral description.**
|
||||
`KyberforgeCopilot.ProactivePhrase` catches it. The phrase steers the Claude Code runtime and
|
||||
does nothing anywhere else, so in a `.agent.md` it is preloaded text that buys no behaviour.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked agent** — see Step 0 of
|
||||
`references/agent-description-quality.md`.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
**An unresolved boundary target is not graded here.** `validate.sh` resolves boundary targets for
|
||||
agent files at both scopes and tiers the verdict itself — route notation (`/name`, an arrow form)
|
||||
is an ERROR, the bare prose form a SUGGESTION unless a second target in the same sentence resolves.
|
||||
Step 1 has already filed it under `### Structure` at that tier. Take the script's verdict rather
|
||||
than re-resolving the name by hand, and do not re-grade it under description: a hand-walk over a
|
||||
different universe can contradict the script, and re-grading puts one target in the report twice.
|
||||
What is left to judgment is semantic and the script cannot reach it: whether a target that *does*
|
||||
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
||||
files manually") should have named one.
|
||||
|
||||
## body, delegation and comment-discipline — `references/agent-body-and-delegation.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- The body restates a procedure owned by a skill the agent can invoke — Fix: invoke `<skill>`
|
||||
instead
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- Frontmatter comments are template scaffolding rather than instruction, or are HTML comments at
|
||||
plugin/APM scope
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- The body does not open with a direct role instruction
|
||||
- The body specifies no error handling — nothing tells the agent what to do with malformed,
|
||||
missing or contradictory input
|
||||
- The job the agent describes is unbounded, or bounded only implicitly
|
||||
- A rationale is missing from a rule the agent is expected to enforce — present but unexplained
|
||||
- Comments are useful but verbose enough to bury the field they annotate
|
||||
|
||||
**Never report an agent body as too long on a word count.** A skill body is gated at 600/900 words;
|
||||
an agent body is deliberately gated at nothing, because an agent body *becomes* the
|
||||
system prompt of a fresh context rather than competing with a live conversation. No number exists
|
||||
to cite. The one length signal that applies is the Copilot runtime's 30,000-character body limit,
|
||||
which `validate.sh` already reports as a SUGGESTION. Length is judged through the delegation FAIL
|
||||
above instead.
|
||||
@@ -0,0 +1,64 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Agent Flow
|
||||
|
||||
Steps 1 to 3 for an agent definition — the target Step 0 matched as a `*.agent.md` file, or as a
|
||||
`.md` file whose immediate parent directory is `agents/`. Work them in order, then return to
|
||||
`SKILL.md` Step 4 to report.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- An agent takes the skill description gates (250 characters SUGGESTION, 400 FAIL) and **no body word gate at all**. Its body becomes the system prompt of a fresh context, so the 900-word skill ceiling does not transfer and no number exists to cite. Judge an over-long agent body through the delegation check.
|
||||
- **Plugin/APM scope only:** provider safety means survival of a verbatim copy to every target, not Claude-Code-versus-Copilot field leakage — `references/agent-scope-plugin-apm.md` carries the contract.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <agent-file>
|
||||
bash scripts/validate-provenance.sh <agent-file>
|
||||
bash scripts/vale-wrap.sh <agent-file> [<counterpart-file>]
|
||||
```
|
||||
|
||||
`validate.sh` takes either half of a project/user-scope pair or the single plugin/APM-scope file, detects the provider from the extension and the scope by walking up, then checks required fields, kebab-case `name`, `FILL IN:` placeholders, template HTML comments left in frontmatter, the description budget (250 chars SUGGESTION, 400 FAIL, measured on the folded YAML value) and the fields that scope permits. Its findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both — except the ones the Step 2 scope contract re-routes.
|
||||
|
||||
If a validation script fails or cannot run — Bash denied, `python3` or `vale` absent, a `scripts/lib-*.sh` library or `references/agent-field-inventory.md` missing — read `references/agent-validation-scripts.md`; what these scripts measure is not reproducible by reading.
|
||||
|
||||
`validate-provenance.sh` prints nothing on success, so read its exit code before you read its silence. **0** is a genuine pass, including the silent exit 0 at project or user scope, where plugin-scope provenance does not apply. **1** means real findings: its FAILs and INFOs become a separate `### Provenance` dimension, and it emits Why and Fix itself — surface those verbatim. **2** means the check never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, no findings and often no stdout at all. On a 2, report `### Provenance` as unverified and quote the stderr reason; never grade it as a clean pass. `validate.sh` uses the same tiers: **1** is real findings, **2** is never ran — report that as `### Structure` unverified, never as a failure.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. At project/user scope pass both files of the pair, not only the one you were handed. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording`, `KyberforgeCopilot.ProactivePhrase` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs`, `Kyberforge.PaddingPhrase` | body |
|
||||
|
||||
## Step 2 — Read the agent and load its scope contract
|
||||
|
||||
Read the agent file end to end, and at project/user scope its counterpart too. A path containing `.apm/agents/` is plugin/APM scope; anything else is project or user scope. Each contract names the dimensions that apply there and where `validate.sh` findings other than Structure belong:
|
||||
|
||||
| Scope | Read |
|
||||
|---|---|
|
||||
| plugin/APM | `references/agent-scope-plugin-apm.md` |
|
||||
| project, user | `references/agent-scope-project-user.md` |
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/agent-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/agent-description-quality.md` |
|
||||
| body, delegation, comment-discipline | `references/agent-body-and-delegation.md` |
|
||||
|
||||
Each rubric is the reasoning behind its criteria, not a second copy of them. Cite file and line number for every finding.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Plugin/APM Scope Contract
|
||||
|
||||
Read this when the agent file sits at `<package>/.apm/agents/<name>.agent.md` — a single
|
||||
vendor-neutral file inside an APM package, with no counterpart anywhere.
|
||||
|
||||
## What is different here
|
||||
|
||||
`apm compile` copies an agent's frontmatter **verbatim** to every target harness. There is no
|
||||
per-target integrator to reconcile a Claude-Code-only field with a Copilot-only one, so the file
|
||||
cannot carry either (ADR-0016). That single fact drives everything below.
|
||||
|
||||
## Frontmatter allowlist
|
||||
|
||||
The permitted keys are the `apm-agent-allowlist` section of `references/agent-field-inventory.md`. Read
|
||||
them from there. Do not recite the list in a finding, do not work from memory, and do not trust any
|
||||
restatement of it you find elsewhere in this repo: the list is data with one home (ADR-0009), it
|
||||
has changed before, and `validate.sh` parses that same section at load time, so a recitation is a
|
||||
copy that can disagree with the check the agent just ran.
|
||||
|
||||
`agent-field-inventory.md` records why a denylist-shaped field is admitted where an allowlist-shaped one
|
||||
is not. Read that note before arguing with a finding about it.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at this scope:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| any frontmatter key outside the allowlist; body over the 30,000-character Copilot limit | Provider safety |
|
||||
| everything else — missing or malformed field, `name` not matching the filename stem, empty body, absent frontmatter, template HTML comments, description length | Structure |
|
||||
| — | Pair consistency never applies |
|
||||
|
||||
**Provider safety means something else here.** At project/user scope it asks whether a field leaked
|
||||
across the Claude Code / Copilot boundary. At this scope there is no boundary and no pair: it asks
|
||||
whether every field survives a verbatim copy to *every* target. Report it in those terms — a
|
||||
finding phrased as "CC-only field in a Copilot file" is the wrong finding here.
|
||||
|
||||
**Pair consistency never applies.** There is one file by design. `validate.sh` never emits a
|
||||
missing-counterpart FAIL at this scope, and neither do you, under any circumstance. Drop
|
||||
`pair-consistency` from the Step 4 coverage line rather than reporting it clean.
|
||||
|
||||
## Behaviour the schema cannot express
|
||||
|
||||
Read the description and body. If either implies a need the vendor-neutral frontmatter can no
|
||||
longer express — a tool restriction, `isolation`, `memory`, or another Claude-only behaviour a
|
||||
hand-authored CC file could have declared — flag it as a **SUGGESTION, never a FAIL**. This is a
|
||||
known upstream schema limitation (ADR-0016), not an authoring mistake, and the finding exists to
|
||||
give the author visibility into the gap rather than to imply the schema can be made to close it.
|
||||
|
||||
Example: a body saying "only use Read and Grep, never Edit" with no `tools` field to enforce it.
|
||||
A denylist-shaped restriction is the available half of that — see `agent-field-inventory.md`.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
source_keys:
|
||||
- context7-websites-code-claude
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Project and User Scope Contract
|
||||
|
||||
Read this when the agent file is not under `.apm/agents/` — a Claude Code `.md` and Copilot CLI
|
||||
`.agent.md` **pair**, at project scope (`<repo>/.claude/agents/` and `<repo>/.github/agents/`) or
|
||||
user scope (`~/.claude/agents/` and `~/.copilot/agents/`). `validate.sh` derives the counterpart
|
||||
from whichever half it was handed; audit both.
|
||||
|
||||
## The pair is a house convention
|
||||
|
||||
Neither platform requires a counterpart file. The pair is a kyberforge convention (ADR-0005), so a
|
||||
missing counterpart is a FAIL against **this repo's** convention and must be labelled that way in
|
||||
the finding, not presented as a platform spec failure.
|
||||
|
||||
## Dimension routing
|
||||
|
||||
`validate.sh` findings land as follows at these scopes:
|
||||
|
||||
| Finding | Dimension |
|
||||
|---|---|
|
||||
| a Claude-Code-only field in the Copilot file, a Copilot-only field in the CC file, a tool the runtime withholds from subagents, body over the 30,000-character Copilot limit | Provider safety |
|
||||
| counterpart file not found | Pair consistency |
|
||||
| everything else — missing or malformed field, name format, empty body, absent frontmatter, description length | Structure |
|
||||
|
||||
The two field lists are the `claude-code-only-fields` and `copilot-only-fields` sections of
|
||||
`references/agent-field-inventory.md`. Read them from there rather than from memory; `validate.sh` parses
|
||||
those same sections, so any restatement is a copy that can disagree with the check (ADR-0009).
|
||||
|
||||
## Field and naming rules that differ by provider
|
||||
|
||||
- `name` must match the filename stem in a **Copilot CLI** `.agent.md`. Claude Code imposes no such
|
||||
rule, so a CC file whose `name` differs from its filename is not a finding.
|
||||
- A Copilot **cloud/IDE** agent — one under `.github/copilot/agents/` — may omit `name` entirely.
|
||||
If it carries one, it still has to be kebab-case.
|
||||
- `Use proactively` is meaningful in a CC description and steers the runtime to offer the agent
|
||||
unprompted. In a Copilot description it does nothing; `KyberforgeCopilot.ProactivePhrase` flags
|
||||
it. The Copilot equivalent is `disable-model-invocation`, which changes the description contract
|
||||
entirely — see `references/agent-description-quality.md`, Step 0.
|
||||
|
||||
## Pair consistency
|
||||
|
||||
Check that:
|
||||
|
||||
- Both files exist.
|
||||
- Both system prompt bodies are non-empty (`validate.sh` covers this; do it by hand only when the
|
||||
script could not run).
|
||||
- The two files describe the **same job**. Divergent capability claims across the pair mean one
|
||||
half was edited and the other was not, which is the defect this dimension exists to catch.
|
||||
- Descriptions may legitimately differ in *shape* when the Copilot half is hand-invoked — that is
|
||||
the Step 0 case in `references/agent-description-quality.md`, not a pair-consistency finding.
|
||||
|
||||
Keep `pair-consistency` in the Step 4 coverage line at these scopes.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
source_keys:
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Validation Scripts Reference
|
||||
|
||||
Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting.
|
||||
Nothing here is needed on a clean run.
|
||||
|
||||
## Report the gap, do not guess
|
||||
|
||||
If a script cannot run at all — Bash denied, `python3` unavailable, `vale` not installed — say so
|
||||
as an **INFO** finding naming the script and the missing dependency, then fall back to the manual
|
||||
checks below. An INFO never changes PASS/FAIL. Silently omitting the dimension a script would have
|
||||
covered reports a clean audit that checked less than it claims to have checked.
|
||||
|
||||
## How the scripts detect scope
|
||||
|
||||
`validate.sh` and `validate-provenance.sh` walk up from the agent file's directory and stop at the
|
||||
first of these:
|
||||
|
||||
1. An `apm.yml` carrying a top-level `type: instructions|skill|hybrid|prompts` line — **plugin/APM
|
||||
scope**, and that directory is the package root. An `apm.yml` with no `type:` is a
|
||||
marketplace-only manifest: skip it and keep walking.
|
||||
2. `$HOME` — **user scope**, checked before `.git` so a dotfiles-managed home directory that is its
|
||||
own repo cannot shadow it.
|
||||
3. A `.git` directory or file — **project scope**.
|
||||
4. The filesystem root — **project scope**.
|
||||
|
||||
`plugin.json` and `.claude-plugin/plugin.json` are not scope signals. A directory holding only a
|
||||
`plugin.json` and no `apm.yml` falls through to project or user scope.
|
||||
|
||||
`validate-provenance.sh` exits 0 silently when that walk does not land on a package root, and again
|
||||
when the package has no provenance data. Check the exit code before you believe the silence:
|
||||
|
||||
- **0** — a pass, not a skip you need to investigate. Both silent cases above land here.
|
||||
- **1** — real findings, on stdout with Why and Fix.
|
||||
- **2** — the check never ran. A missing, doubled, non-file or wrongly-named argument, an
|
||||
undecodable `apm.yml`, or an absent `python3`, each with a diagnostic on stderr and no findings
|
||||
at all. Report the `### Provenance` dimension as unverified and quote the reason. An exit 2 is
|
||||
never a clean pass: empty stdout there means nothing was checked, not that nothing was wrong.
|
||||
|
||||
## Manual fallback
|
||||
|
||||
**Every scope:** required fields present (`name`, `description`, non-empty body); `name` is
|
||||
kebab-case; no `FILL IN:` placeholders in the description or body; the description at or under 400
|
||||
characters measured on the folded YAML value.
|
||||
|
||||
**Plugin/APM scope:** `name` matches the filename stem; no HTML comments left in the frontmatter;
|
||||
no frontmatter key outside the `apm-agent-allowlist` section of `references/agent-field-inventory.md` —
|
||||
open that file, do not work from memory.
|
||||
|
||||
**Project/user scope:** the counterpart file exists; `name` matches the filename stem in the
|
||||
Copilot `.agent.md` only (Claude Code files are exempt); no key from `claude-code-only-fields` in
|
||||
the Copilot file and none from `copilot-only-fields` in the CC file, both read from
|
||||
`references/agent-field-inventory.md`.
|
||||
|
||||
## Script-specific failures
|
||||
|
||||
- **`Error: agent-field-inventory.md not found` (exit 2).** `validate.sh` reads its field lists from
|
||||
`references/agent-field-inventory.md` at load time and refuses to run without it, rather than falling
|
||||
back to a hardcoded list that could disagree with the file (ADR-0009). Restore the file; do not
|
||||
work around it.
|
||||
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full
|
||||
Step 3 judgment for the dimensions it would have covered. The `Kyberforge` style is scoped to
|
||||
`**/agents/*.md` and `**/*.agent.md`, and `KyberforgeCopilot` to `**/*.agent.md` alone — a file
|
||||
outside those globs is silently not linted.
|
||||
- **`E100 Runtime error ... does not exist` (exit 2) from `vale-wrap.sh`.** An explicit relative
|
||||
`--config` was passed. Pass none: the wrapper locates its own `assets/vale/.vale.ini` from its
|
||||
own path. Do not read this exit code as vale being unavailable.
|
||||
- **A path argument that does not exist is a hard error** in `vale-wrap.sh`, deliberately: bare
|
||||
`vale` would fall back to reading stdin and print a clean-looking `0 errors ... in stdin`, which
|
||||
the `0 files` guard above does not catch.
|
||||
@@ -0,0 +1,209 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
---
|
||||
|
||||
# Body Discipline Reference
|
||||
|
||||
Upstream source: agentskills.io — skill-authoring, best-practices.
|
||||
House contract: the context budget.
|
||||
|
||||
## The core test
|
||||
|
||||
For every sentence in the body, ask: **"Would the agent get this wrong without this instruction?"**
|
||||
|
||||
If no — cut it. The agent already knows it from general training. Adding it wastes tokens and
|
||||
dilutes the signal of what matters.
|
||||
|
||||
## What the body is for
|
||||
|
||||
The body carries the **decision procedure only**: ordered steps, decision branches, gates, and
|
||||
which reference to load when.
|
||||
|
||||
Include content the agent lacks:
|
||||
|
||||
- Project-specific conventions and domain procedures it cannot infer
|
||||
- Non-obvious edge cases and environment-specific gotchas
|
||||
- The specific tools or sequences to use — not the full range of options
|
||||
- One default per decision point with one escape hatch
|
||||
|
||||
Move to `references/`, behind an explicit "If X, read `references/<file>.md`" trigger — the literal
|
||||
conditional form, never a generic pointer. Write the real filename in the skill under audit; the
|
||||
angle brackets are a placeholder here, and a literal `references/file.md` in a body is an ERROR
|
||||
from the gate because no such file exists on disk.
|
||||
|
||||
**A dispatch table satisfies this requirement on its own.** A table row already pairs a condition
|
||||
with a target, which is exactly what the literal form encodes; restating each row underneath as a
|
||||
prose conditional duplicates the routing in the one body whose whole purpose is to be short. Where a
|
||||
body dispatches, audit the table for condition/target completeness and stop there — do not require
|
||||
the conditional form as well. The literal form is what a body needs when it loads a reference
|
||||
*without* a dispatch table: a single mid-procedure deepening, an escape hatch, an error path.
|
||||
|
||||
Move:
|
||||
|
||||
- Lookup tables and spec restatements
|
||||
- Output schemas, templates and example blocks
|
||||
- Rationale and justification prose
|
||||
- Anything only one branch of the procedure ever reaches
|
||||
|
||||
Do not include at all:
|
||||
|
||||
- Concepts the agent already knows (what JSON is, how HTTP works, what a CSV is)
|
||||
- Exhaustive option lists — pick a default; the agent does not benefit from choosing
|
||||
- Steps the agent handles independently — over-specifying leads to unproductive paths
|
||||
- Restatements of the description, which is already in context
|
||||
|
||||
## Two length families, measured differently
|
||||
|
||||
Do not conflate these, and do not report them as one finding.
|
||||
|
||||
| Gate | SUGGESTION | FAIL | Counts |
|
||||
|---|---|---|---|
|
||||
| Body budget (house) | 600 words | 900 words | the **body only** — everything after the frontmatter's closing `---` |
|
||||
| Spec conformance (agentskills.io) | — | 2,770 words / 500 lines | the **whole file**, frontmatter included |
|
||||
|
||||
The 2,770-word ceiling is a token-conformance backstop calibrated to the densest prose in the
|
||||
corpus; it says nothing about quality and a file can sit a thousand words inside it while failing
|
||||
the body budget. The 900-word ceiling is the quality gate: a body is loaded into the caller's live
|
||||
context and competes with the conversation already there. `validate.sh` reports both. Cite whichever
|
||||
one actually fired.
|
||||
|
||||
A word count cannot detect the defect it stands in for. Treat both numbers as backstops to the
|
||||
dispatch rule and the Gotchas constraint below, never as a substitute for them.
|
||||
|
||||
## Dispatch is mandatory at two or more mutually exclusive flows
|
||||
|
||||
If a skill handles two or more flows that a single invocation cannot both take — separate
|
||||
subcommands, separate input types, separate lifecycle stages — the body carries a **dispatch
|
||||
table** plus the gates common to every branch, and each flow lives in its own self-contained
|
||||
`references/` file. Inlining all of them is a FAIL regardless of word count, because every
|
||||
invocation then pays for every branch it did not take.
|
||||
|
||||
The reference shape in this repo is `apm-workflow`: a **294-word body** dispatching to 3,154 words
|
||||
of references across five mutually exclusive flows. Its whole-file count is 348 words — cite 294
|
||||
when calibrating a body, or the conflation this section warns against reappears in the finding
|
||||
itself. The 3,154 counts the five flow files only; `references/sources.md` is a provenance record
|
||||
and is never loaded at runtime, so counting it inflates the dispatched total.
|
||||
|
||||
### What earns the wiring exemption
|
||||
|
||||
A dispatch table earns the exemption above on its properties, not on which skill it appears in.
|
||||
Audit any dispatching body against these four:
|
||||
|
||||
- Every flow the skill handles has a row, and every row names a target file that exists on disk.
|
||||
- Each row pairs a condition the agent can evaluate from the request with exactly one target. A row
|
||||
keyed on a literal slash invocation fails this: a model-invoked activation never produces that
|
||||
string, so the routing silently falls to whatever else the row carries.
|
||||
- One line after the table tells the agent to read the file its row matched, and only that one.
|
||||
- The gates every branch needs sit in the body, not inside one flow's file — see the reachability
|
||||
precondition below.
|
||||
|
||||
A table missing any of the four is not exempt, and the literal-conditional requirement applies to it
|
||||
as written. The exemption covers the wiring form only: every other rule in this file applies to a
|
||||
dispatching skill exactly as it applies to any other.
|
||||
|
||||
## Gotchas sections
|
||||
|
||||
The highest-value construct in a body, and the easiest to fill with noise. A Gotcha must state a
|
||||
fact that **contradicts a reasonable default** — something the agent gets wrong precisely by acting
|
||||
sensibly.
|
||||
|
||||
```markdown
|
||||
## Gotchas
|
||||
- The `users` table uses soft deletes. Always include `WHERE deleted_at IS NULL`.
|
||||
- User ID is `user_id` in the database, `uid` in auth, `accountId` in billing. Same value.
|
||||
```
|
||||
|
||||
Constraints:
|
||||
|
||||
- **More than five entries is a SUGGESTION** — five is the guideline, not a ceiling. Past five, the
|
||||
section is usually a summary of the body rather than a set of traps, and the agent stops reading
|
||||
it as a warning. It stays advisory because whether a given gotcha earns its place is judgment;
|
||||
`validate.sh` emits it through `suggest()` and the run still exits 0.
|
||||
- **A Gotcha that paraphrases a step in the body below it is a FAIL.** It has no independent
|
||||
content, and it teaches the agent that Gotchas can be skimmed because the real instruction is
|
||||
coming. This one is the auditor's call — no script detects it. The Fix is conditional: delete the
|
||||
Gotcha only if the surviving copy is reachable from every branch that needs it — see the
|
||||
reachability precondition below.
|
||||
- **A Gotchas section exceeding 25% of the body is a SUGGESTION** — the body has been inverted into
|
||||
a preamble. Same tier and same reasoning as the entry count, and independent of it: either can
|
||||
fire without the other.
|
||||
- Place the section near the top. A gotcha read after the mistake is worthless, which is also why
|
||||
Gotchas is the one construct exempt from moving to `references/`.
|
||||
|
||||
Worked negative example — **`git-commits` v0.1.2 at commit `5e23250`, a fixed pre-retrofit
|
||||
snapshot, not the current file.** The live skill is v0.1.3 and matches none of the citations below;
|
||||
they are quoted as they stood in that snapshot, and are not to be refreshed against
|
||||
`HEAD`. The snapshot is reachable only from a checkout of the authoring repo — an installed plugin
|
||||
cache holds no git history and no such path — so read the citations below as quoted rather than
|
||||
going to look for the file. From a checkout:
|
||||
|
||||
```text
|
||||
git show 5e23250:<the git plugin>/.apm/skills/git-commits/SKILL.md
|
||||
```
|
||||
|
||||
That body carried twelve Gotchas, four of which restated content already below them or already in
|
||||
the description:
|
||||
|
||||
| Gotcha | Restates |
|
||||
|---|---|
|
||||
| `:31` "SemVer mapping is not optional" | the description |
|
||||
| `:32` "Confirmation gates are mandatory for destructive operations" | step 9 at `:52` |
|
||||
| `:33` "Never skip hooks with `--no-verify`" | step 9 at `:52` |
|
||||
| `:36` "Never commit secrets" | step 2 at `:45` |
|
||||
|
||||
All four are FAILs under the paraphrase rule. The entry count and the section's share of the body
|
||||
(387 of 1,102 words, 35%) are two further SUGGESTIONs on top — the script reports both, and neither
|
||||
fails the run on its own. What makes this worth auditing directly is that the four paraphrase FAILs
|
||||
pass every word gate there is; only reading the construct finds them.
|
||||
|
||||
### The paraphrase rule has a reachability precondition
|
||||
|
||||
**A Gotcha that restates a step may be deleted only when the surviving copy is reachable from every
|
||||
branch that needs it.** In a dispatch body it usually is not: each flow file is loaded alone, so a
|
||||
step in one is invisible to an invocation that took another branch. When the restated rule is a
|
||||
safety gate more than one flow needs, the Fix is to **move it into the body's common-gates section**,
|
||||
never to drop it in favour of the per-flow copy.
|
||||
|
||||
Row four is the case that proves it. Following the rule literally, the retrofit deleted the
|
||||
always-loaded secrets Gotcha and kept step 2 of `references/create-commit.md` — but `git-commits`
|
||||
dispatches to exactly one flow file, and `references/rewrite-history.md` stages changes and runs
|
||||
`--amend`, which commits newly staged content exactly as a fresh commit does. A grep for `secret`
|
||||
across the skill in that state returned one hit, on a path two of three branches never reach: that
|
||||
branch could commit a credential with no check anywhere in its loaded context, against this repo's
|
||||
governance hard prohibition. v0.1.3 carries the rule as gate 2 of "Gates on every flow" instead.
|
||||
|
||||
So check reachability before writing the Fix. Rows one to three are unaffected — the description is
|
||||
loaded on every invocation, and confirmation is likewise a common gate rather than a per-flow step.
|
||||
|
||||
## Calibrating control
|
||||
|
||||
**Be prescriptive** when operations are fragile, consistency matters, or a specific sequence must be
|
||||
followed:
|
||||
|
||||
```markdown
|
||||
Run exactly:
|
||||
\`\`\`bash
|
||||
python scripts/migrate.py --verify --backup
|
||||
\`\`\`
|
||||
Do not modify the command or add additional flags.
|
||||
```
|
||||
|
||||
**Give freedom** when multiple approaches are valid. Explaining *why* outperforms rigid directives —
|
||||
agents make better decisions when they understand the purpose.
|
||||
|
||||
## Defaults not menus
|
||||
|
||||
Never present a list of equivalent options — pick one and mention the alternative briefly:
|
||||
|
||||
```markdown
|
||||
# Too many options
|
||||
Use pypdf, pdfplumber, PyMuPDF, or pdf2image...
|
||||
|
||||
# Default with escape hatch
|
||||
Use pdfplumber for text extraction. For scanned PDFs requiring OCR, use pdf2image instead.
|
||||
```
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-optimizing-descriptions
|
||||
---
|
||||
|
||||
# Description Quality Reference
|
||||
|
||||
Upstream source: agentskills.io — optimizing-descriptions, specification.
|
||||
House contract: the context budget. The house contract is narrower than the spec
|
||||
rather than a reinterpretation of it: where both speak, both must be satisfied.
|
||||
|
||||
## Why the description is the expensive part
|
||||
|
||||
At startup an agent loads only the `name` and `description` of every installed skill. The body is
|
||||
never seen until the skill triggers. The description therefore carries the entire triggering
|
||||
burden **and** is paid for in every session, whether the skill fires or not.
|
||||
|
||||
A second cost is less obvious and is a correctness hazard rather than a token cost: a description
|
||||
that summarises the workflow is a shortcut the agent takes *instead of* reading the body. A
|
||||
measured failure upstream — a description saying "code review between tasks" — produced one review
|
||||
where the body's flowchart specified two.
|
||||
|
||||
## Step 0 — establish which contract applies
|
||||
|
||||
Read the frontmatter before judging a single word.
|
||||
|
||||
- **`disable-model-invocation: true`** — the skill is hand-invoked. Its description is never
|
||||
matched against user intent, so it is not a routing string. It carries **one plain human-facing
|
||||
sentence** stating what the skill does. Audit it for that and nothing else. Reporting a missing
|
||||
trigger clause, a missing boundary clause or absent indirect triggers on a hand-invoked skill is
|
||||
a wrong finding, not a strict one.
|
||||
- **No such flag** — the skill is model-invoked and the rest of this file applies.
|
||||
|
||||
## The three-part shape
|
||||
|
||||
A model-invoked description carries exactly three things:
|
||||
|
||||
1. **Trigger clause.** When to invoke, phrased imperatively: `Use when ...`. Not `This skill ...` —
|
||||
the agent is deciding whether to act, not reading a catalogue entry.
|
||||
2. **At most one capability clause.** What it does, in one clause. Never an enumeration.
|
||||
3. **Boundary clause.** Compressed form: `Not <thing> -> <skill-name>.` The target must resolve to
|
||||
a real skill directory or agent file in the authoring source. `validate.sh` checks that
|
||||
deterministically and grades it by notation: an unresolved `/name` or arrow target is an ERROR
|
||||
and reaches the report as a Structure FAIL, while an unresolved prose-form target ("use `y`
|
||||
instead") is only a SUGGESTION unless a second target in the same sentence resolves. Take the
|
||||
script's tier as given and report it once, under Structure.
|
||||
|
||||
Everything else belongs in the body or in `README.md`.
|
||||
|
||||
## Indirect triggers — conditional, never blanket
|
||||
|
||||
Add "even if the user doesn't say X" **only where the user's natural phrasing genuinely omits the
|
||||
domain word.** True for the `gitea-*` family: people say "create an issue", not "create a Gitea
|
||||
issue". False for `git-commits`: nobody asks for a commit without saying commit. A blanket
|
||||
indirect-trigger clause on a skill whose domain word is unavoidable is padding charged to every
|
||||
session.
|
||||
|
||||
## Near-miss exclusions
|
||||
|
||||
Add a boundary clause only where a sibling skill could plausibly steal the activation. Use strong
|
||||
near-misses — queries that share keywords but need something different — not weak ones ("write a
|
||||
fibonacci function"). One boundary clause per genuine near-miss; a list of four is enumeration
|
||||
wearing a boundary's clothes.
|
||||
|
||||
## Before / after
|
||||
|
||||
```yaml
|
||||
# FAIL — enumeration first, mechanics as the opener, a blanket indirect trigger,
|
||||
# and 300+ characters of it preloaded into every session forever.
|
||||
description: >
|
||||
Analyze CSV and tabular data files — compute summary statistics, add derived
|
||||
columns, generate charts, and clean messy data. Use when the user has a CSV,
|
||||
TSV, or Excel file and wants to explore, transform, or visualize the data,
|
||||
even if they don't explicitly mention "CSV" or "analysis."
|
||||
|
||||
# PASS — trigger, one capability clause, boundary. The four verbs the FAIL
|
||||
# version enumerates are the body's job; the router cannot act on them.
|
||||
description: >
|
||||
Use when the user has a CSV, TSV, or Excel file and wants it explored,
|
||||
transformed, or charted. Not schema design -> data-model.
|
||||
```
|
||||
|
||||
(`data-model` is illustrative. In a real description the target has to resolve.)
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
---
|
||||
|
||||
# File Structure and Internal Consistency Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (optional directories, file references).
|
||||
|
||||
Read this when judging the **file-structure** and **internal-consistency** dimensions.
|
||||
|
||||
## Permitted directories
|
||||
|
||||
Only four: `scripts/`, `references/`, `assets/`, `tests/`. The specification permits additional
|
||||
directories; this house does not, because an unlisted directory is content no auditor and no host
|
||||
knows to look at. Flag any other directory as a FAIL.
|
||||
|
||||
- `scripts/` holds only executable code an agent can run, and the sourced libraries those entry
|
||||
points load. A `lib-*.sh` that is never invoked on its own belongs here beside the entry point
|
||||
that sources it — it is executable code, not documentation, so do not flag it for failing to run
|
||||
standalone. Test files (`.bats`, `*_test.*`, `test_*.sh`) there are a FAIL — they belong in
|
||||
`tests/`.
|
||||
- No non-spec files at the skill root: no `META.md`, no stray config outside the four directories.
|
||||
- An optional directory that exists must hold real content, not an unfilled placeholder README.
|
||||
|
||||
## Cross-plugin path references
|
||||
|
||||
A plugin is copied to a cache on install, and a path that climbs out of the skill directory stops
|
||||
resolving there. Flag a path in `SKILL.md`, `scripts/`, `references/` or `assets/` when it
|
||||
**resolves outside the skill directory** — an absolute repo path
|
||||
(`plugins/<plugin>/skills/<other>/` and its APM-native equivalent `.apm/skills/<other>/`), a
|
||||
plugin-root path (`docs/`, `bin/`), or a `../` chain that leaves the skill root.
|
||||
|
||||
Resolve before flagging, twice over:
|
||||
|
||||
- **Resolve the path.** `$SKILL_DIR/../assets/templates` climbs one level from a `scripts/`
|
||||
directory and lands back inside the same skill, so it resolves in a cache install and is not a
|
||||
finding. A bare `../` is not the defect; leaving the skill is.
|
||||
- **Skip fenced code blocks.** A path inside a fenced block is an example, and rubrics quote outside
|
||||
paths deliberately as negative examples of what not to write. Flag a fenced path only when the
|
||||
surrounding prose presents it as the form to copy.
|
||||
|
||||
**Referring to another skill's file.** There is one sanctioned spelling, and it is possessive:
|
||||
`skill-author's references/contract.md`. Write the skill by name and let the reader
|
||||
resolve it — do not spell the repo path. The full path is the thing this section forbids, and
|
||||
`references/contract.md` on its own is a hard ERROR from the gate, which
|
||||
requires an unqualified `references/` pointer to exist in the skill's OWN directory. The
|
||||
possessive form is the only spelling both rules accept; the gate recognises it and skips the
|
||||
on-disk check. Flag any other spelling of a cross-skill reference.
|
||||
|
||||
Two directories are exempt, and the exemptions are structural rather than discretionary:
|
||||
|
||||
- **`references/sources.md`.** Its `Research doc:` fields are development-time provenance pointers,
|
||||
not runtime references. They are expected to be unresolvable after install, so
|
||||
`validate-provenance.sh` does not treat an absent path as a FAIL — it emits an INFO naming the
|
||||
slug and stating that checks 7 and 8 did not run for it. Flagging them as broken references
|
||||
would make every correctly-provenanced skill fail.
|
||||
- **`tests/`.** Test files are dev-only and may reference repo-level infrastructure such as a shared
|
||||
`tests/test_helper/`. The exemption is conditional on the dependency being declared: if `tests/`
|
||||
exists and `tests/README.md` is absent or does not document it, that is a FAIL.
|
||||
|
||||
## Internal consistency
|
||||
|
||||
The skill has to agree with itself. Two checks:
|
||||
|
||||
- `SKILL.md`'s steps match what the scripts actually do — the arguments, the exit codes, and the
|
||||
output shape it tells the agent to expect.
|
||||
- Placeholder READMEs inside `scripts/`, `tests/` and `assets/` say the same thing about each
|
||||
directory that `SKILL.md` does.
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,136 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Finding Criteria
|
||||
|
||||
Every FAIL and SUGGESTION criterion, for every qualitative dimension, and nothing else. The
|
||||
reasoning each criterion stands on, its worked examples and its house rules stay in that
|
||||
dimension's rubric, which Step 3 loads only for a dimension this file puts in play.
|
||||
|
||||
Two rules on using it:
|
||||
|
||||
- A criterion that plainly applies is a finding. Write it up citing file and line.
|
||||
- A criterion that might apply, or whose call the wording here does not settle, is a reason to load
|
||||
that dimension's rubric — never a reason to drop the candidate. This file decides which rubrics
|
||||
to read; it does not settle a close call on its own.
|
||||
|
||||
## description — `references/skill-description-quality.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- **Over 400 characters.** Measured on the folded YAML value, not the raw source lines.
|
||||
`validate.sh` reports the number; do not re-derive it, but do point the Fix at what to cut.
|
||||
- **Internal mechanics appear in the description.** Any of:
|
||||
- capability enumeration or a feature list;
|
||||
- output-format detail ("Produces a compact findings report with Why and Fix per finding");
|
||||
- composition or architecture notes ("composes X rather than duplicating Y", "a cross-cutting
|
||||
shared skill", "the human-facing entry point", "replaces the old flat invocation");
|
||||
- implementation detail ("self-validates via a bundled deterministic script").
|
||||
|
||||
None of it can change a routing decision and all of it is preloaded.
|
||||
`Kyberforge.CompositionNote` catches the common phrasings deterministically; the rest is
|
||||
judgment. This is the rule that deflates a description, so apply it before reaching for length.
|
||||
- **The same trigger stated twice in two registers** — a verb list, then the same verbs re-quoted
|
||||
as user phrasings, usually in the same order. One register, whichever routes better.
|
||||
- **Descriptive rather than imperative phrasing** (`This skill ...`, `This is the ...`).
|
||||
`Kyberforge.DescriptionOpener` catches any opener matching `^This`.
|
||||
- **Vague capabilities** ("helps with APIs" where "parses and validates OpenAPI specs" was
|
||||
available). `Kyberforge.VagueWording` catches the known filler; imprecision outside that list is
|
||||
judgment.
|
||||
- **Trigger-list, boundary or indirect-trigger content on a hand-invoked skill** — see Step 0 of
|
||||
`references/skill-description-quality.md`.
|
||||
- **Over 1024 characters** — the agentskills.io specification ceiling, unchanged and independent
|
||||
of the 400-character house ceiling above.
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- **Over 250 characters** but at or under 400. This tier is what moves the corpus average; the FAIL
|
||||
tier only stops outliers. Report it rather than treating a 399-character description as clean.
|
||||
- A near-miss exclusion is present but targets a weak near-miss.
|
||||
- An indirect trigger is present and warranted but could name the omitted phrasing more precisely.
|
||||
|
||||
**An unresolved boundary target is not graded here.** `validate.sh` owns that call and tiers it by
|
||||
notation — `/name` or an arrow form is an ERROR, the bare prose form a SUGGESTION unless a second
|
||||
target in the same sentence resolves — and Step 1 has already filed it under `### Structure` at that
|
||||
tier. Re-grading it as a description FAIL puts one target in the report twice at two tiers. What is
|
||||
left to judgment here is semantic and the script cannot reach it: whether a target that *does*
|
||||
resolve is the right sibling to exclude, and whether a clause naming no target at all ("examine the
|
||||
files manually") should have named one.
|
||||
|
||||
## body-discipline — `references/skill-body-discipline.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A sentence answers "no" to the core test — it is padding
|
||||
- The body exceeds 900 words counted body-only (`validate.sh` reports it)
|
||||
- Two or more mutually exclusive flows are inlined instead of dispatched
|
||||
- A Gotcha paraphrases a step in the body below it that every branch reaching the Gotcha also
|
||||
reaches
|
||||
- A decision point presents a menu of options with no default
|
||||
- An instruction repeats content already in the description
|
||||
- A prescriptive sequence is used where flexibility is fine, or the reverse
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- The body exceeds 600 words counted body-only but stays at or under 900
|
||||
- The Gotchas section carries more than five entries
|
||||
- The Gotchas section exceeds 25% of the body
|
||||
- A rationale is missing from an include/exclude rule — present but unexplained
|
||||
- Gotchas are correct but placed late in the body rather than near the top
|
||||
- Content that only one branch reaches is inlined where a `references/` file would serve
|
||||
|
||||
## patterns — `references/skill-patterns.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A Gotcha entry is a general tip or a reminder rather than a fact that defies a reasonable
|
||||
assumption
|
||||
- An inner code fence is unescaped inside a markdown block, breaking the render
|
||||
- A checklist wraps a single step
|
||||
- A conditional reference gives no trigger — `Kyberforge.PaddingPhrase` reports the common form
|
||||
- The agent must produce a specific format and no output template is given
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Gotchas are correctly formed but placed late in the body
|
||||
- An output template is present but permissive where the consumer needs it exact
|
||||
- A conditional reference names a trigger that is real but broader than the branch it guards
|
||||
|
||||
## file-structure and internal-consistency — `references/skill-file-structure.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A directory outside the four permitted ones exists
|
||||
- Test files sit in `scripts/`
|
||||
- A non-spec file sits at the skill root
|
||||
- A path that resolves outside the skill directory appears outside the two exempt locations, in
|
||||
prose rather than in a fenced example
|
||||
- `tests/` exists but `tests/README.md` is missing or does not document its repo-level dependency
|
||||
- `SKILL.md` describes a script invocation the script does not accept
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- An optional directory exists but holds only a placeholder README
|
||||
|
||||
## formatting and scripts — `references/skill-formatting-and-scripts.md`
|
||||
|
||||
Flag as FAIL if:
|
||||
|
||||
- A script prompts interactively, in any form
|
||||
- A script exposes no `--help`
|
||||
- A destructive script has no `--dry-run`
|
||||
- Data and diagnostics share a stream, so the output cannot be piped
|
||||
- A relative path named in the body does not resolve
|
||||
- Heading levels are inconsistent enough to break the document's structure
|
||||
|
||||
Flag as SUGGESTION if:
|
||||
|
||||
- Exit codes are meaningful but undocumented in `--help`
|
||||
- A code block is untagged where a language applies
|
||||
- A script is idempotent in practice but does not say so, leaving a re-run's safety unclear
|
||||
- List indentation or section spacing is inconsistent without breaking the render
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Skill Flow
|
||||
|
||||
Steps 1 to 3 for a skill directory — the target Step 0 matched as a directory containing
|
||||
`SKILL.md`, or as a `SKILL.md` file, in which case `<skill-dir>` below is its parent directory.
|
||||
Work them in order, then return to `SKILL.md` Step 4 to report.
|
||||
|
||||
## Gotchas
|
||||
|
||||
- A skill takes two independent length families, and it can sit inside one while failing the other — so report them separately. The 500-line / 2,770-word pair counts the **whole file** for spec conformance. The 250/400-character and 600/900-word pair is the house context budget, and its word half counts the **body only**.
|
||||
|
||||
## Step 1 — Deterministic checks
|
||||
|
||||
Resolve all three paths against this skill's own directory so they work from a repo checkout and an installed plugin cache alike. Run exactly:
|
||||
|
||||
```bash
|
||||
bash scripts/validate.sh <skill-dir>
|
||||
bash scripts/validate-provenance.sh <skill-dir>
|
||||
bash scripts/vale-wrap.sh <skill-dir>/SKILL.md
|
||||
```
|
||||
|
||||
`validate.sh` findings become the `### Structure` dimension — its FAILs and its SUGGESTIONs both, at the tier the script assigned. Report each once; never re-grade one under another dimension. Unresolved boundary targets are where this bites, because their tier turns on notation. It exits **0** when no check failed, **1** on real findings, and **2** when it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr. Report an exit 2 as `### Structure` unverified, quoting that reason, never as a failure or a pass.
|
||||
|
||||
Read `references/skill-validation-scripts.md` when any of the three cannot run or exits non-zero for a reason other than findings, **and whenever `validate-provenance.sh` exits 0 having printed anything**. Ordinary content FAILs are the expected outcome here and need no fallback.
|
||||
|
||||
`validate-provenance.sh` reports through exit code **and** output; neither alone is the verdict. **0, silent** is a genuine pass. **0 with output** is INFO-only findings — still a `### Provenance` dimension; `references/skill-validation-scripts.md` says what each obliges — for a check-9 INFO, reading rather than relaying. **1** is FAILs plus any INFOs; it emits Why and Fix itself — surface those verbatim. **2** means it never ran — an unshaped or missing target, a missing script library, or a missing dependency, reason on stderr, often no stdout — so report `### Provenance` unverified and quote that reason. Never grade an exit 2, or an exit 0 that printed, as a clean pass.
|
||||
|
||||
`vale-wrap.sh` applies the bundled `Kyberforge` style as a prefilter. Pass no `--config`; the wrapper locates its own. Every rule is graded `error`, so every alert is a FAIL. Report each one citing its rule ID, filed under the dimension it belongs to, and do not re-derive it by judgment:
|
||||
|
||||
| Rule | Dimension |
|
||||
|---|---|
|
||||
| `Kyberforge.DescriptionOpener`, `Kyberforge.CompositionNote`, `Kyberforge.VagueWording` | description |
|
||||
| `Kyberforge.SentenceOpenerThereIs` | body-discipline |
|
||||
| `Kyberforge.PaddingPhrase` | patterns |
|
||||
|
||||
## Step 2 — Read the whole skill
|
||||
|
||||
Read `SKILL.md` and every text file under `scripts/`, `references/`, `assets/` and `tests/`. Skip binaries only — internal-consistency findings need the full picture.
|
||||
|
||||
## Step 3 — Qualitative audit
|
||||
|
||||
Read `references/skill-finding-criteria.md` first — every dimension's FAIL and SUGGESTION criteria. Load the rubric below only for a dimension the criteria put in play: one carrying a candidate finding, or one where the criterion alone does not settle the call.
|
||||
|
||||
| Dimension | Rubric |
|
||||
|---|---|
|
||||
| description | `references/skill-description-quality.md` |
|
||||
| body-discipline | `references/skill-body-discipline.md` |
|
||||
| patterns | `references/skill-patterns.md` |
|
||||
| file-structure, internal-consistency | `references/skill-file-structure.md` |
|
||||
| formatting, scripts | `references/skill-formatting-and-scripts.md` |
|
||||
|
||||
Each rubric is self-contained and grounded in the agentskills.io specification plus the house context budget. Cite file and line number for every finding.
|
||||
|
||||
Then return to `SKILL.md` Step 4.
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Formatting and Scripts Reference
|
||||
|
||||
Upstream source: agentskills.io — specification (body content), using-scripts (designing scripts
|
||||
for agentic use).
|
||||
|
||||
Read this when judging the **formatting** and **scripts** dimensions. Both are checklists of static
|
||||
criteria that never vary by skill, which is exactly why they live here rather than in the body.
|
||||
|
||||
## Formatting
|
||||
|
||||
- Heading levels are consistent: H2 for main sections, H3 for subsections. A body that jumps from
|
||||
H2 to H4, or opens on H3, reads as a fragment of a larger document.
|
||||
- Code blocks carry a language tag wherever one applies — `bash`, `markdown`, `python`, `yaml`,
|
||||
`text`. An untagged block loses syntax highlighting and, more importantly, loses the signal of
|
||||
what the agent is meant to do with it.
|
||||
- Whitespace is consistent: a blank line between sections, one list-indentation style throughout.
|
||||
- No broken relative paths in file references. Every `references/…`, `scripts/…` and `assets/…`
|
||||
path named in the body resolves against the skill directory.
|
||||
|
||||
## Scripts
|
||||
|
||||
A script in a skill is run by an agent with no terminal and no human to answer it. The criteria
|
||||
follow from that:
|
||||
|
||||
- **No interactive TTY prompts** — no `read`, no `input()`, no `readline`. A script that blocks on
|
||||
a prompt hangs the run with no diagnostic. `validate.sh` detects the common forms and reports
|
||||
them under Structure; the judgment call is any prompt it cannot pattern-match. What counts is
|
||||
where stdin comes from, not the word `read`: a `read` fed by a here-string, a here-doc, a pipe,
|
||||
or a redirect from a file never touches a terminal and is not a finding. `validate.sh` excludes
|
||||
those forms, so do not rewrite a working `read -r A B <<< "$line"` into parameter expansion to
|
||||
satisfy this rule.
|
||||
- **`--help` is exposed** and gives concise usage.
|
||||
- **Data to stdout, diagnostics to stderr.** A caller piping the script has to be able to separate
|
||||
the result from the commentary.
|
||||
- **Idempotent** — "create if not exists" rather than "create", so a re-run after a partial failure
|
||||
is safe.
|
||||
- **Meaningful exit codes, documented in `--help`.** An agent branches on the exit code; an
|
||||
undocumented one is a coin flip.
|
||||
- **`--dry-run` present for destructive operations.**
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
---
|
||||
|
||||
# Patterns Reference
|
||||
|
||||
Upstream source: agentskills.io — best-practices (instruction patterns), specification.
|
||||
|
||||
Read this when judging the **patterns** dimension: whether each instruction construct a skill uses
|
||||
is the right construct for the job and is correctly formed. Formation, not content — a Gotcha's
|
||||
*content* is judged in `references/skill-body-discipline.md`.
|
||||
|
||||
## The constructs and when each is right
|
||||
|
||||
| Construct | Right when | Wrong when |
|
||||
|---|---|---|
|
||||
| Gotchas | An environment fact contradicts a reasonable default | Used as a summary of the steps below |
|
||||
| Prescriptive sequence | The operation is fragile and flag order or exact arguments must not change | Several approaches are equally valid |
|
||||
| Checklist | A multi-step workflow the agent must complete in order | A single step dressed up as a list |
|
||||
| Conditional reference | Detail is needed on one branch only | The reference is needed on every run and is loaded blind |
|
||||
| Output template | The agent must emit a specific format a caller consumes | The output is prose nobody parses |
|
||||
|
||||
## Formation rules
|
||||
|
||||
**Gotchas** sit near the top of the body, before the steps that would otherwise walk into them.
|
||||
Placement late in the body is a SUGGESTION, not a FAIL — the content is still correct, it is just
|
||||
read after the mistake.
|
||||
|
||||
**Prescriptive sequences** that quote a fenced block inside another markdown block must escape the
|
||||
inner fence as `` \`\`\` ``. An unescaped inner fence terminates the outer block and the remaining
|
||||
instructions render as prose.
|
||||
|
||||
**Conditional references** state a specific trigger, naming a file that exists in the skill's own
|
||||
`references/` directory:
|
||||
|
||||
```text
|
||||
If the API returns a non-200 status, read `references/api-errors.md`.
|
||||
```
|
||||
|
||||
That block is fenced because the filename in it is illustrative — an unfenced `references/` pointer
|
||||
in a `SKILL.md` body must resolve on disk or the gate reports a hard ERROR. The generic
|
||||
form — pointing at the directory and hoping — defeats
|
||||
progressive disclosure, because the agent either loads everything or loads nothing.
|
||||
`Kyberforge.PaddingPhrase` catches the common generic phrasing deterministically; other malformed
|
||||
forms are judgment.
|
||||
|
||||
**Output templates** belong in the body when the agent must emit them on every run, and in
|
||||
`references/` when only one dispatch branch produces that output. A template inlined for a branch
|
||||
most invocations never take is body-discipline padding.
|
||||
|
||||
The FAIL and SUGGESTION criteria for this dimension live in `references/skill-finding-criteria.md`,
|
||||
which Step 3 loads on every run.
|
||||
@@ -0,0 +1,150 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-spec
|
||||
- agentskills-using-scripts
|
||||
---
|
||||
|
||||
# Validation Scripts Reference
|
||||
|
||||
Read this when a Step 1 script fails, cannot run, or reports something that needs interpreting —
|
||||
including `validate-provenance.sh` exiting **0 having printed something**, which is INFO findings,
|
||||
not a clean run. Its silent exit 0 is the only outcome that needs nothing here.
|
||||
|
||||
## Report the gap, do not guess
|
||||
|
||||
If a script cannot run at all — Bash denied, `python3` unavailable, PyYAML not importable, `vale`
|
||||
not installed — say so as an **INFO** finding naming the script and the missing dependency, then
|
||||
fall back to the manual checks below. An INFO never changes PASS/FAIL. Silently omitting the
|
||||
dimension a script would have covered reports a clean audit that checked less than it claims to
|
||||
have checked, and the Step 4 coverage line then names a dimension nothing actually examined.
|
||||
|
||||
## Manual structural fallback
|
||||
|
||||
`validate.sh` needs `python3` **and** PyYAML, and refuses to start without either — the description
|
||||
value has to be measured after YAML folding is resolved, so skipping these gates would be a
|
||||
vacuous pass rather than a partial one. The two are checked separately, so the message already names
|
||||
the right one — report it verbatim rather than diagnosing further:
|
||||
|
||||
```text
|
||||
Error: python3 is required but was not found on PATH.
|
||||
Error: PyYAML is required but is not importable by python3.
|
||||
```
|
||||
|
||||
Without them — or with Bash denied, or on a permission error — work this list
|
||||
by hand and file the results under `### Structure` exactly as the script's output would have been:
|
||||
|
||||
- **`name`** present, 1–64 characters, kebab-case (lowercase letters, digits and hyphens; no
|
||||
leading, trailing or doubled hyphen), and **matching the skill's directory name** exactly.
|
||||
- **`description`** present and non-empty; no unfilled `FILL IN:` placeholder in it. An absent or
|
||||
empty description is a **FAIL**, never a silent skip — it is the one field preloaded into every
|
||||
session, so a skill without one can never be routed to.
|
||||
- **Description length**, measured on the folded YAML value with newlines collapsed to single
|
||||
spaces — not on the raw block scalar, which counts indentation. 250 characters SUGGESTION, 400
|
||||
FAIL (house), 1,024 FAIL (agentskills.io spec).
|
||||
- **Body length**, counting everything after the frontmatter's closing `---`. 600 words
|
||||
SUGGESTION, 900 FAIL (house).
|
||||
- **Whole-file ceilings**, counting the file including frontmatter: 500 lines FAIL, 2,770 words
|
||||
FAIL (agentskills.io spec). These are a different measurement from the two above — report them
|
||||
as separate findings, never merged.
|
||||
- **A boundary clause is present** — either the prose form (`do not` / `instead` / `rather than` /
|
||||
`not for`) or the compressed `Not <thing> -> <name>` arrow. **SUGGESTION**, not FAIL:
|
||||
the absence is deterministic, but whether this skill warrants one is the auditor's call.
|
||||
- **Boundary targets resolve** — **FAIL** on a name that resolves to nothing. See the section
|
||||
below; resolving these by hand is the one item on this list with a procedure of its own.
|
||||
- **Every `references/<file>.md` named in the body exists on disk** — **FAIL**, not a suggestion.
|
||||
A dispatch table or "read X" trigger naming a missing file sends the agent nowhere. Ignore
|
||||
mentions inside fenced code blocks, and ignore a mention whose own line says the file is gone
|
||||
(`removed`, `deleted`, `renamed`, `superseded`, `replaced`, `obsolete`, `deprecated`, `former`,
|
||||
`gone`, `no longer`, `used to`) — that is a historical note, not a dispatch entry.
|
||||
- **Gotchas discipline**, both **SUGGESTION**. Locate the section by a heading that *is* Gotchas
|
||||
(`## Common Gotchas` counts; `## Gotcha handling` and `## Why gotchas matter` do not), running to
|
||||
the next heading at the same level or shallower. More than five top-level entries is one
|
||||
suggestion; a section over 25% of the body word count is a second, independent one. Count
|
||||
entries at column 0 only — an indented child bullet is not an entry — and ignore fenced code
|
||||
blocks for both.
|
||||
- **No unfilled `FILL IN:` placeholder** anywhere in the body.
|
||||
- **Every file in `scripts/`** carries the executable bit and contains no interactive prompt —
|
||||
no bare `read`, no `select`, nothing that blocks on a TTY.
|
||||
|
||||
## Resolving boundary targets by hand
|
||||
|
||||
Targets are read from **both** boundary forms. The compressed `Not <thing> -> <name>` arrow and the
|
||||
prose form are each parsed *and* target-checked, so a typo in prose phrasing fails exactly as an
|
||||
arrow typo does — do not check only the names after an arrow.
|
||||
|
||||
Build the universe by walking up **from the `SKILL.md` under audit**, never from the validator's own
|
||||
location. The nearest ancestor holding `plugins/*/.apm/skills/` or `plugins/*/.apm/agents/` is the
|
||||
authoring root, falling back to the nearest ancestor holding `.git`. When one is found the universe
|
||||
is every skill and agent under `<root>/plugins/*/`, plus the skill's own apm package, plus the
|
||||
packages that package declares in its `apm.yml` under `dependencies.apm`. Deployed `.claude/` and
|
||||
`.agents/` trees are consulted **only** when no authoring root exists — they are gitignored
|
||||
`apm install` output, and reading them would make a fresh clone and a developer machine disagree.
|
||||
|
||||
Three ways to read the result wrong:
|
||||
|
||||
- **A hyphenated name used attributively is not a dangling target.** "Use pre-commit hooks instead
|
||||
of ad-hoc scripts" reads as a route to `pre-commit` on wording alone. What separates a route from
|
||||
prose is grammar: a route target is terminal — followed by punctuation, a conjunction, or a
|
||||
boundary word — whereas a compound modifier is followed by the noun it modifies. A name followed
|
||||
by an ordinary noun still *confirms* a route when it exists, but never raises a FAIL on its own.
|
||||
- **A SUGGESTION-tier unresolved target is not a FAIL you may promote.** Terminal position alone is
|
||||
not evidence of a route: "run `pre-commit` instead", "see `commit-msg`" and "use the clean-up
|
||||
instead" are all terminal and all prose. A prose-form target earns a FAIL only when its own
|
||||
sentence names another target that *does* resolve; otherwise the script reports it and moves on,
|
||||
and so should you. Route notation — `/name` and `-> name` — is exempt and always FAILs, and it is
|
||||
the fix to recommend when the author did mean a route.
|
||||
- **`INFO boundary-target resolution DID NOT RUN` is not a pass.** The script prints it, and exits
|
||||
0, when no universe could be determined for that path — the usual cause being a skill copy
|
||||
audited outside its package. Report it as an INFO naming the unchecked targets and re-run against
|
||||
the real directory; filing it as clean signs off targets nothing verified.
|
||||
|
||||
## Script-specific failures
|
||||
|
||||
- **`validate-provenance.sh` printed nothing *and exited 0*.** That is a pass, not a skip — it
|
||||
exits 0 silently when the skill has no `source_keys` and no `references/sources.md`, and nothing
|
||||
to validate is not a finding. Check the exit code before you believe the silence: a target that
|
||||
is not a directory, a directory holding no `SKILL.md`, a missing or extra argument, and an absent
|
||||
`python3` all exit **2** with a message on stderr. Exit 2 means the script never ran — report it
|
||||
as an unaudited dimension, never as a pass and never as a finding. Exit 1 is findings.
|
||||
- **A check-9 INFO — `'<field>' changed for '<slug>' since <ref>` — means go read, not just relay.**
|
||||
Check 9 diffs the current `references/sources.md` against a base ref and flags a slug whose
|
||||
`Description` or `Contributing files` text differs. It is structurally incapable of telling you
|
||||
whether the new wording is still *true* — it only detects that the text changed — so when this
|
||||
INFO fires, open that slug's own entry: the document named in its `Research doc:` field, and the
|
||||
files its `Contributing files` list names. Read whichever the changed field is a claim *about* —
|
||||
a Description-only change often leaves the file list untouched, so "open the Contributing files"
|
||||
is where to look, not proof that they are what moved. Confirm by reading whether the (possibly
|
||||
strengthened) claim genuinely holds. This is the one provenance finding this script cannot verify
|
||||
for you: every other check here is a structural fact you can relay as-is, but check 9's job is
|
||||
only to tell you *where* to spend that reading effort, not to replace it. Acknowledging the INFO
|
||||
without opening those files is not auditing it. Its companion — `'<field>' removed for '<slug>'
|
||||
since <ref>` — is the same obligation in the other direction: a claim withdrawn rather than
|
||||
rewritten. No other check here requires the field, so confirm the removal was deliberate.
|
||||
- **The check-9 base ref defaults to `git merge-base HEAD origin/main`, and there are two ways to
|
||||
override it.** `--base-ref=<ref>` on the command line, or the `VALIDATE_PROVENANCE_BASE_REF`
|
||||
environment variable; the flag wins when both are given, including when it is given empty
|
||||
(`--base-ref=`), which selects the default resolution and ignores the environment. Reach for one
|
||||
on a fork, a long-lived branch, or a mirror whose remote is not called `origin` — and when a
|
||||
review asks what changed since a specific commit rather than since the branch point.
|
||||
- **A single check-9 INFO naming a whole-check skip is an unaudited dimension, not a finding about
|
||||
the skill.** There are three: "no repo root above the skill directory", "no base ref could be
|
||||
resolved", and "`<path>` is not tracked at `<ref>`". The third is the one to read carefully — it
|
||||
fires when the base ref resolved but `git show <ref>:<path>` did not, which covers both a
|
||||
genuinely new `sources.md` (nothing to flag) and a path git does not know under that name: a
|
||||
renamed skill directory, or an installed, gitignored copy such as a deployed `.claude/skills/`
|
||||
tree. Auditing the deployed copy silently checks nothing; re-run against the authoring path under
|
||||
`plugins/*/.apm/skills/`.
|
||||
- **`vale` reports `0 files`.** Treat the pass as NOT RUN, not as clean, and fall back to full
|
||||
Step 3 judgment for the dimensions it would have covered. The bundled `Kyberforge` style is
|
||||
scoped by glob in `assets/vale/.vale.ini`; a file outside those globs is silently not linted.
|
||||
- **`E100 Runtime error ... does not exist` (exit 2) from `vale-wrap.sh`.** An explicit relative
|
||||
`--config` was passed. Pass none: the wrapper locates its own `assets/vale/.vale.ini` from its
|
||||
own path, so a resolved script path plus an unresolved config path produces exactly this. Do not
|
||||
read this exit code as vale being unavailable — that misreading sends the audit down the
|
||||
fallback path while vale was installed and working the whole time.
|
||||
- **The `vale` binary is genuinely absent** (`command not found`). Report one INFO naming it, then
|
||||
fall back to full Step 3 judgment for the description, body-discipline and patterns dimensions —
|
||||
the prefilter's whole coverage. Judge those by rubric rather than dropping them.
|
||||
- **A path argument that does not exist is a hard error** in `vale-wrap.sh`, deliberately: bare
|
||||
`vale` would fall back to reading stdin and print a clean-looking `0 errors ... in stdin`, which
|
||||
the `0 files` guard above does not catch.
|
||||
@@ -0,0 +1,153 @@
|
||||
---
|
||||
source_keys:
|
||||
- agentskills-home
|
||||
- agentskills-spec
|
||||
- agentskills-best-practices
|
||||
- agentskills-optimizing-descriptions
|
||||
- agentskills-using-scripts
|
||||
- context7-websites-code-claude
|
||||
- claude-code-plugins-docs
|
||||
- claude-code-subagents-docs
|
||||
- context7-github-en-copilot
|
||||
- github-custom-agents-configuration
|
||||
---
|
||||
|
||||
# Sources
|
||||
|
||||
<!-- agentskills.io/llms.txt was used for initial source discovery and is not listed below; it contributed no skill file content directly. -->
|
||||
|
||||
## agentskills-home
|
||||
|
||||
- **URL:** https://agentskills.io/home.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Agent Skills overview — what it is, why it exists, progressive disclosure model, ecosystem of 35+ implementing tools
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-spec
|
||||
|
||||
- **URL:** https://agentskills.io/specification.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Complete SKILL.md format specification — frontmatter fields, constraints, body content, optional directories, progressive disclosure levels, file references, validation
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-description-quality.md, references/skill-patterns.md, references/skill-file-structure.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-best-practices
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/best-practices.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Best practices for skill creators — starting from real expertise, spending context wisely, calibrating control, instruction patterns (gotchas, templates, checklists, validation loops)
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-body-discipline.md, references/skill-patterns.md, references/skill-finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-optimizing-descriptions
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/optimizing-descriptions.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** How to systematically test and improve skill descriptions for triggering accuracy — eval queries, trigger rate testing, train/validation splits, optimization loop
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-description-quality.md, references/skill-finding-criteria.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-evaluating-skills
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/evaluating-skills.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Eval-driven skill quality improvement — test case design, workspace structure, assertion writing, grading, benchmarking, human review, iteration loop
|
||||
- **Contributing files:** (none — eval workflow not directly informing audit dimensions)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-using-scripts
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/using-scripts.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Using scripts in skills — one-off commands, self-contained scripts with inline dependencies, designing scripts for agentic use (no interactive prompts, --help, structured output, idempotency)
|
||||
- **Contributing files:** SKILL.md, references/skill-flow.md, references/skill-formatting-and-scripts.md, references/skill-finding-criteria.md, references/skill-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## agentskills-quickstart
|
||||
|
||||
- **URL:** https://agentskills.io/skill-creation/quickstart.md
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/agentskillsio/sources.md
|
||||
- **Description:** Step-by-step guide to creating a first skill (roll-dice example), how discovery/activation/execution work in practice
|
||||
- **Contributing files:** (none — creation guide not directly informing audit criteria)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-websites-code-claude
|
||||
|
||||
- **URL:** context7:/websites/code_claude
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code documentation site indexed by Context7 — plugin manifest schema, subagent definition types, marketplace JSON format, agent markdown file format
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-plugins-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/plugins
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code plugin authoring guide — plugin structure, manifest fields, loading methods, skill namespacing, agent activation, marketplace submission
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## claude-code-subagents-docs
|
||||
|
||||
- **URL:** https://code.claude.com/docs/en/sub-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/claude-code-plugins/sources.md
|
||||
- **Description:** Official Claude Code subagent reference — definition format, all frontmatter fields, scope priority, built-in agents, CLI flags, environment variables, known limitations
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## context7-github-en-copilot
|
||||
|
||||
- **URL:** context7:/websites/github_en_copilot
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Official GitHub Copilot documentation indexed by Context7; covers CLI plugins, custom agents, SDK, and marketplace
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-project-user.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-custom-agents-configuration
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/custom-agents-configuration
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Reference for cloud and IDE custom agent definition format — frontmatter fields, tool aliases, MCP server config, secrets interpolation, scoping hierarchy
|
||||
- **Contributing files:** SKILL.md, references/agent-flow.md, references/agent-finding-criteria.md, references/agent-field-inventory.md, references/agent-description-quality.md, references/agent-body-and-delegation.md, references/agent-scope-plugin-apm.md, references/agent-scope-project-user.md, references/agent-validation-scripts.md
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-cli-plugin-reference
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** Full CLI plugin reference — plugin.json schema, marketplace.json schema, all CLI commands and flags, install specification formats, loading precedence, env vars, LSP config
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-creating
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-creating
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating Copilot CLI plugins — plugin structure, agent and skill authoring, hooks format, MCP config, development lifecycle
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-finding-installing
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** User-facing guide to discovering and installing CLI plugins — marketplace browsing commands, install/update/uninstall workflow
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-plugins-marketplace
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-marketplace
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** How-to for creating and publishing a plugin marketplace — marketplace.json structure, hosting options, registration commands
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
|
||||
## github-sdk-custom-agents
|
||||
|
||||
- **URL:** https://docs.github.com/en/copilot/how-tos/copilot-sdk/features/custom-agents
|
||||
- **Research doc:** plugins/kyberforge/docs/research/docs/github-copilot-plugins/sources.md
|
||||
- **Description:** SDK custom agent API — CustomAgentConfig fields in all five languages, session config, sub-agent lifecycle events, tool scoping, permission handling
|
||||
- **Contributing files:** (none)
|
||||
- **Status:** `extracted`
|
||||
1104
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh
Executable file
1104
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-boundary-resolver.sh
Executable file
File diff suppressed because it is too large
Load Diff
683
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh
Executable file
683
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-agent.sh
Executable file
@@ -0,0 +1,683 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-checks-agent.sh — SOURCED, never executed.
|
||||
#
|
||||
# agent-audit's structural check suite: everything in its validate.sh that is
|
||||
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
|
||||
# resolver's markers. validate.sh reassembles
|
||||
#
|
||||
# $KYBERFORGE_AGENT_PREAMBLE_PY
|
||||
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
|
||||
# $KYBERFORGE_AGENT_BODY_PY
|
||||
#
|
||||
# in that order — the order the resolver block sat in the original file — and
|
||||
# feeds the result to python3 with the agent file as argv[1] and this scripts/
|
||||
# directory as argv[2]. argv[2] is what the preamble resolves the frontmatter
|
||||
# allowlist against, and the body's detect_scope() walk-up and provider
|
||||
# detection are unchanged from agent-audit's copy.
|
||||
#
|
||||
# The dimension vocabulary and the tiers here are agent-audit's and are
|
||||
# deliberately NOT reconciled with lib-checks-skill.sh's. Agents take the
|
||||
# ADR-0020 description gates and no body word gate at all; adding one would
|
||||
# contradict the ADR.
|
||||
#
|
||||
# Consumed by: validate.sh, agent mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
kyberforge_agent_preflight() {
|
||||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||||
# boundary — same description, two verdicts, depending on which reader ran.
|
||||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||||
# dependencies are declared the same way.
|
||||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||||
# dependency sends the reader to install the wrong thing.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
|
||||
# lib-provenance-*.sh has always exited 2 here; this matches it.
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||||
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||||
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_AGENT_PREAMBLE_PY <<'KYBERFORGE_AGENT_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
import yaml
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and this script's own message text carries em dashes (the
|
||||
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
||||
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
||||
# every check has already run, which loses the whole report and (here) flips a
|
||||
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
||||
# resolver block below pins the reads; this pins the writes.
|
||||
#
|
||||
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
||||
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
||||
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
||||
# byte-identity across all three.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
agent_file = os.path.abspath(sys.argv[1])
|
||||
script_dir = sys.argv[2]
|
||||
|
||||
fname = os.path.basename(agent_file)
|
||||
|
||||
# --- Detect provider (check .agent.md before .md) ---
|
||||
if fname.endswith('.agent.md'):
|
||||
provider = 'copilot'
|
||||
name_stem = fname[:-len('.agent.md')]
|
||||
elif fname.endswith('.md'):
|
||||
provider = 'claude-code'
|
||||
name_stem = fname[:-len('.md')]
|
||||
else:
|
||||
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# --- Load agent-field-inventory.md ---
|
||||
# The merged skill holds one references/ directory for both modes, so every
|
||||
# flow-specific file is name-prefixed and the agent half of the inventory is
|
||||
# agent-field-inventory.md (ADR-0025). There is no fallback to the pre-merge
|
||||
# `field-inventory.md` spelling: agent-audit no longer exists, so a file at that
|
||||
# name would be a stray, and silently reading it would mean auditing against an
|
||||
# inventory this skill does not ship.
|
||||
inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'agent-field-inventory.md'))
|
||||
if not os.path.isfile(inv_path):
|
||||
print(f"Error: agent-field-inventory.md not found at {inv_path}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
# Encoding is pinned to UTF-8 rather than inherited from the locale: under
|
||||
# LC_ALL=C the inherited default is ASCII, and this file legitimately carries
|
||||
# non-ASCII prose. read_text() in the shared resolver block below does the same
|
||||
# thing for every other file; this one is read before that block is defined.
|
||||
try:
|
||||
with open(inv_path, encoding='utf-8') as f:
|
||||
inv_content = f.read()
|
||||
except UnicodeDecodeError as exc:
|
||||
print(f"Error: agent-field-inventory.md at {inv_path} is not valid UTF-8 "
|
||||
f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.",
|
||||
file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
def parse_section_tokens(content, section_name):
|
||||
lines = content.splitlines()
|
||||
for i, line in enumerate(lines):
|
||||
if line.strip() == f'## {section_name}':
|
||||
for j in range(i + 1, len(lines)):
|
||||
stripped = lines[j].strip()
|
||||
if stripped and not stripped.startswith('#') and not stripped.startswith('---'):
|
||||
return set(stripped.split())
|
||||
return set()
|
||||
|
||||
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
|
||||
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
|
||||
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
|
||||
|
||||
# Tools the runtime withholds from subagents regardless of the tools field
|
||||
SUBAGENT_UNAVAILABLE_TOOLS = {
|
||||
'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers',
|
||||
}
|
||||
|
||||
# Copilot body length limit (chars) — content beyond this is silently truncated
|
||||
COPILOT_BODY_LIMIT = 30000
|
||||
|
||||
# ADR-0020 description budget. An agent's name + description is preloaded into
|
||||
# every session exactly like a skill's, so agents take the SAME description
|
||||
# gates. These two constants are DUPLICATED in three places:
|
||||
# scripts/skill-size-check.sh, lib-checks-skill.sh beside this file, and here.
|
||||
# The repo-root hook's copy cannot be shared with this skill — a cache-installed
|
||||
# plugin's scripts cannot read files outside their own plugin directory, and the
|
||||
# hook cannot reach inside the plugin. The two copies INSIDE this skill could be
|
||||
# shared (ADR-0025: two files in one skill may source a third), and are not only
|
||||
# because each mode library is a verbatim lift of the pre-merge suite whose
|
||||
# constants sit in its Python preamble; hoisting them is a separate change.
|
||||
# tests/test-skill-size-check.sh asserts all three agree, so drift fails CI
|
||||
# rather than silently diverging.
|
||||
#
|
||||
# Agents deliberately take NO body word gate, and adding one here would
|
||||
# contradict ADR-0020: a skill body is loaded into the caller's context and
|
||||
# competes with the live conversation, while an agent body becomes the system
|
||||
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
|
||||
# not transfer. Agent body length falls out of the delegation rule instead.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
|
||||
# --- Helpers (shared by every scope) ---
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def fail(msg):
|
||||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||||
# stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo
|
||||
# callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture
|
||||
# `2>&1`, so nothing a human reads moves.
|
||||
global failed
|
||||
failed = True
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
|
||||
def suggest(msg):
|
||||
suggestions.append(msg)
|
||||
|
||||
def info(msg):
|
||||
# A check that DECLINED to run says so out loud, rather than passing
|
||||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||||
print(f"INFO {msg}")
|
||||
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
|
||||
KYBERFORGE_AGENT_PREAMBLE
|
||||
KYBERFORGE_AGENT_PREAMBLE_PY="${KYBERFORGE_AGENT_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_AGENT_BODY_PY <<'KYBERFORGE_AGENT_BODY' || true
|
||||
|
||||
|
||||
def parse_frontmatter(content):
|
||||
m = FRONTMATTER_RE.match(strip_bom(content))
|
||||
if not m:
|
||||
return None, content
|
||||
return m.group(1), strip_bom(content)[m.end():]
|
||||
|
||||
def extract_field(fm, field):
|
||||
"""The raw text after `field:` ON ITS OWN LINE, or None.
|
||||
|
||||
The character class is `[^\\S\\r\\n]`, never `\\s`: under re.MULTILINE a
|
||||
`\\s*` after the colon crosses the newline, so `description:` with no value
|
||||
followed by `model: sonnet` captured `model: sonnet` as the description.
|
||||
That made the value look present, skipped the "missing or empty" failure,
|
||||
and then every ADR-0020 gate early-returned on the genuinely empty folded
|
||||
value — a valueless description exited 0 with zero output on a BLOCKING
|
||||
pre-push gate. This function is now used only for fields with no folding
|
||||
semantics (name, tools); description goes through description_value(), the
|
||||
shared resolver's YAML reader, which is the only thing that can see through
|
||||
`>`, `null`, `''` and a quoted `"description"` key alike.
|
||||
"""
|
||||
m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE)
|
||||
return m.group(1).strip() if m else None
|
||||
|
||||
def get_frontmatter_keys(fm):
|
||||
keys = set()
|
||||
for line in fm.splitlines():
|
||||
m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line)
|
||||
if m:
|
||||
keys.add(m.group(1))
|
||||
return keys
|
||||
|
||||
def agent_description(fm, local_fname):
|
||||
"""The folded description VALUE, or None if it could not be read."""
|
||||
try:
|
||||
return description_value(fm)
|
||||
except FrontmatterError as exc:
|
||||
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or
|
||||
# a description of the wrong type. Do not prefix a diagnosis here; the
|
||||
# last one named a syntax error for two failures that have none.
|
||||
fail(f"{exc} — the ADR-0020 description and boundary-target gates could "
|
||||
f"not run — {local_fname}")
|
||||
return None
|
||||
|
||||
def check_description_budget(value, local_fname, by_hand=False):
|
||||
"""ADR-0020 description gates — identical for every scope.
|
||||
|
||||
`by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent
|
||||
carrying `disable-model-invocation: true` is absent from the model-visible
|
||||
listing, so the 250-character SUGGESTION — a routing-quality budget — has
|
||||
no listing to apply to. The 400-character ceiling is unaffected.
|
||||
"""
|
||||
if not value:
|
||||
return
|
||||
dlen = len(value)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body — {local_fname}")
|
||||
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers "
|
||||
f"— {local_fname}")
|
||||
|
||||
def check_boundary(value, fpath, local_fname, by_hand=False):
|
||||
"""ADR-0020 boundary clause + resolvable boundary targets.
|
||||
|
||||
agent-author's SKILL.md states that an agent's boundary targets must
|
||||
resolve, but until this ran no script checked it — the contract was
|
||||
documented and unenforced. The resolution universe is derived from the
|
||||
AGENT FILE's own location (the authoring root above it, its own apm
|
||||
package, and that package's declared apm dependencies), never from this
|
||||
script's path, and — when an authoring root exists — never from a deployed
|
||||
.claude/ tree, so a fresh clone and a machine that has run `apm install`
|
||||
return the same verdict.
|
||||
"""
|
||||
if not value:
|
||||
return
|
||||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||||
# this particular agent warrants a boundary clause is judgment. All four
|
||||
# agents in this corpus currently lack one.
|
||||
#
|
||||
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could
|
||||
# not parse" are different findings (issue #110). And a hand-invoked agent is
|
||||
# exempt from the clause altogether (issue #108) — the boundary-target
|
||||
# resolution below still runs, because a target it DOES name should still
|
||||
# resolve.
|
||||
status = boundary_clause_status(value) if not by_hand else 'present'
|
||||
if status == 'absent':
|
||||
suggest(f"description has no boundary clause — add the prose form (\"Do not use "
|
||||
f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||||
f"so the router knows where NOT to send this agent — {local_fname}")
|
||||
elif status == 'unparsed':
|
||||
suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no "
|
||||
f"target could be read, so the dangling-target check did not run on it — "
|
||||
f"the clause is PRESENT and unparsed, not missing. Most often the target "
|
||||
f"is a single word, which is deliberately not matchable bare: write it as "
|
||||
f"`name` or /name — {local_fname}")
|
||||
if not by_hand:
|
||||
# One arrow, one target: a second name after the same arrow is resolved
|
||||
# by nothing and reported by nothing (issue #107).
|
||||
for first, second in multi_target_arrow_clauses(value):
|
||||
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
||||
f"'{second}') and only the first is resolved — the second is checked by "
|
||||
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
||||
f"Not Y -> {second}.\" — {local_fname}")
|
||||
targets = boundary_targets(value)
|
||||
if not targets:
|
||||
return
|
||||
known = known_targets(os.path.dirname(os.path.abspath(fpath)))
|
||||
if not known:
|
||||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||||
f"determined for this path (no authoring root above it, no apm package "
|
||||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||||
f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}")
|
||||
return
|
||||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||||
# route notation or its own sentence corroborates it by naming another target
|
||||
# that resolves. See the shared resolver's CORROBORATION note.
|
||||
blocking, reported = unresolved_targets(value, known)
|
||||
for target in blocking:
|
||||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||||
f"target sends the router nowhere — {local_fname}")
|
||||
for target in reported:
|
||||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||||
f"file format or an English compound. If it IS a route, write it as "
|
||||
f"`/{target}` or `-> {target}` and it will be checked properly — "
|
||||
f"{local_fname}")
|
||||
|
||||
def extract_tools_list(fm):
|
||||
"""Tool names from the `tools` field — inline scalar OR YAML block sequence.
|
||||
|
||||
Read off the PARSED mapping, never off extract_field(). That function's
|
||||
capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:`
|
||||
written as a block sequence — the shape Copilot agent files use — captured
|
||||
nothing at all and the subagent-unavailable-tool check silently stopped
|
||||
firing on exactly the files it was written for. Both spellings are legal
|
||||
YAML, so both are read here.
|
||||
"""
|
||||
try:
|
||||
data = yaml.safe_load(fm)
|
||||
except Exception:
|
||||
# Not this function's failure to report: the frontmatter's validity is
|
||||
# decided (and failed) by agent_description() on the same text.
|
||||
return set()
|
||||
if not isinstance(data, dict):
|
||||
return set()
|
||||
val = data.get('tools')
|
||||
if isinstance(val, list):
|
||||
items = [str(item).strip() for item in val]
|
||||
elif isinstance(val, str):
|
||||
items = re.split(r'[\s,]+', val.strip())
|
||||
else:
|
||||
return set()
|
||||
return {item for item in items if item}
|
||||
|
||||
def is_copilot_cloud_ide(fpath):
|
||||
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
|
||||
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
|
||||
|
||||
# --- Detect scope ---
|
||||
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
|
||||
# exactly one of the four package content types. Group 1 captures an optional
|
||||
# opening quote; \1 requires the same character (or nothing) to close it, so
|
||||
# "skill" and '"skill"' both match but a mismatched quote doesn't. The value
|
||||
# must then be followed by whitespace or end-of-line — not just a non-word
|
||||
# character — so a malformed value like `prompts-only` is correctly rejected
|
||||
# instead of false-matching on the `prompts` prefix.
|
||||
APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
|
||||
|
||||
def find_apm_package_root(apm_yml_path):
|
||||
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
|
||||
manifest, not a type:-less marketplace-only apm.yml)."""
|
||||
# errors='replace', not a hard failure: this only asks whether a `type:`
|
||||
# line exists, and a stray undecodable byte elsewhere in someone else's
|
||||
# apm.yml must not abort scope detection.
|
||||
with open(apm_yml_path, encoding='utf-8', errors='replace') as f:
|
||||
for line in f:
|
||||
if APM_TYPE_RE.match(line):
|
||||
return True
|
||||
return False
|
||||
|
||||
def detect_scope(start_dir):
|
||||
home = os.path.expanduser('~')
|
||||
original_start = os.path.abspath(start_dir)
|
||||
# Agent files conventionally live exactly two path segments below their
|
||||
# scope root — <root>/.claude/agents, <root>/.github/agents,
|
||||
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
|
||||
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
|
||||
# recovers the same root new-agent.sh would have been invoked with to
|
||||
# produce this exact file, independent of how far the walk below has to
|
||||
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
|
||||
# `root` vs `current` distinction even though validate.sh is handed a
|
||||
# file's directory, not the scope root itself.
|
||||
#
|
||||
# That arithmetic is only trustworthy when the path actually has this
|
||||
# shape: parent directory literally named "agents", grandparent one of
|
||||
# the four known scope-dir names. A hand-placed or otherwise
|
||||
# non-conventional agent file (never produced by new-agent.sh) has no
|
||||
# such guarantee — blindly trusting two-segments-up there could point at
|
||||
# an unrelated ancestor. conventional_shape gates every use of
|
||||
# conventional_root below; when it's false, the walked-to `current`
|
||||
# directory is used instead, the same fallback this function used before
|
||||
# conventional_root existed.
|
||||
scope_dir_name = os.path.basename(os.path.dirname(original_start))
|
||||
conventional_shape = (
|
||||
os.path.basename(original_start) == 'agents'
|
||||
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
|
||||
)
|
||||
conventional_root = os.path.dirname(os.path.dirname(original_start))
|
||||
current = original_start
|
||||
while True:
|
||||
# The filesystem root is never a candidate, the same guard the shared
|
||||
# resolver's walk-up loops carry. Without it a file under a marker-less
|
||||
# temp directory walked all the way to `/` and returned it as the scope
|
||||
# root, which then reported `counterpart file not found:
|
||||
# /.claude/agents/<name>.md` — a path that names someone else's machine,
|
||||
# not the user's project. When the walk runs out, the agent file's own
|
||||
# directory (or its conventional root) is the honest answer.
|
||||
if _is_fs_root(current):
|
||||
return 'project', conventional_root if conventional_shape else original_start
|
||||
apm_yml = os.path.join(current, 'apm.yml')
|
||||
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
|
||||
return 'plugin', current
|
||||
# $HOME is the user-scope boundary — checked before the .git test
|
||||
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
|
||||
# can't shadow user scope by being its own .git repo. 'user' scope
|
||||
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
|
||||
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
|
||||
# conventional two-segments-below-root depth (i.e. $HOME IS that
|
||||
# root, matching the real ~/.claude/agents or ~/.copilot/agents
|
||||
# shape). Any other walk-up into $HOME — a marker-less directory
|
||||
# nested deeper than that convention — resolves to project scope
|
||||
# instead: a stray directory under $HOME can't be silently
|
||||
# redirected into the shared global ~/.claude or ~/.copilot agent
|
||||
# directories.
|
||||
if current == home:
|
||||
if original_start == home or (conventional_shape and conventional_root == home):
|
||||
return 'user', home
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
|
||||
# in a git worktree — exists() covers both. Returns conventional_root,
|
||||
# not current: new-agent.sh's project-scope file placement always
|
||||
# uses its `$ROOT` argument directly, never the walked-up `.git`
|
||||
# location, so a <root> one or more levels below the repo's .git
|
||||
# (a subdirectory of a larger git-tracked tree — explicitly a
|
||||
# supported case per new-agent.sh's usage text) must resolve to the
|
||||
# same root new-agent.sh actually wrote to, not to the .git dir —
|
||||
# unless the path lacks the conventional shape, in which case that
|
||||
# arithmetic isn't trustworthy and current is used instead.
|
||||
if os.path.exists(os.path.join(current, '.git')):
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
parent = os.path.dirname(current)
|
||||
if parent == current:
|
||||
return 'project', conventional_root if conventional_shape else current
|
||||
current = parent
|
||||
|
||||
agent_dir = os.path.dirname(agent_file)
|
||||
scope, scope_root = detect_scope(agent_dir)
|
||||
|
||||
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
|
||||
def check_apm_agent_file(fpath, allowlist, stem):
|
||||
local_fname = os.path.basename(fpath)
|
||||
try:
|
||||
content = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
||||
f"failure, not a skip — {local_fname}")
|
||||
return
|
||||
except OSError as exc:
|
||||
# A path that cannot be opened gets a FAIL line naming it, not a bare
|
||||
# FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes
|
||||
# this path for an agent file deleted from the worktree but still
|
||||
# tracked in the index — a real, expected state, and the caller needs to
|
||||
# be told which file, not handed an interpreter stack.
|
||||
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
||||
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
fm, body = parse_frontmatter(content)
|
||||
if fm is None:
|
||||
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
||||
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
||||
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
||||
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
# The apm-agent.md template embeds its authoring guidance as HTML
|
||||
# comments inside the frontmatter block (so they render invisible in a
|
||||
# Markdown preview but stay visible in the raw file). get_frontmatter_keys
|
||||
# silently ignores any line that isn't a `key:` match, so a comment left
|
||||
# behind at ship time would otherwise pass unnoticed — yet apm compile
|
||||
# copies this frontmatter verbatim to both harnesses, and `<!-- -->` is
|
||||
# not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016).
|
||||
if re.search(r'<!--|-->', fm):
|
||||
fail(f"frontmatter still contains template HTML comments (<!-- ... -->) "
|
||||
f"— delete them before shipping — {local_fname}")
|
||||
|
||||
# Allowlist: the permitted keys are data, read at load time from
|
||||
# references/agent-field-inventory.md's `## apm-agent-allowlist` section — do not
|
||||
# restate them here, or this comment goes stale the next time that line
|
||||
# changes. apm compile verbatim-copies frontmatter to every target, so a key
|
||||
# outside the list is unsafe on at least one harness (ADR-0016). Note the
|
||||
# list admits denylist-shaped restrictions (disallowedTools) but never
|
||||
# allowlist-shaped ones (tools), whose value shape differs per harness.
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key not in allowlist:
|
||||
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
|
||||
f"({', '.join(sorted(allowlist))}) — {local_fname}")
|
||||
|
||||
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
|
||||
name_val = extract_field(fm, 'name')
|
||||
if not name_val:
|
||||
fail(f"name field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
if name_val != stem:
|
||||
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
||||
|
||||
# description — required, non-empty, no placeholder
|
||||
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
||||
# it on extract_field's raw capture is what let `description:` with no value
|
||||
# pass this gate in total silence: the capture picked up the next key, so
|
||||
# "missing or empty" never fired, and every ADR-0020 check below then
|
||||
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
||||
folded = agent_description(fm, local_fname)
|
||||
if folded is None:
|
||||
pass # frontmatter is not valid YAML — agent_description already failed
|
||||
elif not folded:
|
||||
fail(f"description field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(folded):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
by_hand = hand_invoked(fm)
|
||||
check_description_budget(folded, local_fname, by_hand)
|
||||
check_boundary(folded, fpath, local_fname, by_hand)
|
||||
|
||||
# body — required, non-empty, no placeholder; same Copilot truncation risk
|
||||
# applies since this file compiles verbatim into a real Copilot file downstream.
|
||||
if not body.strip():
|
||||
fail(f"system prompt body is empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(body):
|
||||
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
if len(body) > COPILOT_BODY_LIMIT:
|
||||
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
|
||||
f"content beyond the limit is silently truncated by the Copilot runtime "
|
||||
f"once apm compile emits it downstream — {local_fname}")
|
||||
|
||||
if scope == 'plugin':
|
||||
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
sys.exit(1 if failed else 0)
|
||||
|
||||
# --- Project/user scope: unchanged CC/Copilot pair validation ---
|
||||
|
||||
# --- Derive counterpart path ---
|
||||
if scope == 'project':
|
||||
if provider == 'claude-code':
|
||||
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
|
||||
counterpart_provider = 'copilot'
|
||||
else:
|
||||
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
|
||||
counterpart_provider = 'claude-code'
|
||||
else: # user
|
||||
home = os.path.expanduser('~')
|
||||
if provider == 'claude-code':
|
||||
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
|
||||
counterpart_provider = 'copilot'
|
||||
else:
|
||||
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
|
||||
counterpart_provider = 'claude-code'
|
||||
|
||||
def check_file(fpath, file_provider):
|
||||
local_fname = os.path.basename(fpath)
|
||||
try:
|
||||
content = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
||||
f"failure, not a skip — {local_fname}")
|
||||
return
|
||||
except OSError as exc:
|
||||
# Same reason as check_apm_agent_file's: a diagnostic naming the path
|
||||
# beats a FileNotFoundError traceback. The counterpart is pre-checked at
|
||||
# the bottom of this script, but agent_file itself never was.
|
||||
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
||||
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
fm, body = parse_frontmatter(content)
|
||||
if fm is None:
|
||||
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
||||
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
||||
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
||||
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
||||
return
|
||||
|
||||
# name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents
|
||||
cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath))
|
||||
name_val = extract_field(fm, 'name')
|
||||
if not cloud_ide:
|
||||
if not name_val:
|
||||
fail(f"name field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
# Stem check applies to Copilot CLI only; CC docs say filename need not match name
|
||||
if file_provider == 'copilot':
|
||||
stem = local_fname[:-len('.agent.md')]
|
||||
if name_val != stem:
|
||||
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
||||
elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
||||
# cloud/IDE: name is optional, but if present it must be valid
|
||||
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
||||
|
||||
# description
|
||||
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
||||
# it on extract_field's raw capture is what let `description:` with no value
|
||||
# pass this gate in total silence: the capture picked up the next key, so
|
||||
# "missing or empty" never fired, and every ADR-0020 check below then
|
||||
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
||||
folded = agent_description(fm, local_fname)
|
||||
if folded is None:
|
||||
pass # frontmatter is not valid YAML — agent_description already failed
|
||||
elif not folded:
|
||||
fail(f"description field is missing or empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(folded):
|
||||
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
by_hand = hand_invoked(fm)
|
||||
check_description_budget(folded, local_fname, by_hand)
|
||||
check_boundary(folded, fpath, local_fname, by_hand)
|
||||
|
||||
# body
|
||||
if not body.strip():
|
||||
fail(f"system prompt body is empty — {local_fname}")
|
||||
else:
|
||||
if PLACEHOLDER_RE.search(body):
|
||||
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
||||
# Copilot body length limit
|
||||
if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT:
|
||||
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}")
|
||||
|
||||
# CC-only fields in Copilot file
|
||||
if file_provider == 'copilot':
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key in cc_only_fields:
|
||||
fail(f"CC-only field '{key}' present in Copilot file — {local_fname}")
|
||||
|
||||
# Copilot-only fields in CC file
|
||||
if file_provider == 'claude-code':
|
||||
fm_keys = get_frontmatter_keys(fm)
|
||||
for key in sorted(fm_keys):
|
||||
if key in copilot_only_fields:
|
||||
fail(f"Copilot-only field '{key}' present in CC file — {local_fname}")
|
||||
|
||||
# Subagent-unavailable tools listed in tools field
|
||||
tools = extract_tools_list(fm)
|
||||
unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS
|
||||
for tool in sorted(unavailable):
|
||||
suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}")
|
||||
|
||||
# --- Check counterpart exists ---
|
||||
if not os.path.isfile(counterpart):
|
||||
fail(f"counterpart file not found: {counterpart}")
|
||||
sys.exit(1)
|
||||
|
||||
# --- Check both files ---
|
||||
check_file(agent_file, provider)
|
||||
check_file(counterpart, counterpart_provider)
|
||||
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
|
||||
sys.exit(1 if failed else 0)
|
||||
KYBERFORGE_AGENT_BODY
|
||||
KYBERFORGE_AGENT_BODY_PY="${KYBERFORGE_AGENT_BODY_PY%$'\n'}"
|
||||
621
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh
Executable file
621
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-checks-skill.sh
Executable file
@@ -0,0 +1,621 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-checks-skill.sh — SOURCED, never executed.
|
||||
#
|
||||
# skill-audit's structural check suite: everything in its validate.sh that is
|
||||
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
|
||||
# resolver's markers. validate.sh reassembles
|
||||
#
|
||||
# $KYBERFORGE_SKILL_PREAMBLE_PY
|
||||
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
|
||||
# $KYBERFORGE_SKILL_BODY_PY
|
||||
#
|
||||
# in that order — the order the resolver block sat in the original file — and
|
||||
# feeds the result to python3, so every check runs against the same names it
|
||||
# always did.
|
||||
#
|
||||
# The dimension vocabulary, the message wording and the PASS/FAIL/SUGGESTION/
|
||||
# INFO tiers here are skill-audit's and are deliberately NOT reconciled with
|
||||
# lib-checks-agent.sh's. The two suites disagree on purpose: a skill body is
|
||||
# loaded into the caller's context, an agent body becomes the system prompt of
|
||||
# a fresh one, so ADR-0020 gives skills a body word budget and agents none.
|
||||
#
|
||||
# Consumed by: validate.sh, skill mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
kyberforge_skill_preflight() {
|
||||
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
||||
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
||||
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
||||
# boundary — same description, two verdicts, depending on which reader ran.
|
||||
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
||||
# dependencies are declared the same way.
|
||||
# Check the interpreter separately from the library: `python3 -c` fails the same
|
||||
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
||||
# dependency sends the reader to install the wrong thing.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
|
||||
# lib-provenance-*.sh has always exited 2 here; this matches it.
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
||||
echo "Error: PyYAML is required but is not importable by python3." >&2
|
||||
echo " Why: skipping the ADR-0020 description, body and boundary-target gates would be a vacuous pass." >&2
|
||||
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
||||
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_SKILL_PREAMBLE_PY <<'KYBERFORGE_SKILL_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
import glob
|
||||
|
||||
import yaml
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and this script's own message text carries em dashes (the
|
||||
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
||||
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
||||
# every check has already run, which loses the whole report and (here) flips a
|
||||
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
||||
# resolver block below pins the reads; this pins the writes.
|
||||
#
|
||||
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
||||
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
||||
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
||||
# byte-identity across all three.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
skill_dir = os.path.abspath(sys.argv[1])
|
||||
skill_md = os.path.join(skill_dir, "SKILL.md")
|
||||
|
||||
if not os.path.isfile(skill_md):
|
||||
print(f"Error: '{skill_md}' not found.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
failed = False
|
||||
suggestions = []
|
||||
|
||||
def ok(msg):
|
||||
print(f"PASS {msg}")
|
||||
|
||||
def fail(msg):
|
||||
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
||||
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
||||
# stderr, everything advisory (PASS / SUGGESTION / INFO) goes to stdout.
|
||||
# Both repo callers capture `2>&1`, so nothing a human reads moves.
|
||||
global failed
|
||||
print(f"FAIL {msg}", file=sys.stderr)
|
||||
failed = True
|
||||
|
||||
def suggest(msg):
|
||||
# SUGGESTIONs are printed after every check and NEVER touch the exit code.
|
||||
# factory-audit's SKILL.md Step 4 report counts them into its
|
||||
# `PASS (N suggestions)` result line, which is what makes the ADR-0020 SUGGESTION tier visible
|
||||
# rather than another silently-ignored warning (ADR-0013).
|
||||
suggestions.append(msg)
|
||||
|
||||
def info(msg):
|
||||
# A check that DECLINED to run says so out loud, rather than passing
|
||||
# silently. Silence is what let a whole gate family go missing unnoticed.
|
||||
print(f"INFO {msg}")
|
||||
|
||||
|
||||
KYBERFORGE_SKILL_PREAMBLE
|
||||
KYBERFORGE_SKILL_PREAMBLE_PY="${KYBERFORGE_SKILL_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_SKILL_BODY_PY <<'KYBERFORGE_SKILL_BODY' || true
|
||||
|
||||
|
||||
# A leading BOM is stripped before anything is parsed or counted. It changes
|
||||
# neither count below — it is not a line separator and str.split() does not
|
||||
# treat it as whitespace — but it did defeat the frontmatter match.
|
||||
try:
|
||||
content = strip_bom(read_text(skill_md))
|
||||
except EncodingError as exc:
|
||||
fail(f"SKILL.md is {exc}. Nothing downstream can be measured, so this is a "
|
||||
f"hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
# --- Parse frontmatter ---
|
||||
fm_match = FRONTMATTER_RE.match(content)
|
||||
if not fm_match:
|
||||
fail("No parseable YAML frontmatter block found. Expected a `---` line, the "
|
||||
"fields, then a closing `---` line (a BOM, leading blank lines, trailing "
|
||||
"spaces after either marker and CRLF endings are all tolerated). Nothing "
|
||||
"downstream can be measured, so this is a hard failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
fm = fm_match.group(1)
|
||||
body_start = fm_match.end()
|
||||
|
||||
# Extract name. The character class is `[ \t]`, never `\s`: under re.MULTILINE
|
||||
# a `\s*` after the colon crosses the newline, so a valueless `name:` followed
|
||||
# by `description: ...` captured the NEXT KEY as the name and reported a
|
||||
# mismatch instead of an absence. Same class of bug as the `description:` one
|
||||
# the shared resolver's description_value() docstring records.
|
||||
name_m = re.search(r'^name:[ \t]*(\S+)', fm, re.MULTILINE)
|
||||
name = name_m.group(1).strip('"\'') if name_m else ""
|
||||
|
||||
# Extract description — the VALUE, with YAML folding resolved. Most of this
|
||||
# corpus writes descriptions as `>`-folded block scalars, so the raw lines
|
||||
# carry indentation and newlines that are not part of the value: every length
|
||||
# measurement below is wrong unless the scalar is folded first.
|
||||
try:
|
||||
desc = description_value(fm)
|
||||
except FrontmatterError as exc:
|
||||
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or a
|
||||
# description of the wrong type. Do not prefix a diagnosis here; the last
|
||||
# one named a syntax error for two failures that have none.
|
||||
fail(f"{exc}. Nothing downstream can be measured, so this is a hard "
|
||||
f"failure, not a skip")
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
|
||||
dir_name = os.path.basename(skill_dir)
|
||||
|
||||
# ADR-0020's hand-invocation carve-out (issue #108). `disable-model-invocation:
|
||||
# true` takes the skill out of the model-visible listing entirely, so the
|
||||
# trigger/capability/boundary rules and the 250-character routing target do not
|
||||
# apply to it — the audit's own references/skill-description-quality.md Step 0 says
|
||||
# so, and until this line existed no check here knew the field existed. What the
|
||||
# flag does NOT lift: the body word budget and the 400-character description
|
||||
# ceiling. See the shared resolver's hand_invoked().
|
||||
by_hand = hand_invoked(fm)
|
||||
|
||||
# --- Checks ---
|
||||
|
||||
# name present
|
||||
if name:
|
||||
ok(f"name present: '{name}'")
|
||||
else:
|
||||
fail("name field is missing or empty")
|
||||
|
||||
# name matches directory
|
||||
if name and dir_name:
|
||||
if name == dir_name:
|
||||
ok(f"name '{name}' matches directory '{dir_name}'")
|
||||
else:
|
||||
fail(f"name '{name}' does not match directory '{dir_name}'")
|
||||
|
||||
# name length
|
||||
if name:
|
||||
if len(name) <= 64:
|
||||
ok(f"name length {len(name)} chars (limit: 64)")
|
||||
else:
|
||||
fail(f"name '{name}' is {len(name)} chars — exceeds 64-character limit")
|
||||
|
||||
# name format
|
||||
if name:
|
||||
if re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name):
|
||||
ok("name format valid (kebab-case)")
|
||||
else:
|
||||
fail(f"name '{name}' is invalid — use lowercase letters, numbers, and hyphens only; no leading, trailing, or consecutive hyphens")
|
||||
|
||||
# description present
|
||||
if desc:
|
||||
ok("description present")
|
||||
else:
|
||||
fail("description field is missing or empty")
|
||||
|
||||
# description length — agentskills.io spec backstop. UNCHANGED by ADR-0020:
|
||||
# 1024 is the specification's hard limit, and the ADR-0020 budget gate below
|
||||
# sits underneath it rather than replacing it.
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen <= 1024:
|
||||
ok(f"description length {dlen} chars (agentskills.io spec limit: 1024)")
|
||||
else:
|
||||
fail(f"description length {dlen} chars — exceeds 1024-character limit")
|
||||
|
||||
# Unfilled placeholder detection — matches FILL IN: followed by actual content,
|
||||
# but not backtick-quoted references like `FILL IN:` used in instructions.
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
||||
|
||||
# description contains unfilled placeholder
|
||||
if desc and PLACEHOLDER_RE.search(desc):
|
||||
fail("description still contains 'FILL IN:' placeholder — replace before shipping")
|
||||
else:
|
||||
if desc:
|
||||
ok("description has no unfilled placeholders")
|
||||
|
||||
# --- ADR-0022: metadata.version is mandatory -------------------------------
|
||||
# FAIL, not SUGGESTION, and the tier is set by the gate rather than by taste.
|
||||
# `.pre-commit-config.yaml`'s `skill-size-check` hook REJECTS a SKILL.md with
|
||||
# no `metadata.version`, and rejects a value that is not three-part semver.
|
||||
# skill-author's Step 4 says to run this audit and "resolve every FAIL", so any
|
||||
# tier below FAIL lets that step report done on a skill the commit gate then
|
||||
# refuses — the same audit-disagrees-with-the-gate failure the MAX_LINES note
|
||||
# below warns about, arrived at from the other direction. Verified before this
|
||||
# check existed: a SKILL.md with no `metadata:` block at all reported "All
|
||||
# checks passed".
|
||||
#
|
||||
# The rule is DUPLICATED from that hook for the same cache-isolation reason as
|
||||
# every other constant here — an installed plugin's scripts cannot read the
|
||||
# repo-root config. Keep the two in step: this check must accept exactly what
|
||||
# the hook accepts.
|
||||
SEMVER_RE = re.compile(r'^\d+\.\d+\.\d+$')
|
||||
|
||||
try:
|
||||
fm_data = yaml.safe_load(fm)
|
||||
except Exception:
|
||||
# Unreachable in practice: description_value() above parses the same text
|
||||
# and hard-exits on a YAML error, so anything arriving here already parsed.
|
||||
fm_data = None
|
||||
metadata_block = fm_data.get('metadata') if isinstance(fm_data, dict) else None
|
||||
|
||||
if not isinstance(metadata_block, dict) or metadata_block.get('version') is None:
|
||||
fail("frontmatter has no metadata.version — ADR-0022 makes it mandatory for "
|
||||
"every skill, and the skill-size-check pre-commit hook rejects the file "
|
||||
"without it. Add `metadata:` / ` version: \"1.0.0\"` (new skills start "
|
||||
"at \"0.1.0\")")
|
||||
else:
|
||||
version_value = metadata_block['version']
|
||||
# NOT str()-coerced blind: `version: 1.0` is a YAML float, and its "1.0"
|
||||
# spelling is exactly the two-part value the hook rejects — coercing and
|
||||
# then matching keeps this check and the hook agreeing on that case.
|
||||
version_text = version_value if isinstance(version_value, str) else str(version_value)
|
||||
version_text = version_text.strip()
|
||||
if SEMVER_RE.match(version_text):
|
||||
ok(f"metadata.version present: '{version_text}' (ADR-0022)")
|
||||
else:
|
||||
fail(f"metadata.version '{version_text}' is not three-part semver — the "
|
||||
f"skill-size-check pre-commit hook rejects it. Use MAJOR.MINOR.PATCH, "
|
||||
f"e.g. \"1.0.0\"")
|
||||
|
||||
# SKILL.md size ceilings (agentskills.io skill-authoring.md: 500 lines,
|
||||
# ~5,000 tokens). Both constants are DUPLICATED from the repo-root pre-commit
|
||||
# hook scripts/skill-size-check.sh — a plugin skill's scripts cannot read files
|
||||
# outside the plugin directory once the plugin is cache-installed, so there is
|
||||
# no single source to share. Keep the two in sync by hand: if they drift, this
|
||||
# audit will report a skill ready to ship that the commit hook then rejects.
|
||||
MAX_LINES = 500
|
||||
# Word-count proxy for the ~5,000-token ceiling, calibrated to the densest
|
||||
# prose in the corpus (7.22 chars/word): 2770 words is ~20,000 characters,
|
||||
# ~5,000 tokens at 4 characters per token. See skill-size-check.sh's header
|
||||
# for the full measurement.
|
||||
MAX_WORDS = 2770
|
||||
|
||||
# ADR-0020 context-budget gates. DUPLICATED from scripts/skill-size-check.sh
|
||||
# for exactly the same cache-isolation reason as MAX_LINES/MAX_WORDS above, and
|
||||
# carrying the same warning — tests/test-skill-size-check.sh asserts the copies
|
||||
# agree, so drift fails CI instead of shipping an audit that disagrees with the
|
||||
# commit hook. lib-checks-agent.sh, beside this file, holds a third copy of the
|
||||
# two description constants; per ADR-0020 agents take the description gates and
|
||||
# deliberately take NO body word gate, because an agent body becomes the system
|
||||
# prompt of a fresh context rather than competing with a live conversation.
|
||||
#
|
||||
# These are NOT the same measurements as MAX_LINES/MAX_WORDS and must not be
|
||||
# unified with them: MAX_WORDS counts the WHOLE FILE including frontmatter and
|
||||
# is a spec-conformance backstop; BODY_MAX_WORDS counts the body ONLY and is a
|
||||
# quality gate. Likewise the 1024-character description limit above is the
|
||||
# agentskills.io spec ceiling and stays exactly as it is — DESC_MAX_CHARS sits
|
||||
# underneath it.
|
||||
DESC_SUGGEST_CHARS = 250
|
||||
DESC_MAX_CHARS = 400
|
||||
BODY_SUGGEST_WORDS = 600
|
||||
BODY_MAX_WORDS = 900
|
||||
|
||||
line_count = len(content.splitlines())
|
||||
if line_count <= MAX_LINES:
|
||||
ok(f"SKILL.md line count {line_count} (limit: {MAX_LINES})")
|
||||
else:
|
||||
fail(f"SKILL.md line count {line_count} — exceeds {MAX_LINES}-line limit")
|
||||
|
||||
# str.split() with no argument splits on runs of whitespace, matching the
|
||||
# `wc -w` the hook uses, and counts the whole file including frontmatter.
|
||||
word_count = len(content.split())
|
||||
if word_count <= MAX_WORDS:
|
||||
ok(f"SKILL.md word count {word_count} (limit: {MAX_WORDS}, proxy for ~5,000 tokens)")
|
||||
else:
|
||||
fail(f"SKILL.md word count {word_count} — exceeds {MAX_WORDS}-word limit (proxy for ~5,000 tokens)")
|
||||
|
||||
body = content[body_start:]
|
||||
|
||||
# --- ADR-0020: description budget -----------------------------------------
|
||||
if desc:
|
||||
dlen = len(desc)
|
||||
if dlen > DESC_MAX_CHARS:
|
||||
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
||||
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
||||
f"skill is invoked. Keep a trigger clause, at most one capability clause, "
|
||||
f"and a boundary clause; move capability enumeration, output-format detail, "
|
||||
f"composition notes and implementation detail to the body or README.md")
|
||||
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
||||
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
||||
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
||||
f"what moves the corpus average; the FAIL tier only stops outliers")
|
||||
elif by_hand:
|
||||
ok(f"description length {dlen} chars (hand-invoked: the {DESC_SUGGEST_CHARS}-character "
|
||||
f"routing target does not apply, the {DESC_MAX_CHARS}-character ceiling still does)")
|
||||
else:
|
||||
ok(f"description length {dlen} chars (ADR-0020 target: {DESC_SUGGEST_CHARS})")
|
||||
|
||||
# --- ADR-0020: body budget -------------------------------------------------
|
||||
# Counts the BODY ONLY — everything after the closing --- of the frontmatter.
|
||||
# This is a different measurement from MAX_WORDS above, which counts the whole
|
||||
# file including frontmatter as a spec-conformance backstop. Both are reported.
|
||||
body_word_count = len(body.split())
|
||||
if body_word_count > BODY_MAX_WORDS:
|
||||
fail(f"SKILL.md body is {body_word_count} words — exceeds the {BODY_MAX_WORDS}-word "
|
||||
f"ADR-0020 ceiling (body only; separate from the {MAX_WORDS}-word whole-file "
|
||||
f"limit above). Move lookup tables, spec restatements, output schemas, templates "
|
||||
f"and rationale prose to references/ behind an explicit "
|
||||
f"\"If X, read `references/file.md`\" trigger. At two or more mutually exclusive "
|
||||
f"flows, dispatch is mandatory: the body carries the dispatch table and the gates "
|
||||
f"common to every branch, each flow gets its own self-contained references/ file")
|
||||
elif body_word_count > BODY_SUGGEST_WORDS:
|
||||
suggest(f"SKILL.md body is {body_word_count} words — over the {BODY_SUGGEST_WORDS}-word "
|
||||
f"ADR-0020 target (hard fail at {BODY_MAX_WORDS})")
|
||||
else:
|
||||
ok(f"SKILL.md body word count {body_word_count} (ADR-0020 target: {BODY_SUGGEST_WORDS})")
|
||||
|
||||
# --- Reference pointers must exist -----------------------------------------
|
||||
# FAIL, not SUGGESTION: a dispatch table naming a references/ file that is not
|
||||
# on disk is a hard break, and until this check existed nothing in the
|
||||
# gate/audit/vale stack noticed it — all three exited 0.
|
||||
missing_refs = missing_reference_pointers(body, skill_dir)
|
||||
for ref in missing_refs:
|
||||
fail(f"SKILL.md body points at {ref}, which does not exist on disk — a dispatch "
|
||||
f"table or \"read X\" trigger naming a missing file sends the agent nowhere")
|
||||
if not missing_refs:
|
||||
ok("all referenced references/ files exist")
|
||||
|
||||
# --- Gotchas discipline -----------------------------------------------------
|
||||
# SUGGESTION on both counts: the measurement is deterministic, but whether a
|
||||
# given gotcha earns its place in the body is the auditor's judgment.
|
||||
gotchas = gotcha_stats(body)
|
||||
if gotchas is not None:
|
||||
gotcha_entries, gotcha_words = gotchas
|
||||
if gotcha_entries > GOTCHA_MAX_ENTRIES:
|
||||
suggest(f"Gotchas section has {gotcha_entries} entries — over the "
|
||||
f"{GOTCHA_MAX_ENTRIES}-entry guideline. A list that long is usually a "
|
||||
f"missing references/ file or a design problem written up as a warning")
|
||||
if body_word_count and gotcha_words > body_word_count * GOTCHA_MAX_BODY_FRACTION:
|
||||
suggest(f"Gotchas section is {gotcha_words} of {body_word_count} body words "
|
||||
f"({round(100.0 * gotcha_words / body_word_count)}%) — over the "
|
||||
f"{round(100.0 * GOTCHA_MAX_BODY_FRACTION)}% guideline. Move the durable "
|
||||
f"parts to references/ and keep the section for live traps")
|
||||
|
||||
# --- ADR-0020: boundary clause present -------------------------------------
|
||||
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
||||
# this particular skill warrants a boundary clause is judgment. Both accepted
|
||||
# shapes count — the prose markers and the compressed `Not <thing> -> <name>`.
|
||||
#
|
||||
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could not
|
||||
# parse" are different findings, and reporting the first for the second sends
|
||||
# the author hunting for a problem that is not there (issue #110).
|
||||
#
|
||||
# Skipped entirely for a hand-invoked skill — the contract gives it one plain
|
||||
# sentence with no boundary clause, so the finding would be wrong and its remedy
|
||||
# names a router that cannot see the skill (issue #108).
|
||||
if desc and by_hand:
|
||||
ok("hand-invoked (disable-model-invocation) — the boundary-clause and trigger "
|
||||
"rules do not apply; audited as one plain human-facing sentence")
|
||||
elif desc:
|
||||
status = boundary_clause_status(desc)
|
||||
if status == 'present':
|
||||
ok("description has a boundary clause")
|
||||
elif status == 'absent':
|
||||
suggest("description has no boundary clause — add the prose form (\"Do not use "
|
||||
"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
||||
"so the router knows where NOT to send this skill")
|
||||
else:
|
||||
suggest("description has an arrow boundary clause (\"Not X -> y\") from which no "
|
||||
"target could be read, so the dangling-target check did not run on it — "
|
||||
"the clause is PRESENT and unparsed, not missing. Most often the target is "
|
||||
"a single word, which is deliberately not matchable bare because "
|
||||
"`research`, `triage` and `forge` are all ordinary English: write it as "
|
||||
"`name` or /name")
|
||||
# One arrow, one target. A second name after the same arrow is resolved by
|
||||
# nothing and reported by nothing, so the clause claims coverage it does not
|
||||
# have and this script printed "1 of 1 boundary target(s) resolve" on a
|
||||
# clause naming two (issue #107).
|
||||
for first, second in multi_target_arrow_clauses(desc):
|
||||
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
||||
f"'{second}') and only the first is resolved — the second is checked by "
|
||||
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
||||
f"Not Y -> {second}.\"")
|
||||
|
||||
# --- ADR-0020: resolvable boundary targets ---------------------------------
|
||||
# The resolution universe comes from the SKILL's own location: the authoring
|
||||
# root above it (every sibling plugin in the monorepo), its own apm package, and
|
||||
# the packages that package declares in apm.yml dependencies.apm. It is never
|
||||
# derived from this script's own path, and — when an authoring root exists — it
|
||||
# never reads a deployed .claude/ tree, so a fresh clone and a machine that has
|
||||
# run `apm install` return the same verdict. See the shared resolver's header.
|
||||
if desc:
|
||||
routing_targets = boundary_targets(desc)
|
||||
known = known_targets(skill_dir) if routing_targets else set()
|
||||
if routing_targets and not known:
|
||||
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
||||
f"determined for this path (no authoring root above it, no apm package "
|
||||
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
||||
f"tree). Unchecked target(s): {', '.join(routing_targets)}")
|
||||
elif routing_targets:
|
||||
# blocking vs reported: a target only earns a FAIL when it is written in
|
||||
# route notation or its own sentence corroborates it by naming another
|
||||
# target that resolves. See the shared resolver's CORROBORATION note.
|
||||
unresolved, soft = unresolved_targets(desc, known)
|
||||
for target in unresolved:
|
||||
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
||||
f"target sends the router nowhere")
|
||||
for target in soft:
|
||||
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
||||
f"in this monorepo, in this package, or in a package it declares in "
|
||||
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
||||
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
||||
f"file format or an English compound. If it IS a route, write it as "
|
||||
f"`/{target}` or `-> {target}` and it will be checked properly")
|
||||
if not unresolved:
|
||||
# Counts the targets that ACTUALLY resolve, not every target found:
|
||||
# a confirm-only target (one used attributively — see the resolver's
|
||||
# ATTRIBUTIVE USE note) is exempt from the failure above, so
|
||||
# reporting it as resolved would be a false claim.
|
||||
resolved = [t for t in routing_targets if normalize_target(t) in known]
|
||||
ok(f"{len(resolved)} of {len(routing_targets)} boundary target(s) resolve: "
|
||||
f"{', '.join(resolved) if resolved else '(none)'}")
|
||||
|
||||
# Body unfilled placeholders
|
||||
fill_matches = PLACEHOLDER_RE.findall(body)
|
||||
if fill_matches:
|
||||
fail(f"SKILL.md body contains {len(fill_matches)} unfilled 'FILL IN:' placeholder(s)")
|
||||
else:
|
||||
ok("SKILL.md body has no unfilled placeholders")
|
||||
|
||||
# Interactive prompt heuristic.
|
||||
#
|
||||
# A line-initial `read` only blocks an agent when its stdin is the terminal.
|
||||
# These forms never touch a TTY and are ordinary data plumbing, so flagging
|
||||
# them is a false positive — one that has already cost two authors a
|
||||
# contorted rewrite of working source:
|
||||
#
|
||||
# read -r MODE ROOT <<< "$WALK_OUTPUT" here-string
|
||||
# read -r X <<EOF here-doc
|
||||
# read -r line < "$file" redirect from a file
|
||||
# printf '%s' "$v" | piped stdin — the pipe ends the
|
||||
# read -r X PREVIOUS line, not this one
|
||||
#
|
||||
# So a `read` is reported only when it has neither a stdin redirection on its
|
||||
# own line nor a pipe terminating the previous logical line. `read -r ANSWER`,
|
||||
# `read -p "..." X` and a bare `read` still fail, which is the case the check
|
||||
# exists for.
|
||||
def stdin_redirected(line, prev_line):
|
||||
# Quoted spans are stripped first so a `<` inside a prompt string is not
|
||||
# mistaken for a redirect: `read -p "enter <name>: " X` is interactive and
|
||||
# must still fail.
|
||||
unquoted = re.sub(r'"[^"]*"|\'[^\']*\'', '', line)
|
||||
return '<' in unquoted or prev_line.rstrip().endswith('|')
|
||||
|
||||
# A here-doc body is DATA, not command position. Every script in this corpus
|
||||
# carries a `usage() { cat <<EOF ... EOF; }`, and prose wrapped inside one puts
|
||||
# ordinary English at the start of a line — "read is reported as an INFO ..."
|
||||
# in this skill's own validate-provenance.sh, which made the skill audit
|
||||
# hard-FAIL on its own script. Reflowing that one sentence would have cleared the finding
|
||||
# and left the cause: every future usage text is one wrap away from the same
|
||||
# false positive, and the remedy an author reaches for is contorting working
|
||||
# source, which the note above records has already happened twice.
|
||||
#
|
||||
# Detection is deliberately conservative in the direction that matters. A
|
||||
# here-doc body is skipped only when its terminator is actually found further
|
||||
# down the file; an opener with no terminator — the shape a stray `<<` inside a
|
||||
# string would produce — is ignored rather than allowed to swallow the tail,
|
||||
# because swallowing the tail is a false NEGATIVE and this check exists to fail
|
||||
# closed. `<<<` here-strings open nothing and are excluded by the lookbehind.
|
||||
HEREDOC_START_RE = re.compile(r'(?<!<)<<-?\s*(["\']?)([A-Za-z_][A-Za-z0-9_]*)\1')
|
||||
|
||||
|
||||
def heredoc_delimiter(line):
|
||||
"""The here-doc terminator this line opens, or None."""
|
||||
m = HEREDOC_START_RE.search(line)
|
||||
return m.group(2) if m else None
|
||||
|
||||
|
||||
def heredoc_body_indices(lines):
|
||||
"""Line indices that are here-doc BODY (plus its terminator), not code."""
|
||||
skip = set()
|
||||
i, n = 0, len(lines)
|
||||
while i < n:
|
||||
stripped = lines[i].strip()
|
||||
delim = None if stripped.startswith('#') else heredoc_delimiter(lines[i])
|
||||
if delim:
|
||||
# `<<-` allows an indented terminator, so compare stripped.
|
||||
for j in range(i + 1, n):
|
||||
if lines[j].strip() == delim:
|
||||
skip.update(range(i + 1, j + 1))
|
||||
i = j
|
||||
break
|
||||
i += 1
|
||||
return skip
|
||||
|
||||
|
||||
# The here-doc exemption applies to the `read` heuristic ONLY, and the
|
||||
# asymmetry is the point. `read` is an ordinary English verb, so any prose a
|
||||
# script prints is one line-wrap away from opening with it. `input(` is not a
|
||||
# word — a line beginning `input(` inside a here-doc is an embedded Python
|
||||
# program pausing for a keypress, which is exactly what this check is for, and
|
||||
# these scripts embed Python in a here-doc as a matter of course. Exempting the
|
||||
# whole body would have disarmed the check across every script in the corpus.
|
||||
def interactive_reads(source):
|
||||
hits = []
|
||||
prev_line = ''
|
||||
lines = source.splitlines()
|
||||
in_heredoc = heredoc_body_indices(lines)
|
||||
for idx, line in enumerate(lines):
|
||||
stripped = line.strip()
|
||||
if idx in in_heredoc:
|
||||
if re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
continue
|
||||
if re.match(r'read(\s|$)', stripped):
|
||||
if not stdin_redirected(line, prev_line):
|
||||
hits.append(stripped)
|
||||
elif re.match(r'input\(', stripped):
|
||||
hits.append(stripped)
|
||||
# Blank lines and comments cannot carry the pipe that feeds a
|
||||
# following `read`, so they never displace the previous line.
|
||||
if stripped and not stripped.startswith('#'):
|
||||
prev_line = line
|
||||
return hits
|
||||
|
||||
# Scripts checks
|
||||
scripts_dir = os.path.join(skill_dir, "scripts")
|
||||
if os.path.isdir(scripts_dir):
|
||||
scripts = [f for f in os.listdir(scripts_dir)
|
||||
if os.path.isfile(os.path.join(scripts_dir, f)) and not f.endswith('.md')]
|
||||
for fname in scripts:
|
||||
fpath = os.path.join(scripts_dir, fname)
|
||||
try:
|
||||
sc = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
# The executable-bit check below still runs — one unreadable byte
|
||||
# must not silently drop a second, independent check.
|
||||
sc = None
|
||||
fail(f"scripts/{fname}: {exc} — it could not be scanned for "
|
||||
f"interactive prompts")
|
||||
interactive = interactive_reads(sc) if sc is not None else []
|
||||
if interactive:
|
||||
fail(f"scripts/{fname}: may use interactive input "
|
||||
f"(read/input from a terminal detected): {interactive[0]}")
|
||||
elif sc is not None:
|
||||
ok(f"scripts/{fname}: no interactive prompts detected")
|
||||
# Executable bit
|
||||
if os.access(fpath, os.X_OK):
|
||||
ok(f"scripts/{fname}: is executable")
|
||||
else:
|
||||
fail(f"scripts/{fname}: not executable — run: chmod +x {fpath}")
|
||||
|
||||
# Summary
|
||||
print()
|
||||
for s in suggestions:
|
||||
print(f"SUGGESTION {s}")
|
||||
if suggestions:
|
||||
print()
|
||||
if not failed:
|
||||
if suggestions:
|
||||
# Feeds SKILL.md Step 4's `PASS (N suggestions)` result line. A
|
||||
# SUGGESTION never changes the exit code — only a FAIL does.
|
||||
print(f"All checks passed ({len(suggestions)} suggestion(s)).")
|
||||
else:
|
||||
print("All checks passed.")
|
||||
sys.exit(0)
|
||||
else:
|
||||
print("One or more checks failed.")
|
||||
sys.exit(1)
|
||||
KYBERFORGE_SKILL_BODY
|
||||
KYBERFORGE_SKILL_BODY_PY="${KYBERFORGE_SKILL_BODY_PY%$'\n'}"
|
||||
134
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-contributing-files.sh
Executable file
134
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-contributing-files.sh
Executable file
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-contributing-files.sh — SOURCED, never executed.
|
||||
#
|
||||
# The shared Contributing-files parser, as ONE copy for this skill. Both of
|
||||
# validate-provenance.sh's modes compose it into the Python program they run.
|
||||
# Before the merge it was embedded twice. The two copies drifted once
|
||||
# (484357a) into different spellings of the bullet loop — behaviourally
|
||||
# identical, but unchecked while a docstring asserted they matched — and were
|
||||
# re-unified at 598a7c3, so they were byte-identical when ADR-0025 merged them.
|
||||
# tests/test-adr0020-contract.sh pins this file as the sole authority so that
|
||||
# a second copy cannot reappear.
|
||||
#
|
||||
# Held in a shell variable filled from a QUOTED here-doc for the same reason as
|
||||
# lib-boundary-resolver.sh's block: nothing inside is expanded, so the text
|
||||
# between the two markers below stays byte-identical to the copies the contract
|
||||
# test reads, and the markers stay on lines of their own, at column 0, exactly
|
||||
# once each, so the same sed range extracts the same span.
|
||||
#
|
||||
# The here-doc is consumed by the `read` BUILTIN rather than by `$(cat <<...)`.
|
||||
# validate-provenance.sh sources this file before either mode's python3
|
||||
# preflight, so a `cat` here made coreutils a hard dependency ahead of python3:
|
||||
# on a PATH with neither, the script exited 127 naming `cat` instead of reaching
|
||||
# the preflight that names python3. `read -r -d ''` reads to a NUL that never
|
||||
# arrives and so returns non-zero at EOF — hence the `|| true` — and it keeps the
|
||||
# last line's newline, which the joining newline in the caller would otherwise
|
||||
# double — hence the single strip after the delimiter. It removes exactly ONE
|
||||
# newline, never a run: blank lines at the end of a chunk are program text, and
|
||||
# stripping every trailing newline deleted them. The here-doc itself is
|
||||
# unchanged.
|
||||
#
|
||||
# Consumed by: validate-provenance.sh (both modes), via
|
||||
# $KYBERFORGE_CONTRIBUTING_FILES_PY.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_CONTRIBUTING_FILES_PY <<'KYBERFORGE_CONTRIBUTING_FILES' || true
|
||||
# ===== BEGIN SHARED CONTRIBUTING-FILES PARSER =====
|
||||
# ONE parser, and since ADR-0025 exactly one copy of it: this file, sourced by
|
||||
# validate-provenance.sh for both the skill and the agent flow. It used to be
|
||||
# embedded verbatim in skill-audit's and agent-audit's separate
|
||||
# validate-provenance.sh copies, because a cache-installed plugin's scripts
|
||||
# cannot read files outside their own plugin directory and no single file was
|
||||
# reachable by both skills. Merging those skills removed that constraint: two
|
||||
# files in ONE skill directory can source a third. The two copies had drifted
|
||||
# once before (cosmetically, and re-unified before the merge) while a docstring
|
||||
# claimed they had not, which is why tests/test-adr0020-contract.sh now pins
|
||||
# this file as the SOLE authority — that it exists, that validate-provenance.sh
|
||||
# sources it, and that nothing anywhere has re-inlined the parser. Do not paste
|
||||
# this block into a caller.
|
||||
#
|
||||
# Before the contract test pinned it, the agent-side copy's docstring merely
|
||||
# ASSERTED the two copies were "behaviourally identical" and nothing checked
|
||||
# it — which is how the two diverged spellings of the bullet loop went
|
||||
# unnoticed at 484357a.
|
||||
#
|
||||
# Requires: re (imported by the host script).
|
||||
|
||||
|
||||
def parse_contributing_files(content, slug):
|
||||
"""Find the Contributing files for a given slug H2 in content.
|
||||
|
||||
Both authored forms are accepted, because both are in use across the
|
||||
corpus and only recognising the first silently skipped the contributing-
|
||||
file checks on every sources.md written the other way:
|
||||
|
||||
- **Contributing files:** SKILL.md, references/a.md
|
||||
|
||||
**Contributing files:**
|
||||
- SKILL.md (what this source contributed)
|
||||
- references/a.md (what this source contributed)
|
||||
|
||||
Returns a list of paths with any trailing parenthetical note stripped.
|
||||
Note the bullet form's notes may themselves contain commas, so the list
|
||||
is built per bullet rather than by splitting the joined value.
|
||||
|
||||
The three return values are NOT interchangeable, and callers depend on
|
||||
the distinction:
|
||||
|
||||
[path, ...] the entry names contributing files
|
||||
[] the entry EXPLICITLY records "(none)"
|
||||
None the entry says nothing this parser can read
|
||||
|
||||
Only an explicit "(none)" yields []. A "Contributing files:" heading
|
||||
followed by a numbered list, by `*` bullets, or by prose parses nothing
|
||||
and returns None, never [] — a caller reads [] as a deliberate "no
|
||||
contributing files" record and SKIPS its check on that basis, so a parse
|
||||
failure returning [] would silently disable the check instead of leaving
|
||||
the unreadable entry exposed to it.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
|
||||
re.MULTILINE | re.DOTALL
|
||||
)
|
||||
m = pattern.search(content)
|
||||
if not m:
|
||||
return None
|
||||
block = m.group(1)
|
||||
|
||||
def strip_note(entry):
|
||||
# "references/a.md (why)" -> "references/a.md"
|
||||
return re.sub(r'\s*\(.*$', '', entry).strip()
|
||||
|
||||
# Inline form: value on the same line, comma-separated, no notes.
|
||||
cf_m = re.search(r'^\- \*\*Contributing files:\*\* (.+)$', block, re.MULTILINE)
|
||||
if cf_m:
|
||||
value = cf_m.group(1).strip()
|
||||
if value.startswith("(none"):
|
||||
return []
|
||||
return [p for p in (strip_note(x) for x in value.split(","))
|
||||
if p] or None
|
||||
|
||||
# Bullet form: heading on its own line, one file per following bullet.
|
||||
cf_m = re.search(r'^\*\*Contributing files:\*\*\s*$', block, re.MULTILINE)
|
||||
if not cf_m:
|
||||
return None
|
||||
files = []
|
||||
for line in block[cf_m.end():].splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
if files:
|
||||
break
|
||||
continue
|
||||
if not line.startswith("- "):
|
||||
break
|
||||
entry = line[2:].strip()
|
||||
if entry.startswith("(none"):
|
||||
return []
|
||||
entry = strip_note(entry)
|
||||
if entry:
|
||||
files.append(entry)
|
||||
return files or None
|
||||
# ===== END SHARED CONTRIBUTING-FILES PARSER =====
|
||||
KYBERFORGE_CONTRIBUTING_FILES
|
||||
KYBERFORGE_CONTRIBUTING_FILES_PY="${KYBERFORGE_CONTRIBUTING_FILES_PY%$'\n'}"
|
||||
576
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh
Executable file
576
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-agent.sh
Executable file
@@ -0,0 +1,576 @@
|
||||
#!/usr/bin/env bash
|
||||
# lib-provenance-agent.sh — SOURCED, never executed.
|
||||
#
|
||||
# agent-audit's provenance suite: its validate-provenance.sh, minus the shared
|
||||
# Contributing-files parser (lib-contributing-files.sh holds the one copy) and
|
||||
# minus the --help dispatch that validate-provenance.sh now owns. The bash
|
||||
# argument handling, the preconditions and every exit code are lifted verbatim,
|
||||
# except the extension check, which the dispatcher made unreachable (see
|
||||
# kyberforge_prov_agent_run).
|
||||
#
|
||||
# The two provenance modes have DIFFERENT exit contracts and they are NOT
|
||||
# unified. Agent mode prints NOTHING on a clean run, and exits 0 silently when
|
||||
# the scope walk-up finds no type:-bearing apm.yml above the agent file — that
|
||||
# is a verdict about a real file ("this agent is user or project scope, so
|
||||
# plugin-scope provenance does not apply"), not a rejected input, and
|
||||
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it. Skill mode
|
||||
# (lib-provenance-skill.sh) has no such verdict and instead treats exit 0 with
|
||||
# output as INFO-only findings. Neither contract may be spelled with the
|
||||
# other's codes.
|
||||
#
|
||||
# Agent mode also has no check 9, so it takes no --base-ref flag: a --base-ref
|
||||
# passed alongside an agent target is an extra argument and is rejected with
|
||||
# exit 2, exactly as before the merge.
|
||||
#
|
||||
# Consumed by: validate-provenance.sh, agent mode.
|
||||
# shellcheck shell=bash
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
kyberforge_prov_agent_usage() {
|
||||
cat <<EOF
|
||||
Usage: validate-provenance.sh <agent-file>
|
||||
|
||||
Validate that an agent's sources provenance chain is complete and internally consistent.
|
||||
Operates at plugin/APM scope only (a single vendor-neutral .apm/agents/<name>.agent.md
|
||||
inside a package with a type:-bearing apm.yml) — exits 0 silently for project and user
|
||||
scope agents.
|
||||
|
||||
Arguments:
|
||||
agent-file Path to either the Claude Code .md or Copilot .agent.md agent file.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (or nothing to validate, or not plugin scope)
|
||||
1 One or more checks failed
|
||||
2 Usage error, or the argument is not an agent file this script can read
|
||||
|
||||
An exit code of 2 is NOT a finding. SKILL.md tells the auditor to surface a
|
||||
non-zero exit as findings, so a usage error leaving exit 1 with nothing on
|
||||
stdout was indistinguishable from a clean-but-failing run. Environment and
|
||||
argument problems exit 2; only real findings exit 1.
|
||||
|
||||
Exit 2 and the silent exit 0 answer two DIFFERENT questions, and neither may
|
||||
be spelled with the other's code:
|
||||
|
||||
exit 2 the argument is not something this script can audit at all — it is
|
||||
missing, doubled, not a file, or not named .md / .agent.md. Decided
|
||||
before the scope walk-up runs, from the argument alone.
|
||||
exit 0 the argument IS a readable agent file, and the scope walk-up found
|
||||
no type:-bearing apm.yml above it before hitting the \$HOME, .git or
|
||||
filesystem-root boundary. That is a real verdict about a real file —
|
||||
"this agent is user or project scope, so plugin-scope provenance
|
||||
does not apply to it" — not a rejected input.
|
||||
|
||||
scripts/check-scope-walkup-sync.sh's fixture 6 pins the second: a real agent
|
||||
file under a \$HOME with a type-bearing apm.yml ABOVE it must exit 0 with empty
|
||||
output. Widening exit 2 to cover "the walk-up found no package" would break
|
||||
that fixture AND would be wrong on its own terms, because new-agent.sh happily
|
||||
scaffolds exactly that layout.
|
||||
|
||||
Checks performed:
|
||||
0 source_keys present in agent pair but sources.md absent
|
||||
1 FILL IN: placeholders in sources.md
|
||||
2 source_keys in agent files → slug exists in sources.md
|
||||
3 Contributing files listed in sources.md exist on disk (plugin-root
|
||||
relative). An explicit '(none)' skips silently; a Contributing files block
|
||||
this parser cannot read is reported as an INFO saying checks 3 and 4 did
|
||||
not run, never skipped silently.
|
||||
4 Contributing files back-reference the parent slug in their source_keys
|
||||
5 Research doc field present and not placeholder
|
||||
|
||||
Agent mode has no counterpart to skill mode's checks 6, 7 and 8 (Research
|
||||
doc field / upstream forward / upstream reverse are numbered 6, 7, 8 there and
|
||||
5 here): an agent at plugin scope is a single file with a plugin-root
|
||||
sources.md, so there is no references/ tree to walk and no upstream research
|
||||
source index to cross-check. parse_status() and the sources.md-basename gate
|
||||
that those checks need exist only in lib-provenance-skill.sh.
|
||||
EOF
|
||||
}
|
||||
|
||||
kyberforge_prov_agent_run() {
|
||||
# Usage and environment problems exit 2, findings exit 1. See the usage text
|
||||
# above for why the two must not share a code, and for why "not plugin scope"
|
||||
# is neither of them. This is a deliberate divergence from validate.sh, which
|
||||
# has no 2 tier for content: validate.sh always prints PASS lines, so a usage
|
||||
# error there is visibly not a findings report. This script prints NOTHING on a
|
||||
# clean run, so exit 1 plus empty stdout was the only signal a caller got
|
||||
# either way.
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: agent-file is required." >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_agent_usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Extra positional arguments were silently dropped, so a typo'd flag or a second
|
||||
# path looked like it had been honoured.
|
||||
if [[ $# -gt 1 ]]; then
|
||||
echo "Error: expected exactly one argument, got $#: $*" >&2
|
||||
echo "" >&2
|
||||
kyberforge_prov_agent_usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# python3 is a HARD dependency. Without this preflight a missing interpreter
|
||||
# produced 'line NN: python3: command not found' and exit 127 — an exit code no
|
||||
# caller maps to anything, from a message that names this script's line number
|
||||
# rather than the missing dependency.
|
||||
if ! command -v python3 > /dev/null 2>&1; then
|
||||
echo "Error: python3 is required but was not found on PATH." >&2
|
||||
echo " Why: skipping the provenance checks entirely would be a vacuous pass." >&2
|
||||
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A path that does not exist, or exists but is not a regular file, used to reach
|
||||
# the Python body, get os.path.dirname()'d into some ancestor directory and then
|
||||
# either report a silent exit 0 (no package above it) or — worse — audit a
|
||||
# DIFFERENT agent's package while naming the typo'd path. A typo'd target was
|
||||
# indistinguishable from a clean agent. vale-wrap.sh hard-errors on a
|
||||
# nonexistent path for exactly this reason.
|
||||
#
|
||||
# This is decided from the argument alone, before any walk-up runs, so it cannot
|
||||
# collide with the not-plugin-scope exit 0: that verdict is only ever reached by
|
||||
# a file that got past here.
|
||||
if [[ ! -e "$1" ]]; then
|
||||
echo "Error: no such file: $1" >&2
|
||||
echo " Why: a nonexistent target would otherwise report a silent pass." >&2
|
||||
echo " Fix: pass the path of the agent file to validate." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ ! -f "$1" ]]; then
|
||||
echo "Error: not a regular file: $1" >&2
|
||||
echo " Why: this script audits one agent file, not a directory of them, and reporting a directory as a pass hides the wrong-target mistake." >&2
|
||||
echo " Fix: pass the agent file itself — .apm/agents/<name>.agent.md — not its parent directory." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# No extension check here. The pre-merge script carried one ("unrecognized
|
||||
# extension — expected .md or .agent.md"), but validate-provenance.sh only
|
||||
# dispatches a *.agent.md, or a *.md directly under an agents/ directory, to
|
||||
# this function, and the argument-count check above guarantees $1 IS that
|
||||
# target — so the check could never fire. The "no such file" and "not a
|
||||
# regular file" checks stay: a nonexistent x.agent.md and a FIFO named
|
||||
# x.agent.md both pass the dispatcher and both still reach them.
|
||||
|
||||
# The Python program, reassembled in the order the parser block sat in before
|
||||
# the merge: preamble, shared parser, body.
|
||||
local prog="$KYBERFORGE_PROV_AGENT_PREAMBLE_PY
|
||||
$KYBERFORGE_CONTRIBUTING_FILES_PY
|
||||
$KYBERFORGE_PROV_AGENT_BODY_PY"
|
||||
|
||||
local rc=0
|
||||
python3 -u - "$1" <<< "$prog" || rc=$?
|
||||
# The findings code travels in KYBERFORGE_PROV_RC and this function returns 0,
|
||||
# so the caller can invoke it UNTESTED. See lib-provenance-skill.sh for why:
|
||||
# testing a function's status disables errexit for its whole body.
|
||||
KYBERFORGE_PROV_RC="$rc"
|
||||
return 0
|
||||
}
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_PREAMBLE_PY <<'KYBERFORGE_PROV_AGENT_PREAMBLE' || true
|
||||
import sys
|
||||
import os
|
||||
import re
|
||||
|
||||
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
||||
# default to ASCII, and every finding this script prints contains an em dash.
|
||||
# Pinning only the reads moved the crash from the read to the write — a
|
||||
# UnicodeEncodeError inside print_findings(), which loses the whole report
|
||||
# after all the checks have already run.
|
||||
for _stream in (sys.stdout, sys.stderr):
|
||||
try:
|
||||
_stream.reconfigure(encoding='utf-8')
|
||||
except AttributeError: # pragma: no cover — Python < 3.7
|
||||
pass
|
||||
|
||||
agent_file = os.path.abspath(sys.argv[1])
|
||||
agent_dir = os.path.dirname(agent_file)
|
||||
|
||||
# --- Input ----------------------------------------------------------------
|
||||
# Ported from the skill-audit copy, where the same two problems were already
|
||||
# fixed.
|
||||
#
|
||||
# read_text() pins UTF-8 explicitly instead of inheriting
|
||||
# locale.getpreferredencoding(), which is ASCII under LC_ALL=C — an ordinary em
|
||||
# dash in an agent file or in sources.md then aborted the run with a bare
|
||||
# UnicodeDecodeError traceback, or, at the one call site that wrapped its read
|
||||
# in `except Exception: return []`, reported the unreadable file as having no
|
||||
# source_keys and therefore as clean. A file that genuinely is not UTF-8 still
|
||||
# fails; it just says which file and why.
|
||||
#
|
||||
# strip_bom() runs on every read because a leading BOM defeats
|
||||
# parse_frontmatter()'s `^---` anchor, which silently disabled check 2 on a
|
||||
# BOM-prefixed agent file: no frontmatter parsed means no source_keys parsed
|
||||
# means nothing to validate.
|
||||
|
||||
|
||||
class EncodingError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def strip_bom(text):
|
||||
return text[1:] if text.startswith(u'\ufeff') else text
|
||||
|
||||
|
||||
def read_text(path):
|
||||
"""File contents as text, UTF-8 and BOM-free, with a diagnostic instead of a traceback."""
|
||||
try:
|
||||
with open(path, encoding='utf-8') as fh:
|
||||
return strip_bom(fh.read())
|
||||
except UnicodeDecodeError as exc:
|
||||
raise EncodingError(
|
||||
"not valid UTF-8 (%s at byte %d) — re-save the file as UTF-8; "
|
||||
"this gate does not guess at other encodings"
|
||||
% (exc.reason, exc.start))
|
||||
|
||||
# Matches a top-level `type:` line whose value is exactly one of the four
|
||||
# package content types — identical to validate.sh's APM_TYPE_RE. Group 1's
|
||||
# optional quote must be closed by \1 (or nothing), and the value must be
|
||||
# followed by whitespace/end-of-line so a malformed value like `prompts-only`
|
||||
# doesn't false-match on the `prompts` prefix.
|
||||
TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
|
||||
|
||||
# --- Find package root: walk up for the nearest ancestor apm.yml that
|
||||
# declares a top-level type: field. An apm.yml with no type: field is a
|
||||
# marketplace-only manifest (see monorepo-and-repo-shapes.md) — skip it and
|
||||
# keep walking. Stop at a $HOME boundary, a .git boundary, or the filesystem
|
||||
# root: none of these is plugin/APM scope, so this script has nothing to
|
||||
# check there.
|
||||
#
|
||||
# Returning None here means NOT PLUGIN SCOPE, which is a verdict, not an error:
|
||||
# the caller exits 0 silently, and scripts/check-scope-walkup-sync.sh fixture 6
|
||||
# pins that. It is deliberately NOT folded into the exit-2 tier above.
|
||||
def find_plugin_root(start_dir):
|
||||
home = os.path.expanduser('~')
|
||||
current = os.path.abspath(start_dir)
|
||||
while True:
|
||||
apm_yml = os.path.join(current, 'apm.yml')
|
||||
if os.path.isfile(apm_yml):
|
||||
# An apm.yml is a manifest this script must be able to READ to
|
||||
# classify scope at all. Under LC_ALL=C the old bare open() decoded
|
||||
# as ASCII, so a manifest with an accented author name raised
|
||||
# UnicodeDecodeError mid-walk and killed the run with a traceback.
|
||||
# It is an environment problem, not a finding, so it exits 2 rather
|
||||
# than being swallowed into a silent "no package here".
|
||||
try:
|
||||
content = read_text(apm_yml)
|
||||
except EncodingError as exc:
|
||||
print(
|
||||
"Error: %s is %s" % (apm_yml, exc),
|
||||
file=sys.stderr)
|
||||
sys.exit(2)
|
||||
if any(TYPE_RE.match(line) for line in content.splitlines()):
|
||||
return current
|
||||
# $HOME is a non-plugin-scope boundary — checked before the .git test
|
||||
# below (mirrors validate.sh's detect_scope ordering), so a
|
||||
# dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.) can't shadow
|
||||
# this check by being its own .git repo. Without this, the walk could
|
||||
# continue past $HOME toward the filesystem root looking for a
|
||||
# type-bearing apm.yml, misclassifying a user/project-scope file as
|
||||
# plugin scope in rare ancestor layouts.
|
||||
if current == home:
|
||||
return None
|
||||
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
|
||||
# in a git worktree — exists() covers both.
|
||||
if os.path.exists(os.path.join(current, '.git')):
|
||||
return None
|
||||
parent = os.path.dirname(current)
|
||||
if parent == current:
|
||||
return None
|
||||
current = parent
|
||||
|
||||
plugin_root = find_plugin_root(agent_dir)
|
||||
if plugin_root is None:
|
||||
sys.exit(0)
|
||||
|
||||
sources_md_path = os.path.join(plugin_root, 'sources.md')
|
||||
|
||||
# --- Helpers ---
|
||||
|
||||
# The trailing character class used to be CONSUMING — `[^`\n]` — so a
|
||||
# `FILL IN:` at end of line matched nothing and escaped checks 1 and 5
|
||||
# entirely. `- **Description:** FILL IN:` is the most likely spelling of a
|
||||
# half-written entry, and it was the one spelling the placeholder gate could
|
||||
# not see. The exclusion it was really expressing is "not inside backticks",
|
||||
# which a lookahead states without eating a character.
|
||||
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:(?!`)')
|
||||
|
||||
def parse_frontmatter(content):
|
||||
m = re.match(r'^---\n(.*?)\n---', content, re.DOTALL)
|
||||
if not m:
|
||||
return None, content
|
||||
return m.group(1), content[m.end():]
|
||||
|
||||
def parse_source_keys(fm):
|
||||
"""Extract top-level source_keys list from frontmatter string."""
|
||||
if fm is None:
|
||||
return []
|
||||
keys = []
|
||||
in_source_keys = False
|
||||
for line in fm.splitlines():
|
||||
if re.match(r'^source_keys:', line):
|
||||
in_source_keys = True
|
||||
continue
|
||||
if in_source_keys:
|
||||
m = re.match(r'^[ \t]+-\s+(\S+)', line)
|
||||
if m:
|
||||
keys.append(m.group(1).strip())
|
||||
elif line and not line[0].isspace():
|
||||
in_source_keys = False
|
||||
return keys
|
||||
|
||||
def parse_h2_slugs(content):
|
||||
return re.findall(r'^## (.+)$', content, re.MULTILINE)
|
||||
|
||||
KYBERFORGE_PROV_AGENT_PREAMBLE
|
||||
KYBERFORGE_PROV_AGENT_PREAMBLE_PY="${KYBERFORGE_PROV_AGENT_PREAMBLE_PY%$'\n'}"
|
||||
|
||||
IFS='' read -r -d '' KYBERFORGE_PROV_AGENT_BODY_PY <<'KYBERFORGE_PROV_AGENT_BODY' || true
|
||||
|
||||
def parse_research_docs(content, slug):
|
||||
"""Every Research doc value under a given slug H2, in document order.
|
||||
|
||||
The caller uses the first and reports the rest. Returning only the first —
|
||||
what this did before — meant a second '- **Research doc:**' line in one
|
||||
entry was silently ignored, so an author who added a doc rather than
|
||||
replacing one got check 5 run against the old value and no hint that the
|
||||
new one was never looked at.
|
||||
"""
|
||||
pattern = re.compile(
|
||||
r'^## ' + re.escape(slug) + r'\s*\n(.*?)(?=^## |\Z)',
|
||||
re.MULTILINE | re.DOTALL
|
||||
)
|
||||
m = pattern.search(content)
|
||||
if not m:
|
||||
return []
|
||||
block = m.group(1)
|
||||
return [v.strip() for v in
|
||||
re.findall(r'^\- \*\*Research doc:\*\* (.+)$', block, re.MULTILINE)]
|
||||
|
||||
findings = []
|
||||
has_fail = False
|
||||
|
||||
# A finding identical in every field is the same finding, and the same file is
|
||||
# now reached by more than one check — the agent file is read once for its own
|
||||
# source_keys and again as a contributing file, so an unreadable one would
|
||||
# otherwise be reported twice with the same words. Distinct findings about the
|
||||
# same file still both appear.
|
||||
def _record(entry):
|
||||
if entry not in findings:
|
||||
findings.append(entry)
|
||||
|
||||
def emit_fail(desc, fpath, why, fix):
|
||||
global has_fail
|
||||
has_fail = True
|
||||
_record(("FAIL", desc, fpath, why, fix, None))
|
||||
|
||||
# INFO does not set has_fail and does not change the exit code. It is for a
|
||||
# check that could not RUN — an unverified entry, not a broken one — and it
|
||||
# exists so that "did not run" is never spelled the same way as "passed".
|
||||
def emit_info(desc, fpath, note):
|
||||
_record(("INFO", desc, fpath, None, None, note))
|
||||
|
||||
def print_findings():
|
||||
for entry in findings:
|
||||
kind = entry[0]
|
||||
desc = entry[1]
|
||||
fpath = entry[2]
|
||||
why = entry[3]
|
||||
fix = entry[4]
|
||||
note = entry[5]
|
||||
if kind == "FAIL":
|
||||
print(f"FAIL {desc} — {fpath}")
|
||||
print(f" Why: {why}")
|
||||
print(f" Fix: {fix}")
|
||||
print()
|
||||
else:
|
||||
print(f"INFO {desc} — {fpath}")
|
||||
print(f" Note: {note}")
|
||||
print()
|
||||
|
||||
def emit_unreadable(rel, exc):
|
||||
"""Report a file this script cannot decode. Never a silent skip."""
|
||||
emit_fail(
|
||||
f"File is {exc}",
|
||||
rel,
|
||||
f"'{rel}' cannot be decoded, so its frontmatter — and any source_keys in it — "
|
||||
f"cannot be read. This used to be swallowed by a bare 'except Exception: return []', "
|
||||
f"which reported the unreadable file as having no source_keys and therefore as clean.",
|
||||
f"Re-save '{rel}' as UTF-8."
|
||||
)
|
||||
|
||||
# --- Collect source_keys from agent pair ---
|
||||
def get_source_keys_from_file(fpath, rel):
|
||||
if not os.path.isfile(fpath):
|
||||
return []
|
||||
try:
|
||||
content = read_text(fpath)
|
||||
except EncodingError as exc:
|
||||
emit_unreadable(rel, exc)
|
||||
return []
|
||||
fm, _ = parse_frontmatter(content)
|
||||
return parse_source_keys(fm)
|
||||
|
||||
# Plugin/APM scope is a single vendor-neutral file — no counterpart to merge.
|
||||
rel_given = os.path.relpath(agent_file, plugin_root)
|
||||
given_keys = get_source_keys_from_file(agent_file, rel_given)
|
||||
all_source_keys = given_keys
|
||||
|
||||
sources_md_exists = os.path.isfile(sources_md_path)
|
||||
|
||||
# Early exit: nothing to validate. The read above can itself raise a finding —
|
||||
# an unreadable agent file — so print before leaving; the clean case still
|
||||
# prints nothing and exits 0.
|
||||
if not all_source_keys and not sources_md_exists:
|
||||
print_findings()
|
||||
sys.exit(1 if has_fail else 0)
|
||||
|
||||
sources_content = None
|
||||
sources_slugs = set()
|
||||
if sources_md_exists:
|
||||
try:
|
||||
sources_content = read_text(sources_md_path)
|
||||
except EncodingError as exc:
|
||||
emit_unreadable("sources.md", exc)
|
||||
print_findings()
|
||||
sys.exit(1)
|
||||
sources_slugs = set(parse_h2_slugs(sources_content))
|
||||
|
||||
# --- Check 0: source_keys present but sources.md absent ---
|
||||
if not sources_md_exists and all_source_keys:
|
||||
emit_fail(
|
||||
"source_keys declared but sources.md is absent",
|
||||
rel_given,
|
||||
"source_keys references research provenance that has no sources index to validate against.",
|
||||
"Create sources.md with an H2 entry for each slug referenced by source_keys."
|
||||
)
|
||||
print_findings()
|
||||
sys.exit(1)
|
||||
|
||||
# --- Check 1: FILL IN: placeholders in sources.md ---
|
||||
for line in sources_content.splitlines():
|
||||
if PLACEHOLDER_RE.search(line):
|
||||
emit_fail(
|
||||
"Unfilled FILL IN: placeholder",
|
||||
"sources.md",
|
||||
"sources.md contains an unfilled placeholder, meaning provenance is incomplete.",
|
||||
"Replace all 'FILL IN:' values in sources.md with real content."
|
||||
)
|
||||
break
|
||||
|
||||
# --- Check 2: source_keys in the agent file → slug exists in sources.md ---
|
||||
for fpath, keys in [(agent_file, given_keys)]:
|
||||
if not keys:
|
||||
continue
|
||||
rel = os.path.relpath(fpath, plugin_root)
|
||||
for slug in keys:
|
||||
if slug not in sources_slugs:
|
||||
emit_fail(
|
||||
f"source_keys slug '{slug}' not found in sources.md",
|
||||
rel,
|
||||
f"'{rel}' declares '{slug}' as a source but there is no '## {slug}' heading in sources.md.",
|
||||
f"Add '## {slug}' entry to sources.md or remove '{slug}' from {rel} source_keys."
|
||||
)
|
||||
|
||||
# --- Checks 3, 4, 5: Per-slug checks in sources.md ---
|
||||
|
||||
# Every per-slug parser below — parse_contributing_files, parse_research_docs —
|
||||
# locates its block with pattern.search(), so a slug written twice resolves to
|
||||
# the FIRST block every time. Iterating the raw heading list therefore checked
|
||||
# the first block's fields twice and the second block's never: a duplicated slug
|
||||
# is half-validated, and looked fully validated. The duplicate is announced and
|
||||
# the repeat visit dropped.
|
||||
all_slugs = parse_h2_slugs(sources_content)
|
||||
unique_slugs = []
|
||||
for _slug in all_slugs:
|
||||
if _slug in unique_slugs:
|
||||
continue
|
||||
unique_slugs.append(_slug)
|
||||
_count = all_slugs.count(_slug)
|
||||
if _count > 1:
|
||||
emit_info(
|
||||
f"Duplicate '## {_slug}' entry in sources.md — only the first block is checked",
|
||||
f"sources.md (## {_slug})",
|
||||
f"'## {_slug}' appears {_count} times. Every field parser here takes the first match, so the "
|
||||
f"second and later blocks' Contributing files and Research doc are never validated — "
|
||||
f"checks 3, 4 and 5 did not run for them. "
|
||||
f"Merge the blocks into one entry, or give each a distinct slug and reference it from source_keys."
|
||||
)
|
||||
|
||||
for slug in unique_slugs:
|
||||
# Checks 3 and 4: Contributing files exist (paths relative to plugin root),
|
||||
# and back-reference the slug. `[]` and None are NOT the same answer here.
|
||||
# `[]` is the author writing "(none)" — there is nothing to check and the
|
||||
# skip is correct. None is a Contributing-files block this parser cannot
|
||||
# read, and skipping THAT silently disables both checks on the one entry
|
||||
# least likely to be right, which is the failure mode
|
||||
# parse_contributing_files' own docstring warns about. Say so out loud.
|
||||
cf_files = parse_contributing_files(sources_content, slug)
|
||||
if cf_files is None:
|
||||
emit_info(
|
||||
f"Contributing-file checks skipped for '{slug}' — the Contributing files block could not be parsed",
|
||||
f"sources.md (## {slug})",
|
||||
f"The '## {slug}' entry has no Contributing files list this parser can read — a missing field, a bare heading, '*' bullets, a numbered list, or prose all read as unparsable rather than as an empty declaration. "
|
||||
f"Checks 3 and 4 did not run for this slug, so nothing verified that its contributing files exist or name it back. "
|
||||
f"Write the value as '- **Contributing files:** <comma-separated paths>', or as a '**Contributing files:**' heading followed by '- ' bullets — "
|
||||
f"or record '(none)' if this source contributed no files."
|
||||
)
|
||||
elif cf_files:
|
||||
for cf_rel in cf_files:
|
||||
cf_abs = os.path.join(plugin_root, cf_rel)
|
||||
if not os.path.isfile(cf_abs):
|
||||
emit_fail(
|
||||
f"Contributing file '{cf_rel}' does not exist",
|
||||
f"sources.md (## {slug})",
|
||||
f"sources.md claims '{cf_rel}' was contributed to by slug '{slug}' but the file does not exist.",
|
||||
f"Create '{cf_rel}' relative to the plugin root, or correct the path in sources.md."
|
||||
)
|
||||
else:
|
||||
# Check 4: Bidirectional — file should list slug in its source_keys
|
||||
try:
|
||||
cf_content = read_text(cf_abs)
|
||||
except EncodingError as exc:
|
||||
emit_unreadable(cf_rel, exc)
|
||||
continue
|
||||
cf_fm, _ = parse_frontmatter(cf_content)
|
||||
cf_keys = parse_source_keys(cf_fm)
|
||||
if slug not in cf_keys:
|
||||
emit_fail(
|
||||
f"Contributing file '{cf_rel}' does not list '{slug}' in its source_keys",
|
||||
f"sources.md (## {slug})",
|
||||
f"sources.md says '{cf_rel}' was informed by '{slug}', but '{cf_rel}' does not declare '{slug}' in its top-level source_keys.",
|
||||
f"Add '{slug}' to the top-level source_keys frontmatter in '{cf_rel}'."
|
||||
)
|
||||
|
||||
# Check 5: Research doc field required
|
||||
rd_values = parse_research_docs(sources_content, slug)
|
||||
if len(rd_values) > 1:
|
||||
emit_info(
|
||||
f"Multiple '- **Research doc:**' lines for '{slug}' — only the first is used",
|
||||
f"sources.md (## {slug})",
|
||||
f"The '## {slug}' entry has {len(rd_values)} Research doc lines; check 5 ran against the first "
|
||||
f"('{rd_values[0]}') and never looked at the rest. "
|
||||
f"Keep one Research doc line per entry — if a slug genuinely came from two documents, split it into two slugs, "
|
||||
f"or name the extra document inside the first value's annotation where it is at least visible."
|
||||
)
|
||||
rd_value = rd_values[0] if rd_values else None
|
||||
if rd_value is None:
|
||||
emit_fail(
|
||||
"Research doc field missing",
|
||||
f"sources.md (## {slug})",
|
||||
f"The '## {slug}' entry in sources.md has no '- **Research doc:**' line.",
|
||||
f"Add '- **Research doc:** <path-or-(none)>' to the '## {slug}' entry in sources.md."
|
||||
)
|
||||
elif rd_value == "" or PLACEHOLDER_RE.search(rd_value):
|
||||
emit_fail(
|
||||
"Research doc field is empty or placeholder",
|
||||
f"sources.md (## {slug})",
|
||||
f"The '## {slug}' entry has an unfilled Research doc value.",
|
||||
"Set '- **Research doc:**' to a real path relative to repo root, or '(none)' if not applicable."
|
||||
)
|
||||
|
||||
print_findings()
|
||||
sys.exit(1 if has_fail else 0)
|
||||
KYBERFORGE_PROV_AGENT_BODY
|
||||
KYBERFORGE_PROV_AGENT_BODY_PY="${KYBERFORGE_PROV_AGENT_BODY_PY%$'\n'}"
|
||||
1152
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh
Executable file
1152
plugins/kyberforge/.apm/skills/factory-audit/scripts/lib-provenance-skill.sh
Executable file
File diff suppressed because it is too large
Load Diff
535
plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
Executable file
535
plugins/kyberforge/.apm/skills/factory-audit/scripts/vale-wrap.sh
Executable file
@@ -0,0 +1,535 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Works around a Vale limitation: the `text.frontmatter.description` NLP scope
|
||||
# silently stops matching once the `description:` value spans 2+ physical lines
|
||||
# in any form YAML joins back into one string — a `>`/`>-`/`>+` folded block
|
||||
# scalar (the style used by most skills/agents in this repo), a plain scalar
|
||||
# wrapped onto continuation lines, or a double- or single-quoted scalar wrapped
|
||||
# the same way. A `|`/`|-`/`|+` literal block scalar is NOT affected: its parsed
|
||||
# value keeps exactly the line breaks the source has, and vale matches it fine
|
||||
# (verified against vale 3.15.2), so literal blocks are deliberately left alone.
|
||||
# This script flattens an affected description to a one-line scalar in a scratch
|
||||
# copy — or, for the rare value no inline scalar can spell out verbatim, to a
|
||||
# `|-` literal block with a single content line, which vale matches just as well
|
||||
# (padding with blank lines so every other line number is unchanged), then
|
||||
# runs the real `vale` binary against the copies. Drop-in replacement for calling
|
||||
# `vale` directly: same args, same exit code, bar the two documented divergences
|
||||
# below.
|
||||
#
|
||||
# "Same args" means relative paths — path arguments and the values of the
|
||||
# path-valued flags (`--config`, `--output`, `--path`) alike — resolve against
|
||||
# the caller's current directory, exactly as bare `vale` resolves them. The flag
|
||||
# values are rewritten to absolute form because the run ends up `cd`'d into the
|
||||
# scratch mirror, where a relative one would no longer resolve. (An earlier
|
||||
# version resolved path arguments against the repo root, an invented convention
|
||||
# that hard-errored on `--config ../../.vale.ini` from a subdirectory and, worse,
|
||||
# silently dropped file arguments that didn't happen to resolve from the repo
|
||||
# root — skipping the flattening this script exists for.)
|
||||
#
|
||||
# Divergence 1: with no `--config` at all, this script's own sibling
|
||||
# `assets/vale/.vale.ini` is used instead of vale's upward search. pre-commit
|
||||
# prefixes only `entry[0]` with the hook-repo clone path, so a `--config` in
|
||||
# `.pre-commit-hooks.yaml` would resolve against the *consuming* repo and
|
||||
# hard-fail (E100) for every external consumer. The manifest therefore passes the
|
||||
# script alone, and an explicit `--config` from any other caller still wins.
|
||||
#
|
||||
# Divergence 2: a path-shaped argument that does not exist is a hard error
|
||||
# (exit 2). Bare vale drops it, falls back to reading stdin, and prints
|
||||
# `0 errors ... in stdin` with exit 0 — a typo'd target is then indistinguishable
|
||||
# from a clean run. Both audit skills treat a `0 files` report as NOT RUN rather
|
||||
# than clean, and `in stdin` does not match that guard, so the silent form would
|
||||
# read as "prefilter clean" and skip the LLM fallback. Erroring is the only way
|
||||
# to keep that guard honest. Linting prose piped on stdin is therefore
|
||||
# unsupported here — it already was, since the no-path handoff closes stdin so
|
||||
# vale can't block on a pipe that will never carry content.
|
||||
#
|
||||
# Vale prints each path exactly as it was handed to it, so the scratch tree
|
||||
# mirrors the caller's absolute cwd: a relative path argument is passed through
|
||||
# verbatim and resolves to its flattened copy, keeping the report byte-identical
|
||||
# to bare `vale`'s. An absolute path inside the cwd is relativized to keep that
|
||||
# property. Only an absolute path outside the cwd is rewritten to its scratch
|
||||
# copy and so reports a scratch path — unavoidable, since a file can only be
|
||||
# read from where it actually is.
|
||||
|
||||
cwd="$(pwd -P)"
|
||||
|
||||
# Every array below is expanded as `${arr[@]+"${arr[@]}"}`: bash before 4.4 —
|
||||
# including the 3.2 that macOS still ships as /bin/bash — treats `"${arr[@]}"`
|
||||
# on an empty array as an unbound variable under `set -u`. No expansion site is
|
||||
# reachable while empty on today's control flow, so this is insurance against a
|
||||
# later edit breaking that invariant, not a live fix.
|
||||
vale_args=()
|
||||
path_args=()
|
||||
pending_flag=""
|
||||
config_given=false
|
||||
|
||||
# `--output` takes either one of vale's built-in style names or a template file
|
||||
# path. Only the file form needs absolutizing, and the built-in names have to be
|
||||
# excluded by name *before* the existence test below: a file or directory
|
||||
# literally called `line` in the caller's cwd would otherwise rewrite the
|
||||
# built-in into `$cwd/line`, flipping vale into template mode (`E100 [template]
|
||||
# Runtime error`) where bare vale just uses the built-in. `--path` has no such
|
||||
# names — it is always a path — so the check is keyed on the flag too.
|
||||
is_builtin_output() {
|
||||
case "$2" in
|
||||
line|JSON|CLI) [[ "$1" == "--output" ]] ;;
|
||||
*) false ;;
|
||||
esac
|
||||
}
|
||||
# Absolutizes a `--config` value against the caller's cwd. Shared by both
|
||||
# argument forms below — separated (`--config X`) and joined (`--config=X`)
|
||||
# — so the "already absolute vs. needs $cwd prefixed" check lives in exactly
|
||||
# one place instead of being duplicated per form.
|
||||
abs_config_value() {
|
||||
if [[ "$1" == /* ]]; then
|
||||
printf '%s' "$1"
|
||||
else
|
||||
printf '%s' "$cwd/$1"
|
||||
fi
|
||||
}
|
||||
for arg in "$@"; do
|
||||
if [[ -n "$pending_flag" ]]; then
|
||||
# Value of a separated two-argv flag. It is never a lint target, however
|
||||
# file-like it looks. The run ends up `cd`'d into the scratch mirror, so a
|
||||
# value naming a file has to be absolutized here or it stops resolving.
|
||||
case "$pending_flag" in
|
||||
--config)
|
||||
# Always a path, and required to exist.
|
||||
vale_args+=("$(abs_config_value "$arg")")
|
||||
;;
|
||||
--output|--path)
|
||||
# See `is_builtin_output` above for why the built-in `--output` names
|
||||
# are excluded first. Anything that names nothing is passed through and
|
||||
# left for vale to interpret.
|
||||
if is_builtin_output "$pending_flag" "$arg"; then
|
||||
vale_args+=("$arg")
|
||||
elif [[ "$arg" != /* && -e "$arg" ]]; then
|
||||
vale_args+=("$cwd/$arg")
|
||||
else
|
||||
vale_args+=("$arg")
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
vale_args+=("$arg")
|
||||
;;
|
||||
esac
|
||||
pending_flag=""
|
||||
continue
|
||||
fi
|
||||
case "$arg" in
|
||||
--config)
|
||||
vale_args+=("$arg")
|
||||
pending_flag="$arg"
|
||||
config_given=true
|
||||
continue
|
||||
;;
|
||||
--config=*)
|
||||
vale_args+=("--config=$(abs_config_value "${arg#--config=}")")
|
||||
config_given=true
|
||||
continue
|
||||
;;
|
||||
# Same cwd-relative resolution for the `--flag=value` spelling of the two
|
||||
# other path-valued flags.
|
||||
--output=*|--path=*)
|
||||
flag_val="${arg#*=}"
|
||||
if is_builtin_output "${arg%%=*}" "$flag_val"; then
|
||||
vale_args+=("$arg")
|
||||
elif [[ "$flag_val" != /* && -n "$flag_val" && -e "$flag_val" ]]; then
|
||||
vale_args+=("${arg%%=*}=$cwd/$flag_val")
|
||||
else
|
||||
vale_args+=("$arg")
|
||||
fi
|
||||
continue
|
||||
;;
|
||||
# Vale's remaining value-taking flags, per `vale --help` (3.x). In the
|
||||
# separated two-argv form the value must not be classified as a lint target
|
||||
# — `--output tmpl.tmpl` names a real template file, and treating it as
|
||||
# input both lints the template and reorders argv so vale sees
|
||||
# `--output --no-wrap`. The `--flag=value` form needs no entry here: it
|
||||
# starts with `-` and falls through to vale untouched. A value flag added by
|
||||
# some future vale release is simply absent from this list and lands back on
|
||||
# today's behaviour, so this list going stale is never worse than not having
|
||||
# it.
|
||||
--ext|--filter|--glob|--minAlertLevel|--output|--path)
|
||||
vale_args+=("$arg")
|
||||
pending_flag="$arg"
|
||||
continue
|
||||
;;
|
||||
# Vale's subcommands are bare words that name no file, so they would trip
|
||||
# the not-found error below. A lint target literally named `sync` (no
|
||||
# extension, no slash) is misread as the subcommand — accepted, because the
|
||||
# alternative is failing every `vale-wrap.sh ls-config`.
|
||||
ls-config|ls-dirs|ls-metrics|ls-vars|sync)
|
||||
vale_args+=("$arg")
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
if [[ "$arg" == -* ]]; then
|
||||
vale_args+=("$arg")
|
||||
continue
|
||||
fi
|
||||
# Everything left is a lint target: `vale [options] [input...]` has no third
|
||||
# kind of argument. See divergence 2 above for why a missing one is fatal here.
|
||||
if [[ ! -e "$arg" ]]; then
|
||||
echo "vale-wrap.sh: no such file or directory: $arg" >&2
|
||||
exit 2
|
||||
fi
|
||||
# An absolute path inside the caller's cwd is relativized so the report cites
|
||||
# a path that resolves against the real tree. Left absolute, it would be
|
||||
# rewritten to its scratch copy and printed as `/tmp/tmp.XXXX/...` — a real
|
||||
# path to a file that is deleted on exit, which reads as a bug in any report
|
||||
# quoting it. Absolute paths outside the cwd have no relative form and keep
|
||||
# the scratch-path behaviour documented above.
|
||||
if [[ "$arg" == "$cwd"/* ]]; then
|
||||
path_args+=("${arg#"$cwd"/}")
|
||||
else
|
||||
path_args+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "$config_given" == false ]]; then
|
||||
# `scripts/../assets/vale` begins with neither `/` nor `.`, so `cd` consults
|
||||
# CDPATH for it — and when a CDPATH entry supplies the directory, `cd` PRINTS
|
||||
# the directory it chose. A bare `$(cd ... && pwd)` therefore captured TWO
|
||||
# lines, and the chosen directory could be an unrelated tree entirely: with
|
||||
# CDPATH=/tmp/decoy and /tmp/decoy/scripts present, this resolved to
|
||||
# /tmp/decoy/assets/vale and vale died on a two-line --config path. CDPATH is
|
||||
# cleared for the one command, `--` ends option parsing for a directory named
|
||||
# like a flag, and stdout is discarded so only `pwd` is captured. Same fix as
|
||||
# validate.sh and validate-provenance.sh apply to their SCRIPT_DIR.
|
||||
vale_args+=(--config "$(CDPATH='' cd -- "$(dirname "${BASH_SOURCE[0]}")/../assets/vale" > /dev/null && pwd)/.vale.ini")
|
||||
fi
|
||||
|
||||
if [[ ${#path_args[@]} -eq 0 ]]; then
|
||||
# Nothing to flatten. Hand off directly, with stdin closed so vale doesn't
|
||||
# block waiting on a pipe that will never carry content.
|
||||
exec vale ${vale_args[@]+"${vale_args[@]}"} < /dev/null
|
||||
fi
|
||||
|
||||
# `realpath -m` would be the obvious normalizer, but `-m` (canonicalize-missing)
|
||||
# is a GNU extension the BSD realpath on macOS doesn't have — and every dest
|
||||
# below is a path that doesn't exist yet. python3 is already a hard dependency.
|
||||
abspath() {
|
||||
python3 -c 'import os, sys; print(os.path.abspath(sys.argv[1]))' "$1"
|
||||
}
|
||||
|
||||
flatten() {
|
||||
# Two call shapes: `flatten src dest` (dest already resolved and inside the
|
||||
# scratch tree — the per-markdown-file calls in the directory branch below)
|
||||
# writes straight to `dest`. `flatten src raw_dest tmpdir` (the single-file
|
||||
# branch further down) additionally resolves `raw_dest` the way a separate
|
||||
# `abspath` call used to, applies the same sandbox-escape guard, and prints
|
||||
# the resolved path — folding two python3 spawns per file into one.
|
||||
python3 - "$@" <<'PYTHON'
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
src, dest_input = sys.argv[1], sys.argv[2]
|
||||
tmpdir = sys.argv[3] if len(sys.argv) > 3 else None
|
||||
|
||||
if tmpdir is None:
|
||||
dest = dest_input
|
||||
else:
|
||||
dest = os.path.abspath(dest_input)
|
||||
if not dest.startswith(tmpdir + os.sep):
|
||||
print(
|
||||
f"vale-wrap.sh: refusing to lint '{src}': its scratch copy would "
|
||||
f"land outside {tmpdir}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(2)
|
||||
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||
|
||||
# surrogateescape keeps a non-UTF-8 file (reachable via a directory argument)
|
||||
# a byte-for-byte round trip instead of aborting the whole run on a decode error.
|
||||
with open(src, encoding='utf-8', errors='surrogateescape') as fh:
|
||||
content = fh.read()
|
||||
|
||||
# YAML 1.2 double-quoted escapes (spec 5.7 / 7.3.1). `\<newline>` is handled
|
||||
# separately in unescape_double because it also swallows the next indentation.
|
||||
DQ_ESCAPES = {
|
||||
'0': '\0', 'a': '\a', 'b': '\b', 't': '\t', '\t': '\t', 'n': '\n',
|
||||
'v': '\v', 'f': '\f', 'r': '\r', 'e': '\x1b', ' ': ' ', '"': '"',
|
||||
'/': '/', '\\': '\\', 'N': '\x85', '_': '\xa0', 'L': '\u2028',
|
||||
'P': '\u2029',
|
||||
}
|
||||
|
||||
# First characters that make a plain (unquoted) scalar mean something other than
|
||||
# text: YAML's c-indicator set.
|
||||
PLAIN_UNSAFE_FIRST = '-?:,[]{}#&*!|>\'"%@`'
|
||||
|
||||
|
||||
def unescape_double(text):
|
||||
"""Decode a double-quoted YAML scalar's body to the string YAML parses."""
|
||||
out = []
|
||||
i = 0
|
||||
while i < len(text):
|
||||
char = text[i]
|
||||
if char != '\\':
|
||||
out.append(char)
|
||||
i += 1
|
||||
continue
|
||||
i += 1
|
||||
if i >= len(text):
|
||||
break
|
||||
esc = text[i]
|
||||
if esc == '\n':
|
||||
i += 1
|
||||
while i < len(text) and text[i] in ' \t':
|
||||
i += 1
|
||||
continue
|
||||
if esc in 'xuU':
|
||||
width = {'x': 2, 'u': 4, 'U': 8}[esc]
|
||||
digits = text[i + 1:i + 1 + width]
|
||||
if len(digits) == width:
|
||||
try:
|
||||
out.append(chr(int(digits, 16)))
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
i += 1 + width
|
||||
continue
|
||||
out.append(DQ_ESCAPES.get(esc, esc))
|
||||
i += 1
|
||||
return ''.join(out)
|
||||
|
||||
|
||||
def close_quote(text, quote):
|
||||
"""Index of the closing `quote` in `text`, which starts just past the
|
||||
opening one. None while the scalar is still unterminated."""
|
||||
i = 0
|
||||
while i < len(text):
|
||||
char = text[i]
|
||||
if quote == '"' and char == '\\':
|
||||
i += 2
|
||||
continue
|
||||
if char == quote:
|
||||
if quote == "'" and text[i + 1:i + 2] == "'":
|
||||
i += 2
|
||||
continue
|
||||
return i
|
||||
i += 1
|
||||
return None
|
||||
|
||||
|
||||
def continuation_lines(rest):
|
||||
"""Yield the physical lines of `rest` that continue the value started on the
|
||||
`description:` line. Indentation-based and blank-line-tolerant, per YAML:
|
||||
a blank line (any amount of whitespace) always stays inside; the indent is
|
||||
set by the first content line; the value ends at the first line indented
|
||||
less than that, at any line flush with the key (that is the next mapping
|
||||
key, not a continuation), or at EOF."""
|
||||
indent = None
|
||||
for line in rest.splitlines(keepends=True):
|
||||
text = line.rstrip('\n')
|
||||
if text.strip() == '':
|
||||
yield line
|
||||
continue
|
||||
line_indent = len(text) - len(text.lstrip(' \t'))
|
||||
if line_indent == 0:
|
||||
return
|
||||
if indent is None:
|
||||
indent = line_indent
|
||||
elif line_indent < indent:
|
||||
return
|
||||
yield line
|
||||
|
||||
|
||||
def emit(value):
|
||||
"""Render `value` as a YAML scalar whose source text spells the value out
|
||||
verbatim. Vale locates the description by matching the parsed value back
|
||||
against the source, so a scalar carrying any escape — `''` in a
|
||||
single-quoted scalar, `\\"` or `\\\\` in a double-quoted one — makes the
|
||||
whole `text.frontmatter.description` scope vanish, the same failure this
|
||||
script exists to work around. Verbatim forms only, therefore, tried in
|
||||
descending order of fidelity. The first three occupy one physical line; the
|
||||
`|-` fallback occupies two, which the caller accounts for when padding."""
|
||||
if (value
|
||||
and value[0] not in PLAIN_UNSAFE_FIRST
|
||||
and ': ' not in value
|
||||
and not value.endswith(':')
|
||||
and ' #' not in value):
|
||||
return value # plain: nothing needs escaping at all
|
||||
if "'" not in value:
|
||||
return "'" + value + "'" # single-quoted: only `'` would escape
|
||||
if '"' not in value and '\\' not in value:
|
||||
return '"' + value + '"' # double-quoted: only `"`/`\` would
|
||||
# Last resort: the value needs quoting AND holds an apostrophe AND a double
|
||||
# quote or backslash, so no *inline* scalar can carry it verbatim. A `|-`
|
||||
# literal block can — a block scalar's body has no escape syntax at all, so
|
||||
# `'`, `"`, `\` and `: ` all survive byte for byte, and vale still matches
|
||||
# the description scope against it (the header above says the same of the
|
||||
# `|` blocks this script deliberately leaves alone; verified against vale
|
||||
# 3.15.2). One content line, indented two spaces, `-`-chomped so the parsed
|
||||
# value is exactly `value` with no trailing newline.
|
||||
return '|-\n ' + value
|
||||
|
||||
|
||||
fm_match = re.match(r'^(---\n)(.*?\n)(---\n)', content, re.DOTALL)
|
||||
if fm_match:
|
||||
fm = fm_match.group(2)
|
||||
header_m = re.search(r'^description:[ \t]*', fm, re.MULTILINE)
|
||||
else:
|
||||
header_m = None
|
||||
|
||||
if header_m:
|
||||
head_start = header_m.start()
|
||||
value_start = header_m.end()
|
||||
header_end = fm.find('\n', value_start)
|
||||
header_end = len(fm) if header_end == -1 else header_end
|
||||
first = fm[value_start:header_end]
|
||||
body_start = header_end + 1
|
||||
indicator = first.rstrip()
|
||||
|
||||
block_m = re.fullmatch(r'([|>])([+-]?[0-9]*|[0-9]*[+-]?)', indicator)
|
||||
if block_m and block_m.group(1) == '|':
|
||||
kind = None # literal blocks keep their line breaks; vale is fine
|
||||
elif block_m:
|
||||
kind = 'block' # folded (`>`): the value starts on the next line
|
||||
elif indicator == '':
|
||||
kind = 'block' # bare `description:`: a plain scalar on later lines
|
||||
elif first[:1] == '"':
|
||||
kind = 'double'
|
||||
elif first[:1] == "'":
|
||||
kind = 'single'
|
||||
elif first[:1] in '#&*!':
|
||||
kind = None # comment, anchor, alias or tag — not a plain scalar
|
||||
else:
|
||||
kind = 'plain'
|
||||
|
||||
text = ''
|
||||
value_end = value_start
|
||||
value_lines = 0
|
||||
if kind in ('block', 'plain'):
|
||||
body = ''.join(continuation_lines(fm[body_start:]))
|
||||
value_end = body_start + len(body)
|
||||
if kind == 'block':
|
||||
text = body
|
||||
value_lines = body.count('\n')
|
||||
else:
|
||||
text = fm[value_start:value_end]
|
||||
value_lines = 1 + body.count('\n')
|
||||
if ' #' in text or text.lstrip().startswith('#'):
|
||||
# A `#` opens a comment inside a plain scalar. Folding it in
|
||||
# would lint text YAML never treats as part of the value, so
|
||||
# leave the file alone rather than lint the wrong string.
|
||||
kind = None
|
||||
elif kind in ('double', 'single'):
|
||||
quote = '"' if kind == 'double' else "'"
|
||||
inner_start = value_start + 1
|
||||
acc = fm[inner_start:body_start]
|
||||
idx = close_quote(acc, quote)
|
||||
lines = continuation_lines(fm[body_start:])
|
||||
while idx is None:
|
||||
try:
|
||||
acc += next(lines)
|
||||
except StopIteration:
|
||||
break
|
||||
idx = close_quote(acc, quote)
|
||||
if idx is None:
|
||||
kind = None # unterminated quote: invalid YAML, leave it to vale
|
||||
else:
|
||||
inner = acc[:idx]
|
||||
value_end = inner_start + idx + 1
|
||||
text = unescape_double(inner) if quote == '"' else inner.replace("''", "'")
|
||||
value_lines = 1 + inner.count('\n')
|
||||
|
||||
flat = re.sub(r'\s+', ' ', text).strip()
|
||||
if kind and flat and value_lines >= 2:
|
||||
# `value_end` can land mid-line, just past a closing quote, so extend to
|
||||
# the end of that physical line and carry whatever follows (a trailing
|
||||
# comment) across unchanged.
|
||||
if value_end > 0 and fm[value_end - 1] == '\n':
|
||||
span_end = value_end
|
||||
trailer = ''
|
||||
else:
|
||||
newline = fm.find('\n', value_end)
|
||||
span_end = len(fm) if newline == -1 else newline + 1
|
||||
trailer = fm[value_end:span_end].rstrip('\n')
|
||||
scalar = emit(flat)
|
||||
# A trailing comment carried across from the original line stays on the
|
||||
# `description:` line itself: after a block scalar's `|-` header it is
|
||||
# still a comment, but inside the block body it would become part of the
|
||||
# value.
|
||||
head, newline_sep, block_body = scalar.partition('\n')
|
||||
# The replacement displaces the whole span, so the blank-line pad makes
|
||||
# up the difference between the lines it displaced and the lines it
|
||||
# occupies — every later line number is unchanged. That is one line for
|
||||
# the three inline forms and two for the `|-` block; the span itself is
|
||||
# at least two lines here (`value_lines >= 2` is a precondition), so the
|
||||
# pad count never goes negative.
|
||||
pad = '\n' * (fm[head_start:span_end].count('\n') - 1 - scalar.count('\n'))
|
||||
new_fm = (fm[:head_start] + 'description: ' + head + trailer
|
||||
+ newline_sep + block_body + '\n' + pad + fm[span_end:])
|
||||
content = (fm_match.group(1) + new_fm + fm_match.group(3)
|
||||
+ content[fm_match.end():])
|
||||
|
||||
with open(dest, 'w', encoding='utf-8', errors='surrogateescape') as fh:
|
||||
fh.write(content)
|
||||
|
||||
if tmpdir is not None:
|
||||
print(dest)
|
||||
PYTHON
|
||||
}
|
||||
|
||||
tmpdir="$(cd "$(mktemp -d)" && pwd -P)"
|
||||
trap 'rm -rf "$tmpdir"' EXIT
|
||||
|
||||
# Mirror of the caller's cwd inside the scratch tree; relative path arguments
|
||||
# are resolved from here.
|
||||
mirror="$tmpdir$cwd"
|
||||
mkdir -p "$mirror"
|
||||
|
||||
argv_paths=()
|
||||
for arg in ${path_args[@]+"${path_args[@]}"}; do
|
||||
if [[ "$arg" == /* ]]; then
|
||||
raw_dest="$tmpdir$arg"
|
||||
else
|
||||
raw_dest="$mirror/$arg"
|
||||
fi
|
||||
if [[ -d "$arg" ]]; then
|
||||
dest="$(abspath "$raw_dest")"
|
||||
# A path argument with enough leading `..` to climb past the mirror root would
|
||||
# write outside the scratch dir. The real filesystem clamps such a path at
|
||||
# `/`; the mirror can't, so refuse rather than scribble outside the sandbox.
|
||||
case "$dest" in
|
||||
"$tmpdir"/*) ;;
|
||||
*)
|
||||
echo "vale-wrap.sh: refusing to lint '$arg': its scratch copy would land outside $tmpdir" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
mkdir -p "$(dirname "$dest")"
|
||||
# A directory is mirrored whole — vale applies its own format filtering to
|
||||
# the tree, so any file dropped here would be silently unlinted — and then
|
||||
# every markdown file in the copy is flattened in place. `.git` is pruned:
|
||||
# vale never lints it and copying it can dwarf the rest of the tree.
|
||||
# `find -L` follows symlinks because vale does: it lints both a symlinked
|
||||
# file and a file under a symlinked directory, and a bare `-type f` walk
|
||||
# would report "0 files" where bare vale reports one. (A symlink loop makes
|
||||
# `find` warn on stderr and carry on, which is also what vale does.) The
|
||||
# second walk needs no `-L`: the mirror is all real files by construction.
|
||||
mkdir -p "$dest"
|
||||
while IFS= read -r -d '' rel; do
|
||||
mkdir -p "$dest/$(dirname "$rel")"
|
||||
cp "$arg/$rel" "$dest/$rel"
|
||||
done < <(CDPATH='' cd -- "$arg" && find -L . -name .git -prune -o -type f -print0)
|
||||
while IFS= read -r -d '' md; do
|
||||
flatten "$md" "$md"
|
||||
done < <(find "$dest" -type f -name '*.md' -print0)
|
||||
else
|
||||
# `abspath` + `flatten` folded into one python3 process — see the comment
|
||||
# atop `flatten` above.
|
||||
dest="$(flatten "$arg" "$raw_dest" "$tmpdir")"
|
||||
fi
|
||||
if [[ "$arg" == /* ]]; then
|
||||
argv_paths+=("$dest")
|
||||
else
|
||||
argv_paths+=("$arg")
|
||||
fi
|
||||
done
|
||||
|
||||
cd "$mirror"
|
||||
vale ${vale_args[@]+"${vale_args[@]}"} ${argv_paths[@]+"${argv_paths[@]}"}
|
||||
324
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh
Executable file
324
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate-provenance.sh
Executable file
@@ -0,0 +1,324 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# The ONE entry point for provenance validation. It auto-detects whether the
|
||||
# target is a skill directory or an agent definition file — the same rule
|
||||
# validate.sh uses — and runs the matching suite from lib-provenance-skill.sh or
|
||||
# lib-provenance-agent.sh. The Contributing-files parser both suites need is
|
||||
# sourced once, from lib-contributing-files.sh, instead of being embedded twice.
|
||||
#
|
||||
# The two suites have DIFFERENT exit contracts, and merging the entry point does
|
||||
# not merge those:
|
||||
#
|
||||
# skill mode exits 0 with output when the only findings are INFO — a check
|
||||
# that could not run, announced rather than skipped silently. A
|
||||
# caller must read exit 0 plus output as INFO-only findings.
|
||||
# agent mode prints nothing at all on a clean run, and exits 0 SILENTLY when
|
||||
# the scope walk-up finds no plugin package above the agent file.
|
||||
# That is a verdict about a real file, not a rejected input;
|
||||
# scripts/check-scope-walkup-sync.sh's fixture 6 pins it.
|
||||
#
|
||||
# Exit 2 means the argument is not auditable at all — missing, doubled, the
|
||||
# wrong shape, or an environment problem. It is never a finding.
|
||||
|
||||
# --- Path splitting, with bash builtins only -------------------------------
|
||||
# dirname and basename are EXTERNAL commands, and every call below happens
|
||||
# before the mode's python3 preflight. Using them put coreutils ahead of python3
|
||||
# in the dependency order: on a PATH carrying neither, this script died at exit
|
||||
# 127 naming `dirname` (and, through the sourced libraries, `cat`) instead of
|
||||
# reaching the preflight that names python3 — the exact failure
|
||||
# tests/test-adr0020-contract.sh assertion 2 exists to prevent. The pre-merge
|
||||
# validate-provenance.sh was one self-contained file that reached its preflight
|
||||
# on builtins alone; these two functions, plus the `read`-based loaders in the
|
||||
# sourced libraries, restore that property. `cd` and `pwd` are builtins and may
|
||||
# stay.
|
||||
#
|
||||
# They reproduce dirname/basename semantics for the shapes this script sees:
|
||||
# trailing slashes are stripped, a path with no slash yields "." / itself, and
|
||||
# "/" yields "/".
|
||||
_kf_dirname() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
if [[ "$_p" != */* ]]; then
|
||||
printf '%s' "."
|
||||
return 0
|
||||
fi
|
||||
_p="${_p%/*}"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ -z "$_p" ]]; then
|
||||
_p="/"
|
||||
fi
|
||||
printf '%s' "$_p"
|
||||
}
|
||||
|
||||
_kf_basename() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${_p##*/}"
|
||||
}
|
||||
|
||||
# --- The target's parent directory NAME, resolved -------------------------
|
||||
# The agent rule tests the NAME of the target's parent directory. Reading that
|
||||
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
|
||||
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
|
||||
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
|
||||
# directory was refused as matching neither shape, by an error message naming
|
||||
# that exact shape as valid. The pre-merge agent validator had no path-shape
|
||||
# gate and worked from any working directory.
|
||||
#
|
||||
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
|
||||
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
|
||||
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
|
||||
# still addressed as agents/, which is what the literal test always honoured.
|
||||
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
|
||||
# that cannot be entered — a typo'd path — falls back to the literal name, so
|
||||
# the neither-shape error still fires for it.
|
||||
_kf_parent_name() {
|
||||
local _dir _resolved
|
||||
_dir="$(_kf_dirname "$1")"
|
||||
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
|
||||
_kf_basename "$_resolved"
|
||||
else
|
||||
_kf_basename "$_dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- This script's own directory, and the libraries beside it -------------
|
||||
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
|
||||
# CDPATH exported and the relative invocation the flow references prescribe
|
||||
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
|
||||
# and could resolve through CDPATH to an unrelated directory and source a
|
||||
# same-named file from there. CDPATH is cleared for the one command, `--` ends
|
||||
# option parsing for a directory named like a flag, and stdout is discarded so
|
||||
# only `pwd` is captured.
|
||||
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
|
||||
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
|
||||
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
|
||||
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A sourced library that is missing or unreadable used to kill the script under
|
||||
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
|
||||
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
|
||||
# partial install, or a copy or symlink of this one file taken out of scripts/,
|
||||
# was therefore reported as a failing audit. Checked explicitly instead, and
|
||||
# exit 2, which the same references read as "it never ran".
|
||||
_kf_require_lib() {
|
||||
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
|
||||
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
|
||||
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
|
||||
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
# Each mode's own usage text lives in that mode's library, verbatim, so usage()
|
||||
# needs the libraries — but `--help` must not. Sourcing them unconditionally at
|
||||
# the top made a missing lib-*.sh turn `--help` into exit 2, so the one command
|
||||
# that explains how to use the script was the one command a partial install
|
||||
# could not answer. validate.sh's usage() is self-contained and always works;
|
||||
# this restores the same property without copying the per-mode text down here
|
||||
# and letting it drift from the libraries that own it. When a library is gone,
|
||||
# the shared half of the usage still prints and the mode's half says why it
|
||||
# cannot.
|
||||
#
|
||||
# The two call sites below are spelled out rather than folded into one helper
|
||||
# taking the library as a parameter: a parameterized `.` is a non-constant
|
||||
# source, which is SC1090 at warning severity — the level .pre-commit-config.yaml
|
||||
# runs shellcheck at — and the only way to silence it, a `source=/dev/null`
|
||||
# directive, is a directive that resolves to nothing, which
|
||||
# tests/test-vale-wrap.sh part C rejects outright because a non-resolving
|
||||
# directive silently disarms that file's array-seeding exemption. Two literal
|
||||
# sources with two real directives cost a few lines and keep both gates honest.
|
||||
_kf_lib_readable() {
|
||||
[[ -f "$SCRIPT_DIR/$1" && -r "$SCRIPT_DIR/$1" ]]
|
||||
}
|
||||
|
||||
_kf_usage_lib_missing() {
|
||||
echo "(This mode's usage lives in $1, which is missing or unreadable in"
|
||||
echo "$SCRIPT_DIR. Reinstall the factory-audit skill to restore it. Note that"
|
||||
echo "an audit cannot run in this state either — it would exit 2.)"
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate-provenance.sh <skill-dir> [--base-ref=<ref>]
|
||||
validate-provenance.sh <agent-file>
|
||||
|
||||
Validate that a skill's or an agent's sources provenance chain is complete and
|
||||
internally consistent. The mode is detected from the target:
|
||||
|
||||
skill mode the target is a directory (a skill directory contains SKILL.md),
|
||||
or the target IS a SKILL.md file.
|
||||
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
||||
directory is named 'agents' (.apm/agents, .claude/agents,
|
||||
.github/agents, .copilot/agents).
|
||||
|
||||
The two modes have different checks, different exit contracts and different
|
||||
flags — --base-ref belongs to skill mode's check 9 and agent mode has no
|
||||
check 9 — so each mode's own usage follows below, verbatim.
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (or nothing to validate; in agent mode, also "not plugin
|
||||
scope")
|
||||
1 One or more checks failed
|
||||
2 Usage error, the target matches neither a skill directory nor an agent
|
||||
file this script can read, or a lib-*.sh beside this script is missing or
|
||||
unreadable
|
||||
|
||||
An exit code of 2 is NOT a finding. SKILL.md tells the auditor to surface a
|
||||
non-zero exit as findings, so a usage error leaving exit 1 with nothing on
|
||||
stdout was indistinguishable from a clean-but-failing run. Environment and
|
||||
argument problems exit 2; only real findings exit 1.
|
||||
|
||||
=== skill mode ===
|
||||
EOF
|
||||
if _kf_lib_readable lib-provenance-skill.sh; then
|
||||
# shellcheck source=lib-provenance-skill.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-skill.sh"
|
||||
kyberforge_prov_skill_usage
|
||||
else
|
||||
_kf_usage_lib_missing lib-provenance-skill.sh
|
||||
fi
|
||||
cat <<EOF
|
||||
|
||||
=== agent mode ===
|
||||
EOF
|
||||
if _kf_lib_readable lib-provenance-agent.sh; then
|
||||
# shellcheck source=lib-provenance-agent.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-agent.sh"
|
||||
kyberforge_prov_agent_usage
|
||||
else
|
||||
_kf_usage_lib_missing lib-provenance-agent.sh
|
||||
fi
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Sourced only once an audit is actually going to be attempted. It sat at the
|
||||
# top of the file until `--help` on a partial install exited 2 instead of
|
||||
# printing usage; see the comment above _kf_lib_readable for the whole story.
|
||||
# usage() loads the two provenance libraries on its own when it needs them, so
|
||||
# nothing here is reached by the --help path.
|
||||
_kf_require_lib lib-contributing-files.sh
|
||||
# shellcheck source=lib-contributing-files.sh
|
||||
. "$SCRIPT_DIR/lib-contributing-files.sh"
|
||||
_kf_require_lib lib-provenance-skill.sh
|
||||
# shellcheck source=lib-provenance-skill.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-skill.sh"
|
||||
_kf_require_lib lib-provenance-agent.sh
|
||||
# shellcheck source=lib-provenance-agent.sh
|
||||
. "$SCRIPT_DIR/lib-provenance-agent.sh"
|
||||
|
||||
# --- Detect the mode -------------------------------------------------------
|
||||
# The first non-flag argument decides the mode. Only the mode is decided here:
|
||||
# the argument COUNT, the flag rules and every precondition belong to the mode's
|
||||
# own suite and are applied there, unchanged, over the original "$@". So a
|
||||
# --base-ref handed to an agent target is still an extra argument and is still
|
||||
# rejected, and a second positional is still rejected by whichever mode it
|
||||
# reaches.
|
||||
# _saw_positional is tracked separately because an EMPTY positional and NO
|
||||
# positional are different mistakes with different fixes, and `-z "$TARGET"`
|
||||
# alone cannot tell them apart: `validate-provenance.sh ""` — an unquoted shell
|
||||
# variable that expanded to nothing, the usual way this happens — was reported
|
||||
# as "only flags were given", which is false and sends the reader looking for a
|
||||
# flag they did not type instead of at the variable that came up empty.
|
||||
TARGET=""
|
||||
_saw_positional=false
|
||||
for _arg in "$@"; do
|
||||
case "$_arg" in
|
||||
--base-ref=*) ;;
|
||||
*) TARGET="$_arg"; _saw_positional=true; break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ "$_saw_positional" == false ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo " Why: only flags were given, so there is no target to detect a mode from." >&2
|
||||
echo " Fix: pass the skill directory, or the agent file, as a positional argument." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
if [[ -z "$TARGET" ]]; then
|
||||
echo "Error: the target argument is an empty string." >&2
|
||||
echo " Why: a positional argument was passed, but it is empty, so there is no path to detect a mode from — usually an unquoted or unset shell variable expanding to nothing at the call site, not a missing argument." >&2
|
||||
echo " Fix: check the variable that supplies the target, and pass the skill directory, or the agent file, as a non-empty positional argument." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
TARGET_BASE="$(_kf_basename "$TARGET")"
|
||||
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
|
||||
|
||||
if [[ -d "$TARGET" ]]; then
|
||||
if [[ -f "$TARGET/SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
else
|
||||
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would run the wrong provenance checks." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
||||
exit 2
|
||||
fi
|
||||
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
||||
MODE=agent
|
||||
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
||||
MODE=agent
|
||||
else
|
||||
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would report a silent pass on a typo'd target, which is the failure both suites' preconditions exist to prevent." >&2
|
||||
echo " Fix: pass one of those two shapes." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# In skill mode a SKILL.md target names its directory. The token is replaced in
|
||||
# place rather than assumed to be $1, because --base-ref may precede it; the
|
||||
# suite's own preconditions then apply to that directory, exactly as before the
|
||||
# merge.
|
||||
if [[ "$MODE" == skill && "$TARGET_BASE" == "SKILL.md" && ! -d "$TARGET" ]]; then
|
||||
declare -a _rewritten=()
|
||||
_replaced=false
|
||||
for _arg in "$@"; do
|
||||
if [[ "$_replaced" == false && "$_arg" == "$TARGET" ]]; then
|
||||
_rewritten+=("$(_kf_dirname "$TARGET")")
|
||||
_replaced=true
|
||||
else
|
||||
_rewritten+=("$_arg")
|
||||
fi
|
||||
done
|
||||
# Guarded expansion: bash 3.2 under `set -u` aborts on "${arr[@]}" when the
|
||||
# array is empty, and the loop above cannot prove non-emptiness to a static
|
||||
# scan. tests/test-vale-wrap.sh enforces bash-3.2 portability across this tree.
|
||||
set -- ${_rewritten[@]+"${_rewritten[@]}"}
|
||||
fi
|
||||
|
||||
# Called UNTESTED, on purpose: `f || RC=$?` would disable errexit for the whole
|
||||
# function body. Each run function stashes its findings code in
|
||||
# KYBERFORGE_PROV_RC and returns 0; its error paths exit directly.
|
||||
KYBERFORGE_PROV_RC=0
|
||||
case "$MODE" in
|
||||
skill) kyberforge_prov_skill_run "$@" ;;
|
||||
agent) kyberforge_prov_agent_run "$@" ;;
|
||||
esac
|
||||
RC="$KYBERFORGE_PROV_RC"
|
||||
|
||||
exit "$RC"
|
||||
255
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh
Executable file
255
plugins/kyberforge/.apm/skills/factory-audit/scripts/validate.sh
Executable file
@@ -0,0 +1,255 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# The ONE entry point for structural validation. It auto-detects whether the
|
||||
# target is a skill directory or an agent definition file and runs the matching
|
||||
# check suite; the two suites live in lib-checks-skill.sh and lib-checks-agent.sh
|
||||
# and are unchanged from the skill-audit / agent-audit scripts they came from.
|
||||
# The ADR-0020 boundary resolver both of them need is sourced once, from
|
||||
# lib-boundary-resolver.sh, instead of being embedded twice.
|
||||
#
|
||||
# Detection never guesses. A target that matches neither shape is a hard exit 2
|
||||
# naming the mismatch, because the alternative — picking a mode and letting the
|
||||
# suite fail on its own terms — reports a skill-shaped finding about an agent
|
||||
# file, or the reverse, and sends the reader after the wrong problem.
|
||||
|
||||
# --- Path splitting, with bash builtins only -------------------------------
|
||||
# dirname and basename are EXTERNAL commands, and every call below happens
|
||||
# before the mode-specific python3/PyYAML preflight. Using them put coreutils
|
||||
# ahead of python3 in the dependency order: on a PATH carrying neither, this
|
||||
# script died at exit 127 naming `dirname` instead of reaching the preflight
|
||||
# that names python3 — the exact failure tests/test-adr0020-contract.sh
|
||||
# assertion 2 exists to prevent ("the two are checked separately so the message
|
||||
# names the thing to install rather than the wrong one"). The pre-merge
|
||||
# validate.sh was one self-contained file that reached its preflight on builtins
|
||||
# alone; these two functions restore that property. `cd` and `pwd` are builtins
|
||||
# and may stay.
|
||||
#
|
||||
# They reproduce dirname/basename semantics for the shapes this script sees:
|
||||
# trailing slashes are stripped, a path with no slash yields "." / itself, and
|
||||
# "/" yields "/".
|
||||
_kf_dirname() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
if [[ "$_p" != */* ]]; then
|
||||
printf '%s' "."
|
||||
return 0
|
||||
fi
|
||||
_p="${_p%/*}"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ -z "$_p" ]]; then
|
||||
_p="/"
|
||||
fi
|
||||
printf '%s' "$_p"
|
||||
}
|
||||
|
||||
_kf_basename() {
|
||||
local _p="$1"
|
||||
while [[ "$_p" == */ && "$_p" != "/" ]]; do _p="${_p%/}"; done
|
||||
if [[ "$_p" == "/" ]]; then
|
||||
printf '%s' "/"
|
||||
return 0
|
||||
fi
|
||||
printf '%s' "${_p##*/}"
|
||||
}
|
||||
|
||||
# --- The target's parent directory NAME, resolved -------------------------
|
||||
# The agent rule tests the NAME of the target's parent directory. Reading that
|
||||
# name off the argument text — `_kf_basename "$(_kf_dirname "$TARGET")"` —
|
||||
# returned "." for a bare `git-orchestrate.md` typed from inside .claude/agents/
|
||||
# (and for `./git-orchestrate.md`), so a file that IS directly under an agents/
|
||||
# directory was refused as matching neither shape, by an error message naming
|
||||
# that exact shape as valid. The pre-merge agent validator had no path-shape
|
||||
# gate and worked from any working directory.
|
||||
#
|
||||
# So the parent is resolved with the `cd` and `pwd` builtins in a subshell —
|
||||
# still coreutils-free, for the reason above. It resolves LOGICALLY (`pwd`, not
|
||||
# `pwd -P`): an agents/ directory reached through a symlink named agents/ is
|
||||
# still addressed as agents/, which is what the literal test always honoured.
|
||||
# CDPATH is cleared and cd's output discarded; see SCRIPT_DIR below. A parent
|
||||
# that cannot be entered — a typo'd path — falls back to the literal name, so
|
||||
# the neither-shape error still fires for it.
|
||||
_kf_parent_name() {
|
||||
local _dir _resolved
|
||||
_dir="$(_kf_dirname "$1")"
|
||||
if _resolved="$(CDPATH='' cd -- "$_dir" > /dev/null 2>&1 && pwd)"; then
|
||||
_kf_basename "$_resolved"
|
||||
else
|
||||
_kf_basename "$_dir"
|
||||
fi
|
||||
}
|
||||
|
||||
# --- This script's own directory, and the libraries beside it -------------
|
||||
# `cd` PRINTS the directory it resolved whenever CDPATH supplied it, so with
|
||||
# CDPATH exported and the relative invocation the flow references prescribe
|
||||
# (`bash scripts/<name>.sh`), a bare `$(cd ... && pwd)` captured two lines —
|
||||
# and could resolve through CDPATH to an unrelated directory and source a
|
||||
# same-named file from there. CDPATH is cleared for the one command, `--` ends
|
||||
# option parsing for a directory named like a flag, and stdout is discarded so
|
||||
# only `pwd` is captured.
|
||||
if ! SCRIPT_DIR="$(CDPATH='' cd -- "$(_kf_dirname "${BASH_SOURCE[0]}")" > /dev/null 2>&1 && pwd)"; then
|
||||
echo "Error: cannot enter the directory this script lives in ('$(_kf_dirname "${BASH_SOURCE[0]}")')." >&2
|
||||
echo " Why: the check suites are sourced from files beside this script, so without its own directory nothing can run — and reporting that as findings would pass a broken install off as a failing audit." >&2
|
||||
echo " Fix: invoke the script by a path to its real location inside factory-audit/scripts/." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# A sourced library that is missing or unreadable used to kill the script under
|
||||
# `set -e` with bash's own "No such file or directory" and exit 1 — the tier the
|
||||
# flow references tell the auditor to surface verbatim as REAL FINDINGS. A
|
||||
# partial install, or a copy or symlink of this one file taken out of scripts/,
|
||||
# was therefore reported as a failing audit. Checked explicitly instead, and
|
||||
# exit 2, which the same references read as "it never ran".
|
||||
_kf_require_lib() {
|
||||
if [[ ! -f "$SCRIPT_DIR/$1" || ! -r "$SCRIPT_DIR/$1" ]]; then
|
||||
echo "Error: required library '$SCRIPT_DIR/$1' is missing or unreadable." >&2
|
||||
echo " Why: this script ships together with the lib-*.sh files in factory-audit/scripts/ and cannot run without them; this is an install problem, not a finding about the target." >&2
|
||||
echo " Fix: reinstall the factory-audit skill so its scripts/ directory is complete, and run the script from there rather than from a copy or symlink of the file alone." >&2
|
||||
exit 2
|
||||
fi
|
||||
}
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Usage: validate.sh <skill-dir | agent-file>
|
||||
|
||||
Validate a skill directory against the agentskills.io specification, or an agent
|
||||
definition file against the agent definition spec. The mode is detected from the
|
||||
target:
|
||||
|
||||
skill mode the target is a directory (a skill directory contains SKILL.md),
|
||||
or the target IS a SKILL.md file.
|
||||
agent mode the target is a *.agent.md file, or a *.md file whose parent
|
||||
directory is named 'agents' (.apm/agents, .claude/agents,
|
||||
.github/agents, .copilot/agents).
|
||||
|
||||
Skill mode audits the directory named by <skill-dir>.
|
||||
|
||||
Agent mode: at plugin/APM scope, <agent-file> is a single vendor-neutral
|
||||
.apm/agents/<name>.agent.md file with no counterpart. Its frontmatter allowlist
|
||||
is not restated here: it is read at load time from the apm-agent-allowlist
|
||||
section of references/agent-field-inventory.md, which is the authoritative list.
|
||||
At project or user scope, <agent-file> is either half of a Claude Code .md /
|
||||
Copilot .agent.md pair.
|
||||
|
||||
Arguments:
|
||||
skill-dir Path to the skill directory containing SKILL.md.
|
||||
agent-file Path to the agent file (or either half of a project/user-scope pair).
|
||||
|
||||
Exit codes:
|
||||
0 All checks passed (may include SUGGESTIONs)
|
||||
1 One or more checks failed
|
||||
2 Nothing was audited (no argument, the target matches neither shape, the
|
||||
target does not exist, an unrecognized file extension, a missing
|
||||
references/agent-field-inventory.md, or a missing or unreadable lib-*.sh
|
||||
beside this script)
|
||||
EOF
|
||||
}
|
||||
|
||||
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo "Error: a skill directory or an agent file is required." >&2
|
||||
echo "" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
TARGET="$1"
|
||||
|
||||
# --- The target has to be there --------------------------------------------
|
||||
# Only the directory branch below stats the target; the *.agent.md, the
|
||||
# agents/-parent and the SKILL.md branches classify on NAME alone, so a typo'd
|
||||
# path matching one of those shapes was handed to python3 and came back as a
|
||||
# FAIL at exit 1 — the findings tier, for a target that was never there to have
|
||||
# findings about. The tiers are: 2 nothing is at this path so no check ran, 1
|
||||
# something is there and it is broken.
|
||||
#
|
||||
# This runs on the TYPED path, before the SKILL.md -> parent-directory rewrite
|
||||
# further down: rewritten first, a missing `docs/SKILL.md` would be tested as
|
||||
# `docs`, which exists, and the guard would miss it.
|
||||
#
|
||||
# `-L` deliberately rescues what `-e` rejects. A dangling symlink and a symlink
|
||||
# loop are both FALSE to -e but TRUE to -L, and neither belongs here: something
|
||||
# IS at that path, it just cannot be opened, and "exists but unreadable" is a
|
||||
# real finding the agent suite's check_file reports as a FAIL naming the file.
|
||||
# Catching them here would replace that FAIL with a false "does not exist".
|
||||
if [[ ! -e "$TARGET" && ! -L "$TARGET" ]]; then
|
||||
echo "Error: '$TARGET' does not exist." >&2
|
||||
echo " Why: the path shape says what would be audited, but there is nothing at this path to audit — and auditing a target that is not there would report the absence as findings about it, sending the reader after a spec violation instead of a typo." >&2
|
||||
echo " Fix: check the path, and pass an existing skill directory (or its SKILL.md) or an existing agent file." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- Detect the mode -------------------------------------------------------
|
||||
# Pure path and stat inspection, no interpreter and no external command needed,
|
||||
# so it runs before the python3/PyYAML preflight — which is mode-specific,
|
||||
# because each suite names the gates it would otherwise skip.
|
||||
TARGET_BASE="$(_kf_basename "$TARGET")"
|
||||
TARGET_PARENT="$(_kf_parent_name "$TARGET")"
|
||||
|
||||
if [[ -d "$TARGET" ]]; then
|
||||
if [[ -f "$TARGET/SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
else
|
||||
echo "Error: '$TARGET' is a directory with no SKILL.md in it." >&2
|
||||
echo " Why: a skill directory is identified by its SKILL.md, and an agent target is a file, never a directory — so this path matches neither mode and guessing one would report findings of the wrong kind." >&2
|
||||
echo " Fix: pass the skill directory that holds SKILL.md, or an agent file (<name>.agent.md, or a .md file under an agents/ directory)." >&2
|
||||
exit 2
|
||||
fi
|
||||
elif [[ "$TARGET_BASE" == "SKILL.md" ]]; then
|
||||
MODE=skill
|
||||
TARGET="$(_kf_dirname "$TARGET")"
|
||||
elif [[ "$TARGET_BASE" == *.agent.md ]]; then
|
||||
MODE=agent
|
||||
elif [[ "$TARGET_BASE" == *.md && "$TARGET_PARENT" == "agents" ]]; then
|
||||
MODE=agent
|
||||
else
|
||||
echo "Error: '$TARGET' matches neither a skill directory nor an agent file." >&2
|
||||
echo " Why: skill mode needs a directory containing SKILL.md (or the SKILL.md itself); agent mode needs a <name>.agent.md file, or a .md file directly under an agents/ directory (.apm/agents, .claude/agents, .github/agents, .copilot/agents). Picking a mode anyway would audit this path against the wrong spec." >&2
|
||||
echo " Fix: pass one of those two shapes." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# --- Run the matching suite ------------------------------------------------
|
||||
# Each suite is reassembled in the order the resolver block sat in before the
|
||||
# merge — preamble, resolver, body — so every check runs against exactly the
|
||||
# names and the order it always did.
|
||||
RC=0
|
||||
case "$MODE" in
|
||||
skill)
|
||||
_kf_require_lib lib-boundary-resolver.sh
|
||||
# shellcheck source=lib-boundary-resolver.sh
|
||||
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
||||
_kf_require_lib lib-checks-skill.sh
|
||||
# shellcheck source=lib-checks-skill.sh
|
||||
. "$SCRIPT_DIR/lib-checks-skill.sh"
|
||||
kyberforge_skill_preflight
|
||||
PROG="$KYBERFORGE_SKILL_PREAMBLE_PY
|
||||
$KYBERFORGE_RESOLVER_PY
|
||||
$KYBERFORGE_SKILL_BODY_PY"
|
||||
python3 -u - "$TARGET" <<< "$PROG" || RC=$?
|
||||
;;
|
||||
agent)
|
||||
_kf_require_lib lib-boundary-resolver.sh
|
||||
# shellcheck source=lib-boundary-resolver.sh
|
||||
. "$SCRIPT_DIR/lib-boundary-resolver.sh"
|
||||
_kf_require_lib lib-checks-agent.sh
|
||||
# shellcheck source=lib-checks-agent.sh
|
||||
. "$SCRIPT_DIR/lib-checks-agent.sh"
|
||||
kyberforge_agent_preflight
|
||||
PROG="$KYBERFORGE_AGENT_PREAMBLE_PY
|
||||
$KYBERFORGE_RESOLVER_PY
|
||||
$KYBERFORGE_AGENT_BODY_PY"
|
||||
python3 -u - "$TARGET" "$SCRIPT_DIR" <<< "$PROG" || RC=$?
|
||||
;;
|
||||
esac
|
||||
|
||||
exit "$RC"
|
||||
94
plugins/kyberforge/.apm/skills/factory-audit/tests/README.md
Normal file
94
plugins/kyberforge/.apm/skills/factory-audit/tests/README.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# tests/
|
||||
|
||||
Test files for scripts bundled with this skill.
|
||||
|
||||
## When to add tests
|
||||
|
||||
Add tests here when the skill has scripts in `scripts/` that are complex enough
|
||||
to break silently — validators, parsers, generators, anything with branching
|
||||
logic or edge cases. Test infrastructure (`.bats`, `*_test.*`, `test_*.sh`)
|
||||
belongs here, not in `scripts/`.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Tests require [bats-support](https://github.com/bats-core/bats-support) and
|
||||
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
|
||||
helpers from the repo root's `tests/test_helper/`.
|
||||
|
||||
From the repo root:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
|
||||
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
|
||||
```
|
||||
|
||||
Run all tests for this skill (from the repo root):
|
||||
|
||||
```bash
|
||||
bats plugins/kyberforge/.apm/skills/factory-audit/tests/
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `validate-skill.bats` | `scripts/validate.sh` against skill directories |
|
||||
| `validate-agent.bats` | `scripts/validate.sh` against agent files |
|
||||
| `validate-provenance-skill.bats` | `scripts/validate-provenance.sh` against skill directories |
|
||||
| `validate-provenance-agent.bats` | `scripts/validate-provenance.sh` against agent files |
|
||||
|
||||
## Two scripts, four suites
|
||||
|
||||
`factory-audit` merges what were two skills — `skill-audit` and `agent-audit` —
|
||||
each of which shipped its own `validate.sh` and `validate-provenance.sh`. The
|
||||
merged skill has **one** of each. Every suite here invokes one of those two
|
||||
scripts; the four files are two scripts × two artifact types, not four scripts.
|
||||
|
||||
`validate-skill.bats` and `validate-agent.bats` run the same
|
||||
`scripts/validate.sh` and differ only in the fixtures they point it at. The two
|
||||
provenance suites stand in the same relation to `scripts/validate-provenance.sh`.
|
||||
Do not add a third script path here on the assumption that a differently named
|
||||
suite must mean a differently named script.
|
||||
|
||||
### Auto-detection is pinned across the pair
|
||||
|
||||
Each entry point decides for itself what it was handed. ADR-0025 states the
|
||||
rule: a directory containing `SKILL.md` takes the skill flow; an `.agent.md`
|
||||
file, or a file under a directory named `agents/`, takes the agent flow.
|
||||
Anything else is rejected rather than guessed at. That behaviour is new with the
|
||||
merge — before it, each script was hard-wired to one artifact type and nothing
|
||||
about classification could be wrong — so it is asserted from both sides rather
|
||||
than in one place:
|
||||
|
||||
- the skill-side suites pin the skill-directory classification and the
|
||||
neither-shape rejection,
|
||||
- the agent-side suites pin the two agent rules *separately* — `.agent.md` in a
|
||||
directory that is not `agents/`, and a plain `.md` under `.apm/agents/` — so
|
||||
that a detector implementing only one of them cannot pass both. Plus a control
|
||||
asserting an agent file never picks up a skill-only gate.
|
||||
|
||||
Both skill-side suites additionally pin the `SKILL.md` **file** path, not just
|
||||
the directory: a pre-commit `files:` hook matches files, so every hook-driven
|
||||
invocation hands over a `SKILL.md` path. Each entry point resolves it to the
|
||||
directory, and the suites assert the two spellings produce the identical
|
||||
verdict rather than merely that the file spelling survives.
|
||||
|
||||
A misclassification is silent and total — the wrong rubric runs end to end and
|
||||
reports the artifact clean against gates that never applied to it — and no other
|
||||
fixture in these suites would notice, because every other fixture is already the
|
||||
shape its own suite expects.
|
||||
|
||||
### The two provenance exit contracts are different on purpose
|
||||
|
||||
`scripts/validate-provenance.sh` does **not** behave identically in its two
|
||||
modes, and the difference is deliberate:
|
||||
|
||||
| Mode | Successful run |
|
||||
|------|----------------|
|
||||
| skill | may exit 0 **with** output — INFO findings print, status stays 0 |
|
||||
| agent | exits 0 and prints **nothing** |
|
||||
|
||||
Each half is asserted from its own side, by the `exit contract:` test in each
|
||||
provenance suite. Both are asserted on purpose: a merge that collapsed one
|
||||
contract into the other would still satisfy whichever side was left unasserted,
|
||||
so a single-sided pin would go green on exactly the defect it exists to catch.
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user