Files

Defame1297 620f20b0fd refactor(kyberforge)!: merge skill-audit and agent-audit into factory-audit

Why

The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary
resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser
(93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line
test suite. Sync-by-gate had already failed once: at 484357a the two parser copies drifted into
different spellings of the bullet loop while a docstring asserted they were identical. That drift
was behaviour-neutral and was re-unified by hand at 598a7c3, so the copies were identical at merge
time — but nothing had caught it, and the next drift need not be neutral.

Implementation Notes

Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching
across skill directories, which is why two separate skills needed embedded copies; two files inside
ONE skill may source a third. That is the whole reason the merge removes duplication rather than
relocating it.

The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those
words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path
before any validation, and its table mirrors validate.sh's detection exactly: a directory holding
SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory
(agent); anything else stops without running a validator. Steps 1-3 live in
references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in
that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the
other artifact type, the body restarts at Step 0.

Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that
matches a live sibling reads as ownership rather than membership.

The description carries one arrow per boundary target, because ADR-0020 resolves only the first
target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated
"audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16%
of the body: no size SUGGESTIONs.

The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin
cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml
where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both.
tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources
the resolver and that no third copy exists.

The entry scripts classify the target from its resolved parent directory, so a bare agent filename
typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is
missing, rather than dying with exit 1, the tier the flows relay as real findings.

The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and
return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's
status (`f || RC=$?`) disables errexit for its entire body, and no subshell or
`set -e` inside can re-arm it once the call sits in a condition context
(measured, both spellings). Their error paths use `exit`, which is unaffected
either way; this keeps errexit armed for anything added later.

Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is
access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's
dev environment is root -- so the guard could never fire where it exists to fire.
A read attempt is also the stricter question, catching EIO. This is the reasoning
scripts/check-vale-style-sync.sh carried before this commit deleted it; the
hazard did not go with it.

All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared,
the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise
print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two
remaining bare cd calls take absolute paths, which CDPATH is never consulted for.

Impact

BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to
2.0.0 (catalog 0.4.7).

Check logic is unchanged: differential runs of the old and new validators across every skill and
agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads
differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename.
One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a
target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change,
deliberately (full table in ADR-0025):
- a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is
  by path shape, so a shape-matching path that is simply absent used to reach the validator and come
  back as a FAIL against a file that never existed;
- an unshaped target exits 2 under the generic "matches neither" message, and a directory with no
  SKILL.md under a third, distinct one -- three exit-2 messages, not one;
- a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding
  about the artifact rather than a usage error;
- a SKILL.md file path is audited as its skill directory instead of refused;
- a .md agent outside an agents/ directory is refused rather than audited;
- a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2.
  validate-provenance.sh already exited 2 for the last two; validate.sh now matches it.

.pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both
files: regexes are unchanged; only entry: and description: moved.

scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21),
tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The
checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into
tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the
cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both
hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation
self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite.

Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8.

ADR: 0025
BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by
  factory-audit, which auto-detects whether it was handed a skill directory or an agent file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD

2026-09-16 09:13:57 +00:00

8.7 KiB

Raw Blame History

source_keys

agentskills-home

agentskills-spec

agentskills-best-practices

agentskills-quickstart

agentskills-using-scripts

Creating a new skill

Return to SKILL.md Step 4 once Step 6 below is done — validation, versioning and commit verification are shared with the improve flow and are not repeated here.

Prerequisites

Run /grill-me on the skill's design and research the target domain first. Share those outputs in this conversation: grill context, research docs, examples, constraints.

Design for one coherent user intent — skills too narrow force multiple loads per task; too broad are hard to activate precisely.

Before touching the filesystem, verify you have:

A clear purpose — what specific task will this skill handle?
Trigger scenarios — when should an agent activate it?
Skill name (kebab-case) and destination path

If any are missing, stop and ask the user before proceeding.

Requires /factory-audit — used in SKILL.md Step 4 for final validation. Both skills ship in the kyberforge plugin and are co-installed. If /factory-audit is unavailable, stop and ask the user to install the kyberforge plugin before continuing.

Package-intent gate

Judge whether the destination is meant to be inside an APM package before running the scaffold script — the script cannot tell "no package here" apart from "package not scaffolded yet":

Package intent but no type:-bearing apm.yml found at or above the destination (e.g. "add to my apm package", or a sibling .apm//apm.yml exists nearby) → stop, tell the user to run /apm-workflow configure (apm plugin init, from inside the package directory) first, then retry. Do not fall through to standalone mode.
Otherwise (a ~/-rooted destination, or no package context implied) → continue to Step 1.

Step 1 — Scaffold

Run the copy script with the skill name and a path inside or at the target:

bash scripts/new-skill.sh <skill-name> <path>

The script walks up from <path> for a package boundary: an ancestor apm.yml with a top-level type: field (instructions/skill/hybrid/prompts) means package mode — scaffolds into <package-root>/.apm/skills/<skill-name>/, not under <path> (a subdirectory of the package works fine as <path>). A type:-less apm.yml is a marketplace-only manifest, skipped. Hitting .git or the filesystem root first means standalone mode — scaffolds directly into <path>/<skill-name>/.

Examples:

# Package mode — packages/my-pkg/apm.yml already has `type: skill`
bash scripts/new-skill.sh my-tool packages/my-pkg/

# Standalone mode — no apm.yml/.git above ~/.agents/skills/
bash scripts/new-skill.sh my-tool ~/.agents/skills/

The script prints which mode it used and where the skill landed — read its output.

In package mode, read references/deployment-modes.md before adding any file references to SKILL.md.

Step 2 — Update `apm.yml` includes (package mode only)

Skip in standalone mode. In package mode, check the resolved package's apm.yml: if includes: is an explicit list (not auto), append .apm/skills/<skill-name>/ to it if not already present, preserving YAML formatting. If includes: auto or the field is absent, do nothing — auto already covers the new skill. Use Read/Edit directly on apm.yml; this is not part of scripts/new-skill.sh.

Step 3 — Fill in SKILL.md

Open the new skill's SKILL.md (the path Step 1 printed) and replace every FILL IN: placeholder. The scaffold template carries the body skeleton and the two frontmatter fields that cannot be left as placeholders — name, substituted by the script, and metadata.version, seeded live at "0.1.0" — so fill the template in rather than restructuring it.

name — already set by the scaffold script. Must exactly match the directory name. Format: 1–64 characters, lowercase letters, numbers and hyphens only; no leading, trailing or consecutive hyphens (--).

description — carries the entire triggering burden and is preloaded every session. Write it against references/contract.md, which holds the three-part shape, the banned content, the boundary-clause form and the length tiers. A hand-invoked skill (SKILL.md Step 2) takes one plain sentence and disable-model-invocation: true instead.

metadata.version — required on every skill (ADR-0022), not a per-skill or per-plugin choice, and enforced by the skill-size-check pre-commit hook. The scaffold seeds a new skill at "0.1.0"; leave that value alone here and let SKILL.md Step 4 bump it. ("1.0.0" is the seed for a pre-existing skill retrofitted into the rule, and never applies to a skill created here.)

Optional frontmatter — uncomment and fill in, or remove entirely:

license — include when distributing the skill externally
compatibility — include if the skill requires specific tools, runtimes, or network access (max 500 characters)
metadata — the rest of the map, all of it optional: author and category, plus source_keys now (Step 6) if research sources are in context
allowed-tools — space-separated pre-approved tools; reduces permission prompts (experimental — support varies by client)
disable-model-invocation — hand-invoked skills only

metadata.source_keys — if research sources are in context, list the relevant slugs as you write the body; do not defer this to Step 6. Agents that fill in source_keys late tend to omit it entirely. Example:

metadata:
  source_keys:
    - my-source-slug
    - another-slug

Body — write the decision procedure only, following the body rules and patterns in references/contract.md. Rename the placeholder section headings to ones that fit the skill's structure.

Step 4 — Add scripts (if needed)

Place executable scripts in scripts/. Critical rule: no interactive prompts — agents run non-interactive, and blocking on TTY input hangs indefinitely. Accept all input via flags, env vars, or stdin.

If adding a script, read references/scripts.md first — it covers the full contract: structured output, pinned versions, self-contained deps, idempotency, exit codes, dry-run, error messages, and output size limits.

If no scripts are needed, delete scripts/README.md and the scripts/ directory.

Step 5 — Add references, assets, and tests (if needed)

references/ — additional documentation loaded on demand. One topic per file, named in kebab-case after the topic. Reference conditionally from SKILL.md with the literal form If <condition>, read `references/<file>.md` .

Two hops from SKILL.md, never three. A flow file may route on to a shared contract or sub-topic file — that is the shipped pattern here (SKILL.md → references/create.md → this file's own pointers to contract.md, scripts.md and deployment-modes.md). What does not work is a third hop: a file reachable only through two intermediates is rarely loaded at the moment it is needed. Every hop past the first also needs the same literal conditional form, so the agent knows when to take it.

assets/ — static resources: templates, schemas, lookup tables. Reference by relative path from SKILL.md.

tests/ — test files for scripts in scripts/. Use when scripts are complex enough to break silently. Test infrastructure (.bats, *_test.*) belongs here, not in scripts/. See tests/README.md for setup instructions.

If not needed, delete the placeholder READMEs and their directories.

Step 6 — Populate or delete `references/sources.md`

If a research sources.md is present in the conversation context:

Read it and filter to entries with `extracted` status only.
For each entry, determine which skill files it contributed to (SKILL.md and any files in references/ that drew from it). Update Contributing files accordingly — list skill files, not research topic files.
Write the updated content to references/sources.md. For each entry, include - **Research doc:** <path> where <path> is the relative path from the repo root to the plugin-level research sources file this entry was drawn from (e.g. plugins/myplugin/docs/research/docs/<topic>/sources.md). This field is required on every entry — it makes the provenance chain explicit and is validated by /factory-audit.
Add source_keys to the frontmatter of SKILL.md (under metadata) listing the slugs of sources that informed it.
For each file in references/ that was informed by research sources, add source_keys frontmatter (same format as research topic files) listing the relevant slugs.

If no research sources.md is in context, delete references/sources.md.

Then return to SKILL.md Step 4.

8.7 KiB Raw Blame History Unescape Escape