Why The two audit skills carried 1,724 lines of byte-identical duplication: the ADR-0020 boundary resolver (1,061), vale-wrap.sh (526), the Vale style rules (44) and the Contributing-files parser (93). Nothing shared them — they were held in sync by a 413-line pre-push gate and its 797-line test suite. Sync-by-gate had already failed once: at484357athe two parser copies drifted into different spellings of the bullet loop while a docstring asserted they were identical. That drift was behaviour-neutral and was re-unified by hand at598a7c3, so the copies were identical at merge time — but nothing had caught it, and the next drift need not be neutral. Implementation Notes Self-containment binds BETWEEN skills, not within one. The agentskills.io spec forbids reaching across skill directories, which is why two separate skills needed embedded copies; two files inside ONE skill may source a third. That is the whole reason the merge removes duplication rather than relocating it. The union of both bodies measured 1,532 words against BODY_MAX_WORDS=900, and only 211 of those words were shared, so SKILL.md is a dispatch body. Step 0 resolves the flow from the target path before any validation, and its table mirrors validate.sh's detection exactly: a directory holding SKILL.md or a SKILL.md file (skill); a *.agent.md, or a .md directly under an agents/ directory (agent); anything else stops without running a validator. Steps 1-3 live in references/skill-flow.md and references/agent-flow.md, and gotchas that apply to one flow live in that flow's file, since it is loaded on every invocation anyway. If validate.sh reports on the other artifact type, the body restarts at Step 0. Named factory-audit rather than forge-audit because forge is a live skill, and a family prefix that matches a live sibling reads as ownership rather than membership. The description carries one arrow per boundary target, because ADR-0020 resolves only the first target after an arrow. It drops the quoted "audit this skill"-style phrases, which restated "audited" in a second register (ADR-0020's duplicate-register rule). 241 characters, Gotchas 16% of the body: no size SUGGESTIONs. The boundary resolver stays embedded in two files rather than imported: a cache-installed plugin cannot read outside its own directory, and the repo-root hook resolves via .pre-commit-hooks.yaml where entry[0] is the only token pre-commit rewrites, so no single file is reachable by both. tests/test-adr0020-contract.sh hashes both copies for byte-identity, and asserts validate.sh sources the resolver and that no third copy exists. The entry scripts classify the target from its resolved parent directory, so a bare agent filename typed inside agents/ works; resolve SCRIPT_DIR CDPATH-safely; and exit 2 when a lib-*.sh is missing, rather than dying with exit 1, the tier the flows relay as real findings. The provenance run functions stash their findings code in KYBERFORGE_PROV_RC and return 0, so validate-provenance.sh calls them UNTESTED. Testing a function's status (`f || RC=$?`) disables errexit for its entire body, and no subshell or `set -e` inside can re-arm it once the call sits in a condition context (measured, both spellings). Their error paths use `exit`, which is unaffected either way; this keeps errexit armed for anything added later. Case 0's readability guard reads the file instead of asking `[[ -r ]]`. `-r` is access(2), which answers yes for uid 0 even on a mode-000 file, and this repo's dev environment is root -- so the guard could never fire where it exists to fire. A read attempt is also the stricter question, catching EIO. This is the reasoning scripts/check-vale-style-sync.sh carried before this commit deleted it; the hazard did not go with it. All three entry scripts are CDPATH-safe, vale-wrap.sh included: both of its cd sites are cleared, the --config resolution and the directory-mirror walk, where an exported CDPATH would otherwise print a decoy path into the -print0 stream and build the mirror from the decoy's files. The two remaining bare cd calls take absolute paths, which CDPATH is never consulted for. Impact BREAKING: skill-audit and agent-audit no longer exist as invocable skills. kyberforge goes to 2.0.0 (catalog 0.4.7). Check logic is unchanged: differential runs of the old and new validators across every skill and agent produced byte-identical stdout, stderr and exit codes, and the reconstructed Python payloads differ only in comments and the references/field-inventory.md -> agent-field-inventory.md rename. One doctrine governs the tiers: exit 0 is audited and clean, exit 1 is audited with findings OR a target present but unreadable, exit 2 is that nothing was audited at all. Edge paths DID change, deliberately (full table in ADR-0025): - a missing target exits 2 (never ran), not 1, under its own "does not exist" message; detection is by path shape, so a shape-matching path that is simply absent used to reach the validator and come back as a FAIL against a file that never existed; - an unshaped target exits 2 under the generic "matches neither" message, and a directory with no SKILL.md under a third, distinct one -- three exit-2 messages, not one; - a dangling symlink or a symlink loop stays exit 1: it is present but broken, which is a finding about the artifact rather than a usage error; - a SKILL.md file path is audited as its skill directory instead of refused; - a .md agent outside an agents/ directory is refused rather than audited; - a missing script library, a missing python3, a missing PyYAML, and no argument at all each exit 2. validate-provenance.sh already exited 2 for the last two; validate.sh now matches it. .pre-commit-hooks.yaml is a published contract consumed by external repos. Both hook IDs and both files: regexes are unchanged; only entry: and description: moved. scripts/check-vale-style-sync.sh (413), scripts/sync-vale-styles.sh (21), tests/test-check-vale-style-sync.sh (797) and agent-audit/scripts/README.md (47) are deleted. The checker made 17 assertions: 6 compared the two Vale copies and are moot; 10 are rehomed into tests/test-vale-wrap.sh (case 0, cases 28-31, and the suite's Vale-absent skip); and the cross-manifest files: agreement check, which selected hooks by entry: and so could not survive both hooks sharing one, is ported as case 33 pairing hooks by id:. Cases 28, 30 and 33 carry mutation self-tests; narrowing the local skill prefilter to 6 of 38 SKILL.md files now fails the suite. Skills go 39 to 38. Pre-push goes 9 repo-authored hooks to 8. ADR: 0025 BREAKING-CHANGE: the skill-audit and agent-audit skills are removed. Both flows are served by factory-audit, which auto-detects whether it was handed a skill directory or an agent file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YR2CjVumUbEGWcMikcoXBD
684 lines
34 KiB
Bash
Executable File
684 lines
34 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# lib-checks-agent.sh — SOURCED, never executed.
|
|
#
|
|
# agent-audit's structural check suite: everything in its validate.sh that is
|
|
# NOT the ADR-0020 shared boundary resolver, lifted verbatim and split at the
|
|
# resolver's markers. validate.sh reassembles
|
|
#
|
|
# $KYBERFORGE_AGENT_PREAMBLE_PY
|
|
# $KYBERFORGE_RESOLVER_PY (from lib-boundary-resolver.sh)
|
|
# $KYBERFORGE_AGENT_BODY_PY
|
|
#
|
|
# in that order — the order the resolver block sat in the original file — and
|
|
# feeds the result to python3 with the agent file as argv[1] and this scripts/
|
|
# directory as argv[2]. argv[2] is what the preamble resolves the frontmatter
|
|
# allowlist against, and the body's detect_scope() walk-up and provider
|
|
# detection are unchanged from agent-audit's copy.
|
|
#
|
|
# The dimension vocabulary and the tiers here are agent-audit's and are
|
|
# deliberately NOT reconciled with lib-checks-skill.sh's. Agents take the
|
|
# ADR-0020 description gates and no body word gate at all; adding one would
|
|
# contradict the ADR.
|
|
#
|
|
# Consumed by: validate.sh, agent mode.
|
|
# shellcheck shell=bash
|
|
# shellcheck disable=SC2034
|
|
|
|
kyberforge_agent_preflight() {
|
|
# PyYAML is a HARD dependency, not a nice-to-have. The description VALUE has to
|
|
# be measured after YAML folding is resolved, and the hand-rolled reader that
|
|
# used to stand in for PyYAML disagreed with it across the 400-character FAIL
|
|
# boundary — same description, two verdicts, depending on which reader ran.
|
|
# Refusing to start is the only honest option; the repo's jq / apm / vale
|
|
# dependencies are declared the same way.
|
|
# Check the interpreter separately from the library: `python3 -c` fails the same
|
|
# way whether python3 is missing or PyYAML is, and reporting the wrong missing
|
|
# dependency sends the reader to install the wrong thing.
|
|
if ! command -v python3 > /dev/null 2>&1; then
|
|
echo "Error: python3 is required but was not found on PATH." >&2
|
|
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
|
echo " Fix: install python3 (pre-commit itself is a Python application, so it is almost certainly already present)." >&2
|
|
# Exit 2, the never-ran tier: no check ran, so this is not a findings result.
|
|
# lib-provenance-*.sh has always exited 2 here; this matches it.
|
|
exit 2
|
|
fi
|
|
|
|
if ! python3 -c 'import yaml' > /dev/null 2>&1; then
|
|
echo "Error: PyYAML is required but is not importable by python3." >&2
|
|
echo " Why: skipping the ADR-0020 description and boundary-target gates would be a vacuous pass." >&2
|
|
echo " Fix: python3 -m pip install PyYAML (or your distro's python3-yaml package)." >&2
|
|
# Exit 2, the never-ran tier: a missing hard dependency is not a findings result.
|
|
exit 2
|
|
fi
|
|
}
|
|
|
|
IFS='' read -r -d '' KYBERFORGE_AGENT_PREAMBLE_PY <<'KYBERFORGE_AGENT_PREAMBLE' || true
|
|
import sys
|
|
import os
|
|
import re
|
|
import glob
|
|
|
|
import yaml
|
|
|
|
# Output is UTF-8 for the same reason input is: under LC_ALL=C the streams
|
|
# default to ASCII, and this script's own message text carries em dashes (the
|
|
# ADR-0020 boundary SUGGESTION is one). Pinning only the reads moved the crash
|
|
# from the read to the write — a UnicodeEncodeError raised while PRINTING, after
|
|
# every check has already run, which loses the whole report and (here) flips a
|
|
# clean exit 0 into a traceback and an exit 1. read_text() in the shared
|
|
# resolver block below pins the reads; this pins the writes.
|
|
#
|
|
# Deliberately OUTSIDE the ADR-0020 shared boundary resolver block: the two
|
|
# validate.sh copies print findings, skill-size-check.sh has its own top-level
|
|
# equivalent, and tests/test-adr0020-contract.sh hashes that block for
|
|
# byte-identity across all three.
|
|
for _stream in (sys.stdout, sys.stderr):
|
|
try:
|
|
_stream.reconfigure(encoding='utf-8')
|
|
except AttributeError: # pragma: no cover — Python < 3.7
|
|
pass
|
|
|
|
agent_file = os.path.abspath(sys.argv[1])
|
|
script_dir = sys.argv[2]
|
|
|
|
fname = os.path.basename(agent_file)
|
|
|
|
# --- Detect provider (check .agent.md before .md) ---
|
|
if fname.endswith('.agent.md'):
|
|
provider = 'copilot'
|
|
name_stem = fname[:-len('.agent.md')]
|
|
elif fname.endswith('.md'):
|
|
provider = 'claude-code'
|
|
name_stem = fname[:-len('.md')]
|
|
else:
|
|
print(f"Error: unrecognized extension '{fname}' — expected .md or .agent.md", file=sys.stderr)
|
|
sys.exit(2)
|
|
|
|
# --- Load agent-field-inventory.md ---
|
|
# The merged skill holds one references/ directory for both modes, so every
|
|
# flow-specific file is name-prefixed and the agent half of the inventory is
|
|
# agent-field-inventory.md (ADR-0025). There is no fallback to the pre-merge
|
|
# `field-inventory.md` spelling: agent-audit no longer exists, so a file at that
|
|
# name would be a stray, and silently reading it would mean auditing against an
|
|
# inventory this skill does not ship.
|
|
inv_path = os.path.normpath(os.path.join(script_dir, '..', 'references', 'agent-field-inventory.md'))
|
|
if not os.path.isfile(inv_path):
|
|
print(f"Error: agent-field-inventory.md not found at {inv_path}", file=sys.stderr)
|
|
sys.exit(2)
|
|
|
|
# Encoding is pinned to UTF-8 rather than inherited from the locale: under
|
|
# LC_ALL=C the inherited default is ASCII, and this file legitimately carries
|
|
# non-ASCII prose. read_text() in the shared resolver block below does the same
|
|
# thing for every other file; this one is read before that block is defined.
|
|
try:
|
|
with open(inv_path, encoding='utf-8') as f:
|
|
inv_content = f.read()
|
|
except UnicodeDecodeError as exc:
|
|
print(f"Error: agent-field-inventory.md at {inv_path} is not valid UTF-8 "
|
|
f"({exc.reason} at byte {exc.start}) — re-save it as UTF-8.",
|
|
file=sys.stderr)
|
|
sys.exit(2)
|
|
|
|
def parse_section_tokens(content, section_name):
|
|
lines = content.splitlines()
|
|
for i, line in enumerate(lines):
|
|
if line.strip() == f'## {section_name}':
|
|
for j in range(i + 1, len(lines)):
|
|
stripped = lines[j].strip()
|
|
if stripped and not stripped.startswith('#') and not stripped.startswith('---'):
|
|
return set(stripped.split())
|
|
return set()
|
|
|
|
cc_only_fields = parse_section_tokens(inv_content, 'claude-code-only-fields')
|
|
copilot_only_fields = parse_section_tokens(inv_content, 'copilot-only-fields')
|
|
apm_agent_allowlist = parse_section_tokens(inv_content, 'apm-agent-allowlist')
|
|
|
|
# Tools the runtime withholds from subagents regardless of the tools field
|
|
SUBAGENT_UNAVAILABLE_TOOLS = {
|
|
'AskUserQuestion', 'EnterPlanMode', 'ExitPlanMode', 'ScheduleWakeup', 'WaitForMcpServers',
|
|
}
|
|
|
|
# Copilot body length limit (chars) — content beyond this is silently truncated
|
|
COPILOT_BODY_LIMIT = 30000
|
|
|
|
# ADR-0020 description budget. An agent's name + description is preloaded into
|
|
# every session exactly like a skill's, so agents take the SAME description
|
|
# gates. These two constants are DUPLICATED in three places:
|
|
# scripts/skill-size-check.sh, lib-checks-skill.sh beside this file, and here.
|
|
# The repo-root hook's copy cannot be shared with this skill — a cache-installed
|
|
# plugin's scripts cannot read files outside their own plugin directory, and the
|
|
# hook cannot reach inside the plugin. The two copies INSIDE this skill could be
|
|
# shared (ADR-0025: two files in one skill may source a third), and are not only
|
|
# because each mode library is a verbatim lift of the pre-merge suite whose
|
|
# constants sit in its Python preamble; hoisting them is a separate change.
|
|
# tests/test-skill-size-check.sh asserts all three agree, so drift fails CI
|
|
# rather than silently diverging.
|
|
#
|
|
# Agents deliberately take NO body word gate, and adding one here would
|
|
# contradict ADR-0020: a skill body is loaded into the caller's context and
|
|
# competes with the live conversation, while an agent body becomes the system
|
|
# prompt of a fresh context. The rationale for the 900-word skill ceiling does
|
|
# not transfer. Agent body length falls out of the delegation rule instead.
|
|
DESC_SUGGEST_CHARS = 250
|
|
DESC_MAX_CHARS = 400
|
|
|
|
# --- Helpers (shared by every scope) ---
|
|
failed = False
|
|
suggestions = []
|
|
|
|
def fail(msg):
|
|
# stderr, matching scripts/skill-size-check.sh's ERROR routing. All three
|
|
# scripts in the ADR-0020 family now agree: findings that fail the run go to
|
|
# stderr, everything advisory (SUGGESTION / INFO) goes to stdout. Both repo
|
|
# callers (check-apm-agents-valid.sh, check-scope-walkup-sync.sh) capture
|
|
# `2>&1`, so nothing a human reads moves.
|
|
global failed
|
|
failed = True
|
|
print(f"FAIL {msg}", file=sys.stderr)
|
|
|
|
def suggest(msg):
|
|
suggestions.append(msg)
|
|
|
|
def info(msg):
|
|
# A check that DECLINED to run says so out loud, rather than passing
|
|
# silently. Silence is what let a whole gate family go missing unnoticed.
|
|
print(f"INFO {msg}")
|
|
|
|
PLACEHOLDER_RE = re.compile(r'(?<!`)FILL IN:[^`\n]')
|
|
|
|
|
|
KYBERFORGE_AGENT_PREAMBLE
|
|
KYBERFORGE_AGENT_PREAMBLE_PY="${KYBERFORGE_AGENT_PREAMBLE_PY%$'\n'}"
|
|
|
|
IFS='' read -r -d '' KYBERFORGE_AGENT_BODY_PY <<'KYBERFORGE_AGENT_BODY' || true
|
|
|
|
|
|
def parse_frontmatter(content):
|
|
m = FRONTMATTER_RE.match(strip_bom(content))
|
|
if not m:
|
|
return None, content
|
|
return m.group(1), strip_bom(content)[m.end():]
|
|
|
|
def extract_field(fm, field):
|
|
"""The raw text after `field:` ON ITS OWN LINE, or None.
|
|
|
|
The character class is `[^\\S\\r\\n]`, never `\\s`: under re.MULTILINE a
|
|
`\\s*` after the colon crosses the newline, so `description:` with no value
|
|
followed by `model: sonnet` captured `model: sonnet` as the description.
|
|
That made the value look present, skipped the "missing or empty" failure,
|
|
and then every ADR-0020 gate early-returned on the genuinely empty folded
|
|
value — a valueless description exited 0 with zero output on a BLOCKING
|
|
pre-push gate. This function is now used only for fields with no folding
|
|
semantics (name, tools); description goes through description_value(), the
|
|
shared resolver's YAML reader, which is the only thing that can see through
|
|
`>`, `null`, `''` and a quoted `"description"` key alike.
|
|
"""
|
|
m = re.search(rf'^{re.escape(field)}:[^\S\r\n]*(.+)', fm, re.MULTILINE)
|
|
return m.group(1).strip() if m else None
|
|
|
|
def get_frontmatter_keys(fm):
|
|
keys = set()
|
|
for line in fm.splitlines():
|
|
m = re.match(r'^([a-zA-Z][a-zA-Z0-9_-]*):', line)
|
|
if m:
|
|
keys.add(m.group(1))
|
|
return keys
|
|
|
|
def agent_description(fm, local_fname):
|
|
"""The folded description VALUE, or None if it could not be read."""
|
|
try:
|
|
return description_value(fm)
|
|
except FrontmatterError as exc:
|
|
# `exc` carries the whole clause — invalid YAML, a non-mapping block, or
|
|
# a description of the wrong type. Do not prefix a diagnosis here; the
|
|
# last one named a syntax error for two failures that have none.
|
|
fail(f"{exc} — the ADR-0020 description and boundary-target gates could "
|
|
f"not run — {local_fname}")
|
|
return None
|
|
|
|
def check_description_budget(value, local_fname, by_hand=False):
|
|
"""ADR-0020 description gates — identical for every scope.
|
|
|
|
`by_hand` is ADR-0020's hand-invocation carve-out (issue #108): an agent
|
|
carrying `disable-model-invocation: true` is absent from the model-visible
|
|
listing, so the 250-character SUGGESTION — a routing-quality budget — has
|
|
no listing to apply to. The 400-character ceiling is unaffected.
|
|
"""
|
|
if not value:
|
|
return
|
|
dlen = len(value)
|
|
if dlen > DESC_MAX_CHARS:
|
|
fail(f"description is {dlen} chars — exceeds the {DESC_MAX_CHARS}-character "
|
|
f"ADR-0020 ceiling. It is preloaded into every session whether or not the "
|
|
f"agent is invoked. Keep a trigger clause, at most one capability clause, "
|
|
f"and a boundary clause; move capability enumeration, output-format detail, "
|
|
f"composition notes and implementation detail to the body — {local_fname}")
|
|
elif dlen > DESC_SUGGEST_CHARS and not by_hand:
|
|
suggest(f"description is {dlen} chars — over the {DESC_SUGGEST_CHARS}-character "
|
|
f"ADR-0020 target (hard fail at {DESC_MAX_CHARS}). The SUGGESTION tier is "
|
|
f"what moves the corpus average; the FAIL tier only stops outliers "
|
|
f"— {local_fname}")
|
|
|
|
def check_boundary(value, fpath, local_fname, by_hand=False):
|
|
"""ADR-0020 boundary clause + resolvable boundary targets.
|
|
|
|
agent-author's SKILL.md states that an agent's boundary targets must
|
|
resolve, but until this ran no script checked it — the contract was
|
|
documented and unenforced. The resolution universe is derived from the
|
|
AGENT FILE's own location (the authoring root above it, its own apm
|
|
package, and that package's declared apm dependencies), never from this
|
|
script's path, and — when an authoring root exists — never from a deployed
|
|
.claude/ tree, so a fresh clone and a machine that has run `apm install`
|
|
return the same verdict.
|
|
"""
|
|
if not value:
|
|
return
|
|
# SUGGESTION, not FAIL: detecting the absence is deterministic, but whether
|
|
# this particular agent warrants a boundary clause is judgment. All four
|
|
# agents in this corpus currently lack one.
|
|
#
|
|
# THREE outcomes, not two: "no boundary clause" and "boundary clause I could
|
|
# not parse" are different findings (issue #110). And a hand-invoked agent is
|
|
# exempt from the clause altogether (issue #108) — the boundary-target
|
|
# resolution below still runs, because a target it DOES name should still
|
|
# resolve.
|
|
status = boundary_clause_status(value) if not by_hand else 'present'
|
|
if status == 'absent':
|
|
suggest(f"description has no boundary clause — add the prose form (\"Do not use "
|
|
f"for X — use `y` instead\") or ADR-0020's compressed form (\"Not X -> y\") "
|
|
f"so the router knows where NOT to send this agent — {local_fname}")
|
|
elif status == 'unparsed':
|
|
suggest(f"description has an arrow boundary clause (\"Not X -> y\") from which no "
|
|
f"target could be read, so the dangling-target check did not run on it — "
|
|
f"the clause is PRESENT and unparsed, not missing. Most often the target "
|
|
f"is a single word, which is deliberately not matchable bare: write it as "
|
|
f"`name` or /name — {local_fname}")
|
|
if not by_hand:
|
|
# One arrow, one target: a second name after the same arrow is resolved
|
|
# by nothing and reported by nothing (issue #107).
|
|
for first, second in multi_target_arrow_clauses(value):
|
|
suggest(f"an arrow boundary clause names more than one target ('{first}', then "
|
|
f"'{second}') and only the first is resolved — the second is checked by "
|
|
f"nothing. Split it into one arrow per target: \"Not X -> {first}. "
|
|
f"Not Y -> {second}.\" — {local_fname}")
|
|
targets = boundary_targets(value)
|
|
if not targets:
|
|
return
|
|
known = known_targets(os.path.dirname(os.path.abspath(fpath)))
|
|
if not known:
|
|
info(f"boundary-target resolution DID NOT RUN — no skill universe could be "
|
|
f"determined for this path (no authoring root above it, no apm package "
|
|
f"root, no declared apm dependencies, no deployed .claude/ or .agents/ "
|
|
f"tree). Unchecked target(s): {', '.join(targets)} — {local_fname}")
|
|
return
|
|
# blocking vs reported: a target only earns a FAIL when it is written in
|
|
# route notation or its own sentence corroborates it by naming another target
|
|
# that resolves. See the shared resolver's CORROBORATION note.
|
|
blocking, reported = unresolved_targets(value, known)
|
|
for target in blocking:
|
|
fail(f"description routes to '{target}', which resolves to no skill or agent "
|
|
f"in this monorepo, in this package, or in a package it declares in "
|
|
f"apm.yml dependencies.apm — a boundary clause naming a non-existent "
|
|
f"target sends the router nowhere — {local_fname}")
|
|
for target in reported:
|
|
suggest(f"description routes to '{target}', which resolves to no skill or agent "
|
|
f"in this monorepo, in this package, or in a package it declares in "
|
|
f"apm.yml dependencies.apm — SUGGESTION rather than FAIL because nothing "
|
|
f"else in that sentence resolves, so it is equally likely to be a tool, a "
|
|
f"file format or an English compound. If it IS a route, write it as "
|
|
f"`/{target}` or `-> {target}` and it will be checked properly — "
|
|
f"{local_fname}")
|
|
|
|
def extract_tools_list(fm):
|
|
"""Tool names from the `tools` field — inline scalar OR YAML block sequence.
|
|
|
|
Read off the PARSED mapping, never off extract_field(). That function's
|
|
capture is newline-bounded on purpose (`[^\\S\\r\\n]*(.+)`), so a `tools:`
|
|
written as a block sequence — the shape Copilot agent files use — captured
|
|
nothing at all and the subagent-unavailable-tool check silently stopped
|
|
firing on exactly the files it was written for. Both spellings are legal
|
|
YAML, so both are read here.
|
|
"""
|
|
try:
|
|
data = yaml.safe_load(fm)
|
|
except Exception:
|
|
# Not this function's failure to report: the frontmatter's validity is
|
|
# decided (and failed) by agent_description() on the same text.
|
|
return set()
|
|
if not isinstance(data, dict):
|
|
return set()
|
|
val = data.get('tools')
|
|
if isinstance(val, list):
|
|
items = [str(item).strip() for item in val]
|
|
elif isinstance(val, str):
|
|
items = re.split(r'[\s,]+', val.strip())
|
|
else:
|
|
return set()
|
|
return {item for item in items if item}
|
|
|
|
def is_copilot_cloud_ide(fpath):
|
|
"""True if the file is a cloud/IDE Copilot agent (name is optional for these)."""
|
|
return '.github/copilot/agents' in os.path.abspath(fpath).replace(os.sep, '/')
|
|
|
|
# --- Detect scope ---
|
|
# APM_TYPE_RE matches a top-level (column-0) `type:` line in apm.yml whose value is
|
|
# exactly one of the four package content types. Group 1 captures an optional
|
|
# opening quote; \1 requires the same character (or nothing) to close it, so
|
|
# "skill" and '"skill"' both match but a mismatched quote doesn't. The value
|
|
# must then be followed by whitespace or end-of-line — not just a non-word
|
|
# character — so a malformed value like `prompts-only` is correctly rejected
|
|
# instead of false-matching on the `prompts` prefix.
|
|
APM_TYPE_RE = re.compile(r"^type:\s*(['\"]?)(instructions|skill|hybrid|prompts)\1(?:\s|$)")
|
|
|
|
def find_apm_package_root(apm_yml_path):
|
|
"""Return True if apm_yml_path has a top-level type: line (i.e. is a package
|
|
manifest, not a type:-less marketplace-only apm.yml)."""
|
|
# errors='replace', not a hard failure: this only asks whether a `type:`
|
|
# line exists, and a stray undecodable byte elsewhere in someone else's
|
|
# apm.yml must not abort scope detection.
|
|
with open(apm_yml_path, encoding='utf-8', errors='replace') as f:
|
|
for line in f:
|
|
if APM_TYPE_RE.match(line):
|
|
return True
|
|
return False
|
|
|
|
def detect_scope(start_dir):
|
|
home = os.path.expanduser('~')
|
|
original_start = os.path.abspath(start_dir)
|
|
# Agent files conventionally live exactly two path segments below their
|
|
# scope root — <root>/.claude/agents, <root>/.github/agents,
|
|
# <root>/.copilot/agents, or <root>/.apm/agents (see new-agent.sh's
|
|
# CC_DIR/CP_DIR and user-scope dirs). Stripping those two segments
|
|
# recovers the same root new-agent.sh would have been invoked with to
|
|
# produce this exact file, independent of how far the walk below has to
|
|
# travel to find (or fail to find) a marker — mirrors new-agent.sh's
|
|
# `root` vs `current` distinction even though validate.sh is handed a
|
|
# file's directory, not the scope root itself.
|
|
#
|
|
# That arithmetic is only trustworthy when the path actually has this
|
|
# shape: parent directory literally named "agents", grandparent one of
|
|
# the four known scope-dir names. A hand-placed or otherwise
|
|
# non-conventional agent file (never produced by new-agent.sh) has no
|
|
# such guarantee — blindly trusting two-segments-up there could point at
|
|
# an unrelated ancestor. conventional_shape gates every use of
|
|
# conventional_root below; when it's false, the walked-to `current`
|
|
# directory is used instead, the same fallback this function used before
|
|
# conventional_root existed.
|
|
scope_dir_name = os.path.basename(os.path.dirname(original_start))
|
|
conventional_shape = (
|
|
os.path.basename(original_start) == 'agents'
|
|
and scope_dir_name in ('.claude', '.github', '.copilot', '.apm')
|
|
)
|
|
conventional_root = os.path.dirname(os.path.dirname(original_start))
|
|
current = original_start
|
|
while True:
|
|
# The filesystem root is never a candidate, the same guard the shared
|
|
# resolver's walk-up loops carry. Without it a file under a marker-less
|
|
# temp directory walked all the way to `/` and returned it as the scope
|
|
# root, which then reported `counterpart file not found:
|
|
# /.claude/agents/<name>.md` — a path that names someone else's machine,
|
|
# not the user's project. When the walk runs out, the agent file's own
|
|
# directory (or its conventional root) is the honest answer.
|
|
if _is_fs_root(current):
|
|
return 'project', conventional_root if conventional_shape else original_start
|
|
apm_yml = os.path.join(current, 'apm.yml')
|
|
if os.path.isfile(apm_yml) and find_apm_package_root(apm_yml):
|
|
return 'plugin', current
|
|
# $HOME is the user-scope boundary — checked before the .git test
|
|
# below, so a dotfiles-managed $HOME (yadm, chezmoi bare-repo, etc.)
|
|
# can't shadow user scope by being its own .git repo. 'user' scope
|
|
# requires EITHER start_dir to BE $HOME itself (no walk-up — the
|
|
# new-agent.sh "root exactly $HOME" case) OR start_dir to sit at the
|
|
# conventional two-segments-below-root depth (i.e. $HOME IS that
|
|
# root, matching the real ~/.claude/agents or ~/.copilot/agents
|
|
# shape). Any other walk-up into $HOME — a marker-less directory
|
|
# nested deeper than that convention — resolves to project scope
|
|
# instead: a stray directory under $HOME can't be silently
|
|
# redirected into the shared global ~/.claude or ~/.copilot agent
|
|
# directories.
|
|
if current == home:
|
|
if original_start == home or (conventional_shape and conventional_root == home):
|
|
return 'user', home
|
|
return 'project', conventional_root if conventional_shape else current
|
|
# .git is a directory in a normal checkout but a file (`gitdir: ...`)
|
|
# in a git worktree — exists() covers both. Returns conventional_root,
|
|
# not current: new-agent.sh's project-scope file placement always
|
|
# uses its `$ROOT` argument directly, never the walked-up `.git`
|
|
# location, so a <root> one or more levels below the repo's .git
|
|
# (a subdirectory of a larger git-tracked tree — explicitly a
|
|
# supported case per new-agent.sh's usage text) must resolve to the
|
|
# same root new-agent.sh actually wrote to, not to the .git dir —
|
|
# unless the path lacks the conventional shape, in which case that
|
|
# arithmetic isn't trustworthy and current is used instead.
|
|
if os.path.exists(os.path.join(current, '.git')):
|
|
return 'project', conventional_root if conventional_shape else current
|
|
parent = os.path.dirname(current)
|
|
if parent == current:
|
|
return 'project', conventional_root if conventional_shape else current
|
|
current = parent
|
|
|
|
agent_dir = os.path.dirname(agent_file)
|
|
scope, scope_root = detect_scope(agent_dir)
|
|
|
|
# --- Plugin/APM scope: single vendor-neutral file, no counterpart ---
|
|
def check_apm_agent_file(fpath, allowlist, stem):
|
|
local_fname = os.path.basename(fpath)
|
|
try:
|
|
content = read_text(fpath)
|
|
except EncodingError as exc:
|
|
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
|
f"failure, not a skip — {local_fname}")
|
|
return
|
|
except OSError as exc:
|
|
# A path that cannot be opened gets a FAIL line naming it, not a bare
|
|
# FileNotFoundError traceback. scripts/check-apm-agents-valid.sh takes
|
|
# this path for an agent file deleted from the worktree but still
|
|
# tracked in the index — a real, expected state, and the caller needs to
|
|
# be told which file, not handed an interpreter stack.
|
|
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
|
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
|
return
|
|
|
|
fm, body = parse_frontmatter(content)
|
|
if fm is None:
|
|
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
|
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
|
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
|
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
|
return
|
|
|
|
# The apm-agent.md template embeds its authoring guidance as HTML
|
|
# comments inside the frontmatter block (so they render invisible in a
|
|
# Markdown preview but stay visible in the raw file). get_frontmatter_keys
|
|
# silently ignores any line that isn't a `key:` match, so a comment left
|
|
# behind at ship time would otherwise pass unnoticed — yet apm compile
|
|
# copies this frontmatter verbatim to both harnesses, and `<!-- -->` is
|
|
# not valid YAML, so yaml.safe_load breaks on both downstream (ADR-0016).
|
|
if re.search(r'<!--|-->', fm):
|
|
fail(f"frontmatter still contains template HTML comments (<!-- ... -->) "
|
|
f"— delete them before shipping — {local_fname}")
|
|
|
|
# Allowlist: the permitted keys are data, read at load time from
|
|
# references/agent-field-inventory.md's `## apm-agent-allowlist` section — do not
|
|
# restate them here, or this comment goes stale the next time that line
|
|
# changes. apm compile verbatim-copies frontmatter to every target, so a key
|
|
# outside the list is unsafe on at least one harness (ADR-0016). Note the
|
|
# list admits denylist-shaped restrictions (disallowedTools) but never
|
|
# allowlist-shaped ones (tools), whose value shape differs per harness.
|
|
fm_keys = get_frontmatter_keys(fm)
|
|
for key in sorted(fm_keys):
|
|
if key not in allowlist:
|
|
fail(f"field '{key}' is not in the vendor-neutral APM agent allowlist "
|
|
f"({', '.join(sorted(allowlist))}) — {local_fname}")
|
|
|
|
# name — required, kebab-case, must match filename stem (file is <name>.agent.md)
|
|
name_val = extract_field(fm, 'name')
|
|
if not name_val:
|
|
fail(f"name field is missing or empty — {local_fname}")
|
|
else:
|
|
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
|
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
|
if name_val != stem:
|
|
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
|
|
|
# description — required, non-empty, no placeholder
|
|
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
|
# it on extract_field's raw capture is what let `description:` with no value
|
|
# pass this gate in total silence: the capture picked up the next key, so
|
|
# "missing or empty" never fired, and every ADR-0020 check below then
|
|
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
|
folded = agent_description(fm, local_fname)
|
|
if folded is None:
|
|
pass # frontmatter is not valid YAML — agent_description already failed
|
|
elif not folded:
|
|
fail(f"description field is missing or empty — {local_fname}")
|
|
else:
|
|
if PLACEHOLDER_RE.search(folded):
|
|
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
|
by_hand = hand_invoked(fm)
|
|
check_description_budget(folded, local_fname, by_hand)
|
|
check_boundary(folded, fpath, local_fname, by_hand)
|
|
|
|
# body — required, non-empty, no placeholder; same Copilot truncation risk
|
|
# applies since this file compiles verbatim into a real Copilot file downstream.
|
|
if not body.strip():
|
|
fail(f"system prompt body is empty — {local_fname}")
|
|
else:
|
|
if PLACEHOLDER_RE.search(body):
|
|
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
|
if len(body) > COPILOT_BODY_LIMIT:
|
|
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — "
|
|
f"content beyond the limit is silently truncated by the Copilot runtime "
|
|
f"once apm compile emits it downstream — {local_fname}")
|
|
|
|
if scope == 'plugin':
|
|
check_apm_agent_file(agent_file, apm_agent_allowlist, name_stem)
|
|
for s in suggestions:
|
|
print(f"SUGGESTION {s}")
|
|
sys.exit(1 if failed else 0)
|
|
|
|
# --- Project/user scope: unchanged CC/Copilot pair validation ---
|
|
|
|
# --- Derive counterpart path ---
|
|
if scope == 'project':
|
|
if provider == 'claude-code':
|
|
counterpart = os.path.join(scope_root, '.github', 'agents', name_stem + '.agent.md')
|
|
counterpart_provider = 'copilot'
|
|
else:
|
|
counterpart = os.path.join(scope_root, '.claude', 'agents', name_stem + '.md')
|
|
counterpart_provider = 'claude-code'
|
|
else: # user
|
|
home = os.path.expanduser('~')
|
|
if provider == 'claude-code':
|
|
counterpart = os.path.join(home, '.copilot', 'agents', name_stem + '.agent.md')
|
|
counterpart_provider = 'copilot'
|
|
else:
|
|
counterpart = os.path.join(home, '.claude', 'agents', name_stem + '.md')
|
|
counterpart_provider = 'claude-code'
|
|
|
|
def check_file(fpath, file_provider):
|
|
local_fname = os.path.basename(fpath)
|
|
try:
|
|
content = read_text(fpath)
|
|
except EncodingError as exc:
|
|
fail(f"file is {exc}. Nothing could be measured, so this is a hard "
|
|
f"failure, not a skip — {local_fname}")
|
|
return
|
|
except OSError as exc:
|
|
# Same reason as check_apm_agent_file's: a diagnostic naming the path
|
|
# beats a FileNotFoundError traceback. The counterpart is pre-checked at
|
|
# the bottom of this script, but agent_file itself never was.
|
|
fail(f"could not be read ({exc.strerror or exc}): {fpath}. Nothing could "
|
|
f"be measured, so this is a hard failure, not a skip — {local_fname}")
|
|
return
|
|
|
|
fm, body = parse_frontmatter(content)
|
|
if fm is None:
|
|
fail(f"no parseable YAML frontmatter block — expected a `---` line, the fields, "
|
|
f"then a closing `---` line (a BOM, leading blank lines, trailing spaces "
|
|
f"after either marker and CRLF endings are all tolerated). Nothing could be "
|
|
f"measured, so this is a hard failure, not a skip — {local_fname}")
|
|
return
|
|
|
|
# name — required for CC and Copilot CLI; optional for Copilot cloud/IDE agents
|
|
cloud_ide = (file_provider == 'copilot' and is_copilot_cloud_ide(fpath))
|
|
name_val = extract_field(fm, 'name')
|
|
if not cloud_ide:
|
|
if not name_val:
|
|
fail(f"name field is missing or empty — {local_fname}")
|
|
else:
|
|
if not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
|
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
|
# Stem check applies to Copilot CLI only; CC docs say filename need not match name
|
|
if file_provider == 'copilot':
|
|
stem = local_fname[:-len('.agent.md')]
|
|
if name_val != stem:
|
|
fail(f"name '{name_val}' does not match filename stem '{stem}' — {local_fname}")
|
|
elif name_val and not re.match(r'^[a-z0-9]+(-[a-z0-9]+)*$', name_val):
|
|
# cloud/IDE: name is optional, but if present it must be valid
|
|
fail(f"name '{name_val}' is not kebab-case — {local_fname}")
|
|
|
|
# description
|
|
# Presence is decided on the FOLDED value, never on a line regex. Deciding
|
|
# it on extract_field's raw capture is what let `description:` with no value
|
|
# pass this gate in total silence: the capture picked up the next key, so
|
|
# "missing or empty" never fired, and every ADR-0020 check below then
|
|
# early-returned on the empty folded value. Exit 0, zero output, no gate run.
|
|
folded = agent_description(fm, local_fname)
|
|
if folded is None:
|
|
pass # frontmatter is not valid YAML — agent_description already failed
|
|
elif not folded:
|
|
fail(f"description field is missing or empty — {local_fname}")
|
|
else:
|
|
if PLACEHOLDER_RE.search(folded):
|
|
fail(f"description contains unfilled FILL IN: placeholder — {local_fname}")
|
|
by_hand = hand_invoked(fm)
|
|
check_description_budget(folded, local_fname, by_hand)
|
|
check_boundary(folded, fpath, local_fname, by_hand)
|
|
|
|
# body
|
|
if not body.strip():
|
|
fail(f"system prompt body is empty — {local_fname}")
|
|
else:
|
|
if PLACEHOLDER_RE.search(body):
|
|
fail(f"body contains unfilled FILL IN: placeholder — {local_fname}")
|
|
# Copilot body length limit
|
|
if file_provider == 'copilot' and len(body) > COPILOT_BODY_LIMIT:
|
|
suggest(f"body exceeds {COPILOT_BODY_LIMIT:,} characters ({len(body):,} chars) — content beyond the limit is silently truncated by the Copilot runtime — {local_fname}")
|
|
|
|
# CC-only fields in Copilot file
|
|
if file_provider == 'copilot':
|
|
fm_keys = get_frontmatter_keys(fm)
|
|
for key in sorted(fm_keys):
|
|
if key in cc_only_fields:
|
|
fail(f"CC-only field '{key}' present in Copilot file — {local_fname}")
|
|
|
|
# Copilot-only fields in CC file
|
|
if file_provider == 'claude-code':
|
|
fm_keys = get_frontmatter_keys(fm)
|
|
for key in sorted(fm_keys):
|
|
if key in copilot_only_fields:
|
|
fail(f"Copilot-only field '{key}' present in CC file — {local_fname}")
|
|
|
|
# Subagent-unavailable tools listed in tools field
|
|
tools = extract_tools_list(fm)
|
|
unavailable = tools & SUBAGENT_UNAVAILABLE_TOOLS
|
|
for tool in sorted(unavailable):
|
|
suggest(f"'{tool}' is listed in tools but is never available to subagents — the runtime withholds it regardless — {local_fname}")
|
|
|
|
# --- Check counterpart exists ---
|
|
if not os.path.isfile(counterpart):
|
|
fail(f"counterpart file not found: {counterpart}")
|
|
sys.exit(1)
|
|
|
|
# --- Check both files ---
|
|
check_file(agent_file, provider)
|
|
check_file(counterpart, counterpart_provider)
|
|
|
|
for s in suggestions:
|
|
print(f"SUGGESTION {s}")
|
|
|
|
sys.exit(1 if failed else 0)
|
|
KYBERFORGE_AGENT_BODY
|
|
KYBERFORGE_AGENT_BODY_PY="${KYBERFORGE_AGENT_BODY_PY%$'\n'}"
|