fix(kyberforge): fix scope walk-up and manifest-parsing bugs from PR #93 review

A fresh /code-review of the APM-native authoring retarget (PR #93) found
several correctness bugs beyond the ones already fixed on this branch:

- new-agent.sh silently walked a marker-less subdirectory under $HOME up
  to user scope, contradicting its own usage text ("user scope is checked
  directly, no walk-up") and risking scaffolding into shared global
  ~/.claude or ~/.copilot directories instead of the intended local path.
- The hand-copied apm.yml type: manifest detector in new-agent.sh and
  new-skill.sh accepted mismatched quotes (e.g. `type: "skill'`) that
  validate.sh's regex correctly rejects, and silently dropped a final
  apm.yml line lacking a trailing newline — causing the scaffolder and
  validator to disagree on scope for identical input.
- Plugin-scope agent frontmatter could still contain the apm-agent.md
  template's HTML comments at ship time with no audit signal, yet
  apm compile copies frontmatter verbatim and <!-- --> breaks YAML
  parsing on both downstream harnesses.
- ADR-0016 asserted agent-audit already implements a SUGGESTION heuristic
  for tool-restriction-needing plugin-scope agents; it doesn't.
- agent-audit/README.md still described the old plugin-pair model this
  PR replaced with a single-file allowlist model.
- validate.sh's project/user-scope CC-only/Copilot-only field checks and
  counterpart-missing check lost their only test coverage when the old
  plugin-pair fixture was deleted.

Also replaces an echo-into-sed two-value parse (4 forks per call) with a
single space-separated echo + read in both scaffolders.

Regression tests added for every fix above, including one for a bug this
pass introduced and the test suite caught: an initial two-line
echo + `read` attempt silently dropped the second value, since `read`
consumes only one line regardless of embedded newlines.

Full suite: 158 bats tests, 39 shell-script tests, 12/12 summary
categories, 0 failures.

Refs: #89, #93
This commit is contained in:
2026-08-11 21:49:38 +00:00
parent f037d49b5c
commit 6f6b70781d
8 changed files with 254 additions and 63 deletions

View File

@@ -1,10 +1,27 @@
# agent-audit
Audits a Claude Code and Copilot agent definition file pair for correctness and quality.
Audits an agent definition for correctness and quality — a single vendor-neutral file at
plugin/APM scope, or a Claude Code and Copilot file pair at project/user scope.
## What it does
Accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives the counterpart automatically, and validates both. Runs structural checks via `validate.sh` (required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, silently-ignored fields at plugin scope), provenance chain validation via `validate-provenance.sh` (checks `source_keys` against `sources.md` at the plugin root), then qualitative checks on description phrasing and system prompt quality. Step 1 also runs a Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the `Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is unavailable or reports `0 files` scanned. Produces a compact findings report in the same format as `skill-audit`.
At **plugin/APM scope**, accepts the single `.apm/agents/<name>.agent.md` file — there is no
counterpart. Structural checks via `validate.sh` hard-`FAIL` any frontmatter field outside the
vendor-neutral allowlist (`name`, `description`, `model` — see ADR-0016), since `apm compile`
copies frontmatter verbatim to both harnesses and an unsafe field can't be silently dropped for
just one of them.
At **project/user scope**, accepts either file in a CC `.md` / Copilot `.agent.md` pair, derives
the counterpart automatically, and validates both. Runs structural checks via `validate.sh`
(required fields, kebab-case name, no placeholders, no CC-only fields in the Copilot file, no
Copilot-only fields in the CC file), provenance chain validation via `validate-provenance.sh`
(checks `source_keys` against `sources.md` at the plugin root — plugin/APM scope only), then
qualitative checks on description phrasing and system prompt quality. Step 1 also runs a
Vale-based prose sub-check via `vale-wrap.sh` against both files of the pair, using the
`Kyberforge` style (both files) and `KyberforgeCopilot` style (Copilot file only) — every alert
is a `FAIL`, cited by rule ID — falling back to Step 2 judgment when the `vale` binary is
unavailable or reports `0 files` scanned. Produces a compact findings report in the same format
as `skill-audit`.
## Usage