fix(kyberforge): harden plugin-content sync, reinject Copilot mcpServers
PR #95's review of the issue #90 apm-conversion work found several defects in scripts/sync-plugin-content.sh and the gate wired to it: - --check claimed never to mutate the plugin root, but apm pack still wrote .claude-plugin/plugin.json and .github/plugin/plugin.json into the real plugin_dir on first-time creation. --check now packs a throwaway copy instead. - check-plugin-content-sync hardcoded the six plugin directories instead of deriving them the way check-manifests.sh already does. Added an --all flag that parses .claude-plugin/marketplace.json, and simplified the pre-commit hook to use it. - A missing plugin_dir and one that legitimately has no .apm/ yet both reported SKIP/success; a missing directory now FAILs. - The dispatch loop backgrounded every plugin with no concurrency cap, unlike the JOBS-bounded pattern this same PR added to tests/run-bats.sh and tests/run-tests.sh. Added the same bash-3.2-safe getconf + batched-wait cap here for consistency. - Per-plugin scratch/log/status files were keyed only by basename, with no collision guard across arguments; added a fail-fast check. - sync_hooks_json()'s trailing-newline normalization was duplicated between its --check and write branches; factored into one helper. - tests/test-sync-plugin-content.sh set two competing `trap ... EXIT` statements, so the first (cleaning up $FIXTURE) was silently replaced by the second and its tmp dir leaked every run. Adopted the track()/CLEANUP_DIRS pattern already used in tests/test-check-release-needed.sh. Separately: apm's Copilot-ecosystem plugin.json builder unconditionally strips mcpServers, citing (in its own docstring) that the field is out of schema for Copilot -- a claim this repo's own researched Copilot plugin schema docs contradict. reinject_mcp_servers() narrowly restores it from the plugin's .mcp.json on real syncs only, regenerating plugins/bin/.github/plugin/plugin.json (the only plugin that currently declares any MCP servers). Documented as an amendment to ADR-0017, since it's a deliberate, narrow exception to that ADR's rejection of patching apm's compiled output -- apm's premise for stripping skills/agents/commands/hooks pointers is still accurate; its premise for stripping mcpServers is not. All 12 assertions in tests/test-sync-plugin-content.sh pass individually, plus 5 new regression tests added for this round; the full bats and shell-script suites are green; shellcheck is clean. Refs: #95 ADR: 0017
This commit is contained in:
@@ -22,8 +22,20 @@ set -euo pipefail
|
||||
# refreshes both files from current apm.yml/.apm/ content -- apm pack silently skips
|
||||
# regenerating an existing plugin.json otherwise ("already exists; skipping plugin.json
|
||||
# generation"), which would let them go stale after a name/version/description edit.
|
||||
# --check does NOT pass --force (it must not mutate the plugin root), so plugin.json
|
||||
# staleness is not currently detected by --check -- only fixed by the next real sync.
|
||||
#
|
||||
# apm's Copilot-ecosystem plugin.json builder omits mcpServers entirely -- its own
|
||||
# docstring calls it out-of-schema for Copilot, but this repo's researched Copilot
|
||||
# plugin schema docs (plugins/kyberforge/docs/research/docs/github-copilot-plugins/
|
||||
# configuration.md) document mcpServers as valid there. Real-mode syncs re-inject it
|
||||
# into .github/plugin/plugin.json from the plugin's own .mcp.json after apm pack runs
|
||||
# (see reinject_mcp_servers below); staleness there, like the rest of plugin.json, is
|
||||
# only fixed by the next real sync, not detected by --check.
|
||||
#
|
||||
# apm pack also writes .claude-plugin/plugin.json and .github/plugin/plugin.json into
|
||||
# cwd whenever those files don't already exist yet -- regardless of --force -- so
|
||||
# --check (which must never mutate the real plugin root) never cds into plugin_dir
|
||||
# directly. It packs a throwaway copy instead (see sync_one's pack_cwd); only that
|
||||
# copy's manifest files, never the real ones, can get created as a first-write.
|
||||
#
|
||||
# hooks.json is only synced when .apm/hooks/ actually produces one -- a plugin with
|
||||
# no .apm/hooks/ content is left alone even if a root-level hooks.json already exists
|
||||
@@ -37,7 +49,7 @@ set -euo pipefail
|
||||
# the duplicate and double-running the original under any repo-wide bats/test discovery.
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 [--check] <plugin-dir> [<plugin-dir> ...]" >&2
|
||||
echo "Usage: $0 [--check] (--all | <plugin-dir> [<plugin-dir> ...])" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
@@ -46,13 +58,29 @@ if [[ "${1:-}" == "--check" ]]; then
|
||||
CHECK=1
|
||||
shift
|
||||
fi
|
||||
[[ $# -ge 1 ]] || usage
|
||||
|
||||
ALL=0
|
||||
if [[ "${1:-}" == "--all" ]]; then
|
||||
ALL=1
|
||||
shift
|
||||
fi
|
||||
|
||||
if [[ "$ALL" -eq 1 ]]; then
|
||||
[[ $# -eq 0 ]] || usage
|
||||
else
|
||||
[[ $# -ge 1 ]] || usage
|
||||
fi
|
||||
|
||||
if ! command -v apm &>/dev/null; then
|
||||
echo "Error: apm is required but not installed (see kyberforge:apm-install)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v jq &>/dev/null; then
|
||||
echo "Error: jq is required but not installed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Convention subdirectories apm's plugin exporter can populate from .apm/.
|
||||
MIRROR_DIRS=(agents skills commands instructions extensions)
|
||||
|
||||
@@ -60,6 +88,52 @@ FAIL=0
|
||||
SCRATCH_ROOT="$(mktemp -d)"
|
||||
trap 'rm -rf "$SCRATCH_ROOT"' EXIT
|
||||
|
||||
if [[ "$ALL" -eq 1 ]]; then
|
||||
# Derives the plugin list from marketplace.json the same way
|
||||
# scripts/check-manifests.sh does, instead of hand-maintaining a duplicate list
|
||||
# at every call site (see .pre-commit-config.yaml's check-plugin-content-sync).
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
|
||||
MARKETPLACE="$REPO_ROOT/.claude-plugin/marketplace.json"
|
||||
if [[ ! -f "$MARKETPLACE" ]]; then
|
||||
echo "Error: --all requires $MARKETPLACE" >&2
|
||||
exit 1
|
||||
fi
|
||||
declare -a plugin_dirs=()
|
||||
plugin_count="$(jq '.plugins | length' "$MARKETPLACE")"
|
||||
for ((i = 0; i < plugin_count; i++)); do
|
||||
source_type="$(jq -r ".plugins[$i].source | type" "$MARKETPLACE")"
|
||||
# Remote sources (github, git, npm objects) have no local directory to sync.
|
||||
[[ "$source_type" == "string" ]] || continue
|
||||
source="$(jq -r ".plugins[$i].source" "$MARKETPLACE")"
|
||||
source="${source#./}"
|
||||
plugin_dirs+=("$REPO_ROOT/$source")
|
||||
done
|
||||
else
|
||||
declare -a plugin_dirs=("$@")
|
||||
fi
|
||||
|
||||
# Fail fast on a basename collision rather than letting two plugin_dir arguments
|
||||
# silently share (and corrupt) the same $name.log/$name.status/$name.checkcopy
|
||||
# scratch paths below.
|
||||
declare -a seen_names=()
|
||||
for plugin_dir in "${plugin_dirs[@]}"; do
|
||||
name="$(basename "${plugin_dir%/}")"
|
||||
for seen in ${seen_names[@]+"${seen_names[@]}"}; do
|
||||
if [[ "$seen" == "$name" ]]; then
|
||||
echo "Error: duplicate plugin basename '$name' among arguments -- scratch paths would collide" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
seen_names+=("$name")
|
||||
done
|
||||
|
||||
normalize_trailing_newline() {
|
||||
# apm's bundle exporter writes hooks.json without a trailing newline, which
|
||||
# end-of-file-fixer (pre-commit) would flag on every regeneration -- normalize
|
||||
# instead of fighting that hook on every sync.
|
||||
printf '%s\n' "$(cat "$1")" >"$2"
|
||||
}
|
||||
|
||||
sync_dir() {
|
||||
local plugin_dir="$1" bundle_dir="$2" d="$3"
|
||||
local src="$bundle_dir/$d" dst="$plugin_dir/$d"
|
||||
@@ -98,14 +172,10 @@ sync_hooks_json() {
|
||||
# No .apm/hooks/ content -- hooks.json (if any) is out of scope for this script.
|
||||
[[ -f "$src" ]] || return 0
|
||||
|
||||
# apm's bundle exporter writes hooks.json without a trailing newline, which
|
||||
# end-of-file-fixer (pre-commit) would flag on every regeneration -- normalize both
|
||||
# sides of the comparison (and the real write) to exactly one trailing newline
|
||||
# instead of fighting that hook on every sync.
|
||||
if [[ "$CHECK" -eq 1 ]]; then
|
||||
local normalized_src
|
||||
normalized_src="$(mktemp)"
|
||||
printf '%s\n' "$(cat "$src")" >"$normalized_src"
|
||||
normalize_trailing_newline "$src" "$normalized_src"
|
||||
if [[ ! -f "$dst" ]] || ! diff -q "$normalized_src" "$dst" >/dev/null 2>&1; then
|
||||
echo "DRIFT $dst: out of sync with .apm/hooks/" >&2
|
||||
FAIL=1
|
||||
@@ -114,7 +184,25 @@ sync_hooks_json() {
|
||||
return 0
|
||||
fi
|
||||
|
||||
printf '%s\n' "$(cat "$src")" >"$dst"
|
||||
normalize_trailing_newline "$src" "$dst"
|
||||
}
|
||||
|
||||
reinject_mcp_servers() {
|
||||
local plugin_dir="$1"
|
||||
local mcp_src="$plugin_dir/.mcp.json" dst="$plugin_dir/.github/plugin/plugin.json"
|
||||
[[ -f "$mcp_src" ]] || return 0
|
||||
[[ -f "$dst" ]] || return 0
|
||||
|
||||
# Match apm's own Claude-ecosystem plugin.json builder: mcpServers is omitted
|
||||
# entirely when the plugin declares none, not written out as an empty object.
|
||||
local count
|
||||
count="$(jq '(.mcpServers // {}) | length' "$mcp_src")"
|
||||
[[ "$count" -gt 0 ]] || return 0
|
||||
|
||||
local tmp
|
||||
tmp="$(mktemp)"
|
||||
jq --slurpfile mcp "$mcp_src" '.mcpServers = $mcp[0].mcpServers' "$dst" >"$tmp"
|
||||
mv "$tmp" "$dst"
|
||||
}
|
||||
|
||||
# Runs entirely inside a backgrounded subshell (see the dispatch loop below), so
|
||||
@@ -125,22 +213,36 @@ sync_one() {
|
||||
local apm_dir="$plugin_dir/.apm"
|
||||
FAIL=0
|
||||
|
||||
if [[ ! -d "$plugin_dir" ]]; then
|
||||
echo "FAIL $plugin_dir: plugin directory does not exist" >&2
|
||||
FAIL=1
|
||||
echo "$FAIL" >"$status_file"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [[ ! -d "$apm_dir" ]]; then
|
||||
echo "SKIP $plugin_dir: no .apm/ directory" >&2
|
||||
echo "$FAIL" >"$status_file"
|
||||
return 0
|
||||
fi
|
||||
|
||||
local name scratch bundle_dir pack_log
|
||||
local name scratch bundle_dir pack_log pack_cwd
|
||||
name="$(basename "$plugin_dir")"
|
||||
scratch="$SCRATCH_ROOT/$name"
|
||||
mkdir -p "$scratch"
|
||||
pack_log="$(mktemp)"
|
||||
|
||||
local force_flag=()
|
||||
[[ "$CHECK" -eq 0 ]] && force_flag=(--force)
|
||||
if [[ "$CHECK" -eq 0 ]]; then
|
||||
force_flag=(--force)
|
||||
pack_cwd="$plugin_dir"
|
||||
else
|
||||
pack_cwd="$SCRATCH_ROOT/$name.checkcopy"
|
||||
mkdir -p "$pack_cwd"
|
||||
cp -a "$plugin_dir/." "$pack_cwd/"
|
||||
fi
|
||||
|
||||
if ! (cd "$plugin_dir" && apm pack --format plugin "${force_flag[@]}" -o "$scratch") >"$pack_log" 2>&1; then
|
||||
if ! (cd "$pack_cwd" && apm pack --format plugin "${force_flag[@]}" -o "$scratch") >"$pack_log" 2>&1; then
|
||||
echo "FAIL $plugin_dir: apm pack failed:" >&2
|
||||
sed 's/^/ /' "$pack_log" >&2
|
||||
rm -f "$pack_log"
|
||||
@@ -163,6 +265,9 @@ sync_one() {
|
||||
sync_dir "$plugin_dir" "$bundle_dir" "$d"
|
||||
done
|
||||
sync_hooks_json "$plugin_dir" "$bundle_dir"
|
||||
if [[ "$CHECK" -eq 0 ]]; then
|
||||
reinject_mcp_servers "$plugin_dir"
|
||||
fi
|
||||
echo "$FAIL" >"$status_file"
|
||||
}
|
||||
|
||||
@@ -171,10 +276,22 @@ sync_one() {
|
||||
# than paying that startup cost N times serially. Output is buffered per plugin
|
||||
# (not streamed) so concurrent DRIFT/FAIL messages from different plugins never
|
||||
# interleave; it's flushed in stable $@ order once every job has finished.
|
||||
declare -a plugin_dirs=("$@")
|
||||
#
|
||||
# Batched (not a rolling pool) because a bounded rolling pool needs `wait -n`,
|
||||
# which is bash 4.3+ -- tests/run-tests.sh and tests/run-bats.sh in this same repo
|
||||
# are explicitly bash-3.2-safe, so this script matches their pattern for
|
||||
# consistency. `getconf` over `nproc` for the same reason: `nproc` doesn't exist
|
||||
# on macOS.
|
||||
JOBS_LIMIT="$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo 4)"
|
||||
running=0
|
||||
for plugin_dir in "${plugin_dirs[@]}"; do
|
||||
name="$(basename "${plugin_dir%/}")"
|
||||
(sync_one "$plugin_dir" "$SCRATCH_ROOT/$name.status") >"$SCRATCH_ROOT/$name.log" 2>&1 &
|
||||
running=$((running + 1))
|
||||
if [[ $running -ge $JOBS_LIMIT ]]; then
|
||||
wait
|
||||
running=0
|
||||
fi
|
||||
done
|
||||
wait
|
||||
|
||||
@@ -187,7 +304,11 @@ done
|
||||
|
||||
if [[ "$FAIL" -ne 0 ]]; then
|
||||
if [[ "$CHECK" -eq 1 ]]; then
|
||||
echo "Plugin content mirror is out of sync with .apm/. Fix: bash scripts/sync-plugin-content.sh $*" >&2
|
||||
if [[ "$ALL" -eq 1 ]]; then
|
||||
echo "Plugin content mirror is out of sync with .apm/. Fix: bash scripts/sync-plugin-content.sh --all" >&2
|
||||
else
|
||||
echo "Plugin content mirror is out of sync with .apm/. Fix: bash scripts/sync-plugin-content.sh $*" >&2
|
||||
fi
|
||||
fi
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user