From 6e77c11474809de9524c3ee95808b1c8f5a24287 Mon Sep 17 00:00:00 2001 From: Defame1297 Date: Thu, 13 Aug 2026 17:50:32 +0000 Subject: [PATCH] ci(kyberforge): add apm-native marketplace/audit/pack drift gates Validated the plugin-content-mirror fix (issue #90) against apm's own packing/CI documentation and source: no apm-native mechanism replaces the mirror script (apm's bundler treats .apm/ and root convention dirs as mutually exclusive, by design), but the investigation surfaced a real, separate gap -- this repo ran zero apm-native audit/check commands in CI, relying entirely on custom scripts and Claude Code's own client-side validator. Add three pre-push hooks matching apm's documented producer CI pattern: - apm marketplace check: validates every marketplace.packages[] entry resolves, including live network reachability for remote refs -- a blind spot check-manifests.sh explicitly skips (local sources only). - apm audit --ci: apm's own lockfile/policy/hidden-content integrity gate. - apm pack --check-versions --check-clean: closes issue #90's deferred item 3 (a check-clean-equivalent gate) using apm's native flag instead of bespoke drift logic, verifying .claude-plugin/marketplace.json still matches what apm.yml + .apm/ would currently generate. All three are network-tolerant and whole-repo in scope, so they belong at pre-push alongside check-manifests/check-plugin-content-sync/ validate-plugins -- not pre-commit, which stays fast/offline/per-file. Documented the packing/bundling/releasing/CI findings in docs/research/docs/microsoft-apm/releasing.md (new) and extended testing-and-validation.md with the apm-action wrapper and its documented CI patterns, sourced from Context7 and cross-checked against the installed apm-cli 0.28.0 package directly. Refs: #90 --- .pre-commit-config.yaml | 27 ++++ .../research/docs/microsoft-apm/releasing.md | 133 ++++++++++++++++++ .../research/docs/microsoft-apm/sources.md | 6 +- .../microsoft-apm/testing-and-validation.md | 70 +++++++++ 4 files changed, 234 insertions(+), 2 deletions(-) create mode 100644 plugins/kyberforge/docs/research/docs/microsoft-apm/releasing.md diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index fd0f263..2aff4bb 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -70,6 +70,33 @@ repos: pass_filenames: false always_run: true + - id: apm-marketplace-check + name: apm marketplace check + description: Validate every marketplace.packages[] entry resolves, including network reachability of remote refs -- catches stale/unreachable remote package references that check-manifests.sh deliberately skips (local-source checks only) + entry: apm marketplace check + language: system + stages: [pre-push] + pass_filenames: false + always_run: true + + - id: apm-audit-ci + name: apm audit --ci + description: apm's own producer-side lockfile/policy/hidden-content integrity gate, per apm's documented recommended CI block (see docs/research/docs/microsoft-apm/testing-and-validation.md) + entry: apm audit --ci + language: system + stages: [pre-push] + pass_filenames: false + always_run: true + + - id: apm-pack-check-clean + name: apm pack --check-clean + description: Release gate -- verify .claude-plugin/marketplace.json still matches what apm.yml + .apm/ would currently generate, and that per-package versions agree with the per_package versioning strategy. Closes issue #90's deferred item 3 (a check-clean-equivalent gate) using apm's own flag instead of custom drift logic. + entry: apm pack --check-versions --check-clean --dry-run + language: system + stages: [pre-push] + pass_filenames: false + always_run: true + - id: check-vale-style-sync name: Check Vale style copies are in sync description: Diff skill-audit's Vale copy against agent-audit's canonical copy diff --git a/plugins/kyberforge/docs/research/docs/microsoft-apm/releasing.md b/plugins/kyberforge/docs/research/docs/microsoft-apm/releasing.md new file mode 100644 index 0000000..5be658c --- /dev/null +++ b/plugins/kyberforge/docs/research/docs/microsoft-apm/releasing.md @@ -0,0 +1,133 @@ +--- +topic: releasing +source_keys: + - context7-microsoft-apm + - apm-github-repo +--- + +## `apm pack` full reference + +```bash +apm pack # plugin format (default), directory under ./build/ +apm pack --archive # plugin bundle as .zip (default archive format) +apm pack --archive --archive-format tar.gz # legacy CI pipelines that expect .tar.gz +apm pack --format apm -o ./dist # legacy APM bundle layout, custom output path +apm pack --dry-run -v # resolve and print per-entry detail; write nothing +apm pack --offline # marketplace resolution: cached refs only +apm pack --include-prerelease # marketplace resolution: allow pre-release tags +apm pack --marketplace=claude --json # JSON output for CI pipelines, one format only +apm pack --marketplace-path claude=dist/marketplace.json # override one format's output path +apm pack --legacy-skill-paths # bundle skills under per-client paths, not shared .agents/skills/ +``` + +`apm pack` reads both `apm.yml` and `apm.lock.yaml`. What it produces depends on which blocks `apm.yml` declares: a `dependencies:` block alone produces a bundle; a `marketplace:` block alone produces marketplace artifacts (`.claude-plugin/marketplace.json` etc.); both present produces both. A package with neither block (skills/agents authored directly, no dependency resolution, not marketplace-listed) has nothing for `apm pack` to do beyond the plugin.json/bundle synthesis. + +### `--format plugin` bundle structure + +```text +build/-/ +├── plugin.json # synthesized, schema-conformant per json.schemastore.org/claude-code-plugin.json +├── apm.lock.yaml # enriched copy embedding a per-file bundle_files manifest (integrity check for `apm install `) +├── agents/ +├── skills/ +├── commands/ +└── hooks.json # merged from .apm/hooks/*.json, single file (not a hooks/ directory) +``` + +This is the default (`--format plugin`, no flags) and the one most CI/release workflows use. `--format apm` produces the older, distinct "legacy APM bundle layout" instead — different consumers, not a superset/subset of each other. + +### Exit codes + +| Code | Meaning | +|---|---| +| 0 | Success | +| 1 | Build or runtime error (e.g. network failure, missing reference) | +| 2 | Manifest schema validation error | +| 3 | `--check-versions`: per-package versions don't align with the configured `marketplace.versioning.strategy` | +| 4 | `--check-clean`: working tree is dirty — on-disk marketplace output doesn't match what a fresh pack would produce | + +These four are deliberately structured as **release gates that block, not auto-fix** — none of them rewrite files to make the check pass; a nonzero exit means stop the release and investigate, not "run again to self-heal." + +### `--check-clean` and `--check-versions` scope + +Both are scoped to **marketplace outputs** specifically (whatever `marketplace.claude`/`marketplace.codex` in `apm.yml` configures — i.e. `.claude-plugin/marketplace.json` and equivalents), not to the bundle's plugin content: + +- `--check-clean` regenerates every configured marketplace output to a temp representation (including any `--marketplace-path` override) and diffs it against what's actually on disk at the effective path. Any difference is exit 4. +- `--check-versions` verifies that every marketplace-listed package's version agrees with the configured `marketplace.versioning.strategy` (`lockstep` | `tag_pattern` | `per_package` — see `monorepo-and-repo-shapes.md`). + +Neither one inspects a plugin's `agents/`/`skills/`/`hooks.json` directories at all — there is no apm-native "does the packed bundle match what's committed at the plugin root" check. That gap has to be closed by project-specific tooling if a repo commits compiled bundle content back into its plugin directories (see this repo's `scripts/sync-plugin-content.sh` and its `--check` mode, and `docs/adr/0017-*.md` for why that gap exists and isn't an apm oversight to wait out). + +## The canonical release sequence + +Documented as: `apm pack` (with release gates) → checksum the artifacts → create the GitHub release. A manual version, for repos that need custom packaging steps beyond what `apm-action` covers: + +```yaml +- run: pip install apm-cli +- run: | + apm pack --check-versions --check-clean --json > pack-report.json + for f in build/*.zip .claude-plugin/marketplace.json; do + [ -f "$f" ] || continue + sha256sum "$f" > "${f}.sha256" + done + gh release create "${GITHUB_REF_NAME}" \ + build/*.zip build/*.zip.sha256 \ + .claude-plugin/marketplace.json* \ + --title "${GITHUB_REF_NAME}" --notes-file CHANGELOG.md + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} +``` + +`--json` on the combined `pack --check-versions --check-clean` invocation gives one machine-readable summary of both gates for the release job to inspect/log, rather than parsing prose output. + +The equivalent using the official wrapper, for repos that don't need custom packaging: + +```yaml +on: + push: + tags: ["v*"] +jobs: + release: + permissions: { contents: write } + steps: + - uses: actions/checkout@v5 + - uses: microsoft/apm-action@v1 + with: { mode: release } + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} +``` + +`mode: release` is a third distinct mode on the same action used for install/audit (see `testing-and-validation.md`) — it runs pack + the release gates + GitHub release creation internally, so a repo only needs this when it doesn't need the manual workflow's custom steps (checksums, custom release notes handling, non-GitHub-release publishing target). + +## `apm publish` — registry publish is a different artifact than `apm pack`'s bundle + +```bash +apm publish --package acme/my-skill --dry-run -v # always dry-run first -- not trivially reversible +apm publish --package acme/my-skill +apm publish --package acme/my-skill --registry corp-main # pick a registry when multiple are configured +apm publish --package acme/my-skill --zip ./build/my-package-1.0.0.zip # publish a pre-built zip, skip auto-pack +``` + +By default `apm publish` auto-packs a **flat registry archive** — `apm.yml`, the `.apm/` directory, and standard documentation files — and uploads that. This is explicitly a different layout than the plugin bundle `apm pack --format plugin` produces (no synthesized `plugin.json`, no `agents/`/`skills/` convention-directory flattening, symlinks excluded). Registry consumers install via `apm install `, which resolves and deploys from this registry archive shape, not the plugin bundle shape — the two pack paths exist for two different consumers (a plugin host vs. an apm-aware installer) and aren't interchangeable. + +Constraints: `apm.yml` must have `name` and `version` set. Publishing the same version twice returns a 409 Conflict — versions are immutable once published; fix by bumping the version, not by re-publishing over it. + +### Registries + +A project declares registries in its manifest: + +```yaml +registries: + internal: + url: https://artifactory.example.com/artifactory/api/skills/internal + aliases: + - mirror.example.com + default: internal +``` + +Registries are the "package-level hosting at scale" tier (see `marketplace-and-registries.md`'s "Which mechanism to use" section) — heavier infrastructure than a git-based marketplace, worth it once package count or access-control needs outgrow git-based discovery. + +## Gotchas + +- `apm pack`'s bundle (`--format plugin`, written under `./build/` by default) is a **distribution artifact for `apm install ` consumers** — it is not read by Claude Code's own installer, which clones/scans a plugin's git working directory directly via `marketplace.json`'s `source:` path (see `compile.md`'s note on this same point). Nothing in `apm pack`, `--check-clean`, or `apm audit --ci` reconciles the bundle against a plugin's committed working-tree content — that reconciliation, if a repo needs Claude Code to discover `.apm/`-authored content without waiting on the bundle, is necessarily project-specific tooling, not something to look for as a missing/misused apm flag. +- `--check-clean`'s "regenerate and diff" model only applies to marketplace-format outputs. A repo relying on committed plugin-bundle content (flat `agents/`/`skills/`/`hooks.json` at a plugin root, generated from `.apm/`) needs its own drift check for that content — `--check-clean` will not catch it going stale. +- `apm publish` and `apm pack` are easy to conflate since both start from the same `apm.yml`/`.apm/` source, but they produce structurally different artifacts for different consumers (registry install vs. plugin host install) — verifying one does not verify the other. diff --git a/plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md b/plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md index b1cfd01..f5ae175 100644 --- a/plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md +++ b/plugins/kyberforge/docs/research/docs/microsoft-apm/sources.md @@ -4,12 +4,14 @@ - **URL:** context7:/microsoft/apm - **Description:** Microsoft APM (Agent Package Manager) — open-source dependency manager for AI agent configuration (skills, prompts, instructions, agents, hooks, MCP/LSP deps), applying a declare/lock/install/audit workflow. -- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, examples.md, troubleshooting.md, testing-and-validation.md, marketplace-and-registries.md, monorepo-and-repo-shapes.md, agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md +- **Contributing files:** overview.md, installation.md, configuration.md, cli-reference.md, examples.md, troubleshooting.md, testing-and-validation.md, marketplace-and-registries.md, monorepo-and-repo-shapes.md, agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md, releasing.md - **Status:** `extracted` ## apm-github-repo - **URL:** https://github.com/microsoft/apm - **Description:** APM's own Python source (`src/apm_cli/`) read directly for schema/compile-mapping ground truth where Context7's doc snippets were thin — specifically `primitives/models.py` and `primitives/discovery.py` (primitive dataclasses and discovery globs), `integration/prompt_integrator.py`, `integration/command_integrator.py`, `integration/instruction_integrator.py`, `integration/hook_integrator.py`, `integration/hook_native_formats.py`, `integration/hook_ir.py`, `integration/_hook_dropped_targets.py`, `integration/targets.py` (`KNOWN_TARGETS`/`PrimitiveMapping` per-target deploy config), `compilation/claude_formatter.py` and `compilation/distributed_compiler.py` (instruction fold-in to CLAUDE.md/AGENTS.md), and `models/validation.py`. -- **Contributing files:** agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md +- **Contributing files:** agent-primitive-schema.md, prompt-primitive-schema.md, instructions-primitive-schema.md, hooks-primitive-schema.md, releasing.md - **Status:** `extracted` + +Note: `releasing.md`'s `--check-clean`/`--check-versions` scope, `apm pack` exit-code semantics, and the `.apm/`-vs-root-flat-dir mutual exclusivity referenced there were additionally cross-checked directly against `apm_cli/bundle/plugin_exporter.py`, `apm_cli/commands/pack.py`, and `apm_cli/marketplace/drift_check.py` in the installed `apm-cli` 0.28.0 package (`/root/.local/pipx/venvs/apm-cli/`), not just Context7 doc snippets — confirmed by a live `apm pack --format plugin` run inside `plugins/bin` that reproduced the documented `[!] Skipping root-level skills/ because .apm/ is present` warning. diff --git a/plugins/kyberforge/docs/research/docs/microsoft-apm/testing-and-validation.md b/plugins/kyberforge/docs/research/docs/microsoft-apm/testing-and-validation.md index 390ff86..ed97bca 100644 --- a/plugins/kyberforge/docs/research/docs/microsoft-apm/testing-and-validation.md +++ b/plugins/kyberforge/docs/research/docs/microsoft-apm/testing-and-validation.md @@ -49,3 +49,73 @@ jobs: ## Frozen installs `apm install --frozen` (seen in the private-registry install flow) fails instead of silently re-resolving if the lockfile is out of date — the CI equivalent of `npm ci` vs `npm install`. Worth using in any CI job that shouldn't be allowed to drift the lockfile. + +The documented "recommended CI block" pattern chains these two: `apm install --frozen` first (fail if the lockfile can't reproduce what's declared), then `apm audit --ci` second (fail if the fetched/deployed content doesn't match what's locked). Each step catches a different failure mode — a stale lockfile vs. tampered/drifted deployed files — and both are meant to run on every CI invocation, not just release branches. + +## `microsoft/apm-action` (official GitHub Action) + +A first-party GitHub Action wraps CLI install/compile/audit so CI workflows don't hand-roll `pip install apm-cli` + version pinning: + +```yaml +- uses: microsoft/apm-action@v1 + with: + compile: true # run `apm compile` after install (needed for Codex/Gemini/other + # targets whose instructions require compilation; omit for Claude-only) + audit-report: true # run `apm audit --ci` and surface the report + env: + GITHUB_APM_PAT: ${{ secrets.APM_PAT }} # needed for private-repo dependency resolution +``` + +Bare install (no extra inputs) just runs the equivalent of `apm install`. `mode: release` (see `releasing.md`) is a third, distinct mode from install/audit — the same action handles all three, selected via `mode:`/`compile:`/`audit-report:` inputs rather than three separate actions. + +## Minimal PR gate (`enforce-in-ci` pattern) + +The documented minimal gate scopes itself to only the paths that can actually introduce an APM violation, so it doesn't run on unrelated PRs: + +```yaml +on: + pull_request: + paths: + - 'apm.yml' + - 'apm.lock.yaml' + - '.apm/**' + - '.github/**' + - '.claude/**' + - '.cursor/**' +jobs: + audit: + steps: + - uses: actions/checkout@v4 + - uses: microsoft/apm-action@v1 + - run: apm audit --ci --no-cache +``` + +`--no-cache` forces a fresh policy/dependency resolution rather than trusting a cached prior run — worth using on a gating job specifically (where a stale cache silently passing is worse than a slower job), even though the default cached path is fine for local/dev use. + +## Scheduled drift audit + +Separately from PR-time gating, a nightly scheduled job catches drift that accumulates between PRs (e.g. an upstream dependency ref moving) rather than only checking at commit time: + +```yaml +on: + schedule: [{ cron: '0 6 * * *' }] + workflow_dispatch: +jobs: + audit: + steps: + - uses: actions/checkout@v4 + - run: curl -fsSL https://aka.ms/apm/install | bash + - run: apm install + - run: apm audit --ci --format sarif --output apm-audit.sarif + continue-on-error: true + - uses: github/codeql-action/upload-sarif@v3 + with: { sarif_file: apm-audit.sarif } + - run: apm outdated > apm-outdated.txt + if: always() +``` + +`continue-on-error: true` on the audit step plus uploading SARIF regardless is deliberate for a *scheduled* job: it reports findings as code-scanning annotations instead of hard-failing a job nobody is blocked on merging behind. `apm outdated` (run unconditionally via `if: always()`) is a separate, non-gating command — lists dependencies with newer versions available, informational only. + +## What `apm audit --ci` does NOT cover + +None of `apm audit`, `apm marketplace check`, or the `apm-action` wrapper validate that a plugin's content is actually discoverable by a host's install-time convention scan (e.g. Claude Code's flat `agents/`/`skills/` directory scan) — that is a platform-level concern `apm audit` has no model of; it audits lockfile/policy/hidden-content integrity, not host-side discoverability. See `releasing.md` for why `apm pack`'s bundle output is a distribution artifact, not something the audit/install pipeline reconciles against a plugin's working-tree layout.