feat(core): add AGENTS.md authoring/review tooling #80

Merged
Defame1297 merged 9 commits from feat/79-agentsmd-tooling into main 2026-07-23 17:46:09 +00:00
13 changed files with 816 additions and 0 deletions
Showing only changes of commit 6c8ea8e8f0 - Show all commits

View File

@@ -60,5 +60,11 @@ The three-stage traceability record linking a skill back to its research inputs:
### Bidirectional reference principle ### Bidirectional reference principle
Files that reference other files should declare those references explicitly. The referencing file carries the forward reference (e.g. content index in `CLAUDE.md`, `references:` in frontmatter). The referenced file carries a `when:` field describing when it is loaded. Both sides should agree — divergence signals staleness. The reverse map ("what files reference this file?") is derived by a reference scanner script, not maintained manually. This principle applies to instruction files, skills, and workflow documents. Files that reference other files should declare those references explicitly. The referencing file carries the forward reference (e.g. content index in `CLAUDE.md`, `references:` in frontmatter). The referenced file carries a `when:` field describing when it is loaded. Both sides should agree — divergence signals staleness. The reverse map ("what files reference this file?") is derived by a reference scanner script, not maintained manually. This principle applies to instruction files, skills, and workflow documents.
### agentsmd-author / agentsmd-audit
A skill pair in the `core` plugin for writing, updating, and reviewing a target repo's `AGENTS.md` file(s) (the generic open-standard file — see the `AGENTS.md` entry above — not this repo's own). `agentsmd-author` creates/updates AGENTS.md content, supports nested monorepo placement (per the standard's nearest-file-wins precedence), and closes out by invoking `agentsmd-audit` inline. `agentsmd-audit` runs a single combined pass checking three mandatory baselines: secrets/credentials (governance.md hard prohibition — AGENTS.md is committed content), structural completeness (common-sections checklist from the agents.md spec), and accuracy/drift (do referenced commands and paths actually resolve against the repo). `agentsmd-audit` never inspects provider adapter files (see `provider-adapter-author`) — its scope is AGENTS.md content only. Chosen over folding this into `kyberforge` because kyberforge's scope is meta-tooling for the holocron marketplace itself, not generic target-repo documentation; `core` is the intended home for cross-cutting, repo-agnostic utility skills.
### provider-adapter-author
A companion skill (`core` plugin) that detects a target repo's provider-specific instruction file (`CLAUDE.md`, `.cursor/rules/*.mdc`, `copilot-instructions.md`, etc.) and, where it duplicates content AGENTS.md should own, converts it into a thin adapter that imports AGENTS.md — mirroring this repo's own ADR-0012 pattern. Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`: checks for an import reference, no duplicated headings, size threshold) rather than a separate paired audit skill — the check is mechanical, so a script suffices per governance.md's "prefer deterministic code for repeatable tasks." `agentsmd-author` calls this skill via skill composition when it detects an existing provider file with overlapping content.
### LESSONS.md ### LESSONS.md
Long-loop feedback log for patterns observed across sessions. Three or more entries on the same pattern graduate to the relevant standing file (e.g. a coding convention, a governance rule). Updated by the session-handoff skill or directly by the human. Lives at the repo root. Long-loop feedback log for patterns observed across sessions. Three or more entries on the same pattern graduate to the relevant standing file (e.g. a coding convention, a governance rule). Updated by the session-handoff skill or directly by the human. Lives at the repo root.

View File

@@ -0,0 +1,16 @@
# AGENTS.md tooling lives in `core`, split into three skills
`kyberforge` is scoped to meta-tooling for building and maintaining the holocron marketplace itself (skills, agents, plugins, marketplace entries) — not to generic capabilities for an arbitrary target repo. Authoring and reviewing a target repo's `AGENTS.md` file is repo-agnostic documentation tooling, closer in kind to `bin:write-docs` or `bin:init` than to `skill-author`/`plugin-author`. Research for this topic was initially placed under `plugins/kyberforge/docs/research/docs/agentsmd/` but has moved to `plugins/core/docs/research/docs/agentsmd/` to keep the provenance chain consistent with the plugin the resulting skills live in.
## Decision
Three skills in the `core` plugin (`core`'s first active skills):
- **`agentsmd-author`** — creates/updates a target repo's `AGENTS.md`, including nested monorepo placement (nearest-file-wins). Closes out by invoking `agentsmd-audit` inline, mirroring the `skill-author`/`skill-audit` pattern. When it detects an existing provider-specific file (`CLAUDE.md`, etc.) with content that duplicates what AGENTS.md should own, it calls `provider-adapter-author` via skill composition.
- **`agentsmd-audit`** — a single combined pass checking three mandatory baselines against `AGENTS.md` only: secrets/credentials (governance.md hard prohibition), structural completeness (common-sections checklist from the agents.md spec), and accuracy/drift (do referenced commands/paths resolve against the repo). Never inspects provider adapter files.
- **`provider-adapter-author`** — detects and converts a provider-specific instruction file into a thin adapter that imports `AGENTS.md` (mirroring this repo's own two-tier `CLAUDE.md` pattern). Self-validates via its own bundled deterministic script (`scripts/validate-adapter.sh`) rather than a separate paired audit skill, since the check (import present, no duplicated headings, size threshold) is mechanical.
## Consequences
- `core`'s plugin.json/README will list real skills for the first time.
- `plugins/kyberforge/docs/research/docs/agentsmd/` moves to `plugins/core/docs/research/docs/agentsmd/` before authoring begins.

View File

@@ -0,0 +1,28 @@
# agentsmd-audit
Audit a target repo's AGENTS.md file(s) for embedded secrets, structural completeness, and drift.
## What it does
Runs a single combined pass across every AGENTS.md file in a repo (root and any nested monorepo files): flags embedded secrets/credentials, checks structure against the agents.md common-sections checklist, and resolves referenced commands/paths against the actual repo to catch stale documentation. Outputs a compact findings report — findings only, grouped by dimension, each with Why and Fix. Never inspects provider-specific adapter files (CLAUDE.md, etc.) and never writes or fixes anything.
## Usage
```
/agentsmd-audit
```
Provide the path to the repo root to audit when invoking.
## Files
| File | Purpose |
|------|---------|
| `SKILL.md` | Skill instructions for agents |
| `scripts/validate-secrets.sh` | Scans AGENTS.md files for embedded secrets, API keys, tokens, connection strings |
| `scripts/validate-structure.sh` | Checks for empty/placeholder content, common-sections checklist, nested-vs-root duplication |
| `scripts/validate-drift.sh` | Resolves referenced npm/make commands and file paths against the repo |
| `references/sources.md` | Provenance record — sources that informed this skill and which files each contributed to |
| `tests/validate-secrets.bats` | Bats test suite for `scripts/validate-secrets.sh` |
| `tests/validate-structure.bats` | Bats test suite for `scripts/validate-structure.sh` |
| `tests/validate-drift.bats` | Bats test suite for `scripts/validate-drift.sh` |

View File

@@ -0,0 +1,68 @@
---
name: agentsmd-audit
description: >
Use when the user wants to review a repo's AGENTS.md file, says "audit this
AGENTS.md", "check my AGENTS.md", "is this AGENTS.md any good", or wants to
know if AGENTS.md is safe to commit — even if they don't use the word
"audit". Also invoke proactively after agentsmd-author creates or updates
AGENTS.md, or after a hand-edit made outside agentsmd-author. Audits a
target repo's AGENTS.md file(s) — root and any nested monorepo files — for
embedded secrets/credentials, structural completeness against the
agents.md common-sections checklist, and drift (referenced commands or
paths that no longer resolve against the repo). Produces a compact
findings report (findings only, no PASS noise) with Why and Fix per
finding. Do not use to audit CLAUDE.md, .cursor/rules, or other
provider-specific adapter files — that's provider-adapter-author's
self-contained concern. Do not use to fix or write AGENTS.md content — use
agentsmd-author instead.
allowed-tools: Bash Read
metadata:
category: docs
source_keys:
- agents-md-official
- context7-websites-agents-md
- context7-agentsmd-agents-md
- governance-secrets-hard-prohibition
version: "0.1.0"
---
## Gotchas
- Always run all three checks — this skill does a single combined pass, not staged/gated passes. Don't skip structure or drift checks just because a secrets FAIL was found.
- Never inspect or mention provider-specific adapter files (`CLAUDE.md`, `.cursor/rules/*.mdc`, `copilot-instructions.md`, etc.) — that's out of scope. If one exists and duplicates AGENTS.md content, that's `provider-adapter-author`'s concern, not this skill's.
- A missing common section (e.g. no "Security" heading) is informational, not a failure — not every repo needs every section from the checklist. Only flag a FAIL when the file is empty, entirely unfilled placeholder text, or contains a real embedded secret/stale reference.
- Gather findings internally; don't narrate PASS/FAIL per check as you go — surface them only in the final report.
## Step 1 — Run the validators
```bash
bash scripts/validate-secrets.sh <repo-root>
bash scripts/validate-structure.sh <repo-root>
bash scripts/validate-drift.sh <repo-root>
```
Each script walks the repo for every `AGENTS.md` file (root and nested, excluding `.git`, `node_modules`, `vendor`, and similar) and prints `FAIL`/`INFO`/`SUGGESTION` lines with `Why`/`Fix` (or `Note`) per finding. A nonzero exit means at least one FAIL was found in that dimension. If a script cannot execute (`python3` unavailable, Bash denied), fall back to manual review: scan for real-looking credentials, check common sections are present, and spot-check a few referenced commands/paths by hand.
## Step 2 — Report
Open with a coverage line:
```text
Checked: secrets · structure · drift
```
Then output only findings that were found, in this order within a repo: `### Secrets`, `### Structure`, `### Drift`. Omit a dimension heading entirely if it produced nothing — its absence confirms it passed. Report each finding verbatim as emitted by the scripts (they already carry file:line, Why/Fix or Note).
Close with a result block:
```text
## Result
PASS
PASS · P info
PASS (N suggestions) · P info
FAIL (N fails)
FAIL (N fails) · P info
```
INFO and SUGGESTION findings are observational — they never flip PASS to FAIL. Do not fix anything — this skill reports and proposes only. Point the user to `agentsmd-author` to apply fixes.

View File

@@ -0,0 +1,33 @@
# Sources
## agents-md-official
- **URL:** https://agents.md/
- **Description:** Official agents.md website — format spec, common-sections checklist, precedence rules (nearest-file-wins, no merge across files), monorepo nesting patterns
- **Research doc:** plugins/core/docs/research/docs/agentsmd/sources.md
- **Contributing files:** SKILL.md
- **Status:** `extracted`
## context7-websites-agents-md
- **URL:** context7:/websites/agents_md
- **Description:** Context7 index of the official agents.md website — overview, governance, cross-tool compatibility, configuration examples
- **Research doc:** plugins/core/docs/research/docs/agentsmd/sources.md
- **Contributing files:** SKILL.md
- **Status:** `extracted`
## context7-agentsmd-agents-md
- **URL:** context7:/agentsmd/agents.md
- **Description:** Context7 index of the agentsmd/agents.md repository — format spec, nested monorepo patterns, file structure examples
- **Research doc:** plugins/core/docs/research/docs/agentsmd/sources.md
- **Contributing files:** SKILL.md
- **Status:** `extracted`
## governance-secrets-hard-prohibition
- **URL:** (org convention — not a plugin research corpus entry)
- **Description:** Hard prohibition on placing secrets, API keys, tokens, or credentials in code, config, prompts, or any output. Grounds the secrets/credentials check in `scripts/validate-secrets.sh` and Step 1 of SKILL.md — AGENTS.md is committed content, so an embedded real secret is a hard-prohibition violation, not a style nit.
- **Research doc:** core/instructions/governance.md (org convention file, not a plugin research corpus entry; content is inlined here since plugins must be self-contained and this file may not exist wherever the plugin is installed)
- **Contributing files:** SKILL.md
- **Status:** `extracted`

View File

@@ -0,0 +1,11 @@
# scripts/
Deterministic validators this skill shells out to instead of relying on LLM judgment for mechanical checks.
| File | Purpose |
|------|---------|
| `validate-secrets.sh` | Scans every AGENTS.md file (root + nested) for embedded secrets, API keys, tokens, and connection strings |
| `validate-structure.sh` | Checks for empty/placeholder content, the common-sections checklist, and nested-vs-root duplication |
| `validate-drift.sh` | Resolves referenced npm/make commands and file paths against the actual repo state |
All three take a single `<repo-root>` argument, print `FAIL`/`INFO`/`SUGGESTION` findings to stdout, and exit non-zero only on FAIL.

View File

@@ -0,0 +1,137 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate-drift.sh <repo-root>
Check every AGENTS.md file in a repo (root and nested) for drift: package
manager scripts and file paths referenced in the text that no longer exist
in the repo. Catches the failure mode that matters most in practice — an
agent running a documented command that was renamed or deleted.
Arguments:
repo-root Path to the repository root to scan.
Exit codes:
0 No FAIL findings (INFO may still be printed, e.g. no package.json found)
1 One or more FAIL findings
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: repo-root is required." >&2
echo "" >&2
usage >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
import json
repo_root = os.path.abspath(sys.argv[1])
if not os.path.isdir(repo_root):
print(f"Error: '{repo_root}' is not a directory.", file=sys.stderr)
sys.exit(1)
EXCLUDE_DIRS = {".git", "node_modules", "vendor", ".venv", "venv", "dist", "build"}
def find_agents_md(root):
results = []
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if d not in EXCLUDE_DIRS and not d.startswith(".")]
for fname in filenames:
if fname == "AGENTS.md":
results.append(os.path.join(dirpath, fname))
return sorted(results)
def load_package_scripts(root):
pkg_path = os.path.join(root, "package.json")
if not os.path.isfile(pkg_path):
return None
try:
with open(pkg_path, encoding="utf-8") as f:
data = json.load(f)
except (json.JSONDecodeError, OSError):
return None
return set(data.get("scripts", {}).keys())
def load_make_targets(root):
make_path = os.path.join(root, "Makefile")
if not os.path.isfile(make_path):
return None
with open(make_path, encoding="utf-8", errors="replace") as f:
content = f.read()
return set(re.findall(r'(?m)^([a-zA-Z0-9_-]+)\s*:(?!=)', content))
NPM_RUN_RE = re.compile(r'\b(?:npm|pnpm|yarn)\s+run\s+([a-zA-Z0-9:_-]+)')
MAKE_RE = re.compile(r'\bmake\s+([a-zA-Z0-9_-]+)')
# Backticked relative file paths, e.g. `scripts/bootstrap.sh`, `src/index.ts`.
# Requires a path separator and file extension to avoid matching bare commands/words.
PATH_RE = re.compile(r'`([A-Za-z0-9_.\-]+(?:/[A-Za-z0-9_.\-]+)+\.[A-Za-z0-9]+)`')
has_fail = False
package_scripts = load_package_scripts(repo_root)
make_targets = load_make_targets(repo_root)
for fpath in find_agents_md(repo_root):
rel = os.path.relpath(fpath, repo_root)
with open(fpath, encoding="utf-8", errors="replace") as f:
content = f.read()
for m in NPM_RUN_RE.finditer(content):
script_name = m.group(1)
if package_scripts is None:
print(f"INFO Cannot verify referenced script '{script_name}' — {rel}")
print(f" Note: AGENTS.md references an npm/pnpm/yarn script, but no package.json was found at the repo root to check it against.")
print()
elif script_name not in package_scripts:
has_fail = True
print(f"FAIL Referenced script '{script_name}' not found in package.json — {rel}")
print(f" Why: AGENTS.md tells agents to run '{script_name}', but package.json has no matching \"scripts\" entry — the command will fail.")
print(f" Fix: Update AGENTS.md to reference an existing script, or add '{script_name}' to package.json's scripts.")
print()
for m in MAKE_RE.finditer(content):
target_name = m.group(1)
if make_targets is None:
print(f"INFO Cannot verify referenced make target '{target_name}' — {rel}")
print(f" Note: AGENTS.md references a make target, but no Makefile was found at the repo root to check it against.")
print()
elif target_name not in make_targets:
has_fail = True
print(f"FAIL Referenced make target '{target_name}' not found in Makefile — {rel}")
print(f" Why: AGENTS.md tells agents to run 'make {target_name}', but the Makefile has no matching target — the command will fail.")
print(f" Fix: Update AGENTS.md to reference an existing target, or add '{target_name}' to the Makefile.")
print()
file_dir = os.path.dirname(fpath)
for m in PATH_RE.finditer(content):
candidate = m.group(1)
resolved = (
os.path.isfile(os.path.join(repo_root, candidate))
or os.path.isfile(os.path.join(file_dir, candidate))
or os.path.isdir(os.path.join(repo_root, candidate))
or os.path.isdir(os.path.join(file_dir, candidate))
)
if not resolved:
has_fail = True
print(f"FAIL Referenced path '{candidate}' does not exist — {rel}")
print(f" Why: AGENTS.md points agents to '{candidate}', but it isn't present in the repo (checked relative to repo root and to the AGENTS.md's own directory).")
print(f" Fix: Update AGENTS.md to reference the correct path, or restore/create '{candidate}'.")
print()
if has_fail:
sys.exit(1)
sys.exit(0)
PYTHON

View File

@@ -0,0 +1,120 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate-secrets.sh <repo-root>
Scan every AGENTS.md file in a repo (root and nested) for embedded secrets,
API keys, tokens, or connection strings. AGENTS.md is committed content —
real credentials in it are a hard-prohibition violation, not a style nit.
Placeholders (<your-key>, \$ENV_VAR, YOUR_TOKEN_HERE, example.com, etc.) are
not flagged.
Arguments:
repo-root Path to the repository root to scan.
Exit codes:
0 No findings
1 One or more FAIL findings
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: repo-root is required." >&2
echo "" >&2
usage >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
repo_root = os.path.abspath(sys.argv[1])
if not os.path.isdir(repo_root):
print(f"Error: '{repo_root}' is not a directory.", file=sys.stderr)
sys.exit(1)
EXCLUDE_DIRS = {".git", "node_modules", "vendor", ".venv", "venv", "dist", "build"}
def find_agents_md(root):
results = []
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if d not in EXCLUDE_DIRS and not d.startswith(".")]
for fname in filenames:
if fname == "AGENTS.md":
results.append(os.path.join(dirpath, fname))
return sorted(results)
PLACEHOLDER_RE = re.compile(
r'(?i)(your[_-]|my[_-]|example|xxx+|placeholder|changeme|<[^>]+>|\$\{|\$[A-Z_][A-Z0-9_]*|\.\.\.|redacted)'
)
PATTERNS = [
("AWS access key ID", re.compile(r'AKIA[0-9A-Z]{16}')),
("Private key block", re.compile(r'-----BEGIN [A-Z ]*PRIVATE KEY-----')),
("GitHub token", re.compile(r'gh[pousr]_[A-Za-z0-9]{36,}')),
("Slack token", re.compile(r'xox[baprs]-[A-Za-z0-9-]{10,}')),
("GitLab token", re.compile(r'glpat-[A-Za-z0-9_-]{20,}')),
("Generic API-style secret token", re.compile(r'\bsk-[A-Za-z0-9]{20,}\b')),
(
"Credential-bearing connection string",
re.compile(r'[a-zA-Z][a-zA-Z0-9+.-]*://[^:@/\s]+:[^@/\s]+@[^\s\'"]+'),
),
(
"Assigned secret/password/token literal",
re.compile(
r'(?i)\b(api[_-]?key|secret|token|password|passwd|pwd|access[_-]?key)\b'
r'\s*[:=]\s*[\'"]?([A-Za-z0-9+/_.\-]{12,})[\'"]?'
),
),
]
findings = []
def emit_fail(desc, fpath, lineno, why, fix):
findings.append((desc, fpath, lineno, why, fix))
for fpath in find_agents_md(repo_root):
rel = os.path.relpath(fpath, repo_root)
with open(fpath, encoding="utf-8", errors="replace") as f:
lines = f.readlines()
for i, line in enumerate(lines, start=1):
if PLACEHOLDER_RE.search(line):
continue
for label, pattern in PATTERNS:
m = pattern.search(line)
if not m:
continue
# Re-check placeholder allowlist against just the matched value, in case
# the placeholder marker sits outside the regex's own match span.
value = m.group(0)
if PLACEHOLDER_RE.search(value):
continue
emit_fail(
f"Possible {label}",
f"{rel}:{i}",
i,
"AGENTS.md is committed content; this line matches a real-looking credential pattern rather than a placeholder.",
"Remove the embedded credential and replace it with an environment variable reference or placeholder (e.g. $API_KEY, <your-token>).",
)
break
if not findings:
sys.exit(0)
for desc, fpath, _lineno, why, fix in findings:
print(f"FAIL {desc} — {fpath}")
print(f" Why: {why}")
print(f" Fix: {fix}")
print()
sys.exit(1)
PYTHON

View File

@@ -0,0 +1,118 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<EOF
Usage: validate-structure.sh <repo-root>
Check every AGENTS.md file in a repo (root and nested) for structural
completeness against the agents.md spec's common-sections checklist
(setup/build, code style, testing, security, commit/PR conventions).
Missing individual sections are informational (not every repo needs every
section) — only an empty or entirely unfilled file is a hard failure.
Arguments:
repo-root Path to the repository root to scan.
Exit codes:
0 No FAIL findings (INFO/SUGGESTION may still be printed)
1 One or more FAIL findings
EOF
}
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
fi
if [[ $# -lt 1 ]]; then
echo "Error: repo-root is required." >&2
echo "" >&2
usage >&2
exit 1
fi
python3 -u - "$1" <<'PYTHON'
import sys
import os
import re
PLACEHOLDER_RE = re.compile(r'(?i)FILL IN:|TODO:\s*write|lorem ipsum')
COMMON_SECTIONS = [
("setup/build commands", re.compile(r'(?im)^#{1,3}\s*(setup|install|build|getting started)')),
("code style", re.compile(r'(?im)^#{1,3}\s*(code style|style guide|conventions)')),
("testing instructions", re.compile(r'(?im)^#{1,3}\s*(test|testing)')),
("security considerations", re.compile(r'(?im)^#{1,3}\s*security')),
("commit/PR conventions", re.compile(r'(?im)^#{1,3}\s*(commit|pr|pull request)')),
]
repo_root = os.path.abspath(sys.argv[1])
if not os.path.isdir(repo_root):
print(f"Error: '{repo_root}' is not a directory.", file=sys.stderr)
sys.exit(1)
EXCLUDE_DIRS = {".git", "node_modules", "vendor", ".venv", "venv", "dist", "build"}
def find_agents_md(root):
results = []
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if d not in EXCLUDE_DIRS and not d.startswith(".")]
for fname in filenames:
if fname == "AGENTS.md":
results.append(os.path.join(dirpath, fname))
return sorted(results)
has_fail = False
file_contents = {} # rel path -> content, for the duplication pass below
for fpath in find_agents_md(repo_root):
rel = os.path.relpath(fpath, repo_root)
with open(fpath, encoding="utf-8", errors="replace") as f:
content = f.read()
file_contents[rel] = content
if not content.strip():
has_fail = True
print(f"FAIL AGENTS.md is empty — {rel}")
print(" Why: An empty file provides no instructions and gives agents nothing to act on.")
print(" Fix: Add at least a project overview and setup/test commands, per the agents.md common-sections checklist.")
print()
continue
if PLACEHOLDER_RE.search(content):
has_fail = True
print(f"FAIL Unfilled placeholder content — {rel}")
print(" Why: A 'FILL IN:' or template stub left in place means the file has no repo-specific instructions yet.")
print(" Fix: Replace the placeholder with real, repo-specific content.")
print()
continue
for label, pattern in COMMON_SECTIONS:
if not pattern.search(content):
print(f"INFO No {label} section — {rel}")
print(f" Note: The agents.md common-sections checklist includes {label}; not every repo needs every section, but confirm this omission is deliberate.")
print()
# --- Nested-vs-root duplication check ---
root_content = file_contents.get("AGENTS.md")
if root_content:
root_lines = {ln.strip() for ln in root_content.splitlines() if ln.strip()}
for rel, content in file_contents.items():
if rel == "AGENTS.md":
continue
nested_lines = [ln.strip() for ln in content.splitlines() if ln.strip()]
if not nested_lines:
continue
overlap = sum(1 for ln in nested_lines if ln in root_lines)
ratio = overlap / len(nested_lines)
if ratio >= 0.7:
print(f"SUGGESTION Nested AGENTS.md largely duplicates the root file — {rel}")
print(f" Why: {ratio:.0%} of this file's content lines already appear in the root AGENTS.md; per the spec's nearest-file-wins precedence, nested files don't inherit from the root, but they also shouldn't just restate it.")
print(f" Fix: Trim {rel} down to only what's specific to this package/directory.")
print()
if has_fail:
sys.exit(1)
sys.exit(0)
PYTHON

View File

@@ -0,0 +1,30 @@
# tests/
Test files for scripts bundled with this skill.
## Dependencies
Tests require [bats-support](https://github.com/bats-core/bats-support) and
[bats-assert](https://github.com/bats-core/bats-assert). The test files load
helpers from the repo root's `tests/test_helper/`.
From the repo root:
```bash
git clone https://github.com/bats-core/bats-support tests/test_helper/bats-support
git clone https://github.com/bats-core/bats-assert tests/test_helper/bats-assert
```
Run all tests for this skill (from the repo root):
```bash
bats plugins/core/skills/agentsmd-audit/tests/
```
## Files
| File | Purpose |
|------|---------|
| `validate-secrets.bats` | Bats test suite for `scripts/validate-secrets.sh` |
| `validate-structure.bats` | Bats test suite for `scripts/validate-structure.sh` |
| `validate-drift.bats` | Bats test suite for `scripts/validate-drift.sh` |

View File

@@ -0,0 +1,105 @@
#!/usr/bin/env bats
setup() {
REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../../../../" && pwd)"
load "$REPO_ROOT/tests/test_helper/bats-support/load"
load "$REPO_ROOT/tests/test_helper/bats-assert/load"
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate-drift.sh"
TMPDIR="$(mktemp -d)"
}
teardown() {
rm -rf "$TMPDIR"
}
@test "fails when AGENTS.md references a stale npm script" {
cat > "$TMPDIR/package.json" <<'EOF'
{
"scripts": {
"test": "jest"
}
}
EOF
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Testing
- Run `pnpm run e2e` before committing.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "e2e"
}
@test "passes when the referenced npm script exists" {
cat > "$TMPDIR/package.json" <<'EOF'
{
"scripts": {
"test": "jest"
}
}
EOF
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Testing
- Run `npm run test` before committing.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
}
@test "fails when AGENTS.md references a stale make target" {
cat > "$TMPDIR/Makefile" <<'EOF'
build:
echo building
EOF
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
Run `make deploy` to ship.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "deploy"
}
@test "emits INFO instead of FAIL when there is no package.json to verify an npm script against" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Testing
Run `pnpm run e2e` before committing.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
assert_output --partial "INFO"
assert_output --partial "e2e"
}
@test "fails when a referenced file path does not exist" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
See `scripts/bootstrap.sh` for environment setup.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "scripts/bootstrap.sh"
}
@test "passes when the referenced file path exists" {
mkdir -p "$TMPDIR/scripts"
: > "$TMPDIR/scripts/bootstrap.sh"
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
See `scripts/bootstrap.sh` for environment setup.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
}

View File

@@ -0,0 +1,68 @@
#!/usr/bin/env bats
setup() {
REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../../../../" && pwd)"
load "$REPO_ROOT/tests/test_helper/bats-support/load"
load "$REPO_ROOT/tests/test_helper/bats-assert/load"
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate-secrets.sh"
TMPDIR="$(mktemp -d)"
}
teardown() {
rm -rf "$TMPDIR"
}
@test "passes on AGENTS.md with no secrets, only placeholders" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
- Set `export API_KEY=$API_KEY`
- Token: <your-token-here>
- DB: postgres://user:changeme@localhost/db
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
assert_output ""
}
@test "fails on a real-looking AWS access key" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
- AWS_ACCESS_KEY_ID=AKIAABCDEFGHIJKLMNOP # gitleaks:allow (synthetic fixture — this test verifies validate-secrets.sh catches exactly this pattern)
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "AWS access key ID"
assert_output --partial "AGENTS.md:4"
}
@test "fails on a credential-bearing connection string" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
- DB: postgres://svc_user:h8x2Klm9pQrT@db.internal:5432/prod
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "connection string"
}
@test "detects secrets in a nested AGENTS.md, not just root" {
mkdir -p "$TMPDIR/packages/api"
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
Clean root file.
EOF
cat > "$TMPDIR/packages/api/AGENTS.md" <<'EOF'
# API package
- token: ghp_1234567890abcdefghijklmnopqrstuvwxyz01 # gitleaks:allow (synthetic fixture)
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "packages/api/AGENTS.md"
}

View File

@@ -0,0 +1,76 @@
#!/usr/bin/env bats
setup() {
REPO_ROOT="$(cd "$BATS_TEST_DIRNAME/../../../../../" && pwd)"
load "$REPO_ROOT/tests/test_helper/bats-support/load"
load "$REPO_ROOT/tests/test_helper/bats-assert/load"
SCRIPT="$(cd "$BATS_TEST_DIRNAME/../scripts" && pwd)/validate-structure.sh"
TMPDIR="$(mktemp -d)"
}
teardown() {
rm -rf "$TMPDIR"
}
@test "fails on an empty AGENTS.md" {
: > "$TMPDIR/AGENTS.md"
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "empty"
}
@test "fails on an unfilled placeholder AGENTS.md" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup
FILL IN: describe setup commands here.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_failure
assert_output --partial "placeholder"
}
@test "passes with INFO on real content missing an optional section" {
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup commands
- Install deps: `pnpm install`
- Run tests: `pnpm test`
## Code style
- TypeScript strict mode, single quotes, no semicolons.
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
assert_output --partial "INFO"
assert_output --partial "security"
}
@test "suggests trimming a nested AGENTS.md that duplicates the root file" {
mkdir -p "$TMPDIR/packages/api"
cat > "$TMPDIR/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup commands
- Install deps: `pnpm install`
- Run tests: `pnpm test`
- Lint: `pnpm lint`
- Build: `pnpm build`
EOF
cat > "$TMPDIR/packages/api/AGENTS.md" <<'EOF'
# AGENTS.md
## Setup commands
- Install deps: `pnpm install`
- Run tests: `pnpm test`
- Lint: `pnpm lint`
- Build: `pnpm build`
EOF
run bash "$SCRIPT" "$TMPDIR"
assert_success
assert_output --partial "SUGGESTION"
assert_output --partial "packages/api/AGENTS.md"
}