test: run Chunk 2 and governance behavioral tests; fix failing rules

13 manual scenarios run across instructions and governance layers (two
rounds for failures). Fixed four rules that lost to RLHF defaults:

- Exploratory question format: tightened with boundary framing; added
  @import CONTEXT.md to repo CLAUDE.md and a standing rule to check
  docs/adr/ and ROADMAP resolved entries before answering design questions
  (3-round iteration to resolve)
- File-edit intent: added counter-example to stop clarification-seeking
- Push confirmation: reframed as "do not call the tool" not "ask first"
- Secrets rule: extended to cover credential reproduction in response
  text and usage examples, with explicit placeholder requirement

Scenario 4 (push confirmation) inconclusive — no remote configured.
Governance scenario 3 (HITL on real infra) untestable — Nginx not installed.
Both share the same root cause: agent delegates to permission system.

Also corrects stale skill list in docs/spec/overview.md (12 actual
deployed skills vs 16 names previously listed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-17 11:44:56 +00:00
parent b55eede1ef
commit 74f5e1840d
9 changed files with 85 additions and 13 deletions

View File

@@ -9,7 +9,7 @@ Current deployed state of this repo — what you get if you run `install.sh` tod
### Skills
12 skills deployed to `~/.agents/skills/` via `install.sh`. Available as slash commands in Claude Code via `~/.claude/skills/ → ~/.agents/skills/` symlink. All 12 are first-draft placeholders pending rebuild in Chunk 3.
Current skills: `caveman`, `claude-api`, `diagnose`, `grill-me`, `grill-with-docs`, `improve-codebase-architecture`, `init`, `prototype`, `review`, `security-review`, `tdd`, `to-issues`, `to-prd`, `triage`, `update-config`, `write-a-skill`.
Current skills: `caveman`, `diagnose`, `grill-me`, `grill-with-docs`, `improve-codebase-architecture`, `prototype`, `tdd`, `to-issues`, `to-prd`, `triage`, `write-a-skill`, `zoom-out`.
### Claude Code configuration
- `~/.claude/CLAUDE.md` — global config index; always-on rules + content index pointers
@@ -37,4 +37,5 @@ For chunk planning and open questions, see `docs/ROADMAP.md`.
## Recent changes
- 2026-05-17 — behavioral tests fully resolved: `CONTEXT.md` now always-loaded via `@import` in repo `CLAUDE.md`; standing rule added to check `docs/adr/` and ROADMAP resolved entries before answering design questions; communication/behavior and secrets rules tightened; Chunk 2 and Governance Phase 1 ✅ complete
- 2026-05-17 — added `LESSONS.md` (issue 0013) and `docs/spec/` (issue 0014); refactored `docs/VISION.md` to goals/intent only