fix: architecture review corrections — CONTEXT.md trim and eval backtrack verification

- Remove Fleet, Symbolic oversight, and Data classification tiers from
  CONTEXT.md glossary: Fleet is Phase 2 only; the other two duplicate
  always-on governance.md content
- Drop now-stale governance test assertions for the removed terms
- write-skill eval: add section-ordering assertion; fix duplicate prompt
  across two deterministic tests
- write-eval eval: tighten plan-shown assertion to require per-case
  detail (query + type), not just "a list of test cases"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-17 19:30:39 +00:00
parent 3dcd0f08fb
commit 2631142a0e
4 changed files with 6 additions and 31 deletions

View File

@@ -58,9 +58,9 @@ output_tests:
name: "Deterministic — test plan presented before file is written"
type: deterministic
prompt: "write evals for the diagnose skill"
expected_output: "Skill presents proposed test cases for review and requests confirmation before writing any file"
expected_output: "Skill presents each proposed test case with its id, type, and query before writing any file, then requests confirmation"
assertions:
- "Response includes a proposed test plan or list of test cases before any file is written"
- "Response presents each proposed test case individually — showing at minimum the query and test type — before any file is written"
- "Response requests confirmation before proceeding to write"
- id: deterministic-merge-conflict-flagged