fix: architecture review corrections — CONTEXT.md trim and eval backtrack verification
- Remove Fleet, Symbolic oversight, and Data classification tiers from CONTEXT.md glossary: Fleet is Phase 2 only; the other two duplicate always-on governance.md content - Drop now-stale governance test assertions for the removed terms - write-skill eval: add section-ordering assertion; fix duplicate prompt across two deterministic tests - write-eval eval: tighten plan-shown assertion to require per-case detail (query + type), not just "a list of test cases" Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -58,9 +58,9 @@ output_tests:
|
||||
name: "Deterministic — test plan presented before file is written"
|
||||
type: deterministic
|
||||
prompt: "write evals for the diagnose skill"
|
||||
expected_output: "Skill presents proposed test cases for review and requests confirmation before writing any file"
|
||||
expected_output: "Skill presents each proposed test case with its id, type, and query before writing any file, then requests confirmation"
|
||||
assertions:
|
||||
- "Response includes a proposed test plan or list of test cases before any file is written"
|
||||
- "Response presents each proposed test case individually — showing at minimum the query and test type — before any file is written"
|
||||
- "Response requests confirmation before proceeding to write"
|
||||
|
||||
- id: deterministic-merge-conflict-flagged
|
||||
|
||||
Reference in New Issue
Block a user