fix: architecture review corrections — CONTEXT.md trim and eval backtrack verification
- Remove Fleet, Symbolic oversight, and Data classification tiers from CONTEXT.md glossary: Fleet is Phase 2 only; the other two duplicate always-on governance.md content - Drop now-stale governance test assertions for the removed terms - write-skill eval: add section-ordering assertion; fix duplicate prompt across two deterministic tests - write-eval eval: tighten plan-shown assertion to require per-case detail (query + type), not just "a list of test cases" Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -33,10 +33,10 @@ trigger_tests:
|
||||
|
||||
output_tests:
|
||||
- id: output-has-all-sections
|
||||
name: All 8 body sections present
|
||||
name: All 8 body sections present in order
|
||||
type: deterministic
|
||||
prompt: "Write a new skill for linting markdown files, category: implement"
|
||||
expected_output: A complete SKILL.md containing all 8 required body sections in order.
|
||||
expected_output: A complete SKILL.md containing all 8 required body sections in the prescribed order.
|
||||
assertions:
|
||||
- "Output contains '## Role'"
|
||||
- "Output contains '## When to use / When not to use'"
|
||||
@@ -46,11 +46,12 @@ output_tests:
|
||||
- "Output contains '## Output format'"
|
||||
- "Output contains '## Failure handling'"
|
||||
- "Output contains '## Self-check'"
|
||||
- "Sections appear in this order: ## Role, ## When to use / When not to use, ## Required inputs, ## Constraints, ## Process, ## Output format, ## Failure handling, ## Self-check"
|
||||
|
||||
- id: output-path-correct
|
||||
name: Output path and frontmatter fields correct
|
||||
type: deterministic
|
||||
prompt: "Write a new skill for linting markdown files, category: implement"
|
||||
prompt: "Write a new skill for sending Slack notifications on deploy events, category: deploy"
|
||||
expected_output: A SKILL.md with correct output path stated and all required frontmatter fields present.
|
||||
assertions:
|
||||
- "Output contains '.agents/skills/' in the stated output path"
|
||||
|
||||
Reference in New Issue
Block a user