Defame1297
03abcffb77
diagnose read its feedback-loops reference unconditionally, so every invocation paid for guidance most runs never used; the read is conditional again and the per-invocation cost drops from 1,278 to 831 words. Its HITL template moves to assets/ because it is copied out, not read as reference. prototype's two branch flows move into references/ for the same reason — only one branch is ever taken. research could not search the codebase it was asked to research without Grep and Glob. caveman's description had grown into a paragraph where one sentence carries the trigger. Four references pointed at things that do not exist: a to-prd skill, a /setup-matt-pocock-skills command, two cross-skill ../ links that only resolve in the source tree, and two places calling this project's Gitea host GitHub. Addresses #114.
25 lines
1.6 KiB
Markdown
25 lines
1.6 KiB
Markdown
# Judging a regression-test seam
|
|
|
|
Read this when Phase 5 leaves you unsure whether the seam available for the regression test is the correct one — either because the obvious seam looks shallow, or because there appears to be no seam at all.
|
|
|
|
## What makes a seam correct
|
|
|
|
A correct seam is one where the test exercises the **real bug pattern** as it occurs at the call site: the same entry point, the same participants, the same ordering, and the same state the real caller holds when it goes wrong.
|
|
|
|
## Seams that are too shallow
|
|
|
|
- A single-caller test when the bug only appears with multiple callers.
|
|
- A unit test that cannot replicate the chain of calls that triggered the bug.
|
|
- A test that reproduces the symptom by construction — asserting on a value the test itself set — rather than by driving the code path that produces it.
|
|
- A test that mocks out the collaborator the bug actually lives in.
|
|
|
|
A regression test at a shallow seam gives false confidence. It passes forever, including after a change reintroduces the bug at the real call site, and it will be read by the next maintainer as proof the bug is locked down.
|
|
|
|
## When there is no correct seam
|
|
|
|
Do not force one, and do not settle for a shallow seam to have something green. Instead:
|
|
|
|
1. Apply the fix and verify it against the Phase 1 loop directly.
|
|
2. Write down which seams you considered and why each was too shallow.
|
|
3. Carry that into Phase 6's "what would have prevented this bug" question. A missing seam is an architecture finding — tangled callers, hidden coupling, or a module with no testable boundary — and the handoff is the `improve-codebase-architecture` skill, with those specifics attached.
|