diff options
| author | Craig Jennings <c@cjennings.net> | 2026-07-19 19:51:51 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-07-19 19:51:51 -0500 |
| commit | 179c495fa7fcb2c2dcf7bdb81b1459afaa0e5075 (patch) | |
| tree | dc2fe80817af1f447575ce8d0092d9370c7a29e1 /claude-rules | |
| parent | a760d8e2e645000deb56b4a0a7d922147b2b1ad7 (diff) | |
| download | rulesets-179c495fa7fcb2c2dcf7bdb81b1459afaa0e5075.tar.gz rulesets-179c495fa7fcb2c2dcf7bdb81b1459afaa0e5075.zip | |
docs(testing): sharpen characterization-test guidance, wire it into the quality sweep
The rule already said "write a characterization test before refactoring" but not what one is or how many to write per unit. Two additions close that.
testing.md's "Adding Tests to Existing Untested Code" now defines a characterization test as recording what the code actually does, not what it should (Feathers' assert-wrong-read-the-real-value recipe), and requires the same Normal/Boundary/Error set as any unit rather than one happy-path capture. The reason is the useful part: on a characterization test the negative and boundary cases are the bug-finders, because untested legacy code is weakest at the empty input and the missing upstream, and pinning what it currently does there writes the wrong behavior down where it becomes a visible bug. A pinned case that turns out to be a bug graduates from recording current behavior to asserting correct behavior. In-unit bugs fall to this set. Composition bugs need a functional test.
The refactor-for-testability section gains the framing that when the untestable code is legacy you're hardening, extracting the pure core is the hardening, not a detour around it. A function you can't characterize without mocking tmux or git can't be refactored safely, so "needs too much mocking" is the signal to reshape it, never a license to skip the boundary and error cases.
code-quality.org's passes claim to preserve behavior, so it now states the precondition that makes that true: on untested scope, a characterization net comes first, or "behavior-preserving" is an assertion the green suite can't actually verify.
Diffstat (limited to 'claude-rules')
| -rw-r--r-- | claude-rules/testing.md | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/claude-rules/testing.md b/claude-rules/testing.md index b3fa5bf..81bd391 100644 --- a/claude-rules/testing.md +++ b/claude-rules/testing.md @@ -32,6 +32,31 @@ When working in a codebase without tests: 2. Use the characterization test as a safety net while refactoring 3. Then follow normal TDD for the new change +A characterization test asserts what the code *actually does* right now, not +what it *should* do. Write it by running the code against a fixed input, +reading the exact value or effect it currently produces, and asserting that +value — Feathers' recipe is to assert something you know is wrong, run it, and +paste the real value out of the failure. You don't need to know the correct +answer to write one; you record the observed one. That's what makes it +mechanical enough to bring a large untested surface under test without +re-deriving each unit's spec. + +**Characterize with the same Normal/Boundary/Error set as any unit** (the three +categories below), not one happy-path capture per function. On a characterization +test the negative and boundary cases are the ones that find bugs: untested legacy +code is weakest exactly at the empty input, the malformed value, the missing +upstream, and pinning what it *currently* does there writes the wrong behavior +down in black and white, where it becomes a bug you can see and decide on. When a +pinned case turns out to be a bug rather than behavior worth preserving, that one +test graduates from "record current" to "assert correct" and you fix the code. +The happy-path case is the regression net; the negative and boundary cases are +the audit. + +Bugs that live *inside* a unit are caught by this three-category set; bugs in how +units compose — ordering, shared state handed between them — are invisible to any +per-unit test and need a functional/integration test over the composed path (see +Integration Tests below and the pyramid). + ## Test Categories (Required for All Code) Every unit under test requires coverage across three categories: @@ -287,6 +312,18 @@ Fix: extract focused helpers (one responsibility each), test each in isolation with real inputs, compose them in a thin outer function. Several small unit tests plus one composition test beats one monster test behind a wall of mocks. +When the untestable function is legacy code you're hardening, this extraction +**is** the hardening — not a detour around it. A function whose boundary or +error case can't be exercised without mocking the world (a shell function that +calls `tmux`/`git` directly, a handler that reaches straight into I/O) can't be +characterized, so you can't refactor it safely and you can't pin its edge +behavior. Extracting the pure decision logic into a helper that takes plain +inputs and returns a plain result makes that logic characterizable with the full +Normal/Boundary/Error set; the I/O calls become a thin wrapper you cover once +with a single composition test. "It needs too much mocking to test" is therefore +never a reason to skip the boundary and error cases — it's the signal to reshape +the function so those cases are writable. + ## Coverage Targets - Business logic and domain services: **90%+** |
