summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2025-10-29 02:55:35 -0500
committerCraig Jennings <c@cjennings.net>2025-10-29 02:55:35 -0500
commit7a6c8ef59316fe547da745c29a310ff00b09a907 (patch)
tree7a6f91be28ed7a56d11121b918acc9d0e4b3c13e
parent75291f93e08b65aabc9b49003ca471554360f426 (diff)
docs: add test future-proofing and large-scale refactoring lessons
Document key lessons from chime.el timestamp refactoring project: ## New Sections Added **Test Future-Proofing & Time-Based Testing** - Dynamic timestamp generation patterns and benefits - Never hardcode dates in tests - use relative time helpers - Mock time via function substitution (with-test-time pattern) - Code examples showing before/after patterns **Large-Scale Test Refactoring Strategy** - Strategic planning: tackle biggest challenges first - Execution approach: maintain 100% pass rate throughout - Project management: track progress visibly, celebrate milestones - Know when you're done: not all files need changes **Real-World Example** - chime.el project: 23 files, 339 tests - 16 files refactored (251 tests), 7 files skipped (88 tests) - 100% pass rate maintained across all refactoring - Result: future-proof test suite that never expires ## Key Insights - "Tackle biggest challenge first" eliminates intimidation - Work in batches but commit individually for clean history - Don't let perfectionism create unnecessary work - Strategic approach builds momentum and confidence Added "Hardcoded dates in tests" to Red Flags section. These lessons capture the methodology that successfully completed the hardest refactoring task in the project. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
-rw-r--r--ai-prompts/quality-engineer.org114
1 files changed, 114 insertions, 0 deletions
diff --git a/ai-prompts/quality-engineer.org b/ai-prompts/quality-engineer.org
index d6bb7ecb..385dbec1 100644
--- a/ai-prompts/quality-engineer.org
+++ b/ai-prompts/quality-engineer.org
@@ -638,6 +638,119 @@ Don't duplicate coverage:
- Format test cases as clear, numbered lists within each category
- Focus on practical, implementable tests that catch real-world bugs
+## Test Future-Proofing & Time-Based Testing
+
+*** Dynamic Timestamp Generation
+When tests involve dates and times:
+- **Never hardcode dates** in tests (e.g., "2025-10-24")
+ - Tests will fail when that date passes
+ - Creates maintenance burden (annual test updates)
+ - Makes tests brittle and non-deterministic over time
+
+- **Use dynamic timestamp generation** instead:
+ - Create test utilities that generate timestamps relative to "now"
+ - Examples: `test-time-today-at`, `test-time-tomorrow-at`, `test-time-days-from-now`
+ - Base time can be offset into future to avoid past-date edge cases
+
+- **Mock time via function substitution**:
+ - Replace `current-time` with test-controlled time source
+ - Use macros like `with-test-time` to scope time mocking
+ - Preserves existing function mocking patterns (don't nest cl-letf*)
+
+- **Pattern for time-based tests**:
+ #+begin_src elisp
+ (let* ((now (test-time-today-at 14 0))
+ (event-time (test-time-today-at 14 10))
+ (timestamp-str (test-timestamp-string event-time)))
+ (with-test-time now
+ (cl-letf (((symbol-function 'other-mock) (lambda () ...)))
+ (let* ((result (function-under-test event-time)))
+ (should (equal expected result))))))
+ #+end_src
+
+- **Benefits of dynamic timestamps**:
+ - Tests work indefinitely without modification
+ - Time relationships remain consistent (10 minutes = 10 minutes)
+ - Clearer test intent (relative times vs absolute dates)
+ - Easier to understand what's being tested
+ - No test failures from date expiration
+
+*** Large-Scale Test Refactoring Strategy
+When refactoring many test files (e.g., removing hardcoded timestamps):
+
+**** Strategic Planning
+- **Tackle biggest challenges first** - Largest/most complex files
+ - Eliminates intimidation factor early
+ - Makes remaining work feel manageable
+ - Builds confidence and momentum
+ - Subsequent files feel easier by comparison
+
+- **Establish consistent patterns early**:
+ - First few files set the template
+ - Document patterns in commit messages
+ - Later files follow established conventions
+ - Reduces decision fatigue
+
+- **Identify files that don't need refactoring**:
+ - Not all timestamp references require changes
+ - Copyright dates, version numbers: leave alone
+ - Tests without time dependencies: skip
+ - Focus effort where it matters
+
+**** Execution Approach
+- **Maintain 100% test pass rate throughout**:
+ - Run tests after each file refactored
+ - Never commit broken tests
+ - Validates refactoring correctness immediately
+ - Builds trust in the process
+
+- **Work in batches but commit individually**:
+ - Refactor similar test patterns together
+ - But commit each file separately with detailed message
+ - Makes git history navigable
+ - Easier to revert specific changes if needed
+
+- **Use validation infrastructure**:
+ - Pre-commit hooks catch syntax errors
+ - Makefile targets (`make validate`, `make test-file`)
+ - Automated checks prevent regressions
+
+- **Document patterns in commits**:
+ - Before/after examples in commit messages
+ - List of changes made
+ - Pattern transformations explained
+ - Future maintainers understand the approach
+
+**** Project Management Lessons
+- **Track progress visibly**:
+ - Use todo lists to show what's done vs remaining
+ - Update counts: "16/23 files (70%) complete"
+ - Provides motivation and clarity
+
+- **Celebrate milestones**:
+ - Acknowledge when biggest challenges complete
+ - Note when majority threshold reached (>50%)
+ - Recognize systematic progress
+
+- **Know when you're done**:
+ - Some files genuinely don't need changes
+ - Don't force work where it doesn't apply
+ - Project can complete before all files touched
+
+**** Real-World Example: chime.el Timestamp Refactoring
+Project scope: 23 test files, 339 tests total
+- 16 files needed timestamp refactoring (251 tests)
+- 7 files had no timestamp dependencies (88 tests)
+- Completed over multiple sessions maintaining 100% pass rate
+- Strategic approach: Always chose largest remaining file
+- Result: Future-proof test suite, tests never expire
+
+Key insight: **Not all refactoring projects require touching every file**
+- Analyze which files actually need work
+- Skip files without relevant issues
+- Focus effort on high-impact changes
+- Don't let perfectionism create unnecessary work
+
## Red Flags
Watch for and report these issues:
@@ -652,3 +765,4 @@ Watch for and report these issues:
- Flaky tests that pass/fail intermittently
- Tests that are too slow
- Tests that require manual setup or verification
+- **Hardcoded dates in tests** - Will fail when dates pass, creates maintenance burden