aboutsummaryrefslogtreecommitdiff
path: root/claude-rules/testing.md
Commit message (Collapse)AuthorAgeFilesLines
* docs(rules): add escalation testing and a spike protocol to testing.mdCraig Jennings2026-05-221-1/+61
| | | | Two additions. An "Escalation Beyond Category and Pairwise" section adds property-based testing (for invariants across a broad input domain) and mutation testing (for when high coverage hides thin assertions), both as escalation paths rather than always-on gates. And the "I need to spike first" excuse is formalized into a disciplined spike protocol: TDD stays the default, but a spike is sanctioned only when timeboxed, not committed, and followed by the first failing test before productionizing.
* docs(testing): rewrite time-mocking helper rule to be language-agnosticCraig Jennings2026-04-251-7/+15
| | | | | | | | The original phrasing leaned on Lisp terms (let-bind, defvar) that don't translate to most other languages. Generalize to two named failure modes (infinite recursion against the mocked primitive, scope-shadowing that production callers can't see) with examples across Python, Lisp, Go, and JavaScript so the rule applies regardless of stack.
* docs(commits,testing): add merge strategy + multi-pass gate + time-mocking ↵Craig Jennings2026-04-251-0/+7
| | | | | | | | | | | | | | | | | safety commits.md gets two additions. A new "## Merge Strategy" section makes squash-merge the default for feature branches and requires explicit confirmation of the merge approach before pushing or merging. A new "Multi-pass gate" paragraph in Review and Publish requires every humanizer-flow pass to run and be named when declaring done, so silent skips become defects. testing.md gets a new bullet under "### Determinism": time/clock-mocking helpers must not recurse against the primitive they're mocking, and must not let-bind over a defvar — the binding shadows only inside the test scope, so production code reading the symbol still gets the original value (silent test miss).
* docs: split Linear vs PR structure; propagate content-scope rule to Tier 1 ↵Craig Jennings2026-04-241-0/+4
| | | | | | | | | | | | | | | skills Linear ticket bodies are now Problem + Fix only. PR descriptions keep the four-section format (Problem, Fix, Why this fixes it, How it was tested). Linear's GitHub integration handles the cross-link via the PR body's Linear: line. Cross-ref to the content-scope rule appended at the end of each Tier 1 skill that produces public artifacts: testing.md, arch-document, arch-decide, arch-design, review-code, respond-to-review, brainstorm, codify. Single-source the rule in commits.md, point at it from each output-producing skill.
* feat: adopt pairwise-tests (PICT combinatorial) + cross-reference from ↵Craig Jennings2026-04-191-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | existing testing skills Forked verbatim from omkamal/pypict-claude-skill (MIT). LICENSE preserved. Renamed from `pict-test-designer` to `pairwise-tests` — technique-first naming so users invoking "pairwise" or "combinatorial" find it; PICT remains the tool under the hood. Bundle (skill-runtime only): pairwise-tests/SKILL.md (renamed, description rewritten) pairwise-tests/LICENSE (MIT, preserved) pairwise-tests/references/pict_syntax.md pairwise-tests/references/examples.md pairwise-tests/scripts/pict_helper.py (Python CLI for model gen / output fmt) pairwise-tests/scripts/README.md Upstream's repo-level docs (README, QUICKSTART, CONTRIBUTING, etc.) and `examples/` dir (ATM + gearbox walkthroughs — useful as reading, not as skill-runtime) omitted from the fork. Attribution footer added. Cross-references so /add-tests naturally routes to /pairwise-tests when warranted: - add-tests/SKILL.md Phase 2 step 8: if a function in scope has 3+ parameters each taking multiple values, surface `/pairwise-tests` to the user before proposing normal category coverage. Default continues with /add-tests; user picks pairwise explicitly. - claude-rules/testing.md: new "Combinatorial Coverage" section after the Normal/Boundary/Error categories. Explains when pairwise wins, when to skip (regulated / provably exhaustive contexts, ≤2 parameters, non- parametric testing), and points at /pairwise-tests. - languages/python/claude/rules/python-testing.md: new "Pairwise / Combinatorial for Parameter-Heavy Functions" subsection under the parametrize guidance. Explains the pytest workflow: /pairwise-tests generates the matrix, paste into pytest parametrize block, or use pypict helper directly. Mechanism note: cross-references are judgment-based — Claude reads the nudges in add-tests/testing/python-testing and acts on them when appropriate, not automatic dispatch. Craig can still invoke /pairwise-tests directly when he already knows he wants combinatorial coverage. Makefile SKILLS extended; make install symlinks /pairwise-tests globally.
* feat(rules): port key testing principles from quality-engineer promptCraig Jennings2026-04-191-0/+104
| | | | | | | | | | | | | | | | | | | | | | | Additions to claude-rules/testing.md: - Testing pyramid proportions (70-80% unit / 15-25% integration / 5-10% e2e) - Integration Tests section: docstring must name 'Components integrated:' and mark real vs mocked; when-to-write heuristics - Signs of Overmocking: 'would the test pass if the function body was NotImplementedError?' plus three more sharp questions - Testing Code That Uses Frameworks: test your integration, not the framework itself - Test Real Code, Not Copies: never inline prod code into tests - Error Behavior, Not Error Text: test type + key values, not exact prose - If Tests Are Hard to Write, Refactor the Code: hard-to-test is a code signal, not a test signal; extract focused helpers - Anti-patterns list extended Addition to languages/elisp/claude/rules/elisp-testing.md: - Interactive vs Internal split pattern: cj/foo wraps cj/--foo; test the internal directly, skip UI mocks Source: ~/.emacs.d/ai-prompts/quality-engineer.org (personal reference, kept as an extended prompt separate from these rules).
* refactor: generalize testing.md, split Python specifics, DRY installCraig Jennings2026-04-191-4/+17
| | | | | | | | | | | | | | | | | | | claude-rules/testing.md is now language-agnostic (TDD principles, test categories, coverage targets, anti-patterns). Scope header widened to **/*. Python-specific content (pytest, fixtures, parametrize, anyio, Django DB testing) moved to languages/python/claude/rules/python-testing.md. Added languages/python/ bundle (rules only so far; no CLAUDE.md template or hooks yet — Python validation tooling differs from Elisp). Added install-python shortcut to the Makefile. Updated scripts/install-lang.sh to copy claude-rules/*.md into each target project's .claude/rules/. Bundles no longer need to carry their own verification.md copy — deleted languages/elisp/claude/rules/verification.md. Single source of truth in claude-rules/, fans out via install. Elisp-testing.md now references testing.md as its base (matches the python-testing.md pattern).
* Add general-purpose skills and rules from DeepSat coding-rulesetsCraig Jennings2026-03-291-0/+140
Skills (adapted from DeepSat, stripped of project-specific references): - /review-pr: PR review against engineering standards - /fix-issue: issue-to-merge TDD workflow - /security-check: secrets, OWASP, and dependency audit - /debug: systematic 4-phase debugging - /add-tests: test coverage analysis and generation - /respond-to-review: evaluate and implement code review feedback Rules (general-purpose, copied as-is): - testing.md: universal TDD standards and anti-patterns - verification.md: proof over assumption Makefile updated to install both skills and rules via symlinks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>