From b11cfd66b185a253fecf10ad06080ae165f32a74 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Sun, 19 Apr 2026 16:12:02 -0500 Subject: feat: adopt pairwise-tests (PICT combinatorial) + cross-reference from existing testing skills MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Forked verbatim from omkamal/pypict-claude-skill (MIT). LICENSE preserved. Renamed from `pict-test-designer` to `pairwise-tests` — technique-first naming so users invoking "pairwise" or "combinatorial" find it; PICT remains the tool under the hood. Bundle (skill-runtime only): pairwise-tests/SKILL.md (renamed, description rewritten) pairwise-tests/LICENSE (MIT, preserved) pairwise-tests/references/pict_syntax.md pairwise-tests/references/examples.md pairwise-tests/scripts/pict_helper.py (Python CLI for model gen / output fmt) pairwise-tests/scripts/README.md Upstream's repo-level docs (README, QUICKSTART, CONTRIBUTING, etc.) and `examples/` dir (ATM + gearbox walkthroughs — useful as reading, not as skill-runtime) omitted from the fork. Attribution footer added. Cross-references so /add-tests naturally routes to /pairwise-tests when warranted: - add-tests/SKILL.md Phase 2 step 8: if a function in scope has 3+ parameters each taking multiple values, surface `/pairwise-tests` to the user before proposing normal category coverage. Default continues with /add-tests; user picks pairwise explicitly. - claude-rules/testing.md: new "Combinatorial Coverage" section after the Normal/Boundary/Error categories. Explains when pairwise wins, when to skip (regulated / provably exhaustive contexts, ≤2 parameters, non- parametric testing), and points at /pairwise-tests. - languages/python/claude/rules/python-testing.md: new "Pairwise / Combinatorial for Parameter-Heavy Functions" subsection under the parametrize guidance. Explains the pytest workflow: /pairwise-tests generates the matrix, paste into pytest parametrize block, or use pypict helper directly. Mechanism note: cross-references are judgment-based — Claude reads the nudges in add-tests/testing/python-testing and acts on them when appropriate, not automatic dispatch. Craig can still invoke /pairwise-tests directly when he already knows he wants combinatorial coverage. Makefile SKILLS extended; make install symlinks /pairwise-tests globally. --- claude-rules/testing.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) (limited to 'claude-rules/testing.md') diff --git a/claude-rules/testing.md b/claude-rules/testing.md index f67ace2..b91b76c 100644 --- a/claude-rules/testing.md +++ b/claude-rules/testing.md @@ -58,6 +58,26 @@ Every unit under test requires coverage across three categories: - Resource exhaustion - Malformed data +## Combinatorial Coverage + +For functions with 3+ parameters that each take multiple values (feature-flag +combinations, config matrices, permission/role interactions, multi-field +form validation, API parameter spaces), the exhaustive test count explodes +(M^N) while 3-5 ad-hoc cases miss pair interactions. Use **pairwise / +combinatorial testing** — generate a minimal matrix that hits every 2-way +combination of parameter values. Empirically catches 60-90% of combinatorial +bugs with 80-99% fewer tests. + +Invoke `/pairwise-tests` on the offending function; continue using `/add-tests` +and the Normal/Boundary/Error discipline for the rest. The two approaches +complement: pairwise covers parameter *interactions*; category discipline +covers each parameter's individual edge space. + +Skip pairwise when: the function has 1-2 parameters (just write the cases), +the context requires *provably* exhaustive coverage (regulated systems — document +in an ADR), or the testing target is non-parametric (single happy path, +performance regression, a specific error). + ## Test Organization Typical layout: -- cgit v1.2.3