aboutsummaryrefslogtreecommitdiff
path: root/pairwise-tests/references
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-04-19 16:12:02 -0500
committerCraig Jennings <c@cjennings.net>2026-04-19 16:12:02 -0500
commitb11cfd66b185a253fecf10ad06080ae165f32a74 (patch)
tree95c19d266aff9515acc0ab2dac2a90285dd1103e /pairwise-tests/references
parenta8deb6af6a14bc5e56e86289a2858a0258558388 (diff)
downloadrulesets-b11cfd66b185a253fecf10ad06080ae165f32a74.tar.gz
rulesets-b11cfd66b185a253fecf10ad06080ae165f32a74.zip
feat: adopt pairwise-tests (PICT combinatorial) + cross-reference from existing testing skills
Forked verbatim from omkamal/pypict-claude-skill (MIT). LICENSE preserved. Renamed from `pict-test-designer` to `pairwise-tests` — technique-first naming so users invoking "pairwise" or "combinatorial" find it; PICT remains the tool under the hood. Bundle (skill-runtime only): pairwise-tests/SKILL.md (renamed, description rewritten) pairwise-tests/LICENSE (MIT, preserved) pairwise-tests/references/pict_syntax.md pairwise-tests/references/examples.md pairwise-tests/scripts/pict_helper.py (Python CLI for model gen / output fmt) pairwise-tests/scripts/README.md Upstream's repo-level docs (README, QUICKSTART, CONTRIBUTING, etc.) and `examples/` dir (ATM + gearbox walkthroughs — useful as reading, not as skill-runtime) omitted from the fork. Attribution footer added. Cross-references so /add-tests naturally routes to /pairwise-tests when warranted: - add-tests/SKILL.md Phase 2 step 8: if a function in scope has 3+ parameters each taking multiple values, surface `/pairwise-tests` to the user before proposing normal category coverage. Default continues with /add-tests; user picks pairwise explicitly. - claude-rules/testing.md: new "Combinatorial Coverage" section after the Normal/Boundary/Error categories. Explains when pairwise wins, when to skip (regulated / provably exhaustive contexts, ≤2 parameters, non- parametric testing), and points at /pairwise-tests. - languages/python/claude/rules/python-testing.md: new "Pairwise / Combinatorial for Parameter-Heavy Functions" subsection under the parametrize guidance. Explains the pytest workflow: /pairwise-tests generates the matrix, paste into pytest parametrize block, or use pypict helper directly. Mechanism note: cross-references are judgment-based — Claude reads the nudges in add-tests/testing/python-testing and acts on them when appropriate, not automatic dispatch. Craig can still invoke /pairwise-tests directly when he already knows he wants combinatorial coverage. Makefile SKILLS extended; make install symlinks /pairwise-tests globally.
Diffstat (limited to 'pairwise-tests/references')
-rw-r--r--pairwise-tests/references/examples.md98
-rw-r--r--pairwise-tests/references/pict_syntax.md83
2 files changed, 181 insertions, 0 deletions
diff --git a/pairwise-tests/references/examples.md b/pairwise-tests/references/examples.md
new file mode 100644
index 0000000..258b1ef
--- /dev/null
+++ b/pairwise-tests/references/examples.md
@@ -0,0 +1,98 @@
+# PICT Examples Reference
+
+> **Note**: This is a placeholder file. Comprehensive examples are coming soon!
+>
+> For now, check out the [examples directory](../examples/) for complete real-world examples.
+
+## Available Examples
+
+### Complete Examples
+- **[ATM System Testing](../examples/atm-specification.md)**: Comprehensive banking ATM system with 31 test cases
+
+### Coming Soon
+
+#### Software Testing
+- Function testing with multiple parameters
+- API endpoint testing
+- Database query validation
+- Algorithm testing
+
+#### Web Applications
+- Form validation
+- User authentication
+- E-commerce checkout
+- Shopping cart operations
+
+#### Configuration Testing
+- System configurations
+- Feature flags
+- Environment settings
+- Browser compatibility
+
+#### Mobile Testing
+- Device and OS combinations
+- Screen sizes
+- Network conditions
+- Permissions
+
+## Pattern Library (Coming Soon)
+
+### Common Constraint Patterns
+
+```
+# Dependency constraints
+IF [FeatureA] = "Enabled" THEN [FeatureB] = "Enabled";
+
+# Exclusive options
+IF [PaymentMethod] = "Cash" THEN [InstallmentPlan] = "None";
+
+# Platform limitations
+IF [OS] = "iOS" THEN [Browser] IN {Safari, Chrome};
+
+# Environment restrictions
+IF [Environment] = "Production" THEN [LogLevel] <> "Debug";
+```
+
+### Boundary Value Patterns
+
+```
+# Numeric boundaries
+Age: 0, 17, 18, 64, 65, 100
+
+# Size categories
+FileSize: 0KB, 1KB, 1MB, 100MB, 1GB
+
+# Time periods
+Duration: 0s, 1s, 30s, 60s, 3600s
+```
+
+### Negative Testing Patterns
+
+```
+# Invalid inputs (using ~ prefix in some PICT variants)
+Email: Valid, Invalid, Empty, TooLong
+Password: Strong, Weak, Empty, SpecialChars
+
+# Error conditions
+NetworkStatus: Connected, Slow, Disconnected, Timeout
+```
+
+## Contributing Examples
+
+Have an example to share? We'd love to include it!
+
+1. Create your example following the structure in [examples/README.md](../examples/README.md)
+2. Include:
+ - Original specification
+ - PICT model
+ - Test cases with expected outputs
+ - Learning points
+3. Submit a pull request
+
+See [CONTRIBUTING.md](../CONTRIBUTING.md) for details.
+
+## External Resources
+
+- [Pairwise Testing Tutorial](https://www.pairwisetesting.com/)
+- [NIST Combinatorial Testing Resources](https://csrc.nist.gov/projects/automated-combinatorial-testing-for-software)
+- [Microsoft PICT Examples](https://github.com/microsoft/pict/tree/main/doc)
diff --git a/pairwise-tests/references/pict_syntax.md b/pairwise-tests/references/pict_syntax.md
new file mode 100644
index 0000000..d25fb57
--- /dev/null
+++ b/pairwise-tests/references/pict_syntax.md
@@ -0,0 +1,83 @@
+# PICT Syntax Reference
+
+> **Note**: This is a placeholder file. Complete syntax documentation is coming soon!
+>
+> For now, please refer to the official PICT documentation:
+> - [Microsoft PICT on GitHub](https://github.com/microsoft/pict)
+> - [PICT User Guide](https://github.com/microsoft/pict/blob/main/doc/pict.md)
+
+## Quick Reference
+
+### Basic Model Structure
+
+```
+# Parameters
+ParameterName: Value1, Value2, Value3
+AnotherParameter: ValueA, ValueB, ValueC
+
+# Constraints (optional)
+IF [ParameterName] = "Value1" THEN [AnotherParameter] <> "ValueA";
+```
+
+### Parameter Definition
+
+```
+ParameterName: Value1, Value2, Value3, ...
+```
+
+### Constraint Syntax
+
+```
+IF <condition> THEN <condition>;
+```
+
+### Operators
+
+- `=` - Equal to
+- `<>` - Not equal to
+- `>` - Greater than
+- `<` - Less than
+- `>=` - Greater than or equal to
+- `<=` - Less than or equal to
+- `IN` - Member of set
+- `AND` - Logical AND
+- `OR` - Logical OR
+- `NOT` - Logical NOT
+
+### Example Constraints
+
+```
+# Simple constraint
+IF [OS] = "MacOS" THEN [Browser] <> "IE";
+
+# Multiple conditions
+IF [Environment] = "Production" AND [LogLevel] = "Debug" THEN [Approved] = "False";
+
+# Set membership
+IF [UserRole] = "Guest" THEN [Permission] IN {Read, None};
+```
+
+## Coming Soon
+
+Detailed documentation will include:
+- Complete grammar specification
+- Advanced features (sub-models, aliasing, seeding)
+- Negative testing patterns
+- Weight specifications
+- Order specifications
+- Examples for each feature
+
+## Contributing
+
+If you'd like to help complete this documentation:
+1. Fork the repository
+2. Add content to this file
+3. Submit a pull request
+
+See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
+
+## External Resources
+
+- [Official PICT Documentation](https://github.com/microsoft/pict/blob/main/doc/pict.md)
+- [pypict Documentation](https://github.com/kmaehashi/pypict)
+- [Pairwise Testing Explained](https://www.pairwisetesting.com/)