diff options
| author | Craig Jennings <c@cjennings.net> | 2026-04-19 16:12:02 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-04-19 16:12:02 -0500 |
| commit | b11cfd66b185a253fecf10ad06080ae165f32a74 (patch) | |
| tree | 95c19d266aff9515acc0ab2dac2a90285dd1103e /pairwise-tests/references | |
| parent | a8deb6af6a14bc5e56e86289a2858a0258558388 (diff) | |
| download | rulesets-b11cfd66b185a253fecf10ad06080ae165f32a74.tar.gz rulesets-b11cfd66b185a253fecf10ad06080ae165f32a74.zip | |
feat: adopt pairwise-tests (PICT combinatorial) + cross-reference from existing testing skills
Forked verbatim from omkamal/pypict-claude-skill (MIT). LICENSE preserved.
Renamed from `pict-test-designer` to `pairwise-tests` — technique-first
naming so users invoking "pairwise" or "combinatorial" find it; PICT
remains the tool under the hood.
Bundle (skill-runtime only):
pairwise-tests/SKILL.md (renamed, description rewritten)
pairwise-tests/LICENSE (MIT, preserved)
pairwise-tests/references/pict_syntax.md
pairwise-tests/references/examples.md
pairwise-tests/scripts/pict_helper.py (Python CLI for model gen / output fmt)
pairwise-tests/scripts/README.md
Upstream's repo-level docs (README, QUICKSTART, CONTRIBUTING, etc.) and
`examples/` dir (ATM + gearbox walkthroughs — useful as reading, not as
skill-runtime) omitted from the fork. Attribution footer added.
Cross-references so /add-tests naturally routes to /pairwise-tests when
warranted:
- add-tests/SKILL.md Phase 2 step 8: if a function in scope has 3+ parameters
each taking multiple values, surface `/pairwise-tests` to the user before
proposing normal category coverage. Default continues with /add-tests;
user picks pairwise explicitly.
- claude-rules/testing.md: new "Combinatorial Coverage" section after the
Normal/Boundary/Error categories. Explains when pairwise wins, when to
skip (regulated / provably exhaustive contexts, ≤2 parameters, non-
parametric testing), and points at /pairwise-tests.
- languages/python/claude/rules/python-testing.md: new "Pairwise /
Combinatorial for Parameter-Heavy Functions" subsection under the
parametrize guidance. Explains the pytest workflow: /pairwise-tests
generates the matrix, paste into pytest parametrize block, or use
pypict helper directly.
Mechanism note: cross-references are judgment-based — Claude reads the
nudges in add-tests/testing/python-testing and acts on them when appropriate,
not automatic dispatch. Craig can still invoke /pairwise-tests directly when
he already knows he wants combinatorial coverage.
Makefile SKILLS extended; make install symlinks /pairwise-tests globally.
Diffstat (limited to 'pairwise-tests/references')
| -rw-r--r-- | pairwise-tests/references/examples.md | 98 | ||||
| -rw-r--r-- | pairwise-tests/references/pict_syntax.md | 83 |
2 files changed, 181 insertions, 0 deletions
diff --git a/pairwise-tests/references/examples.md b/pairwise-tests/references/examples.md new file mode 100644 index 0000000..258b1ef --- /dev/null +++ b/pairwise-tests/references/examples.md @@ -0,0 +1,98 @@ +# PICT Examples Reference + +> **Note**: This is a placeholder file. Comprehensive examples are coming soon! +> +> For now, check out the [examples directory](../examples/) for complete real-world examples. + +## Available Examples + +### Complete Examples +- **[ATM System Testing](../examples/atm-specification.md)**: Comprehensive banking ATM system with 31 test cases + +### Coming Soon + +#### Software Testing +- Function testing with multiple parameters +- API endpoint testing +- Database query validation +- Algorithm testing + +#### Web Applications +- Form validation +- User authentication +- E-commerce checkout +- Shopping cart operations + +#### Configuration Testing +- System configurations +- Feature flags +- Environment settings +- Browser compatibility + +#### Mobile Testing +- Device and OS combinations +- Screen sizes +- Network conditions +- Permissions + +## Pattern Library (Coming Soon) + +### Common Constraint Patterns + +``` +# Dependency constraints +IF [FeatureA] = "Enabled" THEN [FeatureB] = "Enabled"; + +# Exclusive options +IF [PaymentMethod] = "Cash" THEN [InstallmentPlan] = "None"; + +# Platform limitations +IF [OS] = "iOS" THEN [Browser] IN {Safari, Chrome}; + +# Environment restrictions +IF [Environment] = "Production" THEN [LogLevel] <> "Debug"; +``` + +### Boundary Value Patterns + +``` +# Numeric boundaries +Age: 0, 17, 18, 64, 65, 100 + +# Size categories +FileSize: 0KB, 1KB, 1MB, 100MB, 1GB + +# Time periods +Duration: 0s, 1s, 30s, 60s, 3600s +``` + +### Negative Testing Patterns + +``` +# Invalid inputs (using ~ prefix in some PICT variants) +Email: Valid, Invalid, Empty, TooLong +Password: Strong, Weak, Empty, SpecialChars + +# Error conditions +NetworkStatus: Connected, Slow, Disconnected, Timeout +``` + +## Contributing Examples + +Have an example to share? We'd love to include it! + +1. Create your example following the structure in [examples/README.md](../examples/README.md) +2. Include: + - Original specification + - PICT model + - Test cases with expected outputs + - Learning points +3. Submit a pull request + +See [CONTRIBUTING.md](../CONTRIBUTING.md) for details. + +## External Resources + +- [Pairwise Testing Tutorial](https://www.pairwisetesting.com/) +- [NIST Combinatorial Testing Resources](https://csrc.nist.gov/projects/automated-combinatorial-testing-for-software) +- [Microsoft PICT Examples](https://github.com/microsoft/pict/tree/main/doc) diff --git a/pairwise-tests/references/pict_syntax.md b/pairwise-tests/references/pict_syntax.md new file mode 100644 index 0000000..d25fb57 --- /dev/null +++ b/pairwise-tests/references/pict_syntax.md @@ -0,0 +1,83 @@ +# PICT Syntax Reference + +> **Note**: This is a placeholder file. Complete syntax documentation is coming soon! +> +> For now, please refer to the official PICT documentation: +> - [Microsoft PICT on GitHub](https://github.com/microsoft/pict) +> - [PICT User Guide](https://github.com/microsoft/pict/blob/main/doc/pict.md) + +## Quick Reference + +### Basic Model Structure + +``` +# Parameters +ParameterName: Value1, Value2, Value3 +AnotherParameter: ValueA, ValueB, ValueC + +# Constraints (optional) +IF [ParameterName] = "Value1" THEN [AnotherParameter] <> "ValueA"; +``` + +### Parameter Definition + +``` +ParameterName: Value1, Value2, Value3, ... +``` + +### Constraint Syntax + +``` +IF <condition> THEN <condition>; +``` + +### Operators + +- `=` - Equal to +- `<>` - Not equal to +- `>` - Greater than +- `<` - Less than +- `>=` - Greater than or equal to +- `<=` - Less than or equal to +- `IN` - Member of set +- `AND` - Logical AND +- `OR` - Logical OR +- `NOT` - Logical NOT + +### Example Constraints + +``` +# Simple constraint +IF [OS] = "MacOS" THEN [Browser] <> "IE"; + +# Multiple conditions +IF [Environment] = "Production" AND [LogLevel] = "Debug" THEN [Approved] = "False"; + +# Set membership +IF [UserRole] = "Guest" THEN [Permission] IN {Read, None}; +``` + +## Coming Soon + +Detailed documentation will include: +- Complete grammar specification +- Advanced features (sub-models, aliasing, seeding) +- Negative testing patterns +- Weight specifications +- Order specifications +- Examples for each feature + +## Contributing + +If you'd like to help complete this documentation: +1. Fork the repository +2. Add content to this file +3. Submit a pull request + +See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines. + +## External Resources + +- [Official PICT Documentation](https://github.com/microsoft/pict/blob/main/doc/pict.md) +- [pypict Documentation](https://github.com/kmaehashi/pypict) +- [Pairwise Testing Explained](https://www.pairwisetesting.com/) |
