Then I talk about randomly sampling from a partition as a way to eventually detect "hidden" partitions you didn't expect. Like Hypothesis does.
When dealing with many parameters, each with several partitions, the variations multiply quickly. To deal with combinatoric explosion, I talk about all pairs testing, also known as "pairwise" testing.
Manual testers should definitely know about equivalence class partitioning, one of the first things I learned when I switched to being a tester.
Also for "pairwise" testing, been a while since I heard that being mentioned!
I think more emphasis should be put on something I only realised recently, which hampered me a lot wrt adopting property testing, and which I still have a hard time with: property tests are not parametrised tests, it’s ok if they can only check some properties of the computation (e.g. post-conditions), that is still valuable.
For the longest time I was convinced I should have “perfect” oracles (unit test style) because that’s what simple examples usually show off, but outside of trivial examples it’s really only possible for limited cases unless you have a complete and perfect oracle (usually an existing known-good implementation).
Indeed, I tend to avoid writing a single test that captures 'all' of a function/unit's behaviour (like the article is doing).
Instead, I would write multiple tests, mostly just asserting one thing each. For example: `test_gcd_always_positive(m, n)`, `test_gcd_divides_arguments(m, n)`, `test_gcd_is_largest_divisor(m, n)`, etc.
I also think in terms of a 'threat model' for our colleagues or past/future selves:
- They're probably not trying to write broken code, so we don't need to catch all incorrect implementations. In other words, they're not adversarial.
- They're probably lazy or mistaken, so we should try to catch incorrect implementations which are simple to write, make obvious mistakes, etc.
For example, having `my_sort = lambda _: []` is a really lazy way to implement the right type signature. Checking that the lengths and set of elements are the same are good tests to catch that, so it seems worth doing. Likewise, checking that the elements are sorted is a good test to avoid the lazy approach of `my_sort = lambda l: l`.
However, the implementation which sorts the set of elements and pads with the largest is pretty convoluted, so a lazy developer will likely just write a basic sort algorithm at that point (e.g. `my_sort = lambda l: [x for y in set(l) for x in l if x == y]`). Also, the test is a bit tricky; if that counting function didn't already exist, I probably wouldn't bother writing that test at all (there are equivalents which might be simpler to write, e.g. for each input element: assert it appears in the output, then remove its first occurrence; then also assert the output ends up empty)
FYI my personal blog lives at https://mathspp.com/blog.