6 ms·

Combinatorial explosions are best mitigated by doing property based testing, design by contract and creating clean, low level abstractions which you test agains

Combinatorial explosions are best mitigated by doing property based testing, design by contract and creating clean, low level abstractions which you test against. In practice I've also run into a lot of combinatorial explosions for which unit testing is worse than useless. E.g. say you have a library that has two dependencies and interacts with browsers you can have: * 30 different versions of library A * 50 different versions of library B * 25 different versions of the language runtime * 30 different browsers That's over a million different potential variations for every test. In practice though, there's a sharp power law. You don't have to test all of those variations, you just have to randomly throw variations at the code until bugs stop cropping up. 99% of the bugs you find probably be found in the first 1,000 iterations.

AllegedAlec 8y ago
Fair enough; in this case unit testing is not very useful. I was more thinking about business logic when I wrote that example.
ajb 8y ago
It's also worth looking at all-pairs testing. The insight is that you can cover all pairs of a set of combinations in only O(N^2) tests rather than O(N^D) irrespective of the number of dimensions. The number of tests ends up being the product of the size of the two largest dimensions. So in your example you can cover all combinations of each pair: * (lib A, lib B) * (lib A, language runtime) * (lib A, browser) * (lib B, language runtime) * (lib A, browser) * (language runtime, browser) in 50*30 = 1500 tests (or a little more to make the analysis tractable)
- Jach 8y ago
  This is clever... is the N^2 behavior just due to nCr(n, 2) = n^2/2 - n/2 growing quadratically?