4 ms·
All-pairs testing. Lets say you've got a complicated piece of code that has a bunch of options (either input or configuration, doesn't matter). You can't test
by ajb 4y ago
All-pairs testing.
Lets say you've got a complicated piece of code that has a bunch of options (either input or configuration, doesn't matter). You can't test it exhaustively because that's exponential in the number of options. But you can test all combinations of each pair of options, in ~O(A x B) where A and B are two largest option cardinalities.
To be concrete about this, suppose there are 10 option variables each having at most 5 possible values, a,b,c,d,e,f,g,h,i,j. Suppose that d and e are have the largest cardinality (5). So you can test all pairs of assignments to d and e in 25 tests. But with the same 25 tests (or epsilon extra ones) you can test all pairs of assignments to every pair of variables - you need hardly any more tests.
Of course, coming up with the table of values is a bit tricky, especially if there are some constraints. But there are libraries to do it.
Takeaway: with a very small number of tests you can find corner cases that are very hard to locate with a standard approach.
This generalises to triples, etc.
(see https://burtleburtle.net/bob/math/jenny.html https://burtleburtle.net/bob/math/jenny.html)