4 ms·
It is normally safe to assume the exhaustive testing isn't possible because the total states to exhaustively tests exceeds the number of atoms in the universe.
by bluGill 3y ago
It is normally safe to assume the exhaustive testing isn't possible because the total states to exhaustively tests exceeds the number of atoms in the universe. There are a few exceptions, but in general we should assume it is always impossible to exhaustively test programs. Which means we need to use something else to find what the limits are and test only those, or use formal methods (both would be my recommendation - though I'll admit I have yet to figure out how to use formal methods)
- dllthomas 3y ago> in general we should assume it is always impossible to exhaustively test programs The whole program? Yes. For testing individual components, there's no need to assume. The answer is very likely either clearly yes (the input is a handful of booleans) or clearly no (the input is several integers or even unbounded) and for those cases where it's not immediately clear (one i16? Three u8s?) it's probably still not hard to think through for your particular case in your particular context.
- mananaysiempre 3y agoBrute forcing a single 32-bit input has also been usefully performed in some cases, such as for single-precision implementations of special functions[1]. [1] https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/ https://randomascii.wordpress.com/2014/01/27/theres-only-fou...
- bluGill 3y ago32 bits is a bit over 4 billion states, so that is doable. However a lot of algorithms can have more than 32 bits of states and so quickly get beyond what is doable.
- dllthomas 3y ago> 32 bits is a bit over 4 billion states, so that is doable. Whether 4B is doable depends on how much work you're doing for each of those states, but yes, "4B" alone certainly doesn't rule it out. There's also a question of whether it goes with your most frequently run tests or is something that's only run occasionally. > However a lot of algorithms can have more than 32 bits of states and so quickly get beyond what is doable. I'm pretty confident we're all in agreement there.