3 ms·
That's a good point—even a few hundred random test cases are usually enough to cover "normal" code. I use QuickCheck for that at work, and it's enough to both f
by tikhonj 9y ago
That's a good point—even a few hundred random test cases are usually enough to cover "normal" code. I use QuickCheck for that at work, and it's enough to both find bugs I wouldn't see otherwise and to make me confident in my code when it doesn't find any bugs. (As a bonus, it also tries to "reduce" failing cases, which can really help with debugging.)
It's worth tuning your random input generator to produce inputs that commonly cause problems like -0.
I can still think of a few situations where exhaustiveness might be worth the effort:
- verifying code passed in from the outside (security considerations?)
- verifying code generated automatically (ie as part of program synthesis)
- verifying very general code you expect to be used in hard-to-predict contexts (ie library routines or compiler optimizations)
- brucedawson 9y agoA few hundred random test cases? Double-precision numbers can have 2,048 different exponents. I've seen multiple bugs that only happen with denormals so you need thousands of tests to have any hope of hitting those. If you're going to do random testing then you need to have a huge set of special numbers (denormals, NaNs, infinity, zero, exact powers of two, one more or less than exact powers of two) and you should do millions or billions of random numbers. Then you can start to feel confident.
- im3w1l 9y ago>It's worth tuning your random input generator to produce inputs that commonly cause problems like -0. This would definitely include denormals. As well as +-inf, nan, odd int, even int, negative odd and even int. +-1. Some .5s.