3 ms·
To clarify: Tests as I see them and described them above do not need to stop when the software is shipped. They are meant to run indefinitely and can continue a
by triska 5y ago
To clarify: Tests as I see them and described them above do not need to stop when the software is shipped. They are meant to run indefinitely and can continue also long after the software is shipped, and independent of the environment in which it was developed. For example, I found several mistakes in released versions of GNU Emacs with test cases that repeatedly do the same thing, over and over, until they reveal a crash or memory leak. A few examples of this approach:
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=496 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=496
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=576 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=576
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=726 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=726
Fuzzing is a useful approach and does reveal many issues with software. As I see it, the fact that fuzzing already tends to reveal mistakes in software is a good motivation to strive for methods that yield greater coverage based on systematic exhaustive search, so that also mistakes that occur in very rare situations are reliably detected.