2 ms·
I'd also recommend introducing mutation testing in addition to fuzz testing. There is some overlap between the two techniques, but they approach the problem fr
by dkubb 11y ago
I'd also recommend introducing mutation testing in addition to fuzz testing.
There is some overlap between the two techniques, but they approach the problem from completely different directions. Fuzz testing randomly generates inputs and uses that to find bugs where the input is improperly handled. Mutation testing randomly changes the code to make sure the tests themselves cover all the code that is written.
A good approach is to use mutation testing to make sure the existing code is fully covered by the tests. Then use fuzz testing to identify new failing test cases that need to be written. Once the implementation handles the new test cases, mutation test that code to make sure there aren't any untested code paths.
I used this same approach on an Relational Algebra + SQL generator lib I wrote a while ago and the result was something that was very stable -- the only bugs were due to flaws in my own understanding of the specification, not bugs in the implementation.
- eru 11y agoI'd be interested to read a more detailed write up of the process.