24 ms·
One thing I've wondered about with this sort of language fuzzing is how well the fuzz inputs cover the program under test. They had all the tools in place to m
by pfdietz 7y ago
One thing I've wondered about with this sort of language fuzzing is how well the fuzz inputs cover the program under test. They had all the tools in place to measure that, but I didn't see them report how well they did.
Ideally, one would like to adjust the fuzzer based on that information to steer executions to lesser covered parts of the code (or, possibly, toward parts you want to focus your testing on.)
- iamxy 7y agoAuthor here. Thanks for your great advice. We will apply this experimental project to testing TiDB in the future and output the report to https://github.com/fuzzdebugplatform/fuzz_debug_platform/issues/3 https://github.com/fuzzdebugplatform/fuzz_debug_platform/iss.... Through the statistics of code block coverage, we can not only identify suspicious code blocks but also count code coverage. The fuzzer we implemented is driven by BNF expressions. We can adjust the inputs of the fuzzer based on the statistics.
- pfdietz 7y agoOne of the tricks you should try is adding Swarm Testing. It tends to make this sort of fuzzing more effective at finding bugs, and I would not be surprised if it improves your coverage also. https://dl.acm.org/citation.cfm?id=2336763 https://dl.acm.org/citation.cfm?id=2336763 https://blog.regehr.org/archives/591 https://blog.regehr.org/archives/591
- iamxy 7y agoThank you for the information, very helpful, I will learn and try it in our test.