3 ms·
Fuzzy is expensive. It's slow to run, and needs a long configuration then calibration time until you don't find false positives anymore. Also, many bugs found
by BiteCode_dev 4y ago
Fuzzy is expensive.
It's slow to run, and needs a long configuration then calibration time until you don't find false positives anymore.
Also, many bugs found by fuzzing are in the "it's ok to have them" range.
10Mo URL crashes my Python interpreter? I'm ok with that.
Not to say fuzzing is not useful, but it's not cheap, and the benefits really shines for mature software.
- morelisp 4y ago> false positives... 10Mo URL crashes my Python interpreter? I'm ok with that. Either you're wrong about the "false" part or you're testing the wrong entry point. Fuzzing has a long warmup time but probably the lowest false positive rate of any testing technique - if it crashes you're wrong about some invariant, even if that's only "I need to fuzz parseX, not decodeX".
- BiteCode_dev 4y agoFor one entry point, fuzzing may found 30 ways to crash it, none of which you care about. A thing is only a bug if it's a problem for use case your users will have an issue with.
- qingdao99 4y agoI think it just depends on how critical the software is. If it's just a webpage running in your browser then it's not a big deal if it crashes on some weird input. If it's a database, each crash can cause huge amounts of damage.
- m463 4y agoI think a lot of these types of tools have a high initial investment, but also a high initial return on investment. The first couple of runs everything breaks, then code is added to NOT catastrophically fail, then you are pretty good. (or move into lots of false failures) Sort of like -Werror or linting/static analysis type tools, or code coverage, or similar...