3 ms·
In my experience very rare. It's so rare I'd say it basically doesn't happen. You'll get some fluctuation, a bug may come up half an hour earlier or not. But r
by hannob 10y ago
In my experience very rare. It's so rare I'd say it basically doesn't happen.
You'll get some fluctuation, a bug may come up half an hour earlier or not. But results tend to be pretty reproducible. If you find a bug with a specific test and tool in x hours then the next time you try for at least x+1 hours you'll find it again.
- moyix 10y agoI think that's right for just picking a different PRNG seed. When you start looking at modifying the search heuristics, mutation operators, or other parts of the "strategy", you definitely start finding different bugs, though.
- hannob 10y agoYes, absolutely. Different strategies can lead to vastly different results, often it's subtle things. Good example: There was a bug I found in openssl that Libfuzzer was unable to find. The Libfuzzer developer was quite interested in this and has now adopted new mutation strategies: https://github.com/google/sanitizers/issues/710 https://github.com/google/sanitizers/issues/710