3 ms·
My understanding is that fuzz testing uses pseudo-random variation of the seed code; given a different seed to the PRNG, how common is it for the same fuzz test
by thyrsus 10y ago
My understanding is that fuzz testing uses pseudo-random variation of the seed code; given a different seed to the PRNG, how common is it for the same fuzz test to identify different flaws?
- hannob 10y agoIn my experience very rare. It's so rare I'd say it basically doesn't happen. You'll get some fluctuation, a bug may come up half an hour earlier or not. But results tend to be pretty reproducible. If you find a bug with a specific test and tool in x hours then the next time you try for at least x+1 hours you'll find it again.
- moyix 10y agoI think that's right for just picking a different PRNG seed. When you start looking at modifying the search heuristics, mutation operators, or other parts of the "strategy", you definitely start finding different bugs, though.
- hannob 10y agoYes, absolutely. Different strategies can lead to vastly different results, often it's subtle things. Good example: There was a bug I found in openssl that Libfuzzer was unable to find. The Libfuzzer developer was quite interested in this and has now adopted new mutation strategies: https://github.com/google/sanitizers/issues/710 https://github.com/google/sanitizers/issues/710