4 ms·
There's another distinction besides the "level" at which they operate. They use different techniques to generate test inputs. QuickCheck-style tools typically
by cakoose 10y ago
There's another distinction besides the "level" at which they operate. They use different techniques to generate test inputs.
QuickCheck-style tools typically generate inputs based on high-level static information, like types or programmer-provided specs. To reach the sophistication of modern fuzzers, you need to be smarter.
For one, you should inspect the code of the function. If you know how the code branches, you can more efficiently generate inputs that exercise both sizes of the branch.
But since you can't always figure out how to exercise both sizes of a branch just based on static analysis, fuzzers also observe the dynamic execution behavior of the program. This allows them to randomly adjust inputs until they know both sides of a branch have been hit.
I imagine that lifting fuzzing techniques to a higher level could have efficiency benefits. Maybe someone builds a fuzzer that runs on Rust MIR? Or maybe Rust can dumps out additional metadata (alongside the LLVM bitcode) to help the fuzzer figure things out?
- eru 10y agoQuickCheck happens to use types (ie static information). But Python's hypothesis shows that QuickCheck works just as well with dynamic languages: https://hypothesis.readthedocs.io/en/latest/ https://hypothesis.readthedocs.io/en/latest/
- cakoose 10y agoYeah, in this case, Hypothesis uses programmer-provided input specs, which are similar but not quite the same as type declarations in a typed programming language. I'd still consider this to be essentially "static" information, since they don't change as the program executes.
- eru 10y agoPerhaps. Though 'shrinking' definitely makes use of dynamic information.