6 ms·
Testing tests if it responds with the correct output for a known input. Fuzzing can't replace testing because a. you still want to test that it works correctly
by KTibow 3y ago
Testing tests if it responds with the correct output for a known input. Fuzzing can't replace testing because a. you still want to test that it works correctly given known inputs b. not all functions need to accept badly formatted input
- bqmjjx0kac 3y agoParadigm shift: fuzzing is testing if you write fuzzers that test properties. For instance, if you have a class that can be parsed and serialized, you may write something like this in a fuzzer: if (serialize(parse(data, size)) != data) { CRASH(); } More directly addressing your point about not all functions accepting malformed input, you can also use a fuzzer to exercise sequences of methods on an object. If your class has any invariants, you can test them along the way.
- Jaxan 3y agoIs this any different from property based testing?
- Loq 3y agoIndeed, what bqmjjx0kac proposes, is orthogonal to fuzzing (which is an algorithmic technique to drive test-coverage), and an instance of defining, and dynamically checking a property. Apropos: https://blog.regehr.org/archives/1687 https://blog.regehr.org/archives/1687
- bqmjjx0kac 3y agoI consider coverage-guided fuzzing with property checks to be a degenerate form of property-based testing à la Haskell's QuickCheck. Defining custom mutators (e.g libprotobuf-mutator) is sort of like implementing the Arbitrary typeclass if you squint. I admit, this might be where the similarities end :)
- costco 3y agoIf it is a common task like decoding base64 then you can compare the results of your function with the results of a well established existing implementation and abort when they are not equal.