3 ms·

For anyone interested there is something called fuzzing that uses usually code coverage based heuristics to generate data to find bugs. For example LLVM's lib

by kuter 6y ago

For anyone interested there is something called fuzzing that uses usually code coverage based heuristics to generate data to find bugs. For example LLVM's lib fuzzer uses instrumentation to track code coverage and mutates data to find invalid behaviour. https://llvm.org/docs/LibFuzzer.html https://llvm.org/docs/LibFuzzer.html It uses a compiler pass to insert code to branch points functions calls etc. I think it uses genetic algorithms to increase coverage by changing the data. There are others that work in similar ways one of them is. https://github.com/google/AFL https://github.com/google/AFL

Vinnl 6y ago
Somewhat similar for web UIs: Quickstrom is a tool that lets you define a set of conditions that should hold (e.g. "there should always be an 'Add todo' button"), and then it'll simulate behaviour that might break that condition. See https://quickstrom.io/ https://quickstrom.io/ (I haven't used it myself yet, but it looks interesting.)
cuillevel3 6y ago
Here is an eight year old presentation on fuzzing X: https://media.ccc.de/v/30C3_-_5499_-_en_-_saal_1_-_201312291830_-_x_security_-_ilja_van_sprundel https://media.ccc.de/v/30C3_-_5499_-_en_-_saal_1_-_201312291...
suyjuris 6y ago
I have used AFL a few times casually in some personal projects, and it has always performed quite well for me. Of course, there are a lot of weird cornercases which would not occur on real-world (non-adversarial) inputs, but it also found some very real bugs. (For example, I once wrote a hash table implementation where the insertion and resizing procedures had slightly different views on wraparound, causing failures on very specific inputs. Another time, I wrote some code to buffer out-of-order messages, which would only occur due to a race condition. It was wrong. Both times I had thought carefully about the code, and the bugs would have been painful to discover otherwise.)
passivate 6y ago
Well, I guess the obvious question to ask is has anyone run this particular fuzzer on the code in question?

by kuter 6y ago

Vinnl 6y ago

Somewhat similar for web UIs: Quickstrom is a tool that lets you define a set of conditions that should hold (e.g. "there should always be an 'Add todo' button"), and then it'll simulate behaviour that might break that condition. See https://quickstrom.io/ https://quickstrom.io/ (I haven't used it myself yet, but it looks interesting.)

cuillevel3 6y ago

Here is an eight year old presentation on fuzzing X: https://media.ccc.de/v/30C3_-_5499_-_en_-_saal_1_-_201312291830_-_x_security_-_ilja_van_sprundel https://media.ccc.de/v/30C3_-_5499_-_en_-_saal_1_-_201312291...

suyjuris 6y ago

I have used AFL a few times casually in some personal projects, and it has always performed quite well for me. Of course, there are a lot of weird cornercases which would not occur on real-world (non-adversarial) inputs, but it also found some very real bugs. (For example, I once wrote a hash table implementation where the insertion and resizing procedures had slightly different views on wraparound, causing failures on very specific inputs. Another time, I wrote some code to buffer out-of-order messages, which would only occur due to a race condition. It was wrong. Both times I had thought carefully about the code, and the bugs would have been painful to discover otherwise.)

passivate 6y ago

Well, I guess the obvious question to ask is has anyone run this particular fuzzer on the code in question?