5 ms·

At risk of sounding ignorant but has anyone fuzzed something like APIs before? Would love to get some anecdotes from fuzzing more abstracted systems.

bch 8y ago
OpenBSD kernel? https://news.ycombinator.com/item?id=17929234 https://news.ycombinator.com/item?id=17929234
paulgdp 8y ago
The Rust ecosystem has a team that tries to fuzz as many crates (rust packages) as possible [1]. Unlike C/C++ and like Python, fuzzing Rust code is not really about finding memory bugs but more about finding logical errors [2]. To do this, a project has been set up with 83 (so far) targets fuzzing the public API of 48 (so far) important crates [3]. All those targets can be fuzzed using any of the three major native code feedback-based fuzzers (AFL, LibFuzzer, and Honggfuzz). [1] https://github.com/rust-fuzz/targets https://github.com/rust-fuzz/targets [2] see the trophy case: https://github.com/rust-fuzz/trophy-case https://github.com/rust-fuzz/trophy-case [3] https://github.com/rust-fuzz/targets/blob/master/common/src/lib.rs https://github.com/rust-fuzz/targets/blob/master/common/src/... Disclaimer: I'm a member of this team and the author of the honggfuzz crate that makes honggfuzz work with Rust code.
- cpeterso 8y ago
  And Cargo has good support for integrating Rust fuzzers into one's own projects: https://medium.com/@seasoned_sw/fuzz-testing-in-rust-with-cargo-fuzz-13b89feecc30 https://medium.com/@seasoned_sw/fuzz-testing-in-rust-with-ca... btw, I'm impressed that the rust-fuzz trophy list includes only one UAF, one uninitialized memory read, and no segfaults. :) https://github.com/rust-fuzz/trophy-case https://github.com/rust-fuzz/trophy-case
paulgdp 8y ago
Also, for a toy rust project, I wrote an implementation of a copy-on-write B+tree and I used a fuzzer (honggfuzz) to generate automatically all the test cases. Ii was insanely effective, there were so much more edge cases than I thought. The fuzzer found all of them and got me a 100% code coverage in no time. All I had to write as test code was a function mapping an array of random data to a series of btree instructions, apply those to both my implementation and a reference and then check that the two structures had the same data.
Joky 8y ago
See the "trophies" here: https://llvm.org/docs/LibFuzzer.html#trophies https://llvm.org/docs/LibFuzzer.html#trophies Also: https://github.com/google/oss-fuzz https://github.com/google/oss-fuzz
QuinnWilton 8y ago
I'm an engineer at Tinfoil Security, working on a security scanner for REST APIs. At a high level, it ingests Swagger specifications, which it uses to build property testing generators to test the implementation against the schema + against various security tests. It isn't ready for public consumption yet, but we have a few customers using a private beta. We're still exploring the problem space, but so far it has been very successful at quickly confirming an API conforms to its specification, and automating the detection of most "input validation" bugs.

bch 8y ago

OpenBSD kernel? https://news.ycombinator.com/item?id=17929234 https://news.ycombinator.com/item?id=17929234

paulgdp 8y ago

The Rust ecosystem has a team that tries to fuzz as many crates (rust packages) as possible [1]. Unlike C/C++ and like Python, fuzzing Rust code is not really about finding memory bugs but more about finding logical errors [2]. To do this, a project has been set up with 83 (so far) targets fuzzing the public API of 48 (so far) important crates [3]. All those targets can be fuzzed using any of the three major native code feedback-based fuzzers (AFL, LibFuzzer, and Honggfuzz). [1] https://github.com/rust-fuzz/targets https://github.com/rust-fuzz/targets [2] see the trophy case: https://github.com/rust-fuzz/trophy-case https://github.com/rust-fuzz/trophy-case [3] https://github.com/rust-fuzz/targets/blob/master/common/src/lib.rs https://github.com/rust-fuzz/targets/blob/master/common/src/... Disclaimer: I'm a member of this team and the author of the honggfuzz crate that makes honggfuzz work with Rust code.

cpeterso 8y ago
And Cargo has good support for integrating Rust fuzzers into one's own projects: https://medium.com/@seasoned_sw/fuzz-testing-in-rust-with-cargo-fuzz-13b89feecc30 https://medium.com/@seasoned_sw/fuzz-testing-in-rust-with-ca... btw, I'm impressed that the rust-fuzz trophy list includes only one UAF, one uninitialized memory read, and no segfaults. :) https://github.com/rust-fuzz/trophy-case https://github.com/rust-fuzz/trophy-case

paulgdp 8y ago

Also, for a toy rust project, I wrote an implementation of a copy-on-write B+tree and I used a fuzzer (honggfuzz) to generate automatically all the test cases. Ii was insanely effective, there were so much more edge cases than I thought. The fuzzer found all of them and got me a 100% code coverage in no time. All I had to write as test code was a function mapping an array of random data to a series of btree instructions, apply those to both my implementation and a reference and then check that the two structures had the same data.

Joky 8y ago

See the "trophies" here: https://llvm.org/docs/LibFuzzer.html#trophies https://llvm.org/docs/LibFuzzer.html#trophies Also: https://github.com/google/oss-fuzz https://github.com/google/oss-fuzz

QuinnWilton 8y ago

I'm an engineer at Tinfoil Security, working on a security scanner for REST APIs. At a high level, it ingests Swagger specifications, which it uses to build property testing generators to test the implementation against the schema + against various security tests. It isn't ready for public consumption yet, but we have a few customers using a private beta. We're still exploring the problem space, but so far it has been very successful at quickly confirming an API conforms to its specification, and automating the detection of most "input validation" bugs.