4 ms·

Any idea as to why the following requirement? This will limit Afl.rs users quite a bit. afl.rs needs to compile against a version of LLVM that matches rust

by cm3 10y ago

Any idea as to why the following requirement? This will limit Afl.rs users quite a bit. afl.rs needs to compile against a version of LLVM that matches rustc's. The easy solution (if you can wait on a slow build) is to build rustc from source and put it in your PATH. Then afl.rs's build script will find llvm-config automatically. Otherwise, the environment variable LLVM_CONFIG should hold the path to llvm-config when you build afl.rs. I was under the impression that Afl can test any application that takes stdin. I'm underinformed for sure, so what's the idea behind explicitly adding code to support Afl fuzzing?

fennecfoxen 10y ago
This might be working with libfuzzer, one of the tools afl lists on its homepage as derivatives/offshoots: http://llvm.org/docs/LibFuzzer.html http://llvm.org/docs/LibFuzzer.html afl is fundamentally instrumentation-based and likes a good look at what sorts of different branches a program is taking, it's a little bit more hands-on than just feeding things to STDIN.
- sanxiyn 10y ago
  No, afl.rs is not using libFuzzer. afl.rs is using llvm_mode code in afl.
zuzun 10y ago
I think it injects code at branches to observe the branch coverage. After all you have to compile your C programs with the afl-gcc binary. Maybe you could ptrace every single instruction executed by the program and then stop at every jump and use the relative values of the IP to identify branches... I don't know if that would work, but it would be much slower.
masklinn 10y ago
Data input goes through stdin, but AFL needs to instrument the binary to observe branch coverage and changes in branch coverage across varying inputs. From the alf-fuzz site: > American fuzzy lop is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. That's why you need to compile C programs with afl-gcc and C++ programs with afl-g++: http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt
- cm3 10y ago
  I see, so does that mean it potentially would be required to explicitly support other code generators (GHC, OCaml, or something else), assuming the existing instrumentation is insufficient?
  - masklinn 10y ago
    I think so, when using an unsupported codegen backend you'd need to add guided fuzzing support.
frewsxcv 10y ago
Author here. AFL is coverage-guided fuzz testing (fuzz testing that relies on code coverage). Someone (not me) wrote an "LLVM pass" that loops through the generated LLVM IR for any LLVM-compiled program and injects the necessary AFL instrumentation such that AFL can work on it: https://github.com/mirrorer/afl/blob/master/llvm_mode/afl-llvm-pass.so.cc https://github.com/mirrorer/afl/blob/master/llvm_mode/afl-ll... Since Rust uses LLVM when compiling a binary, all afl.rs does is incorporate this instrumentation and AFL just needs to be run on the resulting binary. There are multiple versions of LLVM though. Rust is compiled using a specific version of LLVM. Since the LLVM pass above is reading the generated LLVM that Rust produces, the versions can't be too far off otherwise the LLVM pass might see some unrecognized symbols. The README in the project suggests just compiling Rust which also compiles LLVM, then you'll have a version of LLVM that is guaranteed to be the same when you setup the instrumentation. I think it might be sufficient to update the documentation to suggest the user of afl.rs just download and use LLVM 3.8 since this is the major version that Rust uses internally. I need to do more testing to confirm this will work for everyone, then I can update (and greatly simplify) the README.
- cm3 10y ago
  Thanks, because the way which snapshots of LLVM and rustc are required to build a certain rustc release does not make it an easy to use solution for common Rust users. It'd be great if an existing LLVM 3.8 can be leveraged.
  - steveklabnik 10y ago
    Note that with 1.10, the way we do rustc snapshots is changing: it will build with 1.9. And then 1.11 will build with 1.10, etc. so that will help a bit.
    - cm3 10y ago
      Hallelujah. This has been a pain point for anyone who tried to build rustc on non-tier1 platforms (musl as host, etc.).
ZoFreX 10y ago
Others here have explained the benefit - instrumenting. I will add: you do have the option of not instrumenting when using AFL, but instrumenting is a _lot_ faster - so much so that it is completely worth the extra effort. (It's actually not too much extra effort, either)

by cm3 10y ago

fennecfoxen 10y ago

This might be working with libfuzzer, one of the tools afl lists on its homepage as derivatives/offshoots: http://llvm.org/docs/LibFuzzer.html http://llvm.org/docs/LibFuzzer.html afl is fundamentally instrumentation-based and likes a good look at what sorts of different branches a program is taking, it's a little bit more hands-on than just feeding things to STDIN.

sanxiyn 10y ago
No, afl.rs is not using libFuzzer. afl.rs is using llvm_mode code in afl.

zuzun 10y ago

I think it injects code at branches to observe the branch coverage. After all you have to compile your C programs with the afl-gcc binary. Maybe you could ptrace every single instruction executed by the program and then stop at every jump and use the relative values of the IP to identify branches... I don't know if that would work, but it would be much slower.

masklinn 10y ago

Data input goes through stdin, but AFL needs to instrument the binary to observe branch coverage and changes in branch coverage across varying inputs. From the alf-fuzz site: > American fuzzy lop is a security-oriented fuzzer that employs a novel type of compile-time instrumentation and genetic algorithms to automatically discover clean, interesting test cases that trigger new internal states in the targeted binary. That's why you need to compile C programs with afl-gcc and C++ programs with afl-g++: http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt http://lcamtuf.coredump.cx/afl/QuickStartGuide.txt

cm3 10y ago
I see, so does that mean it potentially would be required to explicitly support other code generators (GHC, OCaml, or something else), assuming the existing instrumentation is insufficient?
- masklinn 10y ago
  I think so, when using an unsupported codegen backend you'd need to add guided fuzzing support.

frewsxcv 10y ago

Author here. AFL is coverage-guided fuzz testing (fuzz testing that relies on code coverage). Someone (not me) wrote an "LLVM pass" that loops through the generated LLVM IR for any LLVM-compiled program and injects the necessary AFL instrumentation such that AFL can work on it: https://github.com/mirrorer/afl/blob/master/llvm_mode/afl-llvm-pass.so.cc https://github.com/mirrorer/afl/blob/master/llvm_mode/afl-ll... Since Rust uses LLVM when compiling a binary, all afl.rs does is incorporate this instrumentation and AFL just needs to be run on the resulting binary. There are multiple versions of LLVM though. Rust is compiled using a specific version of LLVM. Since the LLVM pass above is reading the generated LLVM that Rust produces, the versions can't be too far off otherwise the LLVM pass might see some unrecognized symbols. The README in the project suggests just compiling Rust which also compiles LLVM, then you'll have a version of LLVM that is guaranteed to be the same when you setup the instrumentation. I think it might be sufficient to update the documentation to suggest the user of afl.rs just download and use LLVM 3.8 since this is the major version that Rust uses internally. I need to do more testing to confirm this will work for everyone, then I can update (and greatly simplify) the README.

cm3 10y ago
Thanks, because the way which snapshots of LLVM and rustc are required to build a certain rustc release does not make it an easy to use solution for common Rust users. It'd be great if an existing LLVM 3.8 can be leveraged.
- steveklabnik 10y ago
  Note that with 1.10, the way we do rustc snapshots is changing: it will build with 1.9. And then 1.11 will build with 1.10, etc. so that will help a bit.
  - cm3 10y ago
    Hallelujah. This has been a pain point for anyone who tried to build rustc on non-tier1 platforms (musl as host, etc.).

ZoFreX 10y ago

Others here have explained the benefit - instrumenting. I will add: you do have the option of not instrumenting when using AFL, but instrumenting is a _lot_ faster - so much so that it is completely worth the extra effort. (It's actually not too much extra effort, either)