4 ms·

I'm tired of sloppy testing. This issue was discoverable via a fuzzer as noted in the bug report. Software that accepts external inputs needs to be fuzzed regar

by hgs3 3y ago

I'm tired of sloppy testing. This issue was discoverable via a fuzzer as noted in the bug report. Software that accepts external inputs needs to be fuzzed regardless of whether it is written in a memory safe language or not. Uncaught exceptions and panics are still bugs and recovering from one without clearing internal state could lead to a security hole.

DougBTX 3y ago
> I'm tired of sloppy testing Exhaustive testing tops out at about two float 32s[0], so of course do that for small functions, but for full application testing type checks are much cheaper. [0] https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/ https://randomascii.wordpress.com/2014/01/27/theres-only-fou...
- ajross 3y ago
  Fuzzing isn't exhaustive testing, it's dynamic coverage testing. You provide random input until you discover your (compiler-instrumented) binary doing something you haven't seen yet, then permute the input until you get it to do something else, etc... For almost all non-pathological algorithms, it's capable of reaching full coverage in linear time; though some algorithms are always going to have pathological edge and require some whiteboxing.
pjmlp 3y ago
The usual excuse for C and C++ security flaws, "This shouldn't have happened: A vulnerability postmortem " https://googleprojectzero.blogspot.com/2021/12/this-shouldnt-have-happened.html https://googleprojectzero.blogspot.com/2021/12/this-shouldnt... Relevant quote, "All of the NSS fuzzers are represented in combined coverage metrics by oss-fuzz, rather than their individual coverage. This data proved misleading, as the vulnerable code is fuzzed extensively but by fuzzers that could not possibly generate a relevant input."
- ajross 3y ago
  > The usual excuse for C and C++ security flaws, Saying "important software should be rigorously tested" isn't an excuse, it's an actionable and reasonable suggestion. Tut-tutting about implementation language choice in (literally) the platform C runtime is quite a bit less so. Grandparent was being helpful, basically. You're just sniping.

by hgs3 3y ago

DougBTX 3y ago

> I'm tired of sloppy testing Exhaustive testing tops out at about two float 32s[0], so of course do that for small functions, but for full application testing type checks are much cheaper. [0] https://randomascii.wordpress.com/2014/01/27/theres-only-four-billion-floatsso-test-them-all/ https://randomascii.wordpress.com/2014/01/27/theres-only-fou...

ajross 3y ago
Fuzzing isn't exhaustive testing, it's dynamic coverage testing. You provide random input until you discover your (compiler-instrumented) binary doing something you haven't seen yet, then permute the input until you get it to do something else, etc... For almost all non-pathological algorithms, it's capable of reaching full coverage in linear time; though some algorithms are always going to have pathological edge and require some whiteboxing.

pjmlp 3y ago

The usual excuse for C and C++ security flaws, "This shouldn't have happened: A vulnerability postmortem " https://googleprojectzero.blogspot.com/2021/12/this-shouldnt-have-happened.html https://googleprojectzero.blogspot.com/2021/12/this-shouldnt... Relevant quote, "All of the NSS fuzzers are represented in combined coverage metrics by oss-fuzz, rather than their individual coverage. This data proved misleading, as the vulnerable code is fuzzed extensively but by fuzzers that could not possibly generate a relevant input."

ajross 3y ago
> The usual excuse for C and C++ security flaws, Saying "important software should be rigorously tested" isn't an excuse, it's an actionable and reasonable suggestion. Tut-tutting about implementation language choice in (literally) the platform C runtime is quite a bit less so. Grandparent was being helpful, basically. You're just sniping.