3 ms·

Does anyone have code to reduce the search space by a few orders of magnitude to test functions with a float64 argument or two or three float32 arguments in a f

Does anyone have code to reduce the search space by a few orders of magnitude to test functions with a float64 argument or two or three float32 arguments in a few seconds? Perhaps by skipping "boring" floats that aren't near endpoints that are unlikely to produce errors that more interesting floats do?

TheRealPomax 6y ago
all floats are equally boring.
- vortico 6y ago
  No, -0.f is more interesting than 4.1045857301e23f. It's more interesting because it's more likely to fail a test where others don't.
  - TheRealPomax 6y ago
    That's a bold assumption: let's make sure it's correct by testing against all floats, instead of only the ones you consider interesting, in the process making any float as boring as any other float again.
daveFNbuck 6y ago
That's essentially what fuzzy testing libraries do. You tell the library what types of arguments you want, write your test, and it puts in some known corner cases and a few random values that change each time you run the test (but are saved as new corner cases when the test fails in the better libraries).
rigtorp 6y ago
Yes, I've done this using libFuzzer. Supply custom mutation and crossover functions that generates interesting floating point numbers. Then let the coverage guided fuzzer exercise your code. https://rigtorp.se/fuzzing-floating-point-code/ https://rigtorp.se/fuzzing-floating-point-code/
ramshorns 6y ago
Well, if the ceil function failed on 1.0f, maybe even the ordinary numbers are worth testing. But... would randomness help? Take a representative sample of like 1/64th of the double space and you might run into most of the common problems.
recursivecaveat 6y ago
You can't naively do it with the bits of a float, but for most purposes N-wise testing will get you a lot of coverage relative to random or manual cases for little expense. I've had success with it using a library to generate cases testing math kernels with many parameters. https://en.m.wikipedia.org/wiki/All-pairs_testing https://en.m.wikipedia.org/wiki/All-pairs_testing

TheRealPomax 6y ago

all floats are equally boring.

vortico 6y ago
No, -0.f is more interesting than 4.1045857301e23f. It's more interesting because it's more likely to fail a test where others don't.
- TheRealPomax 6y ago
  That's a bold assumption: let's make sure it's correct by testing against all floats, instead of only the ones you consider interesting, in the process making any float as boring as any other float again.

daveFNbuck 6y ago

That's essentially what fuzzy testing libraries do. You tell the library what types of arguments you want, write your test, and it puts in some known corner cases and a few random values that change each time you run the test (but are saved as new corner cases when the test fails in the better libraries).

rigtorp 6y ago

Yes, I've done this using libFuzzer. Supply custom mutation and crossover functions that generates interesting floating point numbers. Then let the coverage guided fuzzer exercise your code. https://rigtorp.se/fuzzing-floating-point-code/ https://rigtorp.se/fuzzing-floating-point-code/

ramshorns 6y ago

Well, if the ceil function failed on 1.0f, maybe even the ordinary numbers are worth testing. But... would randomness help? Take a representative sample of like 1/64th of the double space and you might run into most of the common problems.

recursivecaveat 6y ago

You can't naively do it with the bits of a float, but for most purposes N-wise testing will get you a lot of coverage relative to random or manual cases for little expense. I've had success with it using a library to generate cases testing math kernels with many parameters. https://en.m.wikipedia.org/wiki/All-pairs_testing https://en.m.wikipedia.org/wiki/All-pairs_testing