3 ms·

Hey! Some people in my lab work on that! I also did some work on analyzing floating point exceptions, so I learned a bit about this stuff. AMA, and I’ll do my b

Hey! Some people in my lab work on that! I also did some work on analyzing floating point exceptions, so I learned a bit about this stuff. AMA, and I’ll do my best to answer. :) (For clarification, I work at the University of Utah, so Pavel is the head honcho over Herbie here.)

vanderZwan 3y ago
Can Herbie also look at expressions that are part of summing floating point numbers, or similar operations, to minimize the error of doing so?
- lifthrasiir 3y ago
  I think Herbie only works on a closed-form expression. Besides from that though, if you are indeed summing numbers there should be some library to do that accurately, for example C++ Summation Toolkit [1] or the `accurate` crate in Rust [2]. [1] https://www.partow.net/programming/sumtk/index.html https://www.partow.net/programming/sumtk/index.html [2] https://docs.rs/accurate/latest/accurate/ https://docs.rs/accurate/latest/accurate/
therealcamino 3y ago
This is very cool, and I think would have been useful in a project I worked on a couple of years ago where scaling the units in a tool by 1000X surfaced a whole litany of FP accuracy problems. For that application, it would have been really useful to have even just the part of Herbie that reports the accuracy of an expression. I was interested in how the percentage accuracy metric is computed, but the link from the tutorial was dead. I was wondering if it could be useful to supply a bit more information. The example I was thinking of is linked from the math.js complex sqrt example, where the fastest version is rated more accurate than the original, because it avoids some catastrophic cancellation, but does not even use all the inputs to the problem. (Maybe that's not as bad as it sounds? I didn't get deep into the FP semantics but on the surface it sounded like something you'd be unhappy about if you blindly copy-and-pasted it into your product or library.) It seemed like % accuracy was hiding an important property that you might want to know about, which once you got into the details, you'd have an "aha, but no thanks" moment about. I'm not sure what metrics would make that clearer at an earlier point, but it's sort of an interesting problem. Another question -- I see that you can restrict the ranges of the inputs to get improved answers. Another very common fact you might know about an expression is that one input is expected to be much larger than the other. Could Herbie take that as another assumption, and could it help? Maybe as a suggestion as to what ratios between inputs are most important when evaluating accuracy, or as a runtime test to break it into cases? Which it looks like you are already doing for ranges of specific variables (I guess the search space increases pretty fast when you start considering relationships between the inputs.) Very cool work -- both esoteric and practical all at once!

vanderZwan 3y ago

Can Herbie also look at expressions that are part of summing floating point numbers, or similar operations, to minimize the error of doing so?

lifthrasiir 3y ago
I think Herbie only works on a closed-form expression. Besides from that though, if you are indeed summing numbers there should be some library to do that accurately, for example C++ Summation Toolkit [1] or the `accurate` crate in Rust [2]. [1] https://www.partow.net/programming/sumtk/index.html https://www.partow.net/programming/sumtk/index.html [2] https://docs.rs/accurate/latest/accurate/ https://docs.rs/accurate/latest/accurate/

therealcamino 3y ago

This is very cool, and I think would have been useful in a project I worked on a couple of years ago where scaling the units in a tool by 1000X surfaced a whole litany of FP accuracy problems. For that application, it would have been really useful to have even just the part of Herbie that reports the accuracy of an expression. I was interested in how the percentage accuracy metric is computed, but the link from the tutorial was dead. I was wondering if it could be useful to supply a bit more information. The example I was thinking of is linked from the math.js complex sqrt example, where the fastest version is rated more accurate than the original, because it avoids some catastrophic cancellation, but does not even use all the inputs to the problem. (Maybe that's not as bad as it sounds? I didn't get deep into the FP semantics but on the surface it sounded like something you'd be unhappy about if you blindly copy-and-pasted it into your product or library.) It seemed like % accuracy was hiding an important property that you might want to know about, which once you got into the details, you'd have an "aha, but no thanks" moment about. I'm not sure what metrics would make that clearer at an earlier point, but it's sort of an interesting problem. Another question -- I see that you can restrict the ranges of the inputs to get improved answers. Another very common fact you might know about an expression is that one input is expected to be much larger than the other. Could Herbie take that as another assumption, and could it help? Maybe as a suggestion as to what ratios between inputs are most important when evaluating accuracy, or as a runtime test to break it into cases? Which it looks like you are already doing for ranges of specific variables (I guess the search space increases pretty fast when you start considering relationships between the inputs.) Very cool work -- both esoteric and practical all at once!