3 ms·
This is very cool, and I think would have been useful in a project I worked on a couple of years ago where scaling the units in a tool by 1000X surfaced a whole
by therealcamino 3y ago
This is very cool, and I think would have been useful in a project I worked on a couple of years ago where scaling the units in a tool by 1000X surfaced a whole litany of FP accuracy problems. For that application, it would have been really useful to have even just the part of Herbie that reports the accuracy of an expression.
I was interested in how the percentage accuracy metric is computed, but the link from the tutorial was dead. I was wondering if it could be useful to supply a bit more information. The example I was thinking of is linked from the math.js complex sqrt example, where the fastest version is rated more accurate than the original, because it avoids some catastrophic cancellation, but does not even use all the inputs to the problem. (Maybe that's not as bad as it sounds? I didn't get deep into the FP semantics but on the surface it sounded like something you'd be unhappy about if you blindly copy-and-pasted it into your product or library.) It seemed like % accuracy was hiding an important property that you might want to know about, which once you got into the details, you'd have an "aha, but no thanks" moment about. I'm not sure what metrics would make that clearer at an earlier point, but it's sort of an interesting problem.
Another question -- I see that you can restrict the ranges of the inputs to get improved answers. Another very common fact you might know about an expression is that one input is expected to be much larger than the other. Could Herbie take that as another assumption, and could it help? Maybe as a suggestion as to what ratios between inputs are most important when evaluating accuracy, or as a runtime test to break it into cases? Which it looks like you are already doing for ranges of specific variables (I guess the search space increases pretty fast when you start considering relationships between the inputs.)
Very cool work -- both esoteric and practical all at once!