2 ms·
The first change is quite a bit more than a single bit (...413A vs ...4140), that seems like a bug. The second is just a single bit (...2F68 vs ...2F67) so may
by bsder 7y ago
The first change is quite a bit more than a single bit (...413A vs ...4140), that seems like a bug.
The second is just a single bit (...2F68 vs ...2F67) so may be simply an accuracy increase in the underlying algorithm.
> instead of relying on any result in those last 5 digits (accuracy which is obviously not there), would it be better practice just to truncate anyways?
No. You have absolutely no idea what your truncation just did to your calculation.
This comes up where you have to be careful inverting a matrix because you can get massive cancellation that makes the accuracy of the lower bits quite significant.
In addition, if you are running a Monte Carlo simulation, you may have introduced a bias into your simulation.
IEEE 754 was written by numerical analysts--quite often in obtuse ways that complicate the hardware implementation dramatically. You have to understand the full mathematical rationale behind it before you tamper with it. And, even then, you do so at your own peril.
- crazygringo 7y agoI'm very confused by your reply... none of those example numbers you're using (e.g. 2F68) are in the article. The article gives examples that are far more different: Calculating Math.tanh(0.1) // Node 4 0.099667994624955*90234* // Node 6 0.099667994624955*81907* The article is mostly about trigonometric functions etc., which the article itself states IEEE 754 does not define expected behavior for. So I guess I don't understand the point you're making? Sure I don't understand what truncation is doing... but all the digits after truncation appear to be arbitrarily different between platforms anyways, so I don't see what there is to rely on there? If it's a Monte Carlo simulation as you suggest, there's already potentially arbitrary bias in the last 5 digits, no? I'm just suggesting that if you want repeatable results cross-platform with different implementations, doesn't truncation seem reasonable? Or perhaps rounding, to avoid bias?
- zamadatix 7y agoThe numbers they gave were the hex representation e.g. `(Math.tanh(0.1)).toString(16)` which allows compact binary comparison. The arbitrary bias you'd find is probably less than you'd find by rounding (truncating or otherwise). The problem with truncation to make things repeatable cross-platform is it assumes "cross-platform" to be "any platform where the Nth to last digit is correct" when the standard explicitly makes no such guarantee about the Nth to last digit.
- bsder 7y agoSorry, those are the hexadecimal representation of the underlying IEEE-754 binary floating point number: 0x3FB983D7795F_4140 == 0.09966799462495590234 0x3FB983D7795F_413A == 0.09966799462495581907 0x3FA2F684BDA1_2F68 == 0.03703703703703703498 0x3FA2F684BDA1_2F67 == 0.03703703703703702804 > but all the digits after truncation appear to be arbitrarily different between platforms anyways Not random at all. If you follow a properly characterized algorithm and execute it properly on IEEE-754 hardware, the result will be the same as if you carried it out on infinite-precision hardware unless you get unlucky and hit a Table Maker's Dilemma instance. And, in that instance, the only difference between your algorithm and a slightly different algorithm should be 1ULP (unit in the last place). 5 ULPs is a bug or a garbage library. Both are possible when Javascript is involved. > I'm just suggesting that if you want repeatable results cross-platform with different implementations, doesn't truncation seem reasonable? Or perhaps rounding, to avoid bias? Yes. We call that IEEE-754--it has round-to-nearest-even and a guard bit. Note by William Kahan from Berkeley about exponential functions and keeping the accuracy to about 1ULP. "A Logarithm Too Clever by Half" https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT
- jcranmer 7y ago> 5 ULPs is a bug or a garbage library. I wouldn't say that. There is definitely a trade-off between accuracy and speed, and that trade-off varies for different mathematical functions. And it's not unreasonable for some people to say "I can tolerate losing 10 ULPs if it means the code will run twice as fast." None of the current math library implementations do a good job of letting users explore those trade-offs, and many of them don't even bother to give a claimed accuracy result. (Note that Sun's [1] java.lang.Math does give ULP requirements on math functions, and Math.tanh has a 2.5 ULP accuracy requirement--atan2, sinh, cost, and tanh are the only functions with a >1 ULP accuracy bound). [1] And yes, I'm calling out Sun here and not Oracle, because it was Sun who designed the math library and reported the error bounds, and the standard math library that people use when they're not shelling out to libm is Sun's fdlibm implementation anyways.
- crazygringo 7y agoThanks for the extra info and clarification -- I understand what you're saying a lot better now. Very interesting!