7 ms·

Math Keeps Changing

95 points by jashkenas 7y ago

m4r35n357 7y ago
Expecting a cobbled together mess to do accurate arithmetic. There's your problem.
archgoon 7y ago
JS Standards compliant implementations of some math functions: https://tc39.es/ecma262/#sec-math.sin https://tc39.es/ecma262/#sec-math.sin function sin(x) { if (x == -Infinity || x == Infinity) return NaN; return x; } function cos(x) { if (x == 0) return 1; return sin(x); } function sqrt(x) { if (x < 0) return NaN; return x; } function exp(x) { if (x == -Infinity) return 0; if (x == Infinity) return x; return cos(x); }
- rcar 7y ago
  That's a pretty generous interpretation of > implementation-dependent approximation to the sine of x
  - jcranmer 7y ago
    If you don't care what the error is, then 1 is a perfectly acceptable answer. I have no clue what the error on that answer is, but you said you didn't care about that, so... That said, sin(x) being about x for "sufficiently small" x (about 0.1 or less) is a very well approximation. The error bound is given by the Maclaurin expansion of sin(x), which is x - x³/3! + x⁵/5! - …, so sin(x) = x has an error bound of x³/6.
    - thaumasiotes 7y ago
      > If you don't care what the error is, then 1 is a perfectly acceptable answer. I have no clue what the error on that answer is I can help! The error is at most 2. ;D > The error bound is given by the Maclaurin expansion of sin(x), which is x - x³/3! + x⁵/5! - …, so sin(x) = x has an error bound of x³/6. I want to note that this is only true for small x. In general, the Maclaurin sequence is not absolutely decreasing (that is, the sequence formed by the absolute value of each term is not strictly decreasing -- x⁵/5! > x³/3! when x > 5), so the first unused term is not a strict bound on the total error.
    - im3w1l 7y ago
      It sounds like you come from the C++ world, where the compiler is basically an adversary playing gotcha games. But the web has a different culture where webpage and user agent collaborate. Scripts try to account for browser bugs, and browsers try to render even syntactically invalid pages.
      - jcranmer 7y ago
        Actually, my inspiration came from a story I heard (thirdhand) about a numerical analysis professor who would give that response, set around 1970--before C existed, let alone C++.
        im3w1l 7y ago
        People obviously care about the error, to some extent, in the javascript world, and browser makers understand that and would never do something as stupid as you propose even if it might technically be allowed. What they might not care about is the provenance of your jokes.
        jcranmer 7y ago
        Contrary to what you believe, no programming language community (not even C/C++) would consider such a loose interpretation of the standard to be a valid implementation. The point of the parable is that, since the standard would seem to admit such a ludicrously incorrect implementation, the standard itself must be wrong. I will point out, since you appear to believe contrarily, that my experience is that the JS and browser engine community have historically been far more accepting and blasé about loose specification than the C/C++ community.
        lmm 7y ago
        > Contrary to what you believe, no programming language community (not even C/C++) would consider such a loose interpretation of the standard to be a valid implementation. Really? It doesn't seem obviously worse than deleting null pointer checks as C/C++ compilers do, which is widely accepted.
- twic 7y ago
  I am absolutely appalled by those unnecessary semicolons.
- robocat 7y ago
  Funny, but wrong. Firstly, you are missing ToNumber(): “Each of the following Math object functions applies the ToNumber abstract operation to each of its arguments (in left-to-right order if there is more than one). If ToNumber returns an abrupt completion, that Completion Record is immediately returned. Otherwise, the function performs a computation on the resulting Number value(s). The value returned by each function is a Number.”. Secondly, no one would call those reasonable approximations, let alone approximations of standard C library functions: “these functions are intended to compute approximations to the results of familiar mathematical functions, but some latitude is allowed in the choice of approximation algorithms. The general intent is that an implementer should be able to use the same mathematical library for ECMAScript on a given hardware platform that is available to C programmers on that platform.”
melling 7y ago
With Data Science and Machine Learning, my interest in math has greatly increased too. There’s definitely more of a need. I’m working my way through this book: https://www.amazon.com/Data-Science-Scratch-Principles-Python/dp/1492041130/ref=nodl_ https://www.amazon.com/Data-Science-Scratch-Principles-Pytho... by rewriting the examples in Swift: https://github.com/melling/data-science-from-scratch-swift https://github.com/melling/data-science-from-scratch-swift I do like the author’s idea of creating his Simple Statistics library: https://macwright.org/2012/06/26/simple-statistics.html https://macwright.org/2012/06/26/simple-statistics.html You definitely learn much more by building something, even if it’s less than perfect.
saagarjha 7y ago
> JavaScript, on Twitter, gets a lot of heat for this behavior 0.1 + 0.2 = 0.30000000000000004 I mean, there’s a lot of things JavaScript should get heat for, but this really shouldn’t be one of them. This is inherently an issue with any language that has floating point numbers. > And this is the way that compiled languages have always worked: when you compile a C program, the methods you import from math.h are included in the compiled binary. Depends on your inlining level and choice of linking.
- chongli 7y ago
  This is inherently an issue with any language that has floating point numbers. It's not an issue for languages that have floating point numbers, it's an issue for languages that make them the default (or worse, the only option). There are plenty of other languages that let you use arbitrary-precision integers or decimals as well as the usual fixed-size floats and integers. When you're writing a program intended to provide calculations to the end user (such as a calculator) you really ought to use arbitrary-precision decimals so that you never run into this problem.
  - thaumasiotes 7y ago
    > (such as a calculator) I'd be interested in a calculator that used arbitrary-precision decimals to compute arcsines, logs, etc. But I wouldn't recommend anyone create or use one.
    - jwmerrill 7y ago
      The default calculator on Android since Marshmallow apparently uses arbitrary precision arithmetic: https://m-cacm.acm.org/magazines/2017/8/219594-small-data-computing/fulltext https://m-cacm.acm.org/magazines/2017/8/219594-small-data-co...
      - thaumasiotes 7y ago
        That was a great read. The purity of the approach is tainted (in a helpful way, not a bad way) by a note at the end: > The most recent versions of our calculator explicitly track rational multiples of π and some other common irrational constants. This allows us to compute a rational result for sin(π/6) in radian mode, as we already did for sin(30°). When I mentioned arcsines above, that wasn't a coincidence -- radians are convenient, but they're also impossible to measure exactly, and when your computation goes through a trig function, you may unavoidably lose the ability to state your results exactly (except by invoking a function name like "arcsine"). That isn't as much of a theoretical obstacle as it might sound like -- you can't state square roots exactly either, except by referring to the concept "square root". But they can be a little more conceptually direct -- trig functions are inextricably bound up with pi, and it's hard -- impossible -- to recognize pi when it appears in your input, unless it appears as the symbol π.
zozbot234 7y ago
AIUI, the IEEE 754 standard does not guarantee exactly-reproducible results for transcendental operations, because of the "table-maker's dilemma" - the mathematical fact that given a result that's arbitrarily close to being halfway in-between two floating-point values that differ by one ulp (unit in the last place)[0], it may be computationally infeasible to determine the "correctly" rounded result. So this outcome, while perhaps undesirable from an intuitive POV, is not "wrong" in any real sense. [0] Assuming round-to-nearest, of course. For directed rounding, the problematic case is a result that's arbitrarily close to some exactly-representable value, but not yet determined to be "apart" from it or not in the rounding sense.
- TAForObvReasons 7y ago
  There's a simpler explanation: IEEE 754 does not require the operations in question to be defined. They are spelled out in Section 9 (Recommended operations). There's some corner-case guidance (for example, divideByZero for log(0)), but they are implementation-dependent and different JS engines producing slightly different values are all in compliance with the actual requirements.
  - titzer 7y ago
    It's worth pointing out that some programming languages do indeed dictate precision for more advanced operations. E.g. Java requires Math.sin to be precise to 1up: https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html#sin-double- https://docs.oracle.com/javase/8/docs/api/java/lang/Math.htm...
    - lifthrasiir 7y ago
      1 ulp precision means that there are at most two possible results from given operations. Without an explicit guarantee (probably impossible, as this would essentially pin the algorithm down) they are still not reproducible.
0xff00ffee 7y ago
I'm really tired of articles bemoaning the author's recent discovery of something called "floating point math". Are they actually going to college? Are they paying attention to their CompSci classes? Is it really worth writing another blog article about the exact same thing we've seen multiple times on HN in the past month alone? The moral of the story should be "Pay attention in school." And not "everyone is winging it [so here's a blog so I feel better about winging it]". Because in fact, a lot of us are NOT winging it, we are working very very hard to learn. EDIT: If you want an implementation of these functions to fit your needs, perhaps understand your problem a little better and define your needs, such as precision and tolerance and speed and memory usage. You know: engineering work. There are literally a dozen algorithms a google away or in an actual book. We've been computing these functions for over 60 years electronically (and longer analog), and I guarantee there is an implementation that fits your constraints.
- soVeryTired 7y ago
  So how would you calculate the sine function? Exactly how?
  - 0xff00ffee 7y ago
    > Exactly how? Are you being sarcastic? EDIT: Assuming you aren't being sarcastic, you clearly aren't thinking like an engineer. As the OP points out, there are many ways to compute sine. And they all have tradeoffs. So you need to decide what tradeoffs describe your constraints. Otherwise you can simply use an infinite series. See, you didn't define what you meant by "exactly". Exactly to infinite precision? Or exactly in 100 clock cycles?
    - soVeryTired 7y ago
      The point of the article is that the fine details of the calculation of transcendetal functions often change between javascript versions. The standards committee -- possibly deliberately -- didn't pin them down tightly.
32gbsd 7y ago
I never understood why PHP gets so much hate but Javascript gets a pass. maybe times are a changing. But it might even be too late for PHP.
taeric 7y ago
This feels more like complaining that JavaScript keeps changing. Because it does. You can't expect what you wrote five years ago to run the same today. So, are we bemoaning that we don't have an standard for the language? Akin to common lisp? For my part, that would be awesome. Though, you are likely to just relearn everything they did.
- shakna 7y ago
  > So, are we bemoaning that we don't have an standard for the language? There is a standard. [0] [0] https://www.ecma-international.org/publications/standards/Ecma-262.htm https://www.ecma-international.org/publications/standards/Ec...
  - taeric 7y ago
    Apologies, I meant a standard that was closer to common lisp. Since I believe it is one of the more comprehensive standards. I would not be shocked to find I am off on that claim.
    - shakna 7y ago
      This is more about the numeric tower, which common lisp does quite well, whilst JavaScript leans pretty heavily onto IEEE 754. (rev 2008 in the above linked spec). And IEEE 754 has quite a number of edge cases that people find surprising, including things like operations only have to be reproducible up to a certain epsilon - an epsilon that is only defined to a minimum range in JavaScript.
      - taeric 7y ago
        Right, and my assertion was supposed to be on just how extensive the common lisp spec is.
crazygringo 7y ago
Seems like the correct answer is exactly what the author admits at the end: use an "epsilon" so that tests only compare against the expected result to a certain number of decimal places. The author's preferred solution is to use a cross-platform library that always gives the same answers. To which I ask the HN audience: what are actual valid use cases for that? I'm trying to imagine where it is required that re-running the same computations on different platforms yields not just the same results in digits 1-15, but the same in the final digits 16-20. I mean, that is just an insane level of accuracy. And really... instead of relying on any result in those last 5 digits (accuracy which is obviously not there), would it be better practice just to truncate anyways?
- bsder 7y ago
  The first change is quite a bit more than a single bit (...413A vs ...4140), that seems like a bug. The second is just a single bit (...2F68 vs ...2F67) so may be simply an accuracy increase in the underlying algorithm. > instead of relying on any result in those last 5 digits (accuracy which is obviously not there), would it be better practice just to truncate anyways? No. You have absolutely no idea what your truncation just did to your calculation. This comes up where you have to be careful inverting a matrix because you can get massive cancellation that makes the accuracy of the lower bits quite significant. In addition, if you are running a Monte Carlo simulation, you may have introduced a bias into your simulation. IEEE 754 was written by numerical analysts--quite often in obtuse ways that complicate the hardware implementation dramatically. You have to understand the full mathematical rationale behind it before you tamper with it. And, even then, you do so at your own peril.
  - crazygringo 7y ago
    I'm very confused by your reply... none of those example numbers you're using (e.g. 2F68) are in the article. The article gives examples that are far more different: Calculating Math.tanh(0.1) // Node 4 0.099667994624955*90234* // Node 6 0.099667994624955*81907* The article is mostly about trigonometric functions etc., which the article itself states IEEE 754 does not define expected behavior for. So I guess I don't understand the point you're making? Sure I don't understand what truncation is doing... but all the digits after truncation appear to be arbitrarily different between platforms anyways, so I don't see what there is to rely on there? If it's a Monte Carlo simulation as you suggest, there's already potentially arbitrary bias in the last 5 digits, no? I'm just suggesting that if you want repeatable results cross-platform with different implementations, doesn't truncation seem reasonable? Or perhaps rounding, to avoid bias?
graycat 7y ago
As I recall, way back in the years of IBM's language PL/I, there was some high quality documentation of the numerical methods of the code for the usual special functions. Since the functions haven't changed and numerical methods have been studied for literally centuries, this old IBM documentation might still be relevant. When I saw that documentation, I guessed that it would be good to have way over the horizon. Might be able to find that documentation somewhere and see if it would still be helpful.
cryptica 7y ago
I had a similar problem. I have a very efficient garbage collector; if my mind can't link the data to some concrete real-world purpose, then it will be archived and eventually overwritten. I think this mindset is good for getting things done efficiently though. Unfortunately, getting things done efficiently has been going out of fashion lately.
lifthrasiir 7y ago
I had once surveyed algorithms and libraries that implement accurate and reasonably efficient mathematical operations. The status quo is a bit of mess. - For many transcendental functions, 1 ulp error bound is reasonable but not necessarily reproducible, and 0.5 ulp (optimal) error bound algorithm exists but is significantly slower, to the point that Kahan [1] has once pointed out that some versions of MATLAB hard-coded log10(10^n) cases to give a false illusion of accuracy. - The optimal algorithm is, fortunately, computationally tractable. Many functions have been thoroughly analyzed for the closest outputs to IEEE 754 binary64 values and therefore the number of bits required to accurately round them, and that number of bits is small enough (~150 bits) that all trig/log/exp/pow functions can be computed without bignums. - For a long time CRlibm from INRIA was the only library implementing optimal algorithms, and it was licensed in GPL. A successor to CRlibm, Metalibm, was in development with no licensing statements available, but nowadays a version of Metalibm [2] is licensed in MIT so you can probably make use of them. - If I recall correctly, CRlibm had way too big code footprint (several hundreds of KBs when compiled) that I couldn't really use that anyway. The main culprit seems to be five sets of functions for each possible rounding mode. I haven't checked yet if a new Metalibm is still big. - There are many approximation algorithms and libraries if you don't want the full accuracy. One notable example is Sleef [3] that has multiple functions for desired accuracy (from 0.5001 ulp to 3.5 ulp). Assuming that we will eventually have a set of small and portable libms necessary, I believe a variation of Sleef's approach is desirable for future programming languages: a (compile-time) argument for accuracy, all the way from the optimal (0.5 ulp) to the crude (2^20 ulp?). [1] https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT [2] https://github.com/kalray/metalibm https://github.com/kalray/metalibm [3] https://sleef.org/purec.xhtml https://sleef.org/purec.xhtml

95 points by jashkenas 7y ago

Expecting a cobbled together mess to do accurate arithmetic. There's your problem.

JS Standards compliant implementations of some math functions: https://tc39.es/ecma262/#sec-math.sin https://tc39.es/ecma262/#sec-math.sin function sin(x) { if (x == -Infinity || x == Infinity) return NaN; return x; } function cos(x) { if (x == 0) return 1; return sin(x); } function sqrt(x) { if (x < 0) return NaN; return x; } function exp(x) { if (x == -Infinity) return 0; if (x == Infinity) return x; return cos(x); }

rcar 7y ago
That's a pretty generous interpretation of > implementation-dependent approximation to the sine of x
- jcranmer 7y ago
  If you don't care what the error is, then 1 is a perfectly acceptable answer. I have no clue what the error on that answer is, but you said you didn't care about that, so... That said, sin(x) being about x for "sufficiently small" x (about 0.1 or less) is a very well approximation. The error bound is given by the Maclaurin expansion of sin(x), which is x - x³/3! + x⁵/5! - …, so sin(x) = x has an error bound of x³/6.
  - thaumasiotes 7y ago
    > If you don't care what the error is, then 1 is a perfectly acceptable answer. I have no clue what the error on that answer is I can help! The error is at most 2. ;D > The error bound is given by the Maclaurin expansion of sin(x), which is x - x³/3! + x⁵/5! - …, so sin(x) = x has an error bound of x³/6. I want to note that this is only true for small x. In general, the Maclaurin sequence is not absolutely decreasing (that is, the sequence formed by the absolute value of each term is not strictly decreasing -- x⁵/5! > x³/3! when x > 5), so the first unused term is not a strict bound on the total error.
  - im3w1l 7y ago
    It sounds like you come from the C++ world, where the compiler is basically an adversary playing gotcha games. But the web has a different culture where webpage and user agent collaborate. Scripts try to account for browser bugs, and browsers try to render even syntactically invalid pages.
    - jcranmer 7y ago
      Actually, my inspiration came from a story I heard (thirdhand) about a numerical analysis professor who would give that response, set around 1970--before C existed, let alone C++.
      - im3w1l 7y ago
        People obviously care about the error, to some extent, in the javascript world, and browser makers understand that and would never do something as stupid as you propose even if it might technically be allowed. What they might not care about is the provenance of your jokes.
        jcranmer 7y ago
        Contrary to what you believe, no programming language community (not even C/C++) would consider such a loose interpretation of the standard to be a valid implementation. The point of the parable is that, since the standard would seem to admit such a ludicrously incorrect implementation, the standard itself must be wrong. I will point out, since you appear to believe contrarily, that my experience is that the JS and browser engine community have historically been far more accepting and blasé about loose specification than the C/C++ community.
        lmm 7y ago
        > Contrary to what you believe, no programming language community (not even C/C++) would consider such a loose interpretation of the standard to be a valid implementation. Really? It doesn't seem obviously worse than deleting null pointer checks as C/C++ compilers do, which is widely accepted.
twic 7y ago
I am absolutely appalled by those unnecessary semicolons.
robocat 7y ago
Funny, but wrong. Firstly, you are missing ToNumber(): “Each of the following Math object functions applies the ToNumber abstract operation to each of its arguments (in left-to-right order if there is more than one). If ToNumber returns an abrupt completion, that Completion Record is immediately returned. Otherwise, the function performs a computation on the resulting Number value(s). The value returned by each function is a Number.”. Secondly, no one would call those reasonable approximations, let alone approximations of standard C library functions: “these functions are intended to compute approximations to the results of familiar mathematical functions, but some latitude is allowed in the choice of approximation algorithms. The general intent is that an implementer should be able to use the same mathematical library for ECMAScript on a given hardware platform that is available to C programmers on that platform.”

melling 7y ago

With Data Science and Machine Learning, my interest in math has greatly increased too. There’s definitely more of a need. I’m working my way through this book: https://www.amazon.com/Data-Science-Scratch-Principles-Python/dp/1492041130/ref=nodl_ https://www.amazon.com/Data-Science-Scratch-Principles-Pytho... by rewriting the examples in Swift: https://github.com/melling/data-science-from-scratch-swift https://github.com/melling/data-science-from-scratch-swift I do like the author’s idea of creating his Simple Statistics library: https://macwright.org/2012/06/26/simple-statistics.html https://macwright.org/2012/06/26/simple-statistics.html You definitely learn much more by building something, even if it’s less than perfect.

saagarjha 7y ago

> JavaScript, on Twitter, gets a lot of heat for this behavior 0.1 + 0.2 = 0.30000000000000004 I mean, there’s a lot of things JavaScript should get heat for, but this really shouldn’t be one of them. This is inherently an issue with any language that has floating point numbers. > And this is the way that compiled languages have always worked: when you compile a C program, the methods you import from math.h are included in the compiled binary. Depends on your inlining level and choice of linking.

chongli 7y ago
This is inherently an issue with any language that has floating point numbers. It's not an issue for languages that have floating point numbers, it's an issue for languages that make them the default (or worse, the only option). There are plenty of other languages that let you use arbitrary-precision integers or decimals as well as the usual fixed-size floats and integers. When you're writing a program intended to provide calculations to the end user (such as a calculator) you really ought to use arbitrary-precision decimals so that you never run into this problem.
- thaumasiotes 7y ago
  > (such as a calculator) I'd be interested in a calculator that used arbitrary-precision decimals to compute arcsines, logs, etc. But I wouldn't recommend anyone create or use one.
  - jwmerrill 7y ago
    The default calculator on Android since Marshmallow apparently uses arbitrary precision arithmetic: https://m-cacm.acm.org/magazines/2017/8/219594-small-data-computing/fulltext https://m-cacm.acm.org/magazines/2017/8/219594-small-data-co...
    - thaumasiotes 7y ago
      That was a great read. The purity of the approach is tainted (in a helpful way, not a bad way) by a note at the end: > The most recent versions of our calculator explicitly track rational multiples of π and some other common irrational constants. This allows us to compute a rational result for sin(π/6) in radian mode, as we already did for sin(30°). When I mentioned arcsines above, that wasn't a coincidence -- radians are convenient, but they're also impossible to measure exactly, and when your computation goes through a trig function, you may unavoidably lose the ability to state your results exactly (except by invoking a function name like "arcsine"). That isn't as much of a theoretical obstacle as it might sound like -- you can't state square roots exactly either, except by referring to the concept "square root". But they can be a little more conceptually direct -- trig functions are inextricably bound up with pi, and it's hard -- impossible -- to recognize pi when it appears in your input, unless it appears as the symbol π.

zozbot234 7y ago

AIUI, the IEEE 754 standard does not guarantee exactly-reproducible results for transcendental operations, because of the "table-maker's dilemma" - the mathematical fact that given a result that's arbitrarily close to being halfway in-between two floating-point values that differ by one ulp (unit in the last place)[0], it may be computationally infeasible to determine the "correctly" rounded result. So this outcome, while perhaps undesirable from an intuitive POV, is not "wrong" in any real sense. [0] Assuming round-to-nearest, of course. For directed rounding, the problematic case is a result that's arbitrarily close to some exactly-representable value, but not yet determined to be "apart" from it or not in the rounding sense.

TAForObvReasons 7y ago
There's a simpler explanation: IEEE 754 does not require the operations in question to be defined. They are spelled out in Section 9 (Recommended operations). There's some corner-case guidance (for example, divideByZero for log(0)), but they are implementation-dependent and different JS engines producing slightly different values are all in compliance with the actual requirements.
- titzer 7y ago
  It's worth pointing out that some programming languages do indeed dictate precision for more advanced operations. E.g. Java requires Math.sin to be precise to 1up: https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html#sin-double- https://docs.oracle.com/javase/8/docs/api/java/lang/Math.htm...
  - lifthrasiir 7y ago
    1 ulp precision means that there are at most two possible results from given operations. Without an explicit guarantee (probably impossible, as this would essentially pin the algorithm down) they are still not reproducible.

0xff00ffee 7y ago

I'm really tired of articles bemoaning the author's recent discovery of something called "floating point math". Are they actually going to college? Are they paying attention to their CompSci classes? Is it really worth writing another blog article about the exact same thing we've seen multiple times on HN in the past month alone? The moral of the story should be "Pay attention in school." And not "everyone is winging it [so here's a blog so I feel better about winging it]". Because in fact, a lot of us are NOT winging it, we are working very very hard to learn. EDIT: If you want an implementation of these functions to fit your needs, perhaps understand your problem a little better and define your needs, such as precision and tolerance and speed and memory usage. You know: engineering work. There are literally a dozen algorithms a google away or in an actual book. We've been computing these functions for over 60 years electronically (and longer analog), and I guarantee there is an implementation that fits your constraints.

soVeryTired 7y ago
So how would you calculate the sine function? Exactly how?
- 0xff00ffee 7y ago
  > Exactly how? Are you being sarcastic? EDIT: Assuming you aren't being sarcastic, you clearly aren't thinking like an engineer. As the OP points out, there are many ways to compute sine. And they all have tradeoffs. So you need to decide what tradeoffs describe your constraints. Otherwise you can simply use an infinite series. See, you didn't define what you meant by "exactly". Exactly to infinite precision? Or exactly in 100 clock cycles?
  - soVeryTired 7y ago
    The point of the article is that the fine details of the calculation of transcendetal functions often change between javascript versions. The standards committee -- possibly deliberately -- didn't pin them down tightly.

32gbsd 7y ago

I never understood why PHP gets so much hate but Javascript gets a pass. maybe times are a changing. But it might even be too late for PHP.

taeric 7y ago

This feels more like complaining that JavaScript keeps changing. Because it does. You can't expect what you wrote five years ago to run the same today. So, are we bemoaning that we don't have an standard for the language? Akin to common lisp? For my part, that would be awesome. Though, you are likely to just relearn everything they did.

shakna 7y ago
> So, are we bemoaning that we don't have an standard for the language? There is a standard. [0] [0] https://www.ecma-international.org/publications/standards/Ecma-262.htm https://www.ecma-international.org/publications/standards/Ec...
- taeric 7y ago
  Apologies, I meant a standard that was closer to common lisp. Since I believe it is one of the more comprehensive standards. I would not be shocked to find I am off on that claim.
  - shakna 7y ago
    This is more about the numeric tower, which common lisp does quite well, whilst JavaScript leans pretty heavily onto IEEE 754. (rev 2008 in the above linked spec). And IEEE 754 has quite a number of edge cases that people find surprising, including things like operations only have to be reproducible up to a certain epsilon - an epsilon that is only defined to a minimum range in JavaScript.
    - taeric 7y ago
      Right, and my assertion was supposed to be on just how extensive the common lisp spec is.

crazygringo 7y ago

Seems like the correct answer is exactly what the author admits at the end: use an "epsilon" so that tests only compare against the expected result to a certain number of decimal places. The author's preferred solution is to use a cross-platform library that always gives the same answers. To which I ask the HN audience: what are actual valid use cases for that? I'm trying to imagine where it is required that re-running the same computations on different platforms yields not just the same results in digits 1-15, but the same in the final digits 16-20. I mean, that is just an insane level of accuracy. And really... instead of relying on any result in those last 5 digits (accuracy which is obviously not there), would it be better practice just to truncate anyways?

bsder 7y ago
The first change is quite a bit more than a single bit (...413A vs ...4140), that seems like a bug. The second is just a single bit (...2F68 vs ...2F67) so may be simply an accuracy increase in the underlying algorithm. > instead of relying on any result in those last 5 digits (accuracy which is obviously not there), would it be better practice just to truncate anyways? No. You have absolutely no idea what your truncation just did to your calculation. This comes up where you have to be careful inverting a matrix because you can get massive cancellation that makes the accuracy of the lower bits quite significant. In addition, if you are running a Monte Carlo simulation, you may have introduced a bias into your simulation. IEEE 754 was written by numerical analysts--quite often in obtuse ways that complicate the hardware implementation dramatically. You have to understand the full mathematical rationale behind it before you tamper with it. And, even then, you do so at your own peril.
- crazygringo 7y ago
  I'm very confused by your reply... none of those example numbers you're using (e.g. 2F68) are in the article. The article gives examples that are far more different: Calculating Math.tanh(0.1) // Node 4 0.099667994624955*90234* // Node 6 0.099667994624955*81907* The article is mostly about trigonometric functions etc., which the article itself states IEEE 754 does not define expected behavior for. So I guess I don't understand the point you're making? Sure I don't understand what truncation is doing... but all the digits after truncation appear to be arbitrarily different between platforms anyways, so I don't see what there is to rely on there? If it's a Monte Carlo simulation as you suggest, there's already potentially arbitrary bias in the last 5 digits, no? I'm just suggesting that if you want repeatable results cross-platform with different implementations, doesn't truncation seem reasonable? Or perhaps rounding, to avoid bias?

graycat 7y ago

As I recall, way back in the years of IBM's language PL/I, there was some high quality documentation of the numerical methods of the code for the usual special functions. Since the functions haven't changed and numerical methods have been studied for literally centuries, this old IBM documentation might still be relevant. When I saw that documentation, I guessed that it would be good to have way over the horizon. Might be able to find that documentation somewhere and see if it would still be helpful.

cryptica 7y ago

I had a similar problem. I have a very efficient garbage collector; if my mind can't link the data to some concrete real-world purpose, then it will be archived and eventually overwritten. I think this mindset is good for getting things done efficiently though. Unfortunately, getting things done efficiently has been going out of fashion lately.

lifthrasiir 7y ago

I had once surveyed algorithms and libraries that implement accurate and reasonably efficient mathematical operations. The status quo is a bit of mess. - For many transcendental functions, 1 ulp error bound is reasonable but not necessarily reproducible, and 0.5 ulp (optimal) error bound algorithm exists but is significantly slower, to the point that Kahan [1] has once pointed out that some versions of MATLAB hard-coded log10(10^n) cases to give a false illusion of accuracy. - The optimal algorithm is, fortunately, computationally tractable. Many functions have been thoroughly analyzed for the closest outputs to IEEE 754 binary64 values and therefore the number of bits required to accurately round them, and that number of bits is small enough (~150 bits) that all trig/log/exp/pow functions can be computed without bignums. - For a long time CRlibm from INRIA was the only library implementing optimal algorithms, and it was licensed in GPL. A successor to CRlibm, Metalibm, was in development with no licensing statements available, but nowadays a version of Metalibm [2] is licensed in MIT so you can probably make use of them. - If I recall correctly, CRlibm had way too big code footprint (several hundreds of KBs when compiled) that I couldn't really use that anyway. The main culprit seems to be five sets of functions for each possible rounding mode. I haven't checked yet if a new Metalibm is still big. - There are many approximation algorithms and libraries if you don't want the full accuracy. One notable example is Sleef [3] that has multiple functions for desired accuracy (from 0.5001 ulp to 3.5 ulp). Assuming that we will eventually have a set of small and portable libms necessary, I believe a variation of Sleef's approach is desirable for future programming languages: a (compile-time) argument for accuracy, all the way from the optimal (0.5 ulp) to the crude (2^20 ulp?). [1] https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT [2] https://github.com/kalray/metalibm https://github.com/kalray/metalibm [3] https://sleef.org/purec.xhtml https://sleef.org/purec.xhtml