5 ms·
The problems were not specific to Java, they also exist in most modern programming environments, including FORTRAN and ANSI C. Java just happened to be the hott
by segfaultbuserr 3y ago
The problems were not specific to Java, they also exist in most modern programming environments, including FORTRAN and ANSI C. Java just happened to be the hottest language in the news in the late 90s, so the author decided to specifically target Java - "hurts everyone everywhere" was a wordplay of Java's advertisement.
The main author of this report, W. Kahan, was also the original author of IEEE-754. He was (and is) strongly unsatisfied on the state of floating-point in practical systems, and it was not his first time to complain about these problems. One can find his most recent critique from a few years ago.
My understanding is that, Kahan's IEEE-754 is meant to:
1. The use of double and extended precision should be encouraged to safeguard non-experts from floating-point errors. Everything should be at least double precision by default, and extended precision serves as an additional safeguard. In fact, when IEEE-754 was being drafted, Kahan believed 128-bit floating point should also be supported as computers become more powerful in the future. The computational cost was too high at that time, so he settled on 80-bit as seen on the 8087. He criticized Java for not supporting it.
2. Floating-point exceptions should be used and turned on everywhere to safeguard programmers from making mistakes, and possibly to allow programs to handle them as special cases in the logic at runtime. He criticized Java for not supporting it.
3. Unsafe optimizations should not be done, such as automatically using FMA, or using algebraic identity in compiler optimization. He criticized Java for allowing it in some cases.
Unfortunately, as far as I can see, these ideals of IEEE-754 have all but largely disappeared in real-world applications since then, for various practical reasons.
The industry did not move to 128-bit floating point because its performance overhead is too large, the original assessment of IEEE-754 in the 1980s was too optimistic. Similarly, the industry did not accept 80-bit extended precision as a standard but saw it as an oddity of the Intel 8087, even Intel has abandoned it - x86_64's SSE or AVX has removed all support of that, making it an exclusive feature of obsolete i386/i686 machines. My impression is that anything above double precision is no longer used in the industry (IBM POWER does have native quad-precision support, to their credit).
A secondary overhead of higher-precision floats is the memory wall, arguably a more serious problem today. Memory bandwidth has become the most serious overhead for many numerical programs. Modern computers have a machine balance of 100:1, it means you need to do as many as 100 floating-point operations after loading a single value from RAM to reach the machine's peak performance. But this is not compatible with many algorithms with an inherently low arithmetic intensity, including important physics simulations. The use of of FP80 or FP128 will make them unacceptably slower. As a result, today's trend is moving from FP64 to FP32, and even to FP16 or a custom 16-bit format if possible, not vice versa.
The use of floating-point exceptions similarly became unpopular because of performance problems. I'm not an expert on this, but it was my impression that signaling a floating-point exception was so expensive on both hardware and operation-system level that it was never seriously used in most practical programs. So in contrary to IEEE-754, these exceptions never became an integral part of the programming environment. In addition to an expensive Unix signal, potential exceptions also inhibit the efficient pipelining of modern out-of-order, superscalar CPUs.
Finally, the use of unsafe optimizations is prevalent in many applications when rigor is sacrificed for speed.
So overall, the original spirit of IEEE-754 was long gone - for better or worse, unfortunately.
- SeanLuke 3y agoWas Kahan responsible for denormals being added to the spec?
- lifthrasiir 3y agoOnly partly; James Coonen and Harold Stone are also responsible. The resulting K-C-S draft eventually became IEEE 754 [1] and contained the Gradual Underflow from the very beginning. That said, the eventual decision was collaborative---DEC, a major opponent, even failed to convince its own commissioner. [1] https://people.eecs.berkeley.edu/~wkahan/ieee754status/754story.html https://people.eecs.berkeley.edu/~wkahan/ieee754status/754st...
- pmcjones 3y agoJames => Jerome.
- lou1306 3y ago> Everything should be at least double precision by default, and extended precision serves as an additional safeguard Honestly I don't understand how this would constitute a "solution" or even a "safeguard", really. Using any kind of FP arithmetics without being acutely aware of its quirks is going to cause headache no matter the precision. Conversely, when you do know how to deal with FP, then FP32 can suit many scenarios just fine.
- tails4e 3y agoI recently tried to implement a 16bit 64k point fft in C using FP32. It thought it would work easily. Turns out it was very difficult, what worked fine in double gave huge inaccuracies in float. It was due to the large small issue, if you subtract a small FP32 from a large tje result cannot be represented so it's as if the subtraction did not happen. These kinds of errors accumulated and affected the result. I found a workaround for my particular algorithm, but it was eye opening to me that FP32 was not enough to just work in this case.