9 ms·

How Java’s Floating-Point Hurts Everyone Everywhere (1998) [pdf]

57 points by llambda 13y ago

dekhn 13y ago
I dunno, given what Javascript foisted on a generation of programmers, we should be thankful Java is only as bad as it is. Go Kahan!
acjohnson55 13y ago
I browsed to page 10, but didn't learn what the big problem was, and then gave up.
- amock 13y ago
  There are five points on page 3: 1. Linguistically legislated exact reproducibility is at best mere wishful thinking. 2. Of two traditional policies for mixed precision evaluation, Java chose the worse. 3. Infinities and NaNs unleashed without the protection of floating-point traps and flags mandated by IEEE Standards 754/854 belie Java’s claim to robustness. 4. Every programmer’s prospects for success are diminished by Java’s refusal to grant access to capabilities built into over 95% of today's floating-point hardware. 5. Java has rejected even mildly disciplined infix operator overloading, without which extensions to arithmetic with everyday mathematical types like complex numbers, intervals, matrices, geometrical objects and arbitrarily high precision become extremely inconvenient.
  - kaoD 13y ago
    #1-#4 don't give much information about the real problem. #5 is just a matter of convenience. I'm sure the Java team gave a long thought to adding operator overloading, and I can see why they choose not to include it (Java is meant to be straightforward, with as little magic as possible). The first 20 pages or so are too much "blahblah" about Java, not much about how it's hurting everyone.
    - jerf 13y ago
      #5 isn't a matter of convenience if you're trying to do heavy mathematical operations in Java. Deviation between conventional notation and programming notation is a real problem, because it's a place where bugs can hide. However, I would be very sympathetic to the argument that the solution is "Don't do that". In 1998 and today, for what may be different reasons but still comes to the same outcome, Java is not a language intended for highly mathematical computations. It's probably better just to not pretend that it is one, than to try to force it in. Nor would I even consider this a criticism of Java (though I personally dislike it); compared to its focus area, trying to support heavy-duty math would be a tiny little niche to it that would take wildly disproportionate amounts of effort to support. Let people who care about that niche operate in it. My only point of agreement is that it is true that you ought to be able to get easy feedback about NaN and other failures, indeed precisely because in Java you are unlikely to be doing the sorts of calculations in which those are expected possibilities. $NaN or "infinity hours billed" is something you generally want an exception for, ASAP. It would have been better to write that in as a requirement for the VM and slowly emulate it on the few platforms that didn't natively support it than leave it out. (A brief google search suggests Java still doesn't support this, even today. If I got that wrong... great! I'm glad it has come on board, but I stand by the idea that it should have been in there from day 1, which is the real point I have here.)
  - MrBuddyCasino 13y ago
    Interestingly, omitting infix operator overloading was something the language designers were forced to do for time-to-market reasons, and in hindsight, is one of their biggest regrets. I don't have the source, I think it was in a Josh Bloch interview.
    - Aardwolf 13y ago
      So why not add them to a next Java version?
unwind 13y ago
This was just tweeted by John Carmack (https://twitter.com/ID_AA_Carmack/status/392298557631254528 https://twitter.com/ID_AA_Carmack/status/392298557631254528) which is probably why it was submitted. It's pretty old, it should have a [1998] tag in the title in my opinion. It's also pretty funny. :)
- daurnimator 13y ago
  *2004
  - deleted 13y ago
    [deleted]
  - milliams 13y ago
    Originally presented 1 March 1998 at the invitation of the ACM 1998 Workshop on Java for High–Performance Network Computing held at Stanford University Those exact slides might be from 2004 but the content is from 1998.
pcwalton 13y ago
This is a rant we came across in the early design of Rust and eventually decided was pretty misguided. Very few languages actually address these complaints. The biggest complaint here is that, in Java, when you perform arithmetic on 32-bit floats you perform 32-bit arithmetic with 32-bit intermediate results. Java's behavior is, in other words, what you'd expect. In C, though, if you perform arithmetic on 32-bit floats you're actually performing 64-bit arithmetic on 64-bit intermediate results (at least most of the time; it's been a while since I consulted the rules). Java's behavior (which basically amounts to doing what you asked to do) infuriated the author, so he gave a bunch of examples of scientific calculations that need 64 bits of precision. But that's totally unconvincing to me: if you want 64-bit floating point precision, use doubles. C's implicit conversions in general have caused more harm than good and what the author is complaining about is having to tell the compiler to do 64-bit arithmetic on 32-bit values instead of having it follow complex promotion rules that few programmers even know about.
- pfortuny 13y ago
  You might be right about bit-size (although I disagree) but that is not the only complaint: the lack (at least for that age, I do not know about now) of true 'exceptions' for NaNs, infs, etc... is very very important in many ordinary circumstances and cannot be dismissed as a whim. One should be able to check the status of an operation without having to implement a strange concoction (a 'trap').
- barrkel 13y ago
  Java's FP has historically had problems of its own. The requirement for exact portable behaviour crippled performance when the available hardware primitives weren't a good match. For example, x87 performs all calculations using a configurable intermediate precision, 32, 64 or 80 bit. But if you want to configure it, you need to change the control word - not something you want to do every other instruction. The main other way of specifying precision is to not use most of the processor's internal registers (FP stack), and instead store intermediate results to memory. That's changed a bit over the years, and Java has loosened up somewhat; you need strictfp to get the old behaviour.
saejox 13y ago
Cyrix. That's a word i didn't hear in at least a decade.
betterunix 13y ago
If 95% of programmers have misconceptions about floating point arithmetic, we should stop using it as the default rational / real number type. Floating point is a great optimization, but in a lot of situations it would be a lot better to have arbitrary-precision rational number types, continued fractions representations, or something else that better approximates what programmers expect.
- ErsatzVerkehr 13y ago
  These alternatives would all have their own pitfalls.
japaget 13y ago
What I found most interesting about the article was some of the mathematical techniques presented. See page 44 for an accurate formula for angular distances and page 48 ff. for an alternative formalism for the vector cross product.
Aardwolf 13y ago
Fully agree on the operator overloading complaint. It really is a LOT easier when working with matrices, vectors and scalars to be able to type their math expressions naturally rather than with functions like add(multiply(.....))
- peterashford 13y ago
  What operator do you use for vector dot product? What about cross product (as opposed to vector scaling)? What about the different precedence between primitive operators and overloaded operators ('+' and '*' have the same precedence in vector maths, no so in basic arithmetic). Does adding mathematical notation to programming languages make the code easier for non-mathematicians? Does having two different coding standards make the language easier to use and maintain? What do you do when your set of operator glyphs is less than the number of mathematical operations you need to perform? What happens with your code when the operator overloads you use clash with the operator overloads from a new library? I don't see any language that does operator overloading well and a lot of issues with making a good implementation. I like the idea I just see a lot of problems with the reality. And unless you can solve the problem well, I think it's the right thing to do the simplest thing that might possibly work.
  - Aardwolf 13y ago
    None for dot product. The regular matrix and vector multiplication as * operator makes things a lot more readable. Dot and cross product are less common for 3D rendering (one or none per line of code) so having that as a function doesn't hurt readibility.
    - peterashford 13y ago
      I like writing ray tracers, so I use dot product and cross product quite a lot.
  - bjz_ 13y ago
    I normally have dot and cross as free functions.
PaulHoule 13y ago
Many of the problems they point out aren't just Java problems, they're FORTRAN problems too.
peterashford 13y ago
Another "everything Java does is evil" rant. No wonder HN is all over it. This is tiresome. If you want 64 bit precision - use doubles. That's not hard, is it?

57 points by llambda 13y ago

dekhn 13y ago

I dunno, given what Javascript foisted on a generation of programmers, we should be thankful Java is only as bad as it is. Go Kahan!

acjohnson55 13y ago

I browsed to page 10, but didn't learn what the big problem was, and then gave up.

amock 13y ago
There are five points on page 3: 1. Linguistically legislated exact reproducibility is at best mere wishful thinking. 2. Of two traditional policies for mixed precision evaluation, Java chose the worse. 3. Infinities and NaNs unleashed without the protection of floating-point traps and flags mandated by IEEE Standards 754/854 belie Java’s claim to robustness. 4. Every programmer’s prospects for success are diminished by Java’s refusal to grant access to capabilities built into over 95% of today's floating-point hardware. 5. Java has rejected even mildly disciplined infix operator overloading, without which extensions to arithmetic with everyday mathematical types like complex numbers, intervals, matrices, geometrical objects and arbitrarily high precision become extremely inconvenient.
- kaoD 13y ago
  #1-#4 don't give much information about the real problem. #5 is just a matter of convenience. I'm sure the Java team gave a long thought to adding operator overloading, and I can see why they choose not to include it (Java is meant to be straightforward, with as little magic as possible). The first 20 pages or so are too much "blahblah" about Java, not much about how it's hurting everyone.
  - jerf 13y ago
    #5 isn't a matter of convenience if you're trying to do heavy mathematical operations in Java. Deviation between conventional notation and programming notation is a real problem, because it's a place where bugs can hide. However, I would be very sympathetic to the argument that the solution is "Don't do that". In 1998 and today, for what may be different reasons but still comes to the same outcome, Java is not a language intended for highly mathematical computations. It's probably better just to not pretend that it is one, than to try to force it in. Nor would I even consider this a criticism of Java (though I personally dislike it); compared to its focus area, trying to support heavy-duty math would be a tiny little niche to it that would take wildly disproportionate amounts of effort to support. Let people who care about that niche operate in it. My only point of agreement is that it is true that you ought to be able to get easy feedback about NaN and other failures, indeed precisely because in Java you are unlikely to be doing the sorts of calculations in which those are expected possibilities. $NaN or "infinity hours billed" is something you generally want an exception for, ASAP. It would have been better to write that in as a requirement for the VM and slowly emulate it on the few platforms that didn't natively support it than leave it out. (A brief google search suggests Java still doesn't support this, even today. If I got that wrong... great! I'm glad it has come on board, but I stand by the idea that it should have been in there from day 1, which is the real point I have here.)
- MrBuddyCasino 13y ago
  Interestingly, omitting infix operator overloading was something the language designers were forced to do for time-to-market reasons, and in hindsight, is one of their biggest regrets. I don't have the source, I think it was in a Josh Bloch interview.
  - Aardwolf 13y ago
    So why not add them to a next Java version?

unwind 13y ago

This was just tweeted by John Carmack (https://twitter.com/ID_AA_Carmack/status/392298557631254528 https://twitter.com/ID_AA_Carmack/status/392298557631254528) which is probably why it was submitted. It's pretty old, it should have a [1998] tag in the title in my opinion. It's also pretty funny. :)

daurnimator 13y ago
*2004
- deleted 13y ago
  [deleted]
- milliams 13y ago
  Originally presented 1 March 1998 at the invitation of the ACM 1998 Workshop on Java for High–Performance Network Computing held at Stanford University Those exact slides might be from 2004 but the content is from 1998.

pcwalton 13y ago

This is a rant we came across in the early design of Rust and eventually decided was pretty misguided. Very few languages actually address these complaints. The biggest complaint here is that, in Java, when you perform arithmetic on 32-bit floats you perform 32-bit arithmetic with 32-bit intermediate results. Java's behavior is, in other words, what you'd expect. In C, though, if you perform arithmetic on 32-bit floats you're actually performing 64-bit arithmetic on 64-bit intermediate results (at least most of the time; it's been a while since I consulted the rules). Java's behavior (which basically amounts to doing what you asked to do) infuriated the author, so he gave a bunch of examples of scientific calculations that need 64 bits of precision. But that's totally unconvincing to me: if you want 64-bit floating point precision, use doubles. C's implicit conversions in general have caused more harm than good and what the author is complaining about is having to tell the compiler to do 64-bit arithmetic on 32-bit values instead of having it follow complex promotion rules that few programmers even know about.

pfortuny 13y ago
You might be right about bit-size (although I disagree) but that is not the only complaint: the lack (at least for that age, I do not know about now) of true 'exceptions' for NaNs, infs, etc... is very very important in many ordinary circumstances and cannot be dismissed as a whim. One should be able to check the status of an operation without having to implement a strange concoction (a 'trap').
barrkel 13y ago
Java's FP has historically had problems of its own. The requirement for exact portable behaviour crippled performance when the available hardware primitives weren't a good match. For example, x87 performs all calculations using a configurable intermediate precision, 32, 64 or 80 bit. But if you want to configure it, you need to change the control word - not something you want to do every other instruction. The main other way of specifying precision is to not use most of the processor's internal registers (FP stack), and instead store intermediate results to memory. That's changed a bit over the years, and Java has loosened up somewhat; you need strictfp to get the old behaviour.

saejox 13y ago

Cyrix. That's a word i didn't hear in at least a decade.

betterunix 13y ago

If 95% of programmers have misconceptions about floating point arithmetic, we should stop using it as the default rational / real number type. Floating point is a great optimization, but in a lot of situations it would be a lot better to have arbitrary-precision rational number types, continued fractions representations, or something else that better approximates what programmers expect.

ErsatzVerkehr 13y ago
These alternatives would all have their own pitfalls.

japaget 13y ago

What I found most interesting about the article was some of the mathematical techniques presented. See page 44 for an accurate formula for angular distances and page 48 ff. for an alternative formalism for the vector cross product.

Aardwolf 13y ago

Fully agree on the operator overloading complaint. It really is a LOT easier when working with matrices, vectors and scalars to be able to type their math expressions naturally rather than with functions like add(multiply(.....))

peterashford 13y ago
What operator do you use for vector dot product? What about cross product (as opposed to vector scaling)? What about the different precedence between primitive operators and overloaded operators ('+' and '*' have the same precedence in vector maths, no so in basic arithmetic). Does adding mathematical notation to programming languages make the code easier for non-mathematicians? Does having two different coding standards make the language easier to use and maintain? What do you do when your set of operator glyphs is less than the number of mathematical operations you need to perform? What happens with your code when the operator overloads you use clash with the operator overloads from a new library? I don't see any language that does operator overloading well and a lot of issues with making a good implementation. I like the idea I just see a lot of problems with the reality. And unless you can solve the problem well, I think it's the right thing to do the simplest thing that might possibly work.
- Aardwolf 13y ago
  None for dot product. The regular matrix and vector multiplication as * operator makes things a lot more readable. Dot and cross product are less common for 3D rendering (one or none per line of code) so having that as a function doesn't hurt readibility.
  - peterashford 13y ago
    I like writing ray tracers, so I use dot product and cross product quite a lot.
- bjz_ 13y ago
  I normally have dot and cross as free functions.

PaulHoule 13y ago

Many of the problems they point out aren't just Java problems, they're FORTRAN problems too.

peterashford 13y ago

Another "everything Java does is evil" rant. No wonder HN is all over it. This is tiresome. If you want 64 bit precision - use doubles. That's not hard, is it?