7 ms·
This is a rant we came across in the early design of Rust and eventually decided was pretty misguided. Very few languages actually address these complaints. Th
by pcwalton 13y ago
This is a rant we came across in the early design of Rust and eventually decided was pretty misguided. Very few languages actually address these complaints.
The biggest complaint here is that, in Java, when you perform arithmetic on 32-bit floats you perform 32-bit arithmetic with 32-bit intermediate results. Java's behavior is, in other words, what you'd expect. In C, though, if you perform arithmetic on 32-bit floats you're actually performing 64-bit arithmetic on 64-bit intermediate results (at least most of the time; it's been a while since I consulted the rules). Java's behavior (which basically amounts to doing what you asked to do) infuriated the author, so he gave a bunch of examples of scientific calculations that need 64 bits of precision. But that's totally unconvincing to me: if you want 64-bit floating point precision, use doubles. C's implicit conversions in general have caused more harm than good and what the author is complaining about is having to tell the compiler to do 64-bit arithmetic on 32-bit values instead of having it follow complex promotion rules that few programmers even know about.
- pfortuny 13y agoYou might be right about bit-size (although I disagree) but that is not the only complaint: the lack (at least for that age, I do not know about now) of true 'exceptions' for NaNs, infs, etc... is very very important in many ordinary circumstances and cannot be dismissed as a whim. One should be able to check the status of an operation without having to implement a strange concoction (a 'trap').
- barrkel 13y agoJava's FP has historically had problems of its own. The requirement for exact portable behaviour crippled performance when the available hardware primitives weren't a good match. For example, x87 performs all calculations using a configurable intermediate precision, 32, 64 or 80 bit. But if you want to configure it, you need to change the control word - not something you want to do every other instruction. The main other way of specifying precision is to not use most of the processor's internal registers (FP stack), and instead store intermediate results to memory. That's changed a bit over the years, and Java has loosened up somewhat; you need strictfp to get the old behaviour.
- Pxtl 13y agoYeah, that. Providing portable, perfectly-consistent universal FP is needed as an option (going without it kills whole classes of networked computing applications), but should never be mandatory, not for a language that claims to be general-purpose.
- pcwalton 13y agoWouldn't you only need to change the control word when switching between 32-bit and 64-bit arithmetic? I don't think that would happen in every instruction, at least for the FP-intensive apps I work on (graphics).
- ubasu 13y agoNote that this "rant" is by Velvel Kahan [1], who won the Turing Award for designing the IEEE 754 floating point standard [2] (among other things), which is used for specifying floating point computations on almost all computers today. So perhaps he has a little more context on this than yourself. [1] http://en.wikipedia.org/wiki/William_Kahan http://en.wikipedia.org/wiki/William_Kahan [2] http://en.wikipedia.org/wiki/IEEE_754-2008 http://en.wikipedia.org/wiki/IEEE_754-2008
- kefka 13y agoBut that sounds like an argument to appeal by authority. Firstly, the title page says "presented March 1, 1998". That's pretty old. And then the PDF shows creation date 'July 30, 2004'. I'll assume that there were 6 years of updates to this file between the 2 dates. So that still is about 10 years old document. How have things changed since then?
- pavpanchekha 13y agoAppeal to authority? Ah, I think you misunderstand why that is considered an invalid form of argument. An appeal to authority occurs when one appeals to an authority in an unrelated field in order to bolster an argument from a different field. For example, while watching football yesterday, I saw a commercial where a football player told me that CenturyLink had the fastest internet. This is an implicit appeal to authority. The player is undoubtedly an authority on how to play football, but this authority is unrelated to the question of choosing a good internet service. On the other hand, if this player were to tell me which football video games were most realistic, this would not be an invalid form of argument, since he in fact has some authority on what makes a football video game realistic (Of course, there may be higher authorities yet). In this case, William Kahan actually has authority on the proper use of floating point, as the designer of IEEE 754 and a well-known authority in numerical analysis. His credentials in programming language design, granted, are weaker, but you cannot deny that his authority does in fact extend to the object of our discussion.
- scott_s 13y ago
- acqq 13y agoThe biggest problem is that Java designers ignored the numerical aspects of language almost completely. The IEEE 754 is an international standard for FPU calculations which is implemented on practically all relevant CPU's for at least some 20 years. Java designers practically ignored the standard since they obviously didn't understand it or decided that serious computations will never be important for their target group. A lot of people commenting here don't understand it either. OK, a lot of people don't need it. But those that need such calculations simply can't implement decent numerical calculations in Java. So the next time somebody claims that "Java is comparably fast to C" remember that it can be true only for some of the algorithms. If you ignore IEEE 754 for Rust you're creating one more language that will not be "as good as C." And here I don't mean "C as defined by language lawyers" but "C as implemented in the compilers to at least minimally support the use of the underlying hardware for those who need that." And those who need that are the people who use computers for computations. Don't ignore them. Or at least don't be surprised that they stick to old Fortran and C compilers.
- pcwalton 13y agoSure, I agree we shouldn't neglect the needs of scientific computing. But we need concrete suggestions. It's not our intent to "neglect IEEE 754" (Graydon Hoare was on that committee!) I haven't seen any concrete reason that eliminating implicit floating point promotion affects scientific code. Just use doubles if you want 64-bit precision. We match the underlying LLVM IR much more closely than C does.
- acqq 13y agoConcrete: C didn't have "in a language standard" IEEE support for a while, but it had in practice. C99 had it standardized, and it's not "just libraries." If you make a serious language, at least that's what you shouldn't ignore. http://en.wikipedia.org/wiki/C99 http://en.wikipedia.org/wiki/C99 (IEEE 754 floating point support) Still the practice is even more important than the standardization. In the compilers I used I actually had to do "bit-tricks" to do some parts of manipulations of doubles and floats. Last times I've checked, even initializing values to the desired values was something where you actually "danced" around the standard. It works and it's needed, even if you won't find it in the standards. For an example of what people actually have to use and C "allows" see: http://en.wikipedia.org/wiki/Fast_inverse_square_root http://en.wikipedia.org/wiki/Fast_inverse_square_root If you make a language that is "nice theoretically" but doesn't allow you to use the hardware, it will remain just "nice theoretically." Edit: regarding "just use doubles:" it's about the engineering trade-off: for a lot of uses you want to read from and store to floats but do all the calculations in doubles. Only the calculations which are specifically designed to use floats as partial results should be done with floats. The language should be easy to accommodate this assumption. Computation has simply different logic than "classes" and "theoretical types."
- anonymous 13y agoIn what context would I want less accurate results from my arithmetic expressions? I mean, I have some numbers, I write an expression with them, and then I expect to get the most accurate possible result. Doesn't it make a lot more sense to let users specifically specify that a given expression is to be evaluated only using 32-bit float values, than to make everybody always cast every value in every expression to 64-bit float? I can tell you that having to sprinkle (double) all over your Java code (and then an additional (float) in front) to be sure you're getting the most available precision is pretty tedious.
- pcwalton 13y agoTwo reasons: 1. I mostly write graphics and browser layout code when I use floating point. I never want 64-bit precision. I want the fastest thing available, and I'm willing to accept some error. It's surprisingly tricky to get C to actually do 32-bit floating point arithmetic, and I suspect we may actually gain a little performance over existing browser engines with Rust here (probably negligible, but hey). 2. Wouldn't you expect these two expressions to be the same (assuming b, c, d are all f32)? let a: f32 = b + c / d; and let tmp = c / d; let a: f32 = b + tmp; In Kahan's preferred semantics, they are not the same. This interferes with refactoring, complicates the language, and violates intuition, IMHO. Introducing temporaries without changing the semantics is something programmers should be allowed to do.
- anonymous 13y ago1. Faster? Really? Those values only live in registers for a short while, is saving one or two cycles all that important? 2. That is an interesting point. Gut instinct says tmp should end up being f64 and then both expressions would be the same. Equally valid is the interpretation that the user selected explicitly for conversion to f32 of the intermediate value. We all know computers don't perform platonically perfect math with real numbers, there isn't much you can do about it, but I think the choice which results in the most precision should win at the end of the day. Perhaps in a language which doesn't require specifying types like Rust, the best thing to do is to simply dispense with 32-bit floating point values entirely. I'd be equally happy with compiler flags like -fmore-precision. Or maybe you could have other arithmetic operators like +64 and +80 which cast all their arguments to 64/80-bit floating point numbers and produce a result with that precision. So I'd write let a: f32 = b (+80) c (/80) d as f32 Given that Rust is a pretty young language and still far from a 1.0 release, do consider if you can provide any kind of syntax to actually make full use of computers' numerical capabilities.
- spullara 13y agoThis was fixed in Java 1.2 I think: http://en.wikipedia.org/wiki/Strictfp http://en.wikipedia.org/wiki/Strictfp