5 ms·

In what context would I want less accurate results from my arithmetic expressions? I mean, I have some numbers, I write an expression with them, and then I expe

In what context would I want less accurate results from my arithmetic expressions? I mean, I have some numbers, I write an expression with them, and then I expect to get the most accurate possible result. Doesn't it make a lot more sense to let users specifically specify that a given expression is to be evaluated only using 32-bit float values, than to make everybody always cast every value in every expression to 64-bit float? I can tell you that having to sprinkle (double) all over your Java code (and then an additional (float) in front) to be sure you're getting the most available precision is pretty tedious.

pcwalton 13y ago
Two reasons: 1. I mostly write graphics and browser layout code when I use floating point. I never want 64-bit precision. I want the fastest thing available, and I'm willing to accept some error. It's surprisingly tricky to get C to actually do 32-bit floating point arithmetic, and I suspect we may actually gain a little performance over existing browser engines with Rust here (probably negligible, but hey). 2. Wouldn't you expect these two expressions to be the same (assuming b, c, d are all f32)? let a: f32 = b + c / d; and let tmp = c / d; let a: f32 = b + tmp; In Kahan's preferred semantics, they are not the same. This interferes with refactoring, complicates the language, and violates intuition, IMHO. Introducing temporaries without changing the semantics is something programmers should be allowed to do.
- anonymous 13y ago
  1. Faster? Really? Those values only live in registers for a short while, is saving one or two cycles all that important? 2. That is an interesting point. Gut instinct says tmp should end up being f64 and then both expressions would be the same. Equally valid is the interpretation that the user selected explicitly for conversion to f32 of the intermediate value. We all know computers don't perform platonically perfect math with real numbers, there isn't much you can do about it, but I think the choice which results in the most precision should win at the end of the day. Perhaps in a language which doesn't require specifying types like Rust, the best thing to do is to simply dispense with 32-bit floating point values entirely. I'd be equally happy with compiler flags like -fmore-precision. Or maybe you could have other arithmetic operators like +64 and +80 which cast all their arguments to 64/80-bit floating point numbers and produce a result with that precision. So I'd write let a: f32 = b (+80) c (/80) d as f32 Given that Rust is a pretty young language and still far from a 1.0 release, do consider if you can provide any kind of syntax to actually make full use of computers' numerical capabilities.
  - pcwalton 13y ago
    Regarding point (1), it starts to matter when autovectorization kicks in. If this expression is in a loop and the compiler can autovectorize (which rustc does, if you have a new enough version), you really want to be able to pack more values into a SIMD register if you can. (This is one point in which this paper is out of date…) As for point (2), well, I think that making the size of "tmp" f64 would interact badly with other features like operator overloading and generics, since you're playing fast and loose with types. Gory details (warning, type theory ahead): "+" is implemented by a trait, Add(RHS,Result) where RHS and Result are concrete types and there is a functional dependency so that Self + RHS → Result. This fundep is necessary because otherwise "let tmp: b + c" wouldn't work, as the compiler doesn't know what the type of "tmp" is (is it f32 or f64? It won't guess.) So you can't simultaneously implement f32 : Add(f32,f32) and f32 : Add(f32,f64). We'd have to introduce a lot more experimental type machinery to make this work (a "floating point functional dependency", I guess?) and I'm not sure there wouldn't be fallout (for example, I can foresee issues with higher-kinded type parameters).
    - acqq 13y ago
      It's very simple: as soon as the floating value is read, it should go to the 64-bit virtual register unless you specify that register to be 32-bit too. If you want to use 32-bit partial results, you should say that explicitly. Autovectorization can be applied even if the rules are like suggested.
- bluecalm 13y ago
  I don't know, I wouldn't expect that tbh but I am used to C. I also don't see why I should need it, I am not going to compare non-zero floating point numbers anyway but I would take better accuracy in long computations any day. I think there is value in following the standard (what people doing floating point computations are used to) and you need very good reason to deviate from it. You state that performance gains will be negligible if any in your domain but you could potentially cause a lot of headache in other domains. I don't think elegance/simplicity/negligible performance gains (if you are right about it at all) is enough of a reason to deviate from standard behavior.
  - pcwalton 13y ago
    > I don't think elegance/simplicity (if you are right about it at all) is enough of a reason to deviate from standard behavior. The elegance and simplicity in this case are what are needed to make generics work. (See my other response for the gory details.)
    - acqq 13y ago
      Still the fact is that computation has different goals than "generics" or whatever you want from the rest of the language "cleanness." On x86 and most other CPU's, reading 32-bit FP variable and doing computation with 64-bits doesn't cost you anything: it's just how you fill the bits of the register during the load. As long as you don't have FP registers spills, computation has the same speed, and the final store to the destination is of course faster if you store only 4 bytes instead of 8. You can omit some aspects from your language, but don't be surprised that people who use C continue to use C. If you really want bigger acceptance, even if it's "harder" to do it "properly," you can't pretend that people don't need it.
      - pcwalton 13y ago
        Cleanliness is not about theoretical purity in this case. It's about type soundness, something that C doesn't have but which is a major design goal of Rust. And it does cost you something: autovectorization, as I explained earlier. The problem with these kinds of discussions is that we aren't talking about expressive power: it's obvious that you can do both 64-bit or 32-bit math in either C or Rust. The issue is just what the defaults should be. In these discussions, small communities (like scientific computing in this case) tend to have oversimplified notions of how things should "obviously" be without understanding the subtleties from other perspectives (like PL design). You don't "just" make the "+" operator return different types depending on context in a language with typeclasses. That can break soundness.
- qznc 13y ago
  On page 43, Kahan gives a four rule of thumbs. According to rule 3 (declare temporary/local variables all with the widest ﬁnite precision that is not too slow nor too narrow), I think Kahan would actually agree with you. Maybe you confused Kahans rant with C semantics?
  - qznc 13y ago
    More precisely, I think the scenario is different with type inference. What is the type of "c/d", if c and d are f32? Would it be unreasonable, if the expression had type f64 or even more precise?

pcwalton 13y ago

Two reasons: 1. I mostly write graphics and browser layout code when I use floating point. I never want 64-bit precision. I want the fastest thing available, and I'm willing to accept some error. It's surprisingly tricky to get C to actually do 32-bit floating point arithmetic, and I suspect we may actually gain a little performance over existing browser engines with Rust here (probably negligible, but hey). 2. Wouldn't you expect these two expressions to be the same (assuming b, c, d are all f32)? let a: f32 = b + c / d; and let tmp = c / d; let a: f32 = b + tmp; In Kahan's preferred semantics, they are not the same. This interferes with refactoring, complicates the language, and violates intuition, IMHO. Introducing temporaries without changing the semantics is something programmers should be allowed to do.

anonymous 13y ago
1. Faster? Really? Those values only live in registers for a short while, is saving one or two cycles all that important? 2. That is an interesting point. Gut instinct says tmp should end up being f64 and then both expressions would be the same. Equally valid is the interpretation that the user selected explicitly for conversion to f32 of the intermediate value. We all know computers don't perform platonically perfect math with real numbers, there isn't much you can do about it, but I think the choice which results in the most precision should win at the end of the day. Perhaps in a language which doesn't require specifying types like Rust, the best thing to do is to simply dispense with 32-bit floating point values entirely. I'd be equally happy with compiler flags like -fmore-precision. Or maybe you could have other arithmetic operators like +64 and +80 which cast all their arguments to 64/80-bit floating point numbers and produce a result with that precision. So I'd write let a: f32 = b (+80) c (/80) d as f32 Given that Rust is a pretty young language and still far from a 1.0 release, do consider if you can provide any kind of syntax to actually make full use of computers' numerical capabilities.
- pcwalton 13y ago
  Regarding point (1), it starts to matter when autovectorization kicks in. If this expression is in a loop and the compiler can autovectorize (which rustc does, if you have a new enough version), you really want to be able to pack more values into a SIMD register if you can. (This is one point in which this paper is out of date…) As for point (2), well, I think that making the size of "tmp" f64 would interact badly with other features like operator overloading and generics, since you're playing fast and loose with types. Gory details (warning, type theory ahead): "+" is implemented by a trait, Add(RHS,Result) where RHS and Result are concrete types and there is a functional dependency so that Self + RHS → Result. This fundep is necessary because otherwise "let tmp: b + c" wouldn't work, as the compiler doesn't know what the type of "tmp" is (is it f32 or f64? It won't guess.) So you can't simultaneously implement f32 : Add(f32,f32) and f32 : Add(f32,f64). We'd have to introduce a lot more experimental type machinery to make this work (a "floating point functional dependency", I guess?) and I'm not sure there wouldn't be fallout (for example, I can foresee issues with higher-kinded type parameters).
  - acqq 13y ago
    It's very simple: as soon as the floating value is read, it should go to the 64-bit virtual register unless you specify that register to be 32-bit too. If you want to use 32-bit partial results, you should say that explicitly. Autovectorization can be applied even if the rules are like suggested.
bluecalm 13y ago
I don't know, I wouldn't expect that tbh but I am used to C. I also don't see why I should need it, I am not going to compare non-zero floating point numbers anyway but I would take better accuracy in long computations any day. I think there is value in following the standard (what people doing floating point computations are used to) and you need very good reason to deviate from it. You state that performance gains will be negligible if any in your domain but you could potentially cause a lot of headache in other domains. I don't think elegance/simplicity/negligible performance gains (if you are right about it at all) is enough of a reason to deviate from standard behavior.
- pcwalton 13y ago
  > I don't think elegance/simplicity (if you are right about it at all) is enough of a reason to deviate from standard behavior. The elegance and simplicity in this case are what are needed to make generics work. (See my other response for the gory details.)
  - acqq 13y ago
    Still the fact is that computation has different goals than "generics" or whatever you want from the rest of the language "cleanness." On x86 and most other CPU's, reading 32-bit FP variable and doing computation with 64-bits doesn't cost you anything: it's just how you fill the bits of the register during the load. As long as you don't have FP registers spills, computation has the same speed, and the final store to the destination is of course faster if you store only 4 bytes instead of 8. You can omit some aspects from your language, but don't be surprised that people who use C continue to use C. If you really want bigger acceptance, even if it's "harder" to do it "properly," you can't pretend that people don't need it.
    - pcwalton 13y ago
      Cleanliness is not about theoretical purity in this case. It's about type soundness, something that C doesn't have but which is a major design goal of Rust. And it does cost you something: autovectorization, as I explained earlier. The problem with these kinds of discussions is that we aren't talking about expressive power: it's obvious that you can do both 64-bit or 32-bit math in either C or Rust. The issue is just what the defaults should be. In these discussions, small communities (like scientific computing in this case) tend to have oversimplified notions of how things should "obviously" be without understanding the subtleties from other perspectives (like PL design). You don't "just" make the "+" operator return different types depending on context in a language with typeclasses. That can break soundness.
qznc 13y ago
On page 43, Kahan gives a four rule of thumbs. According to rule 3 (declare temporary/local variables all with the widest ﬁnite precision that is not too slow nor too narrow), I think Kahan would actually agree with you. Maybe you confused Kahans rant with C semantics?
- qznc 13y ago
  More precisely, I think the scenario is different with type inference. What is the type of "c/d", if c and d are f32? Would it be unreasonable, if the expression had type f64 or even more precise?