4 ms·

Faster floating point math with Rust's new API

112 points by subset 2mo ago

pjmlp 2mo ago
Ideally CPython would have a JIT that would be able to do this, depending on the current hardware like other ecosystems, but we are still not there yet.
- IshKebab 2mo ago
  CPython can't do this because it's a change in semantics. You need explicit opt-in from the programmer. Anyway adding this optimisation to CPython would be like putting active aero on a dandy horse.
  - pjmlp 2mo ago
    Some of us would like to have Python finally catch up to Lisp in compiler tooling, but alas. As for change in semantics, apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures.
    - aw1621107 2mo ago
      > apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures. I think it'd be an issue irrespective of the architecture? Optimizations are generally expected to preserve semantics and those languages all specify IEEE 754 semantics which aren't necessarily associative. For instance, from the Java language spec [0]: > Floating-point arithmetic is carried out in accordance with the rules of the IEEE 754 Standard, including for overflow and underflow (§15.4), with the exception of the remainder operator % (§15.17.3). Or the .NET reference [1]: > The Double type complies with the IEC 60559:1989 (IEEE 754) standard for binary floating-point arithmetic. Or the ECMAScript 2027 spec [2]: > Numeric operators such as +, ×, =, and ≥ refer to those operations as determined by the type of the operands. [] When applied to Numbers, the operators refer to the relevant operations within IEEE 754-2019. [0]: https://docs.oracle.com/javase/specs/jls/se26/jls26.pdf https://docs.oracle.com/javase/specs/jls/se26/jls26.pdf [1]: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/types https://learn.microsoft.com/en-us/dotnet/csharp/language-ref... [2]: https://tc39.es/ecma262/#sec-mathematical-operations https://tc39.es/ecma262/#sec-mathematical-operations
      - pjmlp 2mo ago
        Regardless, those optimisations are available on the respective JITs. RyuJIT target CPU modes, breaking change due to dropping support for older hardware, https://github.com/dotnet/docs/issues/48045 https://github.com/dotnet/docs/issues/48045 > Native Image now targets x86-64-v3 architecture by default on AMD64 and provides a new -march option to specify target compatibility. Use -march=compatibility for best compatibility or -march=native for best performance if a native executable is deployed on the same machine or on a machine with the same CPU features. To list all available machine types, use -march=list. https://www.graalvm.org/release-notes/JDK_20 https://www.graalvm.org/release-notes/JDK_20
        aw1621107 2mo ago
        > those optimisations are available on the respective JITs. I'd assume they aren't applied by default and/or without the programmer explicitly opting in to those altered semantics, though. Would you be able to show otherwise? I don't see how changing targeted instruction sets is relevant here as the instructions you use is orthogonal to whether you assume floating point operations are associative.
        pjmlp 2mo ago
        I pasted the links for a reason.
        aw1621107 2mo ago
        Neither of those appear to say anything about treating floating point operations as associative nor reordering them as a standard optimization, let alone allowing the programmer to opt into such an optimization.
        afdbcreid 2mo ago
        > Regardless, those optimisations are available on the respective JITs. I'm pretty sure this is wrong. No JS engine, RyuJIT or or HotSpot break IEE-754. Java does have intrinsics for algebraic floating point operations (but does not apply them by default and I don't think they're exposed), the other I don't think.
- westurner 2mo ago
  RustPython?
GeertB 2mo ago
Signed integer addition is only associative when overflow is defined to wrap around like unsigned arithmetic. This condition is matched here, because only debug builds panic on overflow. However, it's a bit of a gray area that the article completely ignores.
- thrance 2mo ago
  Unlike C, Rust's signed arithmetic is fully specified to wrap around.
  - aw1621107 2mo ago
    > Rust's signed arithmetic is fully specified to wrap around. Well, kind of. It's currently documented to wrap in release mode by default, but it's just that - a default. You're free to enable overflow checks in release mode (or disable them in debug if you really like oddball configurations), and either way overflow is considered a logic error that devs shouldn't rely on (and basically can't rely on when not in control of the end binary since it's the end user who controls overflow checks). The Rust devs are theoretically open to making signed overflow panic by default, but consider such a change unlikely unless "something materially changes" [0]. [0]: https://github.com/rust-lang/rust/issues/47739#issuecomment-3165061176 https://github.com/rust-lang/rust/issues/47739#issuecomment-...
    - raverbashing 2mo ago
      Thanks for clarification, it seems Rust devs (as opposite to C devs) like good defaults and don't like making code accidentally cut yourself just because you looked at it wrong
  - tialaramex 2mo ago
    No. If you want wrapping, ask for it with Wrapping<T> or the specific Wrapping types, or the wrapping arithmetic APIs It's true that since it's safe and faster, release builds default to wrapping rather than panic, but it's still wrong if you overflow any of Rust's default integer types, it's just that in a safe language it won't be Undefined Behaviour. "I can't be bothered to do it correctly" speaks to the quality of the rest of the product, it's a Brown M&M [read about the Van Halen test if you don't know what a Brown M&M means]
rob74 2mo ago
I'm not an expert on floating point math, but the "Does adding a small number do nothing?" example caught my eye, because the numbers used are constants, which are arbitrary precision in some languages (https://stackoverflow.com/questions/57511935/what-is-the-purpose-of-arbitrary-precision-constants-in-go https://stackoverflow.com/questions/57511935/what-is-the-pur...). For instance, Go answers the question with false (but still prints out "1e+16" when you try to print 1e16 + 1): https://go.dev/play/p/mSAktWpCRJA https://go.dev/play/p/mSAktWpCRJA
- tialaramex 2mo ago
  Rust explicitly requires that constants are typed. const FOO: f32 = 0.75; // The 32-bit floating point value three quarters If you try const UNTYPED = 0.75; // Does not compile, pick a type I don't find the SO answer very convincing because it seems like it's trying to argue this is the Reals, and it just isn't, it's only a subset of the Rationals which happened to be convenient for Go to work with it. The Reals are much stranger.
  - ameliaquining 2mo ago
    Sorry, what exactly is your disagreement with the SO answer? It doesn't mention the reals.
  - rob74 2mo ago
    Real numbers include rational numbers (numbers which can be represented as a fraction of two integers) and irrational numbers (numbers that can't be represented as a fraction - the most famous one is probably π). Since irrational numbers have an infinite number of decimal places, they obviously can't be stored as a floating point value and also can't be written down exactly, no matter how many decimal places you use. "Arbitrary precision" constants might get you closer, but yes, you will never be able to store a "true" irrational number in a computer.
    - kibwen 2mo ago
      > you will never be able to store a "true" irrational number in a computer. Joke's on you, in my programming language all numbers are written in phinary: https://en.wikipedia.org/wiki/Golden_ratio_base https://en.wikipedia.org/wiki/Golden_ratio_base
14113 2mo ago
> Floating point math is often slower than integer math because the compiler is being conservative about how it optimizes your code. It's not strictly true to say that it's "being conservative". What is more correct is to say that floating point operations have different semantics to integer operations, and an optimisation that retains the semantics of an expression over integers may not do so when applied to an expression over integers. Hence, it may be possible to apply one optimisation to an integer expression, but applying that to a floating-point expression may result in a different program meaning. C/C++ compilers give you a way out of this with the `--ffast-math` flag, which essentially allows compilers to relax the constraints on floating-point optimisation passes. For an example of how this works in GCC, take a look here: https://gcc.gnu.org/wiki/FloatingPointMath https://gcc.gnu.org/wiki/FloatingPointMath
- duped 2mo ago
  > C/C++ compilers give you a way out of this with the -fast-math People should really use the individual optimization flags they want (no signed zeros, no trapping math, associative math, reciprocal math) and not -ffast-math because the other optimizations it enables leads to surprising code (for example, isinf and isnan may become noops, which will break production code). Basically never use an optimization flag that changes the semantics of your code without understanding exactly what that means. I have had to fix this in a number of codebases because someone thought that flag was as innocuous as -O3. And if you have to enable flush to zero/denormals are zero it should be explicit in your code and scoped.
  - 14113 2mo ago
    Yes, I was being a bit concise: Individual optimisations should be turned on as determined by profiling, application semantics, etc. My point was more that if you want to get as close as possible between integer and floating-point, then there is a flag that does it. That doesn't mean that you should do it, however...
  - tialaramex 2mo ago
    Yes. In fact, even if you don't ask for semantics like the -fast-math flag, the floating point types are worth some extra time to understand before relying on them. They're much stranger than the machine integers. The machine integers are basically like the Integers you were taught in school, except for overflow. That's not nothing but it's a complexity you can ignore entirely so long as you never overflow. In Rust you can have the language keep you safe - if an overflow occurs we'll panic and we're done. However the floating point types are a weird thing entirely invented for the convenience of the machine. They're too often introduced as if, like the machine integers, they're almost familiar numbers from school. Some languages even call these types "real" - but they very much are not actually the Real numbers, not even the approximation that the machine integers were to actual Integers. The programming language can't help you cope today. You can use software like "Herbie" to help you a bit, but today's languages just leave you with it. https://herbie.uwplse.org/ https://herbie.uwplse.org/ Here's an easy example you saw in school, a tenth, written 0.1 in decimal. The floating point types cannot represent this number. When you ask for the 32-bit floating point value 0.1 in a language like C or Rust, you actually get exactly 0.100000001490116119384765625 because that was a number the type can represent and it was deemed "close enough".
conradludgate 2mo ago
See also https://orlp.net/blog/taming-float-sums/ https://orlp.net/blog/taming-float-sums/
Asooka 2mo ago
I like the idea, but I hate how verbose it is. Would be nice if there was also a macro that would transform all arithmetic within a block into algebraic arithmetic. The name is also a bit misleading, as I would expect "alebraic_add" to give an exact algebraic result, but instead it enables optimisations based on associative semantics. As a sidenote, if implemented as a macro, e.g. fn fast_sum_f64(values: &[f64]) -> f64 { let mut total = 0; fp_opt!(associative, { for value in values { total += value; } }); return total; } A question arises what happens when operating on custom types that overload arithmetic operations. I think the cleanest approach here would be to let the custom type define optimised versions, or have another macro that automatically generates them based on the existing ones, i.e. propagate the optimisation flags.
- afdbcreid 2mo ago
  You can create an `struct Algebraic<T>(pub T)` newtype that overloads operators using algebraic methods. In fact such wrapper might be added to std (it was discussed but decided that not yet. Such wrappers exist for wrapping and saturating arithmetic).

112 points by subset 2mo ago

pjmlp 2mo ago

Ideally CPython would have a JIT that would be able to do this, depending on the current hardware like other ecosystems, but we are still not there yet.

IshKebab 2mo ago
CPython can't do this because it's a change in semantics. You need explicit opt-in from the programmer. Anyway adding this optimisation to CPython would be like putting active aero on a dandy horse.
- pjmlp 2mo ago
  Some of us would like to have Python finally catch up to Lisp in compiler tooling, but alas. As for change in semantics, apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures.
  - aw1621107 2mo ago
    > apparently that isn't an issue on JS, Java and .NET JITs in adopting more modern architectures. I think it'd be an issue irrespective of the architecture? Optimizations are generally expected to preserve semantics and those languages all specify IEEE 754 semantics which aren't necessarily associative. For instance, from the Java language spec [0]: > Floating-point arithmetic is carried out in accordance with the rules of the IEEE 754 Standard, including for overflow and underflow (§15.4), with the exception of the remainder operator % (§15.17.3). Or the .NET reference [1]: > The Double type complies with the IEC 60559:1989 (IEEE 754) standard for binary floating-point arithmetic. Or the ECMAScript 2027 spec [2]: > Numeric operators such as +, ×, =, and ≥ refer to those operations as determined by the type of the operands. [] When applied to Numbers, the operators refer to the relevant operations within IEEE 754-2019. [0]: https://docs.oracle.com/javase/specs/jls/se26/jls26.pdf https://docs.oracle.com/javase/specs/jls/se26/jls26.pdf [1]: https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/types https://learn.microsoft.com/en-us/dotnet/csharp/language-ref... [2]: https://tc39.es/ecma262/#sec-mathematical-operations https://tc39.es/ecma262/#sec-mathematical-operations
    - pjmlp 2mo ago
      Regardless, those optimisations are available on the respective JITs. RyuJIT target CPU modes, breaking change due to dropping support for older hardware, https://github.com/dotnet/docs/issues/48045 https://github.com/dotnet/docs/issues/48045 > Native Image now targets x86-64-v3 architecture by default on AMD64 and provides a new -march option to specify target compatibility. Use -march=compatibility for best compatibility or -march=native for best performance if a native executable is deployed on the same machine or on a machine with the same CPU features. To list all available machine types, use -march=list. https://www.graalvm.org/release-notes/JDK_20 https://www.graalvm.org/release-notes/JDK_20
      - aw1621107 2mo ago
        > those optimisations are available on the respective JITs. I'd assume they aren't applied by default and/or without the programmer explicitly opting in to those altered semantics, though. Would you be able to show otherwise? I don't see how changing targeted instruction sets is relevant here as the instructions you use is orthogonal to whether you assume floating point operations are associative.
        pjmlp 2mo ago
        I pasted the links for a reason.
        aw1621107 2mo ago
        Neither of those appear to say anything about treating floating point operations as associative nor reordering them as a standard optimization, let alone allowing the programmer to opt into such an optimization.
      - afdbcreid 2mo ago
        > Regardless, those optimisations are available on the respective JITs. I'm pretty sure this is wrong. No JS engine, RyuJIT or or HotSpot break IEE-754. Java does have intrinsics for algebraic floating point operations (but does not apply them by default and I don't think they're exposed), the other I don't think.
westurner 2mo ago
RustPython?

GeertB 2mo ago

Signed integer addition is only associative when overflow is defined to wrap around like unsigned arithmetic. This condition is matched here, because only debug builds panic on overflow. However, it's a bit of a gray area that the article completely ignores.

thrance 2mo ago
Unlike C, Rust's signed arithmetic is fully specified to wrap around.
- aw1621107 2mo ago
  > Rust's signed arithmetic is fully specified to wrap around. Well, kind of. It's currently documented to wrap in release mode by default, but it's just that - a default. You're free to enable overflow checks in release mode (or disable them in debug if you really like oddball configurations), and either way overflow is considered a logic error that devs shouldn't rely on (and basically can't rely on when not in control of the end binary since it's the end user who controls overflow checks). The Rust devs are theoretically open to making signed overflow panic by default, but consider such a change unlikely unless "something materially changes" [0]. [0]: https://github.com/rust-lang/rust/issues/47739#issuecomment-3165061176 https://github.com/rust-lang/rust/issues/47739#issuecomment-...
  - raverbashing 2mo ago
    Thanks for clarification, it seems Rust devs (as opposite to C devs) like good defaults and don't like making code accidentally cut yourself just because you looked at it wrong
- tialaramex 2mo ago
  No. If you want wrapping, ask for it with Wrapping<T> or the specific Wrapping types, or the wrapping arithmetic APIs It's true that since it's safe and faster, release builds default to wrapping rather than panic, but it's still wrong if you overflow any of Rust's default integer types, it's just that in a safe language it won't be Undefined Behaviour. "I can't be bothered to do it correctly" speaks to the quality of the rest of the product, it's a Brown M&M [read about the Van Halen test if you don't know what a Brown M&M means]

rob74 2mo ago

I'm not an expert on floating point math, but the "Does adding a small number do nothing?" example caught my eye, because the numbers used are constants, which are arbitrary precision in some languages (https://stackoverflow.com/questions/57511935/what-is-the-purpose-of-arbitrary-precision-constants-in-go https://stackoverflow.com/questions/57511935/what-is-the-pur...). For instance, Go answers the question with false (but still prints out "1e+16" when you try to print 1e16 + 1): https://go.dev/play/p/mSAktWpCRJA https://go.dev/play/p/mSAktWpCRJA

tialaramex 2mo ago
Rust explicitly requires that constants are typed. const FOO: f32 = 0.75; // The 32-bit floating point value three quarters If you try const UNTYPED = 0.75; // Does not compile, pick a type I don't find the SO answer very convincing because it seems like it's trying to argue this is the Reals, and it just isn't, it's only a subset of the Rationals which happened to be convenient for Go to work with it. The Reals are much stranger.
- ameliaquining 2mo ago
  Sorry, what exactly is your disagreement with the SO answer? It doesn't mention the reals.
- rob74 2mo ago
  Real numbers include rational numbers (numbers which can be represented as a fraction of two integers) and irrational numbers (numbers that can't be represented as a fraction - the most famous one is probably π). Since irrational numbers have an infinite number of decimal places, they obviously can't be stored as a floating point value and also can't be written down exactly, no matter how many decimal places you use. "Arbitrary precision" constants might get you closer, but yes, you will never be able to store a "true" irrational number in a computer.
  - kibwen 2mo ago
    > you will never be able to store a "true" irrational number in a computer. Joke's on you, in my programming language all numbers are written in phinary: https://en.wikipedia.org/wiki/Golden_ratio_base https://en.wikipedia.org/wiki/Golden_ratio_base

14113 2mo ago

> Floating point math is often slower than integer math because the compiler is being conservative about how it optimizes your code. It's not strictly true to say that it's "being conservative". What is more correct is to say that floating point operations have different semantics to integer operations, and an optimisation that retains the semantics of an expression over integers may not do so when applied to an expression over integers. Hence, it may be possible to apply one optimisation to an integer expression, but applying that to a floating-point expression may result in a different program meaning. C/C++ compilers give you a way out of this with the `--ffast-math` flag, which essentially allows compilers to relax the constraints on floating-point optimisation passes. For an example of how this works in GCC, take a look here: https://gcc.gnu.org/wiki/FloatingPointMath https://gcc.gnu.org/wiki/FloatingPointMath

duped 2mo ago
> C/C++ compilers give you a way out of this with the -fast-math People should really use the individual optimization flags they want (no signed zeros, no trapping math, associative math, reciprocal math) and not -ffast-math because the other optimizations it enables leads to surprising code (for example, isinf and isnan may become noops, which will break production code). Basically never use an optimization flag that changes the semantics of your code without understanding exactly what that means. I have had to fix this in a number of codebases because someone thought that flag was as innocuous as -O3. And if you have to enable flush to zero/denormals are zero it should be explicit in your code and scoped.
- 14113 2mo ago
  Yes, I was being a bit concise: Individual optimisations should be turned on as determined by profiling, application semantics, etc. My point was more that if you want to get as close as possible between integer and floating-point, then there is a flag that does it. That doesn't mean that you should do it, however...
- tialaramex 2mo ago
  Yes. In fact, even if you don't ask for semantics like the -fast-math flag, the floating point types are worth some extra time to understand before relying on them. They're much stranger than the machine integers. The machine integers are basically like the Integers you were taught in school, except for overflow. That's not nothing but it's a complexity you can ignore entirely so long as you never overflow. In Rust you can have the language keep you safe - if an overflow occurs we'll panic and we're done. However the floating point types are a weird thing entirely invented for the convenience of the machine. They're too often introduced as if, like the machine integers, they're almost familiar numbers from school. Some languages even call these types "real" - but they very much are not actually the Real numbers, not even the approximation that the machine integers were to actual Integers. The programming language can't help you cope today. You can use software like "Herbie" to help you a bit, but today's languages just leave you with it. https://herbie.uwplse.org/ https://herbie.uwplse.org/ Here's an easy example you saw in school, a tenth, written 0.1 in decimal. The floating point types cannot represent this number. When you ask for the 32-bit floating point value 0.1 in a language like C or Rust, you actually get exactly 0.100000001490116119384765625 because that was a number the type can represent and it was deemed "close enough".

conradludgate 2mo ago

See also https://orlp.net/blog/taming-float-sums/ https://orlp.net/blog/taming-float-sums/

Asooka 2mo ago

I like the idea, but I hate how verbose it is. Would be nice if there was also a macro that would transform all arithmetic within a block into algebraic arithmetic. The name is also a bit misleading, as I would expect "alebraic_add" to give an exact algebraic result, but instead it enables optimisations based on associative semantics. As a sidenote, if implemented as a macro, e.g. fn fast_sum_f64(values: &[f64]) -> f64 { let mut total = 0; fp_opt!(associative, { for value in values { total += value; } }); return total; } A question arises what happens when operating on custom types that overload arithmetic operations. I think the cleanest approach here would be to let the custom type define optimised versions, or have another macro that automatically generates them based on the existing ones, i.e. propagate the optimisation flags.

afdbcreid 2mo ago
You can create an `struct Algebraic<T>(pub T)` newtype that overloads operators using algebraic methods. In fact such wrapper might be added to std (it was discussed but decided that not yet. Such wrappers exist for wrapping and saturating arithmetic).