10 ms·
Rust and C++ on Floating-Point Intensive Code
- leni536 7y agoSo the difference basically boils down to -ffast-math, right? Is there an equivalent in Rust? Edit: After some search I found these: https://github.com/rust-lang/rust/issues/21690 https://github.com/rust-lang/rust/issues/21690 https://doc.rust-lang.org/core/intrinsics/fn.fadd_fast.html https://doc.rust-lang.org/core/intrinsics/fn.fadd_fast.html Writing a wrapper around f64 that uses these intrinsics shouldn't be too hard. I don't program in Rust though.
- pedrocr 7y agoIt doesn't exist yet and it's not clear it should be replicated as is. The fast-math flag does a bunch of related things that should probably be exposed separately so it's not a footgun in several situations. I'm also partial to exposing it per-function so the control is actually in the hands of the people writing the code that know the context and not subject to someone fiddling with compiler flags and getting incorrect code. For this example you'd probably want -fassociative-math and not the other stuff that may result in incorrect code. -ffast-math was not actually used in the clang compilation and it's possible that the -fvectorize that was used picks a sensible mix of options. Here's a preliminary discussion: https://internals.rust-lang.org/t/pre-rfc-whats-the-best-way-to-implement-ffast-math/5740 https://internals.rust-lang.org/t/pre-rfc-whats-the-best-way... Trying to do an RFC process for this could be useful. The rust development process seems to be pretty good at thinking deeply about these things.
- cannam 7y ago> I'm also partial to exposing it per-function so the control is actually in the hands of the people writing the code that know the context As a C++ programmer who routinely uses fast-math "until something breaks" with DSP code, I would find that capability very attractive.
- fluffything 7y agoThat's kind of at odds with Rust guarantee that your code never breaks.
- stefs 7y agorust doesn't guarantee anything if you opt out of the guarantees. two examples that come to mind: unsafe and maybe bounds checks in release mode. fastmath is probably different anyway, as the "breaking" is on a floating point logic level, as in: results become imprecise, but not exactly "wrong" - as in undefined behaviour. but i don't know fastmath, so i might be wrong.
- steveklabnik 7y ago(bounds checks are not removed in release mode, you have to use unsafe to not have the bounds check) On the /r/rust thread, folks provided examples of why fastmath would produce UB in safe Rust.
- stefs 7y agoah, i think i meant overflow checks. and thanks for the pointer, i'll have a look.
- steveklabnik 7y agoah yes, overflow checks are not in release mode today. They may be in the future. And overflow isn't UB, it's two's compliment wrapping.
- _bxg1 7y agoTechnically Rust only guarantees memory-safety (and only outside of unsafe!{}). It has many features that aid in other kinds of safety - strongly encouraging you to unwrap Option<> and Result<>, requiring all match cases to be covered, allowing for lots of strategic immutability, etc. But it doesn't guarantee that kind of correctness.
- sanxiyn 7y agoClang -Ofast implies -ffast-math.
- paulddraper 7y agoYes, and that doesn't surprise me. In my experience -Ofast/-ffast-math yields very impressive results for FP code. If you can tolerate platform-specific variation in the trailing parts of floating point numbers, it's wonderful.
- gameswithgo 7y agoyes, this is all purely down to Rust not doing ffast-math (on purpose) writing a wrapper is not trivial, at all. what Rust could do though, is add a wrapped float type, that the compiler will forward to llvm saying "you can ffast-math these" That is the approach Rust tends to take for these kinds of things, though no plans are in the works to do this to make floats vectorizeable yet. Maybe we should start such plans?
- zamalek 7y ago> on purpose It's worth explaining why. So far as I understand the situation, dependencies may assume and depend on the absence of ffast-math. Enabling it globally for a compilation is generally considered to be a footgun for this reason.
- dhruvdh 7y agoIt's very easy to do FMA's using .mul_add() on floats in Rust, which the author didn't seem to know about.
- magicalhippo 7y agoIdeally the compiler should be able to do this by itself though, at least with the appropriate flag to enable it.
- jacobolus 7y agoIf you’re doing fiddly numerical work, this must definitely be optional, as swapping separate multiplication and addition for FMA (or vice versa) can compromise correctness. In some cases you need two different algorithms if FMA is present or absent.
- fluffything 7y agoDo you have concrete examples of such algorithms?
- jacobolus 7y agoSome algorithms guarantee that some arithmetic operation(s) applied to 2+ floating point inputs will result in a list of floating point outputs which when summed have exactly the correct result. This gets all screwed up if you mess with the order of operations or the rounding of intermediate results. e.g. https://www.cs.cmu.edu/~quake/robust.html https://www.cs.cmu.edu/~quake/robust.html Some keywords to look for: “compensated arithmetic”, “error-free transformations”. FMAs generally speed up these tools, but you need to be careful and deliberate about how they are used. (Disclaimer: I am not an expert on this, just some guy on the internet.)
- fluffything 7y agoFair, I think it would be very helpful for Rust if some expert actually knows of a specific example for which this is the case. I think there is an RFC about enabling floating-point contraction by default, that would "silently" be able to do some of these transformations depending on the optimization level.
- diamondlovesyou 7y agoI have some experience with this, ie ensuring LLVM optimizes and codegens the "best"! I have been working to generate target independent "kernels" for the Rav1e AV1 encoder and have had to do a lot of unidiomatic things to get LLVM to generate machine code similar in quality to hand written ASM. Granted, this is on integers and not floats, but the same principles should apply. What I've found is that you need to ignore most of Rust: use/load raw pointers, don't use slices, unroll manually, vectorize manually, and check preconditions manually. You'll still get the amazing type system, but the code will have to be more C-like than Rust-like. * raw pointers: LLVM is pretty good at optimizing C code. Rust specific optimization needs some work. (edit: I assumed arrays here, so you'll need the pointer for offsets; references are still okay. You'd also use the pointers for iterating instead of the usual slice iteration patterns) * no slices: index checking is expensive, not to the CPU, the CPU rarely misses the check branches, but to the optimizer. I've found these are mostly left un-elided, even after inlining. * no slices: slice indexing uses overflow checking. For Rav1e's case, the block/plane sizes mean that doing the index calculation using `u32` will never overflow, so calculating the offsets using u32 is fine (I'll have to switch to using a pseudo u24 integer for GPUs though, because u32 is still expensive on them). * unroll manually: LLVM would probably do more of this with profiling info, but I've never found it (this is subjective!) to do any unrolling w/o. Maybe if all the other items here are also done... * vectorize manually: Similar to unrolling. I've observed only limited automatic vectorization. * And to get safety back: check, check, and check before calling the fast kernel! Ie wrap the kernel function with one that does all the checks elided in the kernel. Source: Wrote https://github.com/xiph/rav1e/pull/1716 https://github.com/xiph/rav1e/pull/1716, which speeds up the non-asm encodes by over 2x!
- dochtman 7y agoNot sure I understand the part about raw pointers. As far as I understand, Rust references will surely turn into pointers at the LLVM IR level?
- comex 7y agoThey do. The issue may be that references don't support pointer arithmetic – that is, given `x: &u32` you cannot get `x[1]`. The normal approach is to use a reference to a slice, `x: &[u32]`, and the default bounds-checked indexing. Rust does let you do unchecked indexing on slices (with `unsafe`), but it may be more efficient to avoid indices entirely in favor of pointer arithmetic. LLVM can often optimize index calculations into pointer arithmetic, but not always. Edit: Although, on rereading the above post, I see diamondlovesyou did mention indices, so... not really sure what's going on.
- rathinmadhu 7y agoSupper
- rathinmadhu 7y agoExcellent
- lovasoa 7y agoIt looks like what the author was looking for is [1] f64::mul_add(self, a: f64, b: f64) -> f64 Adding it to the code indeed allows the LLVM to generate the "vfma" instruction. But it didn't significantly improve performance, on my machine at least. $ ./iterators 1000 Normalized Average time = 0.0000000011943495282455513 sumb=89259.51980374461 $ ./mul_add 1000 Normalized Average time = 0.0000000011861410852805122 sumb=89259.52037960211 Maybe the performance gap is not due to what the author thought... [1] https://doc.rust-lang.org/std/primitive.f64.html#method.mul_add https://doc.rust-lang.org/std/primitive.f64.html#method.mul_...
- gpderetta 7y agoHum, did the program get vectorized?
- lovasoa 7y agoAs I said, the compiler did generate FMA instructions. These are SIMD instructions, so yes, the program was vectorized.
- gpderetta 7y agowell technically FMA doesn't necessarily imply vectorization; it depends on whether the P{S,D} vs the S{S,D} suffixed instructions were being used, but if you saw the (P)arallel variants, then yes, it was vectorized.
- gameswithgo 7y agoit isn't always that simple. FMA instructions are tricky to use in a way that actually improves performance, llvm may be doing it right while doing it manually that way may not. also, sometimes a SIMD instruction is used but only on 1 lane at a time. this is actually common with floating point code.
- lovasoa 7y agoWhat do you mean "manually" ? `mul_add` is a rust function that operates on a single f64, it's still up to LLVM to choose which instructions to use and to do the vectorization.
- nestorD 7y agoThe authors ends by noting that FMA would probably have improved the performances for the Rust code. It is interesting to note that, whereas most ffast-math optimization will trade precision for reduced computing time, adding an FMA can only improve the precision of the output (and thus it is a safe optimization).
- gameswithgo 7y agopedantry: ffast-math does not always trade precision. It simply trades the results being the same as if they were not vectorized. A vectorized sum of floats for instance is more accurate, not less.
- toolslive 7y agobefore you look at speed, did you verify you get the exact same math results in Rust and C++ (and for each compiler and platform) ? For C++ code, I have seen the results of calculations vary across compilers (and flags)
- leshow 7y agoWhat's "almost" algebraic about enum? It can definitely be used to construct sum types, and you can make product types with struct or inline in an enum
- uryga 7y agomy best guess is that you can't do recursive enums without explicit boxing [edit: or other forms of indirection, like &T]¹. so you can't do this: enum List<T> { Nil, Cons(T, List<T>) } instead, you have to box/reference-ify the recursive occurrence: enum List<T> { Nil, Cons(T, Box<List<T>>) } so in certain circumstances it doesn't let you "coproduct" two types together, you might need to modify one a bit, which makes it a technically-not-exactly-a-coproduct (i think). a bit of a stretch but it sort of makes sense next to a by-reference-only ML langs where you can (co)product anything as you please (btw, it's the same for recursive products) --- 1 - https://users.rust-lang.org/t/recursive-enum-types/2938/2 https://users.rust-lang.org/t/recursive-enum-types/2938/2
- Ygg2 7y agoI think that's because Rust types are Sized, but I could be wrong. The first example has size = Infinity, while the second has a constant size.
- leshow 7y agoThanks for commenting, that's probably it. I was aware of the requirement for explicit box-ing but it didn't immediately come to mind.
- steveklabnik 7y agoYou don't have to box, but you do need some sort of type to make things sized. This is usually a pointer of some kind, but any kind of pointer works. Take references, for example: enum List<'a, T> { Nil, Cons(T, &'a List<'a, T>) } fn main() { let list = List::Cons("hello", &List::Nil); } Box is usually chosen because it's a good default choice.
- 7y ago
- Myrmornis 7y agoThis is an extremely clear article and apparently well-executed experiment.
- brandmeyer 7y agoThis is interesting to see. But if I'm going to compare numerical C++ against numerical Rust, then I would be using a higher-level library for the comparision. What is Rust's Other Leading Brand (TM) for the Eigen C++ library?
- steveklabnik 7y agoThat comparison (I’m not familiar enough with Eigen to truly say) is going to change over time too; once const generics lands (which is proceeding, finally) the APIs for numerics libraries are going to be significantly different in Rust.
- nravic 7y agoThere's a bunch of Eigen analogues in Rust, which is slightly frustrating. ndarray is pretty great though
- vkaku 7y agoOkay, can someone give an explanation for why Rust does not mimic the -O fast behavior? Is this something they plan to add?
- steveklabnik 7y agoIt leads to undefined behavior in safe code in the general case. We may add a wrapping type, similar to what we do for integer behavior. But in general, adding flags to change major behavior is not something we do.